2024-09-10T03:44:12.029 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T03:44:12.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T03:44:12.147 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603 branch: wip-adk-testing-2024-09-09-1904 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: adking@redhat.com first_in_suite: false job_id: '7898603' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/squid ' name: adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-adk-testing-2024-09-09-1904 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: mon pg warn min per osd: 0 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-adk-testing-2024-09-09-1904 sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c owner: scheduled_adking@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6964 sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c sleep_before_teardown: 0 subset: 1/20 suite: orch:cephadm suite_branch: wip-adk-testing-2024-09-09-1904 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7FmVjbz1R6yBqS9saMeyYXdZGr1C8KerQVp67yt2fZ6jFT97EZOsm0qkx+SEYthPGYxp/dLXVWcNOXbPT159c= smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMDZYzqWc4FVNtrD5H4SDRBwZQDqcOBJHJPfAHtZH8Z0BTiZaA9P0s4JTMIScwilwJ4EP+oKSCNlZDpco+kRiM4= tasks: - install: branch: squid exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: squid conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:squid roleless: true - print: '**** done end installing squid cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_02:59:23 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: adking verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-09-10T03:44:12.147 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa; will attempt to use it 2024-09-10T03:44:12.148 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks 2024-09-10T03:44:12.148 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T03:44:12.149 INFO:teuthology.task.internal:Checking packages... 2024-09-10T03:44:12.171 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3a5e3f16720f432aea965c40a8f6eb480095ec6c' 2024-09-10T03:44:12.171 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:44:12.171 INFO:teuthology.packaging:ref: None 2024-09-10T03:44:12.171 INFO:teuthology.packaging:tag: None 2024-09-10T03:44:12.171 INFO:teuthology.packaging:branch: wip-adk-testing-2024-09-09-1904 2024-09-10T03:44:12.171 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:44:12.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-adk-testing-2024-09-09-1904 2024-09-10T03:44:12.319 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4885.g3a5e3f16 2024-09-10T03:44:12.320 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T03:44:12.328 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T03:44:12.328 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T03:44:12.362 INFO:teuthology.task.internal:Saving configuration 2024-09-10T03:44:12.382 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T03:44:12.395 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T03:44:12.419 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 03:39:12.647846', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7FmVjbz1R6yBqS9saMeyYXdZGr1C8KerQVp67yt2fZ6jFT97EZOsm0qkx+SEYthPGYxp/dLXVWcNOXbPT159c='} 2024-09-10T03:44:12.446 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 03:39:12.646579', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMDZYzqWc4FVNtrD5H4SDRBwZQDqcOBJHJPfAHtZH8Z0BTiZaA9P0s4JTMIScwilwJ4EP+oKSCNlZDpco+kRiM4='} 2024-09-10T03:44:12.446 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T03:44:12.454 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-09-10T03:44:12.454 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-09-10T03:44:12.454 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T03:44:12.531 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbeeec5bbe0>, signals=[15]) 2024-09-10T03:44:12.531 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T03:44:12.539 INFO:teuthology.task.internal:Opening connections... 2024-09-10T03:44:12.539 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T03:44:12.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:44:12.619 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T03:44:12.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:44:12.694 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T03:44:12.702 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-09-10T03:44:12.722 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-09-10T03:44:12.723 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-09-10T03:44:12.780 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-09-10T03:44:12.780 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T03:44:12.781 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-09-10T03:44:12.782 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T03:44:12.782 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T03:44:12.782 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T03:44:12.783 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-09-10T03:44:12.800 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-09-10T03:44:12.817 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-09-10T03:44:12.818 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-09-10T03:44:12.874 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T03:44:12.875 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T03:44:12.876 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T03:44:12.876 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-09-10T03:44:12.900 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T03:44:12.908 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T03:44:12.938 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T03:44:12.938 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-09-10T03:44:12.941 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-09-10T03:44:12.957 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T03:44:12.971 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T03:44:12.971 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-09-10T03:44:12.998 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-09-10T03:44:13.029 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T03:44:13.087 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T03:44:13.102 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T03:44:13.102 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T03:44:13.103 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T03:44:13.103 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T03:44:13.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:44:13.121 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-10T03:44:13.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:44:13.122 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-10T03:44:13.178 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-503.el9.x86_64 2024-09-10T03:44:13.178 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-503.el9.x86_64 2024-09-10T03:44:13.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-09-10T03:44:13.179 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-503.el9.x86_64 2024-09-10T03:44:13.180 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-503.el9.x86_64 2024-09-10T03:44:13.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-09-10T03:44:15.548 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-10T03:44:16.232 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.2 MB 00:01 2024-09-10T03:44:21.476 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-10T03:44:21.834 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-09-10T03:44:26.915 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-10T03:44:27.466 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-10T03:44:28.274 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 114 kB/s | 19 kB 00:00 2024-09-10T03:44:29.008 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-10T03:44:33.541 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 23 MB 00:04 2024-09-10T03:44:34.994 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 3.4 MB/s | 23 MB 00:06 2024-09-10T03:44:39.243 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T03:44:40.386 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T03:44:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T03:44:41.325 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T03:44:41.430 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T03:44:41.437 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:44:41.438 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:44:41.438 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:44:41.438 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-09-10T03:44:41.438 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:44:41.438 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 79 M 2024-09-10T03:44:41.439 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 126 M 2024-09-10T03:44:41.439 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:44:41.728 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-10T03:44:41.751 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-10T03:44:41.948 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:44:42.003 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:44:42.107 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-10T03:44:42.287 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:44:42.425 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T03:44:42.425 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T03:44:42.520 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:44:42.527 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:44:42.527 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-10T03:44:42.527 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:44:42.527 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:44:42.527 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-10T03:44:42.528 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:44:42.529 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 79 M 2024-09-10T03:44:42.529 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 126 M 2024-09-10T03:44:42.529 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:44:42.828 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 17 kB/s | 23 kB 00:01 2024-09-10T03:44:43.022 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:44:43.162 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2024-09-10T03:44:43.227 INFO:teuthology.orchestra.run.smithi046.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.21) 2024-09-10T03:44:43.420 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 8.1 MB/s | 16 MB 00:01 2024-09-10T03:44:44.029 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-10T03:44:44.053 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 13 MB/s | 35 MB 00:02 2024-09-10T03:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 17 MB/s | 35 MB 00:02 2024-09-10T03:44:45.446 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-10T03:44:45.446 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:44:45.446 INFO:teuthology.orchestra.run.smithi070.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-10T03:44:46.128 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:44:46.488 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:44:46.489 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:44:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.9 MB/s | 29 MB 00:04 2024-09-10T03:44:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:44:46.989 INFO:teuthology.orchestra.run.smithi046.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-10T03:44:47.447 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:44:47.447 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:44:47.702 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:44:48.078 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:44:48.078 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:44:49.071 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:44:49.071 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:44:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:44:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:44:50.621 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:44:51.646 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:44:52.322 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:44:52.403 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:44:53.423 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:44:53.566 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:44:55.200 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:44:55.340 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:44:58.399 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:44:58.524 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:00.464 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:45:00.507 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:03.375 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:05.646 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:41.125 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:41.165 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:41.498 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:41.498 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:45:41.498 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:45:41.498 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:45:42.352 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:42.352 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:45:42.353 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:45:42.353 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T03:45:42.353 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T03:45:42.353 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T03:45:42.353 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T03:45:42.353 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:45:42.353 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:45:42.708 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:45:42.793 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:45:42.794 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:45:42.794 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T03:45:42.794 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T03:45:42.794 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T03:45:42.794 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-10T03:45:42.794 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T03:45:42.794 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:45:42.853 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:45:42.853 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:45:42.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 82.el9 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8274670 2024-09-10T03:45:42.893 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:45:42.894 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T03:45:42.895 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T03:45:42.895 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-505.el9.x86_64 2024-09-10T03:45:42.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T03:45:43.554 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-09-10T03:45:44.830 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T03:45:44.851 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-09-10T03:45:44.853 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-10T03:45:44.869 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.16yMmIRE4e 2024-09-10T03:45:44.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.16yMmIRE4e 2024-09-10T03:45:44.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.16yMmIRE4e 2024-09-10T03:45:45.012 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:45.055 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:45.081 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.16yMmIRE4e is 6KB 2024-09-10T03:45:45.130 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.16yMmIRE4e 2024-09-10T03:45:45.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T03:45:45.209 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T03:45:45.210 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T03:45:45.210 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T03:45:45.210 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T03:45:45.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T03:45:45.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-09-10T03:45:45.388 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:45.388 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:45:45.388 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:45:45.388 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:45:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:45:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:45:46.256 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:45:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T03:45:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T03:45:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T03:45:46.256 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T03:45:46.257 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:45:46.257 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:45:46.583 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:45:46.668 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:45:46.668 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:45:46.669 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T03:45:46.669 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T03:45:46.669 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T03:45:46.669 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-10T03:45:46.669 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T03:45:46.669 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:45:46.727 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:45:46.727 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:45:46.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-09-10T03:45:46.758 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 82.el9 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8274670 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T03:45:46.759 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:45:46.760 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T03:45:46.761 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T03:45:46.761 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-505.el9.x86_64 2024-09-10T03:45:46.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T03:45:47.443 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-09-10T03:45:48.879 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T03:45:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-09-10T03:45:48.915 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-09-10T03:45:48.971 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.tPywApph8R 2024-09-10T03:45:48.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tPywApph8R 2024-09-10T03:45:49.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.tPywApph8R 2024-09-10T03:45:49.183 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.tPywApph8R is 6KB 2024-09-10T03:45:49.231 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.tPywApph8R 2024-09-10T03:45:49.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T03:45:49.308 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T03:45:49.308 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T03:45:49.308 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T03:45:49.308 INFO:teuthology.orchestra.run.smithi046.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T03:45:49.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T03:45:49.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-09-10T03:46:15.397 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-10T03:46:15.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T03:46:15.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:46:19.464 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-10T03:46:19.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T03:46:19.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:46:54.141 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-10T03:47:03.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T03:47:03.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:47:15.413 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T03:47:21.533 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-10T03:47:24.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T03:47:24.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:47:24.751 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-10T03:47:25.108 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T03:47:25.108 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T03:47:25.109 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-10T03:47:25.123 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-505.el9.x86_64 2024-09-10T03:47:25.124 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T03:47:25.124 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T03:47:25.124 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T03:47:26.125 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-10T03:47:26.126 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T03:47:26.203 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-09-10T03:47:26.240 DEBUG:teuthology.parallel:result is None 2024-09-10T03:47:33.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T03:47:33.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:47:36.605 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-09-10T03:47:51.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T03:47:51.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:47:51.985 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-10T03:47:52.358 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T03:47:52.358 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T03:47:52.359 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-09-10T03:47:52.376 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-505.el9.x86_64 2024-09-10T03:47:52.376 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T03:47:52.376 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T03:47:52.376 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T03:47:53.376 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-10T03:47:53.377 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T03:47:53.470 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-09-10T03:47:53.493 DEBUG:teuthology.parallel:result is None 2024-09-10T03:47:53.493 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T03:47:53.504 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T03:47:53.504 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T03:47:53.506 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T03:47:53.526 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T03:47:53.550 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T03:47:53.600 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T03:47:53.601 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T03:47:53.603 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T03:47:53.648 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T03:47:53.659 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T03:47:53.660 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T03:47:53.674 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T03:47:53.702 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:47:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:47:53.714 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:47:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:47:53.727 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T03:47:53.738 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T03:47:53.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T03:47:53.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T03:47:53.795 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T03:47:53.805 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T03:47:53.806 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T03:47:53.827 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T03:47:53.851 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:47:53.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:47:53.972 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:47:54.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:47:54.098 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:47:54.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T03:47:54.164 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:47:54.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:47:54.284 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:47:54.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:47:54.367 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:47:54.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T03:47:54.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-09-10T03:47:54.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-09-10T03:47:54.491 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T03:47:54.522 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T03:47:54.875 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T03:47:54.887 INFO:teuthology.task.internal:Starting timer... 2024-09-10T03:47:54.887 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T03:47:54.936 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T03:47:54.985 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T03:47:54.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-09-10T03:47:55.032 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-09-10T03:47:55.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-09-10T03:47:55.106 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-09-10T03:47:55.108 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T03:47:55.109 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-09-10T03:47:55.141 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-09-10T03:47:55.141 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-09-10T03:47:55.160 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-09-10T03:47:55.161 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi070.front.sepia.ceph.com': 'permissive'} 2024-09-10T03:47:55.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T03:47:55.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:47:55.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T03:47:55.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:47:55.236 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T03:47:55.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-09-10T03:47:55.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-09-10T03:47:55.314 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T03:47:55.324 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T03:47:55.328 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T03:47:55.522 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T03:47:55.532 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T03:47:55.533 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi070.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T03:51:38.261 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi070.front.sepia.ceph.com')] 2024-09-10T03:51:38.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T03:51:38.264 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:51:38.343 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-09-10T03:51:38.428 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-09-10T03:51:38.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T03:51:38.429 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:51:38.523 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-10T03:51:38.604 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-10T03:51:38.604 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T03:51:38.770 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T03:51:38.770 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T03:51:38.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T03:51:38.773 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T03:51:38.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T03:51:38.803 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T03:51:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T03:51:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T03:51:38.824 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T03:51:38.842 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-09-10T03:51:38.851 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-09-10T03:51:38.853 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-09-10T03:51:38.863 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-09-10T03:51:38.867 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T03:51:38.880 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T03:51:38.881 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T03:51:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T03:51:38.933 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-10T03:51:38.937 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T03:51:38.937 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-10T03:51:38.937 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.937 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.937 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.937 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.955 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-10T03:51:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T03:51:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-10T03:51:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.958 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:51:38.959 INFO:teuthology.run_tasks:Running task install... 2024-09-10T03:51:38.967 DEBUG:teuthology.task.install:project ceph 2024-09-10T03:51:38.967 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c'}} 2024-09-10T03:51:38.967 DEBUG:teuthology.task.install:config {'branch': 'squid', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c'} 2024-09-10T03:51:38.967 INFO:teuthology.task.install:Using flavor: default 2024-09-10T03:51:38.972 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T03:51:38.973 INFO:teuthology.task.install:extra packages: [] 2024-09-10T03:51:38.973 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'squid', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c', 'tag': None, 'wait_for_package': False} 2024-09-10T03:51:38.973 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:51:38.973 INFO:teuthology.packaging:ref: None 2024-09-10T03:51:38.973 INFO:teuthology.packaging:tag: None 2024-09-10T03:51:38.973 INFO:teuthology.packaging:branch: squid 2024-09-10T03:51:38.974 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:51:38.974 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-10T03:51:38.976 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'squid', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c', 'tag': None, 'wait_for_package': False} 2024-09-10T03:51:38.976 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:51:38.976 INFO:teuthology.packaging:ref: None 2024-09-10T03:51:38.976 INFO:teuthology.packaging:tag: None 2024-09-10T03:51:38.976 INFO:teuthology.packaging:branch: squid 2024-09-10T03:51:38.976 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:51:38.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-10T03:51:39.185 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-10T03:51:39.185 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-10T03:51:39.212 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-10T03:51:39.212 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-10T03:51:39.309 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T03:51:39.309 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:51:39.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T03:51:39.321 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T03:51:39.322 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:51:39.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T03:51:39.336 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T03:51:39.336 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:51:39.336 INFO:teuthology.packaging:ref: None 2024-09-10T03:51:39.336 INFO:teuthology.packaging:tag: None 2024-09-10T03:51:39.336 INFO:teuthology.packaging:branch: squid 2024-09-10T03:51:39.336 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:51:39.337 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/squid/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T03:51:39.350 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T03:51:39.350 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:51:39.350 INFO:teuthology.packaging:ref: None 2024-09-10T03:51:39.350 INFO:teuthology.packaging:tag: None 2024-09-10T03:51:39.350 INFO:teuthology.packaging:branch: squid 2024-09-10T03:51:39.350 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:51:39.351 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/squid/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T03:51:39.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T03:51:39.418 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T03:51:39.477 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T03:51:39.496 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T03:51:39.541 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-10T03:51:39.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-10T03:51:39.562 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-09-10T03:51:39.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-09-10T03:51:39.765 INFO:teuthology.orchestra.run.smithi070.stdout:45 files removed 2024-09-10T03:51:39.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw 2024-09-10T03:51:39.801 INFO:teuthology.orchestra.run.smithi046.stdout:45 files removed 2024-09-10T03:51:39.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw 2024-09-10T03:51:40.352 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-10T03:51:40.406 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-10T03:51:40.636 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T03:51:40.690 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T03:51:40.912 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T03:51:40.974 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-10T03:51:41.470 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-10T03:51:42.682 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-10T03:51:58.171 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 20 MB 00:15 2024-09-10T03:52:02.848 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:19 2024-09-10T03:52:03.152 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 6.5 MB 00:01 2024-09-10T03:52:04.929 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-10T03:52:09.416 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-10T03:52:11.141 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-10T03:52:16.647 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 23 MB 00:11 2024-09-10T03:52:18.343 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 23 MB 00:07 2024-09-10T03:52:21.961 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-10T03:52:24.110 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T03:52:24.346 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:52:24.357 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:52:24.357 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:52:24.357 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:52:24.357 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T03:52:24.358 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-10T03:52:24.359 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:52:24.360 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:52:24.361 INFO:teuthology.orchestra.run.smithi070.stdout:Install 30 Packages 2024-09-10T03:52:24.361 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-09-10T03:52:24.361 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:24.361 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 60 M 2024-09-10T03:52:24.361 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:52:25.220 INFO:teuthology.orchestra.run.smithi070.stdout:(1/32): ceph-base-19.1.1-298.gb53b6d3e.el9.x86_ 9.4 MB/s | 5.4 MB 00:00 2024-09-10T03:52:25.287 INFO:teuthology.orchestra.run.smithi070.stdout:(2/32): ceph-selinux-19.1.1-298.gb53b6d3e.el9.x 378 kB/s | 25 kB 00:00 2024-09-10T03:52:25.379 INFO:teuthology.orchestra.run.smithi070.stdout:(3/32): libcephfs2-19.1.1-298.gb53b6d3e.el9.x86 8.0 MB/s | 752 kB 00:00 2024-09-10T03:52:25.496 INFO:teuthology.orchestra.run.smithi070.stdout:(4/32): ceph-radosgw-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 11 MB 00:00 2024-09-10T03:52:25.530 INFO:teuthology.orchestra.run.smithi070.stdout:(5/32): libradosstriper1-19.1.1-298.gb53b6d3e.e 3.3 MB/s | 506 kB 00:00 2024-09-10T03:52:25.580 INFO:teuthology.orchestra.run.smithi070.stdout:(6/32): python3-ceph-argparse-19.1.1-298.gb53b6 903 kB/s | 45 kB 00:00 2024-09-10T03:52:25.647 INFO:teuthology.orchestra.run.smithi070.stdout:(7/32): python3-ceph-common-19.1.1-298.gb53b6d3 2.0 MB/s | 136 kB 00:00 2024-09-10T03:52:25.872 INFO:teuthology.orchestra.run.smithi070.stdout:(8/32): ceph-common-19.1.1-298.gb53b6d3e.el9.x8 17 MB/s | 21 MB 00:01 2024-09-10T03:52:25.897 INFO:teuthology.orchestra.run.smithi070.stdout:(9/32): python3-cephfs-19.1.1-298.gb53b6d3e.el9 648 kB/s | 162 kB 00:00 2024-09-10T03:52:25.931 INFO:teuthology.orchestra.run.smithi070.stdout:(10/32): python3-rados-19.1.1-298.gb53b6d3e.el9 5.4 MB/s | 321 kB 00:00 2024-09-10T03:52:25.965 INFO:teuthology.orchestra.run.smithi070.stdout:(11/32): python3-rbd-19.1.1-298.gb53b6d3e.el9.x 4.4 MB/s | 301 kB 00:00 2024-09-10T03:52:25.990 INFO:teuthology.orchestra.run.smithi070.stdout:(12/32): python3-rgw-19.1.1-298.gb53b6d3e.el9.x 1.7 MB/s | 100 kB 00:00 2024-09-10T03:52:26.065 INFO:teuthology.orchestra.run.smithi070.stdout:(13/32): librgw2-19.1.1-298.gb53b6d3e.el9.x86_6 9.4 MB/s | 5.3 MB 00:00 2024-09-10T03:52:26.182 INFO:teuthology.orchestra.run.smithi070.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-09-10T03:52:26.207 INFO:teuthology.orchestra.run.smithi070.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 441 kB/s | 107 kB 00:00 2024-09-10T03:52:26.233 INFO:teuthology.orchestra.run.smithi070.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 661 kB/s | 160 kB 00:00 2024-09-10T03:52:26.283 INFO:teuthology.orchestra.run.smithi070.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 845 kB/s | 42 kB 00:00 2024-09-10T03:52:26.308 INFO:teuthology.orchestra.run.smithi070.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-10T03:52:26.334 INFO:teuthology.orchestra.run.smithi070.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-10T03:52:26.359 INFO:teuthology.orchestra.run.smithi070.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-09-10T03:52:26.384 INFO:teuthology.orchestra.run.smithi070.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-10T03:52:26.409 INFO:teuthology.orchestra.run.smithi070.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-10T03:52:26.476 INFO:teuthology.orchestra.run.smithi070.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.6 MB/s | 309 kB 00:00 2024-09-10T03:52:26.501 INFO:teuthology.orchestra.run.smithi070.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 337 kB/s | 26 kB 00:00 2024-09-10T03:52:26.527 INFO:teuthology.orchestra.run.smithi070.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-09-10T03:52:26.552 INFO:teuthology.orchestra.run.smithi070.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T03:52:26.602 INFO:teuthology.orchestra.run.smithi070.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T03:52:26.614 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:52:26.628 INFO:teuthology.orchestra.run.smithi070.stdout:(28/32): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-10T03:52:26.629 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-10T03:52:26.630 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T03:52:26.631 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-09-10T03:52:26.632 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout:Install 30 Packages 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 2 Packages 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 60 M 2024-09-10T03:52:26.633 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:52:26.703 INFO:teuthology.orchestra.run.smithi070.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T03:52:26.786 INFO:teuthology.orchestra.run.smithi070.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-10T03:52:26.904 INFO:teuthology.orchestra.run.smithi070.stdout:(31/32): librados2-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 3.4 MB 00:00 2024-09-10T03:52:26.962 INFO:teuthology.orchestra.run.smithi070.stdout:(32/32): librbd1-19.1.1-298.gb53b6d3e.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-10T03:52:26.967 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:52:26.967 INFO:teuthology.orchestra.run.smithi070.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-10T03:52:27.065 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:52:27.102 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:52:27.102 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:52:27.536 INFO:teuthology.orchestra.run.smithi046.stdout:(1/32): ceph-base-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 5.4 MB 00:00 2024-09-10T03:52:27.595 INFO:teuthology.orchestra.run.smithi046.stdout:(2/32): ceph-selinux-19.1.1-298.gb53b6d3e.el9.x 433 kB/s | 25 kB 00:00 2024-09-10T03:52:27.745 INFO:teuthology.orchestra.run.smithi046.stdout:(3/32): ceph-radosgw-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 11 MB 00:00 2024-09-10T03:52:27.778 INFO:teuthology.orchestra.run.smithi046.stdout:(4/32): libcephfs2-19.1.1-298.gb53b6d3e.el9.x86 4.0 MB/s | 752 kB 00:00 2024-09-10T03:52:27.799 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:52:27.800 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:52:27.820 INFO:teuthology.orchestra.run.smithi046.stdout:(5/32): libradosstriper1-19.1.1-298.gb53b6d3e.e 6.6 MB/s | 506 kB 00:00 2024-09-10T03:52:27.963 INFO:teuthology.orchestra.run.smithi046.stdout:(6/32): python3-ceph-argparse-19.1.1-298.gb53b6 317 kB/s | 45 kB 00:00 2024-09-10T03:52:28.046 INFO:teuthology.orchestra.run.smithi046.stdout:(7/32): librgw2-19.1.1-298.gb53b6d3e.el9.x86_64 20 MB/s | 5.3 MB 00:00 2024-09-10T03:52:28.071 INFO:teuthology.orchestra.run.smithi046.stdout:(8/32): python3-ceph-common-19.1.1-298.gb53b6d3 1.2 MB/s | 136 kB 00:00 2024-09-10T03:52:28.097 INFO:teuthology.orchestra.run.smithi046.stdout:(9/32): python3-cephfs-19.1.1-298.gb53b6d3e.el9 3.2 MB/s | 162 kB 00:00 2024-09-10T03:52:28.122 INFO:teuthology.orchestra.run.smithi046.stdout:(10/32): python3-rados-19.1.1-298.gb53b6d3e.el9 6.3 MB/s | 321 kB 00:00 2024-09-10T03:52:28.147 INFO:teuthology.orchestra.run.smithi046.stdout:(11/32): python3-rbd-19.1.1-298.gb53b6d3e.el9.x 5.9 MB/s | 301 kB 00:00 2024-09-10T03:52:28.173 INFO:teuthology.orchestra.run.smithi046.stdout:(12/32): python3-rgw-19.1.1-298.gb53b6d3e.el9.x 1.9 MB/s | 100 kB 00:00 2024-09-10T03:52:28.406 INFO:teuthology.orchestra.run.smithi046.stdout:(13/32): ceph-common-19.1.1-298.gb53b6d3e.el9.x 16 MB/s | 21 MB 00:01 2024-09-10T03:52:28.448 INFO:teuthology.orchestra.run.smithi046.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 356 kB/s | 107 kB 00:00 2024-09-10T03:52:28.474 INFO:teuthology.orchestra.run.smithi046.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 533 kB/s | 160 kB 00:00 2024-09-10T03:52:28.524 INFO:teuthology.orchestra.run.smithi046.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-10T03:52:28.549 INFO:teuthology.orchestra.run.smithi046.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-10T03:52:28.583 INFO:teuthology.orchestra.run.smithi046.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-10T03:52:28.608 INFO:teuthology.orchestra.run.smithi046.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 504 kB/s | 42 kB 00:00 2024-09-10T03:52:28.633 INFO:teuthology.orchestra.run.smithi046.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 716 kB/s | 36 kB 00:00 2024-09-10T03:52:28.658 INFO:teuthology.orchestra.run.smithi046.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-10T03:52:28.684 INFO:teuthology.orchestra.run.smithi046.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-10T03:52:28.742 INFO:teuthology.orchestra.run.smithi046.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-09-10T03:52:28.768 INFO:teuthology.orchestra.run.smithi046.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-10T03:52:28.793 INFO:teuthology.orchestra.run.smithi046.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 280 kB/s | 26 kB 00:00 2024-09-10T03:52:28.818 INFO:teuthology.orchestra.run.smithi046.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T03:52:28.852 INFO:teuthology.orchestra.run.smithi046.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-10T03:52:28.919 INFO:teuthology.orchestra.run.smithi046.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-09-10T03:52:28.985 INFO:teuthology.orchestra.run.smithi046.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T03:52:29.061 INFO:teuthology.orchestra.run.smithi046.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-10T03:52:29.161 INFO:teuthology.orchestra.run.smithi046.stdout:(31/32): librados2-19.1.1-298.gb53b6d3e.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-10T03:52:29.228 INFO:teuthology.orchestra.run.smithi046.stdout:(32/32): librbd1-19.1.1-298.gb53b6d3e.el9.x86_6 13 MB/s | 3.2 MB 00:00 2024-09-10T03:52:29.233 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:52:29.233 INFO:teuthology.orchestra.run.smithi046.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-10T03:52:29.287 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:52:29.338 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:52:29.368 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-10T03:52:29.379 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:52:29.379 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:52:29.400 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-10T03:52:29.427 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-10T03:52:29.450 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-10T03:52:29.477 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-10T03:52:29.668 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-10T03:52:29.721 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-10T03:52:30.085 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:52:30.085 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:52:30.090 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-10T03:52:30.162 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-10T03:52:30.238 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-10T03:52:30.324 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/34 2024-09-10T03:52:30.358 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-10T03:52:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-10T03:52:30.455 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-10T03:52:30.499 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 11/34 2024-09-10T03:52:30.592 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-10T03:52:30.625 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-10T03:52:30.672 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-10T03:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-10T03:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-10T03:52:30.857 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-10T03:52:30.899 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-10T03:52:30.931 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-10T03:52:30.960 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-10T03:52:30.992 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-10T03:52:31.318 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-10T03:52:31.355 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-10T03:52:31.459 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-10T03:52:31.494 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/34 2024-09-10T03:52:31.538 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-10T03:52:31.605 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:52:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-10T03:52:31.752 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-10T03:52:31.777 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-10T03:52:31.800 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-10T03:52:31.829 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-10T03:52:32.038 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-10T03:52:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-10T03:52:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-10T03:52:32.098 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-10T03:52:32.146 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-10T03:52:32.217 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/34 2024-09-10T03:52:32.245 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 28/34 2024-09-10T03:52:32.493 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-10T03:52:32.554 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-10T03:52:32.647 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-10T03:52:32.737 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/34 2024-09-10T03:52:32.763 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-10T03:52:32.830 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/34 2024-09-10T03:52:32.881 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-10T03:52:32.925 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 11/34 2024-09-10T03:52:32.995 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-10T03:52:33.033 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-10T03:52:33.098 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-10T03:52:33.131 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-10T03:52:33.243 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-10T03:52:33.283 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-10T03:52:33.324 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-10T03:52:33.384 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-09-10T03:52:33.419 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-10T03:52:33.451 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-10T03:52:33.789 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-10T03:52:33.839 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-10T03:52:33.914 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 22/34 2024-09-10T03:52:33.944 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/34 2024-09-10T03:52:33.980 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-10T03:52:34.486 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-10T03:52:34.525 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-10T03:52:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-10T03:52:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-10T03:52:34.596 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/34 2024-09-10T03:52:34.686 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/34 2024-09-10T03:52:34.719 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 28/34 2024-09-10T03:52:35.151 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-10T03:52:35.211 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-10T03:52:35.284 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-10T03:52:35.285 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T03:52:35.285 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T03:52:35.285 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:35.329 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-10T03:52:37.168 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-10T03:52:37.216 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-10T03:52:37.821 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/34 2024-09-10T03:52:37.869 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-10T03:52:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 30/34 2024-09-10T03:52:37.962 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T03:52:37.962 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T03:52:37.962 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:37.995 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-10T03:52:45.217 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-10T03:52:45.217 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-09-10T03:52:45.217 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-09-10T03:52:45.218 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-09-10T03:52:45.218 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-09-10T03:52:45.218 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-09-10T03:52:45.218 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-09-10T03:52:45.218 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-09-10T03:52:45.218 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:46.216 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-10T03:52:46.241 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-10T03:52:46.241 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:52:46.242 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T03:52:46.242 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T03:52:46.242 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T03:52:46.242 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:46.244 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-10T03:52:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-10T03:52:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/34 2024-09-10T03:52:47.502 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 8/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 9/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-10T03:52:47.503 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-10T03:52:47.504 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-10T03:52:47.505 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-10T03:52:47.505 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-10T03:52:47.505 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/34 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-09-10T03:52:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T03:52:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T03:52:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T03:52:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:48.574 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:52:48.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-test 2024-09-10T03:52:49.495 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:28 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:52:49.514 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-10T03:52:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 32/34 2024-09-10T03:52:49.541 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:52:49.541 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T03:52:49.541 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T03:52:49.541 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T03:52:49.541 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:49.543 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-10T03:52:49.627 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:52:49.628 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:52:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:52:49.628 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:52:49.628 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:52:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-10T03:52:49.628 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:52:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:49.629 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 49 M 2024-09-10T03:52:49.630 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 209 M 2024-09-10T03:52:49.630 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:52:49.633 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-10T03:52:49.634 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-10T03:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-10T03:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-10T03:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): libcephsqlite-19.1.1-298.gb53b6d3e.el9.x 732 kB/s | 163 kB 00:00 2024-09-10T03:52:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-10T03:52:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/34 2024-09-10T03:52:50.832 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x 8/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86 9/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-10T03:52:50.833 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-10T03:52:50.834 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-10T03:52:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-10T03:52:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-10T03:52:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 31/34 2024-09-10T03:52:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-10T03:52:50.835 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/34 2024-09-10T03:52:51.183 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): ceph-test-19.1.1-298.gb53b6d3e.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-10T03:52:51.185 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:52:51.186 INFO:teuthology.orchestra.run.smithi070.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-10T03:52:51.213 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:52:51.233 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:52:51.234 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:52:51.618 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:52:51.618 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:52:52.000 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T03:52:52.001 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.002 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:52.003 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:52:52.162 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:52:52.235 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T03:52:52.274 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T03:52:52.302 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-10T03:52:52.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-test 2024-09-10T03:52:52.971 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:28 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:52:53.109 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:52:53.110 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:52:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:52:53.110 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:52:53.110 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:52:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-10T03:52:53.110 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:52:53.110 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:52:53.111 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 49 M 2024-09-10T03:52:53.112 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 209 M 2024-09-10T03:52:53.112 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:52:53.272 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-10T03:52:53.314 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-10T03:52:53.373 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): libcephsqlite-19.1.1-298.gb53b6d3e.el9.x 683 kB/s | 163 kB 00:00 2024-09-10T03:52:54.665 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): ceph-test-19.1.1-298.gb53b6d3e.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-10T03:52:54.667 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:52:54.668 INFO:teuthology.orchestra.run.smithi046.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-10T03:52:54.696 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:52:54.718 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:52:54.718 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:52:55.116 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:52:55.116 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:52:55.665 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:52:55.751 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T03:52:55.790 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T03:52:55.817 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-10T03:52:56.949 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-10T03:52:57.025 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-10T03:52:59.433 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-10T03:52:59.433 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/4 2024-09-10T03:52:59.433 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/4 2024-09-10T03:52:59.433 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T03:52:59.748 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:52:59.749 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:00.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph 2024-09-10T03:53:00.614 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:39 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:00.694 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/4 2024-09-10T03:53:00.758 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:00.760 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T03:53:00.761 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T03:53:00.762 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T03:53:00.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout:Install 37 Packages 2024-09-10T03:53:00.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:00.765 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 30 M 2024-09-10T03:53:00.765 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 108 M 2024-09-10T03:53:00.765 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:53:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-10T03:53:01.502 INFO:teuthology.orchestra.run.smithi070.stdout:(1/37): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.rp 45 kB/s | 6.5 kB 00:00 2024-09-10T03:53:01.669 INFO:teuthology.orchestra.run.smithi070.stdout:(2/37): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_6 4.9 MB/s | 1.5 MB 00:00 2024-09-10T03:53:01.736 INFO:teuthology.orchestra.run.smithi070.stdout:(3/37): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_6 6.3 MB/s | 2.4 MB 00:00 2024-09-10T03:53:01.803 INFO:teuthology.orchestra.run.smithi070.stdout:(4/37): ceph-mgr-modules-core-19.1.1-298.gb53b6 3.7 MB/s | 249 kB 00:00 2024-09-10T03:53:01.919 INFO:teuthology.orchestra.run.smithi070.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-10T03:53:02.020 INFO:teuthology.orchestra.run.smithi070.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 12 MB/s | 1.2 MB 00:00 2024-09-10T03:53:02.204 INFO:teuthology.orchestra.run.smithi070.stdout:(7/37): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_6 32 MB/s | 17 MB 00:00 2024-09-10T03:53:02.229 INFO:teuthology.orchestra.run.smithi070.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 511 kB/s | 106 kB 00:00 2024-09-10T03:53:02.329 INFO:teuthology.orchestra.run.smithi070.stdout:(9/37): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_6 5.7 MB/s | 4.7 MB 00:00 2024-09-10T03:53:02.362 INFO:teuthology.orchestra.run.smithi070.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 936 kB/s | 125 kB 00:00 2024-09-10T03:53:02.396 INFO:teuthology.orchestra.run.smithi070.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 702 kB/s | 135 kB 00:00 2024-09-10T03:53:02.472 INFO:teuthology.orchestra.run.smithi070.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.5 MB/s | 215 kB 00:00 2024-09-10T03:53:02.531 INFO:teuthology.orchestra.run.smithi070.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-10T03:53:02.564 INFO:teuthology.orchestra.run.smithi070.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 937 kB/s | 188 kB 00:00 2024-09-10T03:53:02.581 INFO:teuthology.orchestra.run.smithi070.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 319 kB/s | 35 kB 00:00 2024-09-10T03:53:02.606 INFO:teuthology.orchestra.run.smithi070.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-10T03:53:02.623 INFO:teuthology.orchestra.run.smithi070.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 867 kB/s | 51 kB 00:00 2024-09-10T03:53:02.640 INFO:teuthology.orchestra.run.smithi070.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 707 kB/s | 42 kB 00:00 2024-09-10T03:53:02.690 INFO:teuthology.orchestra.run.smithi070.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-10T03:53:02.716 INFO:teuthology.orchestra.run.smithi070.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 579 kB/s | 43 kB 00:00 2024-09-10T03:53:02.733 INFO:teuthology.orchestra.run.smithi070.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-10T03:53:02.766 INFO:teuthology.orchestra.run.smithi070.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-09-10T03:53:02.783 INFO:teuthology.orchestra.run.smithi070.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 1.2 MB/s | 18 kB 00:00 2024-09-10T03:53:02.800 INFO:teuthology.orchestra.run.smithi070.stdout:(24/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 159 kB/s | 11 kB 00:00 2024-09-10T03:53:02.817 INFO:teuthology.orchestra.run.smithi070.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 690 kB/s | 23 kB 00:00 2024-09-10T03:53:02.834 INFO:teuthology.orchestra.run.smithi070.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 581 kB/s | 19 kB 00:00 2024-09-10T03:53:02.851 INFO:teuthology.orchestra.run.smithi070.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 585 kB/s | 20 kB 00:00 2024-09-10T03:53:02.876 INFO:teuthology.orchestra.run.smithi070.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-10T03:53:02.901 INFO:teuthology.orchestra.run.smithi070.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.9 MB/s | 358 kB 00:00 2024-09-10T03:53:02.935 INFO:teuthology.orchestra.run.smithi070.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 943 kB/s | 79 kB 00:00 2024-09-10T03:53:02.952 INFO:teuthology.orchestra.run.smithi070.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-10T03:53:02.977 INFO:teuthology.orchestra.run.smithi070.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-10T03:53:03.003 INFO:teuthology.orchestra.run.smithi070.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-10T03:53:03.028 INFO:teuthology.orchestra.run.smithi070.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-10T03:53:03.053 INFO:teuthology.orchestra.run.smithi070.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T03:53:03.079 INFO:teuthology.orchestra.run.smithi070.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-09-10T03:53:03.104 INFO:teuthology.orchestra.run.smithi070.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-10T03:53:03.107 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:03.107 INFO:teuthology.orchestra.run.smithi070.stdout:Total 13 MB/s | 30 MB 00:02 2024-09-10T03:53:03.132 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/4 2024-09-10T03:53:03.133 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/4 2024-09-10T03:53:03.133 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/4 2024-09-10T03:53:03.133 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T03:53:03.263 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:53:03.304 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:53:03.304 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:53:03.493 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:03.494 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:03.786 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:53:03.786 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:53:03.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph 2024-09-10T03:53:04.388 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:40 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:04.545 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-10T03:53:04.548 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T03:53:04.549 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T03:53:04.550 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout:Install 37 Packages 2024-09-10T03:53:04.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 30 M 2024-09-10T03:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 108 M 2024-09-10T03:53:04.552 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:53:05.026 INFO:teuthology.orchestra.run.smithi046.stdout:(1/37): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.rp 47 kB/s | 6.5 kB 00:00 2024-09-10T03:53:05.166 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:53:05.184 INFO:teuthology.orchestra.run.smithi046.stdout:(2/37): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_6 5.1 MB/s | 1.5 MB 00:00 2024-09-10T03:53:05.224 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-10T03:53:05.257 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-10T03:53:05.268 INFO:teuthology.orchestra.run.smithi046.stdout:(3/37): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_6 6.3 MB/s | 2.4 MB 00:00 2024-09-10T03:53:05.288 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-10T03:53:05.350 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-10T03:53:05.385 INFO:teuthology.orchestra.run.smithi046.stdout:(4/37): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-10T03:53:05.403 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-10T03:53:05.418 INFO:teuthology.orchestra.run.smithi046.stdout:(5/37): ceph-mgr-modules-core-19.1.1-298.gb53b6 1.6 MB/s | 249 kB 00:00 2024-09-10T03:53:05.436 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-10T03:53:05.527 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-10T03:53:05.535 INFO:teuthology.orchestra.run.smithi046.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-10T03:53:05.577 INFO:teuthology.orchestra.run.smithi046.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T03:53:05.614 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-10T03:53:05.619 INFO:teuthology.orchestra.run.smithi046.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 6.1 MB/s | 1.2 MB 00:00 2024-09-10T03:53:05.645 INFO:teuthology.orchestra.run.smithi046.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-10T03:53:05.664 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-10T03:53:05.670 INFO:teuthology.orchestra.run.smithi046.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-09-10T03:53:05.695 INFO:teuthology.orchestra.run.smithi046.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-10T03:53:05.699 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-10T03:53:05.729 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-10T03:53:05.764 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-10T03:53:05.797 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-10T03:53:05.837 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-10T03:53:05.862 INFO:teuthology.orchestra.run.smithi046.stdout:(12/37): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 25 MB/s | 17 MB 00:00 2024-09-10T03:53:05.872 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-10T03:53:05.907 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-10T03:53:05.955 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-10T03:53:05.971 INFO:teuthology.orchestra.run.smithi046.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-09-10T03:53:05.988 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-10T03:53:06.004 INFO:teuthology.orchestra.run.smithi046.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 557 kB/s | 172 kB 00:00 2024-09-10T03:53:06.030 INFO:teuthology.orchestra.run.smithi046.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 524 kB/s | 188 kB 00:00 2024-09-10T03:53:06.055 INFO:teuthology.orchestra.run.smithi046.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 919 kB/s | 77 kB 00:00 2024-09-10T03:53:06.060 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-10T03:53:06.080 INFO:teuthology.orchestra.run.smithi046.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-09-10T03:53:06.123 INFO:teuthology.orchestra.run.smithi046.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 450 kB/s | 42 kB 00:00 2024-09-10T03:53:06.173 INFO:teuthology.orchestra.run.smithi046.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-10T03:53:06.198 INFO:teuthology.orchestra.run.smithi046.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 156 kB/s | 22 kB 00:00 2024-09-10T03:53:06.200 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-10T03:53:06.223 INFO:teuthology.orchestra.run.smithi046.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-10T03:53:06.257 INFO:teuthology.orchestra.run.smithi046.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 412 kB/s | 11 kB 00:00 2024-09-10T03:53:06.278 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-10T03:53:06.282 INFO:teuthology.orchestra.run.smithi046.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-10T03:53:06.308 INFO:teuthology.orchestra.run.smithi046.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 365 kB/s | 18 kB 00:00 2024-09-10T03:53:06.333 INFO:teuthology.orchestra.run.smithi046.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-10T03:53:06.340 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-10T03:53:06.358 INFO:teuthology.orchestra.run.smithi046.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-10T03:53:06.379 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-10T03:53:06.383 INFO:teuthology.orchestra.run.smithi046.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-10T03:53:06.409 INFO:teuthology.orchestra.run.smithi046.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 924 kB/s | 46 kB 00:00 2024-09-10T03:53:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-10T03:53:06.434 INFO:teuthology.orchestra.run.smithi046.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T03:53:06.459 INFO:teuthology.orchestra.run.smithi046.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-10T03:53:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-10T03:53:06.484 INFO:teuthology.orchestra.run.smithi046.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-10T03:53:06.510 INFO:teuthology.orchestra.run.smithi046.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-10T03:53:06.535 INFO:teuthology.orchestra.run.smithi046.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-10T03:53:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-10T03:53:06.602 INFO:teuthology.orchestra.run.smithi046.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-09-10T03:53:06.627 INFO:teuthology.orchestra.run.smithi046.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-10T03:53:06.653 INFO:teuthology.orchestra.run.smithi046.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-10T03:53:06.657 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:06.657 INFO:teuthology.orchestra.run.smithi046.stdout:Total 14 MB/s | 30 MB 00:02 2024-09-10T03:53:06.674 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-10T03:53:06.727 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-10T03:53:06.796 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-10T03:53:06.810 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:53:06.853 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:53:06.853 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:53:06.884 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-10T03:53:06.918 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-10T03:53:07.034 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-10T03:53:07.190 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 32/37 2024-09-10T03:53:07.212 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-10T03:53:07.240 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-10T03:53:07.241 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:07.241 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T03:53:07.241 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T03:53:07.241 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T03:53:07.241 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:07.350 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:53:07.350 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:53:08.679 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-10T03:53:08.706 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-10T03:53:08.707 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:08.707 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T03:53:08.707 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T03:53:08.707 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T03:53:08.707 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:08.868 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:53:08.939 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-10T03:53:08.976 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-10T03:53:09.004 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-10T03:53:09.078 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-10T03:53:09.120 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-10T03:53:09.138 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-10T03:53:09.161 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-10T03:53:09.162 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-10T03:53:09.162 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:09.162 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T03:53:09.162 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T03:53:09.162 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T03:53:09.163 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:09.258 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-10T03:53:09.342 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-10T03:53:09.389 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-10T03:53:09.406 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-10T03:53:09.413 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-10T03:53:09.413 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:09.413 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T03:53:09.413 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T03:53:09.413 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T03:53:09.413 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:09.440 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-10T03:53:09.470 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-10T03:53:09.477 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-10T03:53:09.504 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-10T03:53:09.538 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-10T03:53:09.579 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-10T03:53:09.622 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-10T03:53:09.656 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-10T03:53:09.706 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-10T03:53:09.746 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-10T03:53:09.830 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-10T03:53:10.097 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-10T03:53:10.181 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-10T03:53:10.357 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-10T03:53:10.395 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-10T03:53:10.453 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-10T03:53:10.517 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-10T03:53:10.722 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-10T03:53:10.775 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-10T03:53:10.857 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-10T03:53:10.945 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-10T03:53:11.100 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-10T03:53:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-10T03:53:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/37 2024-09-10T03:53:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/37 2024-09-10T03:53:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/37 2024-09-10T03:53:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/37 2024-09-10T03:53:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/37 2024-09-10T03:53:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 6/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-10T03:53:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-10T03:53:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-10T03:53:11.228 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-10T03:53:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-10T03:53:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-10T03:53:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-10T03:53:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-10T03:53:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-10T03:53:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-10T03:53:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-10T03:53:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-10T03:53:11.412 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 32/37 2024-09-10T03:53:11.476 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-10T03:53:11.504 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 33/37 2024-09-10T03:53:11.544 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:11.545 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T03:53:11.545 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T03:53:11.545 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T03:53:11.545 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:12.300 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T03:53:12.301 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T03:53:12.302 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:12.303 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:12.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-base 2024-09-10T03:53:13.027 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-10T03:53:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 34/37 2024-09-10T03:53:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T03:53:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T03:53:13.056 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T03:53:13.056 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:13.248 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:13.337 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:53:13.379 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:13.380 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:53:13.380 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:13.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install cephadm 2024-09-10T03:53:13.519 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-10T03:53:13.545 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 35/37 2024-09-10T03:53:13.545 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:13.545 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T03:53:13.545 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T03:53:13.545 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T03:53:13.546 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:13.787 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-10T03:53:13.812 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/37 2024-09-10T03:53:13.812 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:13.812 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T03:53:13.812 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T03:53:13.813 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T03:53:13.813 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:13.879 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-10T03:53:14.012 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:14.144 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:14.144 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:14.144 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 765 k 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 771 k 2024-09-10T03:53:14.145 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:53:14.421 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm-19.1.1-298.gb53b6d3e.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-10T03:53:14.422 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:14.422 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-10T03:53:14.422 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:53:14.427 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:53:14.427 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:53:14.433 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:53:14.433 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:53:14.509 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:53:14.962 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:15.041 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:15.409 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:15.631 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 37/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 5/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.n 6/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-10T03:53:15.632 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-10T03:53:15.633 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-10T03:53:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-10T03:53:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-10T03:53:15.635 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-10T03:53:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-10T03:53:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-10T03:53:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-10T03:53:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-10T03:53:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-10T03:53:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-10T03:53:15.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-10T03:53:15.665 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:15.666 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:15.666 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:53:15.666 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:15.666 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:15.666 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:15.882 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T03:53:16.453 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:16.587 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:16.588 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:16.588 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-10T03:53:16.588 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:16.588 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:53:16.588 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 145 k 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 435 k 2024-09-10T03:53:16.589 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:53:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-10T03:53:16.690 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:16.691 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:53:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:16.691 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T03:53:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T03:53:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T03:53:16.692 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T03:53:16.693 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T03:53:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T03:53:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T03:53:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T03:53:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T03:53:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T03:53:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T03:53:16.694 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T03:53:16.695 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T03:53:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T03:53:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T03:53:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T03:53:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T03:53:16.696 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:16.696 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:16.795 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-immutable-object-cache-19.1.1-298.gb53b6d3 704 kB/s | 145 kB 00:00 2024-09-10T03:53:16.796 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:16.796 INFO:teuthology.orchestra.run.smithi070.stdout:Total 698 kB/s | 145 kB 00:00 2024-09-10T03:53:16.797 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:53:16.804 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:53:16.804 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:53:16.863 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:53:16.863 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:53:17.020 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:53:17.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-base 2024-09-10T03:53:17.113 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:17.661 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:17.755 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:53:17.799 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:17.800 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:53:17.800 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:17.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install cephadm 2024-09-10T03:53:18.058 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-10T03:53:18.058 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:18.058 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:53:18.058 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:18.058 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:18.058 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:18.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr 2024-09-10T03:53:18.454 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:18.592 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:18.592 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:18.592 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 765 k 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 771 k 2024-09-10T03:53:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:53:18.860 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:57 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:18.862 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm-19.1.1-298.gb53b6d3e.el9.noarch.rpm 2.8 MB/s | 765 kB 00:00 2024-09-10T03:53:18.863 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:18.863 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-10T03:53:18.863 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:53:18.868 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:53:18.868 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:53:18.874 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:53:18.875 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:53:18.951 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:53:18.954 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:53:18.997 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:18.998 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:53:18.998 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:19.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T03:53:19.378 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:19.477 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:19.628 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:58 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:19.761 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:19.762 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:19.762 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:53:19.762 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T03:53:19.763 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T03:53:19.764 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T03:53:19.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:19.764 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:53:19.764 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:19.764 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-09-10T03:53:19.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:19.764 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 8.7 M 2024-09-10T03:53:19.765 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 104 M 2024-09-10T03:53:19.765 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:53:19.854 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:20.099 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/1 2024-09-10T03:53:20.099 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:20.099 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:53:20.099 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:20.099 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:20.099 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:20.135 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): ceph-grafana-dashboards-19.1.1-298.gb53 208 kB/s | 32 kB 00:00 2024-09-10T03:53:20.160 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): ceph-prometheus-alerts-19.1.1-298.gb53b 92 kB/s | 16 kB 00:00 2024-09-10T03:53:20.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T03:53:20.377 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): ceph-mgr-dashboard-19.1.1-298.gb53b6d3e 9.0 MB/s | 3.6 MB 00:00 2024-09-10T03:53:20.403 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.1 MB/s | 269 kB 00:00 2024-09-10T03:53:20.461 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-10T03:53:20.495 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 585 kB/s | 19 kB 00:00 2024-09-10T03:53:20.528 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-09-10T03:53:20.604 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 19 MB/s | 2.0 MB 00:00 2024-09-10T03:53:20.629 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.4 MB/s | 144 kB 00:00 2024-09-10T03:53:20.654 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 615 kB/s | 31 kB 00:00 2024-09-10T03:53:20.680 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.8 MB/s | 863 kB 00:00 2024-09-10T03:53:20.705 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-10T03:53:20.709 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:20.709 INFO:teuthology.orchestra.run.smithi070.stdout:Total 9.2 MB/s | 8.7 MB 00:00 2024-09-10T03:53:20.757 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:53:20.773 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:53:20.774 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:53:20.907 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:20.978 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:53:20.979 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:53:21.046 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:21.046 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:21.046 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-10T03:53:21.047 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:21.048 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 145 k 2024-09-10T03:53:21.048 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 435 k 2024-09-10T03:53:21.048 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:53:21.261 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-immutable-object-cache-19.1.1-298.gb53b6d3 681 kB/s | 145 kB 00:00 2024-09-10T03:53:21.261 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:21.261 INFO:teuthology.orchestra.run.smithi046.stdout:Total 677 kB/s | 145 kB 00:00 2024-09-10T03:53:21.262 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:53:21.269 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:53:21.270 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:53:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:53:21.330 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:53:21.484 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:53:21.508 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:53:21.553 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-10T03:53:21.579 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-10T03:53:21.579 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:53:21.579 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T03:53:21.580 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:21.588 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-10T03:53:21.633 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-10T03:53:21.679 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-10T03:53:21.718 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-10T03:53:21.835 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-10T03:53:21.970 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-10T03:53:22.063 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-10T03:53:22.115 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-10T03:53:22.143 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-10T03:53:22.175 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 10/12 2024-09-10T03:53:22.546 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.e 1/1 2024-09-10T03:53:22.546 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:22.546 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:53:22.547 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:53:22.547 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:22.547 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:22.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr 2024-09-10T03:53:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 11/12 2024-09-10T03:53:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-10T03:53:23.364 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:59 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:23.457 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:53:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:53:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:23.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T03:53:24.149 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:24.288 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-10T03:53:24.289 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout:Install 12 Packages 2024-09-10T03:53:24.290 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:24.291 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 8.7 M 2024-09-10T03:53:24.291 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 104 M 2024-09-10T03:53:24.291 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 1/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 2/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 3/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-10T03:53:24.450 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T03:53:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T03:53:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T03:53:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T03:53:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T03:53:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T03:53:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:24.774 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:25.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T03:53:25.057 INFO:teuthology.orchestra.run.smithi046.stdout:(1/12): ceph-grafana-dashboards-19.1.1-298.gb53 224 kB/s | 32 kB 00:00 2024-09-10T03:53:25.082 INFO:teuthology.orchestra.run.smithi046.stdout:(2/12): ceph-prometheus-alerts-19.1.1-298.gb53b 98 kB/s | 16 kB 00:00 2024-09-10T03:53:25.291 INFO:teuthology.orchestra.run.smithi046.stdout:(3/12): ceph-mgr-dashboard-19.1.1-298.gb53b6d3e 9.5 MB/s | 3.6 MB 00:00 2024-09-10T03:53:25.451 INFO:teuthology.orchestra.run.smithi046.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.3 MB/s | 863 kB 00:00 2024-09-10T03:53:25.501 INFO:teuthology.orchestra.run.smithi046.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi046.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-10T03:53:25.570 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:04 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:25.594 INFO:teuthology.orchestra.run.smithi046.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-10T03:53:25.619 INFO:teuthology.orchestra.run.smithi046.stdout:(8/12): python3-protobuf-3.14.0-13.el9.noarch.r 502 kB/s | 269 kB 00:00 2024-09-10T03:53:25.653 INFO:teuthology.orchestra.run.smithi046.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-09-10T03:53:25.678 INFO:teuthology.orchestra.run.smithi046.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 526 kB/s | 31 kB 00:00 2024-09-10T03:53:25.703 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:25.703 INFO:teuthology.orchestra.run.smithi046.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.8 MB/s | 188 kB 00:00 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T03:53:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout:====================================================================================== 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-09-10T03:53:25.705 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:25.706 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 38 M 2024-09-10T03:53:25.706 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 212 M 2024-09-10T03:53:25.706 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:53:25.820 INFO:teuthology.orchestra.run.smithi046.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2024-09-10T03:53:25.825 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:25.825 INFO:teuthology.orchestra.run.smithi046.stdout:Total 5.6 MB/s | 8.7 MB 00:01 2024-09-10T03:53:25.877 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:53:25.897 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:53:25.897 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:53:26.118 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:53:26.119 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:53:26.121 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-10T03:53:26.171 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-10T03:53:26.380 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 126 kB/s | 33 kB 00:00 2024-09-10T03:53:26.481 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): ceph-mgr-diskprediction-local-19.1.1-29 15 MB/s | 7.4 MB 00:00 2024-09-10T03:53:26.498 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 156 kB/s | 18 kB 00:00 2024-09-10T03:53:26.670 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:53:26.746 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-10T03:53:26.748 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 144 kB/s | 38 kB 00:00 2024-09-10T03:53:26.789 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-10T03:53:26.837 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-10T03:53:26.875 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-10T03:53:26.970 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-10T03:53:27.115 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 667 kB/s | 245 kB 00:00 2024-09-10T03:53:27.123 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-10T03:53:27.215 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-10T03:53:27.232 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-10T03:53:27.280 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-10T03:53:27.325 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-10T03:53:27.349 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-10T03:53:27.356 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 10/12 2024-09-10T03:53:28.471 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 11/12 2024-09-10T03:53:28.508 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-10T03:53:29.340 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 1.7 MB/s | 4.9 MB 00:02 2024-09-10T03:53:29.574 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.1 MB/s | 5.1 MB 00:02 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 12/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9 1/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noar 2/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9. 3/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-10T03:53:29.858 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-10T03:53:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-10T03:53:29.859 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-10T03:53:30.180 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T03:53:30.181 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T03:53:30.182 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T03:53:30.182 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T03:53:30.182 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:30.182 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:30.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T03:53:31.005 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:07 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:31.145 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T03:53:31.146 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout:====================================================================================== 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout:Install 12 Packages 2024-09-10T03:53:31.147 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:31.148 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 38 M 2024-09-10T03:53:31.148 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 212 M 2024-09-10T03:53:31.148 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:53:31.762 INFO:teuthology.orchestra.run.smithi046.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-29 16 MB/s | 7.4 MB 00:00 2024-09-10T03:53:31.804 INFO:teuthology.orchestra.run.smithi046.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 378 kB/s | 191 kB 00:00 2024-09-10T03:53:32.029 INFO:teuthology.orchestra.run.smithi046.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 122 kB/s | 33 kB 00:00 2024-09-10T03:53:32.063 INFO:teuthology.orchestra.run.smithi046.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-10T03:53:32.129 INFO:teuthology.orchestra.run.smithi046.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 181 kB/s | 18 kB 00:00 2024-09-10T03:53:32.163 INFO:teuthology.orchestra.run.smithi046.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-10T03:53:32.272 INFO:teuthology.orchestra.run.smithi046.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T03:53:32.547 INFO:teuthology.orchestra.run.smithi046.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.0 MB/s | 3.0 MB 00:00 2024-09-10T03:53:32.656 INFO:teuthology.orchestra.run.smithi046.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 250 kB 00:00 2024-09-10T03:53:32.931 INFO:teuthology.orchestra.run.smithi046.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.7 MB/s | 5.1 MB 00:00 2024-09-10T03:53:33.356 INFO:teuthology.orchestra.run.smithi046.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.0 MB/s | 4.9 MB 00:01 2024-09-10T03:53:46.679 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 861 kB/s | 16 MB 00:19 2024-09-10T03:53:46.680 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:46.681 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.8 MB/s | 38 MB 00:20 2024-09-10T03:53:47.037 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:53:47.063 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:53:47.063 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:53:47.497 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:53:47.497 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:53:48.270 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:53:48.347 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T03:53:48.386 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T03:53:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T03:53:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T03:53:48.731 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T03:53:48.760 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T03:53:48.847 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T03:53:49.499 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T03:53:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T03:53:50.213 INFO:teuthology.orchestra.run.smithi046.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 948 kB/s | 16 MB 00:17 2024-09-10T03:53:50.214 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:50.214 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.0 MB/s | 38 MB 00:19 2024-09-10T03:53:50.582 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:53:50.609 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:53:50.609 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:53:50.843 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T03:53:51.056 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:53:51.056 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:53:51.844 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:53:51.884 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T03:53:51.919 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-10T03:53:51.925 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T03:53:51.970 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T03:53:52.074 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T03:53:52.113 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T03:53:52.305 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T03:53:52.336 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T03:53:52.417 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T03:53:53.095 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T03:53:53.212 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T03:53:54.548 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T03:53:54.642 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-10T03:53:54.642 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 1/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T03:53:54.643 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T03:53:55.045 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T03:53:55.045 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:55.045 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:53:55.045 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:55.045 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T03:53:55.045 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:55.046 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:53:55.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-rook 2024-09-10T03:53:55.626 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T03:53:55.652 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-10T03:53:56.002 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:53:56.138 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:53:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T03:53:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T03:53:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T03:53:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T03:53:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:56.141 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:53:56.141 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:53:56.141 INFO:teuthology.orchestra.run.smithi070.stdout:Install 13 Packages 2024-09-10T03:53:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:53:56.142 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.2 M 2024-09-10T03:53:56.142 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 26 M 2024-09-10T03:53:56.142 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:53:56.421 INFO:teuthology.orchestra.run.smithi070.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 228 kB/s | 19 kB 00:00 2024-09-10T03:53:56.445 INFO:teuthology.orchestra.run.smithi070.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 234 kB/s | 26 kB 00:00 2024-09-10T03:53:56.504 INFO:teuthology.orchestra.run.smithi070.stdout:(3/13): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el9. 293 kB/s | 49 kB 00:00 2024-09-10T03:53:56.530 INFO:teuthology.orchestra.run.smithi070.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-10T03:53:56.555 INFO:teuthology.orchestra.run.smithi070.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-10T03:53:56.580 INFO:teuthology.orchestra.run.smithi070.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-10T03:53:56.622 INFO:teuthology.orchestra.run.smithi070.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-10T03:53:56.647 INFO:teuthology.orchestra.run.smithi070.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-10T03:53:56.672 INFO:teuthology.orchestra.run.smithi070.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.6 MB/s | 279 kB 00:00 2024-09-10T03:53:56.698 INFO:teuthology.orchestra.run.smithi070.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-10T03:53:56.732 INFO:teuthology.orchestra.run.smithi070.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-10T03:53:56.756 INFO:teuthology.orchestra.run.smithi070.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-09-10T03:53:56.798 INFO:teuthology.orchestra.run.smithi070.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-10T03:53:56.801 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:56.801 INFO:teuthology.orchestra.run.smithi070.stdout:Total 3.4 MB/s | 2.2 MB 00:00 2024-09-10T03:53:56.864 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:53:56.879 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:53:56.879 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:53:56.967 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:53:56.967 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:53:57.463 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:53:57.629 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T03:53:57.691 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T03:53:57.755 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T03:53:57.793 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T03:53:57.835 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T03:53:57.943 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T03:53:58.054 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T03:53:58.110 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T03:53:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 12/12 2024-09-10T03:53:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d 1/12 2024-09-10T03:53:58.195 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T03:53:58.196 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T03:53:58.629 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T03:53:58.629 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:58.629 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:53:58.629 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:53:58.629 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T03:53:58.629 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:58.630 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:53:58.821 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T03:53:58.892 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T03:53:58.925 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T03:53:58.973 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T03:53:58.995 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-10T03:53:59.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-rook 2024-09-10T03:53:59.592 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:53:59.734 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T03:53:59.735 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:53:59.736 INFO:teuthology.orchestra.run.smithi046.stdout:Install 13 Packages 2024-09-10T03:53:59.737 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:53:59.737 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 2.2 M 2024-09-10T03:53:59.737 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 26 M 2024-09-10T03:53:59.737 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:53:59.779 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-10T03:53:59.779 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 1/13 2024-09-10T03:53:59.779 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T03:53:59.779 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T03:53:59.779 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T03:53:59.780 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T03:53:59.922 INFO:teuthology.orchestra.run.smithi046.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 145 kB/s | 19 kB 00:00 2024-09-10T03:53:59.947 INFO:teuthology.orchestra.run.smithi046.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 163 kB/s | 26 kB 00:00 2024-09-10T03:53:59.972 INFO:teuthology.orchestra.run.smithi046.stdout:(3/13): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el9. 269 kB/s | 49 kB 00:00 2024-09-10T03:54:00.039 INFO:teuthology.orchestra.run.smithi046.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-10T03:54:00.064 INFO:teuthology.orchestra.run.smithi046.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-10T03:54:00.098 INFO:teuthology.orchestra.run.smithi046.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 915 kB/s | 54 kB 00:00 2024-09-10T03:54:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T03:54:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:00.116 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:54:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T03:54:00.116 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:00.117 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:00.132 INFO:teuthology.orchestra.run.smithi046.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-10T03:54:00.157 INFO:teuthology.orchestra.run.smithi046.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 247 kB/s | 14 kB 00:00 2024-09-10T03:54:00.182 INFO:teuthology.orchestra.run.smithi046.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-10T03:54:00.216 INFO:teuthology.orchestra.run.smithi046.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-10T03:54:00.241 INFO:teuthology.orchestra.run.smithi046.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-10T03:54:00.292 INFO:teuthology.orchestra.run.smithi046.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 7.6 MB/s | 1.0 MB 00:00 2024-09-10T03:54:00.317 INFO:teuthology.orchestra.run.smithi046.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 893 kB/s | 90 kB 00:00 2024-09-10T03:54:00.319 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:00.319 INFO:teuthology.orchestra.run.smithi046.stdout:Total 3.8 MB/s | 2.2 MB 00:00 2024-09-10T03:54:00.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T03:54:00.385 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:00.401 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:00.401 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:00.499 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:00.499 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:00.940 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:01.041 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:01.074 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:01.074 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:54:01.075 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:01.076 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-09-10T03:54:01.076 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:01.076 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 7.0 M 2024-09-10T03:54:01.076 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 33 M 2024-09-10T03:54:01.076 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:54:01.274 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T03:54:01.337 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T03:54:01.392 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T03:54:01.428 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T03:54:01.443 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-09-10T03:54:01.469 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T03:54:01.544 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 6.0 MB 00:00 2024-09-10T03:54:01.569 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e.el 509 kB/s | 146 kB 00:00 2024-09-10T03:54:01.586 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T03:54:01.611 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 888 kB/s | 58 kB 00:00 2024-09-10T03:54:01.627 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-10T03:54:01.653 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.6 MB/s | 548 kB 00:00 2024-09-10T03:54:01.656 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:01.656 INFO:teuthology.orchestra.run.smithi070.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-10T03:54:01.695 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T03:54:01.727 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:54:01.738 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:54:01.738 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:54:01.763 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T03:54:01.838 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:54:01.839 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:54:02.143 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:54:02.310 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T03:54:02.371 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T03:54:02.542 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T03:54:02.602 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T03:54:02.652 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T03:54:02.709 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T03:54:02.746 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-10T03:54:02.906 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T03:54:02.991 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T03:54:03.064 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T03:54:03.092 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-10T03:54:03.482 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 13/13 2024-09-10T03:54:03.482 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 1/13 2024-09-10T03:54:03.482 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T03:54:03.483 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T03:54:03.817 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T03:54:03.818 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:03.819 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:03.924 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-10T03:54:03.925 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/6 2024-09-10T03:54:03.925 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T03:54:03.925 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T03:54:03.925 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T03:54:03.925 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T03:54:04.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T03:54:04.206 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:04.207 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:04.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-fuse 2024-09-10T03:54:04.686 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:04.828 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:04.829 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:04.829 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:54:04.829 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:04.829 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout:Install 6 Packages 2024-09-10T03:54:04.830 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:04.831 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 7.0 M 2024-09-10T03:54:04.831 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 33 M 2024-09-10T03:54:04.831 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:54:05.015 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:44 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:05.150 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-09-10T03:54:05.151 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 977 k 2024-09-10T03:54:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 2.9 M 2024-09-10T03:54:05.152 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:54:05.463 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 482 kB/s | 80 kB 00:00 2024-09-10T03:54:05.588 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86_6 3.0 MB/s | 897 kB 00:00 2024-09-10T03:54:05.589 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:05.589 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.2 MB/s | 977 kB 00:00 2024-09-10T03:54:05.601 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:54:05.624 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:54:05.624 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:54:05.706 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:54:05.706 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:54:05.894 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:54:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:(1/6): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e.el 670 kB/s | 146 kB 00:00 2024-09-10T03:54:05.994 INFO:teuthology.orchestra.run.smithi046.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.1 MB/s | 548 kB 00:00 2024-09-10T03:54:06.019 INFO:teuthology.orchestra.run.smithi046.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-10T03:54:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T03:54:06.053 INFO:teuthology.orchestra.run.smithi046.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-10T03:54:06.096 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-10T03:54:06.520 INFO:teuthology.orchestra.run.smithi046.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 306 kB/s | 248 kB 00:00 2024-09-10T03:54:06.773 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-10T03:54:06.773 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/2 2024-09-10T03:54:06.987 INFO:teuthology.orchestra.run.smithi046.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-10T03:54:06.989 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:06.989 INFO:teuthology.orchestra.run.smithi046.stdout:Total 3.2 MB/s | 7.0 MB 00:02 2024-09-10T03:54:07.023 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T03:54:07.023 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:07.023 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:07.023 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T03:54:07.023 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:07.023 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:07.064 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:07.077 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:07.077 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:07.183 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:07.183 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:07.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados-devel 2024-09-10T03:54:07.509 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:07.685 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T03:54:07.732 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T03:54:07.897 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:08.033 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:08.034 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:08.034 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-10T03:54:08.034 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:08.034 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:54:08.034 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-10T03:54:08.034 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:08.035 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:54:08.035 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:08.035 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-10T03:54:08.035 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:08.035 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 127 k 2024-09-10T03:54:08.035 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 456 k 2024-09-10T03:54:08.035 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:54:08.244 INFO:teuthology.orchestra.run.smithi070.stdout:librados-devel-19.1.1-298.gb53b6d3e.el9.x86_64. 607 kB/s | 127 kB 00:00 2024-09-10T03:54:08.244 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:08.245 INFO:teuthology.orchestra.run.smithi070.stdout:Total 602 kB/s | 127 kB 00:00 2024-09-10T03:54:08.245 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:54:08.250 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:54:08.250 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:54:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T03:54:08.314 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:54:08.314 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:54:08.420 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T03:54:08.468 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:54:08.500 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T03:54:08.543 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-10T03:54:08.583 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:09.000 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:09.288 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:09.289 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:09.289 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 6/6 2024-09-10T03:54:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 1/6 2024-09-10T03:54:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T03:54:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T03:54:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T03:54:09.291 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T03:54:09.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs2 2024-09-10T03:54:09.566 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:09.567 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:09.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-fuse 2024-09-10T03:54:10.117 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:49 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:10.210 INFO:teuthology.orchestra.run.smithi070.stdout:Package libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:10.254 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:10.255 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:54:10.255 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:10.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs-devel 2024-09-10T03:54:10.418 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:10.561 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:10.561 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:10.561 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:Install 2 Packages 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 977 k 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 2.9 M 2024-09-10T03:54:10.562 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:54:10.755 INFO:teuthology.orchestra.run.smithi046.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 542 kB/s | 80 kB 00:00 2024-09-10T03:54:10.881 INFO:teuthology.orchestra.run.smithi046.stdout:(2/2): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86_6 3.2 MB/s | 897 kB 00:00 2024-09-10T03:54:10.881 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:10.882 INFO:teuthology.orchestra.run.smithi046.stdout:Total 3.0 MB/s | 977 kB 00:00 2024-09-10T03:54:10.893 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:10.902 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:49 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:10.916 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:10.917 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:11.036 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:11.037 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:11.037 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-10T03:54:11.037 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:11.037 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:54:11.037 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-10T03:54:11.037 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:11.037 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:54:11.038 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:11.038 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-10T03:54:11.038 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:11.038 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 32 k 2024-09-10T03:54:11.038 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 142 k 2024-09-10T03:54:11.038 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:54:11.181 INFO:teuthology.orchestra.run.smithi070.stdout:libcephfs-devel-19.1.1-298.gb53b6d3e.el9.x86_64 222 kB/s | 32 kB 00:00 2024-09-10T03:54:11.181 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:11.181 INFO:teuthology.orchestra.run.smithi070.stdout:Total 220 kB/s | 32 kB 00:00 2024-09-10T03:54:11.182 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:54:11.185 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:54:11.185 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:54:11.197 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:11.200 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:54:11.200 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:54:11.284 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:54:11.345 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T03:54:11.377 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:11.416 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-10T03:54:11.792 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:12.028 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:12.028 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:12.028 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:12.029 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:12.029 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:12.029 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:12.113 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/2 2024-09-10T03:54:12.113 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/2 2024-09-10T03:54:12.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-09-10T03:54:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T03:54:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:12.425 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:12.425 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T03:54:12.426 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:12.426 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:12.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados-devel 2024-09-10T03:54:12.787 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:51 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:12.877 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:12.920 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:12.921 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:54:12.921 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:12.999 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librbd1 2024-09-10T03:54:13.314 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:49 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:13.453 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:54:13.454 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:13.455 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-10T03:54:13.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:13.455 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 127 k 2024-09-10T03:54:13.455 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 456 k 2024-09-10T03:54:13.455 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:54:13.566 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:52 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:13.636 INFO:teuthology.orchestra.run.smithi046.stdout:librados-devel-19.1.1-298.gb53b6d3e.el9.x86_64. 700 kB/s | 127 kB 00:00 2024-09-10T03:54:13.637 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:13.637 INFO:teuthology.orchestra.run.smithi046.stdout:Total 695 kB/s | 127 kB 00:00 2024-09-10T03:54:13.637 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:13.643 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:13.643 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:13.657 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:13.701 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:13.702 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:54:13.702 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:13.710 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:13.710 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:13.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rados 2024-09-10T03:54:13.872 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:13.987 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:14.338 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:53 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:14.408 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:14.429 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:14.472 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:14.473 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:54:14.473 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:14.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rgw 2024-09-10T03:54:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:14.654 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:14.654 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:14.654 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:14.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs2 2024-09-10T03:54:15.107 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:54 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:15.198 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:15.242 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:15.243 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:54:15.243 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:15.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-cephfs 2024-09-10T03:54:15.530 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:51 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:15.627 INFO:teuthology.orchestra.run.smithi046.stdout:Package libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:15.672 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:15.673 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:54:15.673 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:15.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs-devel 2024-09-10T03:54:15.873 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:54 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:15.963 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:16.006 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:16.007 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:54:16.007 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:16.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rbd 2024-09-10T03:54:16.343 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:52 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:16.483 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:16.483 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 32 k 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 142 k 2024-09-10T03:54:16.484 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:54:16.640 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:55 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:16.641 INFO:teuthology.orchestra.run.smithi046.stdout:libcephfs-devel-19.1.1-298.gb53b6d3e.el9.x86_64 204 kB/s | 32 kB 00:00 2024-09-10T03:54:16.641 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:16.641 INFO:teuthology.orchestra.run.smithi046.stdout:Total 202 kB/s | 32 kB 00:00 2024-09-10T03:54:16.641 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:16.644 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:16.645 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:16.664 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:16.664 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:16.731 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:16.748 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:16.774 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:16.775 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-10T03:54:16.775 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:16.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-fuse 2024-09-10T03:54:16.866 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:17.261 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:17.411 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:56 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:17.497 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:17.497 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:17.498 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:17.498 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:17.545 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:17.546 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:17.546 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-10T03:54:17.546 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:17.546 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:54:17.546 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 85 k 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 231 k 2024-09-10T03:54:17.547 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:54:17.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados2 2024-09-10T03:54:17.725 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 478 kB/s | 85 kB 00:00 2024-09-10T03:54:17.726 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:17.726 INFO:teuthology.orchestra.run.smithi070.stdout:Total 475 kB/s | 85 kB 00:00 2024-09-10T03:54:17.726 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:54:17.733 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:54:17.733 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:54:17.796 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:54:17.796 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:54:17.943 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:54:18.039 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:18.270 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:54 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:18.365 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:18.401 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:18.411 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:18.412 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:54:18.412 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:18.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librbd1 2024-09-10T03:54:18.645 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:18.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:18.649 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:18.649 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:18.649 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:18.649 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:18.884 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-mirror 2024-09-10T03:54:19.074 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:55 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:19.168 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:19.214 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:19.215 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:54:19.215 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:19.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rados 2024-09-10T03:54:19.461 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:58 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:19.595 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:54:19.596 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:19.596 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-10T03:54:19.596 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:19.596 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 3.1 M 2024-09-10T03:54:19.596 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 13 M 2024-09-10T03:54:19.596 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:54:19.884 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:55 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:19.930 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-mirror-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 9.3 MB/s | 3.1 MB 00:00 2024-09-10T03:54:19.930 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:19.931 INFO:teuthology.orchestra.run.smithi070.stdout:Total 9.3 MB/s | 3.1 MB 00:00 2024-09-10T03:54:19.931 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:54:19.939 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:54:19.939 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:54:19.978 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:20.020 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:54:20.020 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:54:20.026 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:20.027 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:54:20.027 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:20.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rgw 2024-09-10T03:54:20.445 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:54:20.497 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:20.522 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:20.522 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:54:20.522 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T03:54:20.522 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T03:54:20.522 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T03:54:20.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:20.685 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:56 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:20.780 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:20.826 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:20.827 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:54:20.828 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:20.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-cephfs 2024-09-10T03:54:21.501 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:57 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:21.594 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:21.627 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:21.627 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:21.627 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:21.627 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:21.627 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:21.628 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:21.641 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:21.642 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:54:21.642 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:21.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rbd 2024-09-10T03:54:21.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-nbd 2024-09-10T03:54:22.301 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:58 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:22.396 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 is already installed. 2024-09-10T03:54:22.442 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:22.443 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-09-10T03:54:22.443 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:22.475 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:02:01 ago on Tue 10 Sep 2024 03:52:21 AM UTC. 2024-09-10T03:54:22.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-fuse 2024-09-10T03:54:22.610 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-10T03:54:22.610 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:22.610 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 171 k 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 494 k 2024-09-10T03:54:22.611 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-10T03:54:22.826 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 796 kB/s | 171 kB 00:00 2024-09-10T03:54:22.827 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:22.827 INFO:teuthology.orchestra.run.smithi070.stdout:Total 790 kB/s | 171 kB 00:00 2024-09-10T03:54:22.827 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-10T03:54:22.834 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-10T03:54:22.835 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-10T03:54:22.902 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-10T03:54:22.902 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-10T03:54:23.053 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-10T03:54:23.103 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:59 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:23.174 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:23.244 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:23.244 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 85 k 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 231 k 2024-09-10T03:54:23.245 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:54:23.485 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 355 kB/s | 85 kB 00:00 2024-09-10T03:54:23.486 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:23.486 INFO:teuthology.orchestra.run.smithi046.stdout:Total 353 kB/s | 85 kB 00:00 2024-09-10T03:54:23.486 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:23.493 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:23.493 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:23.561 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:23.562 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:23.644 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:23.713 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:23.819 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:23.920 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:23.920 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:23.920 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-10T03:54:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:23.921 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:54:23.921 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-10T03:54:24.173 DEBUG:teuthology.parallel:result is None 2024-09-10T03:54:24.188 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:24.481 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:24.481 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:24.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-mirror 2024-09-10T03:54:25.343 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:01 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:25.483 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:25.484 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-09-10T03:54:25.484 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:25.484 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:54:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-10T03:54:25.484 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:25.484 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:54:25.485 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:25.485 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-10T03:54:25.485 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:25.485 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 3.1 M 2024-09-10T03:54:25.485 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 13 M 2024-09-10T03:54:25.485 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:54:25.813 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-mirror-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 9.5 MB/s | 3.1 MB 00:00 2024-09-10T03:54:25.814 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:25.814 INFO:teuthology.orchestra.run.smithi046.stdout:Total 9.4 MB/s | 3.1 MB 00:00 2024-09-10T03:54:25.814 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:25.824 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:25.824 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:25.910 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:25.911 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:26.364 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:26.439 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:26.466 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:26.466 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T03:54:26.466 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T03:54:26.466 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T03:54:26.466 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T03:54:26.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:27.641 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:27.641 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:27.641 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:27.641 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:27.641 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:27.641 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:27.911 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-nbd 2024-09-10T03:54:28.513 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:04 ago on Tue 10 Sep 2024 03:52:24 AM UTC. 2024-09-10T03:54:28.653 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-09-10T03:54:28.653 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-09-10T03:54:28.654 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:28.654 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-09-10T03:54:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-10T03:54:28.654 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:28.654 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-09-10T03:54:28.654 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-09-10T03:54:28.655 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-09-10T03:54:28.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:28.655 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 171 k 2024-09-10T03:54:28.655 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 494 k 2024-09-10T03:54:28.655 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-09-10T03:54:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_64.rpm 770 kB/s | 171 kB 00:00 2024-09-10T03:54:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:Total 764 kB/s | 171 kB 00:00 2024-09-10T03:54:28.878 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-09-10T03:54:28.886 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-09-10T03:54:28.886 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-09-10T03:54:28.954 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-09-10T03:54:28.954 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-09-10T03:54:29.116 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-09-10T03:54:29.227 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:29.663 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:29.928 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/1 2024-09-10T03:54:29.928 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:29.928 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-09-10T03:54:29.928 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-10T03:54:29.928 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:54:29.928 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-09-10T03:54:30.155 DEBUG:teuthology.parallel:result is None 2024-09-10T03:54:30.156 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:54:30.156 INFO:teuthology.packaging:ref: None 2024-09-10T03:54:30.156 INFO:teuthology.packaging:tag: None 2024-09-10T03:54:30.156 INFO:teuthology.packaging:branch: squid 2024-09-10T03:54:30.156 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:54:30.157 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-10T03:54:30.395 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T03:54:30.418 INFO:teuthology.orchestra.run.smithi046.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-10T03:54:30.419 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-10T03:54:30.419 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-10T03:54:30.420 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:54:30.421 INFO:teuthology.packaging:ref: None 2024-09-10T03:54:30.421 INFO:teuthology.packaging:tag: None 2024-09-10T03:54:30.421 INFO:teuthology.packaging:branch: squid 2024-09-10T03:54:30.421 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:54:30.421 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-10T03:54:30.652 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T03:54:30.678 INFO:teuthology.orchestra.run.smithi070.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-10T03:54:30.678 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-10T03:54:30.678 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-10T03:54:30.680 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T03:54:30.681 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:54:30.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T03:54:30.711 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:54:30.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T03:54:30.750 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T03:54:30.750 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:54:30.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T03:54:30.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T03:54:30.851 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:54:30.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T03:54:30.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T03:54:30.947 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T03:54:30.947 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:54:30.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T03:54:30.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T03:54:31.045 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:54:31.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T03:54:31.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T03:54:31.140 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T03:54:31.140 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:54:31.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T03:54:31.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T03:54:31.238 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:54:31.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T03:54:31.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T03:54:31.330 INFO:teuthology.run_tasks:Running task print... 2024-09-10T03:54:31.339 INFO:teuthology.task.print:**** done install task... 2024-09-10T03:54:31.340 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T03:54:31.467 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'squid', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'global': {'mon pg warn min per osd': 0}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}}, 'image': 'quay.ceph.io/ceph-ci/ceph:squid', 'roleless': True, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c'} 2024-09-10T03:54:31.467 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:squid 2024-09-10T03:54:31.468 INFO:tasks.cephadm:Cluster fsid is 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:31.468 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T03:54:31.468 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-10T03:54:31.468 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi046': '172.21.15.46', 'mon.smithi070': '172.21.15.70'} 2024-09-10T03:54:31.468 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T03:54:31.468 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-09-10T03:54:31.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2024-09-10T03:54:31.524 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for squid 2024-09-10T03:54:31.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:54:31.667 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82931/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4885-g3a5e3f16', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4885.g3a5e3f16'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 02:14:59.834535', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-adk-testing-2024-09-09-1904', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T03:54:31.892 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=squid 2024-09-10T03:54:31.894 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-10T03:54:31.894 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-10T03:54:31.894 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T03:54:32.289 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 10 03:54 /home/ubuntu/cephtest/cephadm 2024-09-10T03:54:32.290 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T03:54:32.707 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 10 03:54 /home/ubuntu/cephtest/cephadm 2024-09-10T03:54:32.707 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T03:54:32.728 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T03:54:32.758 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:squid on all hosts... 2024-09-10T03:54:32.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid pull 2024-09-10T03:54:32.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid pull 2024-09-10T03:54:34.357 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2024-09-10T03:54:34.423 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2024-09-10T03:55:23.516 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-10T03:55:23.516 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-10T03:55:23.516 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-10T03:55:23.516 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2024-09-10T03:55:23.517 INFO:teuthology.orchestra.run.smithi070.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-10T03:55:23.517 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-10T03:55:23.517 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-10T03:55:23.871 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T03:55:23.872 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-10T03:55:23.872 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-10T03:55:23.872 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-09-10T03:55:23.872 INFO:teuthology.orchestra.run.smithi046.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-10T03:55:23.872 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-09-10T03:55:23.872 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T03:55:23.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-09-10T03:55:23.932 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2024-09-10T03:55:23.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-09-10T03:55:23.999 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2024-09-10T03:55:24.029 INFO:tasks.cephadm:Writing seed config... 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-10T03:55:24.031 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T03:55:24.032 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T03:55:24.033 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T03:55:24.033 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:55:24.033 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T03:55:24.056 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 628edc22-6f28-11ef-bcea-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T03:55:24.056 DEBUG:teuthology.orchestra.run.smithi046:mon.smithi046> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service 2024-09-10T03:55:24.099 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T03:55:24.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid -v bootstrap --fsid 628edc22-6f28-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T03:55:24.309 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:55:24.309 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:squid', '-v', 'bootstrap', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-09-10T03:55:24.310 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T03:55:24.310 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-09-10T03:55:24.334 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.2.2 2024-09-10T03:55:24.334 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-09-10T03:55:24.334 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-09-10T03:55:24.342 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T03:55:24.342 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T03:55:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T03:55:24.349 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T03:55:24.356 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-09-10T03:55:24.362 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-09-10T03:55:24.362 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-09-10T03:55:24.363 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-09-10T03:55:24.385 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.2.2 2024-09-10T03:55:24.385 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T03:55:24.385 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-09-10T03:55:24.385 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-09-10T03:55:24.392 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T03:55:24.392 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T03:55:24.398 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T03:55:24.398 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T03:55:24.405 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-09-10T03:55:24.412 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-09-10T03:55:24.412 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-09-10T03:55:24.412 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-09-10T03:55:24.412 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:24.413 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 139957544269664 on /run/cephadm/628edc22-6f28-11ef-bcea-c7b262605968.lock 2024-09-10T03:55:24.413 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 139957544269664 acquired on /run/cephadm/628edc22-6f28-11ef-bcea-c7b262605968.lock 2024-09-10T03:55:24.413 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-09-10T03:55:24.413 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-09-10T03:55:24.413 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-10T03:55:24.417 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.46 metric 100 2024-09-10T03:55:24.417 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-09-10T03:55:24.419 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T03:55:24.420 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T03:55:24.420 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-09-10T03:55:24.422 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T03:55:24.422 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T03:55:24.422 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T03:55:24.422 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T03:55:24.422 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-10T03:55:24.422 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T03:55:24.423 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-09-10T03:55:24.423 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-09-10T03:55:24.423 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T03:55:24.424 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T03:55:24.424 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:squid... 2024-09-10T03:55:24.921 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-10T03:55:24.921 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:squid... 2024-09-10T03:55:24.921 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T03:55:24.922 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-10T03:55:24.922 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T03:55:24.922 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-10T03:55:24.922 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T03:55:25.751 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-10T03:55:25.751 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-10T03:55:25.751 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T03:55:26.555 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-09-10T03:55:26.556 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-09-10T03:55:27.436 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQAuw99mp8npMhAAZMBeuPemGvRuO39tcjTTQA== 2024-09-10T03:55:28.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQAvw99mJ0hZLRAA5Avuqi0Oj4vGosnJ0ptNZQ== 2024-09-10T03:55:29.215 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQAww99mAHJNJhAA1UBl5CbvGWb3TWZZBq9AZg== 2024-09-10T03:55:29.216 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-09-10T03:55:30.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T03:55:30.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T03:55:30.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:30.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T03:55:30.139 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for smithi046 [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T03:55:30.139 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-09-10T03:55:30.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:30.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T03:55:30.139 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:55:30.139 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-09-10T03:55:31.052 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.smithi046 on 2024-09-10T03:55:31.272 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T03:55:31.435 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T03:55:31.614 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-628edc22-6f28-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-628edc22-6f28-11ef-bcea-c7b262605968.target. 2024-09-10T03:55:31.614 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-628edc22-6f28-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-628edc22-6f28-11ef-bcea-c7b262605968.target. 2024-09-10T03:55:31.885 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046 2024-09-10T03:55:31.885 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service: Unit ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service not loaded. 2024-09-10T03:55:32.056 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-628edc22-6f28-11ef-bcea-c7b262605968.target.wants/ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service → /etc/systemd/system/ceph-628edc22-6f28-11ef-bcea-c7b262605968@.service. 2024-09-10T03:55:32.593 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:55:32.593 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T03:55:32.600 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:55:32.601 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T03:55:32.601 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T03:55:32.601 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T03:55:32.601 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-09-10T03:55:32.601 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-09-10T03:55:32.679 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:32 smithi046 systemd[1]: Started Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:55:33.635 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: mon.smithi046 is new leader, mons smithi046 in quorum (ranks 0) 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: monmap epoch 1 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: last_changed 2024-09-10T03:55:29.528347+0000 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: min_mon_release 19 (squid) 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: election_strategy: 1 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: fsmap 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: mgrmap e1: no daemons active 2024-09-10T03:55:33.636 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:33 smithi046 ceph-mon[27958]: from='client.? 172.21.15.46:0/3744523901' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T03:55:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T03:55:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T03:55:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-09-10T03:55:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi046 (age 0.511934s) 2024-09-10T03:55:33.640 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-09-10T03:55:33.641 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T03:55:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:34 smithi046 ceph-mon[27958]: from='client.? 172.21.15.46:0/4011070393' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T03:55:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:34 smithi046 ceph-mon[27958]: from='client.? 172.21.15.46:0/4011070393' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T03:55:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T03:55:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-10T03:55:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T03:55:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T03:55:38.495 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T03:55:38.496 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-09-10T03:55:42.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:41 smithi046 ceph-mon[27958]: from='client.? 172.21.15.46:0/575256278' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:55:44.345 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-09-10T03:55:44.646 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:44 smithi046 systemd[1]: Stopping Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T03:55:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:44 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[27954]: 2024-09-10T03:55:44.644+0000 7f1b6d406640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi046 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T03:55:45.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:44 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[27954]: 2024-09-10T03:55:44.644+0000 7f1b6d406640 -1 mon.smithi046@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T03:55:45.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:44 smithi046 podman[28202]: 2024-09-10 03:55:44.891002868 +0000 UTC m=+0.363898614 container died ca687440ba7110a2132b7547d259a514c2b368d22d85041325b6e5fac15e82e7 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T03:55:45.403 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 podman[28202]: 2024-09-10 03:55:45.202330807 +0000 UTC m=+0.675226552 container cleanup ca687440ba7110a2132b7547d259a514c2b368d22d85041325b6e5fac15e82e7 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T03:55:45.404 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 bash[28202]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046 2024-09-10T03:55:45.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 podman[28215]: 2024-09-10 03:55:45.403319863 +0000 UTC m=+0.508656207 container remove ca687440ba7110a2132b7547d259a514c2b368d22d85041325b6e5fac15e82e7 (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T03:55:45.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service: Deactivated successfully. 2024-09-10T03:55:45.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 systemd[1]: Stopped Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:55:45.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 systemd[1]: Starting Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T03:55:45.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 podman[28309]: 2024-09-10 03:55:45.729016499 +0000 UTC m=+0.105474430 container create 9933ea41a29e06352837603dab77083048a972ce9995b598a832d6f5ee21015d (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 podman[28309]: 2024-09-10 03:55:45.639590453 +0000 UTC m=+0.016048357 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay.ceph.io/ceph-ci/ceph:squid 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 podman[28309]: 2024-09-10 03:55:45.922996988 +0000 UTC m=+0.299454881 container init 9933ea41a29e06352837603dab77083048a972ce9995b598a832d6f5ee21015d (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 podman[28309]: 2024-09-10 03:55:45.926161159 +0000 UTC m=+0.302619053 container start 9933ea41a29e06352837603dab77083048a972ce9995b598a832d6f5ee21015d (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: pidfile_write: ignore empty --pid-file 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: load: jerasure load: lrc 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: RocksDB version: 7.9.2 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Git sha 0 2024-09-10T03:55:45.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: DB SUMMARY 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: DB Session ID: O4XZ9MADHS38FN20IG9V 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: CURRENT file: CURRENT 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi046/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi046/store.db: 000009.log size: 95831 ; 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.error_if_exists: 0 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.create_if_missing: 0 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.paranoid_checks: 1 2024-09-10T03:55:46.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.env: 0x561d3d684940 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.info_log: 0x561d3e1718c0 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.statistics: (nil) 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.use_fsync: 0 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_log_file_size: 0 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T03:55:46.001 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.allow_fallocate: 1 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.use_direct_reads: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.db_log_dir: 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.wal_dir: 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T03:55:46.002 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.write_buffer_manager: 0x561d3e180dc0 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T03:55:46.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.unordered_write: 0 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.row_cache: None 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.wal_filter: None 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.two_write_queues: 0 2024-09-10T03:55:46.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.wal_compression: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.atomic_flush: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.log_readahead_size: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T03:55:46.006 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_background_jobs: 2 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_background_compactions: -1 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_subcompactions: 1 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T03:55:46.007 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_open_files: -1 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_background_flushes: -1 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Compression algorithms supported: 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kZSTD supported: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kXpressCompression supported: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kBZip2Compression supported: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kLZ4Compression supported: 1 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kZlibCompression supported: 1 2024-09-10T03:55:46.008 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: kSnappyCompression supported: 1 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi046/store.db/MANIFEST-000010 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.merge_operator: 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_filter: None 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T03:55:46.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561d3e171520) 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: index_type: 0 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: data_block_index_type: 0 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: index_shortening: 1 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: checksum: 4 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: no_block_cache: 0 2024-09-10T03:55:46.010 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache: 0x561d3e1651f0 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache_options: 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: capacity : 536870912 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: num_shard_bits : 4 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: strict_capacity_limit : 0 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache_compressed: (nil) 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: persistent_cache: (nil) 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_size: 4096 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_size_deviation: 10 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_restart_interval: 16 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: index_block_restart_interval: 1 2024-09-10T03:55:46.011 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: metadata_block_size: 4096 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: partition_filters: 0 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: use_delta_encoding: 1 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: filter_policy: bloomfilter 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: whole_key_filtering: 1 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: verify_compression: 0 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: format_version: 5 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: enable_index_compression: 1 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_align: 0 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: max_auto_readahead_size: 262144 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: prepopulate_block_cache: 0 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: initial_auto_readahead_size: 8192 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T03:55:46.012 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression: NoCompression 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.num_levels: 7 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T03:55:46.013 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T03:55:46.014 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T03:55:46.015 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T03:55:46.016 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.inplace_update_support: 0 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T03:55:46.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.bloom_locality: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.max_successive_merges: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.ttl: 2592000 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.enable_blob_files: false 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.min_blob_size: 0 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T03:55:46.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi046/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 248d7314-bca8-471d-9f91-66e3162d0852 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940545952727, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940545953701, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 288, "table_properties": {"data_size": 89873, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11912, "raw_average_key_size": 48, "raw_value_size": 83241, "raw_average_value_size": 337, "num_data_blocks": 12, "num_entries": 247, "num_filter_entries": 247, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725940545, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "248d7314-bca8-471d-9f91-66e3162d0852", "db_session_id": "O4XZ9MADHS38FN20IG9V", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T03:55:46.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940545953798, "job": 1, "event": "recovery_finished"} 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi046/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561d3e192e00 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: DB pointer 0x561d3e28e000 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** DB Stats ** 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:55:46.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: L0 2/0 91.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Sum 2/0 91.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T03:55:46.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative compaction: 0.00 GB write, 27.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval compaction: 0.00 GB write, 27.28 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Block cache BinnedLRUCache@0x561d3e1651f0#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T03:55:46.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: starting mon.smithi046 rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi046 fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???) e1 preinit fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???).mds e1 new map 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???).mds e1 print_map 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: e1 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: btime 2024-09-10T03:55:32:552214+0000 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: legacy client fscid: -1 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T03:55:46.023 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: No filesystems configured 2024-09-10T03:55:46.024 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T03:55:46.024 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:55:46.024 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:55:46.024 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:55:46.024 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 ceph-mon[28328]: mon.smithi046@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T03:55:46.024 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:45 smithi046 bash[28309]: 9933ea41a29e06352837603dab77083048a972ce9995b598a832d6f5ee21015d 2024-09-10T03:55:46.024 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 systemd[1]: Started Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:55:46.024 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T03:55:47.041 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T03:55:47.062 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T03:55:47.062 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-09-10T03:55:47.063 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T03:55:47.063 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T03:55:47.063 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-10T03:55:47.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: mon.smithi046 is new leader, mons smithi046 in quorum (ranks 0) 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: monmap epoch 1 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: last_changed 2024-09-10T03:55:29.528347+0000 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: min_mon_release 19 (squid) 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: election_strategy: 1 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: fsmap 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: mgrmap e1: no daemons active 2024-09-10T03:55:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:46 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2406002697' entity='client.admin' 2024-09-10T03:55:47.341 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-628edc22-6f28-11ef-bcea-c7b262605968@mgr.smithi046.zvckjg 2024-09-10T03:55:47.341 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-628edc22-6f28-11ef-bcea-c7b262605968@mgr.smithi046.zvckjg.service: Unit ceph-628edc22-6f28-11ef-bcea-c7b262605968@mgr.smithi046.zvckjg.service not loaded. 2024-09-10T03:55:47.489 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-628edc22-6f28-11ef-bcea-c7b262605968.target.wants/ceph-628edc22-6f28-11ef-bcea-c7b262605968@mgr.smithi046.zvckjg.service → /etc/systemd/system/ceph-628edc22-6f28-11ef-bcea-c7b262605968@.service. 2024-09-10T03:55:48.001 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:55:48.001 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T03:55:48.009 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:55:48.009 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T03:55:48.009 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T03:55:48.009 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T03:55:48.019 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:55:48.019 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T03:55:48.027 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:55:48.027 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T03:55:48.027 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T03:55:48.027 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-10T03:55:48.027 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-09-10T03:55:48.027 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-09-10T03:55:48.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:48 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2730996298' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T03:55:49.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:49.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:55:49.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "628edc22-6f28-11ef-bcea-c7b262605968", 2024-09-10T03:55:49.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T03:55:49.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T03:55:49.531 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "smithi046" 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:49.532 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T03:55:49.533 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T03:55:49.534 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T03:55:49.537 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T03:55:49.537 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T03:55:49.537 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:49.537 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T03:55:49.537 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T03:55:49.537 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T03:55:32:552214+0000", 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T03:55:49.538 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:49.539 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T03:55:32.552866+0000", 2024-09-10T03:55:49.540 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:55:49.540 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:49.540 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T03:55:49.540 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:55:49.540 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-09-10T03:55:52.173 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:52 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/517343078' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T03:55:52.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:52.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "628edc22-6f28-11ef-bcea-c7b262605968", 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T03:55:52.626 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "smithi046" 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T03:55:52.627 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T03:55:52.628 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T03:55:32:552214+0000", 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T03:55:52.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T03:55:32.552866+0000", 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:55:52.630 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-09-10T03:55:53.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: Activating manager daemon smithi046.zvckjg 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: mgrmap e2: smithi046.zvckjg(active, starting, since 0.00312783s) 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: Manager daemon smithi046.zvckjg is now available 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' 2024-09-10T03:55:53.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:53 smithi046 ceph-mon[28328]: from='mgr.14100 172.21.15.46:0/2554647850' entity='mgr.smithi046.zvckjg' 2024-09-10T03:55:55.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:55 smithi046 ceph-mon[28328]: mgrmap e3: smithi046.zvckjg(active, since 1.00704s) 2024-09-10T03:55:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:56 smithi046 ceph-mon[28328]: mgrmap e4: smithi046.zvckjg(active, since 2s) 2024-09-10T03:55:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:56 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1495495315' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T03:55:56.260 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:56.262 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "628edc22-6f28-11ef-bcea-c7b262605968", 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:56.263 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "smithi046" 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T03:55:56.264 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T03:55:56.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T03:55:56.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T03:55:56.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T03:55:56.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:56.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T03:55:32:552214+0000", 2024-09-10T03:55:56.269 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T03:55:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T03:55:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T03:55:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T03:55:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T03:55:56.270 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T03:55:56.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T03:55:56.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T03:55:56.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T03:55:56.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:55:56.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:55:56.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:56.271 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T03:55:56.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:55:56.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T03:55:32.552866+0000", 2024-09-10T03:55:56.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:55:56.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:55:56.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T03:55:56.272 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:55:56.273 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-09-10T03:55:57.179 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:57 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3774955601' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T03:55:57.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:57.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T03:55:57.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:57.442 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T03:55:57.443 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T03:55:57.444 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T03:55:57.444 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-09-10T03:55:58.461 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:58 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/4110021525' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T03:55:59.290 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:59 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/4110021525' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T03:55:59.291 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:55:59 smithi046 ceph-mon[28328]: mgrmap e5: smithi046.zvckjg(active, since 5s) 2024-09-10T03:56:00.121 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:56:00.121 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T03:56:00.121 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T03:56:00.121 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "smithi046.zvckjg", 2024-09-10T03:56:00.121 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T03:56:00.121 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:56:00.121 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-09-10T03:56:00.122 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-09-10T03:56:00.400 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:00 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1629130842' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T03:56:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: Active manager daemon smithi046.zvckjg restarted 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: Activating manager daemon smithi046.zvckjg 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: mgrmap e6: smithi046.zvckjg(active, starting, since 0.00296685s) 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T03:56:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: Manager daemon smithi046.zvckjg is now available 2024-09-10T03:56:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:56:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:56:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T03:56:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:03 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T03:56:04.927 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:56:04.927 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T03:56:04.927 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T03:56:04.927 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:56:04.927 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-09-10T03:56:04.927 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-09-10T03:56:05.174 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:05 smithi046 ceph-mon[28328]: Found migration_current of "None". Setting to last migration. 2024-09-10T03:56:05.175 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:05 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:05.175 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:05 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:05.175 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:05 smithi046 ceph-mon[28328]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T03:56:05.175 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:05 smithi046 ceph-mon[28328]: mgrmap e7: smithi046.zvckjg(active, since 1.00533s) 2024-09-10T03:56:05.175 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:05 smithi046 ceph-mon[28328]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T03:56:06.967 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:06 smithi046 ceph-mon[28328]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:06.967 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:06 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:06.967 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:06 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:56:06.967 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:06 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:06] ENGINE Bus STARTING 2024-09-10T03:56:06.967 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:06 smithi046 ceph-mon[28328]: mgrmap e8: smithi046.zvckjg(active, since 2s) 2024-09-10T03:56:06.967 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:06 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:56:07.611 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T03:56:07.611 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-09-10T03:56:07.927 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:07 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:06] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T03:56:07.927 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:07 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:06] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:07 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:06] ENGINE Bus STARTED 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:07 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:06] ENGINE Error in HTTPServer.serve 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Traceback (most recent call last): 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._run(expiration_interval) 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:56:07.928 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s = self.context.wrap_socket( 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: return self.sslsocket_class._create( 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self.do_handshake() 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._sslobj.do_handshake() 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:56:07.929 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:07 smithi046 ceph-mon[28328]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:09.921 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:09 smithi046 ceph-mon[28328]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:09.921 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:09 smithi046 ceph-mon[28328]: Generating ssh key... 2024-09-10T03:56:09.921 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:09 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:09.921 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:09 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:10.501 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaPXWUcktW4xqYa38pivS/jlMK4lqdc/XHYF4PkCvDydyDWveXRsCou9+7l+2TcKJH3vOMF+4ESn6L0zThb668FSGr5Q0JvaZivhzgXs/UXLJk6Yxb3KMJucUXAEGlVe8STFGQ1aO1AAHq9E81xh39E1YukGI4ybE0fvQP7zXtN0/IsBG65Plp/KCP88dhmEAPUHta07W9CPipFVhNOBFbUvbf7u9zQjypP5l1txYIsC/f/6JqOdLBJkF2j/ZaUapLcTKSI4GeVYpjrAJMHztTyB3b6plBG61NEecRMnKUDKQpjo07PuC69mi27jHhWythGry9MgnNGP9vE9sjOJ557UtZP+WpYyBFEVFUalHRg24diAm7GmOkzNbgvi1584qg2HgHRUyq04ZnYATseH/Jo8pAnVvkrLOaafYgo0Iv7xQ0UPIC11+H/nYhOFIM0j1Ca48Yyc5U3wvBkWMSD6cS5VO9jbdXaUhVFAHmgscA2Fd1uzkd0MZVK62TnJLtBjs= ceph-628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:10.502 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T03:56:10.502 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T03:56:10.505 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-09-10T03:56:10.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:10 smithi046 ceph-mon[28328]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:11.882 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:11 smithi046 ceph-mon[28328]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:13.411 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:13 smithi046 ceph-mon[28328]: Deploying cephadm binary to smithi046 2024-09-10T03:56:15.670 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-09-10T03:56:15.670 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying mon service with default placement... 2024-09-10T03:56:16.260 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:16 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:16.261 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:16 smithi046 ceph-mon[28328]: Added host smithi046 2024-09-10T03:56:16.261 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:16 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:56:17.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T03:56:17.035 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying mgr service with default placement... 2024-09-10T03:56:17.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:17 smithi046 ceph-mon[28328]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:17.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:17 smithi046 ceph-mon[28328]: Saving service mon spec with placement count:5 2024-09-10T03:56:17.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:17 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:18.236 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T03:56:18.236 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying crash service with default placement... 2024-09-10T03:56:18.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:18 smithi046 ceph-mon[28328]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:18.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:18 smithi046 ceph-mon[28328]: Saving service mgr spec with placement count:2 2024-09-10T03:56:18.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:18 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:19.768 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-10T03:56:19.768 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying ceph-exporter service with default placement... 2024-09-10T03:56:19.893 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:19 smithi046 ceph-mon[28328]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:19.894 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:19 smithi046 ceph-mon[28328]: Saving service crash spec with placement * 2024-09-10T03:56:19.894 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:19 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:21.218 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-09-10T03:56:21.218 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying prometheus service with default placement... 2024-09-10T03:56:21.466 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:21 smithi046 ceph-mon[28328]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:21.466 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:21 smithi046 ceph-mon[28328]: Saving service ceph-exporter spec with placement * 2024-09-10T03:56:21.466 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:21 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:21.466 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:21 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:22.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-10T03:56:22.448 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying grafana service with default placement... 2024-09-10T03:56:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:22 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:22 smithi046 ceph-mon[28328]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:22 smithi046 ceph-mon[28328]: Saving service prometheus spec with placement count:1 2024-09-10T03:56:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:22 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:23.974 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-10T03:56:23.974 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying node-exporter service with default placement... 2024-09-10T03:56:24.307 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:24 smithi046 ceph-mon[28328]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:24.307 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:24 smithi046 ceph-mon[28328]: Saving service grafana spec with placement count:1 2024-09-10T03:56:24.307 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:24 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:24.307 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:24 smithi046 ceph-mon[28328]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:25.331 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-10T03:56:25.331 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying alertmanager service with default placement... 2024-09-10T03:56:25.987 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:25 smithi046 ceph-mon[28328]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:25.987 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:25 smithi046 ceph-mon[28328]: Saving service node-exporter spec with placement * 2024-09-10T03:56:25.987 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:25 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:25.987 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:25 smithi046 ceph-mon[28328]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:26.595 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-10T03:56:27.267 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:26 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:27.268 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:26 smithi046 ceph-mon[28328]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:27.268 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:26 smithi046 ceph-mon[28328]: Saving service alertmanager spec with placement count:1 2024-09-10T03:56:27.268 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:26 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:28.202 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:27 smithi046 ceph-mon[28328]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:28.202 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:27 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/732163345' entity='client.admin' 2024-09-10T03:56:28.202 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:27 smithi046 ceph-mon[28328]: from='mgr.14118 172.21.15.46:0/1586574030' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:29.675 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:29 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2121332275' entity='client.admin' 2024-09-10T03:56:29.676 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:29 smithi046 ceph-mon[28328]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:29.678 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-09-10T03:56:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:30 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/600701384' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T03:56:32.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:31 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/600701384' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T03:56:32.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:31 smithi046 ceph-mon[28328]: mgrmap e9: smithi046.zvckjg(active, since 27s) 2024-09-10T03:56:32.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:56:32.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T03:56:32.760 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T03:56:32.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "smithi046.zvckjg", 2024-09-10T03:56:32.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T03:56:32.761 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:56:32.761 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-09-10T03:56:32.761 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-09-10T03:56:33.019 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:32 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/479726654' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T03:56:36.146 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: Active manager daemon smithi046.zvckjg restarted 2024-09-10T03:56:36.146 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: Activating manager daemon smithi046.zvckjg 2024-09-10T03:56:36.146 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T03:56:36.146 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: mgrmap e10: smithi046.zvckjg(active, starting, since 0.00314974s) 2024-09-10T03:56:36.146 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T03:56:36.146 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T03:56:36.146 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T03:56:36.147 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T03:56:36.147 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T03:56:36.147 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: Manager daemon smithi046.zvckjg is now available 2024-09-10T03:56:36.147 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:56:36.147 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T03:56:36.147 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:35 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T03:56:37.828 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:56:37.828 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T03:56:37.828 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T03:56:37.828 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:56:37.828 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-09-10T03:56:37.828 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:37 smithi046 ceph-mon[28328]: mgrmap e11: smithi046.zvckjg(active, since 1.00551s) 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:37 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:37 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:37] ENGINE Bus STARTING 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:37 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:37] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:37 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:37] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:37 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:37] ENGINE Bus STARTED 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:37 smithi046 ceph-mon[28328]: [10/Sep/2024:03:56:37] ENGINE Error in HTTPServer.serve 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Traceback (most recent call last): 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-10T03:56:38.082 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._run(expiration_interval) 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s = self.context.wrap_socket( 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: return self.sslsocket_class._create( 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self.do_handshake() 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._sslobj.do_handshake() 2024-09-10T03:56:38.083 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:56:39.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T03:56:39.243 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-09-10T03:56:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:39 smithi046 ceph-mon[28328]: mgrmap e12: smithi046.zvckjg(active, since 2s) 2024-09-10T03:56:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:39 smithi046 ceph-mon[28328]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:39.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:39 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:39.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:39 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:39.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:39 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:40.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$LX/6VM9ovki5cegv/gcC1ubzvJKoo87VgkCJGytDXQTdE8MhdRqVK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725940600, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T03:56:40.920 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-09-10T03:56:41.211 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:41 smithi046 ceph-mon[28328]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:41.211 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:41 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:42.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:42 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/797363516' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T03:56:42.636 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T03:56:42.647 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:56:42.647 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-09-10T03:56:42.656 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:56:42.656 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-09-10T03:56:42.656 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-09-10T03:56:42.656 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T03:56:42.658 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-09-10T03:56:42.658 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:42.658 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-09-10T03:56:42.658 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-09-10T03:56:42.658 INFO:teuthology.orchestra.run.smithi046.stdout: Password: nfz940l1vr 2024-09-10T03:56:42.658 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:42.659 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config directory 2024-09-10T03:56:43.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:43 smithi046 ceph-mon[28328]: mgrmap e13: smithi046.zvckjg(active, since 6s) 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.044 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:44.045 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-09-10T03:56:44.093 INFO:tasks.cephadm:Fetching config... 2024-09-10T03:56:44.093 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:56:44.093 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T03:56:44.111 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T03:56:44.112 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:56:44.112 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T03:56:44.169 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T03:56:44.169 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:56:44.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/keyring of=/dev/stdout 2024-09-10T03:56:44.236 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T03:56:44.236 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:56:44.237 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T03:56:44.293 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T03:56:44.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaPXWUcktW4xqYa38pivS/jlMK4lqdc/XHYF4PkCvDydyDWveXRsCou9+7l+2TcKJH3vOMF+4ESn6L0zThb668FSGr5Q0JvaZivhzgXs/UXLJk6Yxb3KMJucUXAEGlVe8STFGQ1aO1AAHq9E81xh39E1YukGI4ybE0fvQP7zXtN0/IsBG65Plp/KCP88dhmEAPUHta07W9CPipFVhNOBFbUvbf7u9zQjypP5l1txYIsC/f/6JqOdLBJkF2j/ZaUapLcTKSI4GeVYpjrAJMHztTyB3b6plBG61NEecRMnKUDKQpjo07PuC69mi27jHhWythGry9MgnNGP9vE9sjOJ557UtZP+WpYyBFEVFUalHRg24diAm7GmOkzNbgvi1584qg2HgHRUyq04ZnYATseH/Jo8pAnVvkrLOaafYgo0Iv7xQ0UPIC11+H/nYhOFIM0j1Ca48Yyc5U3wvBkWMSD6cS5VO9jbdXaUhVFAHmgscA2Fd1uzkd0MZVK62TnJLtBjs= ceph-628edc22-6f28-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T03:56:44.369 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaPXWUcktW4xqYa38pivS/jlMK4lqdc/XHYF4PkCvDydyDWveXRsCou9+7l+2TcKJH3vOMF+4ESn6L0zThb668FSGr5Q0JvaZivhzgXs/UXLJk6Yxb3KMJucUXAEGlVe8STFGQ1aO1AAHq9E81xh39E1YukGI4ybE0fvQP7zXtN0/IsBG65Plp/KCP88dhmEAPUHta07W9CPipFVhNOBFbUvbf7u9zQjypP5l1txYIsC/f/6JqOdLBJkF2j/ZaUapLcTKSI4GeVYpjrAJMHztTyB3b6plBG61NEecRMnKUDKQpjo07PuC69mi27jHhWythGry9MgnNGP9vE9sjOJ557UtZP+WpYyBFEVFUalHRg24diAm7GmOkzNbgvi1584qg2HgHRUyq04ZnYATseH/Jo8pAnVvkrLOaafYgo0Iv7xQ0UPIC11+H/nYhOFIM0j1Ca48Yyc5U3wvBkWMSD6cS5VO9jbdXaUhVFAHmgscA2Fd1uzkd0MZVK62TnJLtBjs= ceph-628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:44.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaPXWUcktW4xqYa38pivS/jlMK4lqdc/XHYF4PkCvDydyDWveXRsCou9+7l+2TcKJH3vOMF+4ESn6L0zThb668FSGr5Q0JvaZivhzgXs/UXLJk6Yxb3KMJucUXAEGlVe8STFGQ1aO1AAHq9E81xh39E1YukGI4ybE0fvQP7zXtN0/IsBG65Plp/KCP88dhmEAPUHta07W9CPipFVhNOBFbUvbf7u9zQjypP5l1txYIsC/f/6JqOdLBJkF2j/ZaUapLcTKSI4GeVYpjrAJMHztTyB3b6plBG61NEecRMnKUDKQpjo07PuC69mi27jHhWythGry9MgnNGP9vE9sjOJ557UtZP+WpYyBFEVFUalHRg24diAm7GmOkzNbgvi1584qg2HgHRUyq04ZnYATseH/Jo8pAnVvkrLOaafYgo0Iv7xQ0UPIC11+H/nYhOFIM0j1Ca48Yyc5U3wvBkWMSD6cS5VO9jbdXaUhVFAHmgscA2Fd1uzkd0MZVK62TnJLtBjs= ceph-628edc22-6f28-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T03:56:44.428 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaPXWUcktW4xqYa38pivS/jlMK4lqdc/XHYF4PkCvDydyDWveXRsCou9+7l+2TcKJH3vOMF+4ESn6L0zThb668FSGr5Q0JvaZivhzgXs/UXLJk6Yxb3KMJucUXAEGlVe8STFGQ1aO1AAHq9E81xh39E1YukGI4ybE0fvQP7zXtN0/IsBG65Plp/KCP88dhmEAPUHta07W9CPipFVhNOBFbUvbf7u9zQjypP5l1txYIsC/f/6JqOdLBJkF2j/ZaUapLcTKSI4GeVYpjrAJMHztTyB3b6plBG61NEecRMnKUDKQpjo07PuC69mi27jHhWythGry9MgnNGP9vE9sjOJ557UtZP+WpYyBFEVFUalHRg24diAm7GmOkzNbgvi1584qg2HgHRUyq04ZnYATseH/Jo8pAnVvkrLOaafYgo0Iv7xQ0UPIC11+H/nYhOFIM0j1Ca48Yyc5U3wvBkWMSD6cS5VO9jbdXaUhVFAHmgscA2Fd1uzkd0MZVK62TnJLtBjs= ceph-628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:44.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T03:56:44.625 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:44 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/4099235809' entity='client.admin' 2024-09-10T03:56:44.693 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:56:46.028 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T03:56:46.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T03:56:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:46 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1527934911' entity='client.admin' 2024-09-10T03:56:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:56:47.525 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:47.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:47.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:56:47.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:47.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T03:56:47.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T03:56:47.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:56:47.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:47 smithi046 ceph-mon[28328]: Deploying daemon ceph-exporter.smithi046 on smithi046 2024-09-10T03:56:48.882 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:48 smithi046 ceph-mon[28328]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:48.882 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:48 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:48.912 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi070 2024-09-10T03:56:48.913 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:56:48.913 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/etc/ceph/ceph.conf 2024-09-10T03:56:48.932 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:56:48.933 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:56:48.991 INFO:tasks.cephadm:Adding host smithi070 to orchestrator... 2024-09-10T03:56:48.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch host add smithi070 2024-09-10T03:56:50.094 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:56:52.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi046", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi046", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:56:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:52 smithi046 ceph-mon[28328]: Deploying daemon crash.smithi046 on smithi046 2024-09-10T03:56:53.528 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:53 smithi046 ceph-mon[28328]: Deploying cephadm binary to smithi070 2024-09-10T03:56:54.926 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi070' with addr '172.21.15.70' 2024-09-10T03:56:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:55 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:55 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:55 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:55 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:55 smithi046 ceph-mon[28328]: Deploying daemon node-exporter.smithi046 on smithi046 2024-09-10T03:56:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:55 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:55.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T03:56:55.790 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:56:56.164 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:56 smithi046 ceph-mon[28328]: Added host smithi070 2024-09-10T03:56:56.165 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:56 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:56:56.646 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:56:56.646 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}] 2024-09-10T03:56:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:57 smithi046 ceph-mon[28328]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:57 smithi046 ceph-mon[28328]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T03:56:57.872 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T03:56:57.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T03:56:58.575 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:56:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-09-10T03:57:00.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:59 smithi046 ceph-mon[28328]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:00.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:59 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:00.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:59 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:00.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:59 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:00.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:59 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:00.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:59 smithi046 ceph-mon[28328]: Deploying daemon alertmanager.smithi046 on smithi046 2024-09-10T03:57:00.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:56:59 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1283215097' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T03:57:00.668 INFO:tasks.cephadm:Adding mon.smithi046 on smithi046 2024-09-10T03:57:00.668 INFO:tasks.cephadm:Adding mon.smithi070 on smithi070 2024-09-10T03:57:00.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch apply mon '2;smithi046:172.21.15.46=smithi046;smithi070:172.21.15.70=smithi070' 2024-09-10T03:57:00.912 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:00.961 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:00 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1283215097' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T03:57:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:00 smithi046 ceph-mon[28328]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:57:01.573 INFO:teuthology.orchestra.run.smithi070.stdout:Scheduled mon update... 2024-09-10T03:57:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:01 smithi046 ceph-mon[28328]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:01 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:02.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:01 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:02.138 DEBUG:teuthology.orchestra.run.smithi070:mon.smithi070> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi070.service 2024-09-10T03:57:02.141 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:02.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:02.421 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:02.465 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:02 smithi046 ceph-mon[28328]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi046:172.21.15.46=smithi046;smithi070:172.21.15.70=smithi070", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:02 smithi046 ceph-mon[28328]: Saving service mon spec with placement smithi046:172.21.15.46=smithi046;smithi070:172.21.15.70=smithi070;count:2 2024-09-10T03:57:03.128 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:03.129 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:03.129 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:03.779 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:03 smithi046 ceph-mon[28328]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:03.780 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:03 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/258868477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:04.688 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:04.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:04.932 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:04.976 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:05.624 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:05.624 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:05.624 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:06.028 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:05 smithi046 ceph-mon[28328]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:06.028 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:05 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3867148256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:07.364 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:07.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:07.604 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:07.649 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T03:57:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:07 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:08.260 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:08.260 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:08.260 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:08 smithi046 ceph-mon[28328]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-10T03:57:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:08 smithi046 ceph-mon[28328]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T03:57:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:08 smithi046 ceph-mon[28328]: Deploying daemon grafana.smithi046 on smithi046 2024-09-10T03:57:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:08 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/1237715639' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:09.799 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:09.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:10.041 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:10.085 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:10.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:09 smithi046 ceph-mon[28328]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:10.670 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:10.670 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:10.671 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:11 smithi046 ceph-mon[28328]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:11 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3553364605' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:11 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:12.203 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:12.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:12.439 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:12.552 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:13.159 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:13.159 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:13.160 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:14.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:13 smithi046 ceph-mon[28328]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:14.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:13 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3688303505' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:14.801 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:14.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:15.042 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:15.086 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:15.697 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:15.697 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:15.697 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:15 smithi046 ceph-mon[28328]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:16 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/1227687511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:17.266 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:17.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:17.509 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:17.553 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:17.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:17.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:17.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:17.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:17.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:17.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:17.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:17.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:17.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:17 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:18.238 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:18.238 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:18.238 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:18.941 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:18 smithi046 ceph-mon[28328]: Deploying daemon prometheus.smithi046 on smithi046 2024-09-10T03:57:18.941 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:18 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/4153899456' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:19.762 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:19.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:19 smithi046 ceph-mon[28328]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:20.008 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:20.051 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:20.813 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:20.814 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:20.814 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:21 smithi046 ceph-mon[28328]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:21 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3429162156' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:21 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:22.369 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:22.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:22.605 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:22.651 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:23.524 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:23.524 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:23 smithi046 ceph-mon[28328]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:23 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3499913905' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:25.134 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:25.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:25.788 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:25 smithi046 ceph-mon[28328]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:26.091 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:26.091 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:26.091 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:26.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:26 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/2908165931' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:27.654 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:27.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:27 smithi046 ceph-mon[28328]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:27.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:27 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:27.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:27 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:27.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:27 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:27.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:27 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T03:57:27.898 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:27.943 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:28.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:28.622 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:28.622 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:29.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:28 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/2413921250' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:29.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:28 smithi046 ceph-mon[28328]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T03:57:29.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:28 smithi046 ceph-mon[28328]: mgrmap e14: smithi046.zvckjg(active, since 52s) 2024-09-10T03:57:30.212 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:30.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:30.451 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:30.495 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:31.127 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:31.128 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:31.128 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:31.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:31 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/1650062354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:32.682 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:32.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:32.927 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:32.971 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:33.883 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:33.883 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:33.883 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:33.979 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: Active manager daemon smithi046.zvckjg restarted 2024-09-10T03:57:33.979 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: Activating manager daemon smithi046.zvckjg 2024-09-10T03:57:33.979 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T03:57:33.979 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: mgrmap e15: smithi046.zvckjg(active, starting, since 0.00323905s) 2024-09-10T03:57:33.979 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T03:57:33.979 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T03:57:33.980 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T03:57:33.980 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T03:57:33.980 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T03:57:33.980 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: Manager daemon smithi046.zvckjg is now available 2024-09-10T03:57:33.980 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:57:33.980 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:33.980 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:57:35.048 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T03:57:35.048 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T03:57:35.049 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:34 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/4071009095' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:35.049 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:35.049 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:34 smithi046 ceph-mon[28328]: mgrmap e16: smithi046.zvckjg(active, since 1.0053s) 2024-09-10T03:57:35.499 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:35.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:35.741 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:35.790 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:36.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:35 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:36.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:35 smithi046 ceph-mon[28328]: [10/Sep/2024:03:57:35] ENGINE Bus STARTING 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:35 smithi046 ceph-mon[28328]: [10/Sep/2024:03:57:35] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:35 smithi046 ceph-mon[28328]: [10/Sep/2024:03:57:35] ENGINE Error in HTTPServer.serve 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Traceback (most recent call last): 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._run(expiration_interval) 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s = self.context.wrap_socket( 2024-09-10T03:57:36.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: return self.sslsocket_class._create( 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self.do_handshake() 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._sslobj.do_handshake() 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:35 smithi046 ceph-mon[28328]: [10/Sep/2024:03:57:35] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:35 smithi046 ceph-mon[28328]: [10/Sep/2024:03:57:35] ENGINE Bus STARTED 2024-09-10T03:57:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:35 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:36.631 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:36.631 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:37 smithi046 ceph-mon[28328]: mgrmap e17: smithi046.zvckjg(active, since 2s) 2024-09-10T03:57:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:37 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/1085949139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:38.516 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:38.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:38.776 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:38.832 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T03:57:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:39.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:39.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:39.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:39.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:57:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:39.890 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:39.890 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/520411411' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T03:57:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:41.634 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:41.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:42.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:41 smithi046 ceph-mon[28328]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-09-10T03:57:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:42.958 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:42.958 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:42.959 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:43 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/4222461346' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:44.548 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:44.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:45.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T03:57:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-10T03:57:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:45 smithi046 ceph-mon[28328]: Deploying daemon crash.smithi070 on smithi070 2024-09-10T03:57:45.454 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:46.290 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:46.290 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:46 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/4240549415' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:48.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:48.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:48.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:48.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:47 smithi046 ceph-mon[28328]: Deploying daemon node-exporter.smithi070 on smithi070 2024-09-10T03:57:48.210 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:48.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:48.455 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:49.306 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:49.307 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:49.307 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:50.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:50.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:57:50.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:49 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3703896477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:51.735 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:51.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:52.488 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:52.488 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:52.488 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:52.488 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:52.488 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T03:57:52.488 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T03:57:52.489 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T03:57:52.489 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:52.489 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:52 smithi046 ceph-mon[28328]: Deploying daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T03:57:52.589 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:53.258 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:53.258 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:53 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/4173183735' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:54.873 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:54.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:54 smithi046 ceph-mon[28328]: Deploying daemon mon.smithi070 on smithi070 2024-09-10T03:57:55.536 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:56 smithi046 ceph-mon[28328]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:57.267 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:57:57.268 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":1,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:55:29.528347Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:57:57.268 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 1 2024-09-10T03:57:58.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:57:57 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/1202227236' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.377 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 podman[35476]: 2024-09-10 03:57:58.355171002 +0000 UTC m=+0.359454965 container start b5145a46761afe21e03716937f5c9e51d4f7409c8059b6b850f34cd7f853fde5 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T03:57:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pidfile_write: ignore empty --pid-file 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: load: jerasure load: lrc 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: RocksDB version: 7.9.2 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Git sha 0 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: DB SUMMARY 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: DB Session ID: KU7CAG0C1GR6JGFHR6VF 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: CURRENT file: CURRENT 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T03:57:58.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi070/store.db dir, Total Num: 0, files: 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi070/store.db: 000004.log size: 511 ; 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.error_if_exists: 0 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.create_if_missing: 0 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.paranoid_checks: 1 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.env: 0x560656ac1940 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T03:57:58.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.info_log: 0x5606580ad8c0 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.statistics: (nil) 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.use_fsync: 0 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_log_file_size: 0 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.allow_fallocate: 1 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T03:57:58.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.use_direct_reads: 0 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.db_log_dir: 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.wal_dir: 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T03:57:58.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.write_buffer_manager: 0x5606580bcdc0 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.unordered_write: 0 2024-09-10T03:57:58.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.row_cache: None 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.wal_filter: None 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.two_write_queues: 0 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.wal_compression: 0 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.atomic_flush: 0 2024-09-10T03:57:58.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.log_readahead_size: 0 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T03:57:58.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_background_jobs: 2 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_background_compactions: -1 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_subcompactions: 1 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_open_files: -1 2024-09-10T03:57:58.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_background_flushes: -1 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Compression algorithms supported: 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kZSTD supported: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kXpressCompression supported: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kBZip2Compression supported: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kLZ4Compression supported: 1 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kZlibCompression supported: 1 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T03:57:58.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: kSnappyCompression supported: 1 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000005 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.merge_operator: 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_filter: None 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5606580ad520) 2024-09-10T03:57:58.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_type: 0 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_index_type: 0 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_shortening: 1 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: checksum: 4 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: no_block_cache: 0 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache: 0x5606580a11f0 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-09-10T03:57:58.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_options: 2024-09-10T03:57:58.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: capacity : 536870912 2024-09-10T03:57:58.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_shard_bits : 4 2024-09-10T03:57:58.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: strict_capacity_limit : 0 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_compressed: (nil) 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: persistent_cache: (nil) 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size: 4096 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size_deviation: 10 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_restart_interval: 16 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_block_restart_interval: 1 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_block_size: 4096 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: partition_filters: 0 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: use_delta_encoding: 1 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: filter_policy: bloomfilter 2024-09-10T03:57:58.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: whole_key_filtering: 1 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: verify_compression: 0 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: format_version: 5 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_index_compression: 1 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_align: 0 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_auto_readahead_size: 262144 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: prepopulate_block_cache: 0 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: initial_auto_readahead_size: 8192 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression: NoCompression 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T03:57:58.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.num_levels: 7 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:57:58.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:57:58.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T03:57:58.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T03:57:58.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.inplace_update_support: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.bloom_locality: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.max_successive_merges: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T03:57:58.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.ttl: 2592000 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.enable_blob_files: false 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.min_blob_size: 0 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T03:57:58.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T03:57:58.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T03:57:58.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c3b06bec-76dd-47d1-b34d-7e10600ae36a 2024-09-10T03:57:58.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940678382151, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T03:57:58.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T03:57:58.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940678382547, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725940678, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c3b06bec-76dd-47d1-b34d-7e10600ae36a", "db_session_id": "KU7CAG0C1GR6JGFHR6VF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T03:57:58.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940678382641, "job": 1, "event": "recovery_finished"} 2024-09-10T03:57:58.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi070/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5606580cee00 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: DB pointer 0x5606581ca000 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** DB Stats ** 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:57:58.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T03:57:58.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T03:57:58.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T03:57:58.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:57:58.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T03:57:58.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T03:57:58.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T03:57:58.864 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache BinnedLRUCache@0x5606580a11f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070 does not exist in monmap, will attempt to join an existing cluster 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: using public_addr v2:172.21.15.70:0/0 -> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: starting mon.smithi070 rank -1 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi070 fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(???) e0 preinit fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:57:58.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).mds e1 new map 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).mds e1 print_map 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: e1 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: btime 2024-09-10T03:55:32:552214+0000 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: legacy client fscid: -1 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: No filesystems configured 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T03:57:58.866 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.867 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon alertmanager.smithi046 on smithi046 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1283215097' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1283215097' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi046:172.21.15.46=smithi046;smithi070:172.21.15.70=smithi070", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Saving service mon spec with placement smithi046:172.21.15.46=smithi046;smithi070:172.21.15.70=smithi070;count:2 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/258868477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3867148256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.868 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon grafana.smithi046 on smithi046 2024-09-10T03:57:58.869 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/1237715639' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3553364605' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3688303505' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/1227687511' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.870 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon prometheus.smithi046 on smithi046 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/4153899456' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3429162156' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3499913905' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/2908165931' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.871 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/2413921250' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14162 172.21.15.46:0/796165313' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mgrmap e14: smithi046.zvckjg(active, since 52s) 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/1650062354' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Active manager daemon smithi046.zvckjg restarted 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Activating manager daemon smithi046.zvckjg 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mgrmap e15: smithi046.zvckjg(active, starting, since 0.00323905s) 2024-09-10T03:57:58.872 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Manager daemon smithi046.zvckjg is now available 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/4071009095' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mgrmap e16: smithi046.zvckjg(active, since 1.0053s) 2024-09-10T03:57:58.873 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: [10/Sep/2024:03:57:35] ENGINE Bus STARTING 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: [10/Sep/2024:03:57:35] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: [10/Sep/2024:03:57:35] ENGINE Error in HTTPServer.serve 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Traceback (most recent call last): 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._run(expiration_interval) 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: s = self.context.wrap_socket( 2024-09-10T03:57:58.874 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: return self.sslsocket_class._create( 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self.do_handshake() 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._sslobj.do_handshake() 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: [10/Sep/2024:03:57:35] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: [10/Sep/2024:03:57:35] ENGINE Bus STARTED 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mgrmap e17: smithi046.zvckjg(active, since 2s) 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/1085949139' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.875 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/520411411' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:57:58.876 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/4222461346' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon crash.smithi070 on smithi070 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/4240549415' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon node-exporter.smithi070 on smithi070 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3703896477' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/4173183735' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.879 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: Deploying daemon mon.smithi070 on smithi070 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/1202227236' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 ceph-mon[35495]: mon.smithi070@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 bash[35476]: b5145a46761afe21e03716937f5c9e51d4f7409c8059b6b850f34cd7f853fde5 2024-09-10T03:57:58.880 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:57:58 smithi070 systemd[1]: Started Ceph mon.smithi070 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:57:58.978 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T03:57:58.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:57:59.222 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T03:58:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: mon.smithi046 calling monitor election 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: mon.smithi070 calling monitor election 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: mon.smithi046 is new leader, mons smithi046,smithi070 in quorum (ranks 0,1) 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: monmap epoch 2 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: last_changed 2024-09-10T03:57:58.410804+0000 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: min_mon_release 19 (squid) 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: election_strategy: 1 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: 1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: fsmap 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: mgrmap e17: smithi046.zvckjg(active, since 29s) 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: overall HEALTH_OK 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: Standby manager daemon smithi070.tezbzn started 2024-09-10T03:58:03.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T03:58:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: mon.smithi046 calling monitor election 2024-09-10T03:58:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: mon.smithi070 calling monitor election 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: mon.smithi046 is new leader, mons smithi046,smithi070 in quorum (ranks 0,1) 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: monmap epoch 2 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:58:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: last_changed 2024-09-10T03:57:58.410804+0000 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: min_mon_release 19 (squid) 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: election_strategy: 1 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: 1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: fsmap 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: mgrmap e17: smithi046.zvckjg(active, since 29s) 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: overall HEALTH_OK 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: Standby manager daemon smithi070.tezbzn started 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T03:58:03.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T03:58:03.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/214364359' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T03:58:03.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:03.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:04.305 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T03:58:04.306 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":2,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:57:58.410804Z","created":"2024-09-10T03:55:29.528347Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"smithi046","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi070","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T03:58:04.306 INFO:teuthology.orchestra.run.smithi070.stderr:dumped monmap epoch 2 2024-09-10T03:58:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:04 smithi070 ceph-mon[35495]: mgrmap e18: smithi046.zvckjg(active, since 29s), standbys: smithi070.tezbzn 2024-09-10T03:58:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi070.tezbzn", "id": "smithi070.tezbzn"}]: dispatch 2024-09-10T03:58:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:58:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:04 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/803592253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:58:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:04.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:04 smithi046 ceph-mon[28328]: mgrmap e18: smithi046.zvckjg(active, since 29s), standbys: smithi070.tezbzn 2024-09-10T03:58:04.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi070.tezbzn", "id": "smithi070.tezbzn"}]: dispatch 2024-09-10T03:58:04.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:58:04.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:04.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:04 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/803592253' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T03:58:04.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T03:58:05.067 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T03:58:05.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T03:58:05.338 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:05.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:05 smithi046 ceph-mon[28328]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:05.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:05 smithi070 ceph-mon[35495]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:06.002 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:58:06.002 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-09-10T03:58:06.002 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T03:58:06.002 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] 2024-09-10T03:58:06.747 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T03:58:06.747 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:58:06.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T03:58:06.777 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:58:06.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:58:06.843 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:58:06.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T03:58:06.873 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:58:06.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:58:06.939 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-10T03:58:06.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-10T03:58:06.941 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-10T03:58:06.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-10T03:58:06.984 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-10T03:58:06.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-10T03:58:06.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1713375535' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:06.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: Reconfiguring mon.smithi046 (unknown last config time)... 2024-09-10T03:58:07.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T03:58:07.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T03:58:07.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:07.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:06 smithi046 ceph-mon[28328]: Reconfiguring daemon mon.smithi046 on smithi046 2024-09-10T03:58:07.000 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-10T03:58:07.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-10T03:58:07.044 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-10T03:58:07.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-10T03:58:07.087 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-10T03:58:07.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-10T03:58:07.133 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-10T03:58:07.133 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-10T03:58:07.177 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-10T03:58:07.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-10T03:58:07.221 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-10T03:58:07.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-10T03:58:07.265 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-10T03:58:07.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-10T03:58:07.309 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-10T03:58:07.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-10T03:58:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:58:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1713375535' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: Reconfiguring mon.smithi046 (unknown last config time)... 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:07.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:06 smithi070 ceph-mon[35495]: Reconfiguring daemon mon.smithi046 on smithi046 2024-09-10T03:58:07.353 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-10T03:58:07.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-10T03:58:07.397 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-10T03:58:07.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-10T03:58:07.441 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T03:58:07.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-10T03:58:07.485 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T03:58:07.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-10T03:58:07.536 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-10T03:58:07.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-10T03:58:07.585 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T03:58:07.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T03:58:07.630 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-10T03:58:07.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-10T03:58:07.674 DEBUG:tasks.cephadm:set 18 configs 2024-09-10T03:58:07.674 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi046') 2024-09-10T03:58:08.257 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:09.269 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: Reconfiguring mgr.smithi046.zvckjg (unknown last config time)... 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi046.zvckjg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: Reconfiguring daemon mgr.smithi046.zvckjg on smithi046 2024-09-10T03:58:09.270 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: Reconfiguring mgr.smithi046.zvckjg (unknown last config time)... 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi046.zvckjg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: Reconfiguring daemon mgr.smithi046.zvckjg on smithi046 2024-09-10T03:58:09.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:10.525 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:10 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3063558837' entity='client.admin' 2024-09-10T03:58:10.525 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:10.525 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:10.525 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T03:58:10.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:10.563 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.582 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:10 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3063558837' entity='client.admin' 2024-09-10T03:58:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T03:58:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:10.590 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.593 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.593 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.597 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.597 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.600 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.600 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.605 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.697 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:10.726 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:11.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:11 smithi046 ceph-mon[28328]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:11 smithi046 ceph-mon[28328]: Reconfiguring ceph-exporter.smithi046 (monmap changed)... 2024-09-10T03:58:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:11 smithi046 ceph-mon[28328]: Reconfiguring daemon ceph-exporter.smithi046 on smithi046 2024-09-10T03:58:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:11 smithi070 ceph-mon[35495]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:11 smithi070 ceph-mon[35495]: Reconfiguring ceph-exporter.smithi046 (monmap changed)... 2024-09-10T03:58:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:11 smithi070 ceph-mon[35495]: Reconfiguring daemon ceph-exporter.smithi046 on smithi046 2024-09-10T03:58:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.078+0000 7f89ff0e4640 1 -- 172.21.15.46:0/671435726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f89f8190110 con 0x7f89f8101a60 2024-09-10T03:58:13.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.082+0000 7f89f5ffb640 1 -- 172.21.15.46:0/671435726 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v13)=0 v13) ==== 120+0+0 (secure 0 0 0) 0x7f89e40848c0 con 0x7f89f8101a60 2024-09-10T03:58:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 -- 172.21.15.46:0/671435726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89e806ddb0 msgr2=0x7f89e8070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 --2- 172.21.15.46:0/671435726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89e806ddb0 0x7f89e8070270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f89f81018c0 tx=0x7f89ec005d30 comp rx=0 tx=0).stop 2024-09-10T03:58:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 -- 172.21.15.46:0/671435726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8101a60 msgr2=0x7f89f8196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 --2- 172.21.15.46:0/671435726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8101a60 0x7f89f8196640 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009230 tx=0x7f89e4005060 comp rx=0 tx=0).stop 2024-09-10T03:58:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 -- 172.21.15.46:0/671435726 shutdown_connections 2024-09-10T03:58:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 --2- 172.21.15.46:0/671435726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89f8101a60 0x7f89f8196640 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 --2- 172.21.15.46:0/671435726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89e806ddb0 0x7f89e8070270 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 --2- 172.21.15.46:0/671435726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89f8100860 0x7f89f8196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 -- 172.21.15.46:0/671435726 >> 172.21.15.46:0/671435726 conn(0x7f89f80fc010 msgr2=0x7f89f8105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 -- 172.21.15.46:0/671435726 shutdown_connections 2024-09-10T03:58:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.084+0000 7f89ff0e4640 1 -- 172.21.15.46:0/671435726 wait complete. 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.086+0000 7f4b09216640 1 -- 172.21.15.46:0/4096057911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b04104d20 msgr2=0x7f4b04107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.086+0000 7f4b09216640 1 --2- 172.21.15.46:0/4096057911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b04104d20 0x7f4b04107110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4af80098b0 tx=0x7f4af802a4d0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.086+0000 7f4b09216640 1 -- 172.21.15.46:0/4096057911 shutdown_connections 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.086+0000 7f4b09216640 1 --2- 172.21.15.46:0/4096057911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b04104d20 0x7f4b04107110 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.086+0000 7f4b09216640 1 --2- 172.21.15.46:0/4096057911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b041023f0 0x7f4b041047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.086+0000 7f4b09216640 1 -- 172.21.15.46:0/4096057911 >> 172.21.15.46:0/4096057911 conn(0x7f4b040fc010 msgr2=0x7f4b040fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.086+0000 7f4b09216640 1 -- 172.21.15.46:0/4096057911 shutdown_connections 2024-09-10T03:58:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.087+0000 7f4b09216640 1 -- 172.21.15.46:0/4096057911 wait complete. 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.087+0000 7f4b09216640 1 Processor -- start 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b09216640 1 -- start start 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b09216640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b041023f0 0x7f4b04101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b09216640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04104d20 0x7f4b041003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b09216640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0410a440 con 0x7f4b041023f0 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b09216640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0410a2c0 con 0x7f4b04104d20 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b02d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b041023f0 0x7f4b04101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b02d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b041023f0 0x7f4b04101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57976/0 (socket says 172.21.15.46:57976) 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b02d9d640 1 -- 172.21.15.46:0/2800004662 learned_addr learned my addr 172.21.15.46:0/2800004662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b0259c640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04104d20 0x7f4b041003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b02d9d640 1 -- 172.21.15.46:0/2800004662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04104d20 msgr2=0x7f4b041003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b02d9d640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04104d20 0x7f4b041003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b02d9d640 1 -- 172.21.15.46:0/2800004662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b041008e0 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.088+0000 7f4b02d9d640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b041023f0 0x7f4b04101ce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4af400d8d0 tx=0x7f4af400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.089+0000 7f4aebfff640 1 -- 172.21.15.46:0/2800004662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4af4017070 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.089+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b04100bd0 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.089+0000 7f4aebfff640 1 -- 172.21.15.46:0/2800004662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(24 keys) ==== 1022+0+0 (secure 0 0 0) 0x7f4af4010460 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.089+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b041011b0 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.089+0000 7f4aebfff640 1 -- 172.21.15.46:0/2800004662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4af400f5f0 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.090+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b0410ac20 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.090+0000 7f4aebfff640 1 -- 172.21.15.46:0/2800004662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f4af4009a30 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.091+0000 7f4aebfff640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4aec06de00 0x7f4aec0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.091+0000 7f4aebfff640 1 -- 172.21.15.46:0/2800004662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4af408c760 con 0x7f4b041023f0 2024-09-10T03:58:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.091+0000 7f4b0259c640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4aec06de00 0x7f4aec0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.091+0000 7f4b0259c640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4aec06de00 0x7f4aec0702c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4af8004d80 tx=0x7f4af8004d10 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.095+0000 7f4aebfff640 1 -- 172.21.15.46:0/2800004662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4af405f110 con 0x7f4b041023f0 2024-09-10T03:58:13.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.276+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f4b041027d0 con 0x7f4b041023f0 2024-09-10T03:58:13.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.278+0000 7f4aebfff640 1 -- 172.21.15.46:0/2800004662 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v14)=0 v14) ==== 125+0+0 (secure 0 0 0) 0x7f4af4063940 con 0x7f4b041023f0 2024-09-10T03:58:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.280+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4aec06de00 msgr2=0x7f4aec0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.280+0000 7f4b09216640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4aec06de00 0x7f4aec0702c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4af8004d80 tx=0x7f4af8004d10 comp rx=0 tx=0).stop 2024-09-10T03:58:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b041023f0 msgr2=0x7f4b04101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b041023f0 0x7f4b04101ce0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4af400d8d0 tx=0x7f4af400dd90 comp rx=0 tx=0).stop 2024-09-10T03:58:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 shutdown_connections 2024-09-10T03:58:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4aec06de00 0x7f4aec0702c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b04104d20 0x7f4b041003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 --2- 172.21.15.46:0/2800004662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b041023f0 0x7f4b04101ce0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 >> 172.21.15.46:0/2800004662 conn(0x7f4b040fc010 msgr2=0x7f4b040fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 shutdown_connections 2024-09-10T03:58:13.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:13.281+0000 7f4b09216640 1 -- 172.21.15.46:0/2800004662 wait complete. 2024-09-10T03:58:13.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:13 smithi046 ceph-mon[28328]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:13.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:13 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1844908197' entity='client.admin' 2024-09-10T03:58:13.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:13 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/671435726' entity='client.admin' 2024-09-10T03:58:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:13 smithi070 ceph-mon[35495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:13 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1844908197' entity='client.admin' 2024-09-10T03:58:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:13 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/671435726' entity='client.admin' 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 -- 172.21.15.46:0/1844908197 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f3ce406de00 msgr2=0x7f3ce40702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 --2- 172.21.15.46:0/1844908197 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f3ce406de00 0x7f3ce40702c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3cf0005fd0 tx=0x7f3cf0005850 comp rx=0 tx=0).stop 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 -- 172.21.15.46:0/1844908197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cfc100860 msgr2=0x7f3cfc1960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 --2- 172.21.15.46:0/1844908197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cfc100860 0x7f3cfc1960a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3cec02a9c0 tx=0x7f3cec004e30 comp rx=0 tx=0).stop 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 -- 172.21.15.46:0/1844908197 shutdown_connections 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 --2- 172.21.15.46:0/1844908197 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f3ce406de00 0x7f3ce40702c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 --2- 172.21.15.46:0/1844908197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3cfc101a60 0x7f3cfc1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 --2- 172.21.15.46:0/1844908197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3cfc100860 0x7f3cfc1960a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 -- 172.21.15.46:0/1844908197 >> 172.21.15.46:0/1844908197 conn(0x7f3cfc0fc010 msgr2=0x7f3cfc105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 -- 172.21.15.46:0/1844908197 shutdown_connections 2024-09-10T03:58:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:12.991+0000 7f3d02c2b640 1 -- 172.21.15.46:0/1844908197 wait complete. 2024-09-10T03:58:14.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:14 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2800004662' entity='client.admin' 2024-09-10T03:58:14.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:14 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2800004662' entity='client.admin' 2024-09-10T03:58:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.833+0000 7f02c8c82640 1 -- 172.21.15.46:0/2722080521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 msgr2=0x7f02c41047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:14.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.833+0000 7f02c8c82640 1 --2- 172.21.15.46:0/2722080521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c41047e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02b40098b0 tx=0x7f02b402a4d0 comp rx=0 tx=0).stop 2024-09-10T03:58:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.834+0000 7f02c8c82640 1 -- 172.21.15.46:0/2722080521 shutdown_connections 2024-09-10T03:58:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.834+0000 7f02c8c82640 1 --2- 172.21.15.46:0/2722080521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c4104d20 0x7f02c4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.834+0000 7f02c8c82640 1 --2- 172.21.15.46:0/2722080521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c41047e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.834+0000 7f02c8c82640 1 -- 172.21.15.46:0/2722080521 >> 172.21.15.46:0/2722080521 conn(0x7f02c40fc010 msgr2=0x7f02c40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.834+0000 7f02c8c82640 1 -- 172.21.15.46:0/2722080521 shutdown_connections 2024-09-10T03:58:14.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.834+0000 7f02c8c82640 1 -- 172.21.15.46:0/2722080521 wait complete. 2024-09-10T03:58:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.834+0000 7f02c8c82640 1 Processor -- start 2024-09-10T03:58:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c8c82640 1 -- start start 2024-09-10T03:58:14.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c8c82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c8c82640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c4104d20 0x7f02c4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c8c82640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410a480 con 0x7f02c41023f0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c8c82640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f02c410a300 con 0x7f02c4104d20 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c4197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58010/0 (socket says 172.21.15.46:58010) 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c259c640 1 -- 172.21.15.46:0/3291135527 learned_addr learned my addr 172.21.15.46:0/3291135527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02bbfff640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c4104d20 0x7f02c4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c259c640 1 -- 172.21.15.46:0/3291135527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c4104d20 msgr2=0x7f02c4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c259c640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c4104d20 0x7f02c4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c259c640 1 -- 172.21.15.46:0/3291135527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c4197ef0 con 0x7f02c41023f0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.835+0000 7f02c259c640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c4197330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009230 tx=0x7f02b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.836+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02b403f070 con 0x7f02c41023f0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.836+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(24 keys) ==== 1022+0+0 (secure 0 0 0) 0x7f02b4005290 con 0x7f02c41023f0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.836+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c419c8d0 con 0x7f02c41023f0 2024-09-10T03:58:14.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.836+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02b4035030 con 0x7f02c41023f0 2024-09-10T03:58:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.836+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c419cee0 con 0x7f02c41023f0 2024-09-10T03:58:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.837+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c410ac60 con 0x7f02c41023f0 2024-09-10T03:58:14.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.838+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f02b4042070 con 0x7f02c41023f0 2024-09-10T03:58:14.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.838+0000 7f02bb7fe640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f02b006ddb0 0x7f02b0070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:14.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.838+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f02b40ad7c0 con 0x7f02c41023f0 2024-09-10T03:58:14.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.838+0000 7f02bbfff640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f02b006ddb0 0x7f02b0070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:14.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.839+0000 7f02bbfff640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f02b006ddb0 0x7f02b0070270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f02ac005fd0 tx=0x7f02ac005e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:14.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.842+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02b407c440 con 0x7f02c41023f0 2024-09-10T03:58:14.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.916+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 7 ==== config(25 keys) ==== 1053+0+0 (secure 0 0 0) 0x7f02b40b3080 con 0x7f02c41023f0 2024-09-10T03:58:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.029+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f02c419d170 con 0x7f02c41023f0 2024-09-10T03:58:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.032+0000 7f02bb7fe640 1 -- 172.21.15.46:0/3291135527 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f02b40800f0 con 0x7f02c41023f0 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f02b006ddb0 msgr2=0x7f02b0070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f02b006ddb0 0x7f02b0070270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f02ac005fd0 tx=0x7f02ac005e20 comp rx=0 tx=0).stop 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 msgr2=0x7f02c4197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c4197330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009230 tx=0x7f02b4004270 comp rx=0 tx=0).stop 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 shutdown_connections 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f02b006ddb0 0x7f02b0070270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02c4104d20 0x7f02c4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 --2- 172.21.15.46:0/3291135527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02c41023f0 0x7f02c4197330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 >> 172.21.15.46:0/3291135527 conn(0x7f02c40fc010 msgr2=0x7f02c40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 shutdown_connections 2024-09-10T03:58:15.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.035+0000 7f02c8c82640 1 -- 172.21.15.46:0/3291135527 wait complete. 2024-09-10T03:58:15.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:15 smithi046 ceph-mon[28328]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:15.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:15 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1979813582' entity='client.admin' 2024-09-10T03:58:15.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:15 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3291135527' entity='client.admin' 2024-09-10T03:58:15.549 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:15 smithi070 ceph-mon[35495]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:15.549 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:15 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1979813582' entity='client.admin' 2024-09-10T03:58:15.549 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:15 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3291135527' entity='client.admin' 2024-09-10T03:58:15.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 -- 172.21.15.46:0/3858186865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078100860 msgr2=0x7fd078100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/3858186865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078100860 0x7fd078100c60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd068009880 tx=0x7fd06802a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 -- 172.21.15.46:0/3858186865 shutdown_connections 2024-09-10T03:58:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/3858186865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd078101a60 0x7fd078101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/3858186865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078100860 0x7fd078100c60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 -- 172.21.15.46:0/3858186865 >> 172.21.15.46:0/3858186865 conn(0x7fd0780fc010 msgr2=0x7fd0780fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 -- 172.21.15.46:0/3858186865 shutdown_connections 2024-09-10T03:58:15.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.714+0000 7fd07dbd0640 1 -- 172.21.15.46:0/3858186865 wait complete. 2024-09-10T03:58:15.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd07dbd0640 1 Processor -- start 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd07dbd0640 1 -- start start 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd07dbd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd078100860 0x7fd078197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd07dbd0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078101a60 0x7fd0781978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd07dbd0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd078109490 con 0x7fd078101a60 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd07dbd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd078109310 con 0x7fd078100860 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd076ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078101a60 0x7fd0781978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd078100860 0x7fd078197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd078100860 0x7fd078197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:36984/0 (socket says 172.21.15.46:36984) 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd076ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078101a60 0x7fd0781978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58022/0 (socket says 172.21.15.46:58022) 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd0777fe640 1 -- 172.21.15.46:0/2420826128 learned_addr learned my addr 172.21.15.46:0/2420826128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd076ffd640 1 -- 172.21.15.46:0/2420826128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd078100860 msgr2=0x7fd078197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd076ffd640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd078100860 0x7fd078197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.715+0000 7fd076ffd640 1 -- 172.21.15.46:0/2420826128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078197f30 con 0x7fd078101a60 2024-09-10T03:58:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.716+0000 7fd076ffd640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078101a60 0x7fd0781978b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00d6a0 tx=0x7fd06c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.716+0000 7fd074ff9640 1 -- 172.21.15.46:0/2420826128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd06c017070 con 0x7fd078101a60 2024-09-10T03:58:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.716+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07819c970 con 0x7fd078101a60 2024-09-10T03:58:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.716+0000 7fd074ff9640 1 -- 172.21.15.46:0/2420826128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(25 keys) ==== 1053+0+0 (secure 0 0 0) 0x7fd06c0054a0 con 0x7fd078101a60 2024-09-10T03:58:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.716+0000 7fd074ff9640 1 -- 172.21.15.46:0/2420826128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd06c0049d0 con 0x7fd078101a60 2024-09-10T03:58:15.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.716+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07819cf80 con 0x7fd078101a60 2024-09-10T03:58:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.717+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd078100c60 con 0x7fd078101a60 2024-09-10T03:58:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.717+0000 7fd074ff9640 1 -- 172.21.15.46:0/2420826128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7fd06c004c90 con 0x7fd078101a60 2024-09-10T03:58:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.718+0000 7fd074ff9640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd06006ddb0 0x7fd060070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.718+0000 7fd074ff9640 1 -- 172.21.15.46:0/2420826128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd06c08c560 con 0x7fd078101a60 2024-09-10T03:58:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.718+0000 7fd0777fe640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd06006ddb0 0x7fd060070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:15.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.718+0000 7fd0777fe640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd06006ddb0 0x7fd060070270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd068000c00 tx=0x7fd068004e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:15.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.722+0000 7fd074ff9640 1 -- 172.21.15.46:0/2420826128 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd06c05ee90 con 0x7fd078101a60 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.898+0000 7f681c107640 1 -- 172.21.15.46:0/2079808672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814101a60 msgr2=0x7f6814101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.898+0000 7f681c107640 1 --2- 172.21.15.46:0/2079808672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814101a60 0x7f6814101ee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6808009880 tx=0x7f680802a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 -- 172.21.15.46:0/2079808672 shutdown_connections 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 --2- 172.21.15.46:0/2079808672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814101a60 0x7f6814101ee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 --2- 172.21.15.46:0/2079808672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6814100860 0x7f6814100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 -- 172.21.15.46:0/2079808672 >> 172.21.15.46:0/2079808672 conn(0x7f68140fc010 msgr2=0x7f68140fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 -- 172.21.15.46:0/2079808672 shutdown_connections 2024-09-10T03:58:15.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 -- 172.21.15.46:0/2079808672 wait complete. 2024-09-10T03:58:15.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fd07819d270 con 0x7fd078101a60 2024-09-10T03:58:15.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 Processor -- start 2024-09-10T03:58:15.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.899+0000 7f681c107640 1 -- start start 2024-09-10T03:58:15.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f681c107640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814100860 0x7f6814197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f681c107640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6814101a60 0x7f68141978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f681c107640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6814109490 con 0x7f6814100860 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f681c107640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6814109310 con 0x7f6814101a60 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f6819ea3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814100860 0x7f6814197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f6819ea3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814100860 0x7f6814197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58038/0 (socket says 172.21.15.46:58038) 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f6819ea3640 1 -- 172.21.15.46:0/428151419 learned_addr learned my addr 172.21.15.46:0/428151419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f68196a2640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6814101a60 0x7f68141978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f6819ea3640 1 -- 172.21.15.46:0/428151419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6814101a60 msgr2=0x7f68141978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f6819ea3640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6814101a60 0x7f68141978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f6819ea3640 1 -- 172.21.15.46:0/428151419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6814197f20 con 0x7f6814100860 2024-09-10T03:58:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.900+0000 7f6819ea3640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814100860 0x7f6814197360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f680400d8d0 tx=0x7f680400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.902+0000 7fd074ff9640 1 -- 172.21.15.46:0/2420826128 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v17)=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fd06c0636c0 con 0x7fd078101a60 2024-09-10T03:58:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.903+0000 7f6812ffd640 1 -- 172.21.15.46:0/428151419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6804017070 con 0x7f6814100860 2024-09-10T03:58:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.903+0000 7f6812ffd640 1 -- 172.21.15.46:0/428151419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(25 keys) ==== 1053+0+0 (secure 0 0 0) 0x7f680400be10 con 0x7f6814100860 2024-09-10T03:58:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.903+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681419c960 con 0x7f6814100860 2024-09-10T03:58:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.903+0000 7f6812ffd640 1 -- 172.21.15.46:0/428151419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6804010e30 con 0x7f6814100860 2024-09-10T03:58:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.903+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f681419cf70 con 0x7f6814100860 2024-09-10T03:58:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6814100c60 con 0x7f6814100860 2024-09-10T03:58:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd06006ddb0 msgr2=0x7fd060070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd06006ddb0 0x7fd060070270 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd068000c00 tx=0x7fd068004e40 comp rx=0 tx=0).stop 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078101a60 msgr2=0x7fd0781978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078101a60 0x7fd0781978b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00d6a0 tx=0x7fd06c00db60 comp rx=0 tx=0).stop 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 shutdown_connections 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd078101a60 0x7fd0781978b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd06006ddb0 0x7fd060070270 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 --2- 172.21.15.46:0/2420826128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd078100860 0x7fd078197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.904+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 >> 172.21.15.46:0/2420826128 conn(0x7fd0780fc010 msgr2=0x7fd078069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.905+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 shutdown_connections 2024-09-10T03:58:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.905+0000 7fd07dbd0640 1 -- 172.21.15.46:0/2420826128 wait complete. 2024-09-10T03:58:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.905+0000 7f6812ffd640 1 -- 172.21.15.46:0/428151419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f680401a070 con 0x7f6814100860 2024-09-10T03:58:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.905+0000 7f6812ffd640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f67fc06de00 0x7f67fc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.906+0000 7f6812ffd640 1 -- 172.21.15.46:0/428151419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f680408c510 con 0x7f6814100860 2024-09-10T03:58:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.906+0000 7f68196a2640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f67fc06de00 0x7f67fc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:15.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.906+0000 7f68196a2640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f67fc06de00 0x7f67fc0702c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6814198810 tx=0x7f6808005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:15.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:15.910+0000 7f6812ffd640 1 -- 172.21.15.46:0/428151419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f680405eec0 con 0x7f6814100860 2024-09-10T03:58:16.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.090+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f681419d260 con 0x7f6814100860 2024-09-10T03:58:16.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.093+0000 7f6812ffd640 1 -- 172.21.15.46:0/428151419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v18)=0 v18) ==== 115+0+0 (secure 0 0 0) 0x7f68040636f0 con 0x7f6814100860 2024-09-10T03:58:16.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f67fc06de00 msgr2=0x7f67fc0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:16.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f67fc06de00 0x7f67fc0702c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6814198810 tx=0x7f6808005a20 comp rx=0 tx=0).stop 2024-09-10T03:58:16.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814100860 msgr2=0x7f6814197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:16.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814100860 0x7f6814197360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f680400d8d0 tx=0x7f680400dd90 comp rx=0 tx=0).stop 2024-09-10T03:58:16.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 shutdown_connections 2024-09-10T03:58:16.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f67fc06de00 0x7f67fc0702c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6814101a60 0x7f68141978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 --2- 172.21.15.46:0/428151419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6814100860 0x7f6814197360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 >> 172.21.15.46:0/428151419 conn(0x7f68140fc010 msgr2=0x7f6814069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:16.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 shutdown_connections 2024-09-10T03:58:16.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.095+0000 7f681c107640 1 -- 172.21.15.46:0/428151419 wait complete. 2024-09-10T03:58:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 -- 172.21.15.46:0/3679359947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c1023f0 msgr2=0x7f0d7c1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 --2- 172.21.15.46:0/3679359947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c1023f0 0x7f0d7c1047e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0098b0 tx=0x7f0d6c02a4d0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 -- 172.21.15.46:0/3679359947 shutdown_connections 2024-09-10T03:58:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 --2- 172.21.15.46:0/3679359947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d7c104d20 0x7f0d7c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 --2- 172.21.15.46:0/3679359947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c1023f0 0x7f0d7c1047e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 -- 172.21.15.46:0/3679359947 >> 172.21.15.46:0/3679359947 conn(0x7f0d7c0fc010 msgr2=0x7f0d7c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 -- 172.21.15.46:0/3679359947 shutdown_connections 2024-09-10T03:58:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.706+0000 7f0d813f2640 1 -- 172.21.15.46:0/3679359947 wait complete. 2024-09-10T03:58:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.707+0000 7f0d813f2640 1 Processor -- start 2024-09-10T03:58:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.707+0000 7f0d813f2640 1 -- start start 2024-09-10T03:58:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.707+0000 7f0d813f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d7c1023f0 0x7f0d7c101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.707+0000 7f0d813f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c104d20 0x7f0d7c1003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.707+0000 7f0d813f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10a5a0 con 0x7f0d7c104d20 2024-09-10T03:58:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.707+0000 7f0d813f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c10a420 con 0x7f0d7c1023f0 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.707+0000 7f0d7a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c104d20 0x7f0d7c1003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d7a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c104d20 0x7f0d7c1003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58056/0 (socket says 172.21.15.46:58056) 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d7a7fc640 1 -- 172.21.15.46:0/2056472038 learned_addr learned my addr 172.21.15.46:0/2056472038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d7affd640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d7c1023f0 0x7f0d7c101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d7a7fc640 1 -- 172.21.15.46:0/2056472038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d7c1023f0 msgr2=0x7f0d7c101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d7a7fc640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d7c1023f0 0x7f0d7c101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d7a7fc640 1 -- 172.21.15.46:0/2056472038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c1008e0 con 0x7f0d7c104d20 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d7a7fc640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c104d20 0x7f0d7c1003a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c8f0 tx=0x7f0d7000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d63fff640 1 -- 172.21.15.46:0/2056472038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d700079b0 con 0x7f0d7c104d20 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c100bd0 con 0x7f0d7c104d20 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d63fff640 1 -- 172.21.15.46:0/2056472038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(25 keys) ==== 1053+0+0 (secure 0 0 0) 0x7f0d70005020 con 0x7f0d7c104d20 2024-09-10T03:58:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d63fff640 1 -- 172.21.15.46:0/2056472038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d70004d00 con 0x7f0d7c104d20 2024-09-10T03:58:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.708+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c1011e0 con 0x7f0d7c104d20 2024-09-10T03:58:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.709+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d7c1027d0 con 0x7f0d7c104d20 2024-09-10T03:58:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.710+0000 7f0d63fff640 1 -- 172.21.15.46:0/2056472038 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f0d700051d0 con 0x7f0d7c104d20 2024-09-10T03:58:16.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.710+0000 7f0d63fff640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0d6406ddb0 0x7f0d64070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.711+0000 7f0d7affd640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0d6406ddb0 0x7f0d64070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.711+0000 7f0d63fff640 1 -- 172.21.15.46:0/2056472038 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0d7008c8a0 con 0x7f0d7c104d20 2024-09-10T03:58:16.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.711+0000 7f0d7affd640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0d6406ddb0 0x7f0d64070270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c004e40 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:16.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.715+0000 7f0d63fff640 1 -- 172.21.15.46:0/2056472038 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d70014030 con 0x7f0d7c104d20 2024-09-10T03:58:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.892+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f0d7c19f4a0 con 0x7f0d7c104d20 2024-09-10T03:58:16.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.896+0000 7f0d63fff640 1 -- 172.21.15.46:0/2056472038 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f0d7005f1d0 con 0x7f0d7c104d20 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0d6406ddb0 msgr2=0x7f0d64070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0d6406ddb0 0x7f0d64070270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c000c00 tx=0x7f0d6c004e40 comp rx=0 tx=0).stop 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c104d20 msgr2=0x7f0d7c1003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c104d20 0x7f0d7c1003a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c8f0 tx=0x7f0d7000cdb0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 shutdown_connections 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d7c104d20 0x7f0d7c1003a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0d6406ddb0 0x7f0d64070270 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 --2- 172.21.15.46:0/2056472038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d7c1023f0 0x7f0d7c101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 >> 172.21.15.46:0/2056472038 conn(0x7f0d7c0fc010 msgr2=0x7f0d7c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 shutdown_connections 2024-09-10T03:58:16.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:16.898+0000 7f0d813f2640 1 -- 172.21.15.46:0/2056472038 wait complete. 2024-09-10T03:58:17.234 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:16 smithi046 ceph-mon[28328]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:17.235 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:16 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2420826128' entity='client.admin' 2024-09-10T03:58:17.235 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:16 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/428151419' entity='client.admin' 2024-09-10T03:58:17.235 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:16 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2056472038' entity='client.admin' 2024-09-10T03:58:17.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:16 smithi070 ceph-mon[35495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:17.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:16 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2420826128' entity='client.admin' 2024-09-10T03:58:17.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:16 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/428151419' entity='client.admin' 2024-09-10T03:58:17.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:16 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2056472038' entity='client.admin' 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 -- 172.21.15.46:0/2540729021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440100860 msgr2=0x7f4440100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 --2- 172.21.15.46:0/2540729021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440100860 0x7f4440100c60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4438009880 tx=0x7f443802a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 -- 172.21.15.46:0/2540729021 shutdown_connections 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 --2- 172.21.15.46:0/2540729021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4440101a60 0x7f4440101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 --2- 172.21.15.46:0/2540729021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440100860 0x7f4440100c60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 -- 172.21.15.46:0/2540729021 >> 172.21.15.46:0/2540729021 conn(0x7f44400fc010 msgr2=0x7f44400fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 -- 172.21.15.46:0/2540729021 shutdown_connections 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 -- 172.21.15.46:0/2540729021 wait complete. 2024-09-10T03:58:17.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.604+0000 7f4447492640 1 Processor -- start 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4447492640 1 -- start start 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4447492640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4440100860 0x7f4440197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4447492640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440101a60 0x7f4440197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4447492640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44401094f0 con 0x7f4440101a60 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4447492640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4440109370 con 0x7f4440100860 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4444a2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440101a60 0x7f4440197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4444a2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440101a60 0x7f4440197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58090/0 (socket says 172.21.15.46:58090) 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4444a2d640 1 -- 172.21.15.46:0/2697583871 learned_addr learned my addr 172.21.15.46:0/2697583871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4444a2d640 1 -- 172.21.15.46:0/2697583871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4440100860 msgr2=0x7f4440197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4444a2d640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4440100860 0x7f4440197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4444a2d640 1 -- 172.21.15.46:0/2697583871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4440197ff0 con 0x7f4440101a60 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f4444a2d640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440101a60 0x7f4440197970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f443400b420 tx=0x7f443400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:17.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.605+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4434011630 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.606+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444019ca30 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.606+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(25 keys) ==== 1053+0+0 (secure 0 0 0) 0x7f4434016070 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.606+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4434010c00 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.606+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444019d040 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.607+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4440100ce0 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.607+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f44340117d0 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.608+0000 7f443e7fc640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f442806ddb0 0x7f4428070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.608+0000 7f444522e640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f442806ddb0 0x7f4428070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.608+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f443408c370 con 0x7f4440101a60 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.608+0000 7f444522e640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f442806ddb0 0x7f4428070270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44380096c0 tx=0x7f44380058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:17.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.612+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f443400e260 con 0x7f4440101a60 2024-09-10T03:58:17.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.709+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 7 ==== config(26 keys) ==== 1084+0+0 (secure 0 0 0) 0x7f443405eca0 con 0x7f4440101a60 2024-09-10T03:58:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.796+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f44401069c0 con 0x7f4440101a60 2024-09-10T03:58:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.800+0000 7f443e7fc640 1 -- 172.21.15.46:0/2697583871 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v21)=0 v21) ==== 120+0+0 (secure 0 0 0) 0x7f443405aff0 con 0x7f4440101a60 2024-09-10T03:58:17.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f442806ddb0 msgr2=0x7f4428070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f442806ddb0 0x7f4428070270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44380096c0 tx=0x7f44380058d0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440101a60 msgr2=0x7f4440197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440101a60 0x7f4440197970 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f443400b420 tx=0x7f443400b8e0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 shutdown_connections 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4440101a60 0x7f4440197970 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f442806ddb0 0x7f4428070270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 --2- 172.21.15.46:0/2697583871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4440100860 0x7f4440197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 >> 172.21.15.46:0/2697583871 conn(0x7f44400fc010 msgr2=0x7f44400fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 shutdown_connections 2024-09-10T03:58:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.802+0000 7f4447492640 1 -- 172.21.15.46:0/2697583871 wait complete. 2024-09-10T03:58:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- 172.21.15.46:0/381233292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288100860 msgr2=0x7fa288100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 --2- 172.21.15.46:0/381233292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288100860 0x7fa288100c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa284009880 tx=0x7fa28402a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- 172.21.15.46:0/381233292 shutdown_connections 2024-09-10T03:58:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 --2- 172.21.15.46:0/381233292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa288101a60 0x7fa288101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 --2- 172.21.15.46:0/381233292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288100860 0x7fa288100c60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- 172.21.15.46:0/381233292 >> 172.21.15.46:0/381233292 conn(0x7fa2880fc010 msgr2=0x7fa2880fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- 172.21.15.46:0/381233292 shutdown_connections 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- 172.21.15.46:0/381233292 wait complete. 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 Processor -- start 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- start start 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa288100860 0x7fa288197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288101a60 0x7fa2881978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa2881093b0 con 0x7fa288101a60 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.724+0000 7fa2905cd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa288109230 con 0x7fa288100860 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28db68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288101a60 0x7fa2881978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28db68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288101a60 0x7fa2881978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57988/0 (socket says 172.21.15.46:57988) 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28db68640 1 -- 172.21.15.46:0/1979813582 learned_addr learned my addr 172.21.15.46:0/1979813582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28e369640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa288100860 0x7fa288197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28db68640 1 -- 172.21.15.46:0/1979813582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa288100860 msgr2=0x7fa288197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28db68640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa288100860 0x7fa288197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28db68640 1 -- 172.21.15.46:0/1979813582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa288197f50 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28e369640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa288100860 0x7fa288197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa28db68640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288101a60 0x7fa2881978d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa27800b420 tx=0x7fa27800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa278011630 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(24 keys) ==== 1022+0+0 (secure 0 0 0) 0x7fa278016070 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa278010b50 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa28819c990 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.725+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa28819cfa0 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.726+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa288109b90 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.727+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7fa278010d10 con 0x7fa288101a60 2024-09-10T03:58:18.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.727+0000 7fa27f7fe640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa27406de00 0x7fa2740702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.727+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa27808c2a0 con 0x7fa288101a60 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.727+0000 7fa28e369640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa27406de00 0x7fa2740702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.728+0000 7fa28e369640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa27406de00 0x7fa2740702c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa284000c00 tx=0x7fa284004900 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.732+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa27805afa0 con 0x7fa288101a60 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.912+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fa28819d2c0 con 0x7fa288101a60 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.915+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 123+0+0 (secure 0 0 0) 0x7fa27805ec50 con 0x7fa288101a60 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.916+0000 7fa27f7fe640 1 -- 172.21.15.46:0/1979813582 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(25 keys) ==== 1053+0+0 (secure 0 0 0) 0x7fa278063480 con 0x7fa288101a60 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.917+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa27406de00 msgr2=0x7fa2740702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.917+0000 7fa2905cd640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa27406de00 0x7fa2740702c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa284000c00 tx=0x7fa284004900 comp rx=0 tx=0).stop 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.917+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288101a60 msgr2=0x7fa2881978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.917+0000 7fa2905cd640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288101a60 0x7fa2881978d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa27800b420 tx=0x7fa27800b8e0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.918+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 shutdown_connections 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.918+0000 7fa2905cd640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa288101a60 0x7fa2881978d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.918+0000 7fa2905cd640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa27406de00 0x7fa2740702c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.918+0000 7fa2905cd640 1 --2- 172.21.15.46:0/1979813582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa288100860 0x7fa288197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.918+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 >> 172.21.15.46:0/1979813582 conn(0x7fa2880fc010 msgr2=0x7fa28806a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.918+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 shutdown_connections 2024-09-10T03:58:18.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:14.918+0000 7fa2905cd640 1 -- 172.21.15.46:0/1979813582 wait complete. 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 -- 172.21.15.46:0/1963531164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 msgr2=0x7f1bec101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 --2- 172.21.15.46:0/1963531164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec101d60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009880 tx=0x7f1bdc02a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 -- 172.21.15.46:0/1963531164 shutdown_connections 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 --2- 172.21.15.46:0/1963531164 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bec102b60 0x7f1bec102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 --2- 172.21.15.46:0/1963531164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec101d60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 -- 172.21.15.46:0/1963531164 >> 172.21.15.46:0/1963531164 conn(0x7f1bec0fd0f0 msgr2=0x7f1bec0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 -- 172.21.15.46:0/1963531164 shutdown_connections 2024-09-10T03:58:18.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.518+0000 7f1bf1247640 1 -- 172.21.15.46:0/1963531164 wait complete. 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.519+0000 7f1bf1247640 1 Processor -- start 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.519+0000 7f1bf1247640 1 -- start start 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.519+0000 7f1bf1247640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.519+0000 7f1bf1247640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bec102b60 0x7f1bec199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.519+0000 7f1bf1247640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10a590 con 0x7f1bec101960 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.519+0000 7f1bf1247640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec10a410 con 0x7f1bec102b60 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bead9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bead9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec1995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58070/0 (socket says 172.21.15.46:58070) 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bead9d640 1 -- 172.21.15.46:0/3479734311 learned_addr learned my addr 172.21.15.46:0/3479734311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bea59c640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bec102b60 0x7f1bec199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bead9d640 1 -- 172.21.15.46:0/3479734311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bec102b60 msgr2=0x7f1bec199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bead9d640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bec102b60 0x7f1bec199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bead9d640 1 -- 172.21.15.46:0/3479734311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec19a190 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bea59c640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bec102b60 0x7f1bec199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bead9d640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec1995d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc004ea0 tx=0x7f1bdc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1bdc03f070 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(25 keys) ==== 1053+0+0 (secure 0 0 0) 0x7f1bdc004b40 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.520+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec19eb00 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.521+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1bdc035030 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.521+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec19f090 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.522+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bec101d60 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.522+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f1bdc042070 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.523+0000 7f1bd3fff640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1bd406ddb0 0x7f1bd4070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.523+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1bdc0ad6b0 con 0x7f1bec101960 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.523+0000 7f1bea59c640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1bd406ddb0 0x7f1bd4070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.523+0000 7f1bea59c640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1bd406ddb0 0x7f1bd4070270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1bec19aa10 tx=0x7f1be000b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.527+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1bdc07ffe0 con 0x7f1bec101960 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.706+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f1bec19f2f0 con 0x7f1bec101960 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.709+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v20)=0 v20) ==== 123+0+0 (secure 0 0 0) 0x7f1bdc084810 con 0x7f1bec101960 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.709+0000 7f1bd3fff640 1 -- 172.21.15.46:0/3479734311 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(26 keys) ==== 1084+0+0 (secure 0 0 0) 0x7f1bdc0318e0 con 0x7f1bec101960 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1bd406ddb0 msgr2=0x7f1bd4070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1bd406ddb0 0x7f1bd4070270 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1bec19aa10 tx=0x7f1be000b390 comp rx=0 tx=0).stop 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 msgr2=0x7f1bec1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec1995d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc004ea0 tx=0x7f1bdc004270 comp rx=0 tx=0).stop 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 shutdown_connections 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1bd406ddb0 0x7f1bd4070270 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1bec102b60 0x7f1bec199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 --2- 172.21.15.46:0/3479734311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bec101960 0x7f1bec1995d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 >> 172.21.15.46:0/3479734311 conn(0x7f1bec0fd0f0 msgr2=0x7f1bec074700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 shutdown_connections 2024-09-10T03:58:18.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.712+0000 7f1bf1247640 1 -- 172.21.15.46:0/3479734311 wait complete. 2024-09-10T03:58:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:18 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3479734311' entity='client.admin' 2024-09-10T03:58:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:18 smithi046 ceph-mon[28328]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:18 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2697583871' entity='client.admin' 2024-09-10T03:58:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:18 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/199863553' entity='client.admin' 2024-09-10T03:58:18.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.877+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/1254554898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4101a60 msgr2=0x7fd4f4101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.877+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/1254554898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4101a60 0x7fd4f4101ee0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009880 tx=0x7fd4e802a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.877+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/1254554898 shutdown_connections 2024-09-10T03:58:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.877+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/1254554898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4101a60 0x7fd4f4101ee0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.877+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/1254554898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f4100860 0x7fd4f4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.877+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/1254554898 >> 172.21.15.46:0/1254554898 conn(0x7fd4f40fc010 msgr2=0x7fd4f40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.877+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/1254554898 shutdown_connections 2024-09-10T03:58:18.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.878+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/1254554898 wait complete. 2024-09-10T03:58:18.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.878+0000 7fd4f8ae5640 1 Processor -- start 2024-09-10T03:58:18.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.878+0000 7fd4f8ae5640 1 -- start start 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f8ae5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4100860 0x7fd4f4197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f8ae5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f4101a60 0x7fd4f4197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f8ae5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41095b0 con 0x7fd4f4100860 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f8ae5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4109430 con 0x7fd4f4101a60 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4100860 0x7fd4f4197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4100860 0x7fd4f4197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39100/0 (socket says 172.21.15.46:39100) 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f259c640 1 -- 172.21.15.46:0/2329662807 learned_addr learned my addr 172.21.15.46:0/2329662807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f1d9b640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f4101a60 0x7fd4f4197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f259c640 1 -- 172.21.15.46:0/2329662807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f4101a60 msgr2=0x7fd4f4197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f259c640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f4101a60 0x7fd4f4197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f259c640 1 -- 172.21.15.46:0/2329662807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f4198010 con 0x7fd4f4100860 2024-09-10T03:58:18.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4f259c640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4100860 0x7fd4f4197450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d6f0 tx=0x7fd4e400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4e37fe640 1 -- 172.21.15.46:0/2329662807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd4e4017070 con 0x7fd4f4100860 2024-09-10T03:58:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.879+0000 7fd4e37fe640 1 -- 172.21.15.46:0/2329662807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1084+0+0 (secure 0 0 0) 0x7fd4e400de00 con 0x7fd4f4100860 2024-09-10T03:58:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.880+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f419ca50 con 0x7fd4f4100860 2024-09-10T03:58:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.880+0000 7fd4e37fe640 1 -- 172.21.15.46:0/2329662807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd4e4003b20 con 0x7fd4f4100860 2024-09-10T03:58:18.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.880+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f419d060 con 0x7fd4f4100860 2024-09-10T03:58:18.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.881+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4100c60 con 0x7fd4f4100860 2024-09-10T03:58:18.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.881+0000 7fd4e37fe640 1 -- 172.21.15.46:0/2329662807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7fd4e4003cc0 con 0x7fd4f4100860 2024-09-10T03:58:18.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.882+0000 7fd4e37fe640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd4dc06de00 0x7fd4dc0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:18.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.882+0000 7fd4e37fe640 1 -- 172.21.15.46:0/2329662807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd4e408c4e0 con 0x7fd4f4100860 2024-09-10T03:58:18.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.882+0000 7fd4f1d9b640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd4dc06de00 0x7fd4dc0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:18.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.882+0000 7fd4f1d9b640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd4dc06de00 0x7fd4dc0702c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4198900 tx=0x7fd4e8004eb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:18.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.886+0000 7fd4e37fe640 1 -- 172.21.15.46:0/2329662807 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4e405ee90 con 0x7fd4f4100860 2024-09-10T03:58:18.979 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:18 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3479734311' entity='client.admin' 2024-09-10T03:58:18.979 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:18 smithi070 ceph-mon[35495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:18.979 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:18 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2697583871' entity='client.admin' 2024-09-10T03:58:18.979 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:18 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/199863553' entity='client.admin' 2024-09-10T03:58:19.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.061+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fd4f419d320 con 0x7fd4f4100860 2024-09-10T03:58:19.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.064+0000 7fd4e37fe640 1 -- 172.21.15.46:0/2329662807 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v23)=0 v23) ==== 106+0+0 (secure 0 0 0) 0x7fd4e40636c0 con 0x7fd4f4100860 2024-09-10T03:58:19.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd4dc06de00 msgr2=0x7fd4dc0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd4dc06de00 0x7fd4dc0702c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4f4198900 tx=0x7fd4e8004eb0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4100860 msgr2=0x7fd4f4197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4100860 0x7fd4f4197450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400d6f0 tx=0x7fd4e400dbb0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 shutdown_connections 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd4dc06de00 0x7fd4dc0702c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4f4101a60 0x7fd4f4197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 --2- 172.21.15.46:0/2329662807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4f4100860 0x7fd4f4197450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 >> 172.21.15.46:0/2329662807 conn(0x7fd4f40fc010 msgr2=0x7fd4f4069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 shutdown_connections 2024-09-10T03:58:19.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.067+0000 7fd4f8ae5640 1 -- 172.21.15.46:0/2329662807 wait complete. 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 -- 172.21.15.46:0/835891902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 msgr2=0x7f98a4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/835891902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4100c60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9894009880 tx=0x7f989402a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 -- 172.21.15.46:0/835891902 shutdown_connections 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/835891902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a4101a60 0x7f98a4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/835891902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4100c60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 -- 172.21.15.46:0/835891902 >> 172.21.15.46:0/835891902 conn(0x7f98a40fc010 msgr2=0x7f98a40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 -- 172.21.15.46:0/835891902 shutdown_connections 2024-09-10T03:58:19.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 -- 172.21.15.46:0/835891902 wait complete. 2024-09-10T03:58:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 Processor -- start 2024-09-10T03:58:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.658+0000 7f98a9bf1640 1 -- start start 2024-09-10T03:58:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a9bf1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a9bf1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a4101a60 0x7f98a4102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a9bf1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4109630 con 0x7f98a4100860 2024-09-10T03:58:19.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a9bf1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41094b0 con 0x7f98a4101a60 2024-09-10T03:58:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39118/0 (socket says 172.21.15.46:39118) 2024-09-10T03:58:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a37fe640 1 -- 172.21.15.46:0/2592313990 learned_addr learned my addr 172.21.15.46:0/2592313990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a2ffd640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a4101a60 0x7f98a4102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a37fe640 1 -- 172.21.15.46:0/2592313990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a4101a60 msgr2=0x7f98a4102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a37fe640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a4101a60 0x7f98a4102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a37fe640 1 -- 172.21.15.46:0/2592313990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a4103380 con 0x7f98a4100860 2024-09-10T03:58:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a2ffd640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a4101a60 0x7f98a4102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T03:58:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a37fe640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4104780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9894006010 tx=0x7f98940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a0ff9640 1 -- 172.21.15.46:0/2592313990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f989403f070 con 0x7f98a4100860 2024-09-10T03:58:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.659+0000 7f98a0ff9640 1 -- 172.21.15.46:0/2592313990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1084+0+0 (secure 0 0 0) 0x7f9894004b80 con 0x7f98a4100860 2024-09-10T03:58:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.660+0000 7f98a0ff9640 1 -- 172.21.15.46:0/2592313990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9894035030 con 0x7f98a4100860 2024-09-10T03:58:19.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.660+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a4103610 con 0x7f98a4100860 2024-09-10T03:58:19.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.660+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a4103bc0 con 0x7f98a4100860 2024-09-10T03:58:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.661+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4100ce0 con 0x7f98a4100860 2024-09-10T03:58:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.661+0000 7f98a0ff9640 1 -- 172.21.15.46:0/2592313990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f9894042070 con 0x7f98a4100860 2024-09-10T03:58:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.662+0000 7f98a0ff9640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f988c06de00 0x7f988c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.662+0000 7f98a0ff9640 1 -- 172.21.15.46:0/2592313990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f98940ad520 con 0x7f98a4100860 2024-09-10T03:58:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.662+0000 7f98a2ffd640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f988c06de00 0x7f988c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.662+0000 7f98a2ffd640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f988c06de00 0x7f988c0702c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9898006fd0 tx=0x7f9898008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:19.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.666+0000 7f98a0ff9640 1 -- 172.21.15.46:0/2592313990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f989407c220 con 0x7f98a4100860 2024-09-10T03:58:19.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.842+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f98a419fc30 con 0x7f98a4100860 2024-09-10T03:58:19.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.845+0000 7f98a0ff9640 1 -- 172.21.15.46:0/2592313990 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v24)=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7f989407fed0 con 0x7f98a4100860 2024-09-10T03:58:19.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f988c06de00 msgr2=0x7f988c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f988c06de00 0x7f988c0702c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9898006fd0 tx=0x7f9898008040 comp rx=0 tx=0).stop 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 msgr2=0x7f98a4104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4104780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9894006010 tx=0x7f98940046c0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 shutdown_connections 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f988c06de00 0x7f988c0702c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98a4101a60 0x7f98a4102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 --2- 172.21.15.46:0/2592313990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98a4100860 0x7f98a4104780 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.847+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 >> 172.21.15.46:0/2592313990 conn(0x7f98a40fc010 msgr2=0x7f98a406a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.848+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 shutdown_connections 2024-09-10T03:58:19.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:19.848+0000 7f98a9bf1640 1 -- 172.21.15.46:0/2592313990 wait complete. 2024-09-10T03:58:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.832+0000 7fa8cb470640 1 -- 172.21.15.46:0/1160240064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4101a80 msgr2=0x7fa8c4101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.832+0000 7fa8cb470640 1 --2- 172.21.15.46:0/1160240064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4101a80 0x7fa8c4101f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009880 tx=0x7fa8b402a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 -- 172.21.15.46:0/1160240064 shutdown_connections 2024-09-10T03:58:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 --2- 172.21.15.46:0/1160240064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4101a80 0x7fa8c4101f00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 --2- 172.21.15.46:0/1160240064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c4100880 0x7fa8c4100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 -- 172.21.15.46:0/1160240064 >> 172.21.15.46:0/1160240064 conn(0x7fa8c40fc010 msgr2=0x7fa8c40fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 -- 172.21.15.46:0/1160240064 shutdown_connections 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 -- 172.21.15.46:0/1160240064 wait complete. 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 Processor -- start 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 -- start start 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4100880 0x7fa8c4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c4101a80 0x7fa8c4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41095f0 con 0x7fa8c4100880 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.833+0000 7fa8cb470640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4109470 con 0x7fa8c4101a80 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c920c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4100880 0x7fa8c4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c920c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4100880 0x7fa8c4197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58100/0 (socket says 172.21.15.46:58100) 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c920c640 1 -- 172.21.15.46:0/199863553 learned_addr learned my addr 172.21.15.46:0/199863553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c8a0b640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c4101a80 0x7fa8c4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:19.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c920c640 1 -- 172.21.15.46:0/199863553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c4101a80 msgr2=0x7fa8c4197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c920c640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c4101a80 0x7fa8c4197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c920c640 1 -- 172.21.15.46:0/199863553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c4197fd0 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8c920c640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4100880 0x7fa8c4197410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000ea20 tx=0x7fa8c000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8ba7fc640 1 -- 172.21.15.46:0/199863553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa8c0016070 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8ba7fc640 1 -- 172.21.15.46:0/199863553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1084+0+0 (secure 0 0 0) 0x7fa8c0003e10 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c419ca10 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8ba7fc640 1 -- 172.21.15.46:0/199863553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa8c0011470 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.834+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c419d020 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.835+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c4100c80 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.836+0000 7fa8ba7fc640 1 -- 172.21.15.46:0/199863553 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7fa8c0019070 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.836+0000 7fa8ba7fc640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa8b006ddb0 0x7fa8b0070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.836+0000 7fa8ba7fc640 1 -- 172.21.15.46:0/199863553 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa8c008ce20 con 0x7fa8c4100880 2024-09-10T03:58:19.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.836+0000 7fa8c8a0b640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa8b006ddb0 0x7fa8b0070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.837+0000 7fa8c8a0b640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa8b006ddb0 0x7fa8b0070270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8c41988c0 tx=0x7fa8b4004eb0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:17.841+0000 7fa8ba7fc640 1 -- 172.21.15.46:0/199863553 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8c005ba00 con 0x7fa8c4100880 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.021+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fa8c4106830 con 0x7fa8c4100880 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.025+0000 7fa8ba7fc640 1 -- 172.21.15.46:0/199863553 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7fa8c005f6b0 con 0x7fa8c4100880 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.026+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa8b006ddb0 msgr2=0x7fa8b0070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.026+0000 7fa8cb470640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa8b006ddb0 0x7fa8b0070270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa8c41988c0 tx=0x7fa8b4004eb0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4100880 msgr2=0x7fa8c4197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4100880 0x7fa8c4197410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000ea20 tx=0x7fa8c000eee0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 shutdown_connections 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa8b006ddb0 0x7fa8b0070270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c4101a80 0x7fa8c4197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 --2- 172.21.15.46:0/199863553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8c4100880 0x7fa8c4197410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 >> 172.21.15.46:0/199863553 conn(0x7fa8c40fc010 msgr2=0x7fa8c40fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:19.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 shutdown_connections 2024-09-10T03:58:19.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:18.027+0000 7fa8cb470640 1 -- 172.21.15.46:0/199863553 wait complete. 2024-09-10T03:58:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:19 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2329662807' entity='client.admin' 2024-09-10T03:58:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:19 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2329662807' entity='client.admin' 2024-09-10T03:58:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.236+0000 7f79d6b9a640 1 -- 172.21.15.46:0/299546421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 msgr2=0x7f79d0101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.236+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/299546421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0101ee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009880 tx=0x7f79c002a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 -- 172.21.15.46:0/299546421 shutdown_connections 2024-09-10T03:58:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/299546421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0101ee0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/299546421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d0100860 0x7f79d0100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 -- 172.21.15.46:0/299546421 >> 172.21.15.46:0/299546421 conn(0x7f79d00fc010 msgr2=0x7f79d00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 -- 172.21.15.46:0/299546421 shutdown_connections 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 -- 172.21.15.46:0/299546421 wait complete. 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 Processor -- start 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 -- start start 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d0100860 0x7f79d0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f79d0109650 con 0x7f79d0101a60 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.237+0000 7f79d6b9a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f79d01094d0 con 0x7f79d0100860 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39144/0 (socket says 172.21.15.46:39144) 2024-09-10T03:58:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c7fff640 1 -- 172.21.15.46:0/373764067 learned_addr learned my addr 172.21.15.46:0/373764067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79d4936640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d0100860 0x7f79d0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c7fff640 1 -- 172.21.15.46:0/373764067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d0100860 msgr2=0x7f79d0196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c7fff640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d0100860 0x7f79d0196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c7fff640 1 -- 172.21.15.46:0/373764067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d0196fc0 con 0x7f79d0101a60 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c7fff640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0196940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f79c0004f10 tx=0x7f79c00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c5ffb640 1 -- 172.21.15.46:0/373764067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f79c003f070 con 0x7f79d0101a60 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c5ffb640 1 -- 172.21.15.46:0/373764067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1084+0+0 (secure 0 0 0) 0x7f79c0004480 con 0x7f79d0101a60 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d019b9a0 con 0x7f79d0101a60 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.238+0000 7f79c5ffb640 1 -- 172.21.15.46:0/373764067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f79c0035030 con 0x7f79d0101a60 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.239+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d019bfb0 con 0x7f79d0101a60 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.239+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d0100ce0 con 0x7f79d0101a60 2024-09-10T03:58:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.240+0000 7f79c5ffb640 1 -- 172.21.15.46:0/373764067 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f79c0042070 con 0x7f79d0101a60 2024-09-10T03:58:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.240+0000 7f79c5ffb640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f79b806de00 0x7f79b80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.240+0000 7f79c5ffb640 1 -- 172.21.15.46:0/373764067 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f79c00ad570 con 0x7f79d0101a60 2024-09-10T03:58:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.241+0000 7f79d4936640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f79b806de00 0x7f79b80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.241+0000 7f79d4936640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f79b806de00 0x7f79b80702c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f79d01018c0 tx=0x7f79cc009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.245+0000 7f79c5ffb640 1 -- 172.21.15.46:0/373764067 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79c007c270 con 0x7f79d0101a60 2024-09-10T03:58:20.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.422+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f79d019c210 con 0x7f79d0101a60 2024-09-10T03:58:20.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.425+0000 7f79c5ffb640 1 -- 172.21.15.46:0/373764067 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v25)=0 v25) ==== 119+0+0 (secure 0 0 0) 0x7f79c007ff20 con 0x7f79d0101a60 2024-09-10T03:58:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f79b806de00 msgr2=0x7f79b80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f79b806de00 0x7f79b80702c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f79d01018c0 tx=0x7f79cc009110 comp rx=0 tx=0).stop 2024-09-10T03:58:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 msgr2=0x7f79d0196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0196940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f79c0004f10 tx=0x7f79c00048f0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 shutdown_connections 2024-09-10T03:58:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f79d0101a60 0x7f79d0196940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f79b806de00 0x7f79b80702c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 --2- 172.21.15.46:0/373764067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d0100860 0x7f79d0196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 >> 172.21.15.46:0/373764067 conn(0x7f79d00fc010 msgr2=0x7f79d00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 shutdown_connections 2024-09-10T03:58:20.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:20.427+0000 7f79d6b9a640 1 -- 172.21.15.46:0/373764067 wait complete. 2024-09-10T03:58:20.948 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:20 smithi046 ceph-mon[28328]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:20.948 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:20 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2592313990' entity='client.admin' 2024-09-10T03:58:20.949 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:20 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/373764067' entity='client.admin' 2024-09-10T03:58:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:20 smithi070 ceph-mon[35495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:20 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2592313990' entity='client.admin' 2024-09-10T03:58:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:20 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/373764067' entity='client.admin' 2024-09-10T03:58:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.461+0000 7fe3e4140640 1 -- 172.21.15.46:0/1505279325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 msgr2=0x7fe3dc100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.461+0000 7fe3e4140640 1 --2- 172.21.15.46:0/1505279325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc100c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009880 tx=0x7fe3d802a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 -- 172.21.15.46:0/1505279325 shutdown_connections 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 --2- 172.21.15.46:0/1505279325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3dc101a80 0x7fe3dc101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 --2- 172.21.15.46:0/1505279325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc100c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 -- 172.21.15.46:0/1505279325 >> 172.21.15.46:0/1505279325 conn(0x7fe3dc0fc010 msgr2=0x7fe3dc0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 -- 172.21.15.46:0/1505279325 shutdown_connections 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 -- 172.21.15.46:0/1505279325 wait complete. 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 Processor -- start 2024-09-10T03:58:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.462+0000 7fe3e4140640 1 -- start start 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e4140640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e4140640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3dc101a80 0x7fe3dc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e4140640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc109650 con 0x7fe3dc100880 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e4140640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1094d0 con 0x7fe3dc101a80 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e1edc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e1edc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39154/0 (socket says 172.21.15.46:39154) 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e1edc640 1 -- 172.21.15.46:0/2347910815 learned_addr learned my addr 172.21.15.46:0/2347910815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e16db640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3dc101a80 0x7fe3dc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e1edc640 1 -- 172.21.15.46:0/2347910815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3dc101a80 msgr2=0x7fe3dc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e1edc640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3dc101a80 0x7fe3dc197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e1edc640 1 -- 172.21.15.46:0/2347910815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc198120 con 0x7fe3dc100880 2024-09-10T03:58:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e1edc640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc197560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8004f10 tx=0x7fe3d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3d803f070 con 0x7fe3dc100880 2024-09-10T03:58:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc19cb00 con 0x7fe3dc100880 2024-09-10T03:58:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1084+0+0 (secure 0 0 0) 0x7fe3d8003a90 con 0x7fe3dc100880 2024-09-10T03:58:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.463+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3d8035030 con 0x7fe3dc100880 2024-09-10T03:58:21.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.464+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc19d090 con 0x7fe3dc100880 2024-09-10T03:58:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.465+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3dc100d00 con 0x7fe3dc100880 2024-09-10T03:58:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.466+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7fe3d8042070 con 0x7fe3dc100880 2024-09-10T03:58:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.466+0000 7fe3ceffd640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe3c806ddb0 0x7fe3c8070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:21.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.466+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe3d80ad670 con 0x7fe3dc100880 2024-09-10T03:58:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.467+0000 7fe3e16db640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe3c806ddb0 0x7fe3c8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.467+0000 7fe3e16db640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe3c806ddb0 0x7fe3c8070270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc198a10 tx=0x7fe3d0008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:21.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.471+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3d807c2f0 con 0x7fe3dc100880 2024-09-10T03:58:21.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.650+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7fe3dc19d3b0 con 0x7fe3dc100880 2024-09-10T03:58:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.653+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v26)=0 v26) ==== 114+0+0 (secure 0 0 0) 0x7fe3d807ffa0 con 0x7fe3dc100880 2024-09-10T03:58:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.654+0000 7fe3ceffd640 1 -- 172.21.15.46:0/2347910815 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3d803b460 con 0x7fe3dc100880 2024-09-10T03:58:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.655+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe3c806ddb0 msgr2=0x7fe3c8070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.655+0000 7fe3e4140640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe3c806ddb0 0x7fe3c8070270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc198a10 tx=0x7fe3d0008040 comp rx=0 tx=0).stop 2024-09-10T03:58:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.655+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 msgr2=0x7fe3dc197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.655+0000 7fe3e4140640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc197560 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8004f10 tx=0x7fe3d80048f0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.656+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 shutdown_connections 2024-09-10T03:58:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.656+0000 7fe3e4140640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe3c806ddb0 0x7fe3c8070270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.656+0000 7fe3e4140640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3dc101a80 0x7fe3dc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.656+0000 7fe3e4140640 1 --2- 172.21.15.46:0/2347910815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc100880 0x7fe3dc197560 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.656+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 >> 172.21.15.46:0/2347910815 conn(0x7fe3dc0fc010 msgr2=0x7fe3dc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.656+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 shutdown_connections 2024-09-10T03:58:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:21.656+0000 7fe3e4140640 1 -- 172.21.15.46:0/2347910815 wait complete. 2024-09-10T03:58:22.778 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:22 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2347910815' entity='client.admin' 2024-09-10T03:58:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:22 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2347910815' entity='client.admin' 2024-09-10T03:58:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:23 smithi046 ceph-mon[28328]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:23.978 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:23 smithi070 ceph-mon[35495]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 -- 172.21.15.46:0/2071632290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c102b60 msgr2=0x7f257c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 --2- 172.21.15.46:0/2071632290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c102b60 0x7f257c102fe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f256c009880 tx=0x7f256c02a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 -- 172.21.15.46:0/2071632290 shutdown_connections 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 --2- 172.21.15.46:0/2071632290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c102b60 0x7f257c102fe0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 --2- 172.21.15.46:0/2071632290 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c101960 0x7f257c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 -- 172.21.15.46:0/2071632290 >> 172.21.15.46:0/2071632290 conn(0x7f257c0fd0f0 msgr2=0x7f257c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 -- 172.21.15.46:0/2071632290 shutdown_connections 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.394+0000 7f2583cbd640 1 -- 172.21.15.46:0/2071632290 wait complete. 2024-09-10T03:58:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2583cbd640 1 Processor -- start 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2583cbd640 1 -- start start 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2583cbd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c101960 0x7f257c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2583cbd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c102b60 0x7f257c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2583cbd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f257c10a820 con 0x7f257c101960 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2583cbd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f257c10a6a0 con 0x7f257c102b60 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2581a59640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c101960 0x7f257c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2581a59640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c101960 0x7f257c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39168/0 (socket says 172.21.15.46:39168) 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2581a59640 1 -- 172.21.15.46:0/1429896103 learned_addr learned my addr 172.21.15.46:0/1429896103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2581258640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c102b60 0x7f257c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2581a59640 1 -- 172.21.15.46:0/1429896103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c102b60 msgr2=0x7f257c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2581a59640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c102b60 0x7f257c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.395+0000 7f2581a59640 1 -- 172.21.15.46:0/1429896103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c074b20 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.396+0000 7f2581a59640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c101960 0x7f257c075f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f257800b3d0 tx=0x7f257800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.396+0000 7f2572ffd640 1 -- 172.21.15.46:0/1429896103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2578015020 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.396+0000 7f2572ffd640 1 -- 172.21.15.46:0/1429896103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2578011af0 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.396+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c074e10 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.396+0000 7f2572ffd640 1 -- 172.21.15.46:0/1429896103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2578010da0 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.396+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c075420 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.397+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257c101d60 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.398+0000 7f2572ffd640 1 -- 172.21.15.46:0/1429896103 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f2578018020 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.398+0000 7f2572ffd640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f256806ddb0 0x7f2568070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.398+0000 7f2572ffd640 1 -- 172.21.15.46:0/1429896103 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f257808c5c0 con 0x7f257c101960 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.398+0000 7f2581258640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f256806ddb0 0x7f2568070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.399+0000 7f2581258640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f256806ddb0 0x7f2568070270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f256c006010 tx=0x7f256c000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:24.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.402+0000 7f2572ffd640 1 -- 172.21.15.46:0/1429896103 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f257801f020 con 0x7f257c101960 2024-09-10T03:58:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.581+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f257c191630 con 0x7f257c101960 2024-09-10T03:58:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.585+0000 7f2572ffd640 1 -- 172.21.15.46:0/1429896103 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v27)=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7f257805b240 con 0x7f257c101960 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f256806ddb0 msgr2=0x7f2568070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f256806ddb0 0x7f2568070270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f256c006010 tx=0x7f256c000bc0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c101960 msgr2=0x7f257c075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c101960 0x7f257c075f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f257800b3d0 tx=0x7f257800b890 comp rx=0 tx=0).stop 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 shutdown_connections 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f256806ddb0 0x7f2568070270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f257c102b60 0x7f257c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 --2- 172.21.15.46:0/1429896103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f257c101960 0x7f257c075f20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 >> 172.21.15.46:0/1429896103 conn(0x7f257c0fd0f0 msgr2=0x7f257c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:24.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 shutdown_connections 2024-09-10T03:58:24.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:24.587+0000 7f2583cbd640 1 -- 172.21.15.46:0/1429896103 wait complete. 2024-09-10T03:58:25.994 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:25 smithi046 ceph-mon[28328]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:25.994 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:25 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1429896103' entity='client.admin' 2024-09-10T03:58:26.023 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:25 smithi070 ceph-mon[35495]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:26.023 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:25 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1429896103' entity='client.admin' 2024-09-10T03:58:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 -- 172.21.15.46:0/3194958196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 msgr2=0x7f5e04100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 --2- 172.21.15.46:0/3194958196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04100c60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009880 tx=0x7f5e0002a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 -- 172.21.15.46:0/3194958196 shutdown_connections 2024-09-10T03:58:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 --2- 172.21.15.46:0/3194958196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e04101a60 0x7f5e04101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 --2- 172.21.15.46:0/3194958196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04100c60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 -- 172.21.15.46:0/3194958196 >> 172.21.15.46:0/3194958196 conn(0x7f5e040fc010 msgr2=0x7f5e040fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 -- 172.21.15.46:0/3194958196 shutdown_connections 2024-09-10T03:58:26.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.722+0000 7f5e0c576640 1 -- 172.21.15.46:0/3194958196 wait complete. 2024-09-10T03:58:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.723+0000 7f5e0c576640 1 Processor -- start 2024-09-10T03:58:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.723+0000 7f5e0c576640 1 -- start start 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.723+0000 7f5e0c576640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.723+0000 7f5e0c576640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e04101a60 0x7f5e04197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.723+0000 7f5e0c576640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e04109700 con 0x7f5e04100860 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.723+0000 7f5e0c576640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5e04109580 con 0x7f5e04101a60 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.723+0000 7f5e0a312640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0a312640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39190/0 (socket says 172.21.15.46:39190) 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0a312640 1 -- 172.21.15.46:0/1142866956 learned_addr learned my addr 172.21.15.46:0/1142866956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e09b11640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e04101a60 0x7f5e04197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0a312640 1 -- 172.21.15.46:0/1142866956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e04101a60 msgr2=0x7f5e04197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0a312640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e04101a60 0x7f5e04197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0a312640 1 -- 172.21.15.46:0/1142866956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e04198130 con 0x7f5e04100860 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0a312640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04197570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e00006010 tx=0x7f5e000048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:26.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5dfb7fe640 1 -- 172.21.15.46:0/1142866956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e0003f070 con 0x7f5e04100860 2024-09-10T03:58:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5dfb7fe640 1 -- 172.21.15.46:0/1142866956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e000044d0 con 0x7f5e04100860 2024-09-10T03:58:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0419cb10 con 0x7f5e04100860 2024-09-10T03:58:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5dfb7fe640 1 -- 172.21.15.46:0/1142866956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e00035030 con 0x7f5e04100860 2024-09-10T03:58:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.724+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0419d0a0 con 0x7f5e04100860 2024-09-10T03:58:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.725+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e04100c60 con 0x7f5e04100860 2024-09-10T03:58:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.726+0000 7f5dfb7fe640 1 -- 172.21.15.46:0/1142866956 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f5e00042070 con 0x7f5e04100860 2024-09-10T03:58:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.726+0000 7f5dfb7fe640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5df006de00 0x7f5df00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.726+0000 7f5dfb7fe640 1 -- 172.21.15.46:0/1142866956 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5e000ad570 con 0x7f5e04100860 2024-09-10T03:58:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.727+0000 7f5e09b11640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5df006de00 0x7f5df00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.727+0000 7f5e09b11640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5df006de00 0x7f5df00702c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e04198a20 tx=0x7f5df4009110 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.731+0000 7f5dfb7fe640 1 -- 172.21.15.46:0/1142866956 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e0007c270 con 0x7f5e04100860 2024-09-10T03:58:26.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.911+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f5e0419d520 con 0x7f5e04100860 2024-09-10T03:58:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.914+0000 7f5dfb7fe640 1 -- 172.21.15.46:0/1142866956 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v28)=0 v28) ==== 108+0+0 (secure 0 0 0) 0x7f5e0007ff20 con 0x7f5e04100860 2024-09-10T03:58:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5df006de00 msgr2=0x7f5df00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5df006de00 0x7f5df00702c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5e04198a20 tx=0x7f5df4009110 comp rx=0 tx=0).stop 2024-09-10T03:58:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 msgr2=0x7f5e04197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04197570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e00006010 tx=0x7f5e000048f0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 shutdown_connections 2024-09-10T03:58:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5df006de00 0x7f5df00702c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e04101a60 0x7f5e04197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 --2- 172.21.15.46:0/1142866956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e04100860 0x7f5e04197570 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 >> 172.21.15.46:0/1142866956 conn(0x7f5e040fc010 msgr2=0x7f5e0406a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 shutdown_connections 2024-09-10T03:58:26.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:26.916+0000 7f5e0c576640 1 -- 172.21.15.46:0/1142866956 wait complete. 2024-09-10T03:58:27.128 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi046') 2024-09-10T03:58:27.129 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi046') 2024-09-10T03:58:27.129 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-10T03:58:27.129 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-10T03:58:27.129 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi046') 2024-09-10T03:58:27.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:27 smithi046 ceph-mon[28328]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:27.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:27 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1142866956' entity='client.admin' 2024-09-10T03:58:28.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:27 smithi070 ceph-mon[35495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:27 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1142866956' entity='client.admin' 2024-09-10T03:58:29.681 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.682 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.683 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.683 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.683 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.683 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:squid', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '628edc22-6f28-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi046') 2024-09-10T03:58:29.683 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T03:58:29.683 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T03:58:29.683 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:58:29.701 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T03:58:29.702 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 03:57:27.002509091 +0000 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 03:51:19.757996083 +0000 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 03:51:19.757996083 +0000 2024-09-10T03:58:29.759 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 03:51:19.757996083 +0000 2024-09-10T03:58:29.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T03:58:29.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:29 smithi046 ceph-mon[28328]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:29.796 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:29.796 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:29.796 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:29 smithi046 ceph-mon[28328]: Reconfiguring crash.smithi046 (monmap changed)... 2024-09-10T03:58:29.796 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi046", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T03:58:29.796 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:29.796 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:29 smithi046 ceph-mon[28328]: Reconfiguring daemon crash.smithi046 on smithi046 2024-09-10T03:58:29.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T03:58:29.823 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T03:58:29.823 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000115436 s, 4.4 MB/s 2024-09-10T03:58:29.824 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T03:58:29.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:29 smithi070 ceph-mon[35495]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:29 smithi070 ceph-mon[35495]: Reconfiguring crash.smithi046 (monmap changed)... 2024-09-10T03:58:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi046", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T03:58:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:29 smithi070 ceph-mon[35495]: Reconfiguring daemon crash.smithi046 on smithi046 2024-09-10T03:58:29.881 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-09-10T03:58:29.938 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 03:57:27.002509091 +0000 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 03:51:20.085992569 +0000 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 03:51:20.085992569 +0000 2024-09-10T03:58:29.939 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 03:51:20.085992569 +0000 2024-09-10T03:58:29.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T03:58:30.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T03:58:30.006 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T03:58:30.006 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000108512 s, 4.7 MB/s 2024-09-10T03:58:30.007 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T03:58:30.064 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-09-10T03:58:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T03:58:30.120 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:30.121 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T03:58:30.121 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:30.121 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:30.121 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 03:57:27.003509080 +0000 2024-09-10T03:58:30.121 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 03:51:20.416989024 +0000 2024-09-10T03:58:30.121 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 03:51:20.416989024 +0000 2024-09-10T03:58:30.121 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 03:51:20.416989024 +0000 2024-09-10T03:58:30.121 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T03:58:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T03:58:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T03:58:30.186 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000107446 s, 4.8 MB/s 2024-09-10T03:58:30.186 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T03:58:30.243 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-09-10T03:58:30.300 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-09-10 03:57:27.003509080 +0000 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 03:51:20.752985425 +0000 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 03:51:20.752985425 +0000 2024-09-10T03:58:30.301 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-09-10 03:51:20.752985425 +0000 2024-09-10T03:58:30.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T03:58:30.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-09-10T03:58:30.364 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-09-10T03:58:30.364 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000102117 s, 5.0 MB/s 2024-09-10T03:58:30.365 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T03:58:30.422 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T03:58:30.422 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:58:30.443 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T03:58:30.444 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-09-10T03:58:30.502 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T03:58:30.502 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:30.502 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-10T03:58:30.503 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:30.503 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:30.503 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 03:57:57.218686777 +0000 2024-09-10T03:58:30.503 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 03:51:17.940195564 +0000 2024-09-10T03:58:30.503 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 03:51:17.940195564 +0000 2024-09-10T03:58:30.503 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 03:51:17.940195564 +0000 2024-09-10T03:58:30.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T03:58:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T03:58:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T03:58:30.571 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000103447 s, 4.9 MB/s 2024-09-10T03:58:30.572 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T03:58:30.631 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-09-10T03:58:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T03:58:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:30.690 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-10T03:58:30.690 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:30.690 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:30.690 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 03:57:57.219686748 +0000 2024-09-10T03:58:30.690 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 03:51:18.253186264 +0000 2024-09-10T03:58:30.690 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 03:51:18.253186264 +0000 2024-09-10T03:58:30.690 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 03:51:18.253186264 +0000 2024-09-10T03:58:30.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T03:58:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T03:58:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T03:58:30.758 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.00012733 s, 4.0 MB/s 2024-09-10T03:58:30.759 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T03:58:30.817 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-09-10T03:58:30.875 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 03:57:57.219686748 +0000 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 03:51:18.567176934 +0000 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 03:51:18.567176934 +0000 2024-09-10T03:58:30.876 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 03:51:18.567176934 +0000 2024-09-10T03:58:30.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T03:58:30.942 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T03:58:30.942 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T03:58:30.942 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000113084 s, 4.5 MB/s 2024-09-10T03:58:30.943 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T03:58:31.002 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-09-10T03:58:31.058 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T03:58:31.058 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:58:31.058 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T03:58:31.059 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:58:31.059 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:58:31.059 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-10 03:57:57.219686748 +0000 2024-09-10T03:58:31.059 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 03:51:18.889167366 +0000 2024-09-10T03:58:31.059 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 03:51:18.889167366 +0000 2024-09-10T03:58:31.059 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-10 03:51:18.889167366 +0000 2024-09-10T03:58:31.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T03:58:31.123 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-10T03:58:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-10T03:58:31.124 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000120999 s, 4.2 MB/s 2024-09-10T03:58:31.125 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T03:58:31.182 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-09-10T03:58:31.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T03:58:31.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:31 smithi046 ceph-mon[28328]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:31.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:31.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:31.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:31 smithi070 ceph-mon[35495]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:32.674 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:32 smithi046 ceph-mon[28328]: Reconfiguring alertmanager.smithi046 (dependencies changed)... 2024-09-10T03:58:32.674 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:32 smithi046 ceph-mon[28328]: Reconfiguring daemon alertmanager.smithi046 on smithi046 2024-09-10T03:58:32.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:32 smithi070 ceph-mon[35495]: Reconfiguring alertmanager.smithi046 (dependencies changed)... 2024-09-10T03:58:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:32 smithi070 ceph-mon[35495]: Reconfiguring daemon alertmanager.smithi046 on smithi046 2024-09-10T03:58:33.705 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:33 smithi046 ceph-mon[28328]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:33.757 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:33.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:33 smithi070 ceph-mon[35495]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:35.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:35 smithi070 ceph-mon[35495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:35.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:35 smithi046 ceph-mon[28328]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:37 smithi070 ceph-mon[35495]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:37 smithi046 ceph-mon[28328]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:38.181 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:58:38.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-09-10T03:58:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:38 smithi070 ceph-mon[35495]: Reconfiguring grafana.smithi046 (dependencies changed)... 2024-09-10T03:58:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:38 smithi070 ceph-mon[35495]: Reconfiguring daemon grafana.smithi046 on smithi046 2024-09-10T03:58:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:38 smithi046 ceph-mon[28328]: Reconfiguring grafana.smithi046 (dependencies changed)... 2024-09-10T03:58:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:38 smithi046 ceph-mon[28328]: Reconfiguring daemon grafana.smithi046 on smithi046 2024-09-10T03:58:38.895 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:58:39.779 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:39 smithi046 ceph-mon[28328]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:39.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:39 smithi070 ceph-mon[35495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.514+0000 7fac6154e640 1 -- 172.21.15.46:0/1888675651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c100860 msgr2=0x7fac5c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.514+0000 7fac6154e640 1 --2- 172.21.15.46:0/1888675651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c100860 0x7fac5c100c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009880 tx=0x7fac4c02a4a0 comp rx=0 tx=0).stop 2024-09-10T03:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 -- 172.21.15.46:0/1888675651 shutdown_connections 2024-09-10T03:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 --2- 172.21.15.46:0/1888675651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5c101a60 0x7fac5c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 --2- 172.21.15.46:0/1888675651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c100860 0x7fac5c100c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 -- 172.21.15.46:0/1888675651 >> 172.21.15.46:0/1888675651 conn(0x7fac5c0fc010 msgr2=0x7fac5c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:58:40.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 -- 172.21.15.46:0/1888675651 shutdown_connections 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 -- 172.21.15.46:0/1888675651 wait complete. 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 Processor -- start 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.515+0000 7fac6154e640 1 -- start start 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac6154e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5c100860 0x7fac5c197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac6154e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c101a60 0x7fac5c197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac6154e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c109700 con 0x7fac5c101a60 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac6154e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c109580 con 0x7fac5c100860 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c101a60 0x7fac5c197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c101a60 0x7fac5c197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43308/0 (socket says 172.21.15.46:43308) 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5a7fc640 1 -- 172.21.15.46:0/3235252567 learned_addr learned my addr 172.21.15.46:0/3235252567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5affd640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5c100860 0x7fac5c197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5a7fc640 1 -- 172.21.15.46:0/3235252567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5c100860 msgr2=0x7fac5c197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:58:40.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5a7fc640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5c100860 0x7fac5c197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5a7fc640 1 -- 172.21.15.46:0/3235252567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5c198130 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.516+0000 7fac5a7fc640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c101a60 0x7fac5c197ab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fac5000b420 tx=0x7fac5000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.518+0000 7fac43fff640 1 -- 172.21.15.46:0/3235252567 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac50011630 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.518+0000 7fac43fff640 1 -- 172.21.15.46:0/3235252567 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac50016070 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.518+0000 7fac43fff640 1 -- 172.21.15.46:0/3235252567 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac50010d40 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.518+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5c19cb70 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.518+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5c19d180 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.519+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5c100c60 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.519+0000 7fac43fff640 1 -- 172.21.15.46:0/3235252567 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7fac5001b460 con 0x7fac5c101a60 2024-09-10T03:58:40.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.520+0000 7fac43fff640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac4406ddc0 0x7fac44070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.520+0000 7fac43fff640 1 -- 172.21.15.46:0/3235252567 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fac5008c2b0 con 0x7fac5c101a60 2024-09-10T03:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.520+0000 7fac5affd640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac4406ddc0 0x7fac44070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.520+0000 7fac5affd640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac4406ddc0 0x7fac44070280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac4c006010 tx=0x7fac4c005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:58:40.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.524+0000 7fac43fff640 1 -- 172.21.15.46:0/3235252567 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac5005af30 con 0x7fac5c101a60 2024-09-10T03:58:40.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:58:40.712+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fac5c19d470 con 0x7fac4406ddc0 2024-09-10T03:58:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:41 smithi046 ceph-mon[28328]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:41 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T03:58:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:41 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T03:58:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:41 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:41 smithi070 ceph-mon[35495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:41 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T03:58:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:41 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T03:58:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:41 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:42.779 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:42 smithi046 ceph-mon[28328]: from='client.14328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:42.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:42 smithi070 ceph-mon[35495]: from='client.14328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:43.779 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:43 smithi046 ceph-mon[28328]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:43.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:43 smithi070 ceph-mon[35495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:45.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:44 smithi046 ceph-mon[28328]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:45.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:44 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:45.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:44 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:45.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:44 smithi046 ceph-mon[28328]: Reconfiguring prometheus.smithi046 (dependencies changed)... 2024-09-10T03:58:45.021 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:44 smithi046 ceph-mon[28328]: Reconfiguring daemon prometheus.smithi046 on smithi046 2024-09-10T03:58:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:44 smithi070 ceph-mon[35495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:44 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:44 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:44 smithi070 ceph-mon[35495]: Reconfiguring prometheus.smithi046 (dependencies changed)... 2024-09-10T03:58:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:44 smithi070 ceph-mon[35495]: Reconfiguring daemon prometheus.smithi046 on smithi046 2024-09-10T03:58:46.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:45 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/363211929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2edfb38-e5a1-4d6b-993a-66efc78d66f2"}]: dispatch 2024-09-10T03:58:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:45 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/363211929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2edfb38-e5a1-4d6b-993a-66efc78d66f2"}]': finished 2024-09-10T03:58:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:45 smithi070 ceph-mon[35495]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T03:58:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:45 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:58:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:45 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2691341553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T03:58:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:45 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/363211929' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c2edfb38-e5a1-4d6b-993a-66efc78d66f2"}]: dispatch 2024-09-10T03:58:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:45 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/363211929' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c2edfb38-e5a1-4d6b-993a-66efc78d66f2"}]': finished 2024-09-10T03:58:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:45 smithi046 ceph-mon[28328]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T03:58:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:58:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:45 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2691341553' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T03:58:47.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:46 smithi070 ceph-mon[35495]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:46 smithi046 ceph-mon[28328]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:49.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:48 smithi070 ceph-mon[35495]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:48 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:48 smithi046 ceph-mon[28328]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:48 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:58:51.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:50 smithi046 ceph-mon[28328]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:50 smithi070 ceph-mon[35495]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:52 smithi070 ceph-mon[35495]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:52 smithi046 ceph-mon[28328]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:54 smithi070 ceph-mon[35495]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:54 smithi046 ceph-mon[28328]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:57.534 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:57 smithi046 ceph-mon[28328]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:57.535 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:57 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:57.535 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:57 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:57.535 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:57 smithi046 ceph-mon[28328]: Reconfiguring ceph-exporter.smithi070 (monmap changed)... 2024-09-10T03:58:57.535 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:57 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T03:58:57.535 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:57 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:57.535 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:57 smithi046 ceph-mon[28328]: Reconfiguring daemon ceph-exporter.smithi070 on smithi070 2024-09-10T03:58:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:57 smithi070 ceph-mon[35495]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:57 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:57 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:57 smithi070 ceph-mon[35495]: Reconfiguring ceph-exporter.smithi070 (monmap changed)... 2024-09-10T03:58:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:57 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T03:58:57.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:57 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:57.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:57 smithi070 ceph-mon[35495]: Reconfiguring daemon ceph-exporter.smithi070 on smithi070 2024-09-10T03:58:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T03:58:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: Deploying daemon osd.0 on smithi046 2024-09-10T03:58:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: Reconfiguring crash.smithi070 (monmap changed)... 2024-09-10T03:58:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T03:58:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:58 smithi070 ceph-mon[35495]: Reconfiguring daemon crash.smithi070 on smithi070 2024-09-10T03:58:58.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T03:58:58.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:58.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: Deploying daemon osd.0 on smithi046 2024-09-10T03:58:58.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:58.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:58.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: Reconfiguring crash.smithi070 (monmap changed)... 2024-09-10T03:58:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T03:58:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:58:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:58 smithi046 ceph-mon[28328]: Reconfiguring daemon crash.smithi070 on smithi070 2024-09-10T03:58:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:59 smithi046 ceph-mon[28328]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:58:59.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:59 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:59.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:59 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:58:59.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:59 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T03:58:59.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:59 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T03:58:59.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:58:59 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:00.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:59 smithi070 ceph-mon[35495]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:00.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:59 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:00.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:59 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:00.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:59 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T03:59:00.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:59 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T03:59:00.061 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:58:59 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:00.921 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:00 smithi070 ceph-mon[35495]: Reconfiguring mgr.smithi070.tezbzn (monmap changed)... 2024-09-10T03:59:00.922 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:00 smithi070 ceph-mon[35495]: Reconfiguring daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T03:59:00.922 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:00 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:00.922 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:00 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:00.922 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:00 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T03:59:00.922 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:00 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T03:59:00.922 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:00 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:00.931 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:00 smithi046 ceph-mon[28328]: Reconfiguring mgr.smithi070.tezbzn (monmap changed)... 2024-09-10T03:59:00.931 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:00 smithi046 ceph-mon[28328]: Reconfiguring daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T03:59:00.931 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:00 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:00.931 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:00 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:00.931 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:00 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T03:59:00.931 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:00 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T03:59:00.931 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:00 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:02.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:01 smithi070 ceph-mon[35495]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:01 smithi070 ceph-mon[35495]: Reconfiguring mon.smithi070 (monmap changed)... 2024-09-10T03:59:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:01 smithi070 ceph-mon[35495]: Reconfiguring daemon mon.smithi070 on smithi070 2024-09-10T03:59:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:01 smithi046 ceph-mon[28328]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:01 smithi046 ceph-mon[28328]: Reconfiguring mon.smithi070 (monmap changed)... 2024-09-10T03:59:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:01 smithi046 ceph-mon[28328]: Reconfiguring daemon mon.smithi070 on smithi070 2024-09-10T03:59:02.933 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi046.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi046.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T03:59:02.934 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:02.935 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:02 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:03.021 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.021 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.021 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T03:59:03.021 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi046.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi046.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.022 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:02 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi046.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi046.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T03:59:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi046.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi046.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:03.982 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:05.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:04 smithi046 ceph-mon[28328]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:05.009 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:04 smithi070 ceph-mon[35495]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:05 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:05 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:06.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:05 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:06.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:05 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:06 smithi070 ceph-mon[35495]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:06 smithi046 ceph-mon[28328]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:09.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.494+0000 7fac43fff640 1 -- 172.21.15.46:0/3235252567 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fac5c19d470 con 0x7fac4406ddc0 2024-09-10T03:59:09.497 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-09-10T03:59:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac4406ddc0 msgr2=0x7fac44070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac4406ddc0 0x7fac44070280 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fac4c006010 tx=0x7fac4c005a20 comp rx=0 tx=0).stop 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c101a60 msgr2=0x7fac5c197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c101a60 0x7fac5c197ab0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fac5000b420 tx=0x7fac5000b8e0 comp rx=0 tx=0).stop 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 shutdown_connections 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac5c101a60 0x7fac5c197ab0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac4406ddc0 0x7fac44070280 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 --2- 172.21.15.46:0/3235252567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac5c100860 0x7fac5c197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 >> 172.21.15.46:0/3235252567 conn(0x7fac5c0fc010 msgr2=0x7fac5c06a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 shutdown_connections 2024-09-10T03:59:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:09.497+0000 7fac6154e640 1 -- 172.21.15.46:0/3235252567 wait complete. 2024-09-10T03:59:09.750 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:09 smithi070 ceph-mon[35495]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:09.750 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:09.750 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:09.750 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:09.750 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:09.750 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:09.751 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:09.763 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:09 smithi046 ceph-mon[28328]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:09.763 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:09.763 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:09.763 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:09.763 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:09.763 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:09.763 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:10.199 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T03:59:10.201 INFO:tasks.cephadm:Deploying osd.1 on smithi046 with /dev/vg_nvme/lv_3... 2024-09-10T03:59:10.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T03:59:10.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:10.790 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:10.790 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:59:11.415 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 03:59:11 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[48974]: 2024-09-10T03:59:11.141+0000 7f68cfd5f640 -1 osd.0 0 log_to_monitors true 2024-09-10T03:59:11.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:11 smithi070 ceph-mon[35495]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:11 smithi070 ceph-mon[35495]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T03:59:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:11 smithi046 ceph-mon[28328]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:11 smithi046 ceph-mon[28328]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T03:59:13.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:12 smithi070 ceph-mon[35495]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:12 smithi070 ceph-mon[35495]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T03:59:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:12 smithi070 ceph-mon[35495]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T03:59:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:12 smithi070 ceph-mon[35495]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T03:59:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:12 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:12 smithi046 ceph-mon[28328]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:12 smithi046 ceph-mon[28328]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T03:59:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:12 smithi046 ceph-mon[28328]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T03:59:13.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:12 smithi046 ceph-mon[28328]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T03:59:13.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:12 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:13.759 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 03:59:13 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[48974]: 2024-09-10T03:59:13.536+0000 7f68cb4bf640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T03:59:13.759 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 03:59:13 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[48974]: 2024-09-10T03:59:13.540+0000 7f68c3af1640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:59:14.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:13 smithi046 ceph-mon[28328]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T03:59:14.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:13 smithi046 ceph-mon[28328]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T03:59:14.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:13 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:14.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:13 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:14.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:13 smithi046 ceph-mon[28328]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' 2024-09-10T03:59:14.021 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:59:14.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-09-10T03:59:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:13 smithi070 ceph-mon[35495]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T03:59:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:13 smithi070 ceph-mon[35495]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T03:59:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:13 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:13 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:13 smithi070 ceph-mon[35495]: from='osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356]' entity='osd.0' 2024-09-10T03:59:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:14 smithi070 ceph-mon[35495]: purged_snaps scrub starts 2024-09-10T03:59:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:14 smithi070 ceph-mon[35495]: purged_snaps scrub ok 2024-09-10T03:59:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:14 smithi070 ceph-mon[35495]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:14 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:15.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:14 smithi070 ceph-mon[35495]: osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] boot 2024-09-10T03:59:15.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:14 smithi070 ceph-mon[35495]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T03:59:15.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:14 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:14 smithi046 ceph-mon[28328]: purged_snaps scrub starts 2024-09-10T03:59:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:14 smithi046 ceph-mon[28328]: purged_snaps scrub ok 2024-09-10T03:59:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:14 smithi046 ceph-mon[28328]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:59:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:14 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:14 smithi046 ceph-mon[28328]: osd.0 [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] boot 2024-09-10T03:59:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:14 smithi046 ceph-mon[28328]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T03:59:15.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:14 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T03:59:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:59:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 -- 172.21.15.46:0/2270320894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 msgr2=0x7f2870101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:16.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 --2- 172.21.15.46:0/2270320894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f2870101d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2860009880 tx=0x7f286002a4a0 comp rx=0 tx=0).stop 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 -- 172.21.15.46:0/2270320894 shutdown_connections 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 --2- 172.21.15.46:0/2270320894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2870102b80 0x7f2870103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 --2- 172.21.15.46:0/2270320894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f2870101d80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 -- 172.21.15.46:0/2270320894 >> 172.21.15.46:0/2270320894 conn(0x7f28700fd110 msgr2=0x7f28700ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 -- 172.21.15.46:0/2270320894 shutdown_connections 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 -- 172.21.15.46:0/2270320894 wait complete. 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.297+0000 7f287585a640 1 Processor -- start 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f287585a640 1 -- start start 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f287585a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f28701974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f287585a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2870102b80 0x7f2870197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f287585a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f287010a7a0 con 0x7f2870101980 2024-09-10T03:59:16.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f287585a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f287010a620 con 0x7f2870102b80 2024-09-10T03:59:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f28701974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:59:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f28701974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55834/0 (socket says 172.21.15.46:55834) 2024-09-10T03:59:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286effd640 1 -- 172.21.15.46:0/184200915 learned_addr learned my addr 172.21.15.46:0/184200915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:59:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286e7fc640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2870102b80 0x7f2870197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:59:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286effd640 1 -- 172.21.15.46:0/184200915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2870102b80 msgr2=0x7f2870197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286effd640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2870102b80 0x7f2870197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:16.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286effd640 1 -- 172.21.15.46:0/184200915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2870198080 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f286effd640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f28701974c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f28600045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.298+0000 7f2874858640 1 -- 172.21.15.46:0/184200915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f286003f070 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.299+0000 7f2874858640 1 -- 172.21.15.46:0/184200915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2860004a50 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.299+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287019ca60 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.299+0000 7f2874858640 1 -- 172.21.15.46:0/184200915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2860035030 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.299+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287019cff0 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.300+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2870101d80 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.300+0000 7f2874858640 1 -- 172.21.15.46:0/184200915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f2860042070 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.301+0000 7f2874858640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f285806de00 0x7f28580702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.301+0000 7f2874858640 1 -- 172.21.15.46:0/184200915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7f28600ad680 con 0x7f2870101980 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.301+0000 7f286e7fc640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f285806de00 0x7f28580702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:59:16.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.301+0000 7f286e7fc640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f285806de00 0x7f28580702c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2870198970 tx=0x7f286400b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:59:16.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.305+0000 7f2874858640 1 -- 172.21.15.46:0/184200915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f286007fdd0 con 0x7f2870101980 2024-09-10T03:59:16.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:16.490+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2870107130 con 0x7f285806de00 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T03:59:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: from='client.14346 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T03:59:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T03:59:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:18.825 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:18 smithi046 ceph-mon[28328]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:18.825 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:18 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:18 smithi070 ceph-mon[35495]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:18 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:19.940 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:19 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/687997296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e2cbb84-8fbe-485f-8bc0-62da12e599c3"}]: dispatch 2024-09-10T03:59:19.941 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:19 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/687997296' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e2cbb84-8fbe-485f-8bc0-62da12e599c3"}]': finished 2024-09-10T03:59:19.941 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:19 smithi046 ceph-mon[28328]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T03:59:19.941 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:19 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/687997296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e2cbb84-8fbe-485f-8bc0-62da12e599c3"}]: dispatch 2024-09-10T03:59:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:19 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/687997296' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e2cbb84-8fbe-485f-8bc0-62da12e599c3"}]': finished 2024-09-10T03:59:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:19 smithi070 ceph-mon[35495]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T03:59:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:21.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:20 smithi070 ceph-mon[35495]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:20 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/914699701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T03:59:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:20 smithi046 ceph-mon[28328]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:20 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/914699701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T03:59:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:22 smithi070 ceph-mon[35495]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:22 smithi046 ceph-mon[28328]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:24 smithi070 ceph-mon[35495]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:24 smithi046 ceph-mon[28328]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:26 smithi070 ceph-mon[35495]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:26 smithi046 ceph-mon[28328]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:29.149 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: Detected new or changed devices on smithi046 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi046 to 11844M 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.150 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:29.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:29.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: Detected new or changed devices on smithi046 2024-09-10T03:59:29.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:59:29.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi046 to 11844M 2024-09-10T03:59:29.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.241 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:29.241 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:29.241 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:29.241 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:30.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T03:59:30.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:30.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:29 smithi046 ceph-mon[28328]: Deploying daemon osd.1 on smithi046 2024-09-10T03:59:30.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:30.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:30.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T03:59:30.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:30.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:29 smithi070 ceph-mon[35495]: Deploying daemon osd.1 on smithi046 2024-09-10T03:59:30.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:30.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:31.085 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:30 smithi046 ceph-mon[28328]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:31.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:30 smithi070 ceph-mon[35495]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:33.158 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:32 smithi046 ceph-mon[28328]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:33.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:32 smithi070 ceph-mon[35495]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:34.119 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:34.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:35.244 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:34 smithi046 ceph-mon[28328]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:35.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:34 smithi070 ceph-mon[35495]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:36.807 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:36 smithi046 ceph-mon[28328]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:36.807 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:36 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:36.807 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:36 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:36.807 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:36 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:36.807 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:36 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:36.807 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:36 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:36 smithi070 ceph-mon[35495]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:36 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:36 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:36 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:36 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:36 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:37.921 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:37.921 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:37.921 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:38.009 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:38.009 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:38.010 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:39.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:39 smithi070 ceph-mon[35495]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:39 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:39 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:39.404 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:39 smithi046 ceph-mon[28328]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:39.404 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:39.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:41 smithi070 ceph-mon[35495]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:41 smithi046 ceph-mon[28328]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:42.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.724+0000 7f2874858640 1 -- 172.21.15.46:0/184200915 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2870107130 con 0x7f285806de00 2024-09-10T03:59:42.726 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 1 on host 'smithi046' 2024-09-10T03:59:42.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f285806de00 msgr2=0x7f28580702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:42.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f285806de00 0x7f28580702c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2870198970 tx=0x7f286400b410 comp rx=0 tx=0).stop 2024-09-10T03:59:42.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 msgr2=0x7f28701974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:42.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f28701974c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2860006010 tx=0x7f28600045e0 comp rx=0 tx=0).stop 2024-09-10T03:59:42.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 shutdown_connections 2024-09-10T03:59:42.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f285806de00 0x7f28580702c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2870102b80 0x7f2870197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 --2- 172.21.15.46:0/184200915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2870101980 0x7f28701974c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.726+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 >> 172.21.15.46:0/184200915 conn(0x7f28700fd110 msgr2=0x7f28700ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:59:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.727+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 shutdown_connections 2024-09-10T03:59:42.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:42.727+0000 7f287585a640 1 -- 172.21.15.46:0/184200915 wait complete. 2024-09-10T03:59:43.467 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:43 smithi046 ceph-mon[28328]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:43.467 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:43 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:43.468 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:43 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:43.497 DEBUG:teuthology.orchestra.run.smithi046:osd.1> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T03:59:43.499 INFO:tasks.cephadm:Deploying osd.2 on smithi046 with /dev/vg_nvme/lv_2... 2024-09-10T03:59:43.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T03:59:43.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:43 smithi070 ceph-mon[35495]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:43 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:43 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:43.978 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:59:44.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:44 smithi070 ceph-mon[35495]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T03:59:44.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:44 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:44.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:44 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:44.431 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:44 smithi046 ceph-mon[28328]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T03:59:44.432 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:44 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:44.432 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:44 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:45.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:45 smithi046 ceph-mon[28328]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:45.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:45 smithi046 ceph-mon[28328]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T03:59:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:45 smithi046 ceph-mon[28328]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T03:59:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:45 smithi046 ceph-mon[28328]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T03:59:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:45 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:45.558 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:45 smithi070 ceph-mon[35495]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:45.558 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:45 smithi070 ceph-mon[35495]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T03:59:45.559 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:45 smithi070 ceph-mon[35495]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T03:59:45.559 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:45 smithi070 ceph-mon[35495]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T03:59:45.559 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:45 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:46.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 03:59:45 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[56628]: 2024-09-10T03:59:45.983+0000 7f81bf1ba640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T03:59:46.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 03:59:45 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[56628]: 2024-09-10T03:59:45.987+0000 7f81b77ec640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:59:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:46 smithi070 ceph-mon[35495]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T03:59:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:46 smithi070 ceph-mon[35495]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T03:59:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:46 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:46 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:46 smithi070 ceph-mon[35495]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' 2024-09-10T03:59:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:46 smithi046 ceph-mon[28328]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T03:59:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:46 smithi046 ceph-mon[28328]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T03:59:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:46 smithi046 ceph-mon[28328]: from='osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844]' entity='osd.1' 2024-09-10T03:59:46.944 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T03:59:46.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-09-10T03:59:47.482 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T03:59:47.572 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:47 smithi046 ceph-mon[28328]: purged_snaps scrub starts 2024-09-10T03:59:47.572 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:47 smithi046 ceph-mon[28328]: purged_snaps scrub ok 2024-09-10T03:59:47.572 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:47 smithi046 ceph-mon[28328]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:47.572 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:47.572 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:47 smithi046 ceph-mon[28328]: osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] boot 2024-09-10T03:59:47.572 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:47 smithi046 ceph-mon[28328]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T03:59:47.572 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:47.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:47 smithi070 ceph-mon[35495]: purged_snaps scrub starts 2024-09-10T03:59:47.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:47 smithi070 ceph-mon[35495]: purged_snaps scrub ok 2024-09-10T03:59:47.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:47 smithi070 ceph-mon[35495]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:59:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:47 smithi070 ceph-mon[35495]: osd.1 [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] boot 2024-09-10T03:59:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:47 smithi070 ceph-mon[35495]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T03:59:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T03:59:48.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 -- 172.21.15.46:0/2809689412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc102b60 msgr2=0x7fe9fc102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 --2- 172.21.15.46:0/2809689412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc102b60 0x7fe9fc102fe0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009880 tx=0x7fe9ec02a4a0 comp rx=0 tx=0).stop 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 -- 172.21.15.46:0/2809689412 shutdown_connections 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 --2- 172.21.15.46:0/2809689412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc102b60 0x7fe9fc102fe0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 --2- 172.21.15.46:0/2809689412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9fc101960 0x7fe9fc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 -- 172.21.15.46:0/2809689412 >> 172.21.15.46:0/2809689412 conn(0x7fe9fc0fd110 msgr2=0x7fe9fc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 -- 172.21.15.46:0/2809689412 shutdown_connections 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.418+0000 7fea03be3640 1 -- 172.21.15.46:0/2809689412 wait complete. 2024-09-10T03:59:48.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea03be3640 1 Processor -- start 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea03be3640 1 -- start start 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea03be3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc101960 0x7fe9fc1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea03be3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9fc102b60 0x7fe9fc1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea03be3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc10a7a0 con 0x7fe9fc101960 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea03be3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc10a620 con 0x7fe9fc102b60 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea0197f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc101960 0x7fe9fc1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea0197f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc101960 0x7fe9fc1974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48414/0 (socket says 172.21.15.46:48414) 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea0197f640 1 -- 172.21.15.46:0/1811958554 learned_addr learned my addr 172.21.15.46:0/1811958554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T03:59:48.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea0117e640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9fc102b60 0x7fe9fc1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea0197f640 1 -- 172.21.15.46:0/1811958554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9fc102b60 msgr2=0x7fe9fc1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea0197f640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9fc102b60 0x7fe9fc1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.419+0000 7fea0197f640 1 -- 172.21.15.46:0/1811958554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc198060 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.420+0000 7fea0197f640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc101960 0x7fe9fc1974a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d8d0 tx=0x7fe9f800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.420+0000 7fe9f2ffd640 1 -- 172.21.15.46:0/1811958554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe9f8017070 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.420+0000 7fe9f2ffd640 1 -- 172.21.15.46:0/1811958554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe9f800be10 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.420+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc19caa0 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.420+0000 7fe9f2ffd640 1 -- 172.21.15.46:0/1811958554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe9f800f3c0 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.420+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc19d000 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.421+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc10af80 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.421+0000 7fe9f2ffd640 1 -- 172.21.15.46:0/1811958554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7fe9f801a070 con 0x7fe9fc101960 2024-09-10T03:59:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.422+0000 7fe9f2ffd640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe9e806ddb0 0x7fe9e8070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T03:59:48.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.422+0000 7fe9f2ffd640 1 -- 172.21.15.46:0/1811958554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2414+0+0 (secure 0 0 0) 0x7fe9f808ca00 con 0x7fe9fc101960 2024-09-10T03:59:48.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.422+0000 7fea0117e640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe9e806ddb0 0x7fe9e8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T03:59:48.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.423+0000 7fea0117e640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe9e806ddb0 0x7fe9e8070270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000c00 tx=0x7fe9ec0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T03:59:48.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.426+0000 7fe9f2ffd640 1 -- 172.21.15.46:0/1811958554 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9f805ef20 con 0x7fe9fc101960 2024-09-10T03:59:48.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T03:59:48.625+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe9fc062630 con 0x7fe9e806ddb0 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:59:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:59:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: pgmap v69: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:49.388 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:49.389 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:59:49.389 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T03:59:49.389 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:49.389 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:49.389 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:49.389 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:50.327 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:50 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi046 to 5922M 2024-09-10T03:59:50.327 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:50 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:50.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:50 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi046 to 5922M 2024-09-10T03:59:50.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:50 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T03:59:51.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:51 smithi046 ceph-mon[28328]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:51.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:51 smithi070 ceph-mon[35495]: pgmap v71: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:52.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:52 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3317969811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "346d282b-5119-44f6-8a94-5cf2f7ece643"}]: dispatch 2024-09-10T03:59:52.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:52 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3317969811' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "346d282b-5119-44f6-8a94-5cf2f7ece643"}]': finished 2024-09-10T03:59:52.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:52 smithi046 ceph-mon[28328]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T03:59:52.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T03:59:52.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:52 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1027556017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T03:59:52.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:52 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3317969811' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "346d282b-5119-44f6-8a94-5cf2f7ece643"}]: dispatch 2024-09-10T03:59:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:52 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3317969811' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "346d282b-5119-44f6-8a94-5cf2f7ece643"}]': finished 2024-09-10T03:59:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:52 smithi070 ceph-mon[35495]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T03:59:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:52 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T03:59:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:52 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1027556017' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T03:59:53.791 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:53 smithi046 ceph-mon[28328]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:53.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:53 smithi070 ceph-mon[35495]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:55 smithi070 ceph-mon[35495]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:55 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:55 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:55 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:55 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:55 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:55 smithi046 ceph-mon[28328]: pgmap v74: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:55 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:55 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:55 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T03:59:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:55 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T03:59:55.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:55 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T03:59:57.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:57 smithi070 ceph-mon[35495]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:57 smithi046 ceph-mon[28328]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:59.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 03:59:59 smithi070 ceph-mon[35495]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:59:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 03:59:59 smithi046 ceph-mon[28328]: pgmap v76: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:00.743 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:00 smithi046 ceph-mon[28328]: overall HEALTH_OK 2024-09-10T04:00:00.743 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:00 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T04:00:00.743 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:00 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:00 smithi070 ceph-mon[35495]: overall HEALTH_OK 2024-09-10T04:00:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:00 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T04:00:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:00 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:01 smithi046 ceph-mon[28328]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:01 smithi046 ceph-mon[28328]: Deploying daemon osd.2 on smithi046 2024-09-10T04:00:01.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:01 smithi070 ceph-mon[35495]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:01 smithi070 ceph-mon[35495]: Deploying daemon osd.2 on smithi046 2024-09-10T04:00:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:03 smithi046 ceph-mon[28328]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:03.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:03 smithi070 ceph-mon[35495]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:04.766 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:04.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:05.666 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:05 smithi046 ceph-mon[28328]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:05.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:05 smithi070 ceph-mon[35495]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:07.699 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:07 smithi046 ceph-mon[28328]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:07.699 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:07 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:07.699 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:07 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:07.700 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:07 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:07.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:07 smithi070 ceph-mon[35495]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:07 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:07 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:07 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:09.741 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:09 smithi046 ceph-mon[28328]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:09.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:09 smithi070 ceph-mon[35495]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:10.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.173+0000 7fe9f2ffd640 1 -- 172.21.15.46:0/1811958554 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe9fc062630 con 0x7fe9e806ddb0 2024-09-10T04:00:10.176 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 2 on host 'smithi046' 2024-09-10T04:00:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe9e806ddb0 msgr2=0x7fe9e8070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:10.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe9e806ddb0 0x7fe9e8070270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000c00 tx=0x7fe9ec0058d0 comp rx=0 tx=0).stop 2024-09-10T04:00:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc101960 msgr2=0x7fe9fc1974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc101960 0x7fe9fc1974a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800d8d0 tx=0x7fe9f800dd90 comp rx=0 tx=0).stop 2024-09-10T04:00:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 shutdown_connections 2024-09-10T04:00:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe9e806ddb0 0x7fe9e8070270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9fc102b60 0x7fe9fc1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 --2- 172.21.15.46:0/1811958554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9fc101960 0x7fe9fc1974a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 >> 172.21.15.46:0/1811958554 conn(0x7fe9fc0fd110 msgr2=0x7fe9fc106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:00:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 shutdown_connections 2024-09-10T04:00:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:00:10.177+0000 7fea03be3640 1 -- 172.21.15.46:0/1811958554 wait complete. 2024-09-10T04:00:11.126 DEBUG:teuthology.orchestra.run.smithi046:osd.2> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T04:00:11.128 INFO:tasks.cephadm:Deploying osd.3 on smithi070 with /dev/vg_nvme/lv_4... 2024-09-10T04:00:11.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T04:00:11.383 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:00:11.503 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:11 smithi046 ceph-mon[28328]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:11.503 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:11.503 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:11.503 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:11 smithi070 ceph-mon[35495]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:11.504 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:11.504 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:12.891 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:12 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[64430]: 2024-09-10T04:00:12.608+0000 7fd87c1dd640 -1 osd.2 0 log_to_monitors true 2024-09-10T04:00:13.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:13 smithi070 ceph-mon[35495]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:13.434 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:13 smithi070 ceph-mon[35495]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T04:00:13.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:13 smithi046 ceph-mon[28328]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:13.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:13 smithi046 ceph-mon[28328]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T04:00:13.973 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T04:00:13.999 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2024-09-10T04:00:14.255 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:00:14.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:14 smithi070 ceph-mon[35495]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T04:00:14.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:14 smithi070 ceph-mon[35495]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T04:00:14.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:14 smithi070 ceph-mon[35495]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T04:00:14.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:14 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:14.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:14 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:14.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:14 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:14.454 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[64430]: 2024-09-10T04:00:14.179+0000 7fd878974640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T04:00:14.454 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[64430]: 2024-09-10T04:00:14.183+0000 7fd873f77640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:00:14.454 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-mon[28328]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T04:00:14.454 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-mon[28328]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T04:00:14.454 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-mon[28328]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-09-10T04:00:14.455 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:14.455 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:14.455 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:14 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 -- 172.21.15.70:0/2873062777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 msgr2=0x7f5b8c1071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 --2- 172.21.15.70:0/2873062777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c1071d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009880 tx=0x7f5b7c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:00:14.733 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 -- 172.21.15.70:0/2873062777 shutdown_connections 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 --2- 172.21.15.70:0/2873062777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c1071d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 --2- 172.21.15.70:0/2873062777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8c1024b0 0x7f5b8c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 -- 172.21.15.70:0/2873062777 >> 172.21.15.70:0/2873062777 conn(0x7f5b8c0fc010 msgr2=0x7f5b8c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 -- 172.21.15.70:0/2873062777 shutdown_connections 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 -- 172.21.15.70:0/2873062777 wait complete. 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 Processor -- start 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.730+0000 7f5b94239640 1 -- start start 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b94239640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8c1024b0 0x7f5b8c101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b94239640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b94239640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10a7a0 con 0x7f5b8c1024b0 2024-09-10T04:00:14.734 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b94239640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c10a620 con 0x7f5b8c104de0 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b917d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b917d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45470/0 (socket says 172.21.15.70:45470) 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b917d4640 1 -- 172.21.15.70:0/2077823726 learned_addr learned my addr 172.21.15.70:0/2077823726 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b91fd5640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8c1024b0 0x7f5b8c101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b917d4640 1 -- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8c1024b0 msgr2=0x7f5b8c101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b917d4640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8c1024b0 0x7f5b8c101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b917d4640 1 -- 172.21.15.70:0/2077823726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c1009a0 con 0x7f5b8c104de0 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b917d4640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c100460 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009230 tx=0x7f5b7c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b7c03f070 con 0x7f5b8c104de0 2024-09-10T04:00:14.735 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b7c0049d0 con 0x7f5b8c104de0 2024-09-10T04:00:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c100c30 con 0x7f5b8c104de0 2024-09-10T04:00:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.731+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b7c035030 con 0x7f5b8c104de0 2024-09-10T04:00:14.736 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.732+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c101160 con 0x7f5b8c104de0 2024-09-10T04:00:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.733+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b8c102890 con 0x7f5b8c104de0 2024-09-10T04:00:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.733+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 18) ==== 92147+0+0 (secure 0 0 0) 0x7f5b7c042070 con 0x7f5b8c104de0 2024-09-10T04:00:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.733+0000 7f5b82ffd640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5b7806ddc0 0x7f5b78070280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:00:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.733+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2566+0+0 (secure 0 0 0) 0x7f5b7c0ad9d0 con 0x7f5b8c104de0 2024-09-10T04:00:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.734+0000 7f5b91fd5640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5b7806ddc0 0x7f5b78070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:00:14.737 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.734+0000 7f5b91fd5640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5b7806ddc0 0x7f5b78070280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009bd0 tx=0x7f5b88005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:00:14.739 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.738+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b7c07fe50 con 0x7f5b8c104de0 2024-09-10T04:00:14.916 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:14.915+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5b8c105840 con 0x7f5b7806ddc0 2024-09-10T04:00:15.423 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:15.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T04:00:15.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T04:00:15.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:15.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:15.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T04:00:15.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T04:00:15.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:15 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:15.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:15.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: from='osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-09-10T04:00:15.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T04:00:15.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:15.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:15.441 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T04:00:15.441 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T04:00:15.441 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:15 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:16.413 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:16 smithi070 ceph-mon[35495]: purged_snaps scrub starts 2024-09-10T04:00:16.413 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:16 smithi070 ceph-mon[35495]: purged_snaps scrub ok 2024-09-10T04:00:16.413 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:16 smithi070 ceph-mon[35495]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:16.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:16 smithi070 ceph-mon[35495]: osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] boot 2024-09-10T04:00:16.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:16 smithi070 ceph-mon[35495]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T04:00:16.414 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:16 smithi046 ceph-mon[28328]: purged_snaps scrub starts 2024-09-10T04:00:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:16 smithi046 ceph-mon[28328]: purged_snaps scrub ok 2024-09-10T04:00:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:16 smithi046 ceph-mon[28328]: from='client.24171 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:16 smithi046 ceph-mon[28328]: osd.2 [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] boot 2024-09-10T04:00:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:16 smithi046 ceph-mon[28328]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T04:00:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:00:17.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:17.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T04:00:17.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/404042090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "503bec3a-eb26-42a1-973e-2c145bd7d10c"}]: dispatch 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "503bec3a-eb26-42a1-973e-2c145bd7d10c"}]: dispatch 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "503bec3a-eb26-42a1-973e-2c145bd7d10c"}]': finished 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T04:00:17.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:17 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/404042090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "503bec3a-eb26-42a1-973e-2c145bd7d10c"}]: dispatch 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "503bec3a-eb26-42a1-973e-2c145bd7d10c"}]: dispatch 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "503bec3a-eb26-42a1-973e-2c145bd7d10c"}]': finished 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T04:00:17.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:17 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:18.086 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:17 smithi046 sudo[68892]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T04:00:18.086 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:17 smithi046 sudo[68892]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:18.086 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:17 smithi046 sudo[68892]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:18.087 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:17 smithi046 sudo[68892]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:18.226 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:18.224+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f5b7c003cb0 con 0x7f5b8c104de0 2024-09-10T04:00:18.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:18 smithi070 ceph-mon[35495]: Detected new or changed devices on smithi046 2024-09-10T04:00:18.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:18 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi046 to 3948M 2024-09-10T04:00:18.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:18 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/749010583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T04:00:18.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:18 smithi070 ceph-mon[35495]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T04:00:18.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:18 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:18.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:18 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:18.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:18 smithi046 ceph-mon[28328]: Detected new or changed devices on smithi046 2024-09-10T04:00:18.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:18 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi046 to 3948M 2024-09-10T04:00:18.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:18 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/749010583' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T04:00:18.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:18 smithi046 ceph-mon[28328]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T04:00:18.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:18 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:18.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:18 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:18.587 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68895]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T04:00:18.587 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68895]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:18.587 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68895]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:18.587 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68895]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:19.086 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68898]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T04:00:19.086 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68898]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:19.086 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68898]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:19.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:18 smithi046 sudo[68898]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:19 smithi070 ceph-mon[35495]: pgmap v91: 1 pgs: 1 active+clean; 128 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:19 smithi070 ceph-mon[35495]: mgrmap e19: smithi046.zvckjg(active, since 2m), standbys: smithi070.tezbzn 2024-09-10T04:00:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:19.560 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T04:00:19.561 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:19.561 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68901]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:19.561 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68901]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:19.561 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:19 smithi046 ceph-mon[28328]: pgmap v91: 1 pgs: 1 active+clean; 128 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:19.562 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:19 smithi046 ceph-mon[28328]: mgrmap e19: smithi046.zvckjg(active, since 2m), standbys: smithi070.tezbzn 2024-09-10T04:00:19.562 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:19.836 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68904]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T04:00:19.836 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:19.836 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68904]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:19.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:19 smithi046 sudo[68904]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:20.337 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68907]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T04:00:20.337 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:20.337 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68907]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:20.337 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68907]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:20.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T04:00:20.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:20.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68910]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:20.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68910]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:21.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68914]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T04:00:21.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68914]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:21.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68914]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:21.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:20 smithi046 sudo[68914]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:21.428 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40399]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T04:00:21.428 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40399]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:21.428 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40399]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:21.428 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40399]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40590]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 sudo[40590]: pam_unix(sudo:session): session closed for user root 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 ceph-mon[35495]: pgmap v93: 1 pgs: 1 active+clean; 128 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:00:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:00:21.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:00:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:21 smithi046 ceph-mon[28328]: pgmap v93: 1 pgs: 1 active+clean; 128 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:21 smithi046 ceph-mon[28328]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T04:00:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:21 smithi046 ceph-mon[28328]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T04:00:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:00:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:00:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:00:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:00:22.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:22 smithi070 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T04:00:22.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:22 smithi070 ceph-mon[35495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T04:00:22.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:22 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:22.780 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:22 smithi046 ceph-mon[28328]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T04:00:22.780 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:22 smithi046 ceph-mon[28328]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T04:00:22.780 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:22 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:23.675 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:23 smithi070 ceph-mon[35495]: pgmap v94: 1 pgs: 1 active+clean; 128 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:23 smithi046 ceph-mon[28328]: pgmap v94: 1 pgs: 1 active+clean; 128 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:25.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:25 smithi070 ceph-mon[35495]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 14 KiB/s rd, 381 KiB/s wr, 36 op/s 2024-09-10T04:00:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:25 smithi046 ceph-mon[28328]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 14 KiB/s rd, 381 KiB/s wr, 36 op/s 2024-09-10T04:00:26.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:26 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T04:00:26.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:26 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:26.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:26 smithi070 ceph-mon[35495]: Deploying daemon osd.3 on smithi070 2024-09-10T04:00:26.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:26 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T04:00:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:26 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:26 smithi046 ceph-mon[28328]: Deploying daemon osd.3 on smithi070 2024-09-10T04:00:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:27 smithi070 ceph-mon[35495]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 286 KiB/s wr, 27 op/s 2024-09-10T04:00:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:27 smithi046 ceph-mon[28328]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 10 KiB/s rd, 286 KiB/s wr, 27 op/s 2024-09-10T04:00:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:29 smithi070 ceph-mon[35495]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 8.3 KiB/s rd, 229 KiB/s wr, 21 op/s 2024-09-10T04:00:29.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:29 smithi046 ceph-mon[28328]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 8.3 KiB/s rd, 229 KiB/s wr, 21 op/s 2024-09-10T04:00:31.765 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:31 smithi070 ceph-mon[35495]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 197 KiB/s wr, 18 op/s 2024-09-10T04:00:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:31 smithi046 ceph-mon[28328]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 197 KiB/s wr, 18 op/s 2024-09-10T04:00:32.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:32 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:32.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:32 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:32.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:32 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:32.827 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:32 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:32.827 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:32 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:32.828 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:32 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:33.531 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:33 smithi070 ceph-mon[35495]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 6.9 KiB/s rd, 190 KiB/s wr, 18 op/s 2024-09-10T04:00:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:33 smithi046 ceph-mon[28328]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 6.9 KiB/s rd, 190 KiB/s wr, 18 op/s 2024-09-10T04:00:34.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:35.726 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:35 smithi070 ceph-mon[35495]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 6.9 KiB/s rd, 190 KiB/s wr, 18 op/s 2024-09-10T04:00:35.766 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.765+0000 7f5b82ffd640 1 -- 172.21.15.70:0/2077823726 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b8c105840 con 0x7f5b7806ddc0 2024-09-10T04:00:35.767 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 3 on host 'smithi070' 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5b7806ddc0 msgr2=0x7f5b78070280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5b7806ddc0 0x7f5b78070280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009bd0 tx=0x7f5b88005f90 comp rx=0 tx=0).stop 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 msgr2=0x7f5b8c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c100460 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009230 tx=0x7f5b7c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 shutdown_connections 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5b7806ddc0 0x7f5b78070280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b8c104de0 0x7f5b8c100460 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:35.769 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 --2- 172.21.15.70:0/2077823726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b8c1024b0 0x7f5b8c101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:35.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.767+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 >> 172.21.15.70:0/2077823726 conn(0x7f5b8c0fc010 msgr2=0x7f5b8c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:00:35.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.768+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 shutdown_connections 2024-09-10T04:00:35.770 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:35.768+0000 7f5b94239640 1 -- 172.21.15.70:0/2077823726 wait complete. 2024-09-10T04:00:35.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:35 smithi046 ceph-mon[28328]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail; 6.9 KiB/s rd, 190 KiB/s wr, 18 op/s 2024-09-10T04:00:36.370 DEBUG:teuthology.orchestra.run.smithi070:osd.3> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T04:00:36.373 INFO:tasks.cephadm:Deploying osd.4 on smithi070 with /dev/vg_nvme/lv_3... 2024-09-10T04:00:36.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T04:00:36.742 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:00:37.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:36 smithi070 ceph-mon[35495]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:36 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:36 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:36 smithi046 ceph-mon[28328]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:37.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:36 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:37.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:36 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:38.459 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:00:38 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[41459]: 2024-09-10T04:00:38.338+0000 7f5c4e64b640 -1 osd.3 0 log_to_monitors true 2024-09-10T04:00:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:38 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:38.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:38 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:38.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:38 smithi070 ceph-mon[35495]: from='osd.3 [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T04:00:38.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:38 smithi070 ceph-mon[35495]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T04:00:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:38 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:38.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:38 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:38.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:38 smithi046 ceph-mon[28328]: from='osd.3 [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T04:00:38.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:38 smithi046 ceph-mon[28328]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T04:00:39.633 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[41459]: 2024-09-10T04:00:39.487+0000 7f5c49dab640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T04:00:39.634 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[41459]: 2024-09-10T04:00:39.495+0000 7f5c463e5640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:00:39.634 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-mon[35495]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:39.634 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-mon[35495]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T04:00:39.634 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-mon[35495]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T04:00:39.634 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-mon[35495]: from='osd.3 [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:00:39.634 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:39.634 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:39 smithi070 ceph-mon[35495]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:00:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:39 smithi046 ceph-mon[28328]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:39 smithi046 ceph-mon[28328]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T04:00:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:39 smithi046 ceph-mon[28328]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T04:00:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:39 smithi046 ceph-mon[28328]: from='osd.3 [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:00:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:39 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:39 smithi046 ceph-mon[28328]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:00:40.023 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T04:00:40.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2024-09-10T04:00:40.589 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:00:40.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:40 smithi070 ceph-mon[35495]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-10T04:00:40.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:40 smithi070 ceph-mon[35495]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T04:00:40.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:40.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:40 smithi046 ceph-mon[28328]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-10T04:00:40.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:40 smithi046 ceph-mon[28328]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T04:00:40.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:40.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:41.100 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 -- 172.21.15.70:0/28191681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63541024b0 msgr2=0x7f63541048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 --2- 172.21.15.70:0/28191681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63541024b0 0x7f63541048a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6344009880 tx=0x7f634402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 -- 172.21.15.70:0/28191681 shutdown_connections 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 --2- 172.21.15.70:0/28191681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6354104de0 0x7f63541071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 --2- 172.21.15.70:0/28191681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63541024b0 0x7f63541048a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 -- 172.21.15.70:0/28191681 >> 172.21.15.70:0/28191681 conn(0x7f63540fbff0 msgr2=0x7f63540fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 -- 172.21.15.70:0/28191681 shutdown_connections 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.079+0000 7f6358a0f640 1 -- 172.21.15.70:0/28191681 wait complete. 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6358a0f640 1 Processor -- start 2024-09-10T04:00:41.101 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6358a0f640 1 -- start start 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6358a0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63541024b0 0x7f6354197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6358a0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6354104de0 0x7f6354197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6358a0f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f635410a7d0 con 0x7f63541024b0 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6358a0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f635410a650 con 0x7f6354104de0 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6351d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6354104de0 0x7f6354197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6351d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6354104de0 0x7f6354197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:57034/0 (socket says 172.21.15.70:57034) 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6351d9b640 1 -- 172.21.15.70:0/1394084925 learned_addr learned my addr 172.21.15.70:0/1394084925 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f635259c640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63541024b0 0x7f6354197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6351d9b640 1 -- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63541024b0 msgr2=0x7f6354197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:00:41.102 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6351d9b640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63541024b0 0x7f6354197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6351d9b640 1 -- 172.21.15.70:0/1394084925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63541980f0 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.080+0000 7f6351d9b640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6354104de0 0x7f6354197a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f634800d6c0 tx=0x7f634800db80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.081+0000 7f63437fe640 1 -- 172.21.15.70:0/1394084925 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6348017070 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.081+0000 7f63437fe640 1 -- 172.21.15.70:0/1394084925 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63480054e0 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.081+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635419cb30 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.082+0000 7f63437fe640 1 -- 172.21.15.70:0/1394084925 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6348004b70 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.082+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635419d0c0 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.082+0000 7f63437fe640 1 -- 172.21.15.70:0/1394084925 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f6348009820 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.082+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6354102890 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.083+0000 7f63437fe640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f633c06dc30 0x7f633c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.083+0000 7f63437fe640 1 -- 172.21.15.70:0/1394084925 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3807+0+0 (secure 0 0 0) 0x7f6348091170 con 0x7f6354104de0 2024-09-10T04:00:41.103 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.083+0000 7f635259c640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f633c06dc30 0x7f633c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:00:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.083+0000 7f635259c640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f633c06dc30 0x7f633c0700f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6344000c00 tx=0x7f63440058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:00:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.088+0000 7f63437fe640 1 -- 172.21.15.70:0/1394084925 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f634805e440 con 0x7f6354104de0 2024-09-10T04:00:41.268 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:00:41.267+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6354105820 con 0x7f633c06dc30 2024-09-10T04:00:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: purged_snaps scrub starts 2024-09-10T04:00:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: purged_snaps scrub ok 2024-09-10T04:00:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: osd.3 [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] boot 2024-09-10T04:00:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T04:00:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:41.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T04:00:41.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T04:00:41.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T04:00:41.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:41 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:41.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: purged_snaps scrub starts 2024-09-10T04:00:41.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: purged_snaps scrub ok 2024-09-10T04:00:41.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-09-10T04:00:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: osd.3 [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] boot 2024-09-10T04:00:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T04:00:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:00:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T04:00:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T04:00:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T04:00:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:41 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:43.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='client.24195 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:43.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: Detected new or changed devices on smithi070 2024-09-10T04:00:43.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:43.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi070 to 14912M 2024-09-10T04:00:43.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:43.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:43.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:43.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:43.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:43.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:42 smithi070 ceph-mon[35495]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T04:00:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='client.24195 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: Detected new or changed devices on smithi070 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi070 to 14912M 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:43.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:43.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:00:43.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:43.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:42 smithi046 ceph-mon[28328]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T04:00:43.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:43 smithi070 ceph-mon[35495]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:43 smithi070 ceph-mon[35495]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T04:00:44.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:43 smithi046 ceph-mon[28328]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:44.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:43 smithi046 ceph-mon[28328]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T04:00:45.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:44 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3160504481' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba74b660-732c-4947-af9b-8f5d75e4da14"}]: dispatch 2024-09-10T04:00:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:44 smithi070 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba74b660-732c-4947-af9b-8f5d75e4da14"}]: dispatch 2024-09-10T04:00:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:44 smithi070 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba74b660-732c-4947-af9b-8f5d75e4da14"}]': finished 2024-09-10T04:00:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:44 smithi070 ceph-mon[35495]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T04:00:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:44 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:00:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:44 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/642874707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T04:00:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:44 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3160504481' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba74b660-732c-4947-af9b-8f5d75e4da14"}]: dispatch 2024-09-10T04:00:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:44 smithi046 ceph-mon[28328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba74b660-732c-4947-af9b-8f5d75e4da14"}]: dispatch 2024-09-10T04:00:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:44 smithi046 ceph-mon[28328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba74b660-732c-4947-af9b-8f5d75e4da14"}]': finished 2024-09-10T04:00:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:44 smithi046 ceph-mon[28328]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T04:00:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:44 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:00:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:44 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/642874707' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T04:00:45.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:45 smithi070 ceph-mon[35495]: pgmap v111: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T04:00:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:45 smithi046 ceph-mon[28328]: pgmap v111: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T04:00:48.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:47 smithi070 ceph-mon[35495]: pgmap v113: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-10T04:00:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:47 smithi046 ceph-mon[28328]: pgmap v113: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-10T04:00:48.980 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:48 smithi070 ceph-mon[35495]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:48 smithi046 ceph-mon[28328]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:50.058 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:50.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:00:51.016 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:50 smithi070 ceph-mon[35495]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T04:00:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:50 smithi046 ceph-mon[28328]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T04:00:52.984 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:52 smithi070 ceph-mon[35495]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:52.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:52 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T04:00:52.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:52 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:52.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:52 smithi070 ceph-mon[35495]: Deploying daemon osd.4 on smithi070 2024-09-10T04:00:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:52 smithi046 ceph-mon[28328]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:53.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T04:00:53.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:52 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:00:53.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:52 smithi046 ceph-mon[28328]: Deploying daemon osd.4 on smithi070 2024-09-10T04:00:54.989 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:54 smithi070 ceph-mon[35495]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:54 smithi046 ceph-mon[28328]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:56 smithi070 ceph-mon[35495]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:56 smithi046 ceph-mon[28328]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:59.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:58 smithi070 ceph-mon[35495]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:00:58 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:58 smithi046 ceph-mon[28328]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:00:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:00:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:00:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:00:58 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:00 smithi070 ceph-mon[35495]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:00 smithi046 ceph-mon[28328]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:02.338 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 4 on host 'smithi070' 2024-09-10T04:01:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.336+0000 7f63437fe640 1 -- 172.21.15.70:0/1394084925 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6354105820 con 0x7f633c06dc30 2024-09-10T04:01:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.338+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f633c06dc30 msgr2=0x7f633c0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.338+0000 7f6358a0f640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f633c06dc30 0x7f633c0700f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6344000c00 tx=0x7f63440058d0 comp rx=0 tx=0).stop 2024-09-10T04:01:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6354104de0 msgr2=0x7f6354197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6354104de0 0x7f6354197a70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f634800d6c0 tx=0x7f634800db80 comp rx=0 tx=0).stop 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 shutdown_connections 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f633c06dc30 0x7f633c0700f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6354104de0 0x7f6354197a70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 --2- 172.21.15.70:0/1394084925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63541024b0 0x7f6354197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 >> 172.21.15.70:0/1394084925 conn(0x7f63540fbff0 msgr2=0x7f63540fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 shutdown_connections 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:02.339+0000 7f6358a0f640 1 -- 172.21.15.70:0/1394084925 wait complete. 2024-09-10T04:01:02.957 DEBUG:teuthology.orchestra.run.smithi070:osd.4> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T04:01:02.959 INFO:tasks.cephadm:Deploying osd.5 on smithi070 with /dev/vg_nvme/lv_2... 2024-09-10T04:01:02.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T04:01:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:01:03.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:03 smithi070 ceph-mon[35495]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:03 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:03 smithi046 ceph-mon[28328]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:03 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:04.644 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:04.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:04.982 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:01:04 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[48052]: 2024-09-10T04:01:04.642+0000 7f74fc4c9640 -1 osd.4 0 log_to_monitors true 2024-09-10T04:01:05.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:05 smithi070 ceph-mon[35495]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:05 smithi070 ceph-mon[35495]: from='osd.4 [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T04:01:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:05 smithi070 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T04:01:05.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:05 smithi046 ceph-mon[28328]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:05.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:05 smithi046 ceph-mon[28328]: from='osd.4 [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T04:01:05.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:05 smithi046 ceph-mon[28328]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T04:01:06.547 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:01:06 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[48052]: 2024-09-10T04:01:06.350+0000 7f74f7c29640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T04:01:06.547 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:01:06 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[48052]: 2024-09-10T04:01:06.356+0000 7f74f4263640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:01:06.548 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:06 smithi070 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T04:01:06.548 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:06 smithi070 ceph-mon[35495]: from='osd.4 [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:06.548 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:06 smithi070 ceph-mon[35495]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T04:01:06.548 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:06 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:06.548 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:06 smithi070 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:06.552 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-10T04:01:06.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2024-09-10T04:01:06.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:06 smithi046 ceph-mon[28328]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T04:01:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:06 smithi046 ceph-mon[28328]: from='osd.4 [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:06 smithi046 ceph-mon[28328]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T04:01:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:06 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:06 smithi046 ceph-mon[28328]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:07.106 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:01:07.562 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:07 smithi070 ceph-mon[35495]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:07.562 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:07 smithi070 ceph-mon[35495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-10T04:01:07.563 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:07 smithi070 ceph-mon[35495]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T04:01:07.563 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:07 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:07.563 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:07 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 -- 172.21.15.70:0/3821527022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8101970 msgr2=0x7f66f8101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/3821527022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8101970 0x7f66f8101d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009880 tx=0x7f66e802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 -- 172.21.15.70:0/3821527022 shutdown_connections 2024-09-10T04:01:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/3821527022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f8102b70 0x7f66f8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/3821527022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8101970 0x7f66f8101d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 -- 172.21.15.70:0/3821527022 >> 172.21.15.70:0/3821527022 conn(0x7f66f80fd140 msgr2=0x7f66f80ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 -- 172.21.15.70:0/3821527022 shutdown_connections 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.646+0000 7f66fc9bf640 1 -- 172.21.15.70:0/3821527022 wait complete. 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.647+0000 7f66fc9bf640 1 Processor -- start 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.647+0000 7f66fc9bf640 1 -- start start 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.647+0000 7f66fc9bf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f8101970 0x7f66f81974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.647+0000 7f66fc9bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8102b70 0x7f66f81979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.647+0000 7f66fc9bf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66f810a7b0 con 0x7f66f8101970 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.647+0000 7f66fc9bf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66f810a630 con 0x7f66f8102b70 2024-09-10T04:01:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8102b70 0x7f66f81979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8102b70 0x7f66f81979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48528/0 (socket says 172.21.15.70:48528) 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66effff640 1 -- 172.21.15.70:0/2610020427 learned_addr learned my addr 172.21.15.70:0/2610020427 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66f659c640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f8101970 0x7f66f81974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66effff640 1 -- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f8101970 msgr2=0x7f66f81974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66effff640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f8101970 0x7f66f81974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66effff640 1 -- 172.21.15.70:0/2610020427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f8198060 con 0x7f66f8102b70 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66effff640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8102b70 0x7f66f81979e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e970 tx=0x7f66e000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66ef7fe640 1 -- 172.21.15.70:0/2610020427 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66e000cac0 con 0x7f66f8102b70 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66ef7fe640 1 -- 172.21.15.70:0/2610020427 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66e00054e0 con 0x7f66f8102b70 2024-09-10T04:01:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66ef7fe640 1 -- 172.21.15.70:0/2610020427 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66e0010430 con 0x7f66f8102b70 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.648+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f819caa0 con 0x7f66f8102b70 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.649+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f8074420 con 0x7f66f8102b70 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.650+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f8101d70 con 0x7f66f8102b70 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.650+0000 7f66ef7fe640 1 -- 172.21.15.70:0/2610020427 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f66e0004f00 con 0x7f66f8102b70 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.651+0000 7f66ef7fe640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f66e40722a0 0x7f66e4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.651+0000 7f66ef7fe640 1 -- 172.21.15.70:0/2610020427 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4206+0+0 (secure 0 0 0) 0x7f66e008dd20 con 0x7f66f8102b70 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.651+0000 7f66f659c640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f66e40722a0 0x7f66e4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.652+0000 7f66f659c640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f66e40722a0 0x7f66e4074760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66e8000c00 tx=0x7f66e8005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.658+0000 7f66ef7fe640 1 -- 172.21.15.70:0/2610020427 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66e005be30 con 0x7f66f8102b70 2024-09-10T04:01:07.729 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:07 smithi046 ceph-mon[28328]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-10T04:01:07.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:07 smithi046 ceph-mon[28328]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-10T04:01:07.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:07 smithi046 ceph-mon[28328]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T04:01:07.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:07 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:07.730 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:07 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:07.842 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:07.842+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f66f8062630 con 0x7f66e40722a0 2024-09-10T04:01:08.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: purged_snaps scrub starts 2024-09-10T04:01:08.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: purged_snaps scrub ok 2024-09-10T04:01:08.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: osd.4 [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] boot 2024-09-10T04:01:08.475 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T04:01:08.476 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:08.476 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T04:01:08.476 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T04:01:08.476 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:08.476 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:08.476 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:08.476 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:08.477 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:08.477 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:08.477 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:08.477 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:08.477 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:08 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: purged_snaps scrub starts 2024-09-10T04:01:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: purged_snaps scrub ok 2024-09-10T04:01:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: osd.4 [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] boot 2024-09-10T04:01:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T04:01:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:01:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:08.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:08.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:08 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:09.439 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:09.439 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: from='client.24213 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:09.439 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: Detected new or changed devices on smithi070 2024-09-10T04:01:09.439 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi070 to 7456M 2024-09-10T04:01:09.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T04:01:09.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:01:09.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:09.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:09.440 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:09 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: from='client.24213 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: Detected new or changed devices on smithi070 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi070 to 7456M 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:09 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:11 smithi070 ceph-mon[35495]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:11 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/3803209689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e092e99e-0c0d-404b-9124-25da794f9eed"}]: dispatch 2024-09-10T04:01:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:11 smithi070 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e092e99e-0c0d-404b-9124-25da794f9eed"}]: dispatch 2024-09-10T04:01:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:11 smithi070 ceph-mon[35495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e092e99e-0c0d-404b-9124-25da794f9eed"}]': finished 2024-09-10T04:01:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:11 smithi070 ceph-mon[35495]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T04:01:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:11.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:11 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/525862544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T04:01:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:11 smithi046 ceph-mon[28328]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:11 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/3803209689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e092e99e-0c0d-404b-9124-25da794f9eed"}]: dispatch 2024-09-10T04:01:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:11 smithi046 ceph-mon[28328]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e092e99e-0c0d-404b-9124-25da794f9eed"}]: dispatch 2024-09-10T04:01:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:11 smithi046 ceph-mon[28328]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e092e99e-0c0d-404b-9124-25da794f9eed"}]': finished 2024-09-10T04:01:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:11 smithi046 ceph-mon[28328]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T04:01:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:11 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/525862544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T04:01:13.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:13 smithi070 ceph-mon[35495]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:13 smithi046 ceph-mon[28328]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:15.548 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:15 smithi070 ceph-mon[35495]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:15 smithi046 ceph-mon[28328]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:17.627 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:17 smithi070 ceph-mon[35495]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:17 smithi046 ceph-mon[28328]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:19 smithi070 ceph-mon[35495]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T04:01:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:19.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:19 smithi046 ceph-mon[28328]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:19.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:19.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T04:01:19.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:20.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:20 smithi070 ceph-mon[35495]: Deploying daemon osd.5 on smithi070 2024-09-10T04:01:20.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:20 smithi046 ceph-mon[28328]: Deploying daemon osd.5 on smithi070 2024-09-10T04:01:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:21 smithi070 ceph-mon[35495]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:21 smithi046 ceph-mon[28328]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:23.620 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:23 smithi070 ceph-mon[35495]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:23 smithi046 ceph-mon[28328]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:25.553 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:25 smithi070 ceph-mon[35495]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:25.554 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:01:25.554 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:25.554 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:25 smithi046 ceph-mon[28328]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:01:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:27.674 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:27 smithi070 ceph-mon[35495]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:27 smithi046 ceph-mon[28328]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:29.309 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.307+0000 7f66ef7fe640 1 -- 172.21.15.70:0/2610020427 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f66f8062630 con 0x7f66e40722a0 2024-09-10T04:01:29.309 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 5 on host 'smithi070' 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f66e40722a0 msgr2=0x7f66e4074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f66e40722a0 0x7f66e4074760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66e8000c00 tx=0x7f66e8005a20 comp rx=0 tx=0).stop 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8102b70 msgr2=0x7f66f81979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8102b70 0x7f66f81979e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66e000e970 tx=0x7f66e000ed70 comp rx=0 tx=0).stop 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 shutdown_connections 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f66e40722a0 0x7f66e4074760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66f8102b70 0x7f66f81979e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 --2- 172.21.15.70:0/2610020427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66f8101970 0x7f66f81974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.309+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 >> 172.21.15.70:0/2610020427 conn(0x7f66f80fd140 msgr2=0x7f66f8106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:29.311 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.310+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 shutdown_connections 2024-09-10T04:01:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:29.310+0000 7f66fc9bf640 1 -- 172.21.15.70:0/2610020427 wait complete. 2024-09-10T04:01:29.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:29.978 DEBUG:teuthology.orchestra.run.smithi070:osd.5> sudo journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T04:01:29.980 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-09-10T04:01:29.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T04:01:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.727+0000 7f6ee2799640 1 -- 172.21.15.46:0/213656445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc101960 msgr2=0x7f6edc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.727+0000 7f6ee2799640 1 --2- 172.21.15.46:0/213656445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc101960 0x7f6edc101d60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009880 tx=0x7f6ecc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.727+0000 7f6ee2799640 1 -- 172.21.15.46:0/213656445 shutdown_connections 2024-09-10T04:01:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.727+0000 7f6ee2799640 1 --2- 172.21.15.46:0/213656445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6edc102b60 0x7f6edc102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.727+0000 7f6ee2799640 1 --2- 172.21.15.46:0/213656445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc101960 0x7f6edc101d60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.727+0000 7f6ee2799640 1 -- 172.21.15.46:0/213656445 >> 172.21.15.46:0/213656445 conn(0x7f6edc0fd0f0 msgr2=0x7f6edc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 -- 172.21.15.46:0/213656445 shutdown_connections 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 -- 172.21.15.46:0/213656445 wait complete. 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 Processor -- start 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 -- start start 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6edc101960 0x7f6edc197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc102b60 0x7f6edc197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc10a800 con 0x7f6edc102b60 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.728+0000 7f6ee2799640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6edc10a680 con 0x7f6edc101960 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc102b60 0x7f6edc197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc102b60 0x7f6edc197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53710/0 (socket says 172.21.15.46:53710) 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edb7fe640 1 -- 172.21.15.46:0/778864898 learned_addr learned my addr 172.21.15.46:0/778864898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edbfff640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6edc101960 0x7f6edc197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edb7fe640 1 -- 172.21.15.46:0/778864898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6edc101960 msgr2=0x7f6edc197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edb7fe640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6edc101960 0x7f6edc197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edb7fe640 1 -- 172.21.15.46:0/778864898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6edc1980d0 con 0x7f6edc102b60 2024-09-10T04:01:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6edb7fe640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc102b60 0x7f6edc197a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000b420 tx=0x7f6ed000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6ed97fa640 1 -- 172.21.15.46:0/778864898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ed0011630 con 0x7f6edc102b60 2024-09-10T04:01:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6edc19cb10 con 0x7f6edc102b60 2024-09-10T04:01:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6ed97fa640 1 -- 172.21.15.46:0/778864898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ed0016070 con 0x7f6edc102b60 2024-09-10T04:01:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6ed97fa640 1 -- 172.21.15.46:0/778864898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ed0010d40 con 0x7f6edc102b60 2024-09-10T04:01:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.729+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6edc19d120 con 0x7f6edc102b60 2024-09-10T04:01:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.731+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6edc101d60 con 0x7f6edc102b60 2024-09-10T04:01:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.731+0000 7f6ed97fa640 1 -- 172.21.15.46:0/778864898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f6ed001b460 con 0x7f6edc102b60 2024-09-10T04:01:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.731+0000 7f6ed97fa640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6ec406de30 0x7f6ec40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.732+0000 7f6ed97fa640 1 -- 172.21.15.46:0/778864898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4321+0+0 (secure 0 0 0) 0x7f6ed008cf50 con 0x7f6edc102b60 2024-09-10T04:01:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.732+0000 7f6edbfff640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6ec406de30 0x7f6ec40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.732+0000 7f6edbfff640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6ec406de30 0x7f6ec40702f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009230 tx=0x7f6ecc0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.736+0000 7f6ed97fa640 1 -- 172.21.15.46:0/778864898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ed005af60 con 0x7f6edc102b60 2024-09-10T04:01:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.910+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6edc062df0 con 0x7f6edc102b60 2024-09-10T04:01:30.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.910+0000 7f6ed97fa640 1 -- 172.21.15.46:0/778864898 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v34) ==== 74+0+130 (secure 0 0 0) 0x7f6ed005ec10 con 0x7f6edc102b60 2024-09-10T04:01:30.913 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:30.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.912+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6ec406de30 msgr2=0x7f6ec40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.912+0000 7f6ee2799640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6ec406de30 0x7f6ec40702f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009230 tx=0x7f6ecc0058d0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.912+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc102b60 msgr2=0x7f6edc197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.912+0000 7f6ee2799640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc102b60 0x7f6edc197a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000b420 tx=0x7f6ed000b8e0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.913+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 shutdown_connections 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.913+0000 7f6ee2799640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6edc102b60 0x7f6edc197a50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.913+0000 7f6ee2799640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6ec406de30 0x7f6ec40702f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.913+0000 7f6ee2799640 1 --2- 172.21.15.46:0/778864898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6edc101960 0x7f6edc197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.913+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 >> 172.21.15.46:0/778864898 conn(0x7f6edc0fd0f0 msgr2=0x7f6edc105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.913+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 shutdown_connections 2024-09-10T04:01:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:30.913+0000 7f6ee2799640 1 -- 172.21.15.46:0/778864898 wait complete. 2024-09-10T04:01:31.397 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:01:31 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[55103]: 2024-09-10T04:01:31.165+0000 7f61c0c18640 -1 osd.5 0 log_to_monitors true 2024-09-10T04:01:31.655 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:31 smithi046 ceph-mon[28328]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:31.656 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:31 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/778864898' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T04:01:31.656 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:31 smithi046 ceph-mon[28328]: from='osd.5 [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T04:01:31.656 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:31 smithi046 ceph-mon[28328]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T04:01:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"num_osds":6,"num_up_osds":5,"osd_up_since":1725940867,"num_in_osds":6,"osd_in_since":1725940870,"num_remapped_pgs":0} 2024-09-10T04:01:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:31 smithi070 ceph-mon[35495]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:31 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/778864898' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T04:01:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:31 smithi070 ceph-mon[35495]: from='osd.5 [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T04:01:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:31 smithi070 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T04:01:32.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T04:01:32.687 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:32 smithi046 ceph-mon[28328]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T04:01:32.687 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:32 smithi046 ceph-mon[28328]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T04:01:32.687 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:32 smithi046 ceph-mon[28328]: from='osd.5 [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:32.687 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:32 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:32.687 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:32 smithi046 ceph-mon[28328]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:32.687 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:32 smithi046 ceph-mon[28328]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-10T04:01:32.792 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:32 smithi070 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T04:01:32.792 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:32 smithi070 ceph-mon[35495]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T04:01:32.792 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:32 smithi070 ceph-mon[35495]: from='osd.5 [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:32.792 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:32 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:32.792 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:32 smithi070 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]: dispatch 2024-09-10T04:01:32.792 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:32 smithi070 ceph-mon[35495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-10T04:01:32.932 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:33.334 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[55103]: 2024-09-10T04:01:33.040+0000 7f61bc378640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T04:01:33.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[55103]: 2024-09-10T04:01:33.046+0000 7f61b49aa640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:01:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 -- 172.21.15.46:0/1310806058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004104d80 msgr2=0x7f2004107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 --2- 172.21.15.46:0/1310806058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004104d80 0x7f2004107170 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009880 tx=0x7f1ff402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 -- 172.21.15.46:0/1310806058 shutdown_connections 2024-09-10T04:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 --2- 172.21.15.46:0/1310806058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004104d80 0x7f2004107170 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 --2- 172.21.15.46:0/1310806058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2004102450 0x7f2004104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 -- 172.21.15.46:0/1310806058 >> 172.21.15.46:0/1310806058 conn(0x7f20040fc010 msgr2=0x7f20040fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 -- 172.21.15.46:0/1310806058 shutdown_connections 2024-09-10T04:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.393+0000 7f200b840640 1 -- 172.21.15.46:0/1310806058 wait complete. 2024-09-10T04:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f200b840640 1 Processor -- start 2024-09-10T04:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f200b840640 1 -- start start 2024-09-10T04:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f200b840640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004102450 0x7f2004199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f200b840640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2004104d80 0x7f2004199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f200b840640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f200410a740 con 0x7f2004102450 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f200b840640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f200410a5c0 con 0x7f2004104d80 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f20095dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004102450 0x7f2004199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f20095dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004102450 0x7f2004199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53736/0 (socket says 172.21.15.46:53736) 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f20095dc640 1 -- 172.21.15.46:0/2698655691 learned_addr learned my addr 172.21.15.46:0/2698655691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.394+0000 7f2008ddb640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2004104d80 0x7f2004199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f20095dc640 1 -- 172.21.15.46:0/2698655691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2004104d80 msgr2=0x7f2004199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f20095dc640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2004104d80 0x7f2004199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f20095dc640 1 -- 172.21.15.46:0/2698655691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200419a250 con 0x7f2004102450 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f20095dc640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004102450 0x7f2004199690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f200000e970 tx=0x7f200000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f1ffa7fc640 1 -- 172.21.15.46:0/2698655691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f200000cac0 con 0x7f2004102450 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f1ffa7fc640 1 -- 172.21.15.46:0/2698655691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20000054e0 con 0x7f2004102450 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200419ec90 con 0x7f2004102450 2024-09-10T04:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f1ffa7fc640 1 -- 172.21.15.46:0/2698655691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2000010430 con 0x7f2004102450 2024-09-10T04:01:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.395+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200419f1a0 con 0x7f2004102450 2024-09-10T04:01:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.396+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200410af20 con 0x7f2004102450 2024-09-10T04:01:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.397+0000 7f1ffa7fc640 1 -- 172.21.15.46:0/2698655691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f2000004e80 con 0x7f2004102450 2024-09-10T04:01:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.397+0000 7f1ffa7fc640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1ff006de30 0x7f1ff00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:33.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.397+0000 7f2008ddb640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1ff006de30 0x7f1ff00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.397+0000 7f1ffa7fc640 1 -- 172.21.15.46:0/2698655691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4358+0+0 (secure 0 0 0) 0x7f200008dda0 con 0x7f2004102450 2024-09-10T04:01:33.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.398+0000 7f2008ddb640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1ff006de30 0x7f1ff00702f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009230 tx=0x7f1ff40058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.402+0000 7f1ffa7fc640 1 -- 172.21.15.46:0/2698655691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f200005bd90 con 0x7f2004102450 2024-09-10T04:01:33.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.579+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f2004062df0 con 0x7f2004102450 2024-09-10T04:01:33.581 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:33.581 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T04:01:33.581 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:33 smithi046 ceph-mon[28328]: from='osd.5 ' entity='osd.5' 2024-09-10T04:01:33.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.580+0000 7f1ffa7fc640 1 -- 172.21.15.46:0/2698655691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v36) ==== 74+0+130 (secure 0 0 0) 0x7f200005fa40 con 0x7f2004102450 2024-09-10T04:01:33.583 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1ff006de30 msgr2=0x7f1ff00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1ff006de30 0x7f1ff00702f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009230 tx=0x7f1ff40058d0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004102450 msgr2=0x7f2004199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004102450 0x7f2004199690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f200000e970 tx=0x7f200000ed70 comp rx=0 tx=0).stop 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 shutdown_connections 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1ff006de30 0x7f1ff00702f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2004104d80 0x7f2004199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 --2- 172.21.15.46:0/2698655691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2004102450 0x7f2004199690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 >> 172.21.15.46:0/2698655691 conn(0x7f20040fc010 msgr2=0x7f2004101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 shutdown_connections 2024-09-10T04:01:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:33.582+0000 7f200b840640 1 -- 172.21.15.46:0/2698655691 wait complete. 2024-09-10T04:01:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:01:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:01:33.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:01:33.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:33 smithi070 ceph-mon[35495]: from='osd.5 ' entity='osd.5' 2024-09-10T04:01:34.319 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"num_osds":6,"num_up_osds":5,"osd_up_since":1725940867,"num_in_osds":6,"osd_in_since":1725940870,"num_remapped_pgs":0} 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: purged_snaps scrub starts 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: purged_snaps scrub ok 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: Detected new or changed devices on smithi070 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi070 to 4970M 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2698655691' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: osd.5 [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] boot 2024-09-10T04:01:34.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T04:01:34.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: purged_snaps scrub starts 2024-09-10T04:01:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: purged_snaps scrub ok 2024-09-10T04:01:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: Detected new or changed devices on smithi070 2024-09-10T04:01:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi070 to 4970M 2024-09-10T04:01:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2698655691' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T04:01:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: osd.5 [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] boot 2024-09-10T04:01:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T04:01:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:01:35.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T04:01:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:35.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:35 smithi046 ceph-mon[28328]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:35.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:35 smithi046 ceph-mon[28328]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T04:01:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:35 smithi070 ceph-mon[35495]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-10T04:01:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:35 smithi070 ceph-mon[35495]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T04:01:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.049+0000 7f8a8459d640 1 -- 172.21.15.46:0/2674603398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c100860 msgr2=0x7f8a7c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.049+0000 7f8a8459d640 1 --2- 172.21.15.46:0/2674603398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c100860 0x7f8a7c100c60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009880 tx=0x7f8a7802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 -- 172.21.15.46:0/2674603398 shutdown_connections 2024-09-10T04:01:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 --2- 172.21.15.46:0/2674603398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a7c101a60 0x7f8a7c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 --2- 172.21.15.46:0/2674603398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c100860 0x7f8a7c100c60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 -- 172.21.15.46:0/2674603398 >> 172.21.15.46:0/2674603398 conn(0x7f8a7c0fc010 msgr2=0x7f8a7c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 -- 172.21.15.46:0/2674603398 shutdown_connections 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 -- 172.21.15.46:0/2674603398 wait complete. 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 Processor -- start 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 -- start start 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.050+0000 7f8a8459d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a7c100860 0x7f8a7c104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a8459d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c101a60 0x7f8a7c102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a8459d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1096a0 con 0x7f8a7c101a60 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a8459d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c109520 con 0x7f8a7c100860 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a81b38640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c101a60 0x7f8a7c102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a81b38640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c101a60 0x7f8a7c102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53754/0 (socket says 172.21.15.46:53754) 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a81b38640 1 -- 172.21.15.46:0/3135596821 learned_addr learned my addr 172.21.15.46:0/3135596821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a82339640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a7c100860 0x7f8a7c104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a81b38640 1 -- 172.21.15.46:0/3135596821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a7c100860 msgr2=0x7f8a7c104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a81b38640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a7c100860 0x7f8a7c104780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a81b38640 1 -- 172.21.15.46:0/3135596821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7c103380 con 0x7f8a7c101a60 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a81b38640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c101a60 0x7f8a7c102e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00d6f0 tx=0x7f8a6c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a737fe640 1 -- 172.21.15.46:0/3135596821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8a6c017070 con 0x7f8a7c101a60 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a737fe640 1 -- 172.21.15.46:0/3135596821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a6c00de00 con 0x7f8a7c101a60 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7c103670 con 0x7f8a7c101a60 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a737fe640 1 -- 172.21.15.46:0/3135596821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8a6c004bb0 con 0x7f8a7c101a60 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.051+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c103c00 con 0x7f8a7c101a60 2024-09-10T04:01:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.052+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a7c100c60 con 0x7f8a7c101a60 2024-09-10T04:01:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.053+0000 7f8a737fe640 1 -- 172.21.15.46:0/3135596821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f8a6c004e60 con 0x7f8a7c101a60 2024-09-10T04:01:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.053+0000 7f8a737fe640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8a6806de30 0x7f8a680702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.053+0000 7f8a737fe640 1 -- 172.21.15.46:0/3135596821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f8a6c08d440 con 0x7f8a7c101a60 2024-09-10T04:01:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.054+0000 7f8a82339640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8a6806de30 0x7f8a680702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.054+0000 7f8a82339640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8a6806de30 0x7f8a680702f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a78000c00 tx=0x7f8a78005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.058+0000 7f8a737fe640 1 -- 172.21.15.46:0/3135596821 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a6c05b320 con 0x7f8a7c101a60 2024-09-10T04:01:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.233+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8a7c19fc40 con 0x7f8a7c101a60 2024-09-10T04:01:36.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.233+0000 7f8a737fe640 1 -- 172.21.15.46:0/3135596821 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) ==== 74+0+130 (secure 0 0 0) 0x7f8a6c05efd0 con 0x7f8a7c101a60 2024-09-10T04:01:36.235 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:36.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8a6806de30 msgr2=0x7f8a680702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:36.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8a6806de30 0x7f8a680702f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8a78000c00 tx=0x7f8a78005a20 comp rx=0 tx=0).stop 2024-09-10T04:01:36.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c101a60 msgr2=0x7f8a7c102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:36.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c101a60 0x7f8a7c102e40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c00d6f0 tx=0x7f8a6c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 shutdown_connections 2024-09-10T04:01:36.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a7c101a60 0x7f8a7c102e40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8a6806de30 0x7f8a680702f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 --2- 172.21.15.46:0/3135596821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a7c100860 0x7f8a7c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:36.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 >> 172.21.15.46:0/3135596821 conn(0x7f8a7c0fc010 msgr2=0x7f8a7c069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:36.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.235+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 shutdown_connections 2024-09-10T04:01:36.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:36.236+0000 7f8a8459d640 1 -- 172.21.15.46:0/3135596821 wait complete. 2024-09-10T04:01:36.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:36 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3135596821' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T04:01:36.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:36 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3135596821' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T04:01:36.894 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"num_osds":6,"num_up_osds":6,"osd_up_since":1725940894,"num_in_osds":6,"osd_in_since":1725940870,"num_remapped_pgs":0} 2024-09-10T04:01:36.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T04:01:37.154 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:37.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:37 smithi046 ceph-mon[28328]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 -- 172.21.15.46:0/1493203400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81067f0 msgr2=0x7f60b8106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 --2- 172.21.15.46:0/1493203400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81067f0 0x7f60b8106bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009880 tx=0x7f60b402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 -- 172.21.15.46:0/1493203400 shutdown_connections 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 --2- 172.21.15.46:0/1493203400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81007f0 0x7f60b8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 --2- 172.21.15.46:0/1493203400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81067f0 0x7f60b8106bd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 -- 172.21.15.46:0/1493203400 >> 172.21.15.46:0/1493203400 conn(0x7f60b80fc550 msgr2=0x7f60b80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 -- 172.21.15.46:0/1493203400 shutdown_connections 2024-09-10T04:01:37.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 -- 172.21.15.46:0/1493203400 wait complete. 2024-09-10T04:01:37.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.590+0000 7f60c027e640 1 Processor -- start 2024-09-10T04:01:37.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60c027e640 1 -- start start 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60c027e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81007f0 0x7f60b819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60c027e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81067f0 0x7f60b819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60c027e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810dca0 con 0x7f60b81007f0 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60c027e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810db20 con 0x7f60b81067f0 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60bd819640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81067f0 0x7f60b819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60be01a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81007f0 0x7f60b819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60bd819640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81067f0 0x7f60b819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:38134/0 (socket says 172.21.15.46:38134) 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.591+0000 7f60bd819640 1 -- 172.21.15.46:0/1773046783 learned_addr learned my addr 172.21.15.46:0/1773046783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60bd819640 1 -- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81007f0 msgr2=0x7f60b819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60bd819640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81007f0 0x7f60b819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:37.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60bd819640 1 -- 172.21.15.46:0/1773046783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b8197750 con 0x7f60b81067f0 2024-09-10T04:01:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60be01a640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81007f0 0x7f60b819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:01:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60bd819640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81067f0 0x7f60b819dbc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d8d0 tx=0x7f60a800dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60af7fe640 1 -- 172.21.15.46:0/1773046783 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f60a8017070 con 0x7f60b81067f0 2024-09-10T04:01:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b8197a40 con 0x7f60b81067f0 2024-09-10T04:01:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60af7fe640 1 -- 172.21.15.46:0/1773046783 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f60a8010460 con 0x7f60b81067f0 2024-09-10T04:01:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60af7fe640 1 -- 172.21.15.46:0/1773046783 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f60a800f5f0 con 0x7f60b81067f0 2024-09-10T04:01:37.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.592+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b8197fa0 con 0x7f60b81067f0 2024-09-10T04:01:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.593+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b8101f30 con 0x7f60b81067f0 2024-09-10T04:01:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.594+0000 7f60af7fe640 1 -- 172.21.15.46:0/1773046783 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f60a8009a30 con 0x7f60b81067f0 2024-09-10T04:01:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.594+0000 7f60af7fe640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f60a406de90 0x7f60a4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.594+0000 7f60af7fe640 1 -- 172.21.15.46:0/1773046783 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f60a808d750 con 0x7f60b81067f0 2024-09-10T04:01:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.594+0000 7f60be01a640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f60a406de90 0x7f60a4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:37.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.595+0000 7f60be01a640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f60a406de90 0x7f60a4070350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60b40096c0 tx=0x7f60b4031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:37.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.599+0000 7f60af7fe640 1 -- 172.21.15.46:0/1773046783 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60a805f240 con 0x7f60b81067f0 2024-09-10T04:01:37.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.775+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60b8198250 con 0x7f60b81067f0 2024-09-10T04:01:37.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.776+0000 7f60af7fe640 1 -- 172.21.15.46:0/1773046783 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+11335 (secure 0 0 0) 0x7f60a8063a70 con 0x7f60b81067f0 2024-09-10T04:01:37.778 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:37.778 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","created":"2024-09-10T03:55:32.552379+0000","modified":"2024-09-10T04:01:35.044377+0000","last_up_change":"2024-09-10T04:01:34.042619+0000","last_in_change":"2024-09-10T04:01:10.397117+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T04:00:16.212497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2edfb38-e5a1-4d6b-993a-66efc78d66f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6803","nonce":3291780356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6805","nonce":3291780356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6809","nonce":3291780356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6807","nonce":3291780356}]},"public_addr":"172.21.15.46:6803/3291780356","cluster_addr":"172.21.15.46:6805/3291780356","heartbeat_back_addr":"172.21.15.46:6809/3291780356","heartbeat_front_addr":"172.21.15.46:6807/3291780356","state":["exists","up"]},{"osd":1,"uuid":"1e2cbb84-8fbe-485f-8bc0-62da12e599c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6811","nonce":1584366844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6813","nonce":1584366844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6817","nonce":1584366844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6815","nonce":1584366844}]},"public_addr":"172.21.15.46:6811/1584366844","cluster_addr":"172.21.15.46:6813/1584366844","heartbeat_back_addr":"172.21.15.46:6817/1584366844","heartbeat_front_addr":"172.21.15.46:6815/1584366844","state":["exists","up"]},{"osd":2,"uuid":"346d282b-5119-44f6-8a94-5cf2f7ece643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6819","nonce":704152147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6821","nonce":704152147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6825","nonce":704152147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6823","nonce":704152147}]},"public_addr":"172.21.15.46:6819/704152147","cluster_addr":"172.21.15.46:6821/704152147","heartbeat_back_addr":"172.21.15.46:6825/704152147","heartbeat_front_addr":"172.21.15.46:6823/704152147","state":["exists","up"]},{"osd":3,"uuid":"503bec3a-eb26-42a1-973e-2c145bd7d10c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6801","nonce":578321990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6803","nonce":578321990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6807","nonce":578321990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6805","nonce":578321990}]},"public_addr":"172.21.15.70:6801/578321990","cluster_addr":"172.21.15.70:6803/578321990","heartbeat_back_addr":"172.21.15.70:6807/578321990","heartbeat_front_addr":"172.21.15.70:6805/578321990","state":["exists","up"]},{"osd":4,"uuid":"ba74b660-732c-4947-af9b-8f5d75e4da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6809","nonce":2152592034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6811","nonce":2152592034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6815","nonce":2152592034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6813","nonce":2152592034}]},"public_addr":"172.21.15.70:6809/2152592034","cluster_addr":"172.21.15.70:6811/2152592034","heartbeat_back_addr":"172.21.15.70:6815/2152592034","heartbeat_front_addr":"172.21.15.70:6813/2152592034","state":["exists","up"]},{"osd":5,"uuid":"e092e99e-0c0d-404b-9124-25da794f9eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6817","nonce":479912299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6819","nonce":479912299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6823","nonce":479912299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6821","nonce":479912299}]},"public_addr":"172.21.15.70:6817/479912299","cluster_addr":"172.21.15.70:6819/479912299","heartbeat_back_addr":"172.21.15.70:6823/479912299","heartbeat_front_addr":"172.21.15.70:6821/479912299","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:12.098977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:44.500164+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:13.640248+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:39.349187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:01:05.658570+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:01:32.135875+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2984295913":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/1609399519":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/4048021550":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/3785267557":"2024-09-11T03:56:03.190805+0000","172.21.15.46:6801/2101940292":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/3311822035":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/1023279878":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6801/1804840410":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/2504291949":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6801/653182070":"2024-09-11T03:56:35.886608+0000","172.21.15.46:0/1078369413":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6800/653182070":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6800/2101940292":"2024-09-11T03:57:33.701336+0000","172.21.15.46:6800/1804840410":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/2611013585":"2024-09-11T03:57:33.701336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T04:01:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f60a406de90 msgr2=0x7f60a4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f60a406de90 0x7f60a4070350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f60b40096c0 tx=0x7f60b4031400 comp rx=0 tx=0).stop 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81067f0 msgr2=0x7f60b819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81067f0 0x7f60b819dbc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d8d0 tx=0x7f60a800dd90 comp rx=0 tx=0).stop 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 shutdown_connections 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f60a406de90 0x7f60a4070350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60b81067f0 0x7f60b819dbc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 --2- 172.21.15.46:0/1773046783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60b81007f0 0x7f60b819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 >> 172.21.15.46:0/1773046783 conn(0x7f60b80fc550 msgr2=0x7f60b8069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 shutdown_connections 2024-09-10T04:01:37.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:37.779+0000 7f60c027e640 1 -- 172.21.15.46:0/1773046783 wait complete. 2024-09-10T04:01:37.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:37 smithi070 ceph-mon[35495]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:38.550 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T04:00:16.212497+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T04:01:38.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T04:01:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:38.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:38 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1773046783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T04:01:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:38 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1773046783' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T04:01:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 -- 172.21.15.46:0/41529022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808101900 msgr2=0x7f9808101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 --2- 172.21.15.46:0/41529022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808101900 0x7f9808101d60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009880 tx=0x7f97fc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 -- 172.21.15.46:0/41529022 shutdown_connections 2024-09-10T04:01:39.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 --2- 172.21.15.46:0/41529022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808101900 0x7f9808101d60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 --2- 172.21.15.46:0/41529022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9808107900 0x7f9808107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 -- 172.21.15.46:0/41529022 >> 172.21.15.46:0/41529022 conn(0x7f98080fd600 msgr2=0x7f98080ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 -- 172.21.15.46:0/41529022 shutdown_connections 2024-09-10T04:01:39.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.233+0000 7f980cae0640 1 -- 172.21.15.46:0/41529022 wait complete. 2024-09-10T04:01:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.234+0000 7f980cae0640 1 Processor -- start 2024-09-10T04:01:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.234+0000 7f980cae0640 1 -- start start 2024-09-10T04:01:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.234+0000 7f980cae0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9808101900 0x7f980819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:39.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.234+0000 7f980cae0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808107900 0x7f980819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.234+0000 7f980cae0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f980810ee30 con 0x7f9808107900 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.234+0000 7f980cae0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f980810ecb0 con 0x7f9808101900 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f9805d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808107900 0x7f980819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f9805d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808107900 0x7f980819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34584/0 (socket says 172.21.15.46:34584) 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f9805d9b640 1 -- 172.21.15.46:0/3253701210 learned_addr learned my addr 172.21.15.46:0/3253701210 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f980659c640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9808101900 0x7f980819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f9805d9b640 1 -- 172.21.15.46:0/3253701210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9808101900 msgr2=0x7f980819d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f9805d9b640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9808101900 0x7f980819d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f9805d9b640 1 -- 172.21.15.46:0/3253701210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9808197840 con 0x7f9808107900 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f980659c640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9808101900 0x7f980819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f9805d9b640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808107900 0x7f980819dc00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f97fc031460 tx=0x7f97fc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:39.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f97f77fe640 1 -- 172.21.15.46:0/3253701210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97fc03f070 con 0x7f9808107900 2024-09-10T04:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f97f77fe640 1 -- 172.21.15.46:0/3253701210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97fc004cf0 con 0x7f9808107900 2024-09-10T04:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9808197ad0 con 0x7f9808107900 2024-09-10T04:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f97f77fe640 1 -- 172.21.15.46:0/3253701210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97fc035030 con 0x7f9808107900 2024-09-10T04:01:39.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.235+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9808198000 con 0x7f9808107900 2024-09-10T04:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.236+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9808103040 con 0x7f9808107900 2024-09-10T04:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.237+0000 7f97f77fe640 1 -- 172.21.15.46:0/3253701210 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f97fc042070 con 0x7f9808107900 2024-09-10T04:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.238+0000 7f97f77fe640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f97f006de90 0x7f97f0070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.238+0000 7f980659c640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f97f006de90 0x7f97f0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.238+0000 7f97f77fe640 1 -- 172.21.15.46:0/3253701210 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f97fc0ae260 con 0x7f9808107900 2024-09-10T04:01:39.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.238+0000 7f980659c640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f97f006de90 0x7f97f0070350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009930 tx=0x7f97f8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:39.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.242+0000 7f97f77fe640 1 -- 172.21.15.46:0/3253701210 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97fc07c1c0 con 0x7f9808107900 2024-09-10T04:01:39.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.415+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9808198d70 con 0x7f9808107900 2024-09-10T04:01:39.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.416+0000 7f97f77fe640 1 -- 172.21.15.46:0/3253701210 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f97fc07fe70 con 0x7f9808107900 2024-09-10T04:01:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-09-10T04:01:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f97f006de90 msgr2=0x7f97f0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f97f006de90 0x7f97f0070350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009930 tx=0x7f97f8008040 comp rx=0 tx=0).stop 2024-09-10T04:01:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808107900 msgr2=0x7f980819dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808107900 0x7f980819dc00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f97fc031460 tx=0x7f97fc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 shutdown_connections 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808107900 0x7f980819dc00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f97f006de90 0x7f97f0070350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 --2- 172.21.15.46:0/3253701210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9808101900 0x7f980819d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 >> 172.21.15.46:0/3253701210 conn(0x7f98080fd600 msgr2=0x7f98080fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 shutdown_connections 2024-09-10T04:01:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:39.418+0000 7f980cae0640 1 -- 172.21.15.46:0/3253701210 wait complete. 2024-09-10T04:01:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:39 smithi046 ceph-mon[28328]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:39 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3253701210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T04:01:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:39 smithi070 ceph-mon[35495]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:39 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3253701210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T04:01:40.092 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T04:01:40.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T04:01:40.353 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 -- 172.21.15.46:0/1226900705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 msgr2=0x7f8f801047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 --2- 172.21.15.46:0/1226900705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f801047e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8f700098b0 tx=0x7f8f7002a4d0 comp rx=0 tx=0).stop 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 -- 172.21.15.46:0/1226900705 shutdown_connections 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 --2- 172.21.15.46:0/1226900705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f80104d20 0x7f8f80107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 --2- 172.21.15.46:0/1226900705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f801047e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 -- 172.21.15.46:0/1226900705 >> 172.21.15.46:0/1226900705 conn(0x7f8f800fc010 msgr2=0x7f8f800fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 -- 172.21.15.46:0/1226900705 shutdown_connections 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 -- 172.21.15.46:0/1226900705 wait complete. 2024-09-10T04:01:40.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.805+0000 7f8f85b85640 1 Processor -- start 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f85b85640 1 -- start start 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f85b85640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f80101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f85b85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f80104d20 0x7f8f801003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f85b85640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8010a7e0 con 0x7f8f801023f0 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f85b85640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8010a660 con 0x7f8f80104d20 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f80101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f80101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34596/0 (socket says 172.21.15.46:34596) 2024-09-10T04:01:40.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f7f7fe640 1 -- 172.21.15.46:0/2129504425 learned_addr learned my addr 172.21.15.46:0/2129504425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f7effd640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f80104d20 0x7f8f801003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f7f7fe640 1 -- 172.21.15.46:0/2129504425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f80104d20 msgr2=0x7f8f801003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f7f7fe640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f80104d20 0x7f8f801003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.806+0000 7f8f7f7fe640 1 -- 172.21.15.46:0/2129504425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f801008e0 con 0x7f8f801023f0 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.807+0000 7f8f7f7fe640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f80101ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f700099d0 tx=0x7f8f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.807+0000 7f8f7cff9640 1 -- 172.21.15.46:0/2129504425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f7003f070 con 0x7f8f801023f0 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.807+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f80100b70 con 0x7f8f801023f0 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.807+0000 7f8f7cff9640 1 -- 172.21.15.46:0/2129504425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f70004a40 con 0x7f8f801023f0 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.807+0000 7f8f7cff9640 1 -- 172.21.15.46:0/2129504425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f70035030 con 0x7f8f801023f0 2024-09-10T04:01:40.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.807+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f801010a0 con 0x7f8f801023f0 2024-09-10T04:01:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.808+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f801027d0 con 0x7f8f801023f0 2024-09-10T04:01:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.808+0000 7f8f7cff9640 1 -- 172.21.15.46:0/2129504425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f8f70042070 con 0x7f8f801023f0 2024-09-10T04:01:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.809+0000 7f8f7cff9640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8f6806de30 0x7f8f680702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.809+0000 7f8f7effd640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8f6806de30 0x7f8f680702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.809+0000 7f8f7cff9640 1 -- 172.21.15.46:0/2129504425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f8f700ae3d0 con 0x7f8f801023f0 2024-09-10T04:01:40.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.809+0000 7f8f7effd640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8f6806de30 0x7f8f680702f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f80101a60 tx=0x7f8f7400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:40.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:40.813+0000 7f8f7cff9640 1 -- 172.21.15.46:0/2129504425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f7007ff60 con 0x7f8f801023f0 2024-09-10T04:01:41.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.059+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f8f8019fa90 con 0x7f8f801023f0 2024-09-10T04:01:41.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.061+0000 7f8f7cff9640 1 -- 172.21.15.46:0/2129504425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) ==== 170+0+59 (secure 0 0 0) 0x7f8f700ae1f0 con 0x7f8f801023f0 2024-09-10T04:01:41.062 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-09-10T04:01:41.062 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQClxN9mQEGhAxAAcboN0bwoVI5qdJULL+vqFA== 2024-09-10T04:01:41.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8f6806de30 msgr2=0x7f8f680702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8f6806de30 0x7f8f680702f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f80101a60 tx=0x7f8f7400b390 comp rx=0 tx=0).stop 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 msgr2=0x7f8f80101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f80101ce0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8f700099d0 tx=0x7f8f700046c0 comp rx=0 tx=0).stop 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 shutdown_connections 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f8f6806de30 0x7f8f680702f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f80104d20 0x7f8f801003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 --2- 172.21.15.46:0/2129504425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f801023f0 0x7f8f80101ce0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 >> 172.21.15.46:0/2129504425 conn(0x7f8f800fc010 msgr2=0x7f8f800fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 shutdown_connections 2024-09-10T04:01:41.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:41.063+0000 7f8f85b85640 1 -- 172.21.15.46:0/2129504425 wait complete. 2024-09-10T04:01:41.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:41 smithi046 ceph-mon[28328]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:41.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:41 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2129504425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T04:01:41.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:41 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2129504425' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:01:41.823 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T04:01:41.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T04:01:41.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T04:01:41.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:41 smithi070 ceph-mon[35495]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:41 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2129504425' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T04:01:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:41 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2129504425' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:01:41.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T04:01:42.123 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:01:42.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 -- 172.21.15.70:0/4195368583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004101a30 msgr2=0x7f3004101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:42.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 --2- 172.21.15.70:0/4195368583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004101a30 0x7f3004101eb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009880 tx=0x7f2ff802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:42.860 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 -- 172.21.15.70:0/4195368583 shutdown_connections 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 --2- 172.21.15.70:0/4195368583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004101a30 0x7f3004101eb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 --2- 172.21.15.70:0/4195368583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3004100830 0x7f3004100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 -- 172.21.15.70:0/4195368583 >> 172.21.15.70:0/4195368583 conn(0x7f30040fbfa0 msgr2=0x7f30040fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 -- 172.21.15.70:0/4195368583 shutdown_connections 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 -- 172.21.15.70:0/4195368583 wait complete. 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.849+0000 7f3009f93640 1 Processor -- start 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3009f93640 1 -- start start 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3009f93640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004100830 0x7f30041963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3009f93640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3004101a30 0x7f3004196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3009f93640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3004109670 con 0x7f3004101a30 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3009f93640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f30041094f0 con 0x7f3004100830 2024-09-10T04:01:42.861 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f30037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004100830 0x7f30041963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f30037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004100830 0x7f30041963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38376/0 (socket says 172.21.15.70:38376) 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f30037fe640 1 -- 172.21.15.70:0/627956986 learned_addr learned my addr 172.21.15.70:0/627956986 (peer_addr_for_me v2:172.21.15.70:0/0) 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3002ffd640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3004101a30 0x7f3004196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f30037fe640 1 -- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3004101a30 msgr2=0x7f3004196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f30037fe640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3004101a30 0x7f3004196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f30037fe640 1 -- 172.21.15.70:0/627956986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3004196fb0 con 0x7f3004100830 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f30037fe640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004100830 0x7f30041963f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d8d0 tx=0x7f2ff400dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3000ff9640 1 -- 172.21.15.70:0/627956986 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2ff4017070 con 0x7f3004100830 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3004069d60 con 0x7f3004100830 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3000ff9640 1 -- 172.21.15.70:0/627956986 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2ff400be10 con 0x7f3004100830 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.850+0000 7f3000ff9640 1 -- 172.21.15.70:0/627956986 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2ff400f3c0 con 0x7f3004100830 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.851+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300406a2f0 con 0x7f3004100830 2024-09-10T04:01:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.852+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3004100c30 con 0x7f3004100830 2024-09-10T04:01:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.852+0000 7f3000ff9640 1 -- 172.21.15.70:0/627956986 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f2ff401a070 con 0x7f3004100830 2024-09-10T04:01:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.852+0000 7f3000ff9640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2fec06de30 0x7f2fec0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.853+0000 7f3000ff9640 1 -- 172.21.15.70:0/627956986 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f2ff408d510 con 0x7f3004100830 2024-09-10T04:01:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.853+0000 7f3002ffd640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2fec06de30 0x7f2fec0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.853+0000 7f3002ffd640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2fec06de30 0x7f2fec0702f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f30041978a0 tx=0x7f2ff8005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:42.857+0000 7f3000ff9640 1 -- 172.21.15.70:0/627956986 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ff405c460 con 0x7f3004100830 2024-09-10T04:01:43.084 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.083+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3004190480 con 0x7f3004100830 2024-09-10T04:01:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.087+0000 7f3000ff9640 1 -- 172.21.15.70:0/627956986 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) ==== 170+0+59 (secure 0 0 0) 0x7f2ff408d330 con 0x7f3004100830 2024-09-10T04:01:43.089 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2024-09-10T04:01:43.089 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCnxN9mGgkVBRAAwWM91Yt/5kCELIJ0Ap/8Gw== 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2fec06de30 msgr2=0x7f2fec0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2fec06de30 0x7f2fec0702f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f30041978a0 tx=0x7f2ff8005a20 comp rx=0 tx=0).stop 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004100830 msgr2=0x7f30041963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004100830 0x7f30041963f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d8d0 tx=0x7f2ff400dd90 comp rx=0 tx=0).stop 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 shutdown_connections 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3004101a30 0x7f3004196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2fec06de30 0x7f2fec0702f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 --2- 172.21.15.70:0/627956986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3004100830 0x7f30041963f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 >> 172.21.15.70:0/627956986 conn(0x7f30040fbfa0 msgr2=0x7f3004104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 shutdown_connections 2024-09-10T04:01:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:01:43.089+0000 7f3009f93640 1 -- 172.21.15.70:0/627956986 wait complete. 2024-09-10T04:01:43.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:43 smithi046 ceph-mon[28328]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:43 smithi046 ceph-mon[28328]: from='client.? 172.21.15.70:0/627956986' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T04:01:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:43 smithi046 ceph-mon[28328]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T04:01:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:43 smithi046 ceph-mon[28328]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:01:43.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:43 smithi070 ceph-mon[35495]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:43 smithi070 ceph-mon[35495]: from='client.? 172.21.15.70:0/627956986' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T04:01:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:43 smithi070 ceph-mon[35495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T04:01:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:43 smithi070 ceph-mon[35495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:01:43.880 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T04:01:43.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T04:01:43.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T04:01:43.918 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-10T04:01:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.640+0000 7f630bf24640 1 -- 172.21.15.46:0/2925083152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 msgr2=0x7f630410dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.640+0000 7f630bf24640 1 --2- 172.21.15.46:0/2925083152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630410dcc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009880 tx=0x7f62f402a490 comp rx=0 tx=0).stop 2024-09-10T04:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.640+0000 7f630bf24640 1 -- 172.21.15.46:0/2925083152 shutdown_connections 2024-09-10T04:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.640+0000 7f630bf24640 1 --2- 172.21.15.46:0/2925083152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630410dcc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 --2- 172.21.15.46:0/2925083152 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304100650 0x7f6304100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 -- 172.21.15.46:0/2925083152 >> 172.21.15.46:0/2925083152 conn(0x7f63040fc520 msgr2=0x7f63040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 -- 172.21.15.46:0/2925083152 shutdown_connections 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 -- 172.21.15.46:0/2925083152 wait complete. 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 Processor -- start 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 -- start start 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304100650 0x7f630419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6304110ff0 con 0x7f6304101000 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.641+0000 7f630bf24640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6304110e70 con 0x7f6304100650 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f63094bf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f63094bf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630419fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34620/0 (socket says 172.21.15.46:34620) 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f63094bf640 1 -- 172.21.15.46:0/4205138337 learned_addr learned my addr 172.21.15.46:0/4205138337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:44.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f6309cc0640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304100650 0x7f630419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f63094bf640 1 -- 172.21.15.46:0/4205138337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304100650 msgr2=0x7f630419f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f63094bf640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304100650 0x7f630419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f63094bf640 1 -- 172.21.15.46:0/4205138337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6304199a80 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f63094bf640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630419fe00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f62f4031460 tx=0x7f62f40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f62faffd640 1 -- 172.21.15.46:0/4205138337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62f403f070 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6304199d10 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f62faffd640 1 -- 172.21.15.46:0/4205138337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f62f4004cf0 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f62faffd640 1 -- 172.21.15.46:0/4205138337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62f4035030 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.642+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630419a2a0 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.643+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630406aa90 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.644+0000 7f62faffd640 1 -- 172.21.15.46:0/4205138337 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f62f4042070 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.644+0000 7f62faffd640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f62f006de30 0x7f62f00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.644+0000 7f62faffd640 1 -- 172.21.15.46:0/4205138337 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f62f40ae330 con 0x7f6304101000 2024-09-10T04:01:44.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.644+0000 7f6309cc0640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f62f006de30 0x7f62f00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:44.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.645+0000 7f6309cc0640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f62f006de30 0x7f62f00702f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f630419abc0 tx=0x7f6300008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:44.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.649+0000 7f62faffd640 1 -- 172.21.15.46:0/4205138337 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62f407c210 con 0x7f6304101000 2024-09-10T04:01:44.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.826+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f630410b3a0 con 0x7f6304101000 2024-09-10T04:01:44.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.826+0000 7f62faffd640 1 -- 172.21.15.46:0/4205138337 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v40) ==== 86+0+160 (secure 0 0 0) 0x7f62f40ae150 con 0x7f6304101000 2024-09-10T04:01:44.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.828+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f62f006de30 msgr2=0x7f62f00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.828+0000 7f630bf24640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f62f006de30 0x7f62f00702f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f630419abc0 tx=0x7f6300008040 comp rx=0 tx=0).stop 2024-09-10T04:01:44.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.828+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 msgr2=0x7f630419fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.828+0000 7f630bf24640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630419fe00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f62f4031460 tx=0x7f62f40048f0 comp rx=0 tx=0).stop 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.829+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 shutdown_connections 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.829+0000 7f630bf24640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6304101000 0x7f630419fe00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.829+0000 7f630bf24640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f62f006de30 0x7f62f00702f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.829+0000 7f630bf24640 1 --2- 172.21.15.46:0/4205138337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304100650 0x7f630419f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.829+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 >> 172.21.15.46:0/4205138337 conn(0x7f63040fc520 msgr2=0x7f63040fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.829+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 shutdown_connections 2024-09-10T04:01:44.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:44.829+0000 7f630bf24640 1 -- 172.21.15.46:0/4205138337 wait complete. 2024-09-10T04:01:45.598 INFO:teuthology.orchestra.run.smithi046.stdout:[{"version":40,"timestamp":"2024-09-10T04:01:33.041772+0000","name":"","changes":[{"name":"osd.5/osd_mclock_max_capacity_iops_ssd","new_value":"7873.536289"}]}] 2024-09-10T04:01:45.598 INFO:tasks.ceph_manager:config epoch is 40 2024-09-10T04:01:45.598 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T04:01:45.598 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T04:01:45.599 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T04:01:45.603 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:45 smithi070 ceph-mon[35495]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:45.603 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:45 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/4205138337' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-10T04:01:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:45 smithi046 ceph-mon[28328]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:45 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/4205138337' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-10T04:01:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 -- 172.21.15.46:0/589138806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 msgr2=0x7efe10103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 --2- 172.21.15.46:0/589138806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe10103f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efe00009880 tx=0x7efe0002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 -- 172.21.15.46:0/589138806 shutdown_connections 2024-09-10T04:01:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 --2- 172.21.15.46:0/589138806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe10103f60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 --2- 172.21.15.46:0/589138806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe10109b90 0x7efe10109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 -- 172.21.15.46:0/589138806 >> 172.21.15.46:0/589138806 conn(0x7efe100ff800 msgr2=0x7efe10101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 -- 172.21.15.46:0/589138806 shutdown_connections 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.292+0000 7efe1865f640 1 -- 172.21.15.46:0/589138806 wait complete. 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.293+0000 7efe1865f640 1 Processor -- start 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.293+0000 7efe1865f640 1 -- start start 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.293+0000 7efe1865f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe1019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.293+0000 7efe1865f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe10109b90 0x7efe1019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.293+0000 7efe1865f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe10110fd0 con 0x7efe10103b00 2024-09-10T04:01:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.293+0000 7efe1865f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efe10110e50 con 0x7efe10109b90 2024-09-10T04:01:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.293+0000 7efe163fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe1019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe163fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe1019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34642/0 (socket says 172.21.15.46:34642) 2024-09-10T04:01:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe163fb640 1 -- 172.21.15.46:0/3578099852 learned_addr learned my addr 172.21.15.46:0/3578099852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe15bfa640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe10109b90 0x7efe1019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe163fb640 1 -- 172.21.15.46:0/3578099852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe10109b90 msgr2=0x7efe1019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe163fb640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe10109b90 0x7efe1019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe163fb640 1 -- 172.21.15.46:0/3578099852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe101999e0 con 0x7efe10103b00 2024-09-10T04:01:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe163fb640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe1019f820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00e970 tx=0x7efe0c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe077fe640 1 -- 172.21.15.46:0/3578099852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe0c00cac0 con 0x7efe10103b00 2024-09-10T04:01:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe10199cd0 con 0x7efe10103b00 2024-09-10T04:01:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe077fe640 1 -- 172.21.15.46:0/3578099852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe0c0054e0 con 0x7efe10103b00 2024-09-10T04:01:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.294+0000 7efe077fe640 1 -- 172.21.15.46:0/3578099852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe0c010430 con 0x7efe10103b00 2024-09-10T04:01:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.295+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe1019a230 con 0x7efe10103b00 2024-09-10T04:01:46.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.296+0000 7efe077fe640 1 -- 172.21.15.46:0/3578099852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7efe0c004f00 con 0x7efe10103b00 2024-09-10T04:01:46.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.296+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe10105200 con 0x7efe10103b00 2024-09-10T04:01:46.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.296+0000 7efe077fe640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7efdfc06de90 0x7efdfc070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:46.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.296+0000 7efe077fe640 1 -- 172.21.15.46:0/3578099852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7efe0c08dd90 con 0x7efe10103b00 2024-09-10T04:01:46.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.297+0000 7efe15bfa640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7efdfc06de90 0x7efdfc070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:46.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.297+0000 7efe15bfa640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7efdfc06de90 0x7efdfc070350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efe1019b1d0 tx=0x7efe00031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.301+0000 7efe077fe640 1 -- 172.21.15.46:0/3578099852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe0c05bcf0 con 0x7efe10103b00 2024-09-10T04:01:46.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.523+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7efe1019af10 con 0x7efe10103b00 2024-09-10T04:01:46.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.526+0000 7efe077fe640 1 -- 172.21.15.46:0/3578099852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) ==== 74+0+176000 (secure 0 0 0) 0x7efe0c05f9a0 con 0x7efe10103b00 2024-09-10T04:01:46.531 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7efdfc06de90 msgr2=0x7efdfc070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7efdfc06de90 0x7efdfc070350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efe1019b1d0 tx=0x7efe00031400 comp rx=0 tx=0).stop 2024-09-10T04:01:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 msgr2=0x7efe1019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe1019f820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00e970 tx=0x7efe0c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 shutdown_connections 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7efdfc06de90 0x7efdfc070350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe10109b90 0x7efe1019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 --2- 172.21.15.46:0/3578099852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe10103b00 0x7efe1019f820 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 >> 172.21.15.46:0/3578099852 conn(0x7efe100ff800 msgr2=0x7efe101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 shutdown_connections 2024-09-10T04:01:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:46.529+0000 7efe1865f640 1 -- 172.21.15.46:0/3578099852 wait complete. 2024-09-10T04:01:47.218 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"flags":0,"active_gid":14217,"active_name":"smithi046.zvckjg","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3538810660},{"type":"v1","addr":"172.21.15.46:6801","nonce":3538810660}]},"active_addr":"172.21.15.46:6801/3538810660","active_change":"2024-09-10T03:57:33.701450+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":14242,"name":"smithi070.tezbzn","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/","prometheus":"http://172.21.15.46:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2676338366}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2203092367}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":4068095835}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3014162921}]}]} 2024-09-10T04:01:47.222 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T04:01:47.222 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T04:01:47.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T04:01:47.486 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:47.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:47 smithi046 ceph-mon[28328]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:47.722 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:47 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3578099852' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T04:01:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:47 smithi070 ceph-mon[35495]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:47 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3578099852' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T04:01:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 -- 172.21.15.46:0/3163963964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8100810 msgr2=0x7f13c8100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3163963964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8100810 0x7f13c8100c70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009880 tx=0x7f13b002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 -- 172.21.15.46:0/3163963964 shutdown_connections 2024-09-10T04:01:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3163963964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8100810 0x7f13c8100c70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3163963964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c8106810 0x7f13c8106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 -- 172.21.15.46:0/3163963964 >> 172.21.15.46:0/3163963964 conn(0x7f13c80fc510 msgr2=0x7f13c80fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 -- 172.21.15.46:0/3163963964 shutdown_connections 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 -- 172.21.15.46:0/3163963964 wait complete. 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 Processor -- start 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.910+0000 7f13ccf51640 1 -- start start 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13ccf51640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c8100810 0x7f13c80fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13ccf51640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8106810 0x7f13c80fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13ccf51640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13c810dcc0 con 0x7f13c8106810 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13ccf51640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13c810db40 con 0x7f13c8100810 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8106810 0x7f13c80fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c5d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8106810 0x7f13c80fd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34662/0 (socket says 172.21.15.46:34662) 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c5d9b640 1 -- 172.21.15.46:0/3527382273 learned_addr learned my addr 172.21.15.46:0/3527382273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c659c640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c8100810 0x7f13c80fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c5d9b640 1 -- 172.21.15.46:0/3527382273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c8100810 msgr2=0x7f13c80fcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:47.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c5d9b640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c8100810 0x7f13c80fcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c5d9b640 1 -- 172.21.15.46:0/3527382273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c80fd680 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c659c640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c8100810 0x7f13c80fcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.911+0000 7f13c5d9b640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8106810 0x7f13c80fd140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13b0031460 tx=0x7f13b00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.912+0000 7f13bf7fe640 1 -- 172.21.15.46:0/3527382273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13b003f070 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.912+0000 7f13bf7fe640 1 -- 172.21.15.46:0/3527382273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13b0004cf0 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.912+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c80fd910 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.912+0000 7f13bf7fe640 1 -- 172.21.15.46:0/3527382273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13b0035030 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.912+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a3d40 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.913+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c8101f50 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.913+0000 7f13bf7fe640 1 -- 172.21.15.46:0/3527382273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f13b0042070 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.914+0000 7f13bf7fe640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f13b406de30 0x7f13b40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.914+0000 7f13bf7fe640 1 -- 172.21.15.46:0/3527382273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f13b00ae2e0 con 0x7f13c8106810 2024-09-10T04:01:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.914+0000 7f13c659c640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f13b406de30 0x7f13b40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.914+0000 7f13c659c640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f13b406de30 0x7f13b40702f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f13c0008e00 tx=0x7f13c0008350 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:47.918+0000 7f13bf7fe640 1 -- 172.21.15.46:0/3527382273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13b007c1c0 con 0x7f13c8106810 2024-09-10T04:01:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.094+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13c8109db0 con 0x7f13c8106810 2024-09-10T04:01:48.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.095+0000 7f13bf7fe640 1 -- 172.21.15.46:0/3527382273 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+11335 (secure 0 0 0) 0x7f13b00ae100 con 0x7f13c8106810 2024-09-10T04:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","created":"2024-09-10T03:55:32.552379+0000","modified":"2024-09-10T04:01:35.044377+0000","last_up_change":"2024-09-10T04:01:34.042619+0000","last_in_change":"2024-09-10T04:01:10.397117+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T04:00:16.212497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2edfb38-e5a1-4d6b-993a-66efc78d66f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6803","nonce":3291780356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6805","nonce":3291780356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6809","nonce":3291780356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6807","nonce":3291780356}]},"public_addr":"172.21.15.46:6803/3291780356","cluster_addr":"172.21.15.46:6805/3291780356","heartbeat_back_addr":"172.21.15.46:6809/3291780356","heartbeat_front_addr":"172.21.15.46:6807/3291780356","state":["exists","up"]},{"osd":1,"uuid":"1e2cbb84-8fbe-485f-8bc0-62da12e599c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6811","nonce":1584366844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6813","nonce":1584366844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6817","nonce":1584366844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6815","nonce":1584366844}]},"public_addr":"172.21.15.46:6811/1584366844","cluster_addr":"172.21.15.46:6813/1584366844","heartbeat_back_addr":"172.21.15.46:6817/1584366844","heartbeat_front_addr":"172.21.15.46:6815/1584366844","state":["exists","up"]},{"osd":2,"uuid":"346d282b-5119-44f6-8a94-5cf2f7ece643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6819","nonce":704152147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6821","nonce":704152147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6825","nonce":704152147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6823","nonce":704152147}]},"public_addr":"172.21.15.46:6819/704152147","cluster_addr":"172.21.15.46:6821/704152147","heartbeat_back_addr":"172.21.15.46:6825/704152147","heartbeat_front_addr":"172.21.15.46:6823/704152147","state":["exists","up"]},{"osd":3,"uuid":"503bec3a-eb26-42a1-973e-2c145bd7d10c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6801","nonce":578321990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6803","nonce":578321990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6807","nonce":578321990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6805","nonce":578321990}]},"public_addr":"172.21.15.70:6801/578321990","cluster_addr":"172.21.15.70:6803/578321990","heartbeat_back_addr":"172.21.15.70:6807/578321990","heartbeat_front_addr":"172.21.15.70:6805/578321990","state":["exists","up"]},{"osd":4,"uuid":"ba74b660-732c-4947-af9b-8f5d75e4da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6809","nonce":2152592034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6811","nonce":2152592034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6815","nonce":2152592034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6813","nonce":2152592034}]},"public_addr":"172.21.15.70:6809/2152592034","cluster_addr":"172.21.15.70:6811/2152592034","heartbeat_back_addr":"172.21.15.70:6815/2152592034","heartbeat_front_addr":"172.21.15.70:6813/2152592034","state":["exists","up"]},{"osd":5,"uuid":"e092e99e-0c0d-404b-9124-25da794f9eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6817","nonce":479912299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6819","nonce":479912299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6823","nonce":479912299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6821","nonce":479912299}]},"public_addr":"172.21.15.70:6817/479912299","cluster_addr":"172.21.15.70:6819/479912299","heartbeat_back_addr":"172.21.15.70:6823/479912299","heartbeat_front_addr":"172.21.15.70:6821/479912299","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:12.098977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:44.500164+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:13.640248+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:39.349187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:01:05.658570+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:01:32.135875+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2984295913":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/1609399519":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/4048021550":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/3785267557":"2024-09-11T03:56:03.190805+0000","172.21.15.46:6801/2101940292":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/3311822035":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/1023279878":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6801/1804840410":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/2504291949":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6801/653182070":"2024-09-11T03:56:35.886608+0000","172.21.15.46:0/1078369413":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6800/653182070":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6800/2101940292":"2024-09-11T03:57:33.701336+0000","172.21.15.46:6800/1804840410":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/2611013585":"2024-09-11T03:57:33.701336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f13b406de30 msgr2=0x7f13b40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f13b406de30 0x7f13b40702f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f13c0008e00 tx=0x7f13c0008350 comp rx=0 tx=0).stop 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8106810 msgr2=0x7f13c80fd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8106810 0x7f13c80fd140 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f13b0031460 tx=0x7f13b00048f0 comp rx=0 tx=0).stop 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 shutdown_connections 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13c8106810 0x7f13c80fd140 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f13b406de30 0x7f13b40702f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 --2- 172.21.15.46:0/3527382273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13c8100810 0x7f13c80fcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 >> 172.21.15.46:0/3527382273 conn(0x7f13c80fc510 msgr2=0x7f13c806a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 shutdown_connections 2024-09-10T04:01:48.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:48.097+0000 7f13ccf51640 1 -- 172.21.15.46:0/3527382273 wait complete. 2024-09-10T04:01:48.716 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:48 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3527382273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T04:01:48.746 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T04:01:48.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T04:01:48.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:48 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3527382273' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T04:01:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:49.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.493+0000 7f398ecd7640 1 -- 172.21.15.46:0/2191848116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988106800 msgr2=0x7f3988106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:49.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.493+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2191848116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988106800 0x7f3988106be0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3978009880 tx=0x7f397802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 -- 172.21.15.46:0/2191848116 shutdown_connections 2024-09-10T04:01:49.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2191848116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3988100800 0x7f3988100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2191848116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988106800 0x7f3988106be0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 -- 172.21.15.46:0/2191848116 >> 172.21.15.46:0/2191848116 conn(0x7f39880fc520 msgr2=0x7f39880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 -- 172.21.15.46:0/2191848116 shutdown_connections 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 -- 172.21.15.46:0/2191848116 wait complete. 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 Processor -- start 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 -- start start 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988100800 0x7f398819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3988106800 0x7f398819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f398810dd30 con 0x7f3988100800 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.494+0000 7f398ecd7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f398810dbb0 con 0x7f3988106800 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ca73640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988100800 0x7f398819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ca73640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988100800 0x7f398819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52152/0 (socket says 172.21.15.46:52152) 2024-09-10T04:01:49.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ca73640 1 -- 172.21.15.46:0/2870548579 learned_addr learned my addr 172.21.15.46:0/2870548579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f3987fff640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3988106800 0x7f398819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ca73640 1 -- 172.21.15.46:0/2870548579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3988106800 msgr2=0x7f398819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ca73640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3988106800 0x7f398819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ca73640 1 -- 172.21.15.46:0/2870548579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3988197820 con 0x7f3988100800 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ca73640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988100800 0x7f398819d6a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3978031460 tx=0x7f39780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f3985ffb640 1 -- 172.21.15.46:0/2870548579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f397803f070 con 0x7f3988100800 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3988197ab0 con 0x7f3988100800 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f3985ffb640 1 -- 172.21.15.46:0/2870548579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3978004cf0 con 0x7f3988100800 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.495+0000 7f3985ffb640 1 -- 172.21.15.46:0/2870548579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3978035030 con 0x7f3988100800 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.496+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3988197fe0 con 0x7f3988100800 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.496+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3988101f40 con 0x7f3988100800 2024-09-10T04:01:49.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.497+0000 7f3985ffb640 1 -- 172.21.15.46:0/2870548579 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f3978042070 con 0x7f3988100800 2024-09-10T04:01:49.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.497+0000 7f3985ffb640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f397006de30 0x7f39700702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:49.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.497+0000 7f3985ffb640 1 -- 172.21.15.46:0/2870548579 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f39780ae330 con 0x7f3988100800 2024-09-10T04:01:49.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.498+0000 7f3987fff640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f397006de30 0x7f39700702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:49.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.498+0000 7f3987fff640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f397006de30 0x7f39700702f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3988104f40 tx=0x7f397c008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:49.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.502+0000 7f3985ffb640 1 -- 172.21.15.46:0/2870548579 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f397807c210 con 0x7f3988100800 2024-09-10T04:01:49.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.682+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3988198d50 con 0x7f3988100800 2024-09-10T04:01:49.685 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:49 smithi046 ceph-mon[28328]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:49.685 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:49.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.683+0000 7f3985ffb640 1 -- 172.21.15.46:0/2870548579 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+11335 (secure 0 0 0) 0x7f39780ae150 con 0x7f3988100800 2024-09-10T04:01:49.686 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:01:49.686 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":38,"fsid":"628edc22-6f28-11ef-bcea-c7b262605968","created":"2024-09-10T03:55:32.552379+0000","modified":"2024-09-10T04:01:35.044377+0000","last_up_change":"2024-09-10T04:01:34.042619+0000","last_in_change":"2024-09-10T04:01:10.397117+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T04:00:16.212497+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c2edfb38-e5a1-4d6b-993a-66efc78d66f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6803","nonce":3291780356}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6805","nonce":3291780356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6809","nonce":3291780356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":3291780356},{"type":"v1","addr":"172.21.15.46:6807","nonce":3291780356}]},"public_addr":"172.21.15.46:6803/3291780356","cluster_addr":"172.21.15.46:6805/3291780356","heartbeat_back_addr":"172.21.15.46:6809/3291780356","heartbeat_front_addr":"172.21.15.46:6807/3291780356","state":["exists","up"]},{"osd":1,"uuid":"1e2cbb84-8fbe-485f-8bc0-62da12e599c3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6811","nonce":1584366844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6813","nonce":1584366844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6817","nonce":1584366844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1584366844},{"type":"v1","addr":"172.21.15.46:6815","nonce":1584366844}]},"public_addr":"172.21.15.46:6811/1584366844","cluster_addr":"172.21.15.46:6813/1584366844","heartbeat_back_addr":"172.21.15.46:6817/1584366844","heartbeat_front_addr":"172.21.15.46:6815/1584366844","state":["exists","up"]},{"osd":2,"uuid":"346d282b-5119-44f6-8a94-5cf2f7ece643","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6819","nonce":704152147}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6821","nonce":704152147}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6825","nonce":704152147}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":704152147},{"type":"v1","addr":"172.21.15.46:6823","nonce":704152147}]},"public_addr":"172.21.15.46:6819/704152147","cluster_addr":"172.21.15.46:6821/704152147","heartbeat_back_addr":"172.21.15.46:6825/704152147","heartbeat_front_addr":"172.21.15.46:6823/704152147","state":["exists","up"]},{"osd":3,"uuid":"503bec3a-eb26-42a1-973e-2c145bd7d10c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6801","nonce":578321990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6803","nonce":578321990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6807","nonce":578321990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":578321990},{"type":"v1","addr":"172.21.15.70:6805","nonce":578321990}]},"public_addr":"172.21.15.70:6801/578321990","cluster_addr":"172.21.15.70:6803/578321990","heartbeat_back_addr":"172.21.15.70:6807/578321990","heartbeat_front_addr":"172.21.15.70:6805/578321990","state":["exists","up"]},{"osd":4,"uuid":"ba74b660-732c-4947-af9b-8f5d75e4da14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6809","nonce":2152592034}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6811","nonce":2152592034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6815","nonce":2152592034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":2152592034},{"type":"v1","addr":"172.21.15.70:6813","nonce":2152592034}]},"public_addr":"172.21.15.70:6809/2152592034","cluster_addr":"172.21.15.70:6811/2152592034","heartbeat_back_addr":"172.21.15.70:6815/2152592034","heartbeat_front_addr":"172.21.15.70:6813/2152592034","state":["exists","up"]},{"osd":5,"uuid":"e092e99e-0c0d-404b-9124-25da794f9eed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6817","nonce":479912299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6819","nonce":479912299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6823","nonce":479912299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":479912299},{"type":"v1","addr":"172.21.15.70:6821","nonce":479912299}]},"public_addr":"172.21.15.70:6817/479912299","cluster_addr":"172.21.15.70:6819/479912299","heartbeat_back_addr":"172.21.15.70:6823/479912299","heartbeat_front_addr":"172.21.15.70:6821/479912299","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:12.098977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:44.500164+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:13.640248+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:39.349187+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:01:05.658570+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:01:32.135875+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/2984295913":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/1609399519":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/4048021550":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/3785267557":"2024-09-11T03:56:03.190805+0000","172.21.15.46:6801/2101940292":"2024-09-11T03:57:33.701336+0000","172.21.15.46:0/3311822035":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/1023279878":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6801/1804840410":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/2504291949":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6801/653182070":"2024-09-11T03:56:35.886608+0000","172.21.15.46:0/1078369413":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6800/653182070":"2024-09-11T03:56:35.886608+0000","172.21.15.46:6800/2101940292":"2024-09-11T03:57:33.701336+0000","172.21.15.46:6800/1804840410":"2024-09-11T03:56:03.190805+0000","172.21.15.46:0/2611013585":"2024-09-11T03:57:33.701336+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T04:01:49.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f397006de30 msgr2=0x7f39700702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f397006de30 0x7f39700702f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3988104f40 tx=0x7f397c008040 comp rx=0 tx=0).stop 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988100800 msgr2=0x7f398819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988100800 0x7f398819d6a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3978031460 tx=0x7f39780048f0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 shutdown_connections 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f397006de30 0x7f39700702f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3988106800 0x7f398819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 --2- 172.21.15.46:0/2870548579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3988100800 0x7f398819d6a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 >> 172.21.15.46:0/2870548579 conn(0x7f39880fc520 msgr2=0x7f398806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 shutdown_connections 2024-09-10T04:01:49.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:49.686+0000 7f398ecd7640 1 -- 172.21.15.46:0/2870548579 wait complete. 2024-09-10T04:01:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:49 smithi070 ceph-mon[35495]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:01:50.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T04:01:50.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T04:01:50.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T04:01:50.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T04:01:50.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T04:01:50.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T04:01:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:50 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2870548579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T04:01:50.640 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:50.651 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:50.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:50 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2870548579' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T04:01:51.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.286+0000 7f2de1eac640 1 -- 172.21.15.46:0/666011905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc107900 msgr2=0x7f2ddc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.286+0000 7f2de1eac640 1 --2- 172.21.15.46:0/666011905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc107900 0x7f2ddc107ce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009880 tx=0x7f2dcc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.286+0000 7f2de1eac640 1 -- 172.21.15.46:0/666011905 shutdown_connections 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.286+0000 7f2de1eac640 1 --2- 172.21.15.46:0/666011905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ddc101900 0x7f2ddc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.286+0000 7f2de1eac640 1 --2- 172.21.15.46:0/666011905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc107900 0x7f2ddc107ce0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.286+0000 7f2de1eac640 1 -- 172.21.15.46:0/666011905 >> 172.21.15.46:0/666011905 conn(0x7f2ddc0fd600 msgr2=0x7f2ddc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.287+0000 7f2de1eac640 1 -- 172.21.15.46:0/666011905 shutdown_connections 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.287+0000 7f2de1eac640 1 -- 172.21.15.46:0/666011905 wait complete. 2024-09-10T04:01:51.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.287+0000 7f2de1eac640 1 Processor -- start 2024-09-10T04:01:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.287+0000 7f2de1eac640 1 -- start start 2024-09-10T04:01:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2de1eac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc101900 0x7f2ddc110b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2de1eac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ddc107900 0x7f2ddc111040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2de1eac640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc075850 con 0x7f2ddc101900 2024-09-10T04:01:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2de1eac640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2ddc0756d0 con 0x7f2ddc107900 2024-09-10T04:01:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc101900 0x7f2ddc110b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc101900 0x7f2ddc110b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52172/0 (socket says 172.21.15.46:52172) 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddb7fe640 1 -- 172.21.15.46:0/1185501643 learned_addr learned my addr 172.21.15.46:0/1185501643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddaffd640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ddc107900 0x7f2ddc111040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddb7fe640 1 -- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ddc107900 msgr2=0x7f2ddc111040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddb7fe640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ddc107900 0x7f2ddc111040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddb7fe640 1 -- 172.21.15.46:0/1185501643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ddc1a2910 con 0x7f2ddc101900 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2ddb7fe640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc101900 0x7f2ddc110b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc031460 tx=0x7f2dcc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.288+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2dcc03f070 con 0x7f2ddc101900 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.289+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ddc1a2ba0 con 0x7f2ddc101900 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.289+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2dcc004cf0 con 0x7f2ddc101900 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.289+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2dcc035030 con 0x7f2ddc101900 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.289+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ddc1a30d0 con 0x7f2ddc101900 2024-09-10T04:01:51.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.289+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ddc1a3690 con 0x7f2ddc101900 2024-09-10T04:01:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.290+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f2dcc042070 con 0x7f2ddc101900 2024-09-10T04:01:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2dd8ff9640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2dc406de30 0x7f2dc40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2ddaffd640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2dc406de30 0x7f2dc40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f2dcc0ae330 con 0x7f2ddc101900 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2dd8ff9640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] conn(0x7f2dc4073c20 0x7f2dc4076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 --> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2dc40766f0 con 0x7f2dc4073c20 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2dcc0b5050 con 0x7f2ddc101900 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2ddaffd640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2dc406de30 0x7f2dc40702f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc1097b0 tx=0x7f2dd0008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.291+0000 7f2ddbfff640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] conn(0x7f2dc4073c20 0x7f2dc4076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.292+0000 7f2ddbfff640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] conn(0x7f2dc4073c20 0x7f2dc4076040 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:51.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.292+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== osd.3 v2:172.21.15.70:6800/578321990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2dc40766f0 con 0x7f2dc4073c20 2024-09-10T04:01:51.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.306+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 --> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ddc10c230 con 0x7f2dc4073c20 2024-09-10T04:01:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.306+0000 7f2dd8ff9640 1 -- 172.21.15.46:0/1185501643 <== osd.3 v2:172.21.15.70:6800/578321990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ddc10c230 con 0x7f2dc4073c20 2024-09-10T04:01:51.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] conn(0x7f2dc4073c20 msgr2=0x7f2dc4076040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] conn(0x7f2dc4073c20 0x7f2dc4076040 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2dc406de30 msgr2=0x7f2dc40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2dc406de30 0x7f2dc40702f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc1097b0 tx=0x7f2dd0008040 comp rx=0 tx=0).stop 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc101900 msgr2=0x7f2ddc110b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc101900 0x7f2ddc110b00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc031460 tx=0x7f2dcc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 shutdown_connections 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:6800/578321990,v1:172.21.15.70:6801/578321990] conn(0x7f2dc4073c20 0x7f2dc4076040 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2dc406de30 0x7f2dc40702f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ddc107900 0x7f2ddc111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 --2- 172.21.15.46:0/1185501643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc101900 0x7f2ddc110b00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 >> 172.21.15.46:0/1185501643 conn(0x7f2ddc0fd600 msgr2=0x7f2ddc10b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:51.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 shutdown_connections 2024-09-10T04:01:51.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.307+0000 7f2de1eac640 1 -- 172.21.15.46:0/1185501643 wait complete. 2024-09-10T04:01:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.485+0000 7f4f68386640 1 -- 172.21.15.46:0/3795597407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60102b70 msgr2=0x7f4f60102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.485+0000 7f4f68386640 1 --2- 172.21.15.46:0/3795597407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60102b70 0x7f4f60102f50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4f58009880 tx=0x7f4f5802a530 comp rx=0 tx=0).stop 2024-09-10T04:01:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 -- 172.21.15.46:0/3795597407 shutdown_connections 2024-09-10T04:01:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 --2- 172.21.15.46:0/3795597407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f60103490 0x7f4f6010b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 --2- 172.21.15.46:0/3795597407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60102b70 0x7f4f60102f50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 -- 172.21.15.46:0/3795597407 >> 172.21.15.46:0/3795597407 conn(0x7f4f600fc550 msgr2=0x7f4f600fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 -- 172.21.15.46:0/3795597407 shutdown_connections 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 -- 172.21.15.46:0/3795597407 wait complete. 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 Processor -- start 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.486+0000 7f4f68386640 1 -- start start 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f68386640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f60102b70 0x7f4f6019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f68386640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60103490 0x7f4f6019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f68386640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6010ede0 con 0x7f4f60103490 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f68386640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6010ec60 con 0x7f4f60102b70 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f65921640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60103490 0x7f4f6019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f65921640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60103490 0x7f4f6019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52186/0 (socket says 172.21.15.46:52186) 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f65921640 1 -- 172.21.15.46:0/1716962396 learned_addr learned my addr 172.21.15.46:0/1716962396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f66122640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f60102b70 0x7f4f6019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f65921640 1 -- 172.21.15.46:0/1716962396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f60102b70 msgr2=0x7f4f6019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f65921640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f60102b70 0x7f4f6019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f65921640 1 -- 172.21.15.46:0/1716962396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f601977b0 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f65921640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60103490 0x7f4f6019db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400d6f0 tx=0x7f4f5400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f54017070 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f5400de00 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f60197aa0 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.487+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f54004bb0 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.488+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f60198030 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.488+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f601985f0 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.489+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f4f54004e60 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.489+0000 7f4f5f7fe640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f4806de30 0x7f4f480702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f4f5408d430 con 0x7f4f60103490 2024-09-10T04:01:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f66122640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f4806de30 0x7f4f480702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f5f7fe640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] conn(0x7f4f48073c20 0x7f4f48076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 --> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f480766f0 con 0x7f4f48073c20 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4f54094050 con 0x7f4f60103490 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f66923640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] conn(0x7f4f48073c20 0x7f4f48076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f66122640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f4806de30 0x7f4f480702f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f580096c0 tx=0x7f4f58004f30 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.490+0000 7f4f66923640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] conn(0x7f4f48073c20 0x7f4f48076040 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.491+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== osd.1 v2:172.21.15.46:6810/1584366844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f480766f0 con 0x7f4f48073c20 2024-09-10T04:01:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.504+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 --> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f600628b0 con 0x7f4f48073c20 2024-09-10T04:01:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f5f7fe640 1 -- 172.21.15.46:0/1716962396 <== osd.1 v2:172.21.15.46:6810/1584366844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f600628b0 con 0x7f4f48073c20 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] conn(0x7f4f48073c20 msgr2=0x7f4f48076040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] conn(0x7f4f48073c20 0x7f4f48076040 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f4806de30 msgr2=0x7f4f480702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f4806de30 0x7f4f480702f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4f580096c0 tx=0x7f4f58004f30 comp rx=0 tx=0).stop 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60103490 msgr2=0x7f4f6019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60103490 0x7f4f6019db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400d6f0 tx=0x7f4f5400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 shutdown_connections 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6810/1584366844,v1:172.21.15.46:6811/1584366844] conn(0x7f4f48073c20 0x7f4f48076040 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f60103490 0x7f4f6019db70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f4806de30 0x7f4f480702f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 --2- 172.21.15.46:0/1716962396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f60102b70 0x7f4f6019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 >> 172.21.15.46:0/1716962396 conn(0x7f4f600fc550 msgr2=0x7f4f60108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 shutdown_connections 2024-09-10T04:01:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:51.505+0000 7f4f68386640 1 -- 172.21.15.46:0/1716962396 wait complete. 2024-09-10T04:01:51.747 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:51 smithi046 ceph-mon[28328]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:51.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:51 smithi070 ceph-mon[35495]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:51.986 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:52.476 INFO:teuthology.orchestra.run.smithi046.stdout:107374182417 2024-09-10T04:01:52.476 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T04:01:52.902 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:52.902 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:52.941 INFO:teuthology.orchestra.run.smithi046.stdout:60129542172 2024-09-10T04:01:52.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T04:01:53.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.399+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/1609510563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 msgr2=0x7f9b4810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:53.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.399+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/1609510563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4810b8d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009880 tx=0x7f9b3c02a530 comp rx=0 tx=0).stop 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.399+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/1609510563 shutdown_connections 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.399+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/1609510563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4810b8d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.400+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/1609510563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b48102ac0 0x7f9b48102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.400+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/1609510563 >> 172.21.15.46:0/1609510563 conn(0x7f9b480fc520 msgr2=0x7f9b480fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.400+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/1609510563 shutdown_connections 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.400+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/1609510563 wait complete. 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.400+0000 7f9b4d8ac640 1 Processor -- start 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b4d8ac640 1 -- start start 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b4d8ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b48102ac0 0x7f9b4819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b4d8ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b4d8ac640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810edd0 con 0x7f9b481033e0 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b4d8ac640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4810ec50 con 0x7f9b48102ac0 2024-09-10T04:01:53.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b467fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52198/0 (socket says 172.21.15.46:52198) 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b467fc640 1 -- 172.21.15.46:0/82918053 learned_addr learned my addr 172.21.15.46:0/82918053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b46ffd640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b48102ac0 0x7f9b4819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b467fc640 1 -- 172.21.15.46:0/82918053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b48102ac0 msgr2=0x7f9b4819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b467fc640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b48102ac0 0x7f9b4819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.401+0000 7f9b467fc640 1 -- 172.21.15.46:0/82918053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b481977e0 con 0x7f9b481033e0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.402+0000 7f9b467fc640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4819dba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c031460 tx=0x7f9b3c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.402+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c03f070 con 0x7f9b481033e0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.402+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b3c0035f0 con 0x7f9b481033e0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.402+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b48197a70 con 0x7f9b481033e0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.402+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b3c035030 con 0x7f9b481033e0 2024-09-10T04:01:53.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.403+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b48197fa0 con 0x7f9b481033e0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.403+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f9b3c042070 con 0x7f9b481033e0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.403+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b48198510 con 0x7f9b481033e0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.404+0000 7f9b4c8aa640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9b3006de30 0x7f9b300702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.404+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f9b3c0af8d0 con 0x7f9b481033e0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.404+0000 7f9b46ffd640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9b3006de30 0x7f9b300702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.404+0000 7f9b4c8aa640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] conn(0x7f9b30073c20 0x7f9b30076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.404+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 --> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b3c0054e0 con 0x7f9b30073c20 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.404+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9b3c0b3060 con 0x7f9b481033e0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.404+0000 7f9b477fe640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] conn(0x7f9b30073c20 0x7f9b30076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.405+0000 7f9b46ffd640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9b3006de30 0x7f9b300702f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b38005fd0 tx=0x7f9b38005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.405+0000 7f9b477fe640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] conn(0x7f9b30073c20 0x7f9b30076040 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:53.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.406+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== osd.2 v2:172.21.15.46:6818/704152147 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b3c0054e0 con 0x7f9b30073c20 2024-09-10T04:01:53.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.433+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 --> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b48062900 con 0x7f9b30073c20 2024-09-10T04:01:53.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.433+0000 7f9b4c8aa640 1 -- 172.21.15.46:0/82918053 <== osd.2 v2:172.21.15.46:6818/704152147 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9b48062900 con 0x7f9b30073c20 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] conn(0x7f9b30073c20 msgr2=0x7f9b30076040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] conn(0x7f9b30073c20 0x7f9b30076040 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9b3006de30 msgr2=0x7f9b300702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9b3006de30 0x7f9b300702f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9b38005fd0 tx=0x7f9b38005850 comp rx=0 tx=0).stop 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 msgr2=0x7f9b4819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4819dba0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c031460 tx=0x7f9b3c004530 comp rx=0 tx=0).stop 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 shutdown_connections 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6818/704152147,v1:172.21.15.46:6819/704152147] conn(0x7f9b30073c20 0x7f9b30076040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b481033e0 0x7f9b4819dba0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9b3006de30 0x7f9b300702f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 --2- 172.21.15.46:0/82918053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b48102ac0 0x7f9b4819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 >> 172.21.15.46:0/82918053 conn(0x7f9b480fc520 msgr2=0x7f9b48108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 shutdown_connections 2024-09-10T04:01:53.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:53.434+0000 7f9b4d8ac640 1 -- 172.21.15.46:0/82918053 wait complete. 2024-09-10T04:01:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:53 smithi046 ceph-mon[28328]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:53.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:53 smithi070 ceph-mon[35495]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:54.032 INFO:teuthology.orchestra.run.smithi046.stdout:81604378646 2024-09-10T04:01:54.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 -- 172.21.15.46:0/1914386414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8100860 msgr2=0x7f14e8100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1914386414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8100860 0x7f14e8100cc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009880 tx=0x7f14d802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 -- 172.21.15.46:0/1914386414 shutdown_connections 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1914386414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8100860 0x7f14e8100cc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1914386414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14e8106860 0x7f14e8106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 -- 172.21.15.46:0/1914386414 >> 172.21.15.46:0/1914386414 conn(0x7f14e80fc580 msgr2=0x7f14e80fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 -- 172.21.15.46:0/1914386414 shutdown_connections 2024-09-10T04:01:54.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.642+0000 7f14ef26b640 1 -- 172.21.15.46:0/1914386414 wait complete. 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ef26b640 1 Processor -- start 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ef26b640 1 -- start start 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ef26b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14e8100860 0x7f14e8195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ef26b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8106860 0x7f14e8195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ef26b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f14e810dd30 con 0x7f14e8106860 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ef26b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f14e810dbb0 con 0x7f14e8100860 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ec806640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8106860 0x7f14e8195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ec806640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8106860 0x7f14e8195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52206/0 (socket says 172.21.15.46:52206) 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ec806640 1 -- 172.21.15.46:0/1813776983 learned_addr learned my addr 172.21.15.46:0/1813776983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.643+0000 7f14ed007640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14e8100860 0x7f14e8195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:54.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14ec806640 1 -- 172.21.15.46:0/1813776983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14e8100860 msgr2=0x7f14e8195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14ec806640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14e8100860 0x7f14e8195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14ec806640 1 -- 172.21.15.46:0/1813776983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e81961b0 con 0x7f14e8106860 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14ed007640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14e8100860 0x7f14e8195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14ec806640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8106860 0x7f14e8195b60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14d8031460 tx=0x7f14d80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14de7fc640 1 -- 172.21.15.46:0/1813776983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14d803f070 con 0x7f14e8106860 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14de7fc640 1 -- 172.21.15.46:0/1813776983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14d8004b10 con 0x7f14e8106860 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e8196440 con 0x7f14e8106860 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14de7fc640 1 -- 172.21.15.46:0/1813776983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f14d8035030 con 0x7f14e8106860 2024-09-10T04:01:54.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.644+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e81a8200 con 0x7f14e8106860 2024-09-10T04:01:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.646+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e8101fa0 con 0x7f14e8106860 2024-09-10T04:01:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.647+0000 7f14de7fc640 1 -- 172.21.15.46:0/1813776983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f14d8042070 con 0x7f14e8106860 2024-09-10T04:01:54.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.647+0000 7f14de7fc640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f14d406de30 0x7f14d40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.647+0000 7f14de7fc640 1 -- 172.21.15.46:0/1813776983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f14d80ae2c0 con 0x7f14e8106860 2024-09-10T04:01:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.647+0000 7f14ed007640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f14d406de30 0x7f14d40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.648+0000 7f14ed007640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f14d406de30 0x7f14d40702f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005fd0 tx=0x7f14e400b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.652+0000 7f14de7fc640 1 -- 172.21.15.46:0/1813776983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14d807fe50 con 0x7f14e8106860 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 -- 172.21.15.46:0/4006482231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7801033d0 msgr2=0x7fd78010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 --2- 172.21.15.46:0/4006482231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7801033d0 0x7fd78010b8c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd774009880 tx=0x7fd77402a530 comp rx=0 tx=0).stop 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 -- 172.21.15.46:0/4006482231 shutdown_connections 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 --2- 172.21.15.46:0/4006482231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7801033d0 0x7fd78010b8c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 --2- 172.21.15.46:0/4006482231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd780102ab0 0x7fd780102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 -- 172.21.15.46:0/4006482231 >> 172.21.15.46:0/4006482231 conn(0x7fd7800fc550 msgr2=0x7fd7800fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 -- 172.21.15.46:0/4006482231 shutdown_connections 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.822+0000 7fd785210640 1 -- 172.21.15.46:0/4006482231 wait complete. 2024-09-10T04:01:54.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.823+0000 7fd785210640 1 Processor -- start 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.823+0000 7fd785210640 1 -- start start 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd785210640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd780102ab0 0x7fd780100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd785210640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7801033d0 0x7fd780100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd785210640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010edc0 con 0x7fd780102ab0 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd785210640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd78010ec40 con 0x7fd7801033d0 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd77ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd780102ab0 0x7fd780100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd77ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd780102ab0 0x7fd780100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52234/0 (socket says 172.21.15.46:52234) 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd77ed9d640 1 -- 172.21.15.46:0/1478921537 learned_addr learned my addr 172.21.15.46:0/1478921537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:54.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd77e59c640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7801033d0 0x7fd780100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd77ed9d640 1 -- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7801033d0 msgr2=0x7fd780100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd77ed9d640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7801033d0 0x7fd780100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.824+0000 7fd77ed9d640 1 -- 172.21.15.46:0/1478921537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd780101190 con 0x7fd780102ab0 2024-09-10T04:01:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.825+0000 7fd77ed9d640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd780102ab0 0x7fd780100710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd77000c8a0 tx=0x7fd77000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.825+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd7700079b0 con 0x7fd780102ab0 2024-09-10T04:01:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.825+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd770007b50 con 0x7fd780102ab0 2024-09-10T04:01:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.825+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd780101480 con 0x7fd780102ab0 2024-09-10T04:01:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.825+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd7700186c0 con 0x7fd780102ab0 2024-09-10T04:01:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.825+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7801a3db0 con 0x7fd780102ab0 2024-09-10T04:01:54.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.826+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd78010b850 con 0x7fd780102ab0 2024-09-10T04:01:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.827+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fd77000f450 con 0x7fd780102ab0 2024-09-10T04:01:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.828+0000 7fd767fff640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd76806de90 0x7fd768070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.828+0000 7fd77e59c640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd76806de90 0x7fd768070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:54.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.829+0000 7fd77e59c640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd76806de90 0x7fd768070350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7801021a0 tx=0x7fd774031400 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.829+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fd77008d940 con 0x7fd780102ab0 2024-09-10T04:01:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.829+0000 7fd767fff640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] conn(0x7fd768073c80 0x7fd7680760a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.829+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 --> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd770012ec0 con 0x7fd768073c80 2024-09-10T04:01:54.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.829+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd7700922a0 con 0x7fd780102ab0 2024-09-10T04:01:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.829+0000 7fd77f59e640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] conn(0x7fd768073c80 0x7fd7680760a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.830+0000 7fd77f59e640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] conn(0x7fd768073c80 0x7fd7680760a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:54.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.831+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== osd.4 v2:172.21.15.70:6808/2152592034 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd770012ec0 con 0x7fd768073c80 2024-09-10T04:01:54.833 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.832+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f14e81a8480 con 0x7f14e8106860 2024-09-10T04:01:54.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.833+0000 7f14de7fc640 1 -- 172.21.15.46:0/1813776983 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f14d8084680 con 0x7f14e8106860 2024-09-10T04:01:54.834 INFO:teuthology.orchestra.run.smithi046.stdout:60129542173 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f14d406de30 msgr2=0x7f14d40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f14d406de30 0x7f14d40702f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f14e4005fd0 tx=0x7f14e400b390 comp rx=0 tx=0).stop 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8106860 msgr2=0x7f14e8195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8106860 0x7f14e8195b60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14d8031460 tx=0x7f14d80045e0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 shutdown_connections 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14e8106860 0x7f14e8195b60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f14d406de30 0x7f14d40702f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 --2- 172.21.15.46:0/1813776983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14e8100860 0x7f14e8195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 >> 172.21.15.46:0/1813776983 conn(0x7f14e80fc580 msgr2=0x7f14e80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 shutdown_connections 2024-09-10T04:01:54.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.836+0000 7f14ef26b640 1 -- 172.21.15.46:0/1813776983 wait complete. 2024-09-10T04:01:54.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.857+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 --> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd78006a580 con 0x7fd768073c80 2024-09-10T04:01:54.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.857+0000 7fd767fff640 1 -- 172.21.15.46:0/1478921537 <== osd.4 v2:172.21.15.70:6808/2152592034 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd78006a580 con 0x7fd768073c80 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] conn(0x7fd768073c80 msgr2=0x7fd7680760a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] conn(0x7fd768073c80 0x7fd7680760a0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd76806de90 msgr2=0x7fd768070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd76806de90 0x7fd768070350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7801021a0 tx=0x7fd774031400 comp rx=0 tx=0).stop 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd780102ab0 msgr2=0x7fd780100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd780102ab0 0x7fd780100710 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd77000c8a0 tx=0x7fd77000cd60 comp rx=0 tx=0).stop 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 shutdown_connections 2024-09-10T04:01:54.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:6808/2152592034,v1:172.21.15.70:6809/2152592034] conn(0x7fd768073c80 0x7fd7680760a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd76806de90 0x7fd768070350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7801033d0 0x7fd780100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 --2- 172.21.15.46:0/1478921537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd780102ab0 0x7fd780100710 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:54.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 >> 172.21.15.46:0/1478921537 conn(0x7fd7800fc550 msgr2=0x7fd78010aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:54.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 shutdown_connections 2024-09-10T04:01:54.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:54.858+0000 7fd785210640 1 -- 172.21.15.46:0/1478921537 wait complete. 2024-09-10T04:01:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 -- 172.21.15.46:0/3877916500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 msgr2=0x7f89e4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/3877916500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e4101d60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009880 tx=0x7f89d802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 -- 172.21.15.46:0/3877916500 shutdown_connections 2024-09-10T04:01:55.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/3877916500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e4101d60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/3877916500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e4107900 0x7f89e4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 -- 172.21.15.46:0/3877916500 >> 172.21.15.46:0/3877916500 conn(0x7f89e40fd600 msgr2=0x7f89e40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 -- 172.21.15.46:0/3877916500 shutdown_connections 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.521+0000 7f89e8a1e640 1 -- 172.21.15.46:0/3877916500 wait complete. 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e8a1e640 1 Processor -- start 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e8a1e640 1 -- start start 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e8a1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e8a1e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e4107900 0x7f89e419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e8a1e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89e410edd0 con 0x7f89e4101900 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e8a1e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f89e410ec50 con 0x7f89e4107900 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52246/0 (socket says 172.21.15.46:52246) 2024-09-10T04:01:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e259c640 1 -- 172.21.15.46:0/346001723 learned_addr learned my addr 172.21.15.46:0/346001723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.522+0000 7f89e1d9b640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e4107900 0x7f89e419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89e259c640 1 -- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e4107900 msgr2=0x7f89e419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89e259c640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e4107900 0x7f89e419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89e259c640 1 -- 172.21.15.46:0/346001723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e41999d0 con 0x7f89e4101900 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89e1d9b640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e4107900 0x7f89e419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89e259c640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e419f810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c8f0 tx=0x7f89d400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f89d40079b0 con 0x7f89e4101900 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e4199cc0 con 0x7f89e4101900 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f89d40054e0 con 0x7f89e4101900 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f89d4004dd0 con 0x7f89e4101900 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.523+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e419a250 con 0x7f89e4101900 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.524+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89e4103040 con 0x7f89e4101900 2024-09-10T04:01:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.524+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f89d4005020 con 0x7f89e4101900 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89d37fe640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89cc06dde0 0x7f89cc0702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f89d408d810 con 0x7f89e4101900 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89e1d9b640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89cc06dde0 0x7f89cc0702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89d37fe640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] conn(0x7f89cc073bd0 0x7f89cc075ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 --> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89d4004490 con 0x7f89cc073bd0 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f89d40922a0 con 0x7f89e4101900 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89e1d9b640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89cc06dde0 0x7f89cc0702a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f89d8031460 tx=0x7f89d80095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.525+0000 7f89e2d9d640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] conn(0x7f89cc073bd0 0x7f89cc075ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.526+0000 7f89e2d9d640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] conn(0x7f89cc073bd0 0x7f89cc075ff0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:55.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.527+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== osd.5 v2:172.21.15.70:6816/479912299 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89d4004490 con 0x7f89cc073bd0 2024-09-10T04:01:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.540+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 --> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f89e4074dd0 con 0x7f89cc073bd0 2024-09-10T04:01:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89d37fe640 1 -- 172.21.15.46:0/346001723 <== osd.5 v2:172.21.15.70:6816/479912299 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f89e4074dd0 con 0x7f89cc073bd0 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] conn(0x7f89cc073bd0 msgr2=0x7f89cc075ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] conn(0x7f89cc073bd0 0x7f89cc075ff0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89cc06dde0 msgr2=0x7f89cc0702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89cc06dde0 0x7f89cc0702a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f89d8031460 tx=0x7f89d80095f0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 msgr2=0x7f89e419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e419f810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c8f0 tx=0x7f89d400cdb0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 shutdown_connections 2024-09-10T04:01:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:6816/479912299,v1:172.21.15.70:6817/479912299] conn(0x7f89cc073bd0 0x7f89cc075ff0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f89cc06dde0 0x7f89cc0702a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89e4107900 0x7f89e419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 --2- 172.21.15.46:0/346001723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89e4101900 0x7f89e419f810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 >> 172.21.15.46:0/346001723 conn(0x7f89e40fd600 msgr2=0x7f89e4074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 shutdown_connections 2024-09-10T04:01:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:55.541+0000 7f89e8a1e640 1 -- 172.21.15.46:0/346001723 wait complete. 2024-09-10T04:01:55.688 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:55 smithi046 ceph-mon[28328]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:55.689 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:55 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1813776983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T04:01:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:55 smithi070 ceph-mon[35495]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:55 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1813776983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T04:01:56.473 INFO:teuthology.orchestra.run.smithi046.stdout:158913789960 2024-09-10T04:01:56.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T04:01:56.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 -- 172.21.15.46:0/2774255928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 msgr2=0x7fe2f8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2774255928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f8102ea0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec009880 tx=0x7fe2ec02a530 comp rx=0 tx=0).stop 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 -- 172.21.15.46:0/2774255928 shutdown_connections 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2774255928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f81033e0 0x7fe2f810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2774255928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f8102ea0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 -- 172.21.15.46:0/2774255928 >> 172.21.15.46:0/2774255928 conn(0x7fe2f80fc520 msgr2=0x7fe2f80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 -- 172.21.15.46:0/2774255928 shutdown_connections 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.030+0000 7fe2ff157640 1 -- 172.21.15.46:0/2774255928 wait complete. 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2ff157640 1 Processor -- start 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2ff157640 1 -- start start 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2ff157640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2ff157640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f81033e0 0x7fe2f819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:56.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2ff157640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f810edf0 con 0x7fe2f8102ac0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2ff157640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f810ec70 con 0x7fe2f81033e0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2fcef3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2fcef3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52266/0 (socket says 172.21.15.46:52266) 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2fcef3640 1 -- 172.21.15.46:0/2278705508 learned_addr learned my addr 172.21.15.46:0/2278705508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2f7fff640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f81033e0 0x7fe2f819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2fcef3640 1 -- 172.21.15.46:0/2278705508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f81033e0 msgr2=0x7fe2f819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2fcef3640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f81033e0 0x7fe2f819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2fcef3640 1 -- 172.21.15.46:0/2278705508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f81977c0 con 0x7fe2f8102ac0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.031+0000 7fe2f7fff640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f81033e0 0x7fe2f819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.032+0000 7fe2fcef3640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f819d640 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec031460 tx=0x7fe2ec0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.032+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe2ec03f070 con 0x7fe2f8102ac0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.032+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f8197a50 con 0x7fe2f8102ac0 2024-09-10T04:01:56.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.032+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2ec0035f0 con 0x7fe2f8102ac0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.032+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe2ec035030 con 0x7fe2f8102ac0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.032+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f8197f80 con 0x7fe2f8102ac0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.033+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2f8198600 con 0x7fe2f8102ac0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fe2ec042070 con 0x7fe2f8102ac0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2f5ffb640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe2e806de30 0x7fe2e80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fe2ec0ae210 con 0x7fe2f8102ac0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2f7fff640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe2e806de30 0x7fe2e80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2f5ffb640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] conn(0x7fe2e8073c20 0x7fe2e8076040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 --> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2ec0054e0 con 0x7fe2e8073c20 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fe2ec0b5050 con 0x7fe2f8102ac0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.034+0000 7fe2fd6f4640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] conn(0x7fe2e8073c20 0x7fe2e8076040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:56.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.035+0000 7fe2f7fff640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe2e806de30 0x7fe2e80702f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4005fd0 tx=0x7fe2e4005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.035+0000 7fe2fd6f4640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] conn(0x7fe2e8073c20 0x7fe2e8076040 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.035+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== osd.0 v2:172.21.15.46:6802/3291780356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2ec0054e0 con 0x7fe2e8073c20 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.049+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 --> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2f810b8c0 con 0x7fe2e8073c20 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.049+0000 7fe2f5ffb640 1 -- 172.21.15.46:0/2278705508 <== osd.0 v2:172.21.15.46:6802/3291780356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe2f810b8c0 con 0x7fe2e8073c20 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.049+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] conn(0x7fe2e8073c20 msgr2=0x7fe2e8076040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.049+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] conn(0x7fe2e8073c20 0x7fe2e8076040 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe2e806de30 msgr2=0x7fe2e80702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:56.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe2e806de30 0x7fe2e80702f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4005fd0 tx=0x7fe2e4005850 comp rx=0 tx=0).stop 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 msgr2=0x7fe2f819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f819d640 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec031460 tx=0x7fe2ec0045e0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 shutdown_connections 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6802/3291780356,v1:172.21.15.46:6803/3291780356] conn(0x7fe2e8073c20 0x7fe2e8076040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe2e806de30 0x7fe2e80702f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f81033e0 0x7fe2f819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 --2- 172.21.15.46:0/2278705508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f8102ac0 0x7fe2f819d640 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 >> 172.21.15.46:0/2278705508 conn(0x7fe2f80fc520 msgr2=0x7fe2f810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:56.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 shutdown_connections 2024-09-10T04:01:56.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:56.050+0000 7fe2ff157640 1 -- 172.21.15.46:0/2278705508 wait complete. 2024-09-10T04:01:57.394 INFO:teuthology.orchestra.run.smithi046.stdout:137438953484 2024-09-10T04:01:57.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T04:01:57.642 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:57 smithi046 ceph-mon[28328]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:57 smithi070 ceph-mon[35495]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:58.065 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542173 for osd.1 2024-09-10T04:01:58.065 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:58.401 INFO:teuthology.orchestra.run.smithi046.stdout:38654705700 2024-09-10T04:01:58.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T04:01:58.589 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 -- 172.21.15.46:0/3248174384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 msgr2=0x7fa0b4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 --2- 172.21.15.46:0/3248174384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b4102e90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009880 tx=0x7fa0a402a530 comp rx=0 tx=0).stop 2024-09-10T04:01:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 -- 172.21.15.46:0/3248174384 shutdown_connections 2024-09-10T04:01:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 --2- 172.21.15.46:0/3248174384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b41033d0 0x7fa0b410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:58.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 --2- 172.21.15.46:0/3248174384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b4102e90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 -- 172.21.15.46:0/3248174384 >> 172.21.15.46:0/3248174384 conn(0x7fa0b40fc4d0 msgr2=0x7fa0b40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 -- 172.21.15.46:0/3248174384 shutdown_connections 2024-09-10T04:01:58.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.597+0000 7fa0b8991640 1 -- 172.21.15.46:0/3248174384 wait complete. 2024-09-10T04:01:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.598+0000 7fa0b8991640 1 Processor -- start 2024-09-10T04:01:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.598+0000 7fa0b8991640 1 -- start start 2024-09-10T04:01:58.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b8991640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b8991640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b41033d0 0x7fa0b419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b8991640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410edc0 con 0x7fa0b4102ab0 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b8991640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b410ec40 con 0x7fa0b41033d0 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52290/0 (socket says 172.21.15.46:52290) 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b259c640 1 -- 172.21.15.46:0/1982408747 learned_addr learned my addr 172.21.15.46:0/1982408747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b1d9b640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b41033d0 0x7fa0b419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b259c640 1 -- 172.21.15.46:0/1982408747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b41033d0 msgr2=0x7fa0b419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b259c640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b41033d0 0x7fa0b419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b259c640 1 -- 172.21.15.46:0/1982408747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41977b0 con 0x7fa0b4102ab0 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.599+0000 7fa0b1d9b640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b41033d0 0x7fa0b419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.600+0000 7fa0b259c640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b419d630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4031460 tx=0x7fa0a40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:58.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.600+0000 7fa0a37fe640 1 -- 172.21.15.46:0/1982408747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0a403f070 con 0x7fa0b4102ab0 2024-09-10T04:01:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.600+0000 7fa0a37fe640 1 -- 172.21.15.46:0/1982408747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa0a4004b10 con 0x7fa0b4102ab0 2024-09-10T04:01:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.600+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b4197a40 con 0x7fa0b4102ab0 2024-09-10T04:01:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.600+0000 7fa0a37fe640 1 -- 172.21.15.46:0/1982408747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0a4035030 con 0x7fa0b4102ab0 2024-09-10T04:01:58.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.600+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b4197f70 con 0x7fa0b4102ab0 2024-09-10T04:01:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.602+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b406ada0 con 0x7fa0b4102ab0 2024-09-10T04:01:58.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.603+0000 7fa0a37fe640 1 -- 172.21.15.46:0/1982408747 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fa0a4042070 con 0x7fa0b4102ab0 2024-09-10T04:01:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.603+0000 7fa0a37fe640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa09c06de80 0x7fa09c070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.604+0000 7fa0a37fe640 1 -- 172.21.15.46:0/1982408747 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fa0a40ae350 con 0x7fa0b4102ab0 2024-09-10T04:01:58.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.604+0000 7fa0b1d9b640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa09c06de80 0x7fa09c070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:58.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.604+0000 7fa0b1d9b640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa09c06de80 0x7fa09c070340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80058c0 tx=0x7fa0a8005850 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.608+0000 7fa0a37fe640 1 -- 172.21.15.46:0/1982408747 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0a407c210 con 0x7fa0b4102ab0 2024-09-10T04:01:58.630 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:01:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.800+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa0b4105f80 con 0x7fa0b4102ab0 2024-09-10T04:01:58.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.800+0000 7fa0a37fe640 1 -- 172.21.15.46:0/1982408747 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa0a407fec0 con 0x7fa0b4102ab0 2024-09-10T04:01:58.802 INFO:teuthology.orchestra.run.smithi046.stdout:81604378647 2024-09-10T04:01:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.802+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa09c06de80 msgr2=0x7fa09c070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa09c06de80 0x7fa09c070340 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80058c0 tx=0x7fa0a8005850 comp rx=0 tx=0).stop 2024-09-10T04:01:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 msgr2=0x7fa0b419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b419d630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4031460 tx=0x7fa0a40045e0 comp rx=0 tx=0).stop 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 shutdown_connections 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa09c06de80 0x7fa09c070340 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b41033d0 0x7fa0b419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 --2- 172.21.15.46:0/1982408747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b4102ab0 0x7fa0b419d630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 >> 172.21.15.46:0/1982408747 conn(0x7fa0b40fc4d0 msgr2=0x7fa0b4108e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 shutdown_connections 2024-09-10T04:01:58.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:58.803+0000 7fa0b8991640 1 -- 172.21.15.46:0/1982408747 wait complete. 2024-09-10T04:01:59.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 -- 172.21.15.46:0/2280627026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 msgr2=0x7fb36c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 --2- 172.21.15.46:0/2280627026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c106bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb3680097e0 tx=0x7fb36802a3f0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 -- 172.21.15.46:0/2280627026 shutdown_connections 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 --2- 172.21.15.46:0/2280627026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb36c1007f0 0x7fb36c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 --2- 172.21.15.46:0/2280627026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c106bd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 -- 172.21.15.46:0/2280627026 >> 172.21.15.46:0/2280627026 conn(0x7fb36c0fc550 msgr2=0x7fb36c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 -- 172.21.15.46:0/2280627026 shutdown_connections 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.565+0000 7fb373bcc640 1 -- 172.21.15.46:0/2280627026 wait complete. 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.566+0000 7fb373bcc640 1 Processor -- start 2024-09-10T04:01:59.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.566+0000 7fb373bcc640 1 -- start start 2024-09-10T04:01:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.566+0000 7fb373bcc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb36c1007f0 0x7fb36c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.566+0000 7fb373bcc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:59.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.566+0000 7fb373bcc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c10dcc0 con 0x7fb36c1067f0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.566+0000 7fb373bcc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c10db40 con 0x7fb36c1007f0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371167640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371167640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44038/0 (socket says 172.21.15.46:44038) 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371167640 1 -- 172.21.15.46:0/3759836075 learned_addr learned my addr 172.21.15.46:0/3759836075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371968640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb36c1007f0 0x7fb36c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371167640 1 -- 172.21.15.46:0/3759836075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb36c1007f0 msgr2=0x7fb36c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371167640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb36c1007f0 0x7fb36c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371167640 1 -- 172.21.15.46:0/3759836075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c1977b0 con 0x7fb36c1067f0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb371167640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c19db70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00e970 tx=0x7fb35c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb362ffd640 1 -- 172.21.15.46:0/3759836075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb35c00cac0 con 0x7fb36c1067f0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb362ffd640 1 -- 172.21.15.46:0/3759836075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb35c0054e0 con 0x7fb36c1067f0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36c197aa0 con 0x7fb36c1067f0 2024-09-10T04:01:59.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb362ffd640 1 -- 172.21.15.46:0/3759836075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb35c004d70 con 0x7fb36c1067f0 2024-09-10T04:01:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.567+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36c198030 con 0x7fb36c1067f0 2024-09-10T04:01:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.569+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36c101f30 con 0x7fb36c1067f0 2024-09-10T04:01:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.569+0000 7fb362ffd640 1 -- 172.21.15.46:0/3759836075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fb35c005020 con 0x7fb36c1067f0 2024-09-10T04:01:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.570+0000 7fb362ffd640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fb35806de30 0x7fb3580702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:59.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.570+0000 7fb371968640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fb35806de30 0x7fb3580702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.570+0000 7fb362ffd640 1 -- 172.21.15.46:0/3759836075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fb35c08dee0 con 0x7fb36c1067f0 2024-09-10T04:01:59.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.570+0000 7fb371968640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fb35806de30 0x7fb3580702f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3680095f0 tx=0x7fb368030890 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:59.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.574+0000 7fb362ffd640 1 -- 172.21.15.46:0/3759836075 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb35c00b2c0 con 0x7fb36c1067f0 2024-09-10T04:01:59.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 -- 172.21.15.46:0/1118060905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 msgr2=0x7f5d3c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/1118060905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c102e90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009880 tx=0x7f5d2c02a530 comp rx=0 tx=0).stop 2024-09-10T04:01:59.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 -- 172.21.15.46:0/1118060905 shutdown_connections 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/1118060905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d3c1033d0 0x7f5d3c10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/1118060905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c102e90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 -- 172.21.15.46:0/1118060905 >> 172.21.15.46:0/1118060905 conn(0x7f5d3c0fc550 msgr2=0x7f5d3c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 -- 172.21.15.46:0/1118060905 shutdown_connections 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.496+0000 7f5d41bb8640 1 -- 172.21.15.46:0/1118060905 wait complete. 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d41bb8640 1 Processor -- start 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d41bb8640 1 -- start start 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d41bb8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d41bb8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d3c1033d0 0x7f5d3c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d41bb8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10edc0 con 0x7f5d3c102ab0 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d41bb8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c10ec40 con 0x7f5d3c1033d0 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:59.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44026/0 (socket says 172.21.15.46:44026) 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d3b7fe640 1 -- 172.21.15.46:0/3865775983 learned_addr learned my addr 172.21.15.46:0/3865775983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.497+0000 7f5d3affd640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d3c1033d0 0x7f5d3c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d3b7fe640 1 -- 172.21.15.46:0/3865775983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d3c1033d0 msgr2=0x7f5d3c100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d3b7fe640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d3c1033d0 0x7f5d3c100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d3b7fe640 1 -- 172.21.15.46:0/3865775983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c101190 con 0x7f5d3c102ab0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d3b7fe640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c100710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c031460 tx=0x7f5d2c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d38ff9640 1 -- 172.21.15.46:0/3865775983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c03f070 con 0x7f5d3c102ab0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d38ff9640 1 -- 172.21.15.46:0/3865775983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d2c004cf0 con 0x7f5d3c102ab0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c101420 con 0x7f5d3c102ab0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d38ff9640 1 -- 172.21.15.46:0/3865775983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c035030 con 0x7f5d3c102ab0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.498+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a3d70 con 0x7f5d3c102ab0 2024-09-10T04:01:59.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.499+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c06a580 con 0x7f5d3c102ab0 2024-09-10T04:01:59.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.500+0000 7f5d38ff9640 1 -- 172.21.15.46:0/3865775983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f5d2c042070 con 0x7f5d3c102ab0 2024-09-10T04:01:59.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.500+0000 7f5d38ff9640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5d2406de30 0x7f5d240702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:01:59.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.500+0000 7f5d38ff9640 1 -- 172.21.15.46:0/3865775983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f5d2c0ae2c0 con 0x7f5d3c102ab0 2024-09-10T04:01:59.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.500+0000 7f5d3affd640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5d2406de30 0x7f5d240702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:01:59.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.501+0000 7f5d3affd640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5d2406de30 0x7f5d240702f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c1021a0 tx=0x7f5d3000a560 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:01:59.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.505+0000 7f5d38ff9640 1 -- 172.21.15.46:0/3865775983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d2c07c1a0 con 0x7f5d3c102ab0 2024-09-10T04:01:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.710+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5d3c105f80 con 0x7f5d3c102ab0 2024-09-10T04:01:59.712 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:59 smithi046 ceph-mon[28328]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:59.712 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:01:59 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1982408747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T04:01:59.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.711+0000 7f5d38ff9640 1 -- 172.21.15.46:0/3865775983 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d2c0ae0e0 con 0x7f5d3c102ab0 2024-09-10T04:01:59.713 INFO:teuthology.orchestra.run.smithi046.stdout:107374182418 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5d2406de30 msgr2=0x7f5d240702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5d2406de30 0x7f5d240702f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c1021a0 tx=0x7f5d3000a560 comp rx=0 tx=0).stop 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 msgr2=0x7f5d3c100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c100710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c031460 tx=0x7f5d2c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 shutdown_connections 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f5d2406de30 0x7f5d240702f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d3c1033d0 0x7f5d3c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 --2- 172.21.15.46:0/3865775983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d3c102ab0 0x7f5d3c100710 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 >> 172.21.15.46:0/3865775983 conn(0x7f5d3c0fc550 msgr2=0x7f5d3c108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 shutdown_connections 2024-09-10T04:01:59.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.713+0000 7f5d41bb8640 1 -- 172.21.15.46:0/3865775983 wait complete. 2024-09-10T04:01:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.777+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb36c109d70 con 0x7fb36c1067f0 2024-09-10T04:01:59.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.777+0000 7fb362ffd640 1 -- 172.21.15.46:0/3759836075 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb35c05bdc0 con 0x7fb36c1067f0 2024-09-10T04:01:59.779 INFO:teuthology.orchestra.run.smithi046.stdout:137438953485 2024-09-10T04:01:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.779+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fb35806de30 msgr2=0x7fb3580702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.779+0000 7fb373bcc640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fb35806de30 0x7fb3580702f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb3680095f0 tx=0x7fb368030890 comp rx=0 tx=0).stop 2024-09-10T04:01:59.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.779+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 msgr2=0x7fb36c19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:01:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.779+0000 7fb373bcc640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c19db70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00e970 tx=0x7fb35c00ee30 comp rx=0 tx=0).stop 2024-09-10T04:01:59.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.780+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 shutdown_connections 2024-09-10T04:01:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.780+0000 7fb373bcc640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb36c1067f0 0x7fb36c19db70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.780+0000 7fb373bcc640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fb35806de30 0x7fb3580702f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.780+0000 7fb373bcc640 1 --2- 172.21.15.46:0/3759836075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb36c1007f0 0x7fb36c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:01:59.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.780+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 >> 172.21.15.46:0/3759836075 conn(0x7fb36c0fc550 msgr2=0x7fb36c069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:01:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.780+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 shutdown_connections 2024-09-10T04:01:59.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:01:59.780+0000 7fb373bcc640 1 -- 172.21.15.46:0/3759836075 wait complete. 2024-09-10T04:01:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:59 smithi070 ceph-mon[35495]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:01:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:01:59 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1982408747' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T04:02:00.200 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:00.230 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378646 got 81604378647 for osd.2 2024-09-10T04:02:00.230 DEBUG:teuthology.parallel:result is None 2024-09-10T04:02:00.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:00 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3865775983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T04:02:00.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:00 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3759836075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T04:02:00.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:00 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3865775983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T04:02:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:00 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3759836075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 -- 172.21.15.46:0/2540256421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 msgr2=0x7f58fc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 --2- 172.21.15.46:0/2540256421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc100c60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009880 tx=0x7f58f002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 -- 172.21.15.46:0/2540256421 shutdown_connections 2024-09-10T04:02:00.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 --2- 172.21.15.46:0/2540256421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc100c60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 --2- 172.21.15.46:0/2540256421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58fc106800 0x7f58fc106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 -- 172.21.15.46:0/2540256421 >> 172.21.15.46:0/2540256421 conn(0x7f58fc0fc520 msgr2=0x7f58fc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 -- 172.21.15.46:0/2540256421 shutdown_connections 2024-09-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 -- 172.21.15.46:0/2540256421 wait complete. 2024-09-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.887+0000 7f5902d23640 1 Processor -- start 2024-09-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5902d23640 1 -- start start 2024-09-10T04:02:00.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5902d23640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5902d23640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58fc106800 0x7f58fc06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5902d23640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10dd30 con 0x7f58fc100800 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5902d23640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc10dbb0 con 0x7f58fc106800 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5900abf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5900abf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc073d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44068/0 (socket says 172.21.15.46:44068) 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5900abf640 1 -- 172.21.15.46:0/2391496072 learned_addr learned my addr 172.21.15.46:0/2391496072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f58fbfff640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58fc106800 0x7f58fc06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:00.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5900abf640 1 -- 172.21.15.46:0/2391496072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58fc106800 msgr2=0x7f58fc06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5900abf640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58fc106800 0x7f58fc06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5900abf640 1 -- 172.21.15.46:0/2391496072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc06c220 con 0x7f58fc100800 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f58fbfff640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58fc106800 0x7f58fc06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.888+0000 7f5900abf640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc073d10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b420 tx=0x7f58ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.889+0000 7f58f9ffb640 1 -- 172.21.15.46:0/2391496072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f58ec011630 con 0x7f58fc100800 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.889+0000 7f58f9ffb640 1 -- 172.21.15.46:0/2391496072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58ec016070 con 0x7f58fc100800 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.889+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc06c510 con 0x7f58fc100800 2024-09-10T04:02:00.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.889+0000 7f58f9ffb640 1 -- 172.21.15.46:0/2391496072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f58ec010d40 con 0x7f58fc100800 2024-09-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.889+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc06caa0 con 0x7f58fc100800 2024-09-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.890+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58fc101f40 con 0x7f58fc100800 2024-09-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.890+0000 7f58f9ffb640 1 -- 172.21.15.46:0/2391496072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f58ec01b460 con 0x7f58fc100800 2024-09-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.891+0000 7f58f9ffb640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f58e406de90 0x7f58e4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.891+0000 7f58f9ffb640 1 -- 172.21.15.46:0/2391496072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f58ec08d010 con 0x7f58fc100800 2024-09-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.891+0000 7f58fbfff640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f58e406de90 0x7f58e4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:00.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.891+0000 7f58fbfff640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f58e406de90 0x7f58e4070350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f58f0031460 tx=0x7f58f00095f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:00.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:00.895+0000 7f58f9ffb640 1 -- 172.21.15.46:0/2391496072 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58ec05af70 con 0x7f58fc100800 2024-09-10T04:02:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.078+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f58fc109da0 con 0x7f58fc100800 2024-09-10T04:02:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.078+0000 7f58f9ffb640 1 -- 172.21.15.46:0/2391496072 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f58ec05ec20 con 0x7f58fc100800 2024-09-10T04:02:01.080 INFO:teuthology.orchestra.run.smithi046.stdout:158913789961 2024-09-10T04:02:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f58e406de90 msgr2=0x7f58e4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f58e406de90 0x7f58e4070350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f58f0031460 tx=0x7f58f00095f0 comp rx=0 tx=0).stop 2024-09-10T04:02:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 msgr2=0x7f58fc073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc073d10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b420 tx=0x7f58ec00b8e0 comp rx=0 tx=0).stop 2024-09-10T04:02:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 shutdown_connections 2024-09-10T04:02:01.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f58e406de90 0x7f58e4070350 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58fc106800 0x7f58fc06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 --2- 172.21.15.46:0/2391496072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58fc100800 0x7f58fc073d10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 >> 172.21.15.46:0/2391496072 conn(0x7f58fc0fc520 msgr2=0x7f58fc06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 shutdown_connections 2024-09-10T04:02:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:01.080+0000 7f5902d23640 1 -- 172.21.15.46:0/2391496072 wait complete. 2024-09-10T04:02:01.437 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953484 got 137438953485 for osd.4 2024-09-10T04:02:01.437 DEBUG:teuthology.parallel:result is None 2024-09-10T04:02:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:01 smithi070 ceph-mon[35495]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:01 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2391496072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T04:02:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:01 smithi046 ceph-mon[28328]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:01 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2391496072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T04:02:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:02.365 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789960 got 158913789961 for osd.5 2024-09-10T04:02:02.365 DEBUG:teuthology.parallel:result is None 2024-09-10T04:02:02.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182417 got 107374182418 for osd.3 2024-09-10T04:02:02.777 DEBUG:teuthology.parallel:result is None 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.222+0000 7f313b7c4640 1 -- 172.21.15.46:0/4114833818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 msgr2=0x7f313410b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.222+0000 7f313b7c4640 1 --2- 172.21.15.46:0/4114833818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313410b940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3128009880 tx=0x7f312802a530 comp rx=0 tx=0).stop 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 -- 172.21.15.46:0/4114833818 shutdown_connections 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 --2- 172.21.15.46:0/4114833818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313410b940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 --2- 172.21.15.46:0/4114833818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134102b30 0x7f3134102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 -- 172.21.15.46:0/4114833818 >> 172.21.15.46:0/4114833818 conn(0x7f31340fc550 msgr2=0x7f31340fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 -- 172.21.15.46:0/4114833818 shutdown_connections 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 -- 172.21.15.46:0/4114833818 wait complete. 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 Processor -- start 2024-09-10T04:02:03.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.223+0000 7f313b7c4640 1 -- start start 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f313b7c4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134102b30 0x7f313419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f313b7c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f313b7c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f313410ee40 con 0x7f3134103450 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f313b7c4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f313410ecc0 con 0x7f3134102b30 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3138d5f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3138d5f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44080/0 (socket says 172.21.15.46:44080) 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3138d5f640 1 -- 172.21.15.46:0/3306419288 learned_addr learned my addr 172.21.15.46:0/3306419288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3139560640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134102b30 0x7f313419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3138d5f640 1 -- 172.21.15.46:0/3306419288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134102b30 msgr2=0x7f313419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3138d5f640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134102b30 0x7f313419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3138d5f640 1 -- 172.21.15.46:0/3306419288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31341999e0 con 0x7f3134103450 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f3138d5f640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313419fd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3128004f90 tx=0x7f31280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:03.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f31327fc640 1 -- 172.21.15.46:0/3306419288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f312803f070 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f31327fc640 1 -- 172.21.15.46:0/3306419288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3128004cf0 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3134199c70 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f31327fc640 1 -- 172.21.15.46:0/3306419288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3128035030 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.224+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313419a1a0 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.225+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313406a500 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.226+0000 7f31327fc640 1 -- 172.21.15.46:0/3306419288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f3128042070 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.227+0000 7f31327fc640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f311c06de90 0x7f311c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.227+0000 7f3139560640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f311c06de90 0x7f311c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.227+0000 7f31327fc640 1 -- 172.21.15.46:0/3306419288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f31280ae1d0 con 0x7f3134103450 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.227+0000 7f3139560640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f311c06de90 0x7f311c070350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f313419ab20 tx=0x7f312400d040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.231+0000 7f31327fc640 1 -- 172.21.15.46:0/3306419288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f312807fde0 con 0x7f3134103450 2024-09-10T04:02:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.409+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f313419af10 con 0x7f3134103450 2024-09-10T04:02:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.409+0000 7f31327fc640 1 -- 172.21.15.46:0/3306419288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3128084610 con 0x7f3134103450 2024-09-10T04:02:03.411 INFO:teuthology.orchestra.run.smithi046.stdout:38654705701 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f311c06de90 msgr2=0x7f311c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f311c06de90 0x7f311c070350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f313419ab20 tx=0x7f312400d040 comp rx=0 tx=0).stop 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 msgr2=0x7f313419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313419fd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3128004f90 tx=0x7f31280048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 shutdown_connections 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3134103450 0x7f313419fd60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f311c06de90 0x7f311c070350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 --2- 172.21.15.46:0/3306419288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3134102b30 0x7f313419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 >> 172.21.15.46:0/3306419288 conn(0x7f31340fc550 msgr2=0x7f3134100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 shutdown_connections 2024-09-10T04:02:03.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:03.411+0000 7f313b7c4640 1 -- 172.21.15.46:0/3306419288 wait complete. 2024-09-10T04:02:03.715 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:03 smithi046 ceph-mon[28328]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:03.715 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:03 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3306419288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T04:02:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:03 smithi070 ceph-mon[35495]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:03 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3306419288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T04:02:04.156 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705700 got 38654705701 for osd.0 2024-09-10T04:02:04.157 DEBUG:teuthology.parallel:result is None 2024-09-10T04:02:04.157 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T04:02:04.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T04:02:04.419 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:04.728 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:04.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.924+0000 7fc528045640 1 -- 172.21.15.46:0/1309793775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 msgr2=0x7fc52010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.924+0000 7fc528045640 1 --2- 172.21.15.46:0/1309793775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52010dcc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc514009880 tx=0x7fc51402a490 comp rx=0 tx=0).stop 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.924+0000 7fc528045640 1 -- 172.21.15.46:0/1309793775 shutdown_connections 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.924+0000 7fc528045640 1 --2- 172.21.15.46:0/1309793775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52010dcc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.924+0000 7fc528045640 1 --2- 172.21.15.46:0/1309793775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520100650 0x7fc520100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.924+0000 7fc528045640 1 -- 172.21.15.46:0/1309793775 >> 172.21.15.46:0/1309793775 conn(0x7fc5200fc520 msgr2=0x7fc5200fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.925+0000 7fc528045640 1 -- 172.21.15.46:0/1309793775 shutdown_connections 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.925+0000 7fc528045640 1 -- 172.21.15.46:0/1309793775 wait complete. 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.925+0000 7fc528045640 1 Processor -- start 2024-09-10T04:02:04.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.925+0000 7fc528045640 1 -- start start 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc528045640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520100650 0x7fc52019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc528045640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc528045640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc520110f70 con 0x7fc520101000 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc528045640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc520110df0 con 0x7fc520100650 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc5255e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc5255e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52019fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44094/0 (socket says 172.21.15.46:44094) 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc5255e0640 1 -- 172.21.15.46:0/3987261268 learned_addr learned my addr 172.21.15.46:0/3987261268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc525de1640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520100650 0x7fc52019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:04.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc5255e0640 1 -- 172.21.15.46:0/3987261268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520100650 msgr2=0x7fc52019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc5255e0640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520100650 0x7fc52019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc5255e0640 1 -- 172.21.15.46:0/3987261268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc520199a80 con 0x7fc520101000 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc5255e0640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52019fe00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc514031460 tx=0x7fc5140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc51effd640 1 -- 172.21.15.46:0/3987261268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc51403f070 con 0x7fc520101000 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc51effd640 1 -- 172.21.15.46:0/3987261268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc514004cf0 con 0x7fc520101000 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc520199d10 con 0x7fc520101000 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc51effd640 1 -- 172.21.15.46:0/3987261268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc514035030 con 0x7fc520101000 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.926+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52019a2a0 con 0x7fc520101000 2024-09-10T04:02:04.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.927+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52006aa90 con 0x7fc520101000 2024-09-10T04:02:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.928+0000 7fc51effd640 1 -- 172.21.15.46:0/3987261268 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fc514042070 con 0x7fc520101000 2024-09-10T04:02:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.928+0000 7fc51effd640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc50806de90 0x7fc508070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.929+0000 7fc51effd640 1 -- 172.21.15.46:0/3987261268 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fc5140ae1f0 con 0x7fc520101000 2024-09-10T04:02:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.929+0000 7fc525de1640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc50806de90 0x7fc508070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:04.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.929+0000 7fc525de1640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc50806de90 0x7fc508070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5180058c0 tx=0x7fc51800a2d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:04.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:04.933+0000 7fc51effd640 1 -- 172.21.15.46:0/3987261268 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc51407fd60 con 0x7fc520101000 2024-09-10T04:02:05.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.104+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc52010b410 con 0x7fc50806de90 2024-09-10T04:02:05.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.104+0000 7fc51effd640 1 -- 172.21.15.46:0/3987261268 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12321 (secure 0 0 0) 0x7fc51800fb30 con 0x7fc50806de90 2024-09-10T04:02:05.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:05.107 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc50806de90 msgr2=0x7fc508070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc50806de90 0x7fc508070350 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc5180058c0 tx=0x7fc51800a2d0 comp rx=0 tx=0).stop 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 msgr2=0x7fc52019fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52019fe00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc514031460 tx=0x7fc5140048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 shutdown_connections 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520101000 0x7fc52019fe00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc50806de90 0x7fc508070350 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 --2- 172.21.15.46:0/3987261268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520100650 0x7fc52019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 >> 172.21.15.46:0/3987261268 conn(0x7fc5200fc520 msgr2=0x7fc5200fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 shutdown_connections 2024-09-10T04:02:05.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:05.107+0000 7fc528045640 1 -- 172.21.15.46:0/3987261268 wait complete. 2024-09-10T04:02:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:05 smithi070 ceph-mon[35495]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:05.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:05 smithi046 ceph-mon[28328]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:05.881 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-10T04:02:03.754125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":190924,"kb_used_data":3780,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":562279988,"statfs":{"total":575970213888,"available":575774707712,"internally_reserved":0,"allocated":3870720,"data_stored":2566116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003690"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":191,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-10T04:01:35.047839+0000","last_change":"2024-09-10T04:01:07.349275+0000","last_active":"2024-09-10T04:01:35.047839+0000","last_peered":"2024-09-10T04:01:35.047839+0000","last_clean":"2024-09-10T04:01:35.047839+0000","last_became_active":"2024-09-10T04:01:07.348851+0000","last_became_peered":"2024-09-10T04:01:07.348851+0000","last_unstale":"2024-09-10T04:01:35.047839+0000","last_undegraded":"2024-09-10T04:01:35.047839+0000","last_fullsized":"2024-09-10T04:01:35.047839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:16.213807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:16.213807+0000","last_clean_scrub_stamp":"2024-09-10T04:00:16.213807+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:09.284474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35688,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709464,"statfs":{"total":95995035648,"available":95958491136,"internally_reserved":0,"allocated":348160,"data_stored":132502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717784,"statfs":{"total":95995035648,"available":95967010816,"internally_reserved":0,"allocated":348160,"data_stored":132502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182419,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":942080,"data_stored":722870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27372,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717780,"statfs":{"total":95995035648,"available":95967006720,"internally_reserved":0,"allocated":348160,"data_stored":132502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":942080,"data_stored":722870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705701,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":942080,"data_stored":722870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T04:02:05.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T04:02:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:06.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:06 smithi046 ceph-mon[28328]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 -- 172.21.15.46:0/3641093831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 msgr2=0x7f1794102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 --2- 172.21.15.46:0/3641093831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794102e90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1784009880 tx=0x7f178402a530 comp rx=0 tx=0).stop 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 -- 172.21.15.46:0/3641093831 shutdown_connections 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 --2- 172.21.15.46:0/3641093831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17941033d0 0x7f179410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 --2- 172.21.15.46:0/3641093831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794102e90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 -- 172.21.15.46:0/3641093831 >> 172.21.15.46:0/3641093831 conn(0x7f17940fc550 msgr2=0x7f17940fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 -- 172.21.15.46:0/3641093831 shutdown_connections 2024-09-10T04:02:06.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.589+0000 7f179a5e1640 1 -- 172.21.15.46:0/3641093831 wait complete. 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.590+0000 7f179a5e1640 1 Processor -- start 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.590+0000 7f179a5e1640 1 -- start start 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.590+0000 7f179a5e1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.590+0000 7f179a5e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17941033d0 0x7f1794100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.590+0000 7f179a5e1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f179410ee40 con 0x7f1794102ab0 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.590+0000 7f179a5e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f179410ecc0 con 0x7f17941033d0 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f1793fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f1793fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44104/0 (socket says 172.21.15.46:44104) 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f1793fff640 1 -- 172.21.15.46:0/2203029617 learned_addr learned my addr 172.21.15.46:0/2203029617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f17937fe640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17941033d0 0x7f1794100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f1793fff640 1 -- 172.21.15.46:0/2203029617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17941033d0 msgr2=0x7f1794100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f1793fff640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17941033d0 0x7f1794100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:06.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f1793fff640 1 -- 172.21.15.46:0/2203029617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1794101190 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f1793fff640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794100710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1784031460 tx=0x7f17840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f17917fa640 1 -- 172.21.15.46:0/2203029617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f178403f070 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f17917fa640 1 -- 172.21.15.46:0/2203029617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1784004cf0 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1794101420 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f17917fa640 1 -- 172.21.15.46:0/2203029617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1784035030 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.591+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17941a3dd0 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.592+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179406a580 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.593+0000 7f17917fa640 1 -- 172.21.15.46:0/2203029617 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f1784042070 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.593+0000 7f17917fa640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f177c06de30 0x7f177c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.594+0000 7f17917fa640 1 -- 172.21.15.46:0/2203029617 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f17840ae280 con 0x7f1794102ab0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.594+0000 7f17937fe640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f177c06de30 0x7f177c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.594+0000 7f17937fe640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f177c06de30 0x7f177c0702f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f17941021a0 tx=0x7f1788009060 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:06.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.598+0000 7f17917fa640 1 -- 172.21.15.46:0/2203029617 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f178407fe10 con 0x7f1794102ab0 2024-09-10T04:02:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.769+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1794062630 con 0x7f177c06de30 2024-09-10T04:02:06.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.770+0000 7f17917fa640 1 -- 172.21.15.46:0/2203029617 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12321 (secure 0 0 0) 0x7f1794062630 con 0x7f177c06de30 2024-09-10T04:02:06.772 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:06.772 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-09-10T04:02:06.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f177c06de30 msgr2=0x7f177c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f177c06de30 0x7f177c0702f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f17941021a0 tx=0x7f1788009060 comp rx=0 tx=0).stop 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 msgr2=0x7f1794100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794100710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1784031460 tx=0x7f17840048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 shutdown_connections 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f177c06de30 0x7f177c0702f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17941033d0 0x7f1794100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 --2- 172.21.15.46:0/2203029617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1794102ab0 0x7f1794100710 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 >> 172.21.15.46:0/2203029617 conn(0x7f17940fc550 msgr2=0x7f1794108e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 shutdown_connections 2024-09-10T04:02:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:06.772+0000 7f179a5e1640 1 -- 172.21.15.46:0/2203029617 wait complete. 2024-09-10T04:02:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:06 smithi070 ceph-mon[35495]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:02:07.469 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-10T04:02:05.754456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":190924,"kb_used_data":3780,"kb_used_omap":9,"kb_used_meta":186998,"kb_avail":562279988,"statfs":{"total":575970213888,"available":575774707712,"internally_reserved":0,"allocated":3870720,"data_stored":2566116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9567,"internal_metadata":191486625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003641"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":191,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-10T04:01:35.047839+0000","last_change":"2024-09-10T04:01:07.349275+0000","last_active":"2024-09-10T04:01:35.047839+0000","last_peered":"2024-09-10T04:01:35.047839+0000","last_clean":"2024-09-10T04:01:35.047839+0000","last_became_active":"2024-09-10T04:01:07.348851+0000","last_became_peered":"2024-09-10T04:01:07.348851+0000","last_unstale":"2024-09-10T04:01:35.047839+0000","last_undegraded":"2024-09-10T04:01:35.047839+0000","last_fullsized":"2024-09-10T04:01:35.047839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:16.213807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:16.213807+0000","last_clean_scrub_stamp":"2024-09-10T04:00:16.213807+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:23:09.284474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":37,"seq":158913789962,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":35688,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709464,"statfs":{"total":95995035648,"available":95958491136,"internally_reserved":0,"allocated":348160,"data_stored":132502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717784,"statfs":{"total":95995035648,"available":95967010816,"internally_reserved":0,"allocated":348160,"data_stored":132502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182420,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":942080,"data_stored":722870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378648,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27372,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717780,"statfs":{"total":95995035648,"available":95967006720,"internally_reserved":0,"allocated":348160,"data_stored":132502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":942080,"data_stored":722870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36268,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708884,"statfs":{"total":95995035648,"available":95957897216,"internally_reserved":0,"allocated":942080,"data_stored":722870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T04:02:07.470 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T04:02:07.470 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T04:02:07.470 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T04:02:07.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T04:02:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:07.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:07 smithi070 ceph-mon[35495]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:07 smithi046 ceph-mon[28328]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/2543235728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 msgr2=0x7f4fc4100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/2543235728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc4100a30 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009880 tx=0x7f4fc002a490 comp rx=0 tx=0).stop 2024-09-10T04:02:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/2543235728 shutdown_connections 2024-09-10T04:02:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/2543235728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc4101000 0x7f4fc410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:08.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/2543235728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc4100a30 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:08.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/2543235728 >> 172.21.15.46:0/2543235728 conn(0x7f4fc40fc520 msgr2=0x7f4fc40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:08.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/2543235728 shutdown_connections 2024-09-10T04:02:08.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.136+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/2543235728 wait complete. 2024-09-10T04:02:08.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fcb3ef640 1 Processor -- start 2024-09-10T04:02:08.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fcb3ef640 1 -- start start 2024-09-10T04:02:08.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fcb3ef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:08.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fcb3ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc4101000 0x7f4fc419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fcb3ef640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4110fb0 con 0x7f4fc4100650 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fcb3ef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4110e30 con 0x7f4fc4101000 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fc918b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fc918b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc419f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44130/0 (socket says 172.21.15.46:44130) 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.137+0000 7f4fc918b640 1 -- 172.21.15.46:0/728598379 learned_addr learned my addr 172.21.15.46:0/728598379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fc898a640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc4101000 0x7f4fc419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fc918b640 1 -- 172.21.15.46:0/728598379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc4101000 msgr2=0x7f4fc419fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fc918b640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc4101000 0x7f4fc419fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fc918b640 1 -- 172.21.15.46:0/728598379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc4199a20 con 0x7f4fc4100650 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fc898a640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc4101000 0x7f4fc419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fc918b640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc419f860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0031460 tx=0x7f4fc00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fba7fc640 1 -- 172.21.15.46:0/728598379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4fc003f070 con 0x7f4fc4100650 2024-09-10T04:02:08.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fba7fc640 1 -- 172.21.15.46:0/728598379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4fc00035f0 con 0x7f4fc4100650 2024-09-10T04:02:08.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fba7fc640 1 -- 172.21.15.46:0/728598379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4fc0035030 con 0x7f4fc4100650 2024-09-10T04:02:08.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc4199cb0 con 0x7f4fc4100650 2024-09-10T04:02:08.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.138+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc419a240 con 0x7f4fc4100650 2024-09-10T04:02:08.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.139+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc406aa90 con 0x7f4fc4100650 2024-09-10T04:02:08.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.140+0000 7f4fba7fc640 1 -- 172.21.15.46:0/728598379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f4fc0042070 con 0x7f4fc4100650 2024-09-10T04:02:08.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.140+0000 7f4fba7fc640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fb006de30 0x7f4fb00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:08.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.140+0000 7f4fba7fc640 1 -- 172.21.15.46:0/728598379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f4fc00ae210 con 0x7f4fc4100650 2024-09-10T04:02:08.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.140+0000 7f4fc898a640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fb006de30 0x7f4fb00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:08.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.141+0000 7f4fc898a640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fb006de30 0x7f4fb00702f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fc41025b0 tx=0x7f4fb4005e50 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.145+0000 7f4fba7fc640 1 -- 172.21.15.46:0/728598379 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fc007c0f0 con 0x7f4fc4100650 2024-09-10T04:02:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.356+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4fc419af50 con 0x7f4fc4100650 2024-09-10T04:02:08.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.356+0000 7f4fba7fc640 1 -- 172.21.15.46:0/728598379 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4fc007fda0 con 0x7f4fc4100650 2024-09-10T04:02:08.358 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:08.358 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T04:02:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.358+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fb006de30 msgr2=0x7f4fb00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.358+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fb006de30 0x7f4fb00702f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fc41025b0 tx=0x7f4fb4005e50 comp rx=0 tx=0).stop 2024-09-10T04:02:08.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 msgr2=0x7f4fc419f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc419f860 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0031460 tx=0x7f4fc00045e0 comp rx=0 tx=0).stop 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 shutdown_connections 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fb006de30 0x7f4fb00702f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fc4101000 0x7f4fc419fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 --2- 172.21.15.46:0/728598379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fc4100650 0x7f4fc419f860 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 >> 172.21.15.46:0/728598379 conn(0x7f4fc40fc520 msgr2=0x7f4fc40fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 shutdown_connections 2024-09-10T04:02:08.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:08.359+0000 7f4fcb3ef640 1 -- 172.21.15.46:0/728598379 wait complete. 2024-09-10T04:02:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:08 smithi070 ceph-mon[35495]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:02:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:08 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/728598379' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T04:02:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:08 smithi046 ceph-mon[28328]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:02:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:08 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/728598379' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T04:02:08.974 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T04:02:08.974 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T04:02:08.974 INFO:teuthology.run_tasks:Running task print... 2024-09-10T04:02:08.984 INFO:teuthology.task.print:**** done end installing squid cephadm ... 2024-09-10T04:02:08.984 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:02:09.010 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:09.010 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-09-10T04:02:09.264 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:09.694 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:09 smithi046 ceph-mon[28328]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 -- 172.21.15.46:0/372097433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54104da0 msgr2=0x7fed54107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 --2- 172.21.15.46:0/372097433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54104da0 0x7fed54107190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fed48009880 tx=0x7fed4802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 -- 172.21.15.46:0/372097433 shutdown_connections 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 --2- 172.21.15.46:0/372097433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54104da0 0x7fed54107190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 --2- 172.21.15.46:0/372097433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed54102470 0x7fed54104860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 -- 172.21.15.46:0/372097433 >> 172.21.15.46:0/372097433 conn(0x7fed540fc010 msgr2=0x7fed540fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 -- 172.21.15.46:0/372097433 shutdown_connections 2024-09-10T04:02:09.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 -- 172.21.15.46:0/372097433 wait complete. 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 Processor -- start 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.711+0000 7fed5ab73640 1 -- start start 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5ab73640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54102470 0x7fed54101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5ab73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed54104da0 0x7fed54100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5ab73640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed5410a7a0 con 0x7fed54102470 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5ab73640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fed5410a620 con 0x7fed54104da0 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5890f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54102470 0x7fed54101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:09.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5890f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54102470 0x7fed54101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48408/0 (socket says 172.21.15.46:48408) 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5890f640 1 -- 172.21.15.46:0/4282736233 learned_addr learned my addr 172.21.15.46:0/4282736233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed53fff640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed54104da0 0x7fed54100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5890f640 1 -- 172.21.15.46:0/4282736233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed54104da0 msgr2=0x7fed54100420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5890f640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed54104da0 0x7fed54100420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5890f640 1 -- 172.21.15.46:0/4282736233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed54100960 con 0x7fed54102470 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5890f640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54102470 0x7fed54101d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d6f0 tx=0x7fed4400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed51ffb640 1 -- 172.21.15.46:0/4282736233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fed44017070 con 0x7fed54102470 2024-09-10T04:02:09.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed51ffb640 1 -- 172.21.15.46:0/4282736233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed4400de00 con 0x7fed54102470 2024-09-10T04:02:09.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed54100c50 con 0x7fed54102470 2024-09-10T04:02:09.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.712+0000 7fed51ffb640 1 -- 172.21.15.46:0/4282736233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fed44004bb0 con 0x7fed54102470 2024-09-10T04:02:09.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.713+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed541011e0 con 0x7fed54102470 2024-09-10T04:02:09.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.714+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed54102850 con 0x7fed54102470 2024-09-10T04:02:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.714+0000 7fed51ffb640 1 -- 172.21.15.46:0/4282736233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fed44009a30 con 0x7fed54102470 2024-09-10T04:02:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.714+0000 7fed51ffb640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed3c06de90 0x7fed3c070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.715+0000 7fed51ffb640 1 -- 172.21.15.46:0/4282736233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fed4408d3a0 con 0x7fed54102470 2024-09-10T04:02:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.715+0000 7fed53fff640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed3c06de90 0x7fed3c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:09.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.715+0000 7fed53fff640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed3c06de90 0x7fed3c070350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fed48006010 tx=0x7fed48000bc0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:09.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.719+0000 7fed51ffb640 1 -- 172.21.15.46:0/4282736233 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed4405b300 con 0x7fed54102470 2024-09-10T04:02:09.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:09 smithi070 ceph-mon[35495]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:09.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.894+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) -- 0x7fed5419fbd0 con 0x7fed54102470 2024-09-10T04:02:09.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.897+0000 7fed51ffb640 1 -- 172.21.15.46:0/4282736233 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v41)=0 v41) ==== 143+0+0 (secure 0 0 0) 0x7fed4405efb0 con 0x7fed54102470 2024-09-10T04:02:09.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.899+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed3c06de90 msgr2=0x7fed3c070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:09.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.899+0000 7fed5ab73640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed3c06de90 0x7fed3c070350 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fed48006010 tx=0x7fed48000bc0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.899+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54102470 msgr2=0x7fed54101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:09.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.899+0000 7fed5ab73640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54102470 0x7fed54101d60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fed4400d6f0 tx=0x7fed4400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.899+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 shutdown_connections 2024-09-10T04:02:09.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.900+0000 7fed5ab73640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed3c06de90 0x7fed3c070350 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.900+0000 7fed5ab73640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed54104da0 0x7fed54100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.900+0000 7fed5ab73640 1 --2- 172.21.15.46:0/4282736233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed54102470 0x7fed54101d60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:09.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.900+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 >> 172.21.15.46:0/4282736233 conn(0x7fed540fc010 msgr2=0x7fed540fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:09.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.900+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 shutdown_connections 2024-09-10T04:02:09.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:09.900+0000 7fed5ab73640 1 -- 172.21.15.46:0/4282736233 wait complete. 2024-09-10T04:02:10.559 INFO:teuthology.run_tasks:Running task print... 2024-09-10T04:02:10.568 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-09-10T04:02:10.568 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:02:10.609 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:10.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T04:02:10.864 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:11.154 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:10 smithi046 ceph-mon[28328]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:11.154 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:10 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/4282736233' entity='client.admin' 2024-09-10T04:02:11.154 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:11.154 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:11.154 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:11.154 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:10 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 -- 172.21.15.46:0/544450800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc1023f0 msgr2=0x7ff6dc1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/544450800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc1023f0 0x7ff6dc1047e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0098b0 tx=0x7ff6cc02a4d0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 -- 172.21.15.46:0/544450800 shutdown_connections 2024-09-10T04:02:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/544450800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6dc104d20 0x7ff6dc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/544450800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc1023f0 0x7ff6dc1047e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 -- 172.21.15.46:0/544450800 >> 172.21.15.46:0/544450800 conn(0x7ff6dc0fc010 msgr2=0x7ff6dc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 -- 172.21.15.46:0/544450800 shutdown_connections 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.328+0000 7ff6e25c2640 1 -- 172.21.15.46:0/544450800 wait complete. 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6e25c2640 1 Processor -- start 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6e25c2640 1 -- start start 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6e25c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6dc1023f0 0x7ff6dc1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6e25c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc104d20 0x7ff6dc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6e25c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc10a830 con 0x7ff6dc104d20 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6e25c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff6dc10a6b0 con 0x7ff6dc1023f0 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc104d20 0x7ff6dc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc104d20 0x7ff6dc197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48418/0 (socket says 172.21.15.46:48418) 2024-09-10T04:02:11.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6db7fe640 1 -- 172.21.15.46:0/3526453676 learned_addr learned my addr 172.21.15.46:0/3526453676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.329+0000 7ff6dbfff640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6dc1023f0 0x7ff6dc1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6db7fe640 1 -- 172.21.15.46:0/3526453676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6dc1023f0 msgr2=0x7ff6dc1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6db7fe640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6dc1023f0 0x7ff6dc1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6db7fe640 1 -- 172.21.15.46:0/3526453676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6dc198180 con 0x7ff6dc104d20 2024-09-10T04:02:11.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6db7fe640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc104d20 0x7ff6dc197b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b3d0 tx=0x7ff6d000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6d97fa640 1 -- 172.21.15.46:0/3526453676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff6d0015020 con 0x7ff6dc104d20 2024-09-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6dc19cbc0 con 0x7ff6dc104d20 2024-09-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6d97fa640 1 -- 172.21.15.46:0/3526453676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff6d0011af0 con 0x7ff6dc104d20 2024-09-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6d97fa640 1 -- 172.21.15.46:0/3526453676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff6d0010d30 con 0x7ff6dc104d20 2024-09-10T04:02:11.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.330+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6dc19d1d0 con 0x7ff6dc104d20 2024-09-10T04:02:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.331+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6dc1027d0 con 0x7ff6dc104d20 2024-09-10T04:02:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.332+0000 7ff6d97fa640 1 -- 172.21.15.46:0/3526453676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7ff6d0018020 con 0x7ff6dc104d20 2024-09-10T04:02:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.332+0000 7ff6d97fa640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff6c406de30 0x7ff6c40702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.332+0000 7ff6d97fa640 1 -- 172.21.15.46:0/3526453676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7ff6d008d2a0 con 0x7ff6dc104d20 2024-09-10T04:02:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.332+0000 7ff6dbfff640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff6c406de30 0x7ff6c40702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:11.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.333+0000 7ff6dbfff640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff6c406de30 0x7ff6c40702f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006010 tx=0x7ff6cc0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:10 smithi070 ceph-mon[35495]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:10 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/4282736233' entity='client.admin' 2024-09-10T04:02:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:11.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:11.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:10 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:11.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.337+0000 7ff6d97fa640 1 -- 172.21.15.46:0/3526453676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6d001f020 con 0x7ff6dc104d20 2024-09-10T04:02:11.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.531+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7ff6dc105700 con 0x7ff6c406de30 2024-09-10T04:02:11.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.532+0000 7ff6d97fa640 1 -- 172.21.15.46:0/3526453676 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7ff6dc105700 con 0x7ff6c406de30 2024-09-10T04:02:11.534 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-09-10T04:02:11.534 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-09-10T04:02:11.534 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-09-10T04:02:11.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff6c406de30 msgr2=0x7ff6c40702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff6c406de30 0x7ff6c40702f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006010 tx=0x7ff6cc0058d0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc104d20 msgr2=0x7ff6dc197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc104d20 0x7ff6dc197b00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000b3d0 tx=0x7ff6d000b890 comp rx=0 tx=0).stop 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 shutdown_connections 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6dc104d20 0x7ff6dc197b00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff6c406de30 0x7ff6c40702f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 --2- 172.21.15.46:0/3526453676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6dc1023f0 0x7ff6dc1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 >> 172.21.15.46:0/3526453676 conn(0x7ff6dc0fc010 msgr2=0x7ff6dc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 shutdown_connections 2024-09-10T04:02:11.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:11.535+0000 7ff6e25c2640 1 -- 172.21.15.46:0/3526453676 wait complete. 2024-09-10T04:02:12.171 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:11 smithi046 ceph-mon[28328]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:12.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T04:02:12.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:11 smithi070 ceph-mon[35495]: from='client.14530 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:12.433 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:12.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 -- 172.21.15.46:0/1591644740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8101a60 msgr2=0x7f52b8101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1591644740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8101a60 0x7f52b8101ee0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009880 tx=0x7f52ac02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 -- 172.21.15.46:0/1591644740 shutdown_connections 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1591644740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8101a60 0x7f52b8101ee0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1591644740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52b8100860 0x7f52b8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 -- 172.21.15.46:0/1591644740 >> 172.21.15.46:0/1591644740 conn(0x7f52b80fc010 msgr2=0x7f52b80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 -- 172.21.15.46:0/1591644740 shutdown_connections 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.894+0000 7f52bdb26640 1 -- 172.21.15.46:0/1591644740 wait complete. 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52bdb26640 1 Processor -- start 2024-09-10T04:02:12.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52bdb26640 1 -- start start 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52bdb26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8100860 0x7f52b81974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52bdb26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52b8101a60 0x7f52b8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52bdb26640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8109680 con 0x7f52b8100860 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52bdb26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8109500 con 0x7f52b8101a60 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8100860 0x7f52b81974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8100860 0x7f52b81974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48440/0 (socket says 172.21.15.46:48440) 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.895+0000 7f52b77fe640 1 -- 172.21.15.46:0/1597165155 learned_addr learned my addr 172.21.15.46:0/1597165155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b6ffd640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52b8101a60 0x7f52b8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b77fe640 1 -- 172.21.15.46:0/1597165155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52b8101a60 msgr2=0x7f52b8197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b77fe640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52b8101a60 0x7f52b8197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b77fe640 1 -- 172.21.15.46:0/1597165155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b8198010 con 0x7f52b8100860 2024-09-10T04:02:12.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b77fe640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8100860 0x7f52b81974e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c8f0 tx=0x7f52a800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:12.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b4ff9640 1 -- 172.21.15.46:0/1597165155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52a8004020 con 0x7f52b8100860 2024-09-10T04:02:12.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b819ca80 con 0x7f52b8100860 2024-09-10T04:02:12.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b4ff9640 1 -- 172.21.15.46:0/1597165155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52a8007e10 con 0x7f52b8100860 2024-09-10T04:02:12.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52b4ff9640 1 -- 172.21.15.46:0/1597165155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f52a8003e10 con 0x7f52b8100860 2024-09-10T04:02:12.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.896+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b819cfe0 con 0x7f52b8100860 2024-09-10T04:02:12.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.898+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b8100c60 con 0x7f52b8100860 2024-09-10T04:02:12.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.898+0000 7f52b4ff9640 1 -- 172.21.15.46:0/1597165155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f52a800f4a0 con 0x7f52b8100860 2024-09-10T04:02:12.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.898+0000 7f52b4ff9640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f52a006de30 0x7f52a00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:12.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.898+0000 7f52b4ff9640 1 -- 172.21.15.46:0/1597165155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f52a808d900 con 0x7f52b8100860 2024-09-10T04:02:12.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.898+0000 7f52b6ffd640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f52a006de30 0x7f52a00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:12.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.899+0000 7f52b6ffd640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f52a006de30 0x7f52a00702f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:12.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:12.903+0000 7f52b4ff9640 1 -- 172.21.15.46:0/1597165155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52a805f490 con 0x7f52b8100860 2024-09-10T04:02:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:12 smithi046 ceph-mon[28328]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.085+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f52b806a930 con 0x7f52a006de30 2024-09-10T04:02:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.093+0000 7f52b4ff9640 1 -- 172.21.15.46:0/1597165155 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3120 (secure 0 0 0) 0x7f52b806a930 con 0x7f52a006de30 2024-09-10T04:02:13.095 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:02:13.095 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (3m) 119s ago 5m 27.7M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:02:13.095 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (5m) 119s ago 5m 7654k - 19.1.1-298-gb53b6d3e 54467abc55f8 3888d245a02a 2024-09-10T04:02:13.095 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (4m) 44s ago 4m 5268k - 19.1.1-298-gb53b6d3e 54467abc55f8 54f37f2cb1f7 2024-09-10T04:02:13.095 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (5m) 119s ago 5m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (4m) 44s ago 4m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (3m) 119s ago 4m 73.5M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:9283,8765,8443 running (6m) 119s ago 6m 489M - 19.1.1-298-gb53b6d3e 54467abc55f8 a4fa65bf88c5 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (4m) 44s ago 4m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 06c733bd8286 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (6m) 119s ago 6m 45.1M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (4m) 44s ago 4m 40.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (5m) 119s ago 5m 15.9M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (4m) 44s ago 4m 17.2M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (3m) 119s ago 3m 47.9M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (2m) 119s ago 2m 46.0M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (2m) 119s ago 2m 12.1M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (101s) 44s ago 101s 43.0M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:02:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (75s) 44s ago 74s 37.0M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:02:13.097 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (48s) 44s ago 48s 11.8M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:02:13.097 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (3m) 119s ago 4m 41.4M - 2.43.0 a07b618ecd1d 15eb5a503403 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f52a006de30 msgr2=0x7f52a00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f52a006de30 0x7f52a00702f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac0058d0 comp rx=0 tx=0).stop 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8100860 msgr2=0x7f52b81974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8100860 0x7f52b81974e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f52a800c8f0 tx=0x7f52a800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 shutdown_connections 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f52a006de30 0x7f52a00702f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52b8101a60 0x7f52b8197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 --2- 172.21.15.46:0/1597165155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52b8100860 0x7f52b81974e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 >> 172.21.15.46:0/1597165155 conn(0x7f52b80fc010 msgr2=0x7f52b806a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 shutdown_connections 2024-09-10T04:02:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:13.096+0000 7f52bdb26640 1 -- 172.21.15.46:0/1597165155 wait complete. 2024-09-10T04:02:13.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:12 smithi070 ceph-mon[35495]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:13.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T04:02:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:14.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:13 smithi046 ceph-mon[28328]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:14.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:13 smithi070 ceph-mon[35495]: from='client.14534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 -- 172.21.15.46:0/339044888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 msgr2=0x7fac78102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 --2- 172.21.15.46:0/339044888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78102fe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fac68009880 tx=0x7fac6802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 -- 172.21.15.46:0/339044888 shutdown_connections 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 --2- 172.21.15.46:0/339044888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78102fe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 --2- 172.21.15.46:0/339044888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac78101960 0x7fac78101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 -- 172.21.15.46:0/339044888 >> 172.21.15.46:0/339044888 conn(0x7fac780fd0f0 msgr2=0x7fac780ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 -- 172.21.15.46:0/339044888 shutdown_connections 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.474+0000 7fac7f96e640 1 -- 172.21.15.46:0/339044888 wait complete. 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7f96e640 1 Processor -- start 2024-09-10T04:02:14.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7f96e640 1 -- start start 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7f96e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac78101960 0x7fac78197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7f96e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7f96e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac7810a820 con 0x7fac78102b60 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7f96e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fac7810a6a0 con 0x7fac78101960 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7cf09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7cf09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48460/0 (socket says 172.21.15.46:48460) 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7cf09640 1 -- 172.21.15.46:0/304674038 learned_addr learned my addr 172.21.15.46:0/304674038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7d70a640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac78101960 0x7fac78197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7cf09640 1 -- 172.21.15.46:0/304674038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac78101960 msgr2=0x7fac78197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:14.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.475+0000 7fac7cf09640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac78101960 0x7fac78197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac7cf09640 1 -- 172.21.15.46:0/304674038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac781980d0 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac7d70a640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac78101960 0x7fac78197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac7cf09640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78197a50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac68009230 tx=0x7fac680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac6e7fc640 1 -- 172.21.15.46:0/304674038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac6803f070 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac6e7fc640 1 -- 172.21.15.46:0/304674038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac680044d0 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac7819cab0 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac6e7fc640 1 -- 172.21.15.46:0/304674038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac68035030 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.476+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac7819d0c0 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.477+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac78101d40 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.477+0000 7fac6e7fc640 1 -- 172.21.15.46:0/304674038 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fac68042070 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.478+0000 7fac6e7fc640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac6406de30 0x7fac640702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.478+0000 7fac6e7fc640 1 -- 172.21.15.46:0/304674038 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7fac680ae580 con 0x7fac78102b60 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.478+0000 7fac7d70a640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac6406de30 0x7fac640702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:14.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.478+0000 7fac7d70a640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac6406de30 0x7fac640702f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac74006fd0 tx=0x7fac74008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:14.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.482+0000 7fac6e7fc640 1 -- 172.21.15.46:0/304674038 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac6807c460 con 0x7fac78102b60 2024-09-10T04:02:14.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.666+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fac78062630 con 0x7fac6406de30 2024-09-10T04:02:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.671+0000 7fac6e7fc640 1 -- 172.21.15.46:0/304674038 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+1290 (secure 0 0 0) 0x7fac78062630 con 0x7fac6406de30 2024-09-10T04:02:14.672 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T04:02:14.672 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-09-10T04:02:14.672 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter 2/2 2m ago 5m * 2024-09-10T04:02:14.673 INFO:teuthology.orchestra.run.smithi046.stdout:crash 2/2 2m ago 5m * 2024-09-10T04:02:14.673 INFO:teuthology.orchestra.run.smithi046.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-09-10T04:02:14.673 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 2/2 2m ago 5m count:2 2024-09-10T04:02:14.673 INFO:teuthology.orchestra.run.smithi046.stdout:mon 2/2 2m ago 5m smithi046:172.21.15.46=smithi046;smithi070:172.21.15.70=smithi070;count:2 2024-09-10T04:02:14.673 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-09-10T04:02:14.673 INFO:teuthology.orchestra.run.smithi046.stdout:osd 6 2m ago - 2024-09-10T04:02:14.673 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac6406de30 msgr2=0x7fac640702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac6406de30 0x7fac640702f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac74006fd0 tx=0x7fac74008040 comp rx=0 tx=0).stop 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 msgr2=0x7fac78197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78197a50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fac68009230 tx=0x7fac680048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 shutdown_connections 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac78102b60 0x7fac78197a50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fac6406de30 0x7fac640702f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 --2- 172.21.15.46:0/304674038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac78101960 0x7fac78197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 >> 172.21.15.46:0/304674038 conn(0x7fac780fd0f0 msgr2=0x7fac78106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 shutdown_connections 2024-09-10T04:02:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:14.673+0000 7fac7f96e640 1 -- 172.21.15.46:0/304674038 wait complete. 2024-09-10T04:02:15.253 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:14 smithi070 ceph-mon[35495]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:15.253 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:14 smithi070 ceph-mon[35495]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:15.257 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:14 smithi046 ceph-mon[28328]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:15.257 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:14 smithi046 ceph-mon[28328]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:15.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T04:02:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:16.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.042+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/4196765715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 msgr2=0x7f2ea4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.042+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/4196765715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea4100c60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009c20 tx=0x7f2e9402a970 comp rx=0 tx=0).stop 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/4196765715 shutdown_connections 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/4196765715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4101a60 0x7f2ea4101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/4196765715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea4100c60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/4196765715 >> 172.21.15.46:0/4196765715 conn(0x7f2ea40fc010 msgr2=0x7f2ea40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/4196765715 shutdown_connections 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/4196765715 wait complete. 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 Processor -- start 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.043+0000 7f2ea9bcc640 1 -- start start 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea9bcc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea41974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea9bcc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4101a60 0x7f2ea4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea9bcc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4109680 con 0x7f2ea4100860 2024-09-10T04:02:16.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea9bcc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4109500 con 0x7f2ea4101a60 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea41974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea41974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48480/0 (socket says 172.21.15.46:48480) 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea37fe640 1 -- 172.21.15.46:0/2547382689 learned_addr learned my addr 172.21.15.46:0/2547382689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea2ffd640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4101a60 0x7f2ea4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea37fe640 1 -- 172.21.15.46:0/2547382689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4101a60 msgr2=0x7f2ea4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea37fe640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4101a60 0x7f2ea4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea37fe640 1 -- 172.21.15.46:0/2547382689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea41980b0 con 0x7f2ea4100860 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea37fe640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea41974f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006010 tx=0x7f2e9402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea0ff9640 1 -- 172.21.15.46:0/2547382689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e94047070 con 0x7f2ea4100860 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea419ca90 con 0x7f2ea4100860 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea0ff9640 1 -- 172.21.15.46:0/2547382689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e94003e30 con 0x7f2ea4100860 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea0ff9640 1 -- 172.21.15.46:0/2547382689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e94034070 con 0x7f2ea4100860 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.044+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea419d020 con 0x7f2ea4100860 2024-09-10T04:02:16.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.045+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea4100c60 con 0x7f2ea4100860 2024-09-10T04:02:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.046+0000 7f2ea0ff9640 1 -- 172.21.15.46:0/2547382689 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f2e9404a070 con 0x7f2ea4100860 2024-09-10T04:02:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.046+0000 7f2ea0ff9640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2e8c06de30 0x7f2e8c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.046+0000 7f2ea0ff9640 1 -- 172.21.15.46:0/2547382689 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f2e940ae490 con 0x7f2ea4100860 2024-09-10T04:02:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.046+0000 7f2ea2ffd640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2e8c06de30 0x7f2e8c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.047+0000 7f2ea2ffd640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2e8c06de30 0x7f2e8c0702f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ea41989a0 tx=0x7f2e98005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:16.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.051+0000 7f2ea0ff9640 1 -- 172.21.15.46:0/2547382689 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e9403f070 con 0x7f2ea4100860 2024-09-10T04:02:16.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.230+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2ea406a400 con 0x7f2e8c06de30 2024-09-10T04:02:16.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.232+0000 7f2ea0ff9640 1 -- 172.21.15.46:0/2547382689 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+145 (secure 0 0 0) 0x7f2ea406a400 con 0x7f2e8c06de30 2024-09-10T04:02:16.233 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-09-10T04:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-09-10T04:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout:smithi070 172.21.15.70 2024-09-10T04:02:16.234 INFO:teuthology.orchestra.run.smithi046.stdout:2 hosts in cluster 2024-09-10T04:02:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2e8c06de30 msgr2=0x7f2e8c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2e8c06de30 0x7f2e8c0702f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ea41989a0 tx=0x7f2e98005f90 comp rx=0 tx=0).stop 2024-09-10T04:02:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 msgr2=0x7f2ea41974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea41974f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2e94006010 tx=0x7f2e9402f5d0 comp rx=0 tx=0).stop 2024-09-10T04:02:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 shutdown_connections 2024-09-10T04:02:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2e8c06de30 0x7f2e8c0702f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ea4101a60 0x7f2ea4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:16.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 --2- 172.21.15.46:0/2547382689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ea4100860 0x7f2ea41974f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 >> 172.21.15.46:0/2547382689 conn(0x7f2ea40fc010 msgr2=0x7f2ea4069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 shutdown_connections 2024-09-10T04:02:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:16.234+0000 7f2ea9bcc640 1 -- 172.21.15.46:0/2547382689 wait complete. 2024-09-10T04:02:16.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T04:02:16.963 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:16 smithi046 ceph-mon[28328]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:16.963 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:16 smithi046 ceph-mon[28328]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:17.194 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:17.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:16 smithi070 ceph-mon[35495]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:17.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:16 smithi070 ceph-mon[35495]: from='client.14542 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 -- 172.21.15.46:0/1066917476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 msgr2=0x7f6f80100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 --2- 172.21.15.46:0/1066917476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f80100c60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009880 tx=0x7f6f7002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 -- 172.21.15.46:0/1066917476 shutdown_connections 2024-09-10T04:02:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 --2- 172.21.15.46:0/1066917476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f80101a60 0x7f6f80101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 --2- 172.21.15.46:0/1066917476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f80100c60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 -- 172.21.15.46:0/1066917476 >> 172.21.15.46:0/1066917476 conn(0x7f6f800fc010 msgr2=0x7f6f800fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 -- 172.21.15.46:0/1066917476 shutdown_connections 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.684+0000 7f6f853c7640 1 -- 172.21.15.46:0/1066917476 wait complete. 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f853c7640 1 Processor -- start 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f853c7640 1 -- start start 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f853c7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f8006aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f853c7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f80101a60 0x7f6f8019c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f853c7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80109680 con 0x7f6f80100860 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f853c7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6f80109500 con 0x7f6f80101a60 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f7effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f8006aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f7effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f8006aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48498/0 (socket says 172.21.15.46:48498) 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f7effd640 1 -- 172.21.15.46:0/3596733822 learned_addr learned my addr 172.21.15.46:0/3596733822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.685+0000 7f6f77fff640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f80101a60 0x7f6f8019c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f7effd640 1 -- 172.21.15.46:0/3596733822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f80101a60 msgr2=0x7f6f8019c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f7effd640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f80101a60 0x7f6f8019c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f7effd640 1 -- 172.21.15.46:0/3596733822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8019c7d0 con 0x7f6f80100860 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f7effd640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f8006aca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009230 tx=0x7f6f700045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f7cff9640 1 -- 172.21.15.46:0/3596733822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6f7003f070 con 0x7f6f80100860 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f7cff9640 1 -- 172.21.15.46:0/3596733822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f70004a50 con 0x7f6f80100860 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8019ca60 con 0x7f6f80100860 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f7cff9640 1 -- 172.21.15.46:0/3596733822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6f70035030 con 0x7f6f80100860 2024-09-10T04:02:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.686+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8019cf90 con 0x7f6f80100860 2024-09-10T04:02:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.687+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f80100c60 con 0x7f6f80100860 2024-09-10T04:02:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.688+0000 7f6f7cff9640 1 -- 172.21.15.46:0/3596733822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f6f70042070 con 0x7f6f80100860 2024-09-10T04:02:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.688+0000 7f6f7cff9640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6f6c06de30 0x7f6f6c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.688+0000 7f6f7cff9640 1 -- 172.21.15.46:0/3596733822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7f6f700ae3a0 con 0x7f6f80100860 2024-09-10T04:02:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.688+0000 7f6f77fff640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6f6c06de30 0x7f6f6c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.689+0000 7f6f77fff640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6f6c06de30 0x7f6f6c0702f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f8006aa20 tx=0x7f6f6800b390 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.693+0000 7f6f7cff9640 1 -- 172.21.15.46:0/3596733822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f7007fea0 con 0x7f6f80100860 2024-09-10T04:02:17.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.873+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f6f80106030 con 0x7f6f6c06de30 2024-09-10T04:02:17.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.875+0000 7f6f7cff9640 1 -- 172.21.15.46:0/3596733822 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+399 (secure 0 0 0) 0x7f6f80106030 con 0x7f6f6c06de30 2024-09-10T04:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T04:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-10T04:02:17.877 INFO:teuthology.orchestra.run.smithi046.stdout:smithi070 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002P480DGN 447G No 45s ago Has a FileSystem, LVM detected 2024-09-10T04:02:17.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.877+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6f6c06de30 msgr2=0x7f6f6c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:17.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.877+0000 7f6f853c7640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6f6c06de30 0x7f6f6c0702f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6f8006aa20 tx=0x7f6f6800b390 comp rx=0 tx=0).stop 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.877+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 msgr2=0x7f6f8006aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.877+0000 7f6f853c7640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f8006aca0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009230 tx=0x7f6f700045c0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.878+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 shutdown_connections 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.878+0000 7f6f853c7640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f6f6c06de30 0x7f6f6c0702f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.878+0000 7f6f853c7640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f80101a60 0x7f6f8019c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.878+0000 7f6f853c7640 1 --2- 172.21.15.46:0/3596733822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f80100860 0x7f6f8006aca0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.878+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 >> 172.21.15.46:0/3596733822 conn(0x7f6f800fc010 msgr2=0x7f6f800fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.878+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 shutdown_connections 2024-09-10T04:02:17.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:17.878+0000 7f6f853c7640 1 -- 172.21.15.46:0/3596733822 wait complete. 2024-09-10T04:02:18.460 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:02:18.470 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:18.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-09-10T04:02:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:19.053 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:18 smithi046 ceph-mon[28328]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:19.054 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:18 smithi046 ceph-mon[28328]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:19.054 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:18 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 -- 172.21.15.46:0/800334789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994101a60 msgr2=0x7ff994101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 --2- 172.21.15.46:0/800334789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994101a60 0x7ff994101ee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff988009880 tx=0x7ff98802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 -- 172.21.15.46:0/800334789 shutdown_connections 2024-09-10T04:02:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 --2- 172.21.15.46:0/800334789 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994101a60 0x7ff994101ee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 --2- 172.21.15.46:0/800334789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff994100860 0x7ff994100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 -- 172.21.15.46:0/800334789 >> 172.21.15.46:0/800334789 conn(0x7ff9940fc010 msgr2=0x7ff9940fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 -- 172.21.15.46:0/800334789 shutdown_connections 2024-09-10T04:02:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.270+0000 7ff99acf6640 1 -- 172.21.15.46:0/800334789 wait complete. 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff99acf6640 1 Processor -- start 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff99acf6640 1 -- start start 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff99acf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994100860 0x7ff994104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff99acf6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff994101a60 0x7ff994102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff99acf6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff994109720 con 0x7ff994100860 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff99acf6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff9941095a0 con 0x7ff994101a60 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff998a92640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994100860 0x7ff994104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.271+0000 7ff998a92640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994100860 0x7ff994104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39502/0 (socket says 172.21.15.46:39502) 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff998a92640 1 -- 172.21.15.46:0/1802647104 learned_addr learned my addr 172.21.15.46:0/1802647104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff993fff640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff994101a60 0x7ff994102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff998a92640 1 -- 172.21.15.46:0/1802647104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff994101a60 msgr2=0x7ff994102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff998a92640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff994101a60 0x7ff994102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff998a92640 1 -- 172.21.15.46:0/1802647104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff994103380 con 0x7ff994100860 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff998a92640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994100860 0x7ff994104780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b3d0 tx=0x7ff98400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff991ffb640 1 -- 172.21.15.46:0/1802647104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff984015020 con 0x7ff994100860 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff994103670 con 0x7ff994100860 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff991ffb640 1 -- 172.21.15.46:0/1802647104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff984011af0 con 0x7ff994100860 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff991ffb640 1 -- 172.21.15.46:0/1802647104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff984010da0 con 0x7ff994100860 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.272+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff994103c80 con 0x7ff994100860 2024-09-10T04:02:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.273+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff994100c60 con 0x7ff994100860 2024-09-10T04:02:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.274+0000 7ff991ffb640 1 -- 172.21.15.46:0/1802647104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7ff984018020 con 0x7ff994100860 2024-09-10T04:02:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.274+0000 7ff991ffb640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff97c06de30 0x7ff97c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.274+0000 7ff991ffb640 1 -- 172.21.15.46:0/1802647104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4638+0+0 (secure 0 0 0) 0x7ff98408d390 con 0x7ff994100860 2024-09-10T04:02:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.274+0000 7ff993fff640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff97c06de30 0x7ff97c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.275+0000 7ff993fff640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff97c06de30 0x7ff97c0702f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff994104500 tx=0x7ff988005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.279+0000 7ff991ffb640 1 -- 172.21.15.46:0/1802647104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff98401f020 con 0x7ff994100860 2024-09-10T04:02:19.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:18 smithi070 ceph-mon[35495]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:19.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:18 smithi070 ceph-mon[35495]: from='client.14546 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:18 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:19.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:19.479+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) -- 0x7ff99419f680 con 0x7ff97c06de30 2024-09-10T04:02:20.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:19 smithi070 ceph-mon[35495]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:20.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-10T04:02:20.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:19 smithi046 ceph-mon[28328]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:02:20.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-10T04:02:20.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.953+0000 7ff991ffb640 1 -- 172.21.15.46:0/1802647104 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff99419f680 con 0x7ff97c06de30 2024-09-10T04:02:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff97c06de30 msgr2=0x7ff97c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff97c06de30 0x7ff97c0702f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff994104500 tx=0x7ff988005a20 comp rx=0 tx=0).stop 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994100860 msgr2=0x7ff994104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994100860 0x7ff994104780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff98400b3d0 tx=0x7ff98400b890 comp rx=0 tx=0).stop 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 shutdown_connections 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ff97c06de30 0x7ff97c0702f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff994101a60 0x7ff994102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 --2- 172.21.15.46:0/1802647104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff994100860 0x7ff994104780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 >> 172.21.15.46:0/1802647104 conn(0x7ff9940fc010 msgr2=0x7ff99406a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 shutdown_connections 2024-09-10T04:02:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:20.956+0000 7ff99acf6640 1 -- 172.21.15.46:0/1802647104 wait complete. 2024-09-10T04:02:21.290 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:20 smithi046 ceph-mon[28328]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:21.290 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:20 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-10T04:02:21.290 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:20 smithi046 ceph-mon[28328]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T04:02:21.290 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:20 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-10T04:02:21.290 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:20 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[28324]: 2024-09-10T04:02:20.932+0000 7f2b3cb79640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T04:02:21.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:20 smithi070 ceph-mon[35495]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:20 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-10T04:02:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:20 smithi070 ceph-mon[35495]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T04:02:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:20 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-10T04:02:21.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T04:02:22.125 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: fsmap cephfs:0 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: Saving service mds.cephfs spec with placement count:4 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:22.126 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:22.127 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:22.127 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.hasdap", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:22.127 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.hasdap", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:22.127 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:22.127 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:21 smithi046 ceph-mon[28328]: Deploying daemon mds.cephfs.smithi046.hasdap on smithi046 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: fsmap cephfs:0 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: Saving service mds.cephfs spec with placement count:4 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:22.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:22.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:22.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:22.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:22.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.hasdap", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:22.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.hasdap", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:22.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:22.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:21 smithi070 ceph-mon[35495]: Deploying daemon mds.cephfs.smithi046.hasdap on smithi046 2024-09-10T04:02:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:22 smithi046 ceph-mon[28328]: pgmap v173: 65 pgs: 13 creating+peering, 6 active+clean, 46 unknown; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:23.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:22 smithi046 ceph-mon[28328]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T04:02:23.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:22 smithi070 ceph-mon[35495]: pgmap v173: 65 pgs: 13 creating+peering, 6 active+clean, 46 unknown; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:23.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:22 smithi070 ceph-mon[35495]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T04:02:23.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 -- 172.21.15.46:0/205020515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c101a60 msgr2=0x7fed7c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 --2- 172.21.15.46:0/205020515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c101a60 0x7fed7c101ee0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fed70009880 tx=0x7fed7002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 -- 172.21.15.46:0/205020515 shutdown_connections 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 --2- 172.21.15.46:0/205020515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c101a60 0x7fed7c101ee0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 --2- 172.21.15.46:0/205020515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7c100860 0x7fed7c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 -- 172.21.15.46:0/205020515 >> 172.21.15.46:0/205020515 conn(0x7fed7c0fc010 msgr2=0x7fed7c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 -- 172.21.15.46:0/205020515 shutdown_connections 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 -- 172.21.15.46:0/205020515 wait complete. 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 Processor -- start 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.468+0000 7fed822ed640 1 -- start start 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed822ed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c100860 0x7fed7c197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed822ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7c101a60 0x7fed7c197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed822ed640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c109680 con 0x7fed7c100860 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed822ed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fed7c109500 con 0x7fed7c101a60 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c100860 0x7fed7c197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c100860 0x7fed7c197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39514/0 (socket says 172.21.15.46:39514) 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7bfff640 1 -- 172.21.15.46:0/3253738449 learned_addr learned my addr 172.21.15.46:0/3253738449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7b7fe640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7c101a60 0x7fed7c197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7bfff640 1 -- 172.21.15.46:0/3253738449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7c101a60 msgr2=0x7fed7c197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7bfff640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7c101a60 0x7fed7c197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7bfff640 1 -- 172.21.15.46:0/3253738449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7c1980e0 con 0x7fed7c100860 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.469+0000 7fed7bfff640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c100860 0x7fed7c197520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00d8d0 tx=0x7fed6c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.470+0000 7fed797fa640 1 -- 172.21.15.46:0/3253738449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fed6c017020 con 0x7fed7c100860 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.470+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7c19cb20 con 0x7fed7c100860 2024-09-10T04:02:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.470+0000 7fed797fa640 1 -- 172.21.15.46:0/3253738449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fed6c010460 con 0x7fed7c100860 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.470+0000 7fed797fa640 1 -- 172.21.15.46:0/3253738449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fed6c00f5b0 con 0x7fed7c100860 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.470+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7c19d080 con 0x7fed7c100860 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.471+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed7c100c60 con 0x7fed7c100860 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.472+0000 7fed797fa640 1 -- 172.21.15.46:0/3253738449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fed6c01a020 con 0x7fed7c100860 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.472+0000 7fed797fa640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed6406de30 0x7fed640702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.472+0000 7fed797fa640 1 -- 172.21.15.46:0/3253738449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fed6c08d9b0 con 0x7fed7c100860 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.472+0000 7fed7b7fe640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed6406de30 0x7fed640702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.473+0000 7fed7b7fe640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed6406de30 0x7fed640702f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed70000c00 tx=0x7fed700058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.478+0000 7fed797fa640 1 -- 172.21.15.46:0/3253738449 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed6c021030 con 0x7fed7c100860 2024-09-10T04:02:23.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.712+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fed7c19da20 con 0x7fed7c100860 2024-09-10T04:02:23.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.712+0000 7fed797fa640 1 -- 172.21.15.46:0/3253738449 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) ==== 75+0+1272 (secure 0 0 0) 0x7fed6c05b550 con 0x7fed7c100860 2024-09-10T04:02:23.714 INFO:teuthology.orchestra.run.smithi046.stdout:e2 2024-09-10T04:02:23.714 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:20:933813+0000 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 2 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:20.933717+0000 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:02:23.715 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 1 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:in 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:02:23.716 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:23.717 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed6406de30 msgr2=0x7fed640702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed6406de30 0x7fed640702f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed70000c00 tx=0x7fed700058d0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c100860 msgr2=0x7fed7c197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c100860 0x7fed7c197520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00d8d0 tx=0x7fed6c00dd90 comp rx=0 tx=0).stop 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 shutdown_connections 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fed6406de30 0x7fed640702f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed7c101a60 0x7fed7c197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 --2- 172.21.15.46:0/3253738449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed7c100860 0x7fed7c197520 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:23.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 >> 172.21.15.46:0/3253738449 conn(0x7fed7c0fc010 msgr2=0x7fed7c069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:23.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 shutdown_connections 2024-09-10T04:02:23.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:23.715+0000 7fed822ed640 1 -- 172.21.15.46:0/3253738449 wait complete. 2024-09-10T04:02:24.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:23 smithi070 ceph-mon[35495]: osdmap e43: 6 total, 6 up, 6 in 2024-09-10T04:02:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:23 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3253738449' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T04:02:24.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:23 smithi046 ceph-mon[28328]: osdmap e43: 6 total, 6 up, 6 in 2024-09-10T04:02:24.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:23 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3253738449' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T04:02:24.453 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:02:24.464 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:24.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-09-10T04:02:24.738 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:25.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 -- 172.21.15.46:0/1453454513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 msgr2=0x7f4f54101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 --2- 172.21.15.46:0/1453454513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54101d70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009880 tx=0x7f4f4402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 -- 172.21.15.46:0/1453454513 shutdown_connections 2024-09-10T04:02:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 --2- 172.21.15.46:0/1453454513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f54102b70 0x7f4f54102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 --2- 172.21.15.46:0/1453454513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54101d70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 -- 172.21.15.46:0/1453454513 >> 172.21.15.46:0/1453454513 conn(0x7f4f540fd140 msgr2=0x7f4f540ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 -- 172.21.15.46:0/1453454513 shutdown_connections 2024-09-10T04:02:25.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.183+0000 7f4f59e13640 1 -- 172.21.15.46:0/1453454513 wait complete. 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f59e13640 1 Processor -- start 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f59e13640 1 -- start start 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f59e13640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f59e13640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f54102b70 0x7f4f540745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f59e13640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410a7b0 con 0x7f4f54101970 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f59e13640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410a630 con 0x7f4f54102b70 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:25.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f52ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f54102b70 0x7f4f540745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39532/0 (socket says 172.21.15.46:39532) 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.184+0000 7f4f537fe640 1 -- 172.21.15.46:0/2795922479 learned_addr learned my addr 172.21.15.46:0/2795922479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f537fe640 1 -- 172.21.15.46:0/2795922479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f54102b70 msgr2=0x7f4f540745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f537fe640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f54102b70 0x7f4f540745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f537fe640 1 -- 172.21.15.46:0/2795922479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f54074b20 con 0x7f4f54101970 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f537fe640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54075f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f44006010 tx=0x7f4f440045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f50ff9640 1 -- 172.21.15.46:0/2795922479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f4403f070 con 0x7f4f54101970 2024-09-10T04:02:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f54074db0 con 0x7f4f54101970 2024-09-10T04:02:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f50ff9640 1 -- 172.21.15.46:0/2795922479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f440049d0 con 0x7f4f54101970 2024-09-10T04:02:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f50ff9640 1 -- 172.21.15.46:0/2795922479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f44035030 con 0x7f4f54101970 2024-09-10T04:02:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.185+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f540752e0 con 0x7f4f54101970 2024-09-10T04:02:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.186+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f54101d70 con 0x7f4f54101970 2024-09-10T04:02:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.187+0000 7f4f50ff9640 1 -- 172.21.15.46:0/2795922479 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f4f44042070 con 0x7f4f54101970 2024-09-10T04:02:25.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.188+0000 7f4f50ff9640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f3c06de30 0x7f4f3c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:25.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.188+0000 7f4f52ffd640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f3c06de30 0x7f4f3c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:25.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.188+0000 7f4f50ff9640 1 -- 172.21.15.46:0/2795922479 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f4f440ae630 con 0x7f4f54101970 2024-09-10T04:02:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.188+0000 7f4f52ffd640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f3c06de30 0x7f4f3c0702f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f54075ca0 tx=0x7f4f48005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.192+0000 7f4f50ff9640 1 -- 172.21.15.46:0/2795922479 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f4407fe80 con 0x7f4f54101970 2024-09-10T04:02:25.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.404+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) -- 0x7f4f5419f600 con 0x7f4f54101970 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: pgmap v176: 65 pgs: 13 creating+peering, 24 active+clean, 28 unknown; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.ehvjgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.ehvjgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] up:boot 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: daemon mds.cephfs.smithi046.hasdap assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: Cluster is now healthy 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: fsmap cephfs:0 1 up:standby 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"}]: dispatch 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: fsmap cephfs:1 {0=cephfs.smithi046.hasdap=up:creating} 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: daemon mds.cephfs.smithi046.hasdap is now active in filesystem cephfs as rank 0 2024-09-10T04:02:25.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:25 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2795922479' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-10T04:02:25.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.953+0000 7f4f50ff9640 1 -- 172.21.15.46:0/2795922479 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) ==== 105+0+0 (secure 0 0 0) 0x7f4f440846b0 con 0x7f4f54101970 2024-09-10T04:02:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f3c06de30 msgr2=0x7f4f3c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f3c06de30 0x7f4f3c0702f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4f54075ca0 tx=0x7f4f48005f90 comp rx=0 tx=0).stop 2024-09-10T04:02:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 msgr2=0x7f4f54075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54075f20 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4f44006010 tx=0x7f4f440045e0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 shutdown_connections 2024-09-10T04:02:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4f3c06de30 0x7f4f3c0702f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f54102b70 0x7f4f540745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 --2- 172.21.15.46:0/2795922479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f54101970 0x7f4f54075f20 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 >> 172.21.15.46:0/2795922479 conn(0x7f4f540fd140 msgr2=0x7f4f540ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.955+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 shutdown_connections 2024-09-10T04:02:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:25.956+0000 7f4f59e13640 1 -- 172.21.15.46:0/2795922479 wait complete. 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: pgmap v176: 65 pgs: 13 creating+peering, 24 active+clean, 28 unknown; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.ehvjgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.ehvjgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] up:boot 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: daemon mds.cephfs.smithi046.hasdap assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T04:02:26.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T04:02:26.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: Cluster is now healthy 2024-09-10T04:02:26.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: fsmap cephfs:0 1 up:standby 2024-09-10T04:02:26.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"}]: dispatch 2024-09-10T04:02:26.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: fsmap cephfs:1 {0=cephfs.smithi046.hasdap=up:creating} 2024-09-10T04:02:26.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: daemon mds.cephfs.smithi046.hasdap is now active in filesystem cephfs as rank 0 2024-09-10T04:02:26.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:25 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2795922479' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-10T04:02:26.743 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:02:26.755 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:26.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-09-10T04:02:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:26 smithi070 ceph-mon[35495]: Deploying daemon mds.cephfs.smithi070.ehvjgr on smithi070 2024-09-10T04:02:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:26 smithi070 ceph-mon[35495]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T04:02:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:26 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2795922479' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-10T04:02:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:26 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] up:active 2024-09-10T04:02:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:26 smithi070 ceph-mon[35495]: fsmap cephfs:1 {0=cephfs.smithi046.hasdap=up:active} 2024-09-10T04:02:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:26 smithi046 ceph-mon[28328]: Deploying daemon mds.cephfs.smithi070.ehvjgr on smithi070 2024-09-10T04:02:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:26 smithi046 ceph-mon[28328]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T04:02:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:26 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2795922479' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-10T04:02:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:26 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] up:active 2024-09-10T04:02:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:26 smithi046 ceph-mon[28328]: fsmap cephfs:1 {0=cephfs.smithi046.hasdap=up:active} 2024-09-10T04:02:27.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 -- 172.21.15.46:0/2740856071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8100880 msgr2=0x7fdab8100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 --2- 172.21.15.46:0/2740856071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8100880 0x7fdab8100c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009880 tx=0x7fdaa802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 -- 172.21.15.46:0/2740856071 shutdown_connections 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 --2- 172.21.15.46:0/2740856071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8101a80 0x7fdab8101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 --2- 172.21.15.46:0/2740856071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8100880 0x7fdab8100c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 -- 172.21.15.46:0/2740856071 >> 172.21.15.46:0/2740856071 conn(0x7fdab80fc030 msgr2=0x7fdab80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 -- 172.21.15.46:0/2740856071 shutdown_connections 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.553+0000 7fdabcf88640 1 -- 172.21.15.46:0/2740856071 wait complete. 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.554+0000 7fdabcf88640 1 Processor -- start 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.554+0000 7fdabcf88640 1 -- start start 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.554+0000 7fdabcf88640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8100880 0x7fdab81974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.554+0000 7fdabcf88640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8101a80 0x7fdab8197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:27.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.554+0000 7fdabcf88640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdab81096a0 con 0x7fdab8101a80 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.554+0000 7fdabcf88640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8109520 con 0x7fdab8100880 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab5d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8101a80 0x7fdab8197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab5d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8101a80 0x7fdab8197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39564/0 (socket says 172.21.15.46:39564) 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab5d9b640 1 -- 172.21.15.46:0/706373124 learned_addr learned my addr 172.21.15.46:0/706373124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab659c640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8100880 0x7fdab81974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab5d9b640 1 -- 172.21.15.46:0/706373124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8100880 msgr2=0x7fdab81974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab5d9b640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8100880 0x7fdab81974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab5d9b640 1 -- 172.21.15.46:0/706373124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab8198020 con 0x7fdab8101a80 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdab5d9b640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8101a80 0x7fdab8197a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00c8a0 tx=0x7fdaac00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdaa77fe640 1 -- 172.21.15.46:0/706373124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdaac0079b0 con 0x7fdab8101a80 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab819ca90 con 0x7fdab8101a80 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdaa77fe640 1 -- 172.21.15.46:0/706373124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdaac007b50 con 0x7fdab8101a80 2024-09-10T04:02:27.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdaa77fe640 1 -- 172.21.15.46:0/706373124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdaac018700 con 0x7fdab8101a80 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.555+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab819cff0 con 0x7fdab8101a80 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.556+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab8100c80 con 0x7fdab8101a80 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.557+0000 7fdaa77fe640 1 -- 172.21.15.46:0/706373124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fdaac007d00 con 0x7fdab8101a80 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.557+0000 7fdaa77fe640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fdaa006de30 0x7fdaa00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.558+0000 7fdaa77fe640 1 -- 172.21.15.46:0/706373124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fdaac08dd70 con 0x7fdab8101a80 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.558+0000 7fdab659c640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fdaa006de30 0x7fdaa00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.558+0000 7fdab659c640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fdaa006de30 0x7fdaa00702f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009230 tx=0x7fdaa80058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:27.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.562+0000 7fdaa77fe640 1 -- 172.21.15.46:0/706373124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdaac014030 con 0x7fdab8101a80 2024-09-10T04:02:27.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:27.788+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fdab819d9e0 con 0x7fdab8101a80 2024-09-10T04:02:28.064 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:27 smithi046 ceph-mon[28328]: pgmap v177: 65 pgs: 4 creating+peering, 61 active+clean; 578 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 635 B/s wr, 2 op/s 2024-09-10T04:02:28.064 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:27 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:28.065 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:27 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:28.065 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:27 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:28.065 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:27 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.cpwyud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:28.065 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:27 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.cpwyud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:28.065 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:27 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:27 smithi070 ceph-mon[35495]: pgmap v177: 65 pgs: 4 creating+peering, 61 active+clean; 578 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 635 B/s wr, 2 op/s 2024-09-10T04:02:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:27 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:27 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:27 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:27 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.cpwyud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:27 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.cpwyud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:27 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.732+0000 7fdaa77fe640 1 -- 172.21.15.46:0/706373124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v8) ==== 121+0+0 (secure 0 0 0) 0x7fdaac05b910 con 0x7fdab8101a80 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fdaa006de30 msgr2=0x7fdaa00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fdaa006de30 0x7fdaa00702f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009230 tx=0x7fdaa80058d0 comp rx=0 tx=0).stop 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8101a80 msgr2=0x7fdab8197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8101a80 0x7fdab8197a30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00c8a0 tx=0x7fdaac00cd60 comp rx=0 tx=0).stop 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 shutdown_connections 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdab8101a80 0x7fdab8197a30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fdaa006de30 0x7fdaa00702f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 --2- 172.21.15.46:0/706373124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdab8100880 0x7fdab81974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.735+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 >> 172.21.15.46:0/706373124 conn(0x7fdab80fc030 msgr2=0x7fdab806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.736+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 shutdown_connections 2024-09-10T04:02:28.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:28.736+0000 7fdabcf88640 1 -- 172.21.15.46:0/706373124 wait complete. 2024-09-10T04:02:28.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: Deploying daemon mds.cephfs.smithi046.cpwyud on smithi046 2024-09-10T04:02:28.997 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] up:boot 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: daemon mds.cephfs.smithi070.ehvjgr assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: fsmap cephfs:1 {0=cephfs.smithi046.hasdap=up:active} 1 up:standby 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"}]: dispatch 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:creating} 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: daemon mds.cephfs.smithi070.ehvjgr is now active in filesystem cephfs as rank 1 2024-09-10T04:02:28.998 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:28 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/706373124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T04:02:29.003 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: Deploying daemon mds.cephfs.smithi046.cpwyud on smithi046 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] up:boot 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: daemon mds.cephfs.smithi070.ehvjgr assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: fsmap cephfs:1 {0=cephfs.smithi046.hasdap=up:active} 1 up:standby 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"}]: dispatch 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:creating} 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: daemon mds.cephfs.smithi070.ehvjgr is now active in filesystem cephfs as rank 1 2024-09-10T04:02:29.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:28 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/706373124' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T04:02:29.543 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:02:29.557 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:29.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-09-10T04:02:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:29 smithi046 ceph-mon[28328]: pgmap v178: 65 pgs: 65 active+clean; 578 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 901 B/s wr, 5 op/s 2024-09-10T04:02:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:29 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/706373124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T04:02:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:29 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] up:active 2024-09-10T04:02:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:29 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2024-09-10T04:02:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:29 smithi070 ceph-mon[35495]: pgmap v178: 65 pgs: 65 active+clean; 578 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 901 B/s wr, 5 op/s 2024-09-10T04:02:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:29 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/706373124' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T04:02:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:29 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] up:active 2024-09-10T04:02:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:29 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2024-09-10T04:02:30.117 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 -- 172.21.15.46:0/3497576296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c104d20 msgr2=0x7fd94c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 --2- 172.21.15.46:0/3497576296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c104d20 0x7fd94c107110 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd9380098b0 tx=0x7fd93802a4d0 comp rx=0 tx=0).stop 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 -- 172.21.15.46:0/3497576296 shutdown_connections 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 --2- 172.21.15.46:0/3497576296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c104d20 0x7fd94c107110 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 --2- 172.21.15.46:0/3497576296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd94c1023f0 0x7fd94c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 -- 172.21.15.46:0/3497576296 >> 172.21.15.46:0/3497576296 conn(0x7fd94c0fc010 msgr2=0x7fd94c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 -- 172.21.15.46:0/3497576296 shutdown_connections 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 -- 172.21.15.46:0/3497576296 wait complete. 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 Processor -- start 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.593+0000 7fd953027640 1 -- start start 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd953027640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c1023f0 0x7fd94c197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd953027640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd94c104d20 0x7fd94c197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd953027640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c10a790 con 0x7fd94c1023f0 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd953027640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd94c10a610 con 0x7fd94c104d20 2024-09-10T04:02:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd950dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c1023f0 0x7fd94c197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd950dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c1023f0 0x7fd94c197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47362/0 (socket says 172.21.15.46:47362) 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd950dc3640 1 -- 172.21.15.46:0/2186932099 learned_addr learned my addr 172.21.15.46:0/2186932099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd94bfff640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd94c104d20 0x7fd94c197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd950dc3640 1 -- 172.21.15.46:0/2186932099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd94c104d20 msgr2=0x7fd94c197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd950dc3640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd94c104d20 0x7fd94c197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd950dc3640 1 -- 172.21.15.46:0/2186932099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94c1980f0 con 0x7fd94c1023f0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd950dc3640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c1023f0 0x7fd94c197530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd94000d8d0 tx=0x7fd94000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd949ffb640 1 -- 172.21.15.46:0/2186932099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd940017030 con 0x7fd94c1023f0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94c19cb30 con 0x7fd94c1023f0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd949ffb640 1 -- 172.21.15.46:0/2186932099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd940004020 con 0x7fd94c1023f0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.594+0000 7fd949ffb640 1 -- 172.21.15.46:0/2186932099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd9400047a0 con 0x7fd94c1023f0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.595+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd94c19d090 con 0x7fd94c1023f0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.596+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd94c10af70 con 0x7fd94c1023f0 2024-09-10T04:02:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.596+0000 7fd949ffb640 1 -- 172.21.15.46:0/2186932099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fd9400041d0 con 0x7fd94c1023f0 2024-09-10T04:02:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.596+0000 7fd949ffb640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd93c06de30 0x7fd93c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.597+0000 7fd949ffb640 1 -- 172.21.15.46:0/2186932099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fd940018070 con 0x7fd94c1023f0 2024-09-10T04:02:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.597+0000 7fd94bfff640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd93c06de30 0x7fd93c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.597+0000 7fd94bfff640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd93c06de30 0x7fd93c0702f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd938005ab0 tx=0x7fd938005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:30.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.601+0000 7fd949ffb640 1 -- 172.21.15.46:0/2186932099 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd94005b560 con 0x7fd94c1023f0 2024-09-10T04:02:30.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:30.818+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd94c1027d0 con 0x7fd94c1023f0 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: pgmap v179: 65 pgs: 65 active+clean; 578 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.myztiz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.myztiz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: Deploying daemon mds.cephfs.smithi070.myztiz on smithi070 2024-09-10T04:02:31.405 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T04:02:31.406 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: Cluster is now healthy 2024-09-10T04:02:31.406 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] up:boot 2024-09-10T04:02:31.406 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby 2024-09-10T04:02:31.406 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"}]: dispatch 2024-09-10T04:02:31.406 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby-replay 2024-09-10T04:02:31.406 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:31 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2186932099' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: pgmap v179: 65 pgs: 65 active+clean; 578 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.myztiz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.myztiz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: Deploying daemon mds.cephfs.smithi070.myztiz on smithi070 2024-09-10T04:02:31.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T04:02:31.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: Cluster is now healthy 2024-09-10T04:02:31.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] up:boot 2024-09-10T04:02:31.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby 2024-09-10T04:02:31.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"}]: dispatch 2024-09-10T04:02:31.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby-replay 2024-09-10T04:02:31.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:31 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2186932099' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T04:02:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.741+0000 7fd949ffb640 1 -- 172.21.15.46:0/2186932099 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) ==== 168+0+0 (secure 0 0 0) 0x7fd94005f210 con 0x7fd94c1023f0 2024-09-10T04:02:31.742 INFO:teuthology.orchestra.run.smithi046.stderr:inline data enabled 2024-09-10T04:02:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd93c06de30 msgr2=0x7fd93c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd93c06de30 0x7fd93c0702f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd938005ab0 tx=0x7fd938005a20 comp rx=0 tx=0).stop 2024-09-10T04:02:31.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c1023f0 msgr2=0x7fd94c197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c1023f0 0x7fd94c197530 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd94000d8d0 tx=0x7fd94000dd90 comp rx=0 tx=0).stop 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 shutdown_connections 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fd93c06de30 0x7fd93c0702f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd94c104d20 0x7fd94c197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 --2- 172.21.15.46:0/2186932099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd94c1023f0 0x7fd94c197530 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.744+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 >> 172.21.15.46:0/2186932099 conn(0x7fd94c0fc010 msgr2=0x7fd94c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.745+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 shutdown_connections 2024-09-10T04:02:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:31.745+0000 7fd953027640 1 -- 172.21.15.46:0/2186932099 wait complete. 2024-09-10T04:02:32.334 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:32 smithi046 ceph-mon[28328]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-10T04:02:32.334 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:32 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2186932099' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T04:02:32.334 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:32 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby-replay 2024-09-10T04:02:32.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:32 smithi070 ceph-mon[35495]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-10T04:02:32.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:32 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2186932099' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T04:02:32.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:32 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby-replay 2024-09-10T04:02:32.370 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:02:32.385 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:32.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T04:02:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:33.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 -- 172.21.15.46:0/199018547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 msgr2=0x7f8844102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:33.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 --2- 172.21.15.46:0/199018547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f8844102fe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8834009880 tx=0x7f883402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:33.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 -- 172.21.15.46:0/199018547 shutdown_connections 2024-09-10T04:02:33.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 --2- 172.21.15.46:0/199018547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f8844102fe0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 --2- 172.21.15.46:0/199018547 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8844101960 0x7f8844101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 -- 172.21.15.46:0/199018547 >> 172.21.15.46:0/199018547 conn(0x7f88440fd0f0 msgr2=0x7f88440ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 -- 172.21.15.46:0/199018547 shutdown_connections 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.173+0000 7f884b29e640 1 -- 172.21.15.46:0/199018547 wait complete. 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f884b29e640 1 Processor -- start 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f884b29e640 1 -- start start 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f884b29e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8844101960 0x7f88441974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f884b29e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f88441979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f884b29e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f884410a7a0 con 0x7f8844102b60 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f884b29e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f884410a620 con 0x7f8844101960 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f8848839640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f88441979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f8848839640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f88441979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47370/0 (socket says 172.21.15.46:47370) 2024-09-10T04:02:33.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f884903a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8844101960 0x7f88441974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.174+0000 7f8848839640 1 -- 172.21.15.46:0/3854927346 learned_addr learned my addr 172.21.15.46:0/3854927346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f8848839640 1 -- 172.21.15.46:0/3854927346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8844101960 msgr2=0x7f88441974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f8848839640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8844101960 0x7f88441974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f8848839640 1 -- 172.21.15.46:0/3854927346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8844198060 con 0x7f8844102b60 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f884903a640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8844101960 0x7f88441974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f8848839640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f88441979e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8834006010 tx=0x7f8834004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f883a7fc640 1 -- 172.21.15.46:0/3854927346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f883403f070 con 0x7f8844102b60 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f883a7fc640 1 -- 172.21.15.46:0/3854927346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f88340049d0 con 0x7f8844102b60 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884419ca40 con 0x7f8844102b60 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f883a7fc640 1 -- 172.21.15.46:0/3854927346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8834035030 con 0x7f8844102b60 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.175+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884419cfd0 con 0x7f8844102b60 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.176+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8844101d60 con 0x7f8844102b60 2024-09-10T04:02:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.177+0000 7f883a7fc640 1 -- 172.21.15.46:0/3854927346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f8834042070 con 0x7f8844102b60 2024-09-10T04:02:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.177+0000 7f883a7fc640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f883006de90 0x7f8830070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.177+0000 7f883a7fc640 1 -- 172.21.15.46:0/3854927346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f88340ae600 con 0x7f8844102b60 2024-09-10T04:02:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.177+0000 7f884903a640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f883006de90 0x7f8830070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.178+0000 7f884903a640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f883006de90 0x7f8830070350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88441029c0 tx=0x7f8840005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.182+0000 7f883a7fc640 1 -- 172.21.15.46:0/3854927346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f883407fed0 con 0x7f8844102b60 2024-09-10T04:02:33.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.420+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8844062df0 con 0x7f8844102b60 2024-09-10T04:02:33.421 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:33 smithi046 ceph-mon[28328]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-10T04:02:33.421 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.421 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.421 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.421 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.421 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.421 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:33 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.420+0000 7f883a7fc640 1 -- 172.21.15.46:0/3854927346 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) ==== 76+0+2004 (secure 0 0 0) 0x7f8834084700 con 0x7f8844102b60 2024-09-10T04:02:33.422 INFO:teuthology.orchestra.run.smithi046.stdout:e13 2024-09-10T04:02:33.422 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:33:138056+0000 2024-09-10T04:02:33.422 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:02:33.422 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:02:33.422 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:02:33.422 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 13 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:33.138048+0000 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:02:33.423 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:02:33.424 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:02:33.425 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 24325,14558 2024-09-10T04:02:33.425 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:02:33.425 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:02:33.425 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 2 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:02:33.425 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:02:33.425 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:33.425 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f883006de90 msgr2=0x7f8830070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f883006de90 0x7f8830070350 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88441029c0 tx=0x7f8840005f90 comp rx=0 tx=0).stop 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 msgr2=0x7f88441979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f88441979e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8834006010 tx=0x7f8834004570 comp rx=0 tx=0).stop 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 shutdown_connections 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8844102b60 0x7f88441979e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f883006de90 0x7f8830070350 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 --2- 172.21.15.46:0/3854927346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8844101960 0x7f88441974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 >> 172.21.15.46:0/3854927346 conn(0x7f88440fd0f0 msgr2=0x7f8844106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 shutdown_connections 2024-09-10T04:02:33.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:33.423+0000 7f884b29e640 1 -- 172.21.15.46:0/3854927346 wait complete. 2024-09-10T04:02:33.497 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:33 smithi070 ceph-mon[35495]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-10T04:02:33.497 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.497 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.497 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.497 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.497 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:33.498 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:33 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:34.327 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] up:boot 2024-09-10T04:02:34.327 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] up:active 2024-09-10T04:02:34.327 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T04:02:34.328 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"}]: dispatch 2024-09-10T04:02:34.328 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:34.328 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3854927346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T04:02:34.328 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:34.328 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:34.328 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] up:boot 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] up:active 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"}]: dispatch 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3854927346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:34.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:34.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-09-10T04:02:35.556 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:35.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:35 smithi070 ceph-mon[35495]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 189 B/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-10T04:02:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:35 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] up:standby-replay 2024-09-10T04:02:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:35 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:35 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:35 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:35 smithi046 ceph-mon[28328]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 189 B/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-10T04:02:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:35 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] up:standby-replay 2024-09-10T04:02:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:35 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:35 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:35 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 -- 172.21.15.46:0/3711085535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 msgr2=0x7fa630107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 --2- 172.21.15.46:0/3711085535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa630107110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa6240098b0 tx=0x7fa62402a4d0 comp rx=0 tx=0).stop 2024-09-10T04:02:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 -- 172.21.15.46:0/3711085535 shutdown_connections 2024-09-10T04:02:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 --2- 172.21.15.46:0/3711085535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa630107110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 --2- 172.21.15.46:0/3711085535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6301023f0 0x7fa6301047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:35.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 -- 172.21.15.46:0/3711085535 >> 172.21.15.46:0/3711085535 conn(0x7fa6300fc010 msgr2=0x7fa6300fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 -- 172.21.15.46:0/3711085535 shutdown_connections 2024-09-10T04:02:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.994+0000 7fa636719640 1 -- 172.21.15.46:0/3711085535 wait complete. 2024-09-10T04:02:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa636719640 1 Processor -- start 2024-09-10T04:02:35.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa636719640 1 -- start start 2024-09-10T04:02:35.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa636719640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6301023f0 0x7fa630101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:35.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa636719640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa6301003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:35.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa636719640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa63010a810 con 0x7fa630104d20 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa636719640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa63010a690 con 0x7fa6301023f0 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa6301003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa6301003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47388/0 (socket says 172.21.15.46:47388) 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa62f7fe640 1 -- 172.21.15.46:0/4197341955 learned_addr learned my addr 172.21.15.46:0/4197341955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa62ffff640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6301023f0 0x7fa630101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa62f7fe640 1 -- 172.21.15.46:0/4197341955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6301023f0 msgr2=0x7fa630101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa62f7fe640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6301023f0 0x7fa630101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.995+0000 7fa62f7fe640 1 -- 172.21.15.46:0/4197341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6301008e0 con 0x7fa630104d20 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.996+0000 7fa62ffff640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6301023f0 0x7fa630101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.996+0000 7fa62f7fe640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa6301003a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa624009230 tx=0x7fa6240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.996+0000 7fa62d7fa640 1 -- 172.21.15.46:0/4197341955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa62403f070 con 0x7fa630104d20 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.996+0000 7fa62d7fa640 1 -- 172.21.15.46:0/4197341955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa6240044d0 con 0x7fa630104d20 2024-09-10T04:02:35.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.996+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa630100b70 con 0x7fa630104d20 2024-09-10T04:02:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.996+0000 7fa62d7fa640 1 -- 172.21.15.46:0/4197341955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa624035030 con 0x7fa630104d20 2024-09-10T04:02:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.996+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa630101120 con 0x7fa630104d20 2024-09-10T04:02:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.997+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6301027d0 con 0x7fa630104d20 2024-09-10T04:02:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.998+0000 7fa62d7fa640 1 -- 172.21.15.46:0/4197341955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fa624042070 con 0x7fa630104d20 2024-09-10T04:02:35.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.998+0000 7fa62d7fa640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa61806de30 0x7fa6180702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:36.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.998+0000 7fa62d7fa640 1 -- 172.21.15.46:0/4197341955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fa6240ae7f0 con 0x7fa630104d20 2024-09-10T04:02:36.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.998+0000 7fa62ffff640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa61806de30 0x7fa6180702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:36.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:35.999+0000 7fa62ffff640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa61806de30 0x7fa6180702f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa620009930 tx=0x7fa620008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:36.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.002+0000 7fa62d7fa640 1 -- 172.21.15.46:0/4197341955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa62407c390 con 0x7fa630104d20 2024-09-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.219+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa63019f870 con 0x7fa630104d20 2024-09-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.220+0000 7fa62d7fa640 1 -- 172.21.15.46:0/4197341955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5352 (secure 0 0 0) 0x7fa624080040 con 0x7fa630104d20 2024-09-10T04:02:36.220 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 15 2024-09-10T04:02:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa61806de30 msgr2=0x7fa6180702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa61806de30 0x7fa6180702f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa620009930 tx=0x7fa620008040 comp rx=0 tx=0).stop 2024-09-10T04:02:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 msgr2=0x7fa6301003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa6301003a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa624009230 tx=0x7fa6240048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 shutdown_connections 2024-09-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa630104d20 0x7fa6301003a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fa61806de30 0x7fa6180702f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 --2- 172.21.15.46:0/4197341955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6301023f0 0x7fa630101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 >> 172.21.15.46:0/4197341955 conn(0x7fa6300fc010 msgr2=0x7fa6300fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 shutdown_connections 2024-09-10T04:02:36.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:36.222+0000 7fa636719640 1 -- 172.21.15.46:0/4197341955 wait complete. 2024-09-10T04:02:36.231 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:02:37.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:37 smithi046 ceph-mon[28328]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T04:02:37.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:37 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] up:active 2024-09-10T04:02:37.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:37 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:37.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:37 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/4197341955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T04:02:37.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-09-10T04:02:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:37 smithi070 ceph-mon[35495]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T04:02:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:37 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] up:active 2024-09-10T04:02:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:37 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:37 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/4197341955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T04:02:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:38.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:38 smithi046 ceph-mon[28328]: mds.? [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] up:standby-replay 2024-09-10T04:02:38.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:38 smithi046 ceph-mon[28328]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:38.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:38 smithi070 ceph-mon[35495]: mds.? [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] up:standby-replay 2024-09-10T04:02:38.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:38 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:02:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 -- 172.21.15.46:0/2483299333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328101960 msgr2=0x7f9328101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 --2- 172.21.15.46:0/2483299333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328101960 0x7f9328101d60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9318009880 tx=0x7f931802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 -- 172.21.15.46:0/2483299333 shutdown_connections 2024-09-10T04:02:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 --2- 172.21.15.46:0/2483299333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9328102b60 0x7f9328102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 --2- 172.21.15.46:0/2483299333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328101960 0x7f9328101d60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 -- 172.21.15.46:0/2483299333 >> 172.21.15.46:0/2483299333 conn(0x7f93280fd0f0 msgr2=0x7f93280ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:39.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 -- 172.21.15.46:0/2483299333 shutdown_connections 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.029+0000 7f932d605640 1 -- 172.21.15.46:0/2483299333 wait complete. 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f932d605640 1 Processor -- start 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f932d605640 1 -- start start 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f932d605640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9328101960 0x7f93281996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f932d605640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328102b60 0x7f9328199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f932d605640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f932810a7a0 con 0x7f9328102b60 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f932d605640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f932810a620 con 0x7f9328101960 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f93267fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328102b60 0x7f9328199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f93267fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328102b60 0x7f9328199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41892/0 (socket says 172.21.15.46:41892) 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f93267fc640 1 -- 172.21.15.46:0/658241460 learned_addr learned my addr 172.21.15.46:0/658241460 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.030+0000 7f9326ffd640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9328101960 0x7f93281996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f93267fc640 1 -- 172.21.15.46:0/658241460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9328101960 msgr2=0x7f93281996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:39.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f93267fc640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9328101960 0x7f93281996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f93267fc640 1 -- 172.21.15.46:0/658241460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932819a290 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f93267fc640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328102b60 0x7f9328199c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f931c00d6f0 tx=0x7f931c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f930ffff640 1 -- 172.21.15.46:0/658241460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f931c017070 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f930ffff640 1 -- 172.21.15.46:0/658241460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f931c00de00 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932819ecd0 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f930ffff640 1 -- 172.21.15.46:0/658241460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f931c004bb0 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.031+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932819f200 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.032+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9328101d60 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.033+0000 7f930ffff640 1 -- 172.21.15.46:0/658241460 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f931c004e60 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.033+0000 7f930ffff640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f931006de30 0x7f93100702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.033+0000 7f9326ffd640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f931006de30 0x7f93100702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.033+0000 7f930ffff640 1 -- 172.21.15.46:0/658241460 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f931c08d6d0 con 0x7f9328102b60 2024-09-10T04:02:39.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.034+0000 7f9326ffd640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f931006de30 0x7f93100702f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f93180058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:39.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.038+0000 7f930ffff640 1 -- 172.21.15.46:0/658241460 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f931c05ef20 con 0x7f9328102b60 2024-09-10T04:02:39.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.314+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) -- 0x7f9328062df0 con 0x7f9328102b60 2024-09-10T04:02:39.315 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:39 smithi046 ceph-mon[28328]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-10T04:02:39.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.314+0000 7f930ffff640 1 -- 172.21.15.46:0/658241460 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v16) ==== 78+0+98 (secure 0 0 0) 0x7f931c063750 con 0x7f9328102b60 2024-09-10T04:02:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.316+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f931006de30 msgr2=0x7f93100702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.316+0000 7f932d605640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f931006de30 0x7f93100702f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9318009230 tx=0x7f93180058d0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.316+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328102b60 msgr2=0x7f9328199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.316+0000 7f932d605640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328102b60 0x7f9328199c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f931c00d6f0 tx=0x7f931c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.317+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 shutdown_connections 2024-09-10T04:02:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.317+0000 7f932d605640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9328102b60 0x7f9328199c10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.317+0000 7f932d605640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f931006de30 0x7f93100702f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.317+0000 7f932d605640 1 --2- 172.21.15.46:0/658241460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9328101960 0x7f93281996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.317+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 >> 172.21.15.46:0/658241460 conn(0x7f93280fd0f0 msgr2=0x7f9328104050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.317+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 shutdown_connections 2024-09-10T04:02:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:39.317+0000 7f932d605640 1 -- 172.21.15.46:0/658241460 wait complete. 2024-09-10T04:02:39.326 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:02:39.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:39 smithi070 ceph-mon[35495]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-09-10T04:02:39.905 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-09-10T04:02:39.919 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T04:02:40.197 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:40.451 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:40.452 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:40.452 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:40 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/658241460' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-10T04:02:40.452 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:40.452 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:40.452 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:40.452 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:40 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:40.496 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:40.497 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:40.497 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:40 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/658241460' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-10T04:02:40.497 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:40.497 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:40.497 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:40.497 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:40 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:02:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 -- 172.21.15.46:0/342966034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb0102ac0 msgr2=0x7f9fb0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 --2- 172.21.15.46:0/342966034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb0102ac0 0x7f9fb0102ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009c20 tx=0x7f9fa002aa00 comp rx=0 tx=0).stop 2024-09-10T04:02:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 -- 172.21.15.46:0/342966034 shutdown_connections 2024-09-10T04:02:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 --2- 172.21.15.46:0/342966034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb01033e0 0x7f9fb010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:41.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 --2- 172.21.15.46:0/342966034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb0102ac0 0x7f9fb0102ea0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 -- 172.21.15.46:0/342966034 >> 172.21.15.46:0/342966034 conn(0x7f9fb00fc520 msgr2=0x7f9fb00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 -- 172.21.15.46:0/342966034 shutdown_connections 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.280+0000 7f9fb776a640 1 -- 172.21.15.46:0/342966034 wait complete. 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb776a640 1 Processor -- start 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb776a640 1 -- start start 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb776a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb0102ac0 0x7f9fb019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb776a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb01033e0 0x7f9fb019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb776a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb010edf0 con 0x7f9fb01033e0 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb776a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9fb010ec70 con 0x7f9fb0102ac0 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb4d05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb01033e0 0x7f9fb019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb4d05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb01033e0 0x7f9fb019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41904/0 (socket says 172.21.15.46:41904) 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb4d05640 1 -- 172.21.15.46:0/3000227467 learned_addr learned my addr 172.21.15.46:0/3000227467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:41.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.281+0000 7f9fb5506640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb0102ac0 0x7f9fb019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fb4d05640 1 -- 172.21.15.46:0/3000227467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb0102ac0 msgr2=0x7f9fb019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fb4d05640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb0102ac0 0x7f9fb019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fb4d05640 1 -- 172.21.15.46:0/3000227467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fb01977e0 con 0x7f9fb01033e0 2024-09-10T04:02:41.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fb4d05640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb01033e0 0x7f9fb019dba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e970 tx=0x7f9fa400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fae7fc640 1 -- 172.21.15.46:0/3000227467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9fa400cac0 con 0x7f9fb01033e0 2024-09-10T04:02:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fae7fc640 1 -- 172.21.15.46:0/3000227467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fa40054e0 con 0x7f9fb01033e0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fb0197ad0 con 0x7f9fb01033e0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fae7fc640 1 -- 172.21.15.46:0/3000227467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9fa4004dd0 con 0x7f9fb01033e0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.282+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fb0198060 con 0x7f9fb01033e0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.283+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fb006aa90 con 0x7f9fb01033e0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.284+0000 7f9fae7fc640 1 -- 172.21.15.46:0/3000227467 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f9fa4005020 con 0x7f9fb01033e0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.285+0000 7f9fae7fc640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9f9806de30 0x7f9f980702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.285+0000 7f9fae7fc640 1 -- 172.21.15.46:0/3000227467 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f9fa408e0b0 con 0x7f9fb01033e0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.285+0000 7f9fb5506640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9f9806de30 0x7f9f980702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:41.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.285+0000 7f9fb5506640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9f9806de30 0x7f9f980702f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009430 tx=0x7f9fa003a040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:41.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.289+0000 7f9fae7fc640 1 -- 172.21.15.46:0/3000227467 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fa405bcd0 con 0x7f9fb01033e0 2024-09-10T04:02:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.511+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fb0105f90 con 0x7f9fb01033e0 2024-09-10T04:02:41.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.512+0000 7f9fae7fc640 1 -- 172.21.15.46:0/3000227467 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5351 (secure 0 0 0) 0x7f9fa405f980 con 0x7f9fb01033e0 2024-09-10T04:02:41.513 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:02:41.513 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"btime":"2024-09-10T04:02:37:146557+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":16,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:02:36.662727+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14558,"mds_1":24325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14558":{"gid":14558,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6827/4038650990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4038650990},{"type":"v1","addr":"172.21.15.46:6827","nonce":4038650990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14574":{"gid":14574,"name":"cephfs.smithi046.cpwyud","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.46:6829/1498169047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1498169047},{"type":"v1","addr":"172.21.15.46:6829","nonce":1498169047}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24325":{"gid":24325,"name":"cephfs.smithi070.ehvjgr","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6825/1155442666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":1155442666},{"type":"v1","addr":"172.21.15.70:6825","nonce":1155442666}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24339":{"gid":24339,"name":"cephfs.smithi070.myztiz","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.70:6827/3341360432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":3341360432},{"type":"v1","addr":"172.21.15.70:6827","nonce":3341360432}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14558,"qdb_cluster":[14558,24325]},"id":1}]} 2024-09-10T04:02:41.513 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9f9806de30 msgr2=0x7f9f980702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9f9806de30 0x7f9f980702f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009430 tx=0x7f9fa003a040 comp rx=0 tx=0).stop 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb01033e0 msgr2=0x7f9fb019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb01033e0 0x7f9fb019dba0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e970 tx=0x7f9fa400ee30 comp rx=0 tx=0).stop 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 shutdown_connections 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb01033e0 0x7f9fb019dba0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f9f9806de30 0x7f9f980702f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 --2- 172.21.15.46:0/3000227467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9fb0102ac0 0x7f9fb019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 >> 172.21.15.46:0/3000227467 conn(0x7f9fb00fc520 msgr2=0x7f9fb0108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:41.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 shutdown_connections 2024-09-10T04:02:41.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:41.514+0000 7f9fb776a640 1 -- 172.21.15.46:0/3000227467 wait complete. 2024-09-10T04:02:41.733 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:41 smithi046 ceph-mon[28328]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T04:02:41.733 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:41 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3000227467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T04:02:42.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:41 smithi070 ceph-mon[35495]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T04:02:42.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:41 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3000227467' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T04:02:42.329 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 50} 2024-09-10T04:02:42.329 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T04:02:42.390 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T04:02:42.391 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-09-10T04:02:42.391 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-10T04:02:42.391 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T04:02:42.391 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:42.391 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-10T04:02:42.391 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T04:02:42.391 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:42.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:42.391 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T04:02:42.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:42.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2024-09-10T04:02:42.478 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2024-09-10T04:02:42.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:02:42.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:42.480 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-10T04:02:42.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:42.499 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-09-10T04:02:42.572 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-09-10T04:02:42.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:02:42.573 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T04:02:42.574 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T04:02:42.574 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T04:02:43.158 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:42 smithi046 ceph-mon[28328]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T04:02:43.158 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:43.158 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:42 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:42 smithi070 ceph-mon[35495]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-10T04:02:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:42 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:43.561 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 -- 172.21.15.46:0/3837315731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc100610 msgr2=0x7f85bc100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/3837315731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc100610 0x7f85bc100a90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f85ac009880 tx=0x7f85ac02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 -- 172.21.15.46:0/3837315731 shutdown_connections 2024-09-10T04:02:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/3837315731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc100610 0x7f85bc100a90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/3837315731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85bc1066a0 0x7f85bc106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 -- 172.21.15.46:0/3837315731 >> 172.21.15.46:0/3837315731 conn(0x7f85bc0fc310 msgr2=0x7f85bc0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 -- 172.21.15.46:0/3837315731 shutdown_connections 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.573+0000 7f85c2f9d640 1 -- 172.21.15.46:0/3837315731 wait complete. 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c2f9d640 1 Processor -- start 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c2f9d640 1 -- start start 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c2f9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85bc100610 0x7f85bc197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c2f9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc1066a0 0x7f85bc197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c2f9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc10db30 con 0x7f85bc1066a0 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c2f9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f85bc10d9b0 con 0x7f85bc100610 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c179a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc1066a0 0x7f85bc197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c179a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc1066a0 0x7f85bc197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41918/0 (socket says 172.21.15.46:41918) 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c179a640 1 -- 172.21.15.46:0/518578388 learned_addr learned my addr 172.21.15.46:0/518578388 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c1f9b640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85bc100610 0x7f85bc197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c179a640 1 -- 172.21.15.46:0/518578388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85bc100610 msgr2=0x7f85bc197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c179a640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85bc100610 0x7f85bc197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.574+0000 7f85c179a640 1 -- 172.21.15.46:0/518578388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85bc19c010 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.575+0000 7f85c179a640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc1066a0 0x7f85bc197c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f85ac031460 tx=0x7f85ac0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.575+0000 7f85b2ffd640 1 -- 172.21.15.46:0/518578388 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85ac03f070 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.575+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85bc19c2a0 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.575+0000 7f85b2ffd640 1 -- 172.21.15.46:0/518578388 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85ac004cf0 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.575+0000 7f85b2ffd640 1 -- 172.21.15.46:0/518578388 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85ac035030 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.575+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85bc19c850 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.576+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85bc101d50 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.577+0000 7f85b2ffd640 1 -- 172.21.15.46:0/518578388 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f85ac042070 con 0x7f85bc1066a0 2024-09-10T04:02:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.577+0000 7f85b2ffd640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f85a806de90 0x7f85a8070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.577+0000 7f85c1f9b640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f85a806de90 0x7f85a8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.577+0000 7f85b2ffd640 1 -- 172.21.15.46:0/518578388 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f85ac0ae5a0 con 0x7f85bc1066a0 2024-09-10T04:02:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.578+0000 7f85c1f9b640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f85a806de90 0x7f85a8070350 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009890 tx=0x7f85b8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.581+0000 7f85b2ffd640 1 -- 172.21.15.46:0/518578388 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85ac07c1c0 con 0x7f85bc1066a0 2024-09-10T04:02:44.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.811+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f85bc198aa0 con 0x7f85bc1066a0 2024-09-10T04:02:44.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.812+0000 7f85b2ffd640 1 -- 172.21.15.46:0/518578388 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+83 (secure 0 0 0) 0x7f85ac07fe70 con 0x7f85bc1066a0 2024-09-10T04:02:44.813 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-10T04:02:44.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f85a806de90 msgr2=0x7f85a8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f85a806de90 0x7f85a8070350 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009890 tx=0x7f85b8008040 comp rx=0 tx=0).stop 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc1066a0 msgr2=0x7f85bc197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc1066a0 0x7f85bc197c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f85ac031460 tx=0x7f85ac0048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 shutdown_connections 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85bc1066a0 0x7f85bc197c80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f85a806de90 0x7f85a8070350 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 --2- 172.21.15.46:0/518578388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85bc100610 0x7f85bc197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 >> 172.21.15.46:0/518578388 conn(0x7f85bc0fc310 msgr2=0x7f85bc069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 shutdown_connections 2024-09-10T04:02:44.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:44.814+0000 7f85c2f9d640 1 -- 172.21.15.46:0/518578388 wait complete. 2024-09-10T04:02:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:44 smithi046 ceph-mon[28328]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-10T04:02:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:44 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/518578388' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T04:02:45.253 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:44 smithi070 ceph-mon[35495]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-10T04:02:45.254 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:44 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/518578388' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T04:02:45.476 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T04:02:45.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:45.477 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-09-10T04:02:45.494 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T04:02:45.494 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:45.495 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-09-10T04:02:45.496 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T04:02:45.496 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42292sec preferred_lft 42292sec 2024-09-10T04:02:45.496 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-09-10T04:02:45.496 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-09-10T04:02:45.496 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T04:02:45.496 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:45.496 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-09-10T04:02:45.496 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T04:02:45.497 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:45.497 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T04:02:45.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-09-10T04:02:45.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-09-10T04:02:45.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-09-10T04:02:45.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T04:02:45.497 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T04:02:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:45.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:45.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:45.767 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T04:02:45.795 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-09-10T04:02:45.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:45.797 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-09-10T04:02:45.876 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-09-10T04:02:45.876 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T04:02:45.876 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T04:02:45.876 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T04:02:45.876 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T04:02:45.877 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:45.877 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T04:02:45.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T04:02:45.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T04:02:45.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T04:02:45.877 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T04:02:45.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:46.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:46.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:46.199 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-09-10T04:02:46.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:46.215 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-09-10T04:02:46.272 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:46.272 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T04:02:46.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T04:02:46.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T04:02:46.272 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T04:02:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:46.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:46.392 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T04:02:46.393 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T04:02:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:46.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:46.525 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:46.526 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-09-10T04:02:46.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-09-10T04:02:46.526 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T04:02:46.526 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-09-10T04:02:46.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:46.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:46.605 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T04:02:46.605 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T04:02:46.605 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:46.621 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T04:02:46.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T04:02:46.622 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:46.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-10T04:02:46.745 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T04:02:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-10T04:02:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-10T04:02:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-10T04:02:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-10T04:02:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-10T04:02:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-10T04:02:46.802 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-10T04:02:46.803 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4a7c44f27de81d236783bb25e05a1a4904cd4b0ce4b018e4ccb17272ba8abdb3/merged 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/797e96acd81cccdee6231c95c50371f77002a242706b3a7cbfacda2897346947/merged 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/3f7b1b20b2db93118c209219ef9d4108aaf005dbe95560e4b725df7f29f6276c/merged 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4e48650ca23388f546acdfe0c7c4470b94a57ef3ac1fa9b4e3026a78e1f70326/merged 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/61fc3c03a5214404a5a9353eec0a51bdbf0e8c40a1883986c393181a02c711df/merged 2024-09-10T04:02:46.804 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/13641633fdf9d48cf658982c836ceb15ad91b0cec8f3dc607242995ab52d0b80/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4ef36c03e93ff3ab5d11857869befeb94c6b7658dbb1d63afb512b7ace425f94/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/984f3322ac093e3f2a042e7b6fcb757ebf1361a3314210ce6e6500362dc7816f/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/5ba6fd1be59ab98a1564c30509dde2354b68760048a21bd516a5896f3bde4ac7/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ac4010ba3d131067f038416cd2563acda9de7201adc20324d060c6f36d0bd0b9/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f38ba55d7bd0060bc1affc26c7aaba66d8b06c22f5663c33a6ed2bb3f40926ce/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4b5250609674467230abb90d123b929fd887e815e890f834491cf0deb6f6bd82/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/b4f11c8c41f533ddb266fad9531c671a65950a6aa738c82fb48d03847004a45a/merged 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T04:02:46.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:46.806 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-10T04:02:46.858 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T04:02:46.858 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T04:02:46.901 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-09-10T04:02:46.926 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T04:02:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2024-09-10T04:02:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2024-09-10T04:02:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2024-09-10T04:02:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2024-09-10T04:02:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2024-09-10T04:02:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2024-09-10T04:02:46.984 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/selinux 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2024-09-10T04:02:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4a7c44f27de81d236783bb25e05a1a4904cd4b0ce4b018e4ccb17272ba8abdb3/merged 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/797e96acd81cccdee6231c95c50371f77002a242706b3a7cbfacda2897346947/merged 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/0 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/3f7b1b20b2db93118c209219ef9d4108aaf005dbe95560e4b725df7f29f6276c/merged 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4e48650ca23388f546acdfe0c7c4470b94a57ef3ac1fa9b4e3026a78e1f70326/merged 2024-09-10T04:02:46.986 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/61fc3c03a5214404a5a9353eec0a51bdbf0e8c40a1883986c393181a02c711df/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/13641633fdf9d48cf658982c836ceb15ad91b0cec8f3dc607242995ab52d0b80/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4ef36c03e93ff3ab5d11857869befeb94c6b7658dbb1d63afb512b7ace425f94/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/984f3322ac093e3f2a042e7b6fcb757ebf1361a3314210ce6e6500362dc7816f/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/5ba6fd1be59ab98a1564c30509dde2354b68760048a21bd516a5896f3bde4ac7/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/ac4010ba3d131067f038416cd2563acda9de7201adc20324d060c6f36d0bd0b9/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/f38ba55d7bd0060bc1affc26c7aaba66d8b06c22f5663c33a6ed2bb3f40926ce/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/4b5250609674467230abb90d123b929fd887e815e890f834491cf0deb6f6bd82/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/containers/storage/overlay/b4f11c8c41f533ddb266fad9531c671a65950a6aa738c82fb48d03847004a45a/merged 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T04:02:46.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:46.988 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-09-10T04:02:47.029 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-09-10T04:02:47.029+0000 7f7214874480 -1 init, newargv = 0x56136115a130 newargc=15 2024-09-10T04:02:47.030 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[76450]: starting ceph client 2024-09-10T04:02:47.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[76450]: starting fuse 2024-09-10T04:02:47.042 INFO:teuthology.orchestra.run.smithi046.stdout:165 2024-09-10T04:02:47.042 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [165] 2024-09-10T04:02:47.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2024-09-10T04:02:47.042 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-09-10T04:02:47.042 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-09-10T04:02:47.043 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-09-10T04:02:47.044 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-09-10T04:02:47.163 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.76450.asok 2024-09-10T04:02:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:47.174 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76450.asok 2024-09-10T04:02:47.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:47.174 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76450.asok status 2024-09-10T04:02:47.311 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:02:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-09-10T04:02:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-10T04:02:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-10T04:02:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "76450", 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 14606, 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 14606 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-10T04:02:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-10T04:02:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-09-10T04:02:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1648660616 2024-09-10T04:02:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:02:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:02:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-09-10T04:02:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1648660616 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.14606 192.168.0.1:0/1648660616", 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/1648660616", 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 16, 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 43, 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-09-10T04:02:47.316 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:02:47.318 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:46 smithi046 ceph-mon[28328]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-10T04:02:47.318 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:47.318 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:47.319 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:47.319 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:47.319 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:47.320 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T04:02:47.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T04:02:47.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:46 smithi070 ceph-mon[35495]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-10T04:02:47.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:47.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:47.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:02:47.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:02:47.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:02:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 -- 172.21.15.46:0/1909838229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 msgr2=0x7fe3ac074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1909838229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac074970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe39c009c20 tx=0x7fe39c02aa00 comp rx=0 tx=0).stop 2024-09-10T04:02:48.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 -- 172.21.15.46:0/1909838229 shutdown_connections 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1909838229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3ac074eb0 0x7fe3ac1123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1909838229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac074970 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 -- 172.21.15.46:0/1909838229 >> 172.21.15.46:0/1909838229 conn(0x7fe3ac0fd600 msgr2=0x7fe3ac0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 -- 172.21.15.46:0/1909838229 shutdown_connections 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.044+0000 7fe3b3970640 1 -- 172.21.15.46:0/1909838229 wait complete. 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b3970640 1 Processor -- start 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b3970640 1 -- start start 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b3970640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b3970640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3ac074eb0 0x7fe3ac19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:48.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b3970640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac1153d0 con 0x7fe3ac074590 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b3970640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac115250 con 0x7fe3ac074eb0 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b170c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b170c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41944/0 (socket says 172.21.15.46:41944) 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.045+0000 7fe3b170c640 1 -- 172.21.15.46:0/1734916462 learned_addr learned my addr 172.21.15.46:0/1734916462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b0f0b640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3ac074eb0 0x7fe3ac19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b170c640 1 -- 172.21.15.46:0/1734916462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3ac074eb0 msgr2=0x7fe3ac19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b170c640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3ac074eb0 0x7fe3ac19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b170c640 1 -- 172.21.15.46:0/1734916462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac1a28a0 con 0x7fe3ac074590 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b0f0b640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3ac074eb0 0x7fe3ac19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b170c640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac19dfd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe39c02af20 tx=0x7fe39c02ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:48.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3a27fc640 1 -- 172.21.15.46:0/1734916462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe39c047070 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac1a2b30 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3a27fc640 1 -- 172.21.15.46:0/1734916462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe39c003e30 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3a27fc640 1 -- 172.21.15.46:0/1734916462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe39c033050 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.046+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac1a3060 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.047+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ac075cd0 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.048+0000 7fe3a27fc640 1 -- 172.21.15.46:0/1734916462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fe39c04a070 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.048+0000 7fe3a27fc640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe39406de30 0x7fe3940702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.048+0000 7fe3a27fc640 1 -- 172.21.15.46:0/1734916462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fe39c03e030 con 0x7fe3ac074590 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.048+0000 7fe3b0f0b640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe39406de30 0x7fe3940702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.049+0000 7fe3b0f0b640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe39406de30 0x7fe3940702f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac104880 tx=0x7fe3a800b5f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:48.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.053+0000 7fe3a27fc640 1 -- 172.21.15.46:0/1734916462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe39c07c860 con 0x7fe3ac074590 2024-09-10T04:02:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.263+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe3ac19f330 con 0x7fe3ac074590 2024-09-10T04:02:48.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.263+0000 7fe3a27fc640 1 -- 172.21.15.46:0/1734916462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+83 (secure 0 0 0) 0x7fe39c080510 con 0x7fe3ac074590 2024-09-10T04:02:48.264 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-10T04:02:48.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe39406de30 msgr2=0x7fe3940702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:48.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe39406de30 0x7fe3940702f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac104880 tx=0x7fe3a800b5f0 comp rx=0 tx=0).stop 2024-09-10T04:02:48.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 msgr2=0x7fe3ac19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:48.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac19dfd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe39c02af20 tx=0x7fe39c02ee60 comp rx=0 tx=0).stop 2024-09-10T04:02:48.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 shutdown_connections 2024-09-10T04:02:48.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fe39406de30 0x7fe3940702f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:48.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3ac074eb0 0x7fe3ac19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:48.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 --2- 172.21.15.46:0/1734916462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3ac074590 0x7fe3ac19dfd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:48.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 >> 172.21.15.46:0/1734916462 conn(0x7fe3ac0fd600 msgr2=0x7fe3ac104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:48.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 shutdown_connections 2024-09-10T04:02:48.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:48.265+0000 7fe3b3970640 1 -- 172.21.15.46:0/1734916462 wait complete. 2024-09-10T04:02:48.889 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T04:02:48.889 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T04:02:48.889 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2024-09-10T04:02:48.889 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T04:02:48.889 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:48.890 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T04:02:48.890 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T04:02:48.890 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T04:02:48.890 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T04:02:48.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:48.890 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2024-09-10T04:02:48.910 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:48.911 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42271sec preferred_lft 42271sec 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2024-09-10T04:02:48.912 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2024-09-10T04:02:48.912 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T04:02:48.913 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:48.913 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T04:02:48.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2024-09-10T04:02:48.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2024-09-10T04:02:48.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2024-09-10T04:02:48.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T04:02:48.913 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T04:02:49.010 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:49.123 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:49.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:49.129 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T04:02:49.153 INFO:teuthology.orchestra.run.smithi070.stdout:1 2024-09-10T04:02:49.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:49.155 DEBUG:teuthology.orchestra.run.smithi070:> route 2024-09-10T04:02:49.216 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2024-09-10T04:02:49.216 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T04:02:49.216 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T04:02:49.216 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T04:02:49.217 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T04:02:49.217 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:49.217 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T04:02:49.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T04:02:49.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T04:02:49.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T04:02:49.217 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T04:02:49.304 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:49.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:49 smithi070 ceph-mon[35495]: pgmap v188: 65 pgs: 65 active+clean; 581 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 4 op/s 2024-09-10T04:02:49.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:49 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1734916462' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T04:02:49.312 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:49.323 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:49 smithi046 ceph-mon[28328]: pgmap v188: 65 pgs: 65 active+clean; 581 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 85 B/s wr, 4 op/s 2024-09-10T04:02:49.323 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:49 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1734916462' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T04:02:49.323 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:02:49.517 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:49.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:49.524 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2024-09-10T04:02:49.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:49.540 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2024-09-10T04:02:49.594 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:49.595 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T04:02:49.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T04:02:49.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T04:02:49.595 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T04:02:49.683 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:49.712 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T04:02:49.713 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T04:02:49.761 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:49.834 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:49.841 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T04:02:49.841 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2024-09-10T04:02:49.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2024-09-10T04:02:49.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T04:02:49.841 DEBUG:teuthology.orchestra.run.smithi070:> ') 2024-09-10T04:02:49.889 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T04:02:49.917 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:49.922 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T04:02:49.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T04:02:49.923 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:49.938 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T04:02:49.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T04:02:49.938 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:49.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2024-09-10T04:02:50.059 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2024-09-10T04:02:50.116 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/bpf 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T04:02:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/0 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/e2797147c3a97991ddd354b25a1f33fa3452131836d6a5f29e8318a7351bdcdd/merged 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/7b067f1c0d161cad6f2f41ecb21bf353f9f960875add2bc333ac6971cccf88aa/merged 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ae98d3a6f0e14c9435a13efda46346d43c6885ec362ffd48360d85892b4701a1/merged 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/b17ff58a5bdfaae317fc376db15484d085827c2a232296dfc7de5582012555ec/merged 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/d39615bfc3be69e5510c53b891c6d5e35f334612d76258ecabe499804d02170b/merged 2024-09-10T04:02:50.118 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ab6e817dbd48cb0ff2e972acea5dd205a7a6e0c330c82f04691fd400b9d72535/merged 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/7ec0a60afd0d41671ff3ff841d98ec2f7ae77ae3760ce1afad9a1d27d3fe85bb/merged 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/d492195217beea6851262a48d1181a691a24fbd6930bcc9366f006b08da1fa77/merged 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/02945a3d6e9ebb7f5f8761b746441c48d7a563f3546e85a454ff21db66f2435d/merged 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/0a5c6155b2b8ee2c0d51c0dc4b1464a4c3309d60ee3550f421434d3c96a598a2/merged 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T04:02:50.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:50.119 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2024-09-10T04:02:50.171 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T04:02:50.171 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T04:02:50.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2024-09-10T04:02:50.238 DEBUG:teuthology.orchestra.run.smithi070:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T04:02:50.294 INFO:teuthology.orchestra.run.smithi070.stdout:/proc 2024-09-10T04:02:50.294 INFO:teuthology.orchestra.run.smithi070.stdout:/sys 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/dev 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/security 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/shm 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/pts 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/run 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/cgroup 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/pstore 2024-09-10T04:02:50.295 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/bpf 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/ 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/selinux 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/debug 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/mqueue 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/dev/hugepages 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/tracing 2024-09-10T04:02:50.296 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/fs/fuse/connections 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/sys/kernel/config 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/1000 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/ceph 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/run/user/0 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/e2797147c3a97991ddd354b25a1f33fa3452131836d6a5f29e8318a7351bdcdd/merged 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/7b067f1c0d161cad6f2f41ecb21bf353f9f960875add2bc333ac6971cccf88aa/merged 2024-09-10T04:02:50.297 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ae98d3a6f0e14c9435a13efda46346d43c6885ec362ffd48360d85892b4701a1/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/b17ff58a5bdfaae317fc376db15484d085827c2a232296dfc7de5582012555ec/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/d39615bfc3be69e5510c53b891c6d5e35f334612d76258ecabe499804d02170b/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/ab6e817dbd48cb0ff2e972acea5dd205a7a6e0c330c82f04691fd400b9d72535/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/7ec0a60afd0d41671ff3ff841d98ec2f7ae77ae3760ce1afad9a1d27d3fe85bb/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/d492195217beea6851262a48d1181a691a24fbd6930bcc9366f006b08da1fa77/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/02945a3d6e9ebb7f5f8761b746441c48d7a563f3546e85a454ff21db66f2435d/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/var/lib/containers/storage/overlay/0a5c6155b2b8ee2c0d51c0dc4b1464a4c3309d60ee3550f421434d3c96a598a2/merged 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run.smithi070.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T04:02:50.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:50.299 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2024-09-10T04:02:50.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi070.stderr:2024-09-10T04:02:50.334+0000 7f3ff2e2e480 -1 init, newargv = 0x559cb390f0b0 newargc=15 2024-09-10T04:02:50.336 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi070.stderr:ceph-fuse[61700]: starting ceph client 2024-09-10T04:02:50.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi070.stderr:ceph-fuse[61700]: starting fuse 2024-09-10T04:02:50.351 INFO:teuthology.orchestra.run.smithi070.stdout:129 2024-09-10T04:02:50.351 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [129] 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> import re 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> import os 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2024-09-10T04:02:50.352 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> return f 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.1")) 2024-09-10T04:02:50.353 DEBUG:teuthology.orchestra.run.smithi070:> ' 2024-09-10T04:02:50.469 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.1.61700.asok 2024-09-10T04:02:50.472 INFO:teuthology.orchestra.run.smithi070.stderr:2024-09-10T04:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T04:02:50.479 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.61700.asok 2024-09-10T04:02:50.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:50.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.61700.asok status 2024-09-10T04:02:50.611 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc", 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "1", 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "61700", 2024-09-10T04:02:50.612 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 24367, 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 24367 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-10T04:02:50.613 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2709434340 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2709434340 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.24367 192.168.0.1:0/2709434340", 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2709434340", 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 16, 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 43, 2024-09-10T04:02:50.614 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2024-09-10T04:02:50.615 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2024-09-10T04:02:50.615 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2024-09-10T04:02:50.615 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-10T04:02:50.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:50.620 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-09-10T04:02:50.620 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:50.641 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-09-10T04:02:50.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:50.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:50.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:50.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:50.739 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2024-09-10T04:02:50.739 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:50.755 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2024-09-10T04:02:50.755 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:50.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:02:50.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:51.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:51 smithi070 ceph-mon[35495]: pgmap v189: 65 pgs: 65 active+clean; 581 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-10T04:02:51.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:51 smithi046 ceph-mon[28328]: pgmap v189: 65 pgs: 65 active+clean; 581 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-10T04:02:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:53 smithi046 ceph-mon[28328]: pgmap v190: 65 pgs: 65 active+clean; 582 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-10T04:02:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:53 smithi070 ceph-mon[35495]: pgmap v190: 65 pgs: 65 active+clean; 582 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-10T04:02:54.589 INFO:teuthology.run_tasks:Running task print... 2024-09-10T04:02:54.599 INFO:teuthology.task.print:**** done client 2024-09-10T04:02:54.599 INFO:teuthology.run_tasks:Running task parallel... 2024-09-10T04:02:54.643 INFO:teuthology.task.parallel:starting parallel... 2024-09-10T04:02:54.643 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-10T04:02:54.644 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T04:02:54.644 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:02:54.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-10T04:02:54.645 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-10T04:02:54.645 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-09-10T04:02:54.649 INFO:tasks.workunit:Pulling workunits from ref 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:02:54.650 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T04:02:54.650 INFO:tasks.workunit:timeout=3h 2024-09-10T04:02:54.650 INFO:tasks.workunit:cleanup=True 2024-09-10T04:02:54.650 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:54.668 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T04:02:54.668 INFO:teuthology.orchestra.run.smithi046.stdout:Device: a5h/165d Inode: 1 Links: 2 2024-09-10T04:02:54.669 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T04:02:54.669 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T04:02:54.669 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-10T04:02:54.669 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-09-10 04:02:24.952941112 +0000 2024-09-10T04:02:54.669 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-09-10 04:02:50.709583482 +0000 2024-09-10T04:02:54.669 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2024-09-10T04:02:54.669 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T04:02:54.669 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T04:02:54.740 DEBUG:teuthology.orchestra.run.smithi070:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:54.761 INFO:teuthology.orchestra.run.smithi070.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 81h/129d Inode: 1 Links: 3 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-10 04:02:54.733048497 +0000 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-10 04:02:54.733048497 +0000 2024-09-10T04:02:54.762 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: - 2024-09-10T04:02:54.762 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-10T04:02:54.762 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-10T04:02:54.838 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:02:54.838 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:02:54.856 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T04:02:54.896 INFO:tasks.workunit.client.1.smithi070.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-10T04:02:54.917 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:55.302 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:55 smithi046 ceph-mon[28328]: pgmap v191: 65 pgs: 65 active+clean; 582 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-10T04:02:55.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:55 smithi070 ceph-mon[35495]: pgmap v191: 65 pgs: 65 active+clean; 582 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s rd, 682 B/s wr, 4 op/s 2024-09-10T04:02:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 -- 172.21.15.46:0/3410118459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 msgr2=0x7f60141005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 --2- 172.21.15.46:0/3410118459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f60141005e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6010009880 tx=0x7f601002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 -- 172.21.15.46:0/3410118459 shutdown_connections 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 --2- 172.21.15.46:0/3410118459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6014100b20 0x7f6014100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 --2- 172.21.15.46:0/3410118459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f60141005e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 -- 172.21.15.46:0/3410118459 >> 172.21.15.46:0/3410118459 conn(0x7f60140fc010 msgr2=0x7f60140fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 -- 172.21.15.46:0/3410118459 shutdown_connections 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 -- 172.21.15.46:0/3410118459 wait complete. 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 Processor -- start 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.390+0000 7f601b5c5640 1 -- start start 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f601b5c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f6014199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f601b5c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6014100b20 0x7f6014199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f601b5c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f601410c990 con 0x7f60141001e0 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f601b5c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f601410c810 con 0x7f6014100b20 2024-09-10T04:02:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6019361640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f6014199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6019361640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f6014199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42388/0 (socket says 172.21.15.46:42388) 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6019361640 1 -- 172.21.15.46:0/4109218386 learned_addr learned my addr 172.21.15.46:0/4109218386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6018b60640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6014100b20 0x7f6014199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6019361640 1 -- 172.21.15.46:0/4109218386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6014100b20 msgr2=0x7f6014199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6019361640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6014100b20 0x7f6014199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6019361640 1 -- 172.21.15.46:0/4109218386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601419a240 con 0x7f60141001e0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f6019361640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f6014199680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6010006010 tx=0x7f60100048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f600a7fc640 1 -- 172.21.15.46:0/4109218386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f601003f070 con 0x7f60141001e0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f600a7fc640 1 -- 172.21.15.46:0/4109218386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f60100044d0 con 0x7f60141001e0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f600a7fc640 1 -- 172.21.15.46:0/4109218386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6010035030 con 0x7f60141001e0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601419ec20 con 0x7f60141001e0 2024-09-10T04:02:55.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.391+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601419f1b0 con 0x7f60141001e0 2024-09-10T04:02:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.392+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60141005e0 con 0x7f60141001e0 2024-09-10T04:02:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.393+0000 7f600a7fc640 1 -- 172.21.15.46:0/4109218386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f6010042070 con 0x7f60141001e0 2024-09-10T04:02:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.393+0000 7f600a7fc640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f600006de90 0x7f6000070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.393+0000 7f600a7fc640 1 -- 172.21.15.46:0/4109218386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f60100ae810 con 0x7f60141001e0 2024-09-10T04:02:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.393+0000 7f6018b60640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f600006de90 0x7f6000070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.394+0000 7f6018b60640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f600006de90 0x7f6000070350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6004005fd0 tx=0x7f6004005e20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.398+0000 7f600a7fc640 1 -- 172.21.15.46:0/4109218386 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f601007c430 con 0x7f60141001e0 2024-09-10T04:02:55.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.572+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) -- 0x7f601419f540 con 0x7f60141001e0 2024-09-10T04:02:55.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.573+0000 7f600a7fc640 1 -- 172.21.15.46:0/4109218386 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v42)=0 v42) ==== 155+0+0 (secure 0 0 0) 0x7f60100800e0 con 0x7f60141001e0 2024-09-10T04:02:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.574+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f600006de90 msgr2=0x7f6000070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.574+0000 7f601b5c5640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f600006de90 0x7f6000070350 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6004005fd0 tx=0x7f6004005e20 comp rx=0 tx=0).stop 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 msgr2=0x7f6014199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f6014199680 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6010006010 tx=0x7f60100048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 shutdown_connections 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f600006de90 0x7f6000070350 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6014100b20 0x7f6014199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 --2- 172.21.15.46:0/4109218386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60141001e0 0x7f6014199680 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 >> 172.21.15.46:0/4109218386 conn(0x7f60140fc010 msgr2=0x7f60140fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 shutdown_connections 2024-09-10T04:02:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:55.575+0000 7f601b5c5640 1 -- 172.21.15.46:0/4109218386 wait complete. 2024-09-10T04:02:56.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-10T04:02:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2358005773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 msgr2=0x7f1d84100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2358005773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84100c60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009880 tx=0x7f1d8002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2358005773 shutdown_connections 2024-09-10T04:02:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2358005773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d84101a60 0x7f1d84101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2358005773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84100c60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2358005773 >> 172.21.15.46:0/2358005773 conn(0x7f1d840fc010 msgr2=0x7f1d840fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:56.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2358005773 shutdown_connections 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.921+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2358005773 wait complete. 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8c50d640 1 Processor -- start 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8c50d640 1 -- start start 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8c50d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8c50d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d84101a60 0x7f1d84199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8c50d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84109700 con 0x7f1d84100860 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8c50d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84109580 con 0x7f1d84101a60 2024-09-10T04:02:56.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8a2a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8a2a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42412/0 (socket says 172.21.15.46:42412) 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8a2a9640 1 -- 172.21.15.46:0/2481383487 learned_addr learned my addr 172.21.15.46:0/2481383487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d89aa8640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d84101a60 0x7f1d84199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8a2a9640 1 -- 172.21.15.46:0/2481383487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d84101a60 msgr2=0x7f1d84199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8a2a9640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d84101a60 0x7f1d84199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.922+0000 7f1d8a2a9640 1 -- 172.21.15.46:0/2481383487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8419a330 con 0x7f1d84100860 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.923+0000 7f1d8a2a9640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84199770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006010 tx=0x7f1d800048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.923+0000 7f1d7b7fe640 1 -- 172.21.15.46:0/2481383487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1d8003f070 con 0x7f1d84100860 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.923+0000 7f1d7b7fe640 1 -- 172.21.15.46:0/2481383487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d800044d0 con 0x7f1d84100860 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.923+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8419ed10 con 0x7f1d84100860 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.923+0000 7f1d7b7fe640 1 -- 172.21.15.46:0/2481383487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1d80035030 con 0x7f1d84100860 2024-09-10T04:02:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.923+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8419f260 con 0x7f1d84100860 2024-09-10T04:02:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.924+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d84100c60 con 0x7f1d84100860 2024-09-10T04:02:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.925+0000 7f1d7b7fe640 1 -- 172.21.15.46:0/2481383487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f1d80042070 con 0x7f1d84100860 2024-09-10T04:02:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.925+0000 7f1d7b7fe640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1d7006de90 0x7f1d70070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.925+0000 7f1d7b7fe640 1 -- 172.21.15.46:0/2481383487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f1d800ae770 con 0x7f1d84100860 2024-09-10T04:02:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.925+0000 7f1d89aa8640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1d7006de90 0x7f1d70070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.925+0000 7f1d89aa8640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1d7006de90 0x7f1d70070350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d8419ac20 tx=0x7f1d74008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:56.930+0000 7f1d7b7fe640 1 -- 172.21.15.46:0/2481383487 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d8007c390 con 0x7f1d84100860 2024-09-10T04:02:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.107+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) -- 0x7f1d8419fb80 con 0x7f1d84100860 2024-09-10T04:02:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.108+0000 7f1d7b7fe640 1 -- 172.21.15.46:0/2481383487 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v42)=0 v42) ==== 163+0+0 (secure 0 0 0) 0x7f1d800ae020 con 0x7f1d84100860 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1d7006de90 msgr2=0x7f1d70070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1d7006de90 0x7f1d70070350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1d8419ac20 tx=0x7f1d74008040 comp rx=0 tx=0).stop 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 msgr2=0x7f1d84199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84199770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006010 tx=0x7f1d800048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 shutdown_connections 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f1d7006de90 0x7f1d70070350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d84101a60 0x7f1d84199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 --2- 172.21.15.46:0/2481383487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d84100860 0x7f1d84199770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 >> 172.21.15.46:0/2481383487 conn(0x7f1d840fc010 msgr2=0x7f1d8406a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 shutdown_connections 2024-09-10T04:02:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:57.110+0000 7f1d8c50d640 1 -- 172.21.15.46:0/2481383487 wait complete. 2024-09-10T04:02:57.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:57 smithi070 ceph-mon[35495]: pgmap v192: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T04:02:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:57 smithi046 ceph-mon[28328]: pgmap v192: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T04:02:57.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-10T04:02:58.013 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:02:58.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 -- 172.21.15.46:0/3587980187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 msgr2=0x7f4fdc100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:58.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/3587980187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc100e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009880 tx=0x7f4fcc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 -- 172.21.15.46:0/3587980187 shutdown_connections 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/3587980187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc100e10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/3587980187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0ffc20 0x7f4fdc100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 -- 172.21.15.46:0/3587980187 >> 172.21.15.46:0/3587980187 conn(0x7f4fdc0fb770 msgr2=0x7f4fdc0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 -- 172.21.15.46:0/3587980187 shutdown_connections 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.443+0000 7f4fe3edd640 1 -- 172.21.15.46:0/3587980187 wait complete. 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe3edd640 1 Processor -- start 2024-09-10T04:02:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe3edd640 1 -- start start 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe3edd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0ffc20 0x7f4fdc199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe3edd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe3edd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc10ca00 con 0x7f4fdc100990 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe3edd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc10c880 con 0x7f4fdc0ffc20 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe1478640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe1478640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42434/0 (socket says 172.21.15.46:42434) 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.444+0000 7f4fe1478640 1 -- 172.21.15.46:0/2539803137 learned_addr learned my addr 172.21.15.46:0/2539803137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe1c79640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0ffc20 0x7f4fdc199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:58.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe1478640 1 -- 172.21.15.46:0/2539803137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0ffc20 msgr2=0x7f4fdc199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe1478640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0ffc20 0x7f4fdc199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe1478640 1 -- 172.21.15.46:0/2539803137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc19a2f0 con 0x7f4fdc100990 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe1c79640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0ffc20 0x7f4fdc199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe1478640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc199c70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006010 tx=0x7f4fcc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fd2ffd640 1 -- 172.21.15.46:0/2539803137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc03f070 con 0x7f4fdc100990 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc19ecd0 con 0x7f4fdc100990 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fd2ffd640 1 -- 172.21.15.46:0/2539803137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4fcc0044d0 con 0x7f4fdc100990 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fd2ffd640 1 -- 172.21.15.46:0/2539803137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc035030 con 0x7f4fdc100990 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.445+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc19f230 con 0x7f4fdc100990 2024-09-10T04:02:58.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.446+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc100020 con 0x7f4fdc100990 2024-09-10T04:02:58.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.447+0000 7f4fd2ffd640 1 -- 172.21.15.46:0/2539803137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f4fcc042070 con 0x7f4fdc100990 2024-09-10T04:02:58.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.447+0000 7f4fd2ffd640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fc806de30 0x7f4fc80702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:02:58.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.447+0000 7f4fd2ffd640 1 -- 172.21.15.46:0/2539803137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f4fcc0ae8b0 con 0x7f4fdc100990 2024-09-10T04:02:58.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.447+0000 7f4fe1c79640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fc806de30 0x7f4fc80702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:02:58.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.448+0000 7f4fe1c79640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fc806de30 0x7f4fc80702f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009930 tx=0x7f4fd8008040 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:02:58.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.452+0000 7f4fd2ffd640 1 -- 172.21.15.46:0/2539803137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fcc07c3b0 con 0x7f4fdc100990 2024-09-10T04:02:58.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.633+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) -- 0x7f4fdc19fc70 con 0x7f4fdc100990 2024-09-10T04:02:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.633+0000 7f4fd2ffd640 1 -- 172.21.15.46:0/2539803137 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v42)=0 v42) ==== 135+0+0 (secure 0 0 0) 0x7f4fcc080060 con 0x7f4fdc100990 2024-09-10T04:02:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fc806de30 msgr2=0x7f4fc80702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fc806de30 0x7f4fc80702f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8009930 tx=0x7f4fd8008040 comp rx=0 tx=0).stop 2024-09-10T04:02:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 msgr2=0x7f4fdc199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:02:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc199c70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc006010 tx=0x7f4fcc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 shutdown_connections 2024-09-10T04:02:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fdc100990 0x7f4fdc199c70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4fc806de30 0x7f4fc80702f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 --2- 172.21.15.46:0/2539803137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fdc0ffc20 0x7f4fdc199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:02:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.635+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 >> 172.21.15.46:0/2539803137 conn(0x7f4fdc0fb770 msgr2=0x7f4fdc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:02:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.636+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 shutdown_connections 2024-09-10T04:02:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:02:58.636+0000 7f4fe3edd640 1 -- 172.21.15.46:0/2539803137 wait complete. 2024-09-10T04:02:59.236 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:02:59 smithi046 ceph-mon[28328]: pgmap v193: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T04:02:59.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-09-10T04:02:59.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:02:59 smithi070 ceph-mon[35495]: pgmap v193: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T04:02:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:03:00.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 -- 172.21.15.46:0/353658642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 msgr2=0x7f0bdc102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:03:00.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 --2- 172.21.15.46:0/353658642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc102fe0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009880 tx=0x7f0bcc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:03:00.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 -- 172.21.15.46:0/353658642 shutdown_connections 2024-09-10T04:03:00.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 --2- 172.21.15.46:0/353658642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc102fe0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 --2- 172.21.15.46:0/353658642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc101960 0x7f0bdc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 -- 172.21.15.46:0/353658642 >> 172.21.15.46:0/353658642 conn(0x7f0bdc0fd0f0 msgr2=0x7f0bdc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 -- 172.21.15.46:0/353658642 shutdown_connections 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 -- 172.21.15.46:0/353658642 wait complete. 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 Processor -- start 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.010+0000 7f0be4057640 1 -- start start 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be4057640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc101960 0x7f0bdc197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be4057640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be4057640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10a780 con 0x7f0bdc102b60 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be4057640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc10a600 con 0x7f0bdc101960 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be15f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:03:00.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be15f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52792/0 (socket says 172.21.15.46:52792) 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be15f2640 1 -- 172.21.15.46:0/767955165 learned_addr learned my addr 172.21.15.46:0/767955165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be1df3640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc101960 0x7f0bdc197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be15f2640 1 -- 172.21.15.46:0/767955165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc101960 msgr2=0x7f0bdc197490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be15f2640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc101960 0x7f0bdc197490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be15f2640 1 -- 172.21.15.46:0/767955165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc197fc0 con 0x7f0bdc102b60 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be1df3640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc101960 0x7f0bdc197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0be15f2640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc1979d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc006010 tx=0x7f0bcc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.011+0000 7f0bd2ffd640 1 -- 172.21.15.46:0/767955165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0bcc03f070 con 0x7f0bdc102b60 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.012+0000 7f0bd2ffd640 1 -- 172.21.15.46:0/767955165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bcc0044d0 con 0x7f0bdc102b60 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.012+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc19c9d0 con 0x7f0bdc102b60 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.012+0000 7f0bd2ffd640 1 -- 172.21.15.46:0/767955165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0bcc035030 con 0x7f0bdc102b60 2024-09-10T04:03:00.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.012+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc19ced0 con 0x7f0bdc102b60 2024-09-10T04:03:00.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.013+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bdc101d60 con 0x7f0bdc102b60 2024-09-10T04:03:00.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.013+0000 7f0bd2ffd640 1 -- 172.21.15.46:0/767955165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f0bcc042070 con 0x7f0bdc102b60 2024-09-10T04:03:00.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.014+0000 7f0bd2ffd640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0bc806de90 0x7f0bc8070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:03:00.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.014+0000 7f0bd2ffd640 1 -- 172.21.15.46:0/767955165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f0bcc0ae8b0 con 0x7f0bdc102b60 2024-09-10T04:03:00.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.014+0000 7f0be1df3640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0bc806de90 0x7f0bc8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:03:00.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.015+0000 7f0be1df3640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0bc806de90 0x7f0bc8070350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc1029c0 tx=0x7f0bd800a5e0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:03:00.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.019+0000 7f0bd2ffd640 1 -- 172.21.15.46:0/767955165 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0bcc07c430 con 0x7f0bdc102b60 2024-09-10T04:03:00.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:03:00.213+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) -- 0x7f0bdc02c6f0 con 0x7f0bc806de90 2024-09-10T04:03:01.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:01 smithi070 ceph-mon[35495]: pgmap v194: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T04:03:01.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:01 smithi070 ceph-mon[35495]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:03:01.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:01 smithi046 ceph-mon[28328]: pgmap v194: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T04:03:01.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:01 smithi046 ceph-mon[28328]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:03:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:03 smithi070 ceph-mon[35495]: pgmap v195: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T04:03:03.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:03 smithi046 ceph-mon[28328]: pgmap v195: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T04:03:04.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:04.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:05.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:05 smithi070 ceph-mon[35495]: pgmap v196: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-10T04:03:05.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:05 smithi046 ceph-mon[28328]: pgmap v196: 65 pgs: 65 active+clean; 587 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-10T04:03:07.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:07 smithi070 ceph-mon[35495]: pgmap v197: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T04:03:07.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:07 smithi046 ceph-mon[28328]: pgmap v197: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T04:03:09.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:09 smithi070 ceph-mon[35495]: pgmap v198: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T04:03:09.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:09 smithi046 ceph-mon[28328]: pgmap v198: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T04:03:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:11 smithi070 ceph-mon[35495]: pgmap v199: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T04:03:11.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:11 smithi046 ceph-mon[28328]: pgmap v199: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T04:03:13.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:13 smithi070 ceph-mon[35495]: pgmap v200: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T04:03:13.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:13 smithi046 ceph-mon[28328]: pgmap v200: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T04:03:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:15 smithi046 ceph-mon[28328]: pgmap v201: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T04:03:15.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:15 smithi070 ceph-mon[35495]: pgmap v201: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T04:03:17.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:17 smithi070 ceph-mon[35495]: pgmap v202: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-10T04:03:17.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:17 smithi046 ceph-mon[28328]: pgmap v202: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-09-10T04:03:19.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:19 smithi070 ceph-mon[35495]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:19.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:19 smithi046 ceph-mon[28328]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:19.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:21.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:21 smithi070 ceph-mon[35495]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:03:21.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:21 smithi046 ceph-mon[28328]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:03:23.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:23 smithi070 ceph-mon[35495]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-10T04:03:23.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:23 smithi046 ceph-mon[28328]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-10T04:03:25.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:25 smithi070 ceph-mon[35495]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:03:25.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:25 smithi046 ceph-mon[28328]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:03:27.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:27 smithi070 ceph-mon[35495]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:03:27.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:27 smithi046 ceph-mon[28328]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:03:29.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:29 smithi070 ceph-mon[35495]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:03:29.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:29 smithi046 ceph-mon[28328]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:03:31.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:31 smithi070 ceph-mon[35495]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:03:31.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:31 smithi046 ceph-mon[28328]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:03:33.078 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:33 smithi046 ceph-mon[28328]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:03:33.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:33 smithi070 ceph-mon[35495]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:03:34.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:34.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:35.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:35 smithi070 ceph-mon[35495]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:35.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:35 smithi046 ceph-mon[28328]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:37.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:37 smithi070 ceph-mon[35495]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:03:37.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:37 smithi046 ceph-mon[28328]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:03:39.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:39 smithi070 ceph-mon[35495]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:39 smithi046 ceph-mon[28328]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:41 smithi070 ceph-mon[35495]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:03:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:41 smithi046 ceph-mon[28328]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:03:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:43 smithi070 ceph-mon[35495]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:03:43.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:43 smithi046 ceph-mon[28328]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:03:45.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:45 smithi070 ceph-mon[35495]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:45 smithi046 ceph-mon[28328]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:46 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:03:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:46 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:03:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:47 smithi070 ceph-mon[35495]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:03:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:03:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:03:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:47 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:03:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:47 smithi046 ceph-mon[28328]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:03:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:03:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:03:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:47 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:03:49.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:49 smithi070 ceph-mon[35495]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:49.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:49 smithi046 ceph-mon[28328]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:03:51.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:51 smithi070 ceph-mon[35495]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:03:51.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:51 smithi046 ceph-mon[28328]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:03:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:53 smithi070 ceph-mon[35495]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:03:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:53 smithi046 ceph-mon[28328]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:03:55.454 INFO:tasks.workunit.client.1.smithi070.stderr:Note: switching to '3a5e3f16720f432aea965c40a8f6eb480095ec6c'. 2024-09-10T04:03:55.454 INFO:tasks.workunit.client.1.smithi070.stderr: 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr: 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr: 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr: git switch -c 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr: 2024-09-10T04:03:55.455 INFO:tasks.workunit.client.1.smithi070.stderr:Or undo this operation with: 2024-09-10T04:03:55.456 INFO:tasks.workunit.client.1.smithi070.stderr: 2024-09-10T04:03:55.456 INFO:tasks.workunit.client.1.smithi070.stderr: git switch - 2024-09-10T04:03:55.456 INFO:tasks.workunit.client.1.smithi070.stderr: 2024-09-10T04:03:55.456 INFO:tasks.workunit.client.1.smithi070.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T04:03:55.456 INFO:tasks.workunit.client.1.smithi070.stderr: 2024-09-10T04:03:55.456 INFO:tasks.workunit.client.1.smithi070.stderr:HEAD is now at 3a5e3f16720 Merge branch 'cephadm-enable-module-bootstrap-retry' of https://github.com/adk3798/ceph into wip-adk-testing-2024-09-09-1904 2024-09-10T04:03:55.459 DEBUG:teuthology.orchestra.run.smithi070:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-10T04:03:55.523 INFO:tasks.workunit.client.1.smithi070.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T04:03:55.525 INFO:tasks.workunit.client.1.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-10T04:03:55.525 INFO:tasks.workunit.client.1.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T04:03:55.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:55 smithi070 ceph-mon[35495]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:55.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:55 smithi046 ceph-mon[28328]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:03:56.221 INFO:tasks.workunit.client.1.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T04:03:56.294 INFO:tasks.workunit.client.1.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T04:03:56.333 INFO:tasks.workunit.client.1.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-10T04:03:56.335 INFO:tasks.workunit.client.1.smithi070.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-10T04:03:56.335 INFO:tasks.workunit.client.1.smithi070.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T04:03:56.373 INFO:tasks.workunit.client.1.smithi070.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-10T04:03:56.376 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-10T04:03:56.376 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-10T04:03:56.433 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-09-10T04:03:56.436 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-10T04:03:56.436 DEBUG:teuthology.orchestra.run.smithi070:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3a5e3f16720f432aea965c40a8f6eb480095ec6c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-09-10T04:03:56.500 INFO:tasks.workunit.client.1.smithi070.stderr:+ mkdir -p fsstress 2024-09-10T04:03:56.503 INFO:tasks.workunit.client.1.smithi070.stderr:+ pushd fsstress 2024-09-10T04:03:56.505 INFO:tasks.workunit.client.1.smithi070.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-10T04:03:56.505 INFO:tasks.workunit.client.1.smithi070.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-10T04:03:57.077 INFO:tasks.workunit.client.1.smithi070.stderr:+ tar xzf ltp-full.tgz 2024-09-10T04:03:57.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:57 smithi070 ceph-mon[35495]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:03:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:57 smithi046 ceph-mon[28328]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 191 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:03:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:03:59 smithi070 ceph-mon[35495]: pgmap v223: 65 pgs: 65 active+clean; 6.9 MiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 541 KiB/s wr, 4 op/s 2024-09-10T04:03:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:03:59 smithi046 ceph-mon[28328]: pgmap v223: 65 pgs: 65 active+clean; 6.9 MiB data, 195 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 541 KiB/s wr, 4 op/s 2024-09-10T04:04:01.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:01 smithi070 ceph-mon[35495]: pgmap v224: 65 pgs: 65 active+clean; 11 MiB data, 207 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 919 KiB/s wr, 11 op/s 2024-09-10T04:04:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:01 smithi046 ceph-mon[28328]: pgmap v224: 65 pgs: 65 active+clean; 11 MiB data, 207 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 919 KiB/s wr, 11 op/s 2024-09-10T04:04:03.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:03 smithi070 ceph-mon[35495]: pgmap v225: 65 pgs: 65 active+clean; 14 MiB data, 222 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2024-09-10T04:04:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:03 smithi046 ceph-mon[28328]: pgmap v225: 65 pgs: 65 active+clean; 14 MiB data, 222 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2024-09-10T04:04:04.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:04.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:05.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:05 smithi070 ceph-mon[35495]: pgmap v226: 65 pgs: 65 active+clean; 15 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-09-10T04:04:05.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:05 smithi046 ceph-mon[28328]: pgmap v226: 65 pgs: 65 active+clean; 15 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 136 KiB/s rd, 1.2 MiB/s wr, 39 op/s 2024-09-10T04:04:07.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:07 smithi070 ceph-mon[35495]: pgmap v227: 65 pgs: 65 active+clean; 18 MiB data, 254 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-10T04:04:07.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:07 smithi046 ceph-mon[28328]: pgmap v227: 65 pgs: 65 active+clean; 18 MiB data, 254 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s rd, 1.5 MiB/s wr, 57 op/s 2024-09-10T04:04:09.510 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 79% (10107/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T04:04:09.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:09 smithi070 ceph-mon[35495]: pgmap v228: 65 pgs: 65 active+clean; 23 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-09-10T04:04:09.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:09 smithi046 ceph-mon[28328]: pgmap v228: 65 pgs: 65 active+clean; 23 MiB data, 263 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-09-10T04:04:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.176+0000 7f0bd2ffd640 1 -- 172.21.15.46:0/767955165 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+89 (secure 0 0 0) 0x7f0bdc02c6f0 con 0x7f0bc806de90 2024-09-10T04:04:10.177 INFO:teuthology.orchestra.run.smithi046.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:04:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0bc806de90 msgr2=0x7f0bc8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0bc806de90 0x7f0bc8070350 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc1029c0 tx=0x7f0bd800a5e0 comp rx=0 tx=0).stop 2024-09-10T04:04:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 msgr2=0x7f0bdc1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc1979d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc006010 tx=0x7f0bcc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:04:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 shutdown_connections 2024-09-10T04:04:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bdc102b60 0x7f0bdc1979d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f0bc806de90 0x7f0bc8070350 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 --2- 172.21.15.46:0/767955165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bdc101960 0x7f0bdc197490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 >> 172.21.15.46:0/767955165 conn(0x7f0bdc0fd0f0 msgr2=0x7f0bdc106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 shutdown_connections 2024-09-10T04:04:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:10.179+0000 7f0be4057640 1 -- 172.21.15.46:0/767955165 wait complete. 2024-09-10T04:04:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to '3a5e3f16720f432aea965c40a8f6eb480095ec6c'. 2024-09-10T04:04:10.381 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-10T04:04:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T04:04:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T04:04:10.381 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T04:04:10.382 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-09-10T04:04:10.383 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at 3a5e3f16720 Merge branch 'cephadm-enable-module-bootstrap-retry' of https://github.com/adk3798/ceph into wip-adk-testing-2024-09-09-1904 2024-09-10T04:04:10.386 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T04:04:10.450 INFO:tasks.workunit.client.0.smithi046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T04:04:10.453 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T04:04:10.453 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T04:04:11.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-09-10T04:04:11.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: pgmap v229: 65 pgs: 65 active+clean; 25 MiB data, 269 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-09-10T04:04:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:04:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:04:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:04:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:04:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:04:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:04:11.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:11 smithi046 ceph-mon[28328]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:04:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: pgmap v229: 65 pgs: 65 active+clean; 25 MiB data, 269 MiB used, 536 GiB / 536 GiB avail; 684 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:04:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:11 smithi070 ceph-mon[35495]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:04:12.041 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T04:04:12.169 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T04:04:12.211 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T04:04:12.213 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T04:04:12.213 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T04:04:12.267 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T04:04:12.271 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T04:04:12.271 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T04:04:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 -- 172.21.15.46:0/3915235237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 msgr2=0x7f6410104840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3915235237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 0x7f6410104840 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f640c009880 tx=0x7f640c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 -- 172.21.15.46:0/3915235237 shutdown_connections 2024-09-10T04:04:12.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3915235237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6410104d80 0x7f6410107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3915235237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 0x7f6410104840 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 -- 172.21.15.46:0/3915235237 >> 172.21.15.46:0/3915235237 conn(0x7f64100fc010 msgr2=0x7f64100fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 -- 172.21.15.46:0/3915235237 shutdown_connections 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.218+0000 7f6417dbb640 1 -- 172.21.15.46:0/3915235237 wait complete. 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6417dbb640 1 Processor -- start 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6417dbb640 1 -- start start 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6417dbb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 0x7f6410197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6417dbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6410104d80 0x7f6410197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6417dbb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f641010a750 con 0x7f6410102450 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6417dbb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f641010a5d0 con 0x7f6410104d80 2024-09-10T04:04:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6415b57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 0x7f6410197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6415356640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6410104d80 0x7f6410197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.219+0000 7f6415b57640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 0x7f6410197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54868/0 (socket says 172.21.15.46:54868) 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6415b57640 1 -- 172.21.15.46:0/3126556476 learned_addr learned my addr 172.21.15.46:0/3126556476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6415356640 1 -- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 msgr2=0x7f6410197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6415356640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 0x7f6410197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6415356640 1 -- 172.21.15.46:0/3126556476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6410198010 con 0x7f6410104d80 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6415356640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6410104d80 0x7f6410197990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f640000e940 tx=0x7f640000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6406ffd640 1 -- 172.21.15.46:0/3126556476 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f640000cac0 con 0x7f6410104d80 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6406ffd640 1 -- 172.21.15.46:0/3126556476 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f640000cc60 con 0x7f6410104d80 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.220+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641019ca50 con 0x7f6410104d80 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.221+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641019cfe0 con 0x7f6410104d80 2024-09-10T04:04:12.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.221+0000 7f6406ffd640 1 -- 172.21.15.46:0/3126556476 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f64000186e0 con 0x7f6410104d80 2024-09-10T04:04:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.222+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6410102830 con 0x7f6410104d80 2024-09-10T04:04:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.224+0000 7f6406ffd640 1 -- 172.21.15.46:0/3126556476 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f6400010430 con 0x7f6410104d80 2024-09-10T04:04:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.224+0000 7f6406ffd640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f63fc06de30 0x7f63fc0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.224+0000 7f6406ffd640 1 -- 172.21.15.46:0/3126556476 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f640008e8f0 con 0x7f6410104d80 2024-09-10T04:04:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.224+0000 7f6415b57640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f63fc06de30 0x7f63fc0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.225+0000 7f6415b57640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f63fc06de30 0x7f63fc0702f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f640c0096c0 tx=0x7f640c005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:12.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.228+0000 7f6406ffd640 1 -- 172.21.15.46:0/3126556476 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6400026070 con 0x7f6410104d80 2024-09-10T04:04:12.289 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-10T04:04:12.290 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-10T04:04:12.290 DEBUG:teuthology.orchestra.run.smithi046:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3a5e3f16720f432aea965c40a8f6eb480095ec6c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-10T04:04:12.361 INFO:tasks.workunit.client.0.smithi046.stderr:+ mkdir -p fsstress 2024-09-10T04:04:12.366 INFO:tasks.workunit.client.0.smithi046.stderr:+ pushd fsstress 2024-09-10T04:04:12.367 INFO:tasks.workunit.client.0.smithi046.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-10T04:04:12.368 INFO:tasks.workunit.client.0.smithi046.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-10T04:04:12.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.475+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6410105640 con 0x7f63fc06de30 2024-09-10T04:04:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.476+0000 7f6406ffd640 1 -- 172.21.15.46:0/3126556476 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f6410105640 con 0x7f63fc06de30 2024-09-10T04:04:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.479+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f63fc06de30 msgr2=0x7f63fc0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.479+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f63fc06de30 0x7f63fc0702f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f640c0096c0 tx=0x7f640c005a20 comp rx=0 tx=0).stop 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.479+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6410104d80 msgr2=0x7f6410197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.479+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6410104d80 0x7f6410197990 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f640000e940 tx=0x7f640000ee00 comp rx=0 tx=0).stop 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.480+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 shutdown_connections 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.480+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f63fc06de30 0x7f63fc0702f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.480+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6410104d80 0x7f6410197990 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.480+0000 7f6417dbb640 1 --2- 172.21.15.46:0/3126556476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6410102450 0x7f6410197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.480+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 >> 172.21.15.46:0/3126556476 conn(0x7f64100fc010 msgr2=0x7f64100fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.480+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 shutdown_connections 2024-09-10T04:04:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.480+0000 7f6417dbb640 1 -- 172.21.15.46:0/3126556476 wait complete. 2024-09-10T04:04:12.492 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 -- 172.21.15.46:0/2133633128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc100860 msgr2=0x7f7cfc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 --2- 172.21.15.46:0/2133633128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc100860 0x7f7cfc100c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009880 tx=0x7f7cf802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 -- 172.21.15.46:0/2133633128 shutdown_connections 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 --2- 172.21.15.46:0/2133633128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc101a60 0x7f7cfc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 --2- 172.21.15.46:0/2133633128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc100860 0x7f7cfc100c60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 -- 172.21.15.46:0/2133633128 >> 172.21.15.46:0/2133633128 conn(0x7f7cfc0fc010 msgr2=0x7f7cfc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 -- 172.21.15.46:0/2133633128 shutdown_connections 2024-09-10T04:04:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.614+0000 7f7d04099640 1 -- 172.21.15.46:0/2133633128 wait complete. 2024-09-10T04:04:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.615+0000 7f7d04099640 1 Processor -- start 2024-09-10T04:04:12.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.615+0000 7f7d04099640 1 -- start start 2024-09-10T04:04:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d04099640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc100860 0x7f7cfc06a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d04099640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc101a60 0x7f7cfc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d01e35640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc100860 0x7f7cfc06a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d01634640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc101a60 0x7f7cfc19e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d01634640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc101a60 0x7f7cfc19e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54874/0 (socket says 172.21.15.46:54874) 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d01e35640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc100860 0x7f7cfc06a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:53616/0 (socket says 172.21.15.46:53616) 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d01634640 1 -- 172.21.15.46:0/3472966232 learned_addr learned my addr 172.21.15.46:0/3472966232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc109680 con 0x7f7cfc101a60 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.616+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc109500 con 0x7f7cfc100860 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.617+0000 7f7d01634640 1 -- 172.21.15.46:0/3472966232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc100860 msgr2=0x7f7cfc06a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.617+0000 7f7d01634640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc100860 0x7f7cfc06a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.617+0000 7f7d01634640 1 -- 172.21.15.46:0/3472966232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc19eb40 con 0x7f7cfc101a60 2024-09-10T04:04:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.617+0000 7f7d01634640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc101a60 0x7f7cfc19e600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00ece0 tx=0x7f7cec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.617+0000 7f7cf2ffd640 1 -- 172.21.15.46:0/3472966232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cec00ee90 con 0x7f7cfc101a60 2024-09-10T04:04:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.617+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc19ed10 con 0x7f7cfc101a60 2024-09-10T04:04:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.617+0000 7f7cf2ffd640 1 -- 172.21.15.46:0/3472966232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cec00cdb0 con 0x7f7cfc101a60 2024-09-10T04:04:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.618+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc19f240 con 0x7f7cfc101a60 2024-09-10T04:04:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.618+0000 7f7cf2ffd640 1 -- 172.21.15.46:0/3472966232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cec01c070 con 0x7f7cfc101a60 2024-09-10T04:04:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.619+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc100c60 con 0x7f7cfc101a60 2024-09-10T04:04:12.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.620+0000 7f7cf2ffd640 1 -- 172.21.15.46:0/3472966232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f7cec004020 con 0x7f7cfc101a60 2024-09-10T04:04:12.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.621+0000 7f7cf2ffd640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f7ce806dfd0 0x7f7ce8070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.621+0000 7f7cf2ffd640 1 -- 172.21.15.46:0/3472966232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f7cec08daf0 con 0x7f7cfc101a60 2024-09-10T04:04:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.621+0000 7f7d01e35640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f7ce806dfd0 0x7f7ce8070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.622+0000 7f7d01e35640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f7ce806dfd0 0x7f7ce8070490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009230 tx=0x7f7cf80058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:12.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.628+0000 7f7cf2ffd640 1 -- 172.21.15.46:0/3472966232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7cec05b5a0 con 0x7f7cfc101a60 2024-09-10T04:04:12.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.965+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7cfc106010 con 0x7f7ce806dfd0 2024-09-10T04:04:12.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.967+0000 7f7cf2ffd640 1 -- 172.21.15.46:0/3472966232 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f7cfc106010 con 0x7f7ce806dfd0 2024-09-10T04:04:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f7ce806dfd0 msgr2=0x7f7ce8070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f7ce806dfd0 0x7f7ce8070490 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009230 tx=0x7f7cf80058d0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc101a60 msgr2=0x7f7cfc19e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:12.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc101a60 0x7f7cfc19e600 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00ece0 tx=0x7f7cec00c6a0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 shutdown_connections 2024-09-10T04:04:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cfc101a60 0x7f7cfc19e600 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f7ce806dfd0 0x7f7ce8070490 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 --2- 172.21.15.46:0/3472966232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cfc100860 0x7f7cfc06a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 >> 172.21.15.46:0/3472966232 conn(0x7f7cfc0fc010 msgr2=0x7f7cfc0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 shutdown_connections 2024-09-10T04:04:12.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:12.970+0000 7f7d04099640 1 -- 172.21.15.46:0/3472966232 wait complete. 2024-09-10T04:04:13.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.078+0000 7fc64b1aa640 1 -- 172.21.15.46:0/801089644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6440ff9e0 msgr2=0x7fc6440ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.078+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/801089644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6440ff9e0 0x7fc6440ffe60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc6300097e0 tx=0x7fc63002a3f0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.078+0000 7fc64b1aa640 1 -- 172.21.15.46:0/801089644 shutdown_connections 2024-09-10T04:04:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.079+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/801089644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6440ff9e0 0x7fc6440ffe60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.079+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/801089644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc644101ee0 0x7fc6441022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.079+0000 7fc64b1aa640 1 -- 172.21.15.46:0/801089644 >> 172.21.15.46:0/801089644 conn(0x7fc6440fb770 msgr2=0x7fc6440fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.079+0000 7fc64b1aa640 1 -- 172.21.15.46:0/801089644 shutdown_connections 2024-09-10T04:04:13.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.079+0000 7fc64b1aa640 1 -- 172.21.15.46:0/801089644 wait complete. 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.079+0000 7fc64b1aa640 1 Processor -- start 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc64b1aa640 1 -- start start 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc64b1aa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440ff9e0 0x7fc64419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc64b1aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc644101ee0 0x7fc64419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc64b1aa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc64410ca20 con 0x7fc6440ff9e0 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc64b1aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc64410c8a0 con 0x7fc644101ee0 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc648f46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440ff9e0 0x7fc64419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc643fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc644101ee0 0x7fc64419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc648f46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440ff9e0 0x7fc64419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54880/0 (socket says 172.21.15.46:54880) 2024-09-10T04:04:13.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.080+0000 7fc648f46640 1 -- 172.21.15.46:0/3830461987 learned_addr learned my addr 172.21.15.46:0/3830461987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc643fff640 1 -- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440ff9e0 msgr2=0x7fc64419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc643fff640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440ff9e0 0x7fc64419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc643fff640 1 -- 172.21.15.46:0/3830461987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64419e690 con 0x7fc644101ee0 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc648f46640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440ff9e0 0x7fc64419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc643fff640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc644101ee0 0x7fc64419e010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc63002a910 tx=0x7fc6300043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc641ffb640 1 -- 172.21.15.46:0/3830461987 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc630040070 con 0x7fc644101ee0 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc641ffb640 1 -- 172.21.15.46:0/3830461987 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc630030c00 con 0x7fc644101ee0 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6441a3070 con 0x7fc644101ee0 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc641ffb640 1 -- 172.21.15.46:0/3830461987 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc630003c80 con 0x7fc644101ee0 2024-09-10T04:04:13.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.081+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6441a3600 con 0x7fc644101ee0 2024-09-10T04:04:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.083+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6440ffe60 con 0x7fc644101ee0 2024-09-10T04:04:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.083+0000 7fc641ffb640 1 -- 172.21.15.46:0/3830461987 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fc630039040 con 0x7fc644101ee0 2024-09-10T04:04:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.084+0000 7fc641ffb640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc63406de90 0x7fc634070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.084+0000 7fc641ffb640 1 -- 172.21.15.46:0/3830461987 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fc6300aed50 con 0x7fc644101ee0 2024-09-10T04:04:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.084+0000 7fc648f46640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc63406de90 0x7fc634070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.084+0000 7fc648f46640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc63406de90 0x7fc634070350 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6380058c0 tx=0x7fc638009260 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.088+0000 7fc641ffb640 1 -- 172.21.15.46:0/3830461987 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc63007c910 con 0x7fc644101ee0 2024-09-10T04:04:13.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:13 smithi046 ceph-mon[28328]: pgmap v230: 65 pgs: 65 active+clean; 33 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-10T04:04:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:13 smithi046 ceph-mon[28328]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:13.338 INFO:tasks.workunit.client.0.smithi046.stderr:+ tar xzf ltp-full.tgz 2024-09-10T04:04:13.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.366+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc64410a480 con 0x7fc63406de90 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.377+0000 7fc641ffb640 1 -- 172.21.15.46:0/3830461987 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7fc64410a480 con 0x7fc63406de90 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (5m) 87s ago 7m 26.6M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (7m) 87s ago 7m 8254k - 19.1.1-298-gb53b6d3e 54467abc55f8 3888d245a02a 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (6m) 91s ago 6m 5268k - 19.1.1-298-gb53b6d3e 54467abc55f8 54f37f2cb1f7 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (7m) 87s ago 7m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (6m) 91s ago 6m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:04:13.378 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (5m) 87s ago 6m 80.0M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (103s) 87s ago 103s 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (109s) 87s ago 108s 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (106s) 91s ago 105s 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (101s) 91s ago 100s 13.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:9283,8765,8443 running (8m) 87s ago 8m 498M - 19.1.1-298-gb53b6d3e 54467abc55f8 a4fa65bf88c5 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (6m) 91s ago 6m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 06c733bd8286 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (8m) 87s ago 8m 47.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (6m) 91s ago 6m 34.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (7m) 87s ago 7m 15.3M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:04:13.379 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 91s ago 6m 17.0M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:04:13.380 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (5m) 87s ago 5m 57.0M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:04:13.380 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (4m) 87s ago 4m 54.7M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:04:13.380 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (4m) 87s ago 4m 46.4M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:04:13.380 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (3m) 91s ago 3m 48.0M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:04:13.380 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (3m) 91s ago 3m 42.1M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:04:13.380 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (2m) 91s ago 2m 49.3M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:04:13.380 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (5m) 87s ago 6m 61.3M - 2.43.0 a07b618ecd1d 15eb5a503403 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc63406de90 msgr2=0x7fc634070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc63406de90 0x7fc634070350 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6380058c0 tx=0x7fc638009260 comp rx=0 tx=0).stop 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc644101ee0 msgr2=0x7fc64419e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc644101ee0 0x7fc64419e010 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc63002a910 tx=0x7fc6300043b0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 shutdown_connections 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc63406de90 0x7fc634070350 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc644101ee0 0x7fc64419e010 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 --2- 172.21.15.46:0/3830461987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6440ff9e0 0x7fc64419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.379+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 >> 172.21.15.46:0/3830461987 conn(0x7fc6440fb770 msgr2=0x7fc6440fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:13.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.380+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 shutdown_connections 2024-09-10T04:04:13.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.380+0000 7fc64b1aa640 1 -- 172.21.15.46:0/3830461987 wait complete. 2024-09-10T04:04:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.486+0000 7fc50d0a0640 1 -- 172.21.15.46:0/2948010552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc508104d20 msgr2=0x7fc508107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.486+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/2948010552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc508104d20 0x7fc508107110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0098b0 tx=0x7fc4fc02a4d0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.487+0000 7fc50d0a0640 1 -- 172.21.15.46:0/2948010552 shutdown_connections 2024-09-10T04:04:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.487+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/2948010552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc508104d20 0x7fc508107110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.487+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/2948010552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5081023f0 0x7fc5081047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.487+0000 7fc50d0a0640 1 -- 172.21.15.46:0/2948010552 >> 172.21.15.46:0/2948010552 conn(0x7fc5080fc010 msgr2=0x7fc5080fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.487+0000 7fc50d0a0640 1 -- 172.21.15.46:0/2948010552 shutdown_connections 2024-09-10T04:04:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.487+0000 7fc50d0a0640 1 -- 172.21.15.46:0/2948010552 wait complete. 2024-09-10T04:04:13.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.487+0000 7fc50d0a0640 1 Processor -- start 2024-09-10T04:04:13.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50d0a0640 1 -- start start 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50d0a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081023f0 0x7fc508197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50d0a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc508104d20 0x7fc508197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50d0a0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810a790 con 0x7fc5081023f0 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50d0a0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc50810a610 con 0x7fc508104d20 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc506d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081023f0 0x7fc508197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc506d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081023f0 0x7fc508197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54896/0 (socket says 172.21.15.46:54896) 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc506d9d640 1 -- 172.21.15.46:0/1431639766 learned_addr learned my addr 172.21.15.46:0/1431639766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50659c640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc508104d20 0x7fc508197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50659c640 1 -- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081023f0 msgr2=0x7fc508197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.488+0000 7fc50659c640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081023f0 0x7fc508197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.489+0000 7fc50659c640 1 -- 172.21.15.46:0/1431639766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081980d0 con 0x7fc508104d20 2024-09-10T04:04:13.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.489+0000 7fc50659c640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc508104d20 0x7fc508197a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009230 tx=0x7fc4fc0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.489+0000 7fc4effff640 1 -- 172.21.15.46:0/1431639766 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc4fc03f070 con 0x7fc508104d20 2024-09-10T04:04:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.489+0000 7fc4effff640 1 -- 172.21.15.46:0/1431639766 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4fc0035f0 con 0x7fc508104d20 2024-09-10T04:04:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.489+0000 7fc4effff640 1 -- 172.21.15.46:0/1431639766 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc4fc035030 con 0x7fc508104d20 2024-09-10T04:04:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.489+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50819cab0 con 0x7fc508104d20 2024-09-10T04:04:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.489+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc50819d040 con 0x7fc508104d20 2024-09-10T04:04:13.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.490+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5081027d0 con 0x7fc508104d20 2024-09-10T04:04:13.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.492+0000 7fc4effff640 1 -- 172.21.15.46:0/1431639766 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fc4fc042070 con 0x7fc508104d20 2024-09-10T04:04:13.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.492+0000 7fc4effff640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc4f006de30 0x7fc4f00702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.492+0000 7fc4effff640 1 -- 172.21.15.46:0/1431639766 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fc4fc0ae5b0 con 0x7fc508104d20 2024-09-10T04:04:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.492+0000 7fc506d9d640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc4f006de30 0x7fc4f00702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.493+0000 7fc506d9d640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc4f006de30 0x7fc4f00702f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009c90 tx=0x7fc4f8005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:13.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.497+0000 7fc4effff640 1 -- 172.21.15.46:0/1431639766 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4fc07fe00 con 0x7fc508104d20 2024-09-10T04:04:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:13 smithi070 ceph-mon[35495]: pgmap v230: 65 pgs: 65 active+clean; 33 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 95 op/s 2024-09-10T04:04:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:13 smithi070 ceph-mon[35495]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:13.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.878+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc50819d9c0 con 0x7fc508104d20 2024-09-10T04:04:13.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.879+0000 7fc4effff640 1 -- 172.21.15.46:0/1431639766 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+633 (secure 0 0 0) 0x7fc4fc084630 con 0x7fc508104d20 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:04:13.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:04:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:04:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:04:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 14 2024-09-10T04:04:13.881 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:04:13.881 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc4f006de30 msgr2=0x7fc4f00702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc4f006de30 0x7fc4f00702f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009c90 tx=0x7fc4f8005f90 comp rx=0 tx=0).stop 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc508104d20 msgr2=0x7fc508197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc508104d20 0x7fc508197a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009230 tx=0x7fc4fc0045e0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 shutdown_connections 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fc4f006de30 0x7fc4f00702f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc508104d20 0x7fc508197a50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 --2- 172.21.15.46:0/1431639766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081023f0 0x7fc508197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 >> 172.21.15.46:0/1431639766 conn(0x7fc5080fc010 msgr2=0x7fc5080fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 shutdown_connections 2024-09-10T04:04:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.882+0000 7fc50d0a0640 1 -- 172.21.15.46:0/1431639766 wait complete. 2024-09-10T04:04:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 -- 172.21.15.46:0/4100148256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94101a60 msgr2=0x7ffb94101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 --2- 172.21.15.46:0/4100148256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94101a60 0x7ffb94101ee0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009880 tx=0x7ffb8802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 -- 172.21.15.46:0/4100148256 shutdown_connections 2024-09-10T04:04:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 --2- 172.21.15.46:0/4100148256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94101a60 0x7ffb94101ee0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 --2- 172.21.15.46:0/4100148256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb94100860 0x7ffb94100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 -- 172.21.15.46:0/4100148256 >> 172.21.15.46:0/4100148256 conn(0x7ffb940fc010 msgr2=0x7ffb940fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 -- 172.21.15.46:0/4100148256 shutdown_connections 2024-09-10T04:04:13.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.974+0000 7ffb98d03640 1 -- 172.21.15.46:0/4100148256 wait complete. 2024-09-10T04:04:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.975+0000 7ffb98d03640 1 Processor -- start 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.975+0000 7ffb98d03640 1 -- start start 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.975+0000 7ffb98d03640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94100860 0x7ffb94104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.975+0000 7ffb98d03640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb94101a60 0x7ffb94102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.975+0000 7ffb98d03640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94109680 con 0x7ffb94100860 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.975+0000 7ffb98d03640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94109500 con 0x7ffb94101a60 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.975+0000 7ffb9259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94100860 0x7ffb94104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.976+0000 7ffb9259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94100860 0x7ffb94104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54914/0 (socket says 172.21.15.46:54914) 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.976+0000 7ffb9259c640 1 -- 172.21.15.46:0/3436130435 learned_addr learned my addr 172.21.15.46:0/3436130435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.976+0000 7ffb91d9b640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb94101a60 0x7ffb94102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.976+0000 7ffb9259c640 1 -- 172.21.15.46:0/3436130435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb94101a60 msgr2=0x7ffb94102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.976+0000 7ffb9259c640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb94101a60 0x7ffb94102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:13.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.976+0000 7ffb9259c640 1 -- 172.21.15.46:0/3436130435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb94103380 con 0x7ffb94100860 2024-09-10T04:04:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.977+0000 7ffb9259c640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94100860 0x7ffb94104780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e940 tx=0x7ffb8400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.977+0000 7ffb837fe640 1 -- 172.21.15.46:0/3436130435 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffb8400cac0 con 0x7ffb94100860 2024-09-10T04:04:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.977+0000 7ffb837fe640 1 -- 172.21.15.46:0/3436130435 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb8400cc60 con 0x7ffb94100860 2024-09-10T04:04:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.977+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb94103670 con 0x7ffb94100860 2024-09-10T04:04:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.977+0000 7ffb837fe640 1 -- 172.21.15.46:0/3436130435 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffb84003de0 con 0x7ffb94100860 2024-09-10T04:04:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.977+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb94103bd0 con 0x7ffb94100860 2024-09-10T04:04:13.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.978+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb94100c60 con 0x7ffb94100860 2024-09-10T04:04:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.979+0000 7ffb837fe640 1 -- 172.21.15.46:0/3436130435 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7ffb84003f80 con 0x7ffb94100860 2024-09-10T04:04:13.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.980+0000 7ffb837fe640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ffb7c06dee0 0x7ffb7c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.980+0000 7ffb837fe640 1 -- 172.21.15.46:0/3436130435 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7ffb8408e1d0 con 0x7ffb94100860 2024-09-10T04:04:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.980+0000 7ffb91d9b640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ffb7c06dee0 0x7ffb7c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:13.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.981+0000 7ffb91d9b640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ffb7c06dee0 0x7ffb7c0703a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffb88000c00 tx=0x7ffb880058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:13.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:13.987+0000 7ffb837fe640 1 -- 172.21.15.46:0/3436130435 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb8405bd50 con 0x7ffb94100860 2024-09-10T04:04:14.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.278+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ffb9406a400 con 0x7ffb7c06dee0 2024-09-10T04:04:14.280 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:04:14.280 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", 2024-09-10T04:04:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:04:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:04:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [], 2024-09-10T04:04:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "", 2024-09-10T04:04:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c image", 2024-09-10T04:04:14.281 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:04:14.282 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:04:14.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.280+0000 7ffb837fe640 1 -- 172.21.15.46:0/3436130435 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7ffb9406a400 con 0x7ffb7c06dee0 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ffb7c06dee0 msgr2=0x7ffb7c0703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ffb7c06dee0 0x7ffb7c0703a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffb88000c00 tx=0x7ffb880058d0 comp rx=0 tx=0).stop 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94100860 msgr2=0x7ffb94104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94100860 0x7ffb94104780 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e940 tx=0x7ffb8400ee00 comp rx=0 tx=0).stop 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 shutdown_connections 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7ffb7c06dee0 0x7ffb7c0703a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb94101a60 0x7ffb94102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 --2- 172.21.15.46:0/3436130435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb94100860 0x7ffb94104780 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 >> 172.21.15.46:0/3436130435 conn(0x7ffb940fc010 msgr2=0x7ffb94069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:14.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 shutdown_connections 2024-09-10T04:04:14.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:14.283+0000 7ffb98d03640 1 -- 172.21.15.46:0/3436130435 wait complete. 2024-09-10T04:04:14.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:14 smithi070 ceph-mon[35495]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:14.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:14 smithi070 ceph-mon[35495]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:14.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:14 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1431639766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:04:14.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:14 smithi046 ceph-mon[28328]: from='client.14636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:14.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:14 smithi046 ceph-mon[28328]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:14.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:14 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1431639766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:04:15.526 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:15 smithi046 ceph-mon[28328]: pgmap v231: 65 pgs: 65 active+clean; 36 MiB data, 312 MiB used, 536 GiB / 536 GiB avail; 892 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-10T04:04:15.527 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:15 smithi046 ceph-mon[28328]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:15.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:15 smithi070 ceph-mon[35495]: pgmap v231: 65 pgs: 65 active+clean; 36 MiB data, 312 MiB used, 536 GiB / 536 GiB avail; 892 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-10T04:04:15.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:15 smithi070 ceph-mon[35495]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:04:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:16 smithi070 ceph-mon[35495]: Upgrade: Target is version 19.3.0-4885-g3a5e3f16 (squid) 2024-09-10T04:04:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:16 smithi070 ceph-mon[35495]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca'] 2024-09-10T04:04:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:16 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:04:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:16 smithi070 ceph-mon[35495]: Upgrade: Need to upgrade myself (mgr.smithi046.zvckjg) 2024-09-10T04:04:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:04:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:16 smithi046 ceph-mon[28328]: Upgrade: Target is version 19.3.0-4885-g3a5e3f16 (squid) 2024-09-10T04:04:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:16 smithi046 ceph-mon[28328]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca'] 2024-09-10T04:04:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:16 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:04:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:16 smithi046 ceph-mon[28328]: Upgrade: Need to upgrade myself (mgr.smithi046.zvckjg) 2024-09-10T04:04:17.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:17 smithi070 ceph-mon[35495]: pgmap v232: 65 pgs: 65 active+clean; 58 MiB data, 367 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 128 op/s 2024-09-10T04:04:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:17 smithi070 ceph-mon[35495]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca on smithi070 2024-09-10T04:04:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:17 smithi046 ceph-mon[28328]: pgmap v232: 65 pgs: 65 active+clean; 58 MiB data, 367 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 128 op/s 2024-09-10T04:04:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:17 smithi046 ceph-mon[28328]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca on smithi070 2024-09-10T04:04:19.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:19 smithi070 ceph-mon[35495]: pgmap v233: 65 pgs: 65 active+clean; 62 MiB data, 395 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 135 op/s 2024-09-10T04:04:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:19 smithi046 ceph-mon[28328]: pgmap v233: 65 pgs: 65 active+clean; 62 MiB data, 395 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 135 op/s 2024-09-10T04:04:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:21.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:21 smithi070 ceph-mon[35495]: pgmap v234: 65 pgs: 65 active+clean; 63 MiB data, 407 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 128 op/s 2024-09-10T04:04:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:21 smithi046 ceph-mon[28328]: pgmap v234: 65 pgs: 65 active+clean; 63 MiB data, 407 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 128 op/s 2024-09-10T04:04:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:23 smithi070 ceph-mon[35495]: pgmap v235: 65 pgs: 65 active+clean; 73 MiB data, 429 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 160 op/s 2024-09-10T04:04:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:23 smithi046 ceph-mon[28328]: pgmap v235: 65 pgs: 65 active+clean; 73 MiB data, 429 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 160 op/s 2024-09-10T04:04:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:25 smithi070 ceph-mon[35495]: pgmap v236: 65 pgs: 65 active+clean; 74 MiB data, 420 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 157 op/s 2024-09-10T04:04:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:25 smithi046 ceph-mon[28328]: pgmap v236: 65 pgs: 65 active+clean; 74 MiB data, 420 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 157 op/s 2024-09-10T04:04:27.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:27 smithi070 ceph-mon[35495]: pgmap v237: 65 pgs: 65 active+clean; 85 MiB data, 436 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 178 op/s 2024-09-10T04:04:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:27 smithi046 ceph-mon[28328]: pgmap v237: 65 pgs: 65 active+clean; 85 MiB data, 436 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.2 MiB/s wr, 178 op/s 2024-09-10T04:04:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:29 smithi070 ceph-mon[35495]: pgmap v238: 65 pgs: 65 active+clean; 86 MiB data, 461 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 153 op/s 2024-09-10T04:04:29.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:29 smithi046 ceph-mon[28328]: pgmap v238: 65 pgs: 65 active+clean; 86 MiB data, 461 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 153 op/s 2024-09-10T04:04:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:31 smithi046 ceph-mon[28328]: pgmap v239: 65 pgs: 65 active+clean; 94 MiB data, 455 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 155 op/s 2024-09-10T04:04:32.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:31 smithi070 ceph-mon[35495]: pgmap v239: 65 pgs: 65 active+clean; 94 MiB data, 455 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 155 op/s 2024-09-10T04:04:34.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:33 smithi070 ceph-mon[35495]: pgmap v240: 65 pgs: 65 active+clean; 109 MiB data, 508 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-09-10T04:04:34.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:33 smithi046 ceph-mon[28328]: pgmap v240: 65 pgs: 65 active+clean; 109 MiB data, 508 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 193 op/s 2024-09-10T04:04:35.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:36.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:35 smithi070 ceph-mon[35495]: pgmap v241: 65 pgs: 65 active+clean; 113 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 190 op/s 2024-09-10T04:04:36.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:35 smithi046 ceph-mon[28328]: pgmap v241: 65 pgs: 65 active+clean; 113 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 190 op/s 2024-09-10T04:04:38.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:37 smithi070 ceph-mon[35495]: pgmap v242: 65 pgs: 65 active+clean; 125 MiB data, 592 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 234 op/s 2024-09-10T04:04:38.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:37 smithi046 ceph-mon[28328]: pgmap v242: 65 pgs: 65 active+clean; 125 MiB data, 592 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 234 op/s 2024-09-10T04:04:40.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:39 smithi070 ceph-mon[35495]: pgmap v243: 65 pgs: 65 active+clean; 130 MiB data, 616 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 227 op/s 2024-09-10T04:04:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:39 smithi046 ceph-mon[28328]: pgmap v243: 65 pgs: 65 active+clean; 130 MiB data, 616 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 227 op/s 2024-09-10T04:04:42.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:41 smithi070 ceph-mon[35495]: pgmap v244: 65 pgs: 65 active+clean; 139 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 240 op/s 2024-09-10T04:04:42.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:41 smithi046 ceph-mon[28328]: pgmap v244: 65 pgs: 65 active+clean; 139 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 240 op/s 2024-09-10T04:04:44.004 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:43 smithi070 ceph-mon[35495]: pgmap v245: 65 pgs: 65 active+clean; 145 MiB data, 656 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 278 op/s 2024-09-10T04:04:44.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:43 smithi046 ceph-mon[28328]: pgmap v245: 65 pgs: 65 active+clean; 145 MiB data, 656 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 278 op/s 2024-09-10T04:04:44.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 -- 172.21.15.46:0/2596327289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc1024b0 msgr2=0x7f94fc1048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 --2- 172.21.15.46:0/2596327289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc1024b0 0x7f94fc1048a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009880 tx=0x7f94f802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 -- 172.21.15.46:0/2596327289 shutdown_connections 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 --2- 172.21.15.46:0/2596327289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94fc104de0 0x7f94fc1071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 --2- 172.21.15.46:0/2596327289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc1024b0 0x7f94fc1048a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 -- 172.21.15.46:0/2596327289 >> 172.21.15.46:0/2596327289 conn(0x7f94fc0fc010 msgr2=0x7f94fc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 -- 172.21.15.46:0/2596327289 shutdown_connections 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.398+0000 7f9503576640 1 -- 172.21.15.46:0/2596327289 wait complete. 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9503576640 1 Processor -- start 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9503576640 1 -- start start 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9503576640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94fc1024b0 0x7f94fc101da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:44.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9503576640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc104de0 0x7f94fc100460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:44.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9503576640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc10a7b0 con 0x7f94fc104de0 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9503576640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc10a630 con 0x7f94fc1024b0 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9500b11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc104de0 0x7f94fc100460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9501312640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94fc1024b0 0x7f94fc101da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9500b11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc104de0 0x7f94fc100460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49572/0 (socket says 172.21.15.46:49572) 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9501312640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94fc1024b0 0x7f94fc101da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:42658/0 (socket says 172.21.15.46:42658) 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.399+0000 7f9500b11640 1 -- 172.21.15.46:0/2994713881 learned_addr learned my addr 172.21.15.46:0/2994713881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f9500b11640 1 -- 172.21.15.46:0/2994713881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94fc1024b0 msgr2=0x7f94fc101da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f9500b11640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94fc1024b0 0x7f94fc101da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f9500b11640 1 -- 172.21.15.46:0/2994713881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc1009a0 con 0x7f94fc104de0 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f9500b11640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc104de0 0x7f94fc100460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e940 tx=0x7f94ec00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f94f27fc640 1 -- 172.21.15.46:0/2994713881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f94ec00cac0 con 0x7f94fc104de0 2024-09-10T04:04:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc100c90 con 0x7f94fc104de0 2024-09-10T04:04:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc101220 con 0x7f94fc104de0 2024-09-10T04:04:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.400+0000 7f94f27fc640 1 -- 172.21.15.46:0/2994713881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94ec0054e0 con 0x7f94fc104de0 2024-09-10T04:04:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.401+0000 7f94f27fc640 1 -- 172.21.15.46:0/2994713881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f94ec004e40 con 0x7f94fc104de0 2024-09-10T04:04:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.402+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc102890 con 0x7f94fc104de0 2024-09-10T04:04:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.402+0000 7f94f27fc640 1 -- 172.21.15.46:0/2994713881 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f94ec004fe0 con 0x7f94fc104de0 2024-09-10T04:04:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.403+0000 7f94f27fc640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f94e806de90 0x7f94e8070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.403+0000 7f94f27fc640 1 -- 172.21.15.46:0/2994713881 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f94ec08e030 con 0x7f94fc104de0 2024-09-10T04:04:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.403+0000 7f9501312640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f94e806de90 0x7f94e8070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.404+0000 7f9501312640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f94e806de90 0x7f94e8070350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94f8000c00 tx=0x7f94f80058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.408+0000 7f94f27fc640 1 -- 172.21.15.46:0/2994713881 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94ec05bc50 con 0x7f94fc104de0 2024-09-10T04:04:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.684+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f94fc19f5c0 con 0x7f94e806de90 2024-09-10T04:04:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.685+0000 7f94f27fc640 1 -- 172.21.15.46:0/2994713881 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+439 (secure 0 0 0) 0x7f94fc19f5c0 con 0x7f94e806de90 2024-09-10T04:04:44.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f94e806de90 msgr2=0x7f94e8070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f94e806de90 0x7f94e8070350 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94f8000c00 tx=0x7f94f80058d0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc104de0 msgr2=0x7f94fc100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc104de0 0x7f94fc100460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00e940 tx=0x7f94ec00ee00 comp rx=0 tx=0).stop 2024-09-10T04:04:44.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 shutdown_connections 2024-09-10T04:04:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94fc104de0 0x7f94fc100460 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f94e806de90 0x7f94e8070350 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 --2- 172.21.15.46:0/2994713881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94fc1024b0 0x7f94fc101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 >> 172.21.15.46:0/2994713881 conn(0x7f94fc0fc010 msgr2=0x7f94fc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:44.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 shutdown_connections 2024-09-10T04:04:44.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.687+0000 7f9503576640 1 -- 172.21.15.46:0/2994713881 wait complete. 2024-09-10T04:04:44.698 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:04:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 -- 172.21.15.46:0/350162179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac102410 msgr2=0x7f4eac104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/350162179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac102410 0x7f4eac104800 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0098b0 tx=0x7f4e9c02a4d0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 -- 172.21.15.46:0/350162179 shutdown_connections 2024-09-10T04:04:44.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/350162179 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac104d40 0x7f4eac107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/350162179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac102410 0x7f4eac104800 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 -- 172.21.15.46:0/350162179 >> 172.21.15.46:0/350162179 conn(0x7f4eac0fc010 msgr2=0x7f4eac0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 -- 172.21.15.46:0/350162179 shutdown_connections 2024-09-10T04:04:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.792+0000 7f4eb0afe640 1 -- 172.21.15.46:0/350162179 wait complete. 2024-09-10T04:04:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.793+0000 7f4eb0afe640 1 Processor -- start 2024-09-10T04:04:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.793+0000 7f4eb0afe640 1 -- start start 2024-09-10T04:04:44.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.793+0000 7f4eb0afe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac102410 0x7f4eac197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.793+0000 7f4eb0afe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac104d40 0x7f4eac197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.793+0000 7f4eb0afe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10a7e0 con 0x7f4eac104d40 2024-09-10T04:04:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.793+0000 7f4eb0afe640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac10a660 con 0x7f4eac102410 2024-09-10T04:04:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.793+0000 7f4eaa59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac102410 0x7f4eac197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.794+0000 7f4eaa59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac102410 0x7f4eac197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:42678/0 (socket says 172.21.15.46:42678) 2024-09-10T04:04:44.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.794+0000 7f4eaa59c640 1 -- 172.21.15.46:0/1133348833 learned_addr learned my addr 172.21.15.46:0/1133348833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.794+0000 7f4ea9d9b640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac104d40 0x7f4eac197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.794+0000 7f4eaa59c640 1 -- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac104d40 msgr2=0x7f4eac197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.794+0000 7f4eaa59c640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac104d40 0x7f4eac197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:44.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.794+0000 7f4eaa59c640 1 -- 172.21.15.46:0/1133348833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac198140 con 0x7f4eac102410 2024-09-10T04:04:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.794+0000 7f4eaa59c640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac102410 0x7f4eac197580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0099d0 tx=0x7f4e9c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.795+0000 7f4e9b7fe640 1 -- 172.21.15.46:0/1133348833 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e9c03f070 con 0x7f4eac102410 2024-09-10T04:04:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.795+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac19cb20 con 0x7f4eac102410 2024-09-10T04:04:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.795+0000 7f4e9b7fe640 1 -- 172.21.15.46:0/1133348833 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e9c004a40 con 0x7f4eac102410 2024-09-10T04:04:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.795+0000 7f4e9b7fe640 1 -- 172.21.15.46:0/1133348833 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e9c035030 con 0x7f4eac102410 2024-09-10T04:04:44.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.795+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac19d0b0 con 0x7f4eac102410 2024-09-10T04:04:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.796+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eac1027f0 con 0x7f4eac102410 2024-09-10T04:04:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.797+0000 7f4e9b7fe640 1 -- 172.21.15.46:0/1133348833 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f4e9c042070 con 0x7f4eac102410 2024-09-10T04:04:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.798+0000 7f4e9b7fe640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4e9406de90 0x7f4e94070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.798+0000 7f4e9b7fe640 1 -- 172.21.15.46:0/1133348833 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f4e9c0ae5c0 con 0x7f4eac102410 2024-09-10T04:04:44.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.798+0000 7f4ea9d9b640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4e9406de90 0x7f4e94070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:44.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.799+0000 7f4ea9d9b640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4e9406de90 0x7f4e94070350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4eac198a30 tx=0x7f4ea000b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:44.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:44.803+0000 7f4e9b7fe640 1 -- 172.21.15.46:0/1133348833 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e9c07fdf0 con 0x7f4eac102410 2024-09-10T04:04:45.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.096+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4eac105740 con 0x7f4e9406de90 2024-09-10T04:04:45.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.097+0000 7f4e9b7fe640 1 -- 172.21.15.46:0/1133348833 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+439 (secure 0 0 0) 0x7f4eac105740 con 0x7f4e9406de90 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4e9406de90 msgr2=0x7f4e94070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4e9406de90 0x7f4e94070350 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4eac198a30 tx=0x7f4ea000b410 comp rx=0 tx=0).stop 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac102410 msgr2=0x7f4eac197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac102410 0x7f4eac197580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0099d0 tx=0x7f4e9c0046c0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 shutdown_connections 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4eac104d40 0x7f4eac197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f4e9406de90 0x7f4e94070350 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 --2- 172.21.15.46:0/1133348833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4eac102410 0x7f4eac197580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 >> 172.21.15.46:0/1133348833 conn(0x7f4eac0fc010 msgr2=0x7f4eac0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 shutdown_connections 2024-09-10T04:04:45.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.100+0000 7f4eb0afe640 1 -- 172.21.15.46:0/1133348833 wait complete. 2024-09-10T04:04:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 -- 172.21.15.46:0/2886860043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d81023f0 msgr2=0x7f72d81047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 --2- 172.21.15.46:0/2886860043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d81023f0 0x7f72d81047e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f72cc0098b0 tx=0x7f72cc02a4d0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 -- 172.21.15.46:0/2886860043 shutdown_connections 2024-09-10T04:04:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 --2- 172.21.15.46:0/2886860043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72d8104d20 0x7f72d8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 --2- 172.21.15.46:0/2886860043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d81023f0 0x7f72d81047e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 -- 172.21.15.46:0/2886860043 >> 172.21.15.46:0/2886860043 conn(0x7f72d80fc010 msgr2=0x7f72d80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:45.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 -- 172.21.15.46:0/2886860043 shutdown_connections 2024-09-10T04:04:45.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.221+0000 7f72dd065640 1 -- 172.21.15.46:0/2886860043 wait complete. 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.222+0000 7f72dd065640 1 Processor -- start 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.222+0000 7f72dd065640 1 -- start start 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72dd065640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72d81023f0 0x7f72d8197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72dd065640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d8104d20 0x7f72d8197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72dd065640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72d810a7e0 con 0x7f72d8104d20 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72dd065640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f72d810a660 con 0x7f72d81023f0 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d8104d20 0x7f72d8197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d8104d20 0x7f72d8197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49626/0 (socket says 172.21.15.46:49626) 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72d81023f0 0x7f72d8197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:45.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d659c640 1 -- 172.21.15.46:0/787974589 learned_addr learned my addr 172.21.15.46:0/787974589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d659c640 1 -- 172.21.15.46:0/787974589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72d81023f0 msgr2=0x7f72d8197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d659c640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72d81023f0 0x7f72d8197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d659c640 1 -- 172.21.15.46:0/787974589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d81980c0 con 0x7f72d8104d20 2024-09-10T04:04:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.223+0000 7f72d659c640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d8104d20 0x7f72d8197a40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d6c0 tx=0x7f72c800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:45.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.224+0000 7f72bffff640 1 -- 172.21.15.46:0/787974589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f72c8017070 con 0x7f72d8104d20 2024-09-10T04:04:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.224+0000 7f72bffff640 1 -- 172.21.15.46:0/787974589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f72c80054e0 con 0x7f72d8104d20 2024-09-10T04:04:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.224+0000 7f72bffff640 1 -- 172.21.15.46:0/787974589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f72c8004b70 con 0x7f72d8104d20 2024-09-10T04:04:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.224+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d81001e0 con 0x7f72d8104d20 2024-09-10T04:04:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.224+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d8100710 con 0x7f72d8104d20 2024-09-10T04:04:45.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.226+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d81027d0 con 0x7f72d8104d20 2024-09-10T04:04:45.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.227+0000 7f72bffff640 1 -- 172.21.15.46:0/787974589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f72c8009820 con 0x7f72d8104d20 2024-09-10T04:04:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.228+0000 7f72bffff640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f72c00722e0 0x7f72c00747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.228+0000 7f72bffff640 1 -- 172.21.15.46:0/787974589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f72c808d5f0 con 0x7f72d8104d20 2024-09-10T04:04:45.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.228+0000 7f72d6d9d640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f72c00722e0 0x7f72c00747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:45.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.229+0000 7f72d6d9d640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f72c00722e0 0x7f72c00747a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f72cc006010 tx=0x7f72cc0058d0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:45.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.235+0000 7f72bffff640 1 -- 172.21.15.46:0/787974589 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72c805eea0 con 0x7f72d8104d20 2024-09-10T04:04:45.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.569+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f72d8105680 con 0x7f72c00722e0 2024-09-10T04:04:45.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.581+0000 7f72bffff640 1 -- 172.21.15.46:0/787974589 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3768 (secure 0 0 0) 0x7f72d8105680 con 0x7f72c00722e0 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (6m) 119s ago 7m 26.6M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (7m) 119s ago 7m 8254k - 19.1.1-298-gb53b6d3e 54467abc55f8 3888d245a02a 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (7m) 2m ago 7m 5268k - 19.1.1-298-gb53b6d3e 54467abc55f8 54f37f2cb1f7 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (7m) 119s ago 7m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (6m) 2m ago 6m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (6m) 119s ago 7m 80.0M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (2m) 119s ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (2m) 119s ago 2m 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (2m) 2m ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:04:45.583 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (2m) 2m ago 2m 13.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:9283,8765,8443 running (8m) 119s ago 8m 498M - 19.1.1-298-gb53b6d3e 54467abc55f8 a4fa65bf88c5 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (6m) 2m ago 6m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 06c733bd8286 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (8m) 119s ago 9m 47.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (6m) 2m ago 6m 34.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (7m) 119s ago 7m 15.3M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 2m ago 6m 17.0M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (5m) 119s ago 5m 57.0M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (5m) 119s ago 5m 54.7M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (4m) 119s ago 4m 46.4M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (4m) 2m ago 4m 48.0M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (3m) 2m ago 3m 42.1M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (3m) 2m ago 3m 49.3M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:04:45.584 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (5m) 119s ago 7m 61.3M - 2.43.0 a07b618ecd1d 15eb5a503403 2024-09-10T04:04:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f72c00722e0 msgr2=0x7f72c00747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f72c00722e0 0x7f72c00747a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f72cc006010 tx=0x7f72cc0058d0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d8104d20 msgr2=0x7f72d8197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d8104d20 0x7f72d8197a40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72c800d6c0 tx=0x7f72c800db80 comp rx=0 tx=0).stop 2024-09-10T04:04:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 shutdown_connections 2024-09-10T04:04:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72d8104d20 0x7f72d8197a40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f72c00722e0 0x7f72c00747a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 --2- 172.21.15.46:0/787974589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72d81023f0 0x7f72d8197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 >> 172.21.15.46:0/787974589 conn(0x7f72d80fc010 msgr2=0x7f72d80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 shutdown_connections 2024-09-10T04:04:45.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.584+0000 7f72dd065640 1 -- 172.21.15.46:0/787974589 wait complete. 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 -- 172.21.15.46:0/2032474391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100ba0 msgr2=0x7f513c101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 --2- 172.21.15.46:0/2032474391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100ba0 0x7f513c101000 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51300097e0 tx=0x7f513002a3f0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 -- 172.21.15.46:0/2032474391 shutdown_connections 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 --2- 172.21.15.46:0/2032474391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100ba0 0x7f513c101000 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f51300097e0 tx=0x7f513002a3f0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 --2- 172.21.15.46:0/2032474391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c100260 0x7f513c100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 -- 172.21.15.46:0/2032474391 >> 172.21.15.46:0/2032474391 conn(0x7f513c0fc010 msgr2=0x7f513c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 -- 172.21.15.46:0/2032474391 shutdown_connections 2024-09-10T04:04:45.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.702+0000 7f51415e8640 1 -- 172.21.15.46:0/2032474391 wait complete. 2024-09-10T04:04:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.703+0000 7f51415e8640 1 Processor -- start 2024-09-10T04:04:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.703+0000 7f51415e8640 1 -- start start 2024-09-10T04:04:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f51415e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100260 0x7f513c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:45.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f51415e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c199df0 0x7f513c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:45.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f51415e8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f513c10c980 con 0x7f513c199df0 2024-09-10T04:04:45.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f51415e8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f513c10c800 con 0x7f513c100260 2024-09-10T04:04:45.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100260 0x7f513c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:45.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100260 0x7f513c1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:42708/0 (socket says 172.21.15.46:42708) 2024-09-10T04:04:45.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513affd640 1 -- 172.21.15.46:0/1092329853 learned_addr learned my addr 172.21.15.46:0/1092329853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:45.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513a7fc640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c199df0 0x7f513c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513affd640 1 -- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c199df0 msgr2=0x7f513c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513affd640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c199df0 0x7f513c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513affd640 1 -- 172.21.15.46:0/1092329853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c19a320 con 0x7f513c100260 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513a7fc640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c199df0 0x7f513c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.704+0000 7f513affd640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100260 0x7f513c1998b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f513400ef30 tx=0x7f513400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.705+0000 7f5123fff640 1 -- 172.21.15.46:0/1092329853 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f513401b070 con 0x7f513c100260 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.705+0000 7f5123fff640 1 -- 172.21.15.46:0/1092329853 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f513400ba30 con 0x7f513c100260 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.705+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513c19f330 con 0x7f513c100260 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.705+0000 7f5123fff640 1 -- 172.21.15.46:0/1092329853 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5134016b90 con 0x7f513c100260 2024-09-10T04:04:45.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.705+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513c19f830 con 0x7f513c100260 2024-09-10T04:04:45.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.706+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513c101000 con 0x7f513c100260 2024-09-10T04:04:45.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.707+0000 7f5123fff640 1 -- 172.21.15.46:0/1092329853 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f513401e070 con 0x7f513c100260 2024-09-10T04:04:45.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.707+0000 7f5123fff640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f512406ded0 0x7f5124070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:45.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.708+0000 7f5123fff640 1 -- 172.21.15.46:0/1092329853 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f5134013070 con 0x7f513c100260 2024-09-10T04:04:45.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.708+0000 7f513a7fc640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f512406ded0 0x7f5124070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:45.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.708+0000 7f513a7fc640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f512406ded0 0x7f5124070390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51300097b0 tx=0x7f51300308b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:45.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:45.712+0000 7f5123fff640 1 -- 172.21.15.46:0/1092329853 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51340606e0 con 0x7f513c100260 2024-09-10T04:04:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:45 smithi046 ceph-mon[28328]: pgmap v246: 65 pgs: 65 active+clean; 150 MiB data, 672 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-09-10T04:04:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:45 smithi046 ceph-mon[28328]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:46.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:45 smithi070 ceph-mon[35495]: pgmap v246: 65 pgs: 65 active+clean; 150 MiB data, 672 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-09-10T04:04:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:45 smithi070 ceph-mon[35495]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.090+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f513c062df0 con 0x7f513c100260 2024-09-10T04:04:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.091+0000 7f5123fff640 1 -- 172.21.15.46:0/1092329853 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+633 (secure 0 0 0) 0x7f513405a480 con 0x7f513c100260 2024-09-10T04:04:46.092 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:04:46.093 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:04:46.094 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 14 2024-09-10T04:04:46.094 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:04:46.094 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:04:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f512406ded0 msgr2=0x7f5124070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f512406ded0 0x7f5124070390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51300097b0 tx=0x7f51300308b0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100260 msgr2=0x7f513c1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100260 0x7f513c1998b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f513400ef30 tx=0x7f513400c800 comp rx=0 tx=0).stop 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 shutdown_connections 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f513c199df0 0x7f513c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f512406ded0 0x7f5124070390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 --2- 172.21.15.46:0/1092329853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f513c100260 0x7f513c1998b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.093+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 >> 172.21.15.46:0/1092329853 conn(0x7f513c0fc010 msgr2=0x7f513c108c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.094+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 shutdown_connections 2024-09-10T04:04:46.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.094+0000 7f51415e8640 1 -- 172.21.15.46:0/1092329853 wait complete. 2024-09-10T04:04:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 -- 172.21.15.46:0/4208581000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 msgr2=0x7f39b4100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/4208581000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 0x7f39b4100640 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009880 tx=0x7f39a402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 -- 172.21.15.46:0/4208581000 shutdown_connections 2024-09-10T04:04:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/4208581000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b4100b80 0x7f39b4100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/4208581000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 0x7f39b4100640 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 -- 172.21.15.46:0/4208581000 >> 172.21.15.46:0/4208581000 conn(0x7f39b40fc010 msgr2=0x7f39b40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 -- 172.21.15.46:0/4208581000 shutdown_connections 2024-09-10T04:04:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.205+0000 7f39b8fe1640 1 -- 172.21.15.46:0/4208581000 wait complete. 2024-09-10T04:04:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.206+0000 7f39b8fe1640 1 Processor -- start 2024-09-10T04:04:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.206+0000 7f39b8fe1640 1 -- start start 2024-09-10T04:04:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.206+0000 7f39b8fe1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 0x7f39b41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.206+0000 7f39b8fe1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b4100b80 0x7f39b4199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.206+0000 7f39b8fe1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39b410c990 con 0x7f39b4100240 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.206+0000 7f39b8fe1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f39b410c810 con 0x7f39b4100b80 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 0x7f39b41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 0x7f39b41996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49674/0 (socket says 172.21.15.46:49674) 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b259c640 1 -- 172.21.15.46:0/953595605 learned_addr learned my addr 172.21.15.46:0/953595605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b1d9b640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b4100b80 0x7f39b4199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b1d9b640 1 -- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 msgr2=0x7f39b41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b1d9b640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 0x7f39b41996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b1d9b640 1 -- 172.21.15.46:0/953595605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b419a290 con 0x7f39b4100b80 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39b1d9b640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b4100b80 0x7f39b4199c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e970 tx=0x7f39a800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.207+0000 7f39a37fe640 1 -- 172.21.15.46:0/953595605 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f39a800cac0 con 0x7f39b4100b80 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.208+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b419ecd0 con 0x7f39b4100b80 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.208+0000 7f39a37fe640 1 -- 172.21.15.46:0/953595605 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f39a800cc60 con 0x7f39b4100b80 2024-09-10T04:04:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.208+0000 7f39a37fe640 1 -- 172.21.15.46:0/953595605 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f39a80047a0 con 0x7f39b4100b80 2024-09-10T04:04:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.208+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b419f200 con 0x7f39b4100b80 2024-09-10T04:04:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.209+0000 7f39a37fe640 1 -- 172.21.15.46:0/953595605 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f39a8004030 con 0x7f39b4100b80 2024-09-10T04:04:46.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.209+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b4100640 con 0x7f39b4100b80 2024-09-10T04:04:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.210+0000 7f39a37fe640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f399c06de30 0x7f399c0702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:04:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.210+0000 7f39b259c640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f399c06de30 0x7f399c0702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:04:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.210+0000 7f39a37fe640 1 -- 172.21.15.46:0/953595605 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f39a808e0b0 con 0x7f39b4100b80 2024-09-10T04:04:46.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.211+0000 7f39b259c640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f399c06de30 0x7f399c0702f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39a40096c0 tx=0x7f39a4005a20 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:04:46.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.215+0000 7f39a37fe640 1 -- 172.21.15.46:0/953595605 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39a805bbb0 con 0x7f39b4100b80 2024-09-10T04:04:46.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.530+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f39b4062630 con 0x7f399c06de30 2024-09-10T04:04:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.531+0000 7f39a37fe640 1 -- 172.21.15.46:0/953595605 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+439 (secure 0 0 0) 0x7f39b4062630 con 0x7f399c06de30 2024-09-10T04:04:46.533 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:04:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:04:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:04:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:04:46.533 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [], 2024-09-10T04:04:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "0/2 daemons upgraded", 2024-09-10T04:04:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca image on host smithi070", 2024-09-10T04:04:46.534 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:04:46.534 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:04:46.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f399c06de30 msgr2=0x7f399c0702f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f399c06de30 0x7f399c0702f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f39a40096c0 tx=0x7f39a4005a20 comp rx=0 tx=0).stop 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b4100b80 msgr2=0x7f39b4199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b4100b80 0x7f39b4199c10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e970 tx=0x7f39a800ee30 comp rx=0 tx=0).stop 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 shutdown_connections 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f399c06de30 0x7f399c0702f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39b4100b80 0x7f39b4199c10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 --2- 172.21.15.46:0/953595605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39b4100240 0x7f39b41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 >> 172.21.15.46:0/953595605 conn(0x7f39b40fc010 msgr2=0x7f39b4108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 shutdown_connections 2024-09-10T04:04:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:04:46.535+0000 7f39b8fe1640 1 -- 172.21.15.46:0/953595605 wait complete. 2024-09-10T04:04:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:46 smithi046 ceph-mon[28328]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:46 smithi046 ceph-mon[28328]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:46 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1092329853' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:04:47.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:46 smithi070 ceph-mon[35495]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:46 smithi070 ceph-mon[35495]: from='client.14660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:46 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1092329853' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:04:47.786 INFO:tasks.workunit.client.1.smithi070.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-10T04:04:47.796 INFO:tasks.workunit.client.1.smithi070.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-10T04:04:47.796 INFO:tasks.workunit.client.1.smithi070.stderr:+ make 2024-09-10T04:04:47.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:47 smithi046 ceph-mon[28328]: pgmap v247: 65 pgs: 65 active+clean; 161 MiB data, 701 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-09-10T04:04:47.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:47 smithi046 ceph-mon[28328]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:48.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:47 smithi070 ceph-mon[35495]: pgmap v247: 65 pgs: 65 active+clean; 161 MiB data, 701 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-09-10T04:04:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:47 smithi070 ceph-mon[35495]: from='client.24415 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:04:48.838 INFO:tasks.workunit.client.1.smithi070.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-10T04:04:49.332 INFO:tasks.workunit.client.1.smithi070.stderr:++ readlink -f fsstress 2024-09-10T04:04:49.334 INFO:tasks.workunit.client.1.smithi070.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-10T04:04:49.334 INFO:tasks.workunit.client.1.smithi070.stderr:+ popd 2024-09-10T04:04:49.335 INFO:tasks.workunit.client.1.smithi070.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-10T04:04:49.336 INFO:tasks.workunit.client.1.smithi070.stderr:+ popd 2024-09-10T04:04:49.336 INFO:tasks.workunit.client.1.smithi070.stdout:~/cephtest/mnt.1/client.1/tmp 2024-09-10T04:04:49.337 INFO:tasks.workunit.client.1.smithi070.stderr:++ mktemp -d -p . 2024-09-10T04:04:49.340 INFO:tasks.workunit.client.1.smithi070.stderr:+ T=./tmp.2QsmQqCNVi 2024-09-10T04:04:49.340 INFO:tasks.workunit.client.1.smithi070.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.2QsmQqCNVi -l 1 -n 1000 -p 10 -v 2024-09-10T04:04:49.344 INFO:tasks.workunit.client.1.smithi070.stdout:seed = 1725596812 2024-09-10T04:04:49.348 INFO:tasks.workunit.client.1.smithi070.stdout:0/0: fdatasync - no filename 2024-09-10T04:04:49.348 INFO:tasks.workunit.client.1.smithi070.stdout:0/1: chown . 55145368 1 2024-09-10T04:04:49.349 INFO:tasks.workunit.client.1.smithi070.stdout:1/0: stat - no entries 2024-09-10T04:04:49.350 INFO:tasks.workunit.client.1.smithi070.stdout:0/2: creat f0 x:0 0 0 2024-09-10T04:04:49.351 INFO:tasks.workunit.client.1.smithi070.stdout:0/3: write f0 [551770,121314] 0 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/0: readlink - no filename 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/1: dread - no filename 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:1/1: getdents . 0 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/2: chown . 729717108 1 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/3: dwrite - no filename 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/4: dread - no filename 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/5: write - no filename 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/6: readlink - no filename 2024-09-10T04:04:49.352 INFO:tasks.workunit.client.1.smithi070.stdout:2/7: truncate - no filename 2024-09-10T04:04:49.354 INFO:tasks.workunit.client.1.smithi070.stdout:0/4: creat f1 x:0 0 0 2024-09-10T04:04:49.354 INFO:tasks.workunit.client.1.smithi070.stdout:0/5: read - f1 zero size 2024-09-10T04:04:49.355 INFO:tasks.workunit.client.1.smithi070.stdout:5/0: readlink - no filename 2024-09-10T04:04:49.355 INFO:tasks.workunit.client.1.smithi070.stdout:5/1: dread - no filename 2024-09-10T04:04:49.355 INFO:tasks.workunit.client.1.smithi070.stdout:5/2: write - no filename 2024-09-10T04:04:49.356 INFO:tasks.workunit.client.1.smithi070.stdout:1/2: creat f0 x:0 0 0 2024-09-10T04:04:49.356 INFO:tasks.workunit.client.1.smithi070.stdout:2/8: getdents . 0 2024-09-10T04:04:49.356 INFO:tasks.workunit.client.1.smithi070.stdout:2/9: fdatasync - no filename 2024-09-10T04:04:49.356 INFO:tasks.workunit.client.1.smithi070.stdout:2/10: write - no filename 2024-09-10T04:04:49.357 INFO:tasks.workunit.client.1.smithi070.stdout:4/0: creat f0 x:0 0 0 2024-09-10T04:04:49.358 INFO:tasks.workunit.client.1.smithi070.stdout:4/1: stat f0 0 2024-09-10T04:04:49.359 INFO:tasks.workunit.client.1.smithi070.stdout:1/3: write f0 [804134,32037] 0 2024-09-10T04:04:49.359 INFO:tasks.workunit.client.1.smithi070.stdout:6/0: unlink - no file 2024-09-10T04:04:49.360 INFO:tasks.workunit.client.1.smithi070.stdout:6/1: chown . 451 1 2024-09-10T04:04:49.360 INFO:tasks.workunit.client.1.smithi070.stdout:5/3: creat f0 x:0 0 0 2024-09-10T04:04:49.360 INFO:tasks.workunit.client.1.smithi070.stdout:5/4: rmdir - no directory 2024-09-10T04:04:49.361 INFO:tasks.workunit.client.1.smithi070.stdout:2/11: symlink l0 0 2024-09-10T04:04:49.362 INFO:tasks.workunit.client.1.smithi070.stdout:4/2: dread - f0 zero size 2024-09-10T04:04:49.364 INFO:tasks.workunit.client.1.smithi070.stdout:2/12: mknod c1 0 2024-09-10T04:04:49.364 INFO:tasks.workunit.client.1.smithi070.stdout:2/13: dwrite - no filename 2024-09-10T04:04:49.364 INFO:tasks.workunit.client.1.smithi070.stdout:1/4: creat f1 x:0 0 0 2024-09-10T04:04:49.364 INFO:tasks.workunit.client.1.smithi070.stdout:6/2: mkdir d0 0 2024-09-10T04:04:49.365 INFO:tasks.workunit.client.1.smithi070.stdout:6/3: rename d0 to d0/d1 22 2024-09-10T04:04:49.365 INFO:tasks.workunit.client.1.smithi070.stdout:1/5: write f1 [315250,8164] 0 2024-09-10T04:04:49.367 INFO:tasks.workunit.client.1.smithi070.stdout:5/5: symlink l1 0 2024-09-10T04:04:49.368 INFO:tasks.workunit.client.1.smithi070.stdout:4/3: creat f1 x:0 0 0 2024-09-10T04:04:49.369 INFO:tasks.workunit.client.1.smithi070.stdout:0/6: dwrite f0 [0,4194304] 0 2024-09-10T04:04:49.369 INFO:tasks.workunit.client.1.smithi070.stdout:3/0: dread - no filename 2024-09-10T04:04:49.369 INFO:tasks.workunit.client.1.smithi070.stdout:3/1: dread - no filename 2024-09-10T04:04:49.369 INFO:tasks.workunit.client.1.smithi070.stdout:2/14: unlink l0 0 2024-09-10T04:04:49.370 INFO:tasks.workunit.client.1.smithi070.stdout:5/6: write f0 [991843,2721] 0 2024-09-10T04:04:49.370 INFO:tasks.workunit.client.1.smithi070.stdout:1/6: symlink l2 0 2024-09-10T04:04:49.370 INFO:tasks.workunit.client.1.smithi070.stdout:4/4: dread - f0 zero size 2024-09-10T04:04:49.371 INFO:tasks.workunit.client.1.smithi070.stdout:2/15: chown c1 7588500 1 2024-09-10T04:04:49.371 INFO:tasks.workunit.client.1.smithi070.stdout:2/16: dread - no filename 2024-09-10T04:04:49.372 INFO:tasks.workunit.client.1.smithi070.stdout:4/5: dread - f0 zero size 2024-09-10T04:04:49.373 INFO:tasks.workunit.client.1.smithi070.stdout:2/17: chown c1 613 1 2024-09-10T04:04:49.373 INFO:tasks.workunit.client.1.smithi070.stdout:8/0: fdatasync - no filename 2024-09-10T04:04:49.373 INFO:tasks.workunit.client.1.smithi070.stdout:8/1: rename - no filename 2024-09-10T04:04:49.375 INFO:tasks.workunit.client.1.smithi070.stdout:7/0: creat f0 x:0 0 0 2024-09-10T04:04:49.375 INFO:tasks.workunit.client.1.smithi070.stdout:6/4: symlink d0/l2 0 2024-09-10T04:04:49.375 INFO:tasks.workunit.client.1.smithi070.stdout:6/5: dread - no filename 2024-09-10T04:04:49.375 INFO:tasks.workunit.client.1.smithi070.stdout:1/7: mkdir d3 0 2024-09-10T04:04:49.376 INFO:tasks.workunit.client.1.smithi070.stdout:1/8: chown d3 0 1 2024-09-10T04:04:49.376 INFO:tasks.workunit.client.1.smithi070.stdout:6/6: stat d0 0 2024-09-10T04:04:49.376 INFO:tasks.workunit.client.1.smithi070.stdout:6/7: dwrite - no filename 2024-09-10T04:04:49.376 INFO:tasks.workunit.client.1.smithi070.stdout:6/8: dread - no filename 2024-09-10T04:04:49.376 INFO:tasks.workunit.client.1.smithi070.stdout:6/9: truncate - no filename 2024-09-10T04:04:49.377 INFO:tasks.workunit.client.1.smithi070.stdout:6/10: read - no filename 2024-09-10T04:04:49.378 INFO:tasks.workunit.client.1.smithi070.stdout:6/11: chown d0/l2 2162 1 2024-09-10T04:04:49.378 INFO:tasks.workunit.client.1.smithi070.stdout:7/1: write f0 [833615,31468] 0 2024-09-10T04:04:49.379 INFO:tasks.workunit.client.1.smithi070.stdout:4/6: mknod c2 0 2024-09-10T04:04:49.380 INFO:tasks.workunit.client.1.smithi070.stdout:2/18: symlink l2 0 2024-09-10T04:04:49.380 INFO:tasks.workunit.client.1.smithi070.stdout:2/19: truncate - no filename 2024-09-10T04:04:49.380 INFO:tasks.workunit.client.1.smithi070.stdout:9/0: dwrite - no filename 2024-09-10T04:04:49.380 INFO:tasks.workunit.client.1.smithi070.stdout:5/7: dread f0 [0,4194304] 0 2024-09-10T04:04:49.381 INFO:tasks.workunit.client.1.smithi070.stdout:3/2: mknod c0 0 2024-09-10T04:04:49.382 INFO:tasks.workunit.client.1.smithi070.stdout:6/12: chown d0 11 1 2024-09-10T04:04:49.382 INFO:tasks.workunit.client.1.smithi070.stdout:4/7: write f1 [795340,18431] 0 2024-09-10T04:04:49.382 INFO:tasks.workunit.client.1.smithi070.stdout:2/20: chown l2 182487013 1 2024-09-10T04:04:49.382 INFO:tasks.workunit.client.1.smithi070.stdout:6/13: stat d0 0 2024-09-10T04:04:49.383 INFO:tasks.workunit.client.1.smithi070.stdout:6/14: dread - no filename 2024-09-10T04:04:49.383 INFO:tasks.workunit.client.1.smithi070.stdout:6/15: dread - no filename 2024-09-10T04:04:49.383 INFO:tasks.workunit.client.1.smithi070.stdout:6/16: write - no filename 2024-09-10T04:04:49.383 INFO:tasks.workunit.client.1.smithi070.stdout:6/17: dwrite - no filename 2024-09-10T04:04:49.384 INFO:tasks.workunit.client.1.smithi070.stdout:8/2: creat f0 x:0 0 0 2024-09-10T04:04:49.385 INFO:tasks.workunit.client.1.smithi070.stdout:5/8: fdatasync f0 0 2024-09-10T04:04:49.385 INFO:tasks.workunit.client.1.smithi070.stdout:8/3: dread - f0 zero size 2024-09-10T04:04:49.386 INFO:tasks.workunit.client.1.smithi070.stdout:7/2: creat f1 x:0 0 0 2024-09-10T04:04:49.387 INFO:tasks.workunit.client.1.smithi070.stdout:4/8: symlink l3 0 2024-09-10T04:04:49.388 INFO:tasks.workunit.client.1.smithi070.stdout:2/21: unlink l2 0 2024-09-10T04:04:49.388 INFO:tasks.workunit.client.1.smithi070.stdout:2/22: write - no filename 2024-09-10T04:04:49.388 INFO:tasks.workunit.client.1.smithi070.stdout:2/23: dread - no filename 2024-09-10T04:04:49.389 INFO:tasks.workunit.client.1.smithi070.stdout:6/18: mkdir d0/d3 0 2024-09-10T04:04:49.390 INFO:tasks.workunit.client.1.smithi070.stdout:9/1: symlink l0 0 2024-09-10T04:04:49.390 INFO:tasks.workunit.client.1.smithi070.stdout:9/2: fdatasync - no filename 2024-09-10T04:04:49.390 INFO:tasks.workunit.client.1.smithi070.stdout:9/3: dwrite - no filename 2024-09-10T04:04:49.391 INFO:tasks.workunit.client.1.smithi070.stdout:8/4: mknod c1 0 2024-09-10T04:04:49.391 INFO:tasks.workunit.client.1.smithi070.stdout:7/3: mkdir d2 0 2024-09-10T04:04:49.393 INFO:tasks.workunit.client.1.smithi070.stdout:4/9: creat f4 x:0 0 0 2024-09-10T04:04:49.394 INFO:tasks.workunit.client.1.smithi070.stdout:2/24: unlink c1 0 2024-09-10T04:04:49.394 INFO:tasks.workunit.client.1.smithi070.stdout:2/25: fdatasync - no filename 2024-09-10T04:04:49.394 INFO:tasks.workunit.client.1.smithi070.stdout:2/26: chown . 173293 1 2024-09-10T04:04:49.394 INFO:tasks.workunit.client.1.smithi070.stdout:6/19: symlink d0/l4 0 2024-09-10T04:04:49.396 INFO:tasks.workunit.client.1.smithi070.stdout:9/4: rename l0 to l1 0 2024-09-10T04:04:49.397 INFO:tasks.workunit.client.1.smithi070.stdout:5/9: link f0 f2 0 2024-09-10T04:04:49.397 INFO:tasks.workunit.client.1.smithi070.stdout:8/5: symlink l2 0 2024-09-10T04:04:49.398 INFO:tasks.workunit.client.1.smithi070.stdout:6/20: creat d0/f5 x:0 0 0 2024-09-10T04:04:49.399 INFO:tasks.workunit.client.1.smithi070.stdout:7/4: mkdir d2/d3 0 2024-09-10T04:04:49.400 INFO:tasks.workunit.client.1.smithi070.stdout:6/21: dread - d0/f5 zero size 2024-09-10T04:04:49.402 INFO:tasks.workunit.client.1.smithi070.stdout:7/5: chown f0 880831774 1 2024-09-10T04:04:49.402 INFO:tasks.workunit.client.1.smithi070.stdout:8/6: creat f3 x:0 0 0 2024-09-10T04:04:49.403 INFO:tasks.workunit.client.1.smithi070.stdout:6/22: mknod d0/c6 0 2024-09-10T04:04:49.404 INFO:tasks.workunit.client.1.smithi070.stdout:7/6: write f0 [1103890,42057] 0 2024-09-10T04:04:49.406 INFO:tasks.workunit.client.1.smithi070.stdout:6/23: chown d0/c6 148184150 1 2024-09-10T04:04:49.406 INFO:tasks.workunit.client.1.smithi070.stdout:7/7: write f0 [657722,16062] 0 2024-09-10T04:04:49.409 INFO:tasks.workunit.client.1.smithi070.stdout:6/24: dread - d0/f5 zero size 2024-09-10T04:04:49.413 INFO:tasks.workunit.client.1.smithi070.stdout:5/10: dwrite f0 [0,4194304] 0 2024-09-10T04:04:49.416 INFO:tasks.workunit.client.1.smithi070.stdout:6/25: mknod d0/d3/c7 0 2024-09-10T04:04:49.419 INFO:tasks.workunit.client.1.smithi070.stdout:6/26: dread - d0/f5 zero size 2024-09-10T04:04:49.420 INFO:tasks.workunit.client.1.smithi070.stdout:7/8: dwrite f1 [0,4194304] 0 2024-09-10T04:04:49.420 INFO:tasks.workunit.client.1.smithi070.stdout:6/27: write d0/f5 [1037371,11786] 0 2024-09-10T04:04:49.427 INFO:tasks.workunit.client.1.smithi070.stdout:6/28: dread d0/f5 [0,4194304] 0 2024-09-10T04:04:49.428 INFO:tasks.workunit.client.1.smithi070.stdout:5/11: fdatasync f0 0 2024-09-10T04:04:49.432 INFO:tasks.workunit.client.1.smithi070.stdout:6/29: creat d0/f8 x:0 0 0 2024-09-10T04:04:49.432 INFO:tasks.workunit.client.1.smithi070.stdout:7/9: dwrite f0 [0,4194304] 0 2024-09-10T04:04:49.434 INFO:tasks.workunit.client.1.smithi070.stdout:5/12: rename f2 to f3 0 2024-09-10T04:04:49.439 INFO:tasks.workunit.client.1.smithi070.stdout:5/13: dread f3 [0,4194304] 0 2024-09-10T04:04:49.442 INFO:tasks.workunit.client.1.smithi070.stdout:5/14: link f3 f4 0 2024-09-10T04:04:50.035 INFO:tasks.workunit.client.1.smithi070.stdout:0/7: getdents . 0 2024-09-10T04:04:50.035 INFO:tasks.workunit.client.1.smithi070.stdout:0/8: chown f0 2 1 2024-09-10T04:04:50.036 INFO:tasks.workunit.client.1.smithi070.stdout:0/9: write f1 [703822,14986] 0 2024-09-10T04:04:50.036 INFO:tasks.workunit.client.1.smithi070.stdout:0/10: readlink - no filename 2024-09-10T04:04:50.040 INFO:tasks.workunit.client.1.smithi070.stdout:0/11: dread f0 [0,4194304] 0 2024-09-10T04:04:50.046 INFO:tasks.workunit.client.1.smithi070.stdout:1/9: fsync f1 0 2024-09-10T04:04:50.064 INFO:tasks.workunit.client.1.smithi070.stdout:3/3: getdents . 0 2024-09-10T04:04:50.065 INFO:tasks.workunit.client.1.smithi070.stdout:3/4: chown c0 551792 1 2024-09-10T04:04:50.065 INFO:tasks.workunit.client.1.smithi070.stdout:3/5: dread - no filename 2024-09-10T04:04:50.065 INFO:tasks.workunit.client.1.smithi070.stdout:3/6: rmdir - no directory 2024-09-10T04:04:50.073 INFO:tasks.workunit.client.1.smithi070.stdout:4/10: getdents . 0 2024-09-10T04:04:50.074 INFO:tasks.workunit.client.1.smithi070.stdout:4/11: write f0 [264898,121697] 0 2024-09-10T04:04:50.075 INFO:tasks.workunit.client.1.smithi070.stdout:4/12: write f4 [996182,53172] 0 2024-09-10T04:04:50.075 INFO:tasks.workunit.client.1.smithi070.stdout:4/13: rmdir - no directory 2024-09-10T04:04:50.075 INFO:tasks.workunit.client.1.smithi070.stdout:4/14: chown f1 5275091 1 2024-09-10T04:04:50.075 INFO:tasks.workunit.client.1.smithi070.stdout:4/15: readlink l3 0 2024-09-10T04:04:50.076 INFO:tasks.workunit.client.1.smithi070.stdout:4/16: chown f1 2272714 1 2024-09-10T04:04:50.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:49 smithi070 ceph-mon[35495]: pgmap v248: 65 pgs: 65 active+clean; 168 MiB data, 718 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-09-10T04:04:50.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:49 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:50.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:49 smithi046 ceph-mon[28328]: pgmap v248: 65 pgs: 65 active+clean; 168 MiB data, 718 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 270 op/s 2024-09-10T04:04:50.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:49 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:04:50.093 INFO:tasks.workunit.client.1.smithi070.stdout:5/15: write f3 [5011533,20717] 0 2024-09-10T04:04:50.094 INFO:tasks.workunit.client.1.smithi070.stdout:5/16: readlink l1 0 2024-09-10T04:04:50.095 INFO:tasks.workunit.client.1.smithi070.stdout:5/17: fdatasync f3 0 2024-09-10T04:04:50.097 INFO:tasks.workunit.client.1.smithi070.stdout:5/18: write f4 [4105771,130147] 0 2024-09-10T04:04:50.097 INFO:tasks.workunit.client.1.smithi070.stdout:6/30: fsync d0/f8 0 2024-09-10T04:04:50.098 INFO:tasks.workunit.client.1.smithi070.stdout:6/31: write d0/f8 [179886,23235] 0 2024-09-10T04:04:50.101 INFO:tasks.workunit.client.1.smithi070.stdout:6/32: dread d0/f5 [0,4194304] 0 2024-09-10T04:04:50.198 INFO:tasks.workunit.client.1.smithi070.stdout:0/12: creat f2 x:0 0 0 2024-09-10T04:04:50.202 INFO:tasks.workunit.client.1.smithi070.stdout:9/5: rename l1 to l2 0 2024-09-10T04:04:50.206 INFO:tasks.workunit.client.1.smithi070.stdout:5/19: mkdir d5 0 2024-09-10T04:04:50.211 INFO:tasks.workunit.client.1.smithi070.stdout:1/10: symlink d3/l4 0 2024-09-10T04:04:50.212 INFO:tasks.workunit.client.1.smithi070.stdout:6/33: write d0/f5 [1949633,93910] 0 2024-09-10T04:04:50.212 INFO:tasks.workunit.client.1.smithi070.stdout:0/13: symlink l3 0 2024-09-10T04:04:50.213 INFO:tasks.workunit.client.1.smithi070.stdout:3/7: getdents . 0 2024-09-10T04:04:50.213 INFO:tasks.workunit.client.1.smithi070.stdout:3/8: dwrite - no filename 2024-09-10T04:04:50.215 INFO:tasks.workunit.client.1.smithi070.stdout:3/9: chown c0 21094 1 2024-09-10T04:04:50.215 INFO:tasks.workunit.client.1.smithi070.stdout:4/17: getdents . 0 2024-09-10T04:04:50.216 INFO:tasks.workunit.client.1.smithi070.stdout:6/34: write d0/f8 [606187,58060] 0 2024-09-10T04:04:50.216 INFO:tasks.workunit.client.1.smithi070.stdout:5/20: dwrite f0 [0,4194304] 0 2024-09-10T04:04:50.218 INFO:tasks.workunit.client.1.smithi070.stdout:6/35: write d0/f5 [828013,105284] 0 2024-09-10T04:04:50.220 INFO:tasks.workunit.client.1.smithi070.stdout:0/14: creat f4 x:0 0 0 2024-09-10T04:04:50.220 INFO:tasks.workunit.client.1.smithi070.stdout:1/11: creat d3/f5 x:0 0 0 2024-09-10T04:04:50.228 INFO:tasks.workunit.client.1.smithi070.stdout:6/36: dread d0/f5 [0,4194304] 0 2024-09-10T04:04:50.228 INFO:tasks.workunit.client.1.smithi070.stdout:3/10: getdents . 0 2024-09-10T04:04:50.228 INFO:tasks.workunit.client.1.smithi070.stdout:3/11: dread - no filename 2024-09-10T04:04:50.230 INFO:tasks.workunit.client.1.smithi070.stdout:4/18: dread f4 [0,4194304] 0 2024-09-10T04:04:50.232 INFO:tasks.workunit.client.1.smithi070.stdout:3/12: creat f1 x:0 0 0 2024-09-10T04:04:50.233 INFO:tasks.workunit.client.1.smithi070.stdout:5/21: rmdir d5 0 2024-09-10T04:04:50.235 INFO:tasks.workunit.client.1.smithi070.stdout:5/22: chown f3 0 1 2024-09-10T04:04:50.239 INFO:tasks.workunit.client.1.smithi070.stdout:0/15: dwrite f2 [0,4194304] 0 2024-09-10T04:04:50.240 INFO:tasks.workunit.client.1.smithi070.stdout:4/19: dread f1 [0,4194304] 0 2024-09-10T04:04:50.240 INFO:tasks.workunit.client.1.smithi070.stdout:6/37: write d0/f5 [415759,63767] 0 2024-09-10T04:04:50.244 INFO:tasks.workunit.client.1.smithi070.stdout:0/16: truncate f1 852249 0 2024-09-10T04:04:50.244 INFO:tasks.workunit.client.1.smithi070.stdout:6/38: fsync d0/f8 0 2024-09-10T04:04:50.244 INFO:tasks.workunit.client.1.smithi070.stdout:4/20: chown c2 5172 1 2024-09-10T04:04:50.245 INFO:tasks.workunit.client.1.smithi070.stdout:1/12: dwrite d3/f5 [0,4194304] 0 2024-09-10T04:04:50.254 INFO:tasks.workunit.client.1.smithi070.stdout:3/13: dwrite f1 [0,4194304] 0 2024-09-10T04:04:50.257 INFO:tasks.workunit.client.1.smithi070.stdout:4/21: link f1 f5 0 2024-09-10T04:04:50.258 INFO:tasks.workunit.client.1.smithi070.stdout:1/13: link f1 d3/f6 0 2024-09-10T04:04:50.259 INFO:tasks.workunit.client.1.smithi070.stdout:6/39: getdents d0/d3 0 2024-09-10T04:04:50.262 INFO:tasks.workunit.client.1.smithi070.stdout:0/17: dwrite f2 [0,4194304] 0 2024-09-10T04:04:50.262 INFO:tasks.workunit.client.1.smithi070.stdout:4/22: unlink f1 0 2024-09-10T04:04:50.264 INFO:tasks.workunit.client.1.smithi070.stdout:4/23: write f0 [796372,107781] 0 2024-09-10T04:04:50.264 INFO:tasks.workunit.client.1.smithi070.stdout:4/24: readlink l3 0 2024-09-10T04:04:50.269 INFO:tasks.workunit.client.1.smithi070.stdout:0/18: rename f2 to f5 0 2024-09-10T04:04:50.271 INFO:tasks.workunit.client.1.smithi070.stdout:4/25: creat f6 x:0 0 0 2024-09-10T04:04:50.275 INFO:tasks.workunit.client.1.smithi070.stdout:6/40: link d0/l4 d0/l9 0 2024-09-10T04:04:50.275 INFO:tasks.workunit.client.1.smithi070.stdout:4/26: dread f4 [0,4194304] 0 2024-09-10T04:04:50.277 INFO:tasks.workunit.client.1.smithi070.stdout:6/41: mknod d0/ca 0 2024-09-10T04:04:50.279 INFO:tasks.workunit.client.1.smithi070.stdout:4/27: mknod c7 0 2024-09-10T04:04:50.279 INFO:tasks.workunit.client.1.smithi070.stdout:1/14: dwrite d3/f6 [0,4194304] 0 2024-09-10T04:04:50.281 INFO:tasks.workunit.client.1.smithi070.stdout:6/42: write d0/f8 [697648,60887] 0 2024-09-10T04:04:50.283 INFO:tasks.workunit.client.1.smithi070.stdout:6/43: write d0/f5 [77189,16221] 0 2024-09-10T04:04:50.285 INFO:tasks.workunit.client.1.smithi070.stdout:1/15: creat d3/f7 x:0 0 0 2024-09-10T04:04:50.286 INFO:tasks.workunit.client.1.smithi070.stdout:4/28: write f4 [2000238,57153] 0 2024-09-10T04:04:50.287 INFO:tasks.workunit.client.1.smithi070.stdout:6/44: write d0/f8 [636893,88252] 0 2024-09-10T04:04:50.288 INFO:tasks.workunit.client.1.smithi070.stdout:0/19: dwrite f5 [0,4194304] 0 2024-09-10T04:04:50.292 INFO:tasks.workunit.client.1.smithi070.stdout:1/16: mkdir d3/d8 0 2024-09-10T04:04:50.292 INFO:tasks.workunit.client.1.smithi070.stdout:6/45: mknod d0/d3/cb 0 2024-09-10T04:04:50.293 INFO:tasks.workunit.client.1.smithi070.stdout:0/20: mknod c6 0 2024-09-10T04:04:50.295 INFO:tasks.workunit.client.1.smithi070.stdout:1/17: mknod d3/c9 0 2024-09-10T04:04:50.298 INFO:tasks.workunit.client.1.smithi070.stdout:6/46: rename d0/d3/cb to d0/cc 0 2024-09-10T04:04:50.298 INFO:tasks.workunit.client.1.smithi070.stdout:0/21: dread f5 [0,4194304] 0 2024-09-10T04:04:50.301 INFO:tasks.workunit.client.1.smithi070.stdout:1/18: dread f1 [0,4194304] 0 2024-09-10T04:04:50.301 INFO:tasks.workunit.client.1.smithi070.stdout:0/22: symlink l7 0 2024-09-10T04:04:50.302 INFO:tasks.workunit.client.1.smithi070.stdout:1/19: read f0 [817210,36305] 0 2024-09-10T04:04:50.303 INFO:tasks.workunit.client.1.smithi070.stdout:1/20: read f1 [3094725,95243] 0 2024-09-10T04:04:50.305 INFO:tasks.workunit.client.1.smithi070.stdout:0/23: unlink f4 0 2024-09-10T04:04:50.305 INFO:tasks.workunit.client.1.smithi070.stdout:0/24: chown l7 171849949 1 2024-09-10T04:04:50.306 INFO:tasks.workunit.client.1.smithi070.stdout:0/25: write f1 [833195,92740] 0 2024-09-10T04:04:50.315 INFO:tasks.workunit.client.1.smithi070.stdout:0/26: dwrite f5 [0,4194304] 0 2024-09-10T04:04:50.319 INFO:tasks.workunit.client.1.smithi070.stdout:0/27: mkdir d8 0 2024-09-10T04:04:50.322 INFO:tasks.workunit.client.1.smithi070.stdout:0/28: read f1 [809458,127730] 0 2024-09-10T04:04:50.322 INFO:tasks.workunit.client.1.smithi070.stdout:0/29: readlink l7 0 2024-09-10T04:04:50.322 INFO:tasks.workunit.client.1.smithi070.stdout:0/30: chown d8 6 1 2024-09-10T04:04:50.325 INFO:tasks.workunit.client.1.smithi070.stdout:0/31: write f0 [1888928,3731] 0 2024-09-10T04:04:50.330 INFO:tasks.workunit.client.1.smithi070.stdout:0/32: dread f0 [0,4194304] 0 2024-09-10T04:04:50.330 INFO:tasks.workunit.client.1.smithi070.stdout:0/33: readlink l3 0 2024-09-10T04:04:50.334 INFO:tasks.workunit.client.1.smithi070.stdout:0/34: symlink d8/l9 0 2024-09-10T04:04:50.335 INFO:tasks.workunit.client.1.smithi070.stdout:0/35: write f5 [2994540,114286] 0 2024-09-10T04:04:50.338 INFO:tasks.workunit.client.1.smithi070.stdout:0/36: unlink l3 0 2024-09-10T04:04:50.478 INFO:tasks.workunit.client.1.smithi070.stdout:0/37: dwrite f0 [4194304,4194304] 0 2024-09-10T04:04:50.480 INFO:tasks.workunit.client.1.smithi070.stdout:9/6: unlink l2 0 2024-09-10T04:04:50.482 INFO:tasks.workunit.client.1.smithi070.stdout:0/38: creat d8/fa x:0 0 0 2024-09-10T04:04:50.482 INFO:tasks.workunit.client.1.smithi070.stdout:9/7: symlink l3 0 2024-09-10T04:04:50.482 INFO:tasks.workunit.client.1.smithi070.stdout:0/39: readlink d8/l9 0 2024-09-10T04:04:50.484 INFO:tasks.workunit.client.1.smithi070.stdout:9/8: rename l3 to l4 0 2024-09-10T04:04:50.484 INFO:tasks.workunit.client.1.smithi070.stdout:9/9: read - no filename 2024-09-10T04:04:50.484 INFO:tasks.workunit.client.1.smithi070.stdout:9/10: rmdir - no directory 2024-09-10T04:04:50.485 INFO:tasks.workunit.client.1.smithi070.stdout:9/11: truncate - no filename 2024-09-10T04:04:50.485 INFO:tasks.workunit.client.1.smithi070.stdout:9/12: fdatasync - no filename 2024-09-10T04:04:50.485 INFO:tasks.workunit.client.1.smithi070.stdout:9/13: read - no filename 2024-09-10T04:04:50.485 INFO:tasks.workunit.client.1.smithi070.stdout:9/14: fdatasync - no filename 2024-09-10T04:04:50.485 INFO:tasks.workunit.client.1.smithi070.stdout:0/40: symlink d8/lb 0 2024-09-10T04:04:50.498 INFO:tasks.workunit.client.1.smithi070.stdout:3/14: truncate f1 3405001 0 2024-09-10T04:04:50.507 INFO:tasks.workunit.client.1.smithi070.stdout:1/21: dwrite f1 [4194304,4194304] 0 2024-09-10T04:04:50.508 INFO:tasks.workunit.client.1.smithi070.stdout:6/47: chown d0/cc 2391 1 2024-09-10T04:04:50.509 INFO:tasks.workunit.client.1.smithi070.stdout:1/22: chown d3/f5 25 1 2024-09-10T04:04:50.524 INFO:tasks.workunit.client.1.smithi070.stdout:6/48: dwrite d0/f8 [0,4194304] 0 2024-09-10T04:04:50.524 INFO:tasks.workunit.client.1.smithi070.stdout:1/23: dwrite d3/f7 [0,4194304] 0 2024-09-10T04:04:50.553 INFO:tasks.workunit.client.1.smithi070.stdout:9/15: stat l4 0 2024-09-10T04:04:50.554 INFO:tasks.workunit.client.1.smithi070.stdout:9/16: symlink l5 0 2024-09-10T04:04:50.555 INFO:tasks.workunit.client.1.smithi070.stdout:9/17: fdatasync - no filename 2024-09-10T04:04:50.555 INFO:tasks.workunit.client.1.smithi070.stdout:9/18: write - no filename 2024-09-10T04:04:50.555 INFO:tasks.workunit.client.1.smithi070.stdout:9/19: readlink l5 0 2024-09-10T04:04:50.555 INFO:tasks.workunit.client.1.smithi070.stdout:3/15: unlink f1 0 2024-09-10T04:04:50.556 INFO:tasks.workunit.client.1.smithi070.stdout:3/16: read - no filename 2024-09-10T04:04:50.556 INFO:tasks.workunit.client.1.smithi070.stdout:3/17: dread - no filename 2024-09-10T04:04:50.558 INFO:tasks.workunit.client.1.smithi070.stdout:9/20: rename l5 to l6 0 2024-09-10T04:04:50.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/21: write - no filename 2024-09-10T04:04:50.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/22: dread - no filename 2024-09-10T04:04:50.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/23: write - no filename 2024-09-10T04:04:50.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/24: write - no filename 2024-09-10T04:04:50.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/25: readlink l4 0 2024-09-10T04:04:50.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/18: unlink c0 0 2024-09-10T04:04:50.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/19: readlink - no filename 2024-09-10T04:04:50.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/20: write - no filename 2024-09-10T04:04:50.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/21: readlink - no filename 2024-09-10T04:04:50.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/22: chown . 1933303 1 2024-09-10T04:04:50.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/23: fdatasync - no filename 2024-09-10T04:04:50.564 INFO:tasks.workunit.client.1.smithi070.stdout:6/49: truncate d0/f8 3289698 0 2024-09-10T04:04:50.567 INFO:tasks.workunit.client.1.smithi070.stdout:3/24: mknod c2 0 2024-09-10T04:04:50.568 INFO:tasks.workunit.client.1.smithi070.stdout:3/25: chown c2 2123959 1 2024-09-10T04:04:50.568 INFO:tasks.workunit.client.1.smithi070.stdout:3/26: readlink - no filename 2024-09-10T04:04:50.570 INFO:tasks.workunit.client.1.smithi070.stdout:3/27: creat f3 x:0 0 0 2024-09-10T04:04:50.573 INFO:tasks.workunit.client.1.smithi070.stdout:3/28: rename c2 to c4 0 2024-09-10T04:04:50.574 INFO:tasks.workunit.client.1.smithi070.stdout:3/29: write f3 [826687,104710] 0 2024-09-10T04:04:50.583 INFO:tasks.workunit.client.1.smithi070.stdout:3/30: dwrite f3 [0,4194304] 0 2024-09-10T04:04:50.584 INFO:tasks.workunit.client.1.smithi070.stdout:9/26: rename l6 to l7 0 2024-09-10T04:04:50.584 INFO:tasks.workunit.client.1.smithi070.stdout:9/27: dread - no filename 2024-09-10T04:04:50.584 INFO:tasks.workunit.client.1.smithi070.stdout:9/28: dwrite - no filename 2024-09-10T04:04:50.584 INFO:tasks.workunit.client.1.smithi070.stdout:9/29: dread - no filename 2024-09-10T04:04:50.585 INFO:tasks.workunit.client.1.smithi070.stdout:3/31: write f3 [2688414,15764] 0 2024-09-10T04:04:50.587 INFO:tasks.workunit.client.1.smithi070.stdout:6/50: dread d0/f8 [0,4194304] 0 2024-09-10T04:04:50.587 INFO:tasks.workunit.client.1.smithi070.stdout:3/32: write f3 [2981251,61644] 0 2024-09-10T04:04:50.592 INFO:tasks.workunit.client.1.smithi070.stdout:3/33: symlink l5 0 2024-09-10T04:04:50.592 INFO:tasks.workunit.client.1.smithi070.stdout:3/34: chown f3 6 1 2024-09-10T04:04:50.593 INFO:tasks.workunit.client.1.smithi070.stdout:6/51: mkdir d0/d3/dd 0 2024-09-10T04:04:50.597 INFO:tasks.workunit.client.1.smithi070.stdout:3/35: dread f3 [0,4194304] 0 2024-09-10T04:04:50.599 INFO:tasks.workunit.client.1.smithi070.stdout:6/52: rmdir d0/d3 39 2024-09-10T04:04:50.600 INFO:tasks.workunit.client.1.smithi070.stdout:6/53: write d0/f5 [2845232,59285] 0 2024-09-10T04:04:50.603 INFO:tasks.workunit.client.1.smithi070.stdout:3/36: creat f6 x:0 0 0 2024-09-10T04:04:50.603 INFO:tasks.workunit.client.1.smithi070.stdout:3/37: dread - f6 zero size 2024-09-10T04:04:50.606 INFO:tasks.workunit.client.1.smithi070.stdout:9/30: getdents . 0 2024-09-10T04:04:50.608 INFO:tasks.workunit.client.1.smithi070.stdout:3/38: creat f7 x:0 0 0 2024-09-10T04:04:50.610 INFO:tasks.workunit.client.1.smithi070.stdout:9/31: rename l7 to l8 0 2024-09-10T04:04:50.610 INFO:tasks.workunit.client.1.smithi070.stdout:9/32: read - no filename 2024-09-10T04:04:50.612 INFO:tasks.workunit.client.1.smithi070.stdout:3/39: rename l5 to l8 0 2024-09-10T04:04:50.612 INFO:tasks.workunit.client.1.smithi070.stdout:6/54: dwrite d0/f8 [0,4194304] 0 2024-09-10T04:04:50.614 INFO:tasks.workunit.client.1.smithi070.stdout:9/33: mkdir d9 0 2024-09-10T04:04:50.620 INFO:tasks.workunit.client.1.smithi070.stdout:6/55: mknod d0/ce 0 2024-09-10T04:04:50.621 INFO:tasks.workunit.client.1.smithi070.stdout:9/34: symlink d9/la 0 2024-09-10T04:04:50.622 INFO:tasks.workunit.client.1.smithi070.stdout:3/40: dwrite f7 [0,4194304] 0 2024-09-10T04:04:50.622 INFO:tasks.workunit.client.1.smithi070.stdout:9/35: rename d9 to d9/db 22 2024-09-10T04:04:50.623 INFO:tasks.workunit.client.1.smithi070.stdout:6/56: write d0/f5 [3204684,42100] 0 2024-09-10T04:04:50.623 INFO:tasks.workunit.client.1.smithi070.stdout:9/36: rename d9 to d9/dc 22 2024-09-10T04:04:50.623 INFO:tasks.workunit.client.1.smithi070.stdout:9/37: dwrite - no filename 2024-09-10T04:04:50.623 INFO:tasks.workunit.client.1.smithi070.stdout:9/38: dwrite - no filename 2024-09-10T04:04:50.627 INFO:tasks.workunit.client.1.smithi070.stdout:9/39: mkdir d9/dd 0 2024-09-10T04:04:50.635 INFO:tasks.workunit.client.1.smithi070.stdout:6/57: dwrite d0/f5 [0,4194304] 0 2024-09-10T04:04:50.639 INFO:tasks.workunit.client.1.smithi070.stdout:9/40: stat l8 0 2024-09-10T04:04:50.642 INFO:tasks.workunit.client.1.smithi070.stdout:9/41: mkdir d9/de 0 2024-09-10T04:04:50.642 INFO:tasks.workunit.client.1.smithi070.stdout:9/42: write - no filename 2024-09-10T04:04:50.653 INFO:tasks.workunit.client.1.smithi070.stdout:6/58: fdatasync d0/f5 0 2024-09-10T04:04:50.653 INFO:tasks.workunit.client.1.smithi070.stdout:6/59: fsync d0/f5 0 2024-09-10T04:04:50.654 INFO:tasks.workunit.client.1.smithi070.stdout:6/60: write d0/f8 [3893449,50024] 0 2024-09-10T04:04:50.659 INFO:tasks.workunit.client.1.smithi070.stdout:6/61: mknod d0/d3/cf 0 2024-09-10T04:04:50.663 INFO:tasks.workunit.client.1.smithi070.stdout:3/41: truncate f7 992137 0 2024-09-10T04:04:50.665 INFO:tasks.workunit.client.1.smithi070.stdout:6/62: creat d0/f10 x:0 0 0 2024-09-10T04:04:50.669 INFO:tasks.workunit.client.1.smithi070.stdout:3/42: rename l8 to l9 0 2024-09-10T04:04:50.670 INFO:tasks.workunit.client.1.smithi070.stdout:6/63: creat d0/d3/dd/f11 x:0 0 0 2024-09-10T04:04:50.670 INFO:tasks.workunit.client.1.smithi070.stdout:6/64: stat d0/ce 0 2024-09-10T04:04:50.671 INFO:tasks.workunit.client.1.smithi070.stdout:3/43: chown c4 243 1 2024-09-10T04:04:50.673 INFO:tasks.workunit.client.1.smithi070.stdout:6/65: creat d0/d3/f12 x:0 0 0 2024-09-10T04:04:50.680 INFO:tasks.workunit.client.1.smithi070.stdout:6/66: dwrite d0/d3/dd/f11 [0,4194304] 0 2024-09-10T04:04:50.681 INFO:tasks.workunit.client.1.smithi070.stdout:6/67: fdatasync d0/f8 0 2024-09-10T04:04:50.682 INFO:tasks.workunit.client.1.smithi070.stdout:6/68: stat d0/l2 0 2024-09-10T04:04:50.686 INFO:tasks.workunit.client.1.smithi070.stdout:6/69: readlink d0/l4 0 2024-09-10T04:04:50.689 INFO:tasks.workunit.client.1.smithi070.stdout:6/70: mknod d0/d3/c13 0 2024-09-10T04:04:50.693 INFO:tasks.workunit.client.1.smithi070.stdout:6/71: creat d0/d3/dd/f14 x:0 0 0 2024-09-10T04:04:50.698 INFO:tasks.workunit.client.1.smithi070.stdout:3/44: write f7 [631115,75062] 0 2024-09-10T04:04:50.698 INFO:tasks.workunit.client.1.smithi070.stdout:3/45: fdatasync f6 0 2024-09-10T04:04:50.699 INFO:tasks.workunit.client.1.smithi070.stdout:6/72: rename d0/f5 to d0/f15 0 2024-09-10T04:04:50.700 INFO:tasks.workunit.client.1.smithi070.stdout:6/73: read - d0/d3/f12 zero size 2024-09-10T04:04:50.704 INFO:tasks.workunit.client.1.smithi070.stdout:6/74: creat d0/d3/f16 x:0 0 0 2024-09-10T04:04:50.705 INFO:tasks.workunit.client.1.smithi070.stdout:6/75: dread - d0/d3/dd/f14 zero size 2024-09-10T04:04:50.708 INFO:tasks.workunit.client.1.smithi070.stdout:6/76: mkdir d0/d3/dd/d17 0 2024-09-10T04:04:50.710 INFO:tasks.workunit.client.1.smithi070.stdout:6/77: mkdir d0/d3/d18 0 2024-09-10T04:04:50.711 INFO:tasks.workunit.client.1.smithi070.stdout:6/78: truncate d0/f15 4409035 0 2024-09-10T04:04:50.712 INFO:tasks.workunit.client.1.smithi070.stdout:6/79: chown d0/d3/f12 4853 1 2024-09-10T04:04:50.715 INFO:tasks.workunit.client.1.smithi070.stdout:6/80: write d0/f15 [4580029,50176] 0 2024-09-10T04:04:50.716 INFO:tasks.workunit.client.1.smithi070.stdout:6/81: dread - d0/d3/f16 zero size 2024-09-10T04:04:50.721 INFO:tasks.workunit.client.1.smithi070.stdout:6/82: creat d0/d3/f19 x:0 0 0 2024-09-10T04:04:50.728 INFO:tasks.workunit.client.1.smithi070.stdout:6/83: dwrite d0/d3/dd/f11 [0,4194304] 0 2024-09-10T04:04:50.736 INFO:tasks.workunit.client.1.smithi070.stdout:6/84: truncate d0/f8 3414281 0 2024-09-10T04:04:50.738 INFO:tasks.workunit.client.1.smithi070.stdout:6/85: symlink d0/d3/l1a 0 2024-09-10T04:04:50.758 INFO:tasks.workunit.client.1.smithi070.stdout:6/86: dread d0/f8 [0,4194304] 0 2024-09-10T04:04:50.766 INFO:tasks.workunit.client.1.smithi070.stdout:6/87: dwrite d0/f10 [0,4194304] 0 2024-09-10T04:04:50.773 INFO:tasks.workunit.client.1.smithi070.stdout:6/88: dwrite d0/d3/dd/f11 [0,4194304] 0 2024-09-10T04:04:50.777 INFO:tasks.workunit.client.1.smithi070.stdout:6/89: rename d0/d3/dd to d0/d3/d18/d1b 0 2024-09-10T04:04:50.781 INFO:tasks.workunit.client.1.smithi070.stdout:6/90: rename d0/l4 to d0/d3/l1c 0 2024-09-10T04:04:50.782 INFO:tasks.workunit.client.1.smithi070.stdout:6/91: fsync d0/d3/d18/d1b/f14 0 2024-09-10T04:04:50.783 INFO:tasks.workunit.client.1.smithi070.stdout:6/92: dread - d0/d3/d18/d1b/f14 zero size 2024-09-10T04:04:50.787 INFO:tasks.workunit.client.1.smithi070.stdout:6/93: symlink d0/l1d 0 2024-09-10T04:04:50.794 INFO:tasks.workunit.client.1.smithi070.stdout:6/94: dread d0/d3/d18/d1b/f11 [0,4194304] 0 2024-09-10T04:04:50.798 INFO:tasks.workunit.client.1.smithi070.stdout:6/95: link d0/d3/cf d0/d3/d18/d1b/c1e 0 2024-09-10T04:04:50.800 INFO:tasks.workunit.client.1.smithi070.stdout:6/96: mkdir d0/d3/d1f 0 2024-09-10T04:04:50.804 INFO:tasks.workunit.client.1.smithi070.stdout:6/97: link d0/l9 d0/d3/d18/d1b/d17/l20 0 2024-09-10T04:04:50.806 INFO:tasks.workunit.client.1.smithi070.stdout:6/98: symlink d0/d3/d1f/l21 0 2024-09-10T04:04:50.809 INFO:tasks.workunit.client.1.smithi070.stdout:6/99: creat d0/d3/d1f/f22 x:0 0 0 2024-09-10T04:04:50.810 INFO:tasks.workunit.client.1.smithi070.stdout:6/100: readlink d0/d3/d1f/l21 0 2024-09-10T04:04:50.817 INFO:tasks.workunit.client.1.smithi070.stdout:6/101: dwrite d0/d3/d18/d1b/f11 [0,4194304] 0 2024-09-10T04:04:50.825 INFO:tasks.workunit.client.1.smithi070.stdout:6/102: link d0/d3/cf d0/c23 0 2024-09-10T04:04:50.851 INFO:tasks.workunit.client.1.smithi070.stdout:6/103: write d0/f8 [261358,6664] 0 2024-09-10T04:04:50.853 INFO:tasks.workunit.client.1.smithi070.stdout:6/104: write d0/d3/d18/d1b/f11 [5240723,32030] 0 2024-09-10T04:04:50.862 INFO:tasks.workunit.client.1.smithi070.stdout:6/105: truncate d0/f8 2007625 0 2024-09-10T04:04:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:51 smithi070 ceph-mon[35495]: pgmap v249: 65 pgs: 65 active+clean; 179 MiB data, 730 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-09-10T04:04:52.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:51 smithi046 ceph-mon[28328]: pgmap v249: 65 pgs: 65 active+clean; 179 MiB data, 730 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-09-10T04:04:53.987 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:53 smithi046 ceph-mon[28328]: pgmap v250: 65 pgs: 65 active+clean; 220 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 302 op/s 2024-09-10T04:04:54.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:53 smithi070 ceph-mon[35495]: pgmap v250: 65 pgs: 65 active+clean; 220 MiB data, 811 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 302 op/s 2024-09-10T04:04:55.942 INFO:tasks.workunit.client.1.smithi070.stdout:2/27: sync 2024-09-10T04:04:55.943 INFO:tasks.workunit.client.1.smithi070.stdout:2/28: rename - no filename 2024-09-10T04:04:55.943 INFO:tasks.workunit.client.1.smithi070.stdout:2/29: fdatasync - no filename 2024-09-10T04:04:55.943 INFO:tasks.workunit.client.1.smithi070.stdout:2/30: dread - no filename 2024-09-10T04:04:55.943 INFO:tasks.workunit.client.1.smithi070.stdout:2/31: write - no filename 2024-09-10T04:04:55.943 INFO:tasks.workunit.client.1.smithi070.stdout:2/32: truncate - no filename 2024-09-10T04:04:55.943 INFO:tasks.workunit.client.1.smithi070.stdout:2/33: creat f3 x:0 0 0 2024-09-10T04:04:55.944 INFO:tasks.workunit.client.1.smithi070.stdout:2/34: truncate f3 702265 0 2024-09-10T04:04:56.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:55 smithi070 ceph-mon[35495]: pgmap v251: 65 pgs: 65 active+clean; 241 MiB data, 911 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 259 op/s 2024-09-10T04:04:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:55 smithi046 ceph-mon[28328]: pgmap v251: 65 pgs: 65 active+clean; 241 MiB data, 911 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 259 op/s 2024-09-10T04:04:58.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:57 smithi070 ceph-mon[35495]: pgmap v252: 65 pgs: 65 active+clean; 264 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 278 op/s 2024-09-10T04:04:58.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:57 smithi046 ceph-mon[28328]: pgmap v252: 65 pgs: 65 active+clean; 264 MiB data, 1013 MiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 13 MiB/s wr, 278 op/s 2024-09-10T04:05:00.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:04:59 smithi070 ceph-mon[35495]: pgmap v253: 65 pgs: 65 active+clean; 265 MiB data, 1010 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 259 op/s 2024-09-10T04:05:00.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:04:59 smithi046 ceph-mon[28328]: pgmap v253: 65 pgs: 65 active+clean; 265 MiB data, 1010 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 259 op/s 2024-09-10T04:05:02.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:01 smithi070 ceph-mon[35495]: pgmap v254: 65 pgs: 65 active+clean; 266 MiB data, 1017 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 243 op/s 2024-09-10T04:05:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:01 smithi046 ceph-mon[28328]: pgmap v254: 65 pgs: 65 active+clean; 266 MiB data, 1017 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 243 op/s 2024-09-10T04:05:02.241 INFO:tasks.workunit.client.0.smithi046.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-10T04:05:02.245 INFO:tasks.workunit.client.0.smithi046.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-10T04:05:02.245 INFO:tasks.workunit.client.0.smithi046.stderr:+ make 2024-09-10T04:05:02.460 INFO:tasks.workunit.client.0.smithi046.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-10T04:05:03.565 INFO:tasks.workunit.client.0.smithi046.stderr:++ readlink -f fsstress 2024-09-10T04:05:03.566 INFO:tasks.workunit.client.0.smithi046.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-10T04:05:03.566 INFO:tasks.workunit.client.0.smithi046.stderr:+ popd 2024-09-10T04:05:03.567 INFO:tasks.workunit.client.0.smithi046.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-10T04:05:03.567 INFO:tasks.workunit.client.0.smithi046.stderr:+ popd 2024-09-10T04:05:03.567 INFO:tasks.workunit.client.0.smithi046.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-10T04:05:03.568 INFO:tasks.workunit.client.0.smithi046.stderr:++ mktemp -d -p . 2024-09-10T04:05:03.570 INFO:tasks.workunit.client.0.smithi046.stderr:+ T=./tmp.WKIhNanA7D 2024-09-10T04:05:03.570 INFO:tasks.workunit.client.0.smithi046.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.WKIhNanA7D -l 1 -n 1000 -p 10 -v 2024-09-10T04:05:03.573 INFO:tasks.workunit.client.0.smithi046.stdout:seed = 1725397709 2024-09-10T04:05:03.576 INFO:tasks.workunit.client.0.smithi046.stdout:0/0: truncate - no filename 2024-09-10T04:05:03.577 INFO:tasks.workunit.client.0.smithi046.stdout:0/1: chown . 13 1 2024-09-10T04:05:03.577 INFO:tasks.workunit.client.0.smithi046.stdout:0/2: dwrite - no filename 2024-09-10T04:05:03.577 INFO:tasks.workunit.client.0.smithi046.stdout:0/3: chown . 9514 1 2024-09-10T04:05:03.577 INFO:tasks.workunit.client.0.smithi046.stdout:0/4: rename - no filename 2024-09-10T04:05:03.577 INFO:tasks.workunit.client.0.smithi046.stdout:0/5: truncate - no filename 2024-09-10T04:05:03.577 INFO:tasks.workunit.client.0.smithi046.stdout:0/6: chown . 5 1 2024-09-10T04:05:03.578 INFO:tasks.workunit.client.0.smithi046.stdout:2/0: write - no filename 2024-09-10T04:05:03.578 INFO:tasks.workunit.client.0.smithi046.stdout:2/1: dread - no filename 2024-09-10T04:05:03.579 INFO:tasks.workunit.client.0.smithi046.stdout:2/2: write - no filename 2024-09-10T04:05:03.579 INFO:tasks.workunit.client.0.smithi046.stdout:2/3: dwrite - no filename 2024-09-10T04:05:03.579 INFO:tasks.workunit.client.0.smithi046.stdout:2/4: chown . 52 1 2024-09-10T04:05:03.579 INFO:tasks.workunit.client.0.smithi046.stdout:2/5: stat - no entries 2024-09-10T04:05:03.579 INFO:tasks.workunit.client.0.smithi046.stdout:2/6: link - no file 2024-09-10T04:05:03.580 INFO:tasks.workunit.client.0.smithi046.stdout:1/0: creat f0 x:0 0 0 2024-09-10T04:05:03.581 INFO:tasks.workunit.client.0.smithi046.stdout:2/7: symlink l0 0 2024-09-10T04:05:03.582 INFO:tasks.workunit.client.0.smithi046.stdout:2/8: rmdir - no directory 2024-09-10T04:05:03.582 INFO:tasks.workunit.client.0.smithi046.stdout:2/9: dwrite - no filename 2024-09-10T04:05:03.582 INFO:tasks.workunit.client.0.smithi046.stdout:2/10: readlink l0 0 2024-09-10T04:05:03.582 INFO:tasks.workunit.client.0.smithi046.stdout:2/11: chown l0 8291 1 2024-09-10T04:05:03.583 INFO:tasks.workunit.client.0.smithi046.stdout:2/12: write - no filename 2024-09-10T04:05:03.583 INFO:tasks.workunit.client.0.smithi046.stdout:2/13: dwrite - no filename 2024-09-10T04:05:03.583 INFO:tasks.workunit.client.0.smithi046.stdout:4/0: stat - no entries 2024-09-10T04:05:03.584 INFO:tasks.workunit.client.0.smithi046.stdout:2/14: symlink l1 0 2024-09-10T04:05:03.584 INFO:tasks.workunit.client.0.smithi046.stdout:2/15: dread - no filename 2024-09-10T04:05:03.585 INFO:tasks.workunit.client.0.smithi046.stdout:5/0: write - no filename 2024-09-10T04:05:03.585 INFO:tasks.workunit.client.0.smithi046.stdout:5/1: truncate - no filename 2024-09-10T04:05:03.586 INFO:tasks.workunit.client.0.smithi046.stdout:2/16: mkdir d2 0 2024-09-10T04:05:03.586 INFO:tasks.workunit.client.0.smithi046.stdout:2/17: fsync - no filename 2024-09-10T04:05:03.586 INFO:tasks.workunit.client.0.smithi046.stdout:4/1: mknod c0 0 2024-09-10T04:05:03.586 INFO:tasks.workunit.client.0.smithi046.stdout:4/2: truncate - no filename 2024-09-10T04:05:03.587 INFO:tasks.workunit.client.0.smithi046.stdout:6/0: readlink - no filename 2024-09-10T04:05:03.588 INFO:tasks.workunit.client.0.smithi046.stdout:4/3: creat f1 x:0 0 0 2024-09-10T04:05:03.589 INFO:tasks.workunit.client.0.smithi046.stdout:5/2: creat f0 x:0 0 0 2024-09-10T04:05:03.590 INFO:tasks.workunit.client.0.smithi046.stdout:5/3: chown f0 30071483 1 2024-09-10T04:05:03.590 INFO:tasks.workunit.client.0.smithi046.stdout:5/4: dread - f0 zero size 2024-09-10T04:05:03.591 INFO:tasks.workunit.client.0.smithi046.stdout:7/0: fsync - no filename 2024-09-10T04:05:03.591 INFO:tasks.workunit.client.0.smithi046.stdout:7/1: fdatasync - no filename 2024-09-10T04:05:03.592 INFO:tasks.workunit.client.0.smithi046.stdout:2/18: creat d2/f3 x:0 0 0 2024-09-10T04:05:03.592 INFO:tasks.workunit.client.0.smithi046.stdout:2/19: chown d2/f3 37612879 1 2024-09-10T04:05:03.593 INFO:tasks.workunit.client.0.smithi046.stdout:2/20: write d2/f3 [834184,104900] 0 2024-09-10T04:05:03.594 INFO:tasks.workunit.client.0.smithi046.stdout:5/5: creat f1 x:0 0 0 2024-09-10T04:05:03.594 INFO:tasks.workunit.client.0.smithi046.stdout:5/6: dread - f1 zero size 2024-09-10T04:05:03.595 INFO:tasks.workunit.client.0.smithi046.stdout:5/7: chown f0 81831901 1 2024-09-10T04:05:03.595 INFO:tasks.workunit.client.0.smithi046.stdout:8/0: fsync - no filename 2024-09-10T04:05:03.595 INFO:tasks.workunit.client.0.smithi046.stdout:8/1: fsync - no filename 2024-09-10T04:05:03.598 INFO:tasks.workunit.client.0.smithi046.stdout:7/2: creat f0 x:0 0 0 2024-09-10T04:05:03.599 INFO:tasks.workunit.client.0.smithi046.stdout:7/3: truncate f0 484261 0 2024-09-10T04:05:03.600 INFO:tasks.workunit.client.0.smithi046.stdout:5/8: creat f2 x:0 0 0 2024-09-10T04:05:03.601 INFO:tasks.workunit.client.0.smithi046.stdout:4/4: link c0 c2 0 2024-09-10T04:05:03.604 INFO:tasks.workunit.client.0.smithi046.stdout:7/4: creat f1 x:0 0 0 2024-09-10T04:05:03.605 INFO:tasks.workunit.client.0.smithi046.stdout:7/5: write f1 [87885,9016] 0 2024-09-10T04:05:03.605 INFO:tasks.workunit.client.0.smithi046.stdout:8/2: creat f0 x:0 0 0 2024-09-10T04:05:03.607 INFO:tasks.workunit.client.0.smithi046.stdout:4/5: mkdir d3 0 2024-09-10T04:05:03.607 INFO:tasks.workunit.client.0.smithi046.stdout:4/6: chown f1 0 1 2024-09-10T04:05:03.608 INFO:tasks.workunit.client.0.smithi046.stdout:9/0: symlink l0 0 2024-09-10T04:05:03.608 INFO:tasks.workunit.client.0.smithi046.stdout:9/1: dwrite - no filename 2024-09-10T04:05:03.608 INFO:tasks.workunit.client.0.smithi046.stdout:9/2: fsync - no filename 2024-09-10T04:05:03.608 INFO:tasks.workunit.client.0.smithi046.stdout:9/3: fsync - no filename 2024-09-10T04:05:03.608 INFO:tasks.workunit.client.0.smithi046.stdout:4/7: dread - f1 zero size 2024-09-10T04:05:03.609 INFO:tasks.workunit.client.0.smithi046.stdout:4/8: write f1 [530966,112816] 0 2024-09-10T04:05:03.609 INFO:tasks.workunit.client.0.smithi046.stdout:7/6: creat f2 x:0 0 0 2024-09-10T04:05:03.610 INFO:tasks.workunit.client.0.smithi046.stdout:8/3: mkdir d1 0 2024-09-10T04:05:03.612 INFO:tasks.workunit.client.0.smithi046.stdout:8/4: rename d1 to d1/d2 22 2024-09-10T04:05:03.623 INFO:tasks.workunit.client.0.smithi046.stdout:4/9: dwrite f1 [0,4194304] 0 2024-09-10T04:05:03.624 INFO:tasks.workunit.client.0.smithi046.stdout:7/7: dwrite f2 [0,4194304] 0 2024-09-10T04:05:03.629 INFO:tasks.workunit.client.0.smithi046.stdout:7/8: rename f1 to f3 0 2024-09-10T04:05:03.632 INFO:tasks.workunit.client.0.smithi046.stdout:7/9: dread f0 [0,4194304] 0 2024-09-10T04:05:03.632 INFO:tasks.workunit.client.0.smithi046.stdout:4/10: dread f1 [0,4194304] 0 2024-09-10T04:05:03.633 INFO:tasks.workunit.client.0.smithi046.stdout:4/11: chown d3 0 1 2024-09-10T04:05:03.640 INFO:tasks.workunit.client.0.smithi046.stdout:7/10: link f0 f4 0 2024-09-10T04:05:03.640 INFO:tasks.workunit.client.0.smithi046.stdout:7/11: readlink - no filename 2024-09-10T04:05:03.640 INFO:tasks.workunit.client.0.smithi046.stdout:7/12: stat f4 0 2024-09-10T04:05:03.640 INFO:tasks.workunit.client.0.smithi046.stdout:4/12: write f1 [2513385,21265] 0 2024-09-10T04:05:03.642 INFO:tasks.workunit.client.0.smithi046.stdout:7/13: mkdir d5 0 2024-09-10T04:05:03.644 INFO:tasks.workunit.client.0.smithi046.stdout:7/14: write f3 [139265,88676] 0 2024-09-10T04:05:03.645 INFO:tasks.workunit.client.0.smithi046.stdout:7/15: write f3 [870480,49571] 0 2024-09-10T04:05:03.683 INFO:tasks.workunit.client.0.smithi046.stdout:1/1: fsync f0 0 2024-09-10T04:05:03.684 INFO:tasks.workunit.client.0.smithi046.stdout:1/2: creat f1 x:0 0 0 2024-09-10T04:05:03.685 INFO:tasks.workunit.client.0.smithi046.stdout:1/3: dread - f1 zero size 2024-09-10T04:05:03.686 INFO:tasks.workunit.client.0.smithi046.stdout:1/4: write f0 [236197,55423] 0 2024-09-10T04:05:03.686 INFO:tasks.workunit.client.0.smithi046.stdout:1/5: chown f1 56511701 1 2024-09-10T04:05:03.688 INFO:tasks.workunit.client.0.smithi046.stdout:1/6: read f0 [116,33507] 0 2024-09-10T04:05:03.691 INFO:tasks.workunit.client.0.smithi046.stdout:5/9: getdents . 0 2024-09-10T04:05:03.692 INFO:tasks.workunit.client.0.smithi046.stdout:2/21: truncate d2/f3 350653 0 2024-09-10T04:05:03.693 INFO:tasks.workunit.client.0.smithi046.stdout:8/5: fsync f0 0 2024-09-10T04:05:03.702 INFO:tasks.workunit.client.0.smithi046.stdout:7/16: truncate f2 3530404 0 2024-09-10T04:05:03.703 INFO:tasks.workunit.client.0.smithi046.stdout:5/10: dwrite f1 [0,4194304] 0 2024-09-10T04:05:03.708 INFO:tasks.workunit.client.0.smithi046.stdout:5/11: write f2 [574797,91207] 0 2024-09-10T04:05:03.708 INFO:tasks.workunit.client.0.smithi046.stdout:8/6: dwrite f0 [0,4194304] 0 2024-09-10T04:05:03.710 INFO:tasks.workunit.client.0.smithi046.stdout:2/22: dread d2/f3 [0,4194304] 0 2024-09-10T04:05:03.713 INFO:tasks.workunit.client.0.smithi046.stdout:2/23: dread d2/f3 [0,4194304] 0 2024-09-10T04:05:03.714 INFO:tasks.workunit.client.0.smithi046.stdout:2/24: truncate d2/f3 811504 0 2024-09-10T04:05:03.715 INFO:tasks.workunit.client.0.smithi046.stdout:8/7: unlink f0 0 2024-09-10T04:05:03.715 INFO:tasks.workunit.client.0.smithi046.stdout:8/8: write - no filename 2024-09-10T04:05:03.715 INFO:tasks.workunit.client.0.smithi046.stdout:8/9: chown d1 2985 1 2024-09-10T04:05:03.715 INFO:tasks.workunit.client.0.smithi046.stdout:8/10: write - no filename 2024-09-10T04:05:03.716 INFO:tasks.workunit.client.0.smithi046.stdout:8/11: dread - no filename 2024-09-10T04:05:03.716 INFO:tasks.workunit.client.0.smithi046.stdout:8/12: link - no file 2024-09-10T04:05:03.716 INFO:tasks.workunit.client.0.smithi046.stdout:2/25: mknod d2/c4 0 2024-09-10T04:05:03.716 INFO:tasks.workunit.client.0.smithi046.stdout:2/26: rename d2 to d2/d5 22 2024-09-10T04:05:03.719 INFO:tasks.workunit.client.0.smithi046.stdout:5/12: fsync f1 0 2024-09-10T04:05:03.720 INFO:tasks.workunit.client.0.smithi046.stdout:2/27: write d2/f3 [1297596,81215] 0 2024-09-10T04:05:03.721 INFO:tasks.workunit.client.0.smithi046.stdout:8/13: creat d1/f3 x:0 0 0 2024-09-10T04:05:03.721 INFO:tasks.workunit.client.0.smithi046.stdout:8/14: chown d1/f3 7719 1 2024-09-10T04:05:03.722 INFO:tasks.workunit.client.0.smithi046.stdout:8/15: dread - d1/f3 zero size 2024-09-10T04:05:03.723 INFO:tasks.workunit.client.0.smithi046.stdout:5/13: rename f2 to f3 0 2024-09-10T04:05:03.724 INFO:tasks.workunit.client.0.smithi046.stdout:2/28: mkdir d2/d6 0 2024-09-10T04:05:03.725 INFO:tasks.workunit.client.0.smithi046.stdout:7/17: truncate f2 3535809 0 2024-09-10T04:05:03.726 INFO:tasks.workunit.client.0.smithi046.stdout:8/16: mknod d1/c4 0 2024-09-10T04:05:03.728 INFO:tasks.workunit.client.0.smithi046.stdout:2/29: creat d2/d6/f7 x:0 0 0 2024-09-10T04:05:03.729 INFO:tasks.workunit.client.0.smithi046.stdout:8/17: creat d1/f5 x:0 0 0 2024-09-10T04:05:03.730 INFO:tasks.workunit.client.0.smithi046.stdout:7/18: mkdir d5/d6 0 2024-09-10T04:05:03.731 INFO:tasks.workunit.client.0.smithi046.stdout:2/30: mkdir d2/d8 0 2024-09-10T04:05:03.731 INFO:tasks.workunit.client.0.smithi046.stdout:8/18: rmdir d1 39 2024-09-10T04:05:03.732 INFO:tasks.workunit.client.0.smithi046.stdout:2/31: chown d2/d6/f7 203113663 1 2024-09-10T04:05:03.734 INFO:tasks.workunit.client.0.smithi046.stdout:7/19: link f3 d5/f7 0 2024-09-10T04:05:03.736 INFO:tasks.workunit.client.0.smithi046.stdout:8/19: mkdir d1/d6 0 2024-09-10T04:05:03.736 INFO:tasks.workunit.client.0.smithi046.stdout:2/32: dread d2/f3 [0,4194304] 0 2024-09-10T04:05:03.738 INFO:tasks.workunit.client.0.smithi046.stdout:8/20: rename d1/f5 to d1/f7 0 2024-09-10T04:05:03.739 INFO:tasks.workunit.client.0.smithi046.stdout:7/20: dread f3 [0,4194304] 0 2024-09-10T04:05:03.741 INFO:tasks.workunit.client.0.smithi046.stdout:2/33: symlink d2/l9 0 2024-09-10T04:05:03.744 INFO:tasks.workunit.client.0.smithi046.stdout:7/21: mknod d5/c8 0 2024-09-10T04:05:03.745 INFO:tasks.workunit.client.0.smithi046.stdout:2/34: dread d2/f3 [0,4194304] 0 2024-09-10T04:05:03.746 INFO:tasks.workunit.client.0.smithi046.stdout:7/22: mkdir d5/d9 0 2024-09-10T04:05:03.748 INFO:tasks.workunit.client.0.smithi046.stdout:8/21: dwrite d1/f3 [0,4194304] 0 2024-09-10T04:05:03.748 INFO:tasks.workunit.client.0.smithi046.stdout:8/22: chown d1/d6 27 1 2024-09-10T04:05:03.750 INFO:tasks.workunit.client.0.smithi046.stdout:2/35: mknod d2/d8/ca 0 2024-09-10T04:05:03.751 INFO:tasks.workunit.client.0.smithi046.stdout:7/23: mknod d5/d6/ca 0 2024-09-10T04:05:03.753 INFO:tasks.workunit.client.0.smithi046.stdout:8/23: symlink d1/d6/l8 0 2024-09-10T04:05:03.753 INFO:tasks.workunit.client.0.smithi046.stdout:7/24: dread f0 [0,4194304] 0 2024-09-10T04:05:03.754 INFO:tasks.workunit.client.0.smithi046.stdout:8/24: read d1/f3 [3782317,10684] 0 2024-09-10T04:05:03.756 INFO:tasks.workunit.client.0.smithi046.stdout:8/25: dread - d1/f7 zero size 2024-09-10T04:05:03.756 INFO:tasks.workunit.client.0.smithi046.stdout:2/36: rename l1 to d2/d8/lb 0 2024-09-10T04:05:03.759 INFO:tasks.workunit.client.0.smithi046.stdout:2/37: dread d2/f3 [0,4194304] 0 2024-09-10T04:05:03.767 INFO:tasks.workunit.client.0.smithi046.stdout:2/38: unlink d2/d8/lb 0 2024-09-10T04:05:03.780 INFO:tasks.workunit.client.0.smithi046.stdout:8/26: dwrite d1/f7 [0,4194304] 0 2024-09-10T04:05:03.781 INFO:tasks.workunit.client.0.smithi046.stdout:8/27: read d1/f7 [412461,17244] 0 2024-09-10T04:05:03.783 INFO:tasks.workunit.client.0.smithi046.stdout:8/28: write d1/f7 [1963853,16676] 0 2024-09-10T04:05:03.783 INFO:tasks.workunit.client.0.smithi046.stdout:7/25: dwrite f3 [0,4194304] 0 2024-09-10T04:05:03.861 INFO:tasks.workunit.client.0.smithi046.stdout:2/39: mknod d2/d6/cc 0 2024-09-10T04:05:03.864 INFO:tasks.workunit.client.0.smithi046.stdout:8/29: unlink d1/f3 0 2024-09-10T04:05:03.865 INFO:tasks.workunit.client.0.smithi046.stdout:8/30: chown d1/d6/l8 1891043 1 2024-09-10T04:05:03.865 INFO:tasks.workunit.client.0.smithi046.stdout:7/26: mkdir d5/d6/db 0 2024-09-10T04:05:03.866 INFO:tasks.workunit.client.0.smithi046.stdout:2/40: mknod d2/d8/cd 0 2024-09-10T04:05:03.868 INFO:tasks.workunit.client.0.smithi046.stdout:8/31: symlink d1/l9 0 2024-09-10T04:05:03.869 INFO:tasks.workunit.client.0.smithi046.stdout:2/41: symlink d2/le 0 2024-09-10T04:05:03.871 INFO:tasks.workunit.client.0.smithi046.stdout:8/32: creat d1/fa x:0 0 0 2024-09-10T04:05:03.872 INFO:tasks.workunit.client.0.smithi046.stdout:2/42: symlink d2/lf 0 2024-09-10T04:05:03.873 INFO:tasks.workunit.client.0.smithi046.stdout:2/43: chown d2/lf 0 1 2024-09-10T04:05:03.874 INFO:tasks.workunit.client.0.smithi046.stdout:8/33: creat d1/d6/fb x:0 0 0 2024-09-10T04:05:03.877 INFO:tasks.workunit.client.0.smithi046.stdout:2/44: creat d2/f10 x:0 0 0 2024-09-10T04:05:03.877 INFO:tasks.workunit.client.0.smithi046.stdout:7/27: dwrite f4 [0,4194304] 0 2024-09-10T04:05:03.880 INFO:tasks.workunit.client.0.smithi046.stdout:2/45: chown d2/d8/ca 1 1 2024-09-10T04:05:03.881 INFO:tasks.workunit.client.0.smithi046.stdout:7/28: creat d5/fc x:0 0 0 2024-09-10T04:05:03.884 INFO:tasks.workunit.client.0.smithi046.stdout:7/29: chown d5/d6/ca 1939 1 2024-09-10T04:05:03.885 INFO:tasks.workunit.client.0.smithi046.stdout:2/46: unlink d2/d6/cc 0 2024-09-10T04:05:03.888 INFO:tasks.workunit.client.0.smithi046.stdout:8/34: dwrite d1/d6/fb [0,4194304] 0 2024-09-10T04:05:03.888 INFO:tasks.workunit.client.0.smithi046.stdout:2/47: truncate d2/d6/f7 544470 0 2024-09-10T04:05:03.889 INFO:tasks.workunit.client.0.smithi046.stdout:2/48: chown d2/d6/f7 0 1 2024-09-10T04:05:03.891 INFO:tasks.workunit.client.0.smithi046.stdout:8/35: mknod d1/cc 0 2024-09-10T04:05:03.893 INFO:tasks.workunit.client.0.smithi046.stdout:2/49: rename d2/lf to d2/d8/l11 0 2024-09-10T04:05:03.893 INFO:tasks.workunit.client.0.smithi046.stdout:2/50: dread - d2/f10 zero size 2024-09-10T04:05:03.894 INFO:tasks.workunit.client.0.smithi046.stdout:2/51: read - d2/f10 zero size 2024-09-10T04:05:03.895 INFO:tasks.workunit.client.0.smithi046.stdout:8/36: write d1/f7 [4645357,64404] 0 2024-09-10T04:05:03.896 INFO:tasks.workunit.client.0.smithi046.stdout:8/37: chown d1/d6 1423546 1 2024-09-10T04:05:03.897 INFO:tasks.workunit.client.0.smithi046.stdout:8/38: stat d1/c4 0 2024-09-10T04:05:03.898 INFO:tasks.workunit.client.0.smithi046.stdout:8/39: write d1/d6/fb [3487989,28257] 0 2024-09-10T04:05:03.898 INFO:tasks.workunit.client.0.smithi046.stdout:2/52: creat d2/d8/f12 x:0 0 0 2024-09-10T04:05:03.900 INFO:tasks.workunit.client.0.smithi046.stdout:2/53: rmdir d2/d8 39 2024-09-10T04:05:03.905 INFO:tasks.workunit.client.0.smithi046.stdout:2/54: creat d2/d8/f13 x:0 0 0 2024-09-10T04:05:03.907 INFO:tasks.workunit.client.0.smithi046.stdout:2/55: dread - d2/d8/f12 zero size 2024-09-10T04:05:03.908 INFO:tasks.workunit.client.0.smithi046.stdout:2/56: stat d2/d8 0 2024-09-10T04:05:03.909 INFO:tasks.workunit.client.0.smithi046.stdout:8/40: dwrite d1/fa [0,4194304] 0 2024-09-10T04:05:03.910 INFO:tasks.workunit.client.0.smithi046.stdout:2/57: write d2/d8/f13 [854676,91212] 0 2024-09-10T04:05:03.921 INFO:tasks.workunit.client.0.smithi046.stdout:8/41: dread d1/fa [0,4194304] 0 2024-09-10T04:05:03.923 INFO:tasks.workunit.client.0.smithi046.stdout:2/58: dwrite d2/f10 [0,4194304] 0 2024-09-10T04:05:04.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:03 smithi070 ceph-mon[35495]: pgmap v255: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 279 op/s 2024-09-10T04:05:04.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:03 smithi046 ceph-mon[28328]: pgmap v255: 65 pgs: 65 active+clean; 273 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 279 op/s 2024-09-10T04:05:04.115 INFO:tasks.workunit.client.0.smithi046.stdout:2/59: write d2/d8/f13 [779835,40252] 0 2024-09-10T04:05:04.117 INFO:tasks.workunit.client.0.smithi046.stdout:2/60: fsync d2/d6/f7 0 2024-09-10T04:05:04.121 INFO:tasks.workunit.client.0.smithi046.stdout:8/42: dwrite d1/f7 [4194304,4194304] 0 2024-09-10T04:05:04.122 INFO:tasks.workunit.client.0.smithi046.stdout:8/43: chown d1/d6 231 1 2024-09-10T04:05:04.124 INFO:tasks.workunit.client.0.smithi046.stdout:7/30: truncate d5/f7 34213 0 2024-09-10T04:05:04.125 INFO:tasks.workunit.client.0.smithi046.stdout:7/31: readlink - no filename 2024-09-10T04:05:04.125 INFO:tasks.workunit.client.0.smithi046.stdout:8/44: creat d1/fd x:0 0 0 2024-09-10T04:05:04.127 INFO:tasks.workunit.client.0.smithi046.stdout:2/61: unlink d2/d8/f13 0 2024-09-10T04:05:04.128 INFO:tasks.workunit.client.0.smithi046.stdout:7/32: fdatasync f2 0 2024-09-10T04:05:04.129 INFO:tasks.workunit.client.0.smithi046.stdout:7/33: write d5/fc [881322,104279] 0 2024-09-10T04:05:04.130 INFO:tasks.workunit.client.0.smithi046.stdout:8/45: rename d1/f7 to d1/d6/fe 0 2024-09-10T04:05:04.131 INFO:tasks.workunit.client.0.smithi046.stdout:2/62: symlink d2/d6/l14 0 2024-09-10T04:05:04.135 INFO:tasks.workunit.client.0.smithi046.stdout:8/46: creat d1/d6/ff x:0 0 0 2024-09-10T04:05:04.137 INFO:tasks.workunit.client.0.smithi046.stdout:2/63: mkdir d2/d6/d15 0 2024-09-10T04:05:04.138 INFO:tasks.workunit.client.0.smithi046.stdout:7/34: mkdir d5/d6/db/dd 0 2024-09-10T04:05:04.139 INFO:tasks.workunit.client.0.smithi046.stdout:7/35: write d5/fc [984818,56750] 0 2024-09-10T04:05:04.141 INFO:tasks.workunit.client.0.smithi046.stdout:8/47: unlink d1/d6/fb 0 2024-09-10T04:05:04.142 INFO:tasks.workunit.client.0.smithi046.stdout:2/64: chown d2/d8/l11 179597629 1 2024-09-10T04:05:04.142 INFO:tasks.workunit.client.0.smithi046.stdout:7/36: read d5/fc [831695,95999] 0 2024-09-10T04:05:04.148 INFO:tasks.workunit.client.0.smithi046.stdout:7/37: unlink d5/d6/ca 0 2024-09-10T04:05:04.148 INFO:tasks.workunit.client.0.smithi046.stdout:8/48: dwrite d1/fd [0,4194304] 0 2024-09-10T04:05:04.151 INFO:tasks.workunit.client.0.smithi046.stdout:2/65: mkdir d2/d6/d15/d16 0 2024-09-10T04:05:04.155 INFO:tasks.workunit.client.0.smithi046.stdout:7/38: creat d5/d9/fe x:0 0 0 2024-09-10T04:05:04.157 INFO:tasks.workunit.client.0.smithi046.stdout:7/39: write d5/d9/fe [280071,34617] 0 2024-09-10T04:05:04.160 INFO:tasks.workunit.client.0.smithi046.stdout:7/40: rename d5/d6/db to d5/d6/db/dd/df 22 2024-09-10T04:05:04.161 INFO:tasks.workunit.client.0.smithi046.stdout:2/66: dwrite d2/d6/f7 [0,4194304] 0 2024-09-10T04:05:04.162 INFO:tasks.workunit.client.0.smithi046.stdout:2/67: stat d2/le 0 2024-09-10T04:05:04.163 INFO:tasks.workunit.client.0.smithi046.stdout:7/41: symlink d5/d9/l10 0 2024-09-10T04:05:04.164 INFO:tasks.workunit.client.0.smithi046.stdout:7/42: readlink d5/d9/l10 0 2024-09-10T04:05:04.165 INFO:tasks.workunit.client.0.smithi046.stdout:7/43: chown d5/fc 3104846 1 2024-09-10T04:05:04.166 INFO:tasks.workunit.client.0.smithi046.stdout:2/68: symlink d2/l17 0 2024-09-10T04:05:04.166 INFO:tasks.workunit.client.0.smithi046.stdout:7/44: chown d5/d9/fe 134 1 2024-09-10T04:05:04.167 INFO:tasks.workunit.client.0.smithi046.stdout:7/45: fdatasync d5/d9/fe 0 2024-09-10T04:05:04.169 INFO:tasks.workunit.client.0.smithi046.stdout:2/69: unlink d2/d6/f7 0 2024-09-10T04:05:04.173 INFO:tasks.workunit.client.0.smithi046.stdout:8/49: truncate d1/fa 1014240 0 2024-09-10T04:05:04.174 INFO:tasks.workunit.client.0.smithi046.stdout:2/70: creat d2/d6/f18 x:0 0 0 2024-09-10T04:05:04.175 INFO:tasks.workunit.client.0.smithi046.stdout:8/50: symlink d1/d6/l10 0 2024-09-10T04:05:04.176 INFO:tasks.workunit.client.0.smithi046.stdout:2/71: rename d2/l17 to d2/d6/d15/l19 0 2024-09-10T04:05:04.178 INFO:tasks.workunit.client.0.smithi046.stdout:2/72: truncate d2/d6/f18 862655 0 2024-09-10T04:05:04.179 INFO:tasks.workunit.client.0.smithi046.stdout:8/51: rename d1/d6/l10 to d1/l11 0 2024-09-10T04:05:04.180 INFO:tasks.workunit.client.0.smithi046.stdout:7/46: write d5/fc [576517,21857] 0 2024-09-10T04:05:04.180 INFO:tasks.workunit.client.0.smithi046.stdout:8/52: rmdir d1 39 2024-09-10T04:05:04.182 INFO:tasks.workunit.client.0.smithi046.stdout:2/73: rename d2/f10 to d2/d6/f1a 0 2024-09-10T04:05:04.184 INFO:tasks.workunit.client.0.smithi046.stdout:7/47: mkdir d5/d6/db/d11 0 2024-09-10T04:05:04.185 INFO:tasks.workunit.client.0.smithi046.stdout:8/53: mkdir d1/d6/d12 0 2024-09-10T04:05:04.189 INFO:tasks.workunit.client.0.smithi046.stdout:7/48: dread f4 [0,4194304] 0 2024-09-10T04:05:04.192 INFO:tasks.workunit.client.0.smithi046.stdout:7/49: dread f2 [0,4194304] 0 2024-09-10T04:05:04.194 INFO:tasks.workunit.client.0.smithi046.stdout:2/74: link d2/d8/l11 d2/d6/d15/d16/l1b 0 2024-09-10T04:05:04.196 INFO:tasks.workunit.client.0.smithi046.stdout:8/54: rename d1/l11 to d1/l13 0 2024-09-10T04:05:04.199 INFO:tasks.workunit.client.0.smithi046.stdout:8/55: mkdir d1/d14 0 2024-09-10T04:05:04.200 INFO:tasks.workunit.client.0.smithi046.stdout:2/75: unlink d2/d6/d15/d16/l1b 0 2024-09-10T04:05:04.203 INFO:tasks.workunit.client.0.smithi046.stdout:8/56: rename d1/d14 to d1/d6/d12/d15 0 2024-09-10T04:05:04.203 INFO:tasks.workunit.client.0.smithi046.stdout:2/76: write d2/d6/f1a [1854187,60240] 0 2024-09-10T04:05:04.205 INFO:tasks.workunit.client.0.smithi046.stdout:8/57: getdents d1/d6/d12/d15 0 2024-09-10T04:05:04.207 INFO:tasks.workunit.client.0.smithi046.stdout:8/58: creat d1/d6/d12/f16 x:0 0 0 2024-09-10T04:05:04.211 INFO:tasks.workunit.client.0.smithi046.stdout:2/77: dwrite d2/d8/f12 [0,4194304] 0 2024-09-10T04:05:04.215 INFO:tasks.workunit.client.0.smithi046.stdout:8/59: truncate d1/fa 1761948 0 2024-09-10T04:05:04.217 INFO:tasks.workunit.client.0.smithi046.stdout:2/78: rename d2/d8/ca to d2/c1c 0 2024-09-10T04:05:04.218 INFO:tasks.workunit.client.0.smithi046.stdout:8/60: mkdir d1/d6/d17 0 2024-09-10T04:05:04.219 INFO:tasks.workunit.client.0.smithi046.stdout:2/79: symlink d2/d6/d15/l1d 0 2024-09-10T04:05:04.226 INFO:tasks.workunit.client.0.smithi046.stdout:8/61: dwrite d1/d6/ff [0,4194304] 0 2024-09-10T04:05:04.226 INFO:tasks.workunit.client.0.smithi046.stdout:2/80: getdents d2/d8 0 2024-09-10T04:05:04.228 INFO:tasks.workunit.client.0.smithi046.stdout:8/62: write d1/d6/d12/f16 [493685,49] 0 2024-09-10T04:05:04.230 INFO:tasks.workunit.client.0.smithi046.stdout:2/81: mknod d2/c1e 0 2024-09-10T04:05:04.232 INFO:tasks.workunit.client.0.smithi046.stdout:2/82: symlink d2/d6/d15/l1f 0 2024-09-10T04:05:04.234 INFO:tasks.workunit.client.0.smithi046.stdout:2/83: fsync d2/f3 0 2024-09-10T04:05:04.234 INFO:tasks.workunit.client.0.smithi046.stdout:8/63: dread d1/fa [0,4194304] 0 2024-09-10T04:05:04.237 INFO:tasks.workunit.client.0.smithi046.stdout:8/64: rename d1/cc to d1/d6/d17/c18 0 2024-09-10T04:05:04.240 INFO:tasks.workunit.client.0.smithi046.stdout:8/65: chown d1/d6/fe 64549 1 2024-09-10T04:05:04.241 INFO:tasks.workunit.client.0.smithi046.stdout:8/66: fsync d1/fa 0 2024-09-10T04:05:04.241 INFO:tasks.workunit.client.0.smithi046.stdout:2/84: dwrite d2/d8/f12 [0,4194304] 0 2024-09-10T04:05:04.243 INFO:tasks.workunit.client.0.smithi046.stdout:8/67: mkdir d1/d6/d19 0 2024-09-10T04:05:04.245 INFO:tasks.workunit.client.0.smithi046.stdout:8/68: stat d1/d6/fe 0 2024-09-10T04:05:04.250 INFO:tasks.workunit.client.0.smithi046.stdout:8/69: getdents d1 0 2024-09-10T04:05:04.250 INFO:tasks.workunit.client.0.smithi046.stdout:2/85: dwrite d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:04.253 INFO:tasks.workunit.client.0.smithi046.stdout:2/86: symlink d2/l20 0 2024-09-10T04:05:04.253 INFO:tasks.workunit.client.0.smithi046.stdout:8/70: mknod d1/d6/d19/c1a 0 2024-09-10T04:05:04.254 INFO:tasks.workunit.client.0.smithi046.stdout:2/87: symlink d2/d8/l21 0 2024-09-10T04:05:04.255 INFO:tasks.workunit.client.0.smithi046.stdout:8/71: creat d1/d6/f1b x:0 0 0 2024-09-10T04:05:04.259 INFO:tasks.workunit.client.0.smithi046.stdout:2/88: getdents d2/d6/d15/d16 0 2024-09-10T04:05:04.261 INFO:tasks.workunit.client.0.smithi046.stdout:2/89: fdatasync d2/d6/f18 0 2024-09-10T04:05:04.263 INFO:tasks.workunit.client.0.smithi046.stdout:2/90: write d2/f3 [438839,35530] 0 2024-09-10T04:05:04.263 INFO:tasks.workunit.client.0.smithi046.stdout:8/72: dwrite d1/d6/ff [0,4194304] 0 2024-09-10T04:05:04.264 INFO:tasks.workunit.client.0.smithi046.stdout:2/91: chown d2/d6/l14 743983300 1 2024-09-10T04:05:04.267 INFO:tasks.workunit.client.0.smithi046.stdout:2/92: symlink d2/d6/l22 0 2024-09-10T04:05:04.273 INFO:tasks.workunit.client.0.smithi046.stdout:2/93: dwrite d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:04.275 INFO:tasks.workunit.client.0.smithi046.stdout:2/94: mkdir d2/d6/d23 0 2024-09-10T04:05:04.282 INFO:tasks.workunit.client.0.smithi046.stdout:2/95: dwrite d2/f3 [0,4194304] 0 2024-09-10T04:05:04.283 INFO:tasks.workunit.client.0.smithi046.stdout:2/96: write d2/d8/f12 [4942174,9896] 0 2024-09-10T04:05:04.284 INFO:tasks.workunit.client.0.smithi046.stdout:2/97: write d2/f3 [3995846,7909] 0 2024-09-10T04:05:04.285 INFO:tasks.workunit.client.0.smithi046.stdout:2/98: write d2/f3 [3100558,117845] 0 2024-09-10T04:05:04.287 INFO:tasks.workunit.client.0.smithi046.stdout:2/99: truncate d2/d6/f18 1161352 0 2024-09-10T04:05:04.290 INFO:tasks.workunit.client.0.smithi046.stdout:2/100: rename d2/d6/d15/l1d to d2/d6/d23/l24 0 2024-09-10T04:05:04.294 INFO:tasks.workunit.client.0.smithi046.stdout:2/101: dread d2/d8/f12 [0,4194304] 0 2024-09-10T04:05:04.296 INFO:tasks.workunit.client.0.smithi046.stdout:2/102: getdents d2/d6/d15/d16 0 2024-09-10T04:05:04.629 INFO:tasks.workunit.client.0.smithi046.stdout:8/73: sync 2024-09-10T04:05:04.629 INFO:tasks.workunit.client.0.smithi046.stdout:0/7: sync 2024-09-10T04:05:04.629 INFO:tasks.workunit.client.0.smithi046.stdout:6/1: sync 2024-09-10T04:05:04.629 INFO:tasks.workunit.client.0.smithi046.stdout:6/2: write - no filename 2024-09-10T04:05:04.629 INFO:tasks.workunit.client.0.smithi046.stdout:6/3: rename - no filename 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:7/50: sync 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:5/14: sync 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:3/0: sync 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:3/1: dwrite - no filename 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:4/13: sync 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:9/4: sync 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:1/7: sync 2024-09-10T04:05:04.630 INFO:tasks.workunit.client.0.smithi046.stdout:2/103: sync 2024-09-10T04:05:04.631 INFO:tasks.workunit.client.0.smithi046.stdout:4/14: fsync f1 0 2024-09-10T04:05:04.634 INFO:tasks.workunit.client.0.smithi046.stdout:1/8: write f0 [98137,1282] 0 2024-09-10T04:05:04.635 INFO:tasks.workunit.client.0.smithi046.stdout:5/15: creat f4 x:0 0 0 2024-09-10T04:05:04.636 INFO:tasks.workunit.client.0.smithi046.stdout:8/74: dread d1/d6/d12/f16 [0,4194304] 0 2024-09-10T04:05:04.636 INFO:tasks.workunit.client.0.smithi046.stdout:5/16: chown f3 0 1 2024-09-10T04:05:04.636 INFO:tasks.workunit.client.0.smithi046.stdout:7/51: mkdir d5/d9/d12 0 2024-09-10T04:05:04.637 INFO:tasks.workunit.client.0.smithi046.stdout:5/17: truncate f0 259414 0 2024-09-10T04:05:04.637 INFO:tasks.workunit.client.0.smithi046.stdout:0/8: creat f0 x:0 0 0 2024-09-10T04:05:04.637 INFO:tasks.workunit.client.0.smithi046.stdout:0/9: rmdir - no directory 2024-09-10T04:05:04.638 INFO:tasks.workunit.client.0.smithi046.stdout:3/2: creat f0 x:0 0 0 2024-09-10T04:05:04.639 INFO:tasks.workunit.client.0.smithi046.stdout:0/10: write f0 [682384,68630] 0 2024-09-10T04:05:04.641 INFO:tasks.workunit.client.0.smithi046.stdout:0/11: fdatasync f0 0 2024-09-10T04:05:04.647 INFO:tasks.workunit.client.0.smithi046.stdout:3/3: dwrite f0 [0,4194304] 0 2024-09-10T04:05:04.657 INFO:tasks.workunit.client.0.smithi046.stdout:6/4: getdents . 0 2024-09-10T04:05:04.659 INFO:tasks.workunit.client.0.smithi046.stdout:2/104: creat d2/f25 x:0 0 0 2024-09-10T04:05:04.660 INFO:tasks.workunit.client.0.smithi046.stdout:8/75: symlink d1/l1c 0 2024-09-10T04:05:04.661 INFO:tasks.workunit.client.0.smithi046.stdout:7/52: mknod d5/c13 0 2024-09-10T04:05:04.662 INFO:tasks.workunit.client.0.smithi046.stdout:5/18: symlink l5 0 2024-09-10T04:05:04.667 INFO:tasks.workunit.client.0.smithi046.stdout:0/12: mkdir d1 0 2024-09-10T04:05:04.668 INFO:tasks.workunit.client.0.smithi046.stdout:5/19: dread f1 [0,4194304] 0 2024-09-10T04:05:04.672 INFO:tasks.workunit.client.0.smithi046.stdout:3/4: creat f1 x:0 0 0 2024-09-10T04:05:04.672 INFO:tasks.workunit.client.0.smithi046.stdout:3/5: rmdir - no directory 2024-09-10T04:05:04.674 INFO:tasks.workunit.client.0.smithi046.stdout:4/15: creat d3/f4 x:0 0 0 2024-09-10T04:05:04.676 INFO:tasks.workunit.client.0.smithi046.stdout:1/9: link f1 f2 0 2024-09-10T04:05:04.677 INFO:tasks.workunit.client.0.smithi046.stdout:1/10: dread - f1 zero size 2024-09-10T04:05:04.678 INFO:tasks.workunit.client.0.smithi046.stdout:9/5: getdents . 0 2024-09-10T04:05:04.678 INFO:tasks.workunit.client.0.smithi046.stdout:9/6: chown l0 10242 1 2024-09-10T04:05:04.679 INFO:tasks.workunit.client.0.smithi046.stdout:2/105: mknod d2/d6/d15/c26 0 2024-09-10T04:05:04.682 INFO:tasks.workunit.client.0.smithi046.stdout:5/20: write f3 [245014,80528] 0 2024-09-10T04:05:04.683 INFO:tasks.workunit.client.0.smithi046.stdout:3/6: rename f1 to f2 0 2024-09-10T04:05:04.684 INFO:tasks.workunit.client.0.smithi046.stdout:4/16: symlink d3/l5 0 2024-09-10T04:05:04.684 INFO:tasks.workunit.client.0.smithi046.stdout:4/17: readlink d3/l5 0 2024-09-10T04:05:04.687 INFO:tasks.workunit.client.0.smithi046.stdout:5/21: dread f3 [0,4194304] 0 2024-09-10T04:05:04.688 INFO:tasks.workunit.client.0.smithi046.stdout:6/5: creat f0 x:0 0 0 2024-09-10T04:05:04.689 INFO:tasks.workunit.client.0.smithi046.stdout:1/11: mkdir d3 0 2024-09-10T04:05:04.690 INFO:tasks.workunit.client.0.smithi046.stdout:1/12: chown f1 83 1 2024-09-10T04:05:04.691 INFO:tasks.workunit.client.0.smithi046.stdout:9/7: symlink l1 0 2024-09-10T04:05:04.692 INFO:tasks.workunit.client.0.smithi046.stdout:7/53: creat d5/d6/db/d11/f14 x:0 0 0 2024-09-10T04:05:04.694 INFO:tasks.workunit.client.0.smithi046.stdout:2/106: creat d2/d8/f27 x:0 0 0 2024-09-10T04:05:04.695 INFO:tasks.workunit.client.0.smithi046.stdout:0/13: mkdir d1/d2 0 2024-09-10T04:05:04.696 INFO:tasks.workunit.client.0.smithi046.stdout:0/14: chown f0 59034003 1 2024-09-10T04:05:04.700 INFO:tasks.workunit.client.0.smithi046.stdout:4/18: mknod d3/c6 0 2024-09-10T04:05:04.700 INFO:tasks.workunit.client.0.smithi046.stdout:0/15: dread f0 [0,4194304] 0 2024-09-10T04:05:04.702 INFO:tasks.workunit.client.0.smithi046.stdout:5/22: rename f0 to f6 0 2024-09-10T04:05:04.704 INFO:tasks.workunit.client.0.smithi046.stdout:6/6: creat f1 x:0 0 0 2024-09-10T04:05:04.706 INFO:tasks.workunit.client.0.smithi046.stdout:9/8: rename l0 to l2 0 2024-09-10T04:05:04.708 INFO:tasks.workunit.client.0.smithi046.stdout:2/107: rename d2/d6/d15/l1f to d2/d6/d15/l28 0 2024-09-10T04:05:04.711 INFO:tasks.workunit.client.0.smithi046.stdout:3/7: link f2 f3 0 2024-09-10T04:05:04.714 INFO:tasks.workunit.client.0.smithi046.stdout:5/23: mkdir d7 0 2024-09-10T04:05:04.715 INFO:tasks.workunit.client.0.smithi046.stdout:1/13: getdents d3 0 2024-09-10T04:05:04.716 INFO:tasks.workunit.client.0.smithi046.stdout:6/7: symlink l2 0 2024-09-10T04:05:04.716 INFO:tasks.workunit.client.0.smithi046.stdout:1/14: write f0 [580360,27983] 0 2024-09-10T04:05:04.717 INFO:tasks.workunit.client.0.smithi046.stdout:6/8: dread - f1 zero size 2024-09-10T04:05:04.719 INFO:tasks.workunit.client.0.smithi046.stdout:2/108: creat d2/d8/f29 x:0 0 0 2024-09-10T04:05:04.720 INFO:tasks.workunit.client.0.smithi046.stdout:3/8: dwrite f0 [0,4194304] 0 2024-09-10T04:05:04.720 INFO:tasks.workunit.client.0.smithi046.stdout:6/9: write f1 [1025002,102099] 0 2024-09-10T04:05:04.720 INFO:tasks.workunit.client.0.smithi046.stdout:3/9: dread - f3 zero size 2024-09-10T04:05:04.722 INFO:tasks.workunit.client.0.smithi046.stdout:3/10: chown f0 375116 1 2024-09-10T04:05:04.723 INFO:tasks.workunit.client.0.smithi046.stdout:4/19: truncate f1 2272891 0 2024-09-10T04:05:04.724 INFO:tasks.workunit.client.0.smithi046.stdout:4/20: truncate d3/f4 857842 0 2024-09-10T04:05:04.725 INFO:tasks.workunit.client.0.smithi046.stdout:0/16: mkdir d1/d2/d3 0 2024-09-10T04:05:04.728 INFO:tasks.workunit.client.0.smithi046.stdout:5/24: dread f3 [0,4194304] 0 2024-09-10T04:05:04.729 INFO:tasks.workunit.client.0.smithi046.stdout:2/109: symlink d2/d6/l2a 0 2024-09-10T04:05:04.729 INFO:tasks.workunit.client.0.smithi046.stdout:5/25: chown f6 15491 1 2024-09-10T04:05:04.730 INFO:tasks.workunit.client.0.smithi046.stdout:6/10: creat f3 x:0 0 0 2024-09-10T04:05:04.732 INFO:tasks.workunit.client.0.smithi046.stdout:3/11: creat f4 x:0 0 0 2024-09-10T04:05:04.733 INFO:tasks.workunit.client.0.smithi046.stdout:3/12: fsync f0 0 2024-09-10T04:05:04.733 INFO:tasks.workunit.client.0.smithi046.stdout:4/21: symlink d3/l7 0 2024-09-10T04:05:04.735 INFO:tasks.workunit.client.0.smithi046.stdout:0/17: dwrite f0 [0,4194304] 0 2024-09-10T04:05:04.735 INFO:tasks.workunit.client.0.smithi046.stdout:1/15: link f0 d3/f4 0 2024-09-10T04:05:04.736 INFO:tasks.workunit.client.0.smithi046.stdout:4/22: write d3/f4 [1362490,66078] 0 2024-09-10T04:05:04.737 INFO:tasks.workunit.client.0.smithi046.stdout:2/110: rmdir d2/d6 39 2024-09-10T04:05:04.741 INFO:tasks.workunit.client.0.smithi046.stdout:6/11: mknod c4 0 2024-09-10T04:05:04.748 INFO:tasks.workunit.client.0.smithi046.stdout:4/23: creat d3/f8 x:0 0 0 2024-09-10T04:05:04.754 INFO:tasks.workunit.client.0.smithi046.stdout:9/9: rename l2 to l3 0 2024-09-10T04:05:04.755 INFO:tasks.workunit.client.0.smithi046.stdout:9/10: dwrite - no filename 2024-09-10T04:05:04.756 INFO:tasks.workunit.client.0.smithi046.stdout:3/13: dwrite f4 [0,4194304] 0 2024-09-10T04:05:04.763 INFO:tasks.workunit.client.0.smithi046.stdout:0/18: dwrite f0 [0,4194304] 0 2024-09-10T04:05:04.764 INFO:tasks.workunit.client.0.smithi046.stdout:5/26: dwrite f6 [0,4194304] 0 2024-09-10T04:05:04.764 INFO:tasks.workunit.client.0.smithi046.stdout:2/111: symlink d2/d6/d23/l2b 0 2024-09-10T04:05:04.768 INFO:tasks.workunit.client.0.smithi046.stdout:9/11: mkdir d4 0 2024-09-10T04:05:04.768 INFO:tasks.workunit.client.0.smithi046.stdout:9/12: dwrite - no filename 2024-09-10T04:05:04.769 INFO:tasks.workunit.client.0.smithi046.stdout:5/27: chown l5 39057337 1 2024-09-10T04:05:04.769 INFO:tasks.workunit.client.0.smithi046.stdout:7/54: dwrite f2 [0,4194304] 0 2024-09-10T04:05:04.774 INFO:tasks.workunit.client.0.smithi046.stdout:3/14: rename f2 to f5 0 2024-09-10T04:05:04.778 INFO:tasks.workunit.client.0.smithi046.stdout:0/19: mkdir d1/d2/d4 0 2024-09-10T04:05:04.778 INFO:tasks.workunit.client.0.smithi046.stdout:3/15: chown f3 289 1 2024-09-10T04:05:04.778 INFO:tasks.workunit.client.0.smithi046.stdout:4/24: dwrite d3/f8 [0,4194304] 0 2024-09-10T04:05:04.796 INFO:tasks.workunit.client.0.smithi046.stdout:1/16: getdents d3 0 2024-09-10T04:05:04.797 INFO:tasks.workunit.client.0.smithi046.stdout:5/28: rename f4 to d7/f8 0 2024-09-10T04:05:04.798 INFO:tasks.workunit.client.0.smithi046.stdout:6/12: getdents . 0 2024-09-10T04:05:04.798 INFO:tasks.workunit.client.0.smithi046.stdout:4/25: read d3/f4 [641911,127067] 0 2024-09-10T04:05:04.798 INFO:tasks.workunit.client.0.smithi046.stdout:7/55: dread f3 [0,4194304] 0 2024-09-10T04:05:04.799 INFO:tasks.workunit.client.0.smithi046.stdout:3/16: creat f6 x:0 0 0 2024-09-10T04:05:04.800 INFO:tasks.workunit.client.0.smithi046.stdout:0/20: unlink f0 0 2024-09-10T04:05:04.800 INFO:tasks.workunit.client.0.smithi046.stdout:0/21: dwrite - no filename 2024-09-10T04:05:04.800 INFO:tasks.workunit.client.0.smithi046.stdout:7/56: read d5/f7 [2757,24830] 0 2024-09-10T04:05:04.801 INFO:tasks.workunit.client.0.smithi046.stdout:7/57: write f2 [5161212,114961] 0 2024-09-10T04:05:04.815 INFO:tasks.workunit.client.0.smithi046.stdout:8/76: sync 2024-09-10T04:05:04.816 INFO:tasks.workunit.client.0.smithi046.stdout:8/77: write d1/d6/ff [333788,100732] 0 2024-09-10T04:05:04.911 INFO:tasks.workunit.client.0.smithi046.stdout:9/13: creat d4/f5 x:0 0 0 2024-09-10T04:05:04.911 INFO:tasks.workunit.client.0.smithi046.stdout:9/14: dread - d4/f5 zero size 2024-09-10T04:05:04.914 INFO:tasks.workunit.client.0.smithi046.stdout:2/112: creat d2/f2c x:0 0 0 2024-09-10T04:05:04.915 INFO:tasks.workunit.client.0.smithi046.stdout:5/29: rename f3 to d7/f9 0 2024-09-10T04:05:04.916 INFO:tasks.workunit.client.0.smithi046.stdout:6/13: symlink l5 0 2024-09-10T04:05:04.916 INFO:tasks.workunit.client.0.smithi046.stdout:6/14: dread - f3 zero size 2024-09-10T04:05:04.917 INFO:tasks.workunit.client.0.smithi046.stdout:4/26: creat d3/f9 x:0 0 0 2024-09-10T04:05:04.919 INFO:tasks.workunit.client.0.smithi046.stdout:3/17: creat f7 x:0 0 0 2024-09-10T04:05:04.921 INFO:tasks.workunit.client.0.smithi046.stdout:7/58: fsync f4 0 2024-09-10T04:05:04.922 INFO:tasks.workunit.client.0.smithi046.stdout:3/18: dread f4 [0,4194304] 0 2024-09-10T04:05:04.923 INFO:tasks.workunit.client.0.smithi046.stdout:8/78: symlink d1/d6/l1d 0 2024-09-10T04:05:04.923 INFO:tasks.workunit.client.0.smithi046.stdout:3/19: stat f3 0 2024-09-10T04:05:04.923 INFO:tasks.workunit.client.0.smithi046.stdout:3/20: fsync f6 0 2024-09-10T04:05:04.924 INFO:tasks.workunit.client.0.smithi046.stdout:3/21: fsync f6 0 2024-09-10T04:05:04.924 INFO:tasks.workunit.client.0.smithi046.stdout:8/79: write d1/fd [3944927,35400] 0 2024-09-10T04:05:04.925 INFO:tasks.workunit.client.0.smithi046.stdout:2/113: rename d2/d6/l2a to d2/d6/d15/l2d 0 2024-09-10T04:05:04.926 INFO:tasks.workunit.client.0.smithi046.stdout:2/114: dread - d2/d8/f29 zero size 2024-09-10T04:05:04.927 INFO:tasks.workunit.client.0.smithi046.stdout:6/15: creat f6 x:0 0 0 2024-09-10T04:05:04.930 INFO:tasks.workunit.client.0.smithi046.stdout:0/22: mkdir d1/d2/d3/d5 0 2024-09-10T04:05:04.930 INFO:tasks.workunit.client.0.smithi046.stdout:0/23: readlink - no filename 2024-09-10T04:05:04.931 INFO:tasks.workunit.client.0.smithi046.stdout:7/59: mknod d5/d6/db/d11/c15 0 2024-09-10T04:05:04.932 INFO:tasks.workunit.client.0.smithi046.stdout:7/60: stat f2 0 2024-09-10T04:05:04.932 INFO:tasks.workunit.client.0.smithi046.stdout:0/24: chown d1/d2/d3/d5 6973 1 2024-09-10T04:05:04.933 INFO:tasks.workunit.client.0.smithi046.stdout:0/25: unlink - no file 2024-09-10T04:05:04.933 INFO:tasks.workunit.client.0.smithi046.stdout:0/26: dwrite - no filename 2024-09-10T04:05:04.933 INFO:tasks.workunit.client.0.smithi046.stdout:3/22: fdatasync f3 0 2024-09-10T04:05:04.934 INFO:tasks.workunit.client.0.smithi046.stdout:8/80: read d1/d6/d12/f16 [234368,105759] 0 2024-09-10T04:05:04.939 INFO:tasks.workunit.client.0.smithi046.stdout:5/30: dwrite d7/f8 [0,4194304] 0 2024-09-10T04:05:04.943 INFO:tasks.workunit.client.0.smithi046.stdout:5/31: stat f1 0 2024-09-10T04:05:04.946 INFO:tasks.workunit.client.0.smithi046.stdout:6/16: dwrite f3 [0,4194304] 0 2024-09-10T04:05:04.954 INFO:tasks.workunit.client.0.smithi046.stdout:0/27: rmdir d1/d2/d3/d5 0 2024-09-10T04:05:04.958 INFO:tasks.workunit.client.0.smithi046.stdout:7/61: dwrite f4 [0,4194304] 0 2024-09-10T04:05:04.959 INFO:tasks.workunit.client.0.smithi046.stdout:2/115: dwrite d2/d8/f12 [4194304,4194304] 0 2024-09-10T04:05:04.960 INFO:tasks.workunit.client.0.smithi046.stdout:3/23: dwrite f3 [0,4194304] 0 2024-09-10T04:05:04.962 INFO:tasks.workunit.client.0.smithi046.stdout:2/116: chown d2/d8/cd 34 1 2024-09-10T04:05:04.962 INFO:tasks.workunit.client.0.smithi046.stdout:8/81: rmdir d1/d6/d12/d15 0 2024-09-10T04:05:04.963 INFO:tasks.workunit.client.0.smithi046.stdout:3/24: dread - f6 zero size 2024-09-10T04:05:04.967 INFO:tasks.workunit.client.0.smithi046.stdout:7/62: symlink d5/d9/l16 0 2024-09-10T04:05:04.968 INFO:tasks.workunit.client.0.smithi046.stdout:3/25: creat f8 x:0 0 0 2024-09-10T04:05:04.968 INFO:tasks.workunit.client.0.smithi046.stdout:7/63: chown d5 351966 1 2024-09-10T04:05:04.970 INFO:tasks.workunit.client.0.smithi046.stdout:2/117: creat d2/d6/d15/d16/f2e x:0 0 0 2024-09-10T04:05:04.976 INFO:tasks.workunit.client.0.smithi046.stdout:8/82: link d1/d6/d12/f16 d1/d6/d12/f1e 0 2024-09-10T04:05:04.980 INFO:tasks.workunit.client.0.smithi046.stdout:2/118: dwrite d2/d6/d15/d16/f2e [0,4194304] 0 2024-09-10T04:05:04.981 INFO:tasks.workunit.client.0.smithi046.stdout:2/119: fdatasync d2/f2c 0 2024-09-10T04:05:04.983 INFO:tasks.workunit.client.0.smithi046.stdout:2/120: mknod d2/d6/d15/d16/c2f 0 2024-09-10T04:05:04.984 INFO:tasks.workunit.client.0.smithi046.stdout:2/121: chown d2/d6/d23 15 1 2024-09-10T04:05:05.018 INFO:tasks.workunit.client.0.smithi046.stdout:9/15: fsync d4/f5 0 2024-09-10T04:05:05.020 INFO:tasks.workunit.client.0.smithi046.stdout:9/16: creat d4/f6 x:0 0 0 2024-09-10T04:05:05.021 INFO:tasks.workunit.client.0.smithi046.stdout:9/17: mkdir d4/d7 0 2024-09-10T04:05:05.025 INFO:tasks.workunit.client.0.smithi046.stdout:2/122: getdents d2/d6 0 2024-09-10T04:05:05.025 INFO:tasks.workunit.client.0.smithi046.stdout:6/17: getdents . 0 2024-09-10T04:05:05.027 INFO:tasks.workunit.client.0.smithi046.stdout:6/18: mknod c7 0 2024-09-10T04:05:05.028 INFO:tasks.workunit.client.0.smithi046.stdout:2/123: symlink d2/d8/l30 0 2024-09-10T04:05:05.033 INFO:tasks.workunit.client.0.smithi046.stdout:2/124: write d2/d8/f12 [3539612,70540] 0 2024-09-10T04:05:05.034 INFO:tasks.workunit.client.0.smithi046.stdout:9/18: dwrite d4/f5 [0,4194304] 0 2024-09-10T04:05:05.035 INFO:tasks.workunit.client.0.smithi046.stdout:6/19: dread f1 [0,4194304] 0 2024-09-10T04:05:05.036 INFO:tasks.workunit.client.0.smithi046.stdout:2/125: chown d2/c4 690 1 2024-09-10T04:05:05.037 INFO:tasks.workunit.client.0.smithi046.stdout:4/27: dwrite f1 [0,4194304] 0 2024-09-10T04:05:05.037 INFO:tasks.workunit.client.0.smithi046.stdout:9/19: truncate d4/f6 187737 0 2024-09-10T04:05:05.038 INFO:tasks.workunit.client.0.smithi046.stdout:6/20: write f1 [1979356,16646] 0 2024-09-10T04:05:05.041 INFO:tasks.workunit.client.0.smithi046.stdout:9/20: dread d4/f6 [0,4194304] 0 2024-09-10T04:05:05.042 INFO:tasks.workunit.client.0.smithi046.stdout:4/28: unlink d3/f8 0 2024-09-10T04:05:05.045 INFO:tasks.workunit.client.0.smithi046.stdout:6/21: rename f3 to f8 0 2024-09-10T04:05:05.046 INFO:tasks.workunit.client.0.smithi046.stdout:4/29: creat d3/fa x:0 0 0 2024-09-10T04:05:05.048 INFO:tasks.workunit.client.0.smithi046.stdout:6/22: creat f9 x:0 0 0 2024-09-10T04:05:05.049 INFO:tasks.workunit.client.0.smithi046.stdout:6/23: read f8 [3609171,46471] 0 2024-09-10T04:05:05.050 INFO:tasks.workunit.client.0.smithi046.stdout:4/30: creat d3/fb x:0 0 0 2024-09-10T04:05:05.050 INFO:tasks.workunit.client.0.smithi046.stdout:2/126: link d2/d6/d15/l2d d2/d8/l31 0 2024-09-10T04:05:05.050 INFO:tasks.workunit.client.0.smithi046.stdout:6/24: chown f1 1507 1 2024-09-10T04:05:05.051 INFO:tasks.workunit.client.0.smithi046.stdout:2/127: read - d2/d8/f29 zero size 2024-09-10T04:05:05.051 INFO:tasks.workunit.client.0.smithi046.stdout:6/25: mkdir da 0 2024-09-10T04:05:05.053 INFO:tasks.workunit.client.0.smithi046.stdout:4/31: unlink c0 0 2024-09-10T04:05:05.053 INFO:tasks.workunit.client.0.smithi046.stdout:2/128: symlink d2/d6/d23/l32 0 2024-09-10T04:05:05.054 INFO:tasks.workunit.client.0.smithi046.stdout:4/32: truncate d3/f4 685175 0 2024-09-10T04:05:05.054 INFO:tasks.workunit.client.0.smithi046.stdout:2/129: write d2/d6/f1a [2856940,27285] 0 2024-09-10T04:05:05.055 INFO:tasks.workunit.client.0.smithi046.stdout:6/26: mkdir da/db 0 2024-09-10T04:05:05.056 INFO:tasks.workunit.client.0.smithi046.stdout:4/33: creat d3/fc x:0 0 0 2024-09-10T04:05:05.057 INFO:tasks.workunit.client.0.smithi046.stdout:2/130: creat d2/d6/d23/f33 x:0 0 0 2024-09-10T04:05:05.058 INFO:tasks.workunit.client.0.smithi046.stdout:6/27: mkdir da/dc 0 2024-09-10T04:05:05.059 INFO:tasks.workunit.client.0.smithi046.stdout:2/131: fdatasync d2/d8/f27 0 2024-09-10T04:05:05.061 INFO:tasks.workunit.client.0.smithi046.stdout:2/132: rmdir d2/d6/d15/d16 39 2024-09-10T04:05:05.064 INFO:tasks.workunit.client.0.smithi046.stdout:2/133: write d2/f25 [992593,45021] 0 2024-09-10T04:05:05.066 INFO:tasks.workunit.client.0.smithi046.stdout:4/34: dwrite d3/fc [0,4194304] 0 2024-09-10T04:05:05.068 INFO:tasks.workunit.client.0.smithi046.stdout:2/134: mkdir d2/d6/d23/d34 0 2024-09-10T04:05:05.069 INFO:tasks.workunit.client.0.smithi046.stdout:4/35: truncate d3/f9 22670 0 2024-09-10T04:05:05.070 INFO:tasks.workunit.client.0.smithi046.stdout:6/28: dwrite f9 [0,4194304] 0 2024-09-10T04:05:05.080 INFO:tasks.workunit.client.0.smithi046.stdout:1/17: sync 2024-09-10T04:05:05.081 INFO:tasks.workunit.client.0.smithi046.stdout:2/135: rename d2/d8/f12 to d2/f35 0 2024-09-10T04:05:05.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:04 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:04 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:05.086 INFO:tasks.workunit.client.0.smithi046.stdout:6/29: dwrite f6 [0,4194304] 0 2024-09-10T04:05:05.087 INFO:tasks.workunit.client.0.smithi046.stdout:4/36: dwrite d3/fc [0,4194304] 0 2024-09-10T04:05:05.090 INFO:tasks.workunit.client.0.smithi046.stdout:4/37: mknod d3/cd 0 2024-09-10T04:05:05.092 INFO:tasks.workunit.client.0.smithi046.stdout:4/38: write d3/fa [467868,77024] 0 2024-09-10T04:05:05.094 INFO:tasks.workunit.client.0.smithi046.stdout:6/30: dread f8 [0,4194304] 0 2024-09-10T04:05:05.098 INFO:tasks.workunit.client.0.smithi046.stdout:6/31: rename f0 to da/fd 0 2024-09-10T04:05:05.099 INFO:tasks.workunit.client.0.smithi046.stdout:6/32: read f9 [4067046,80433] 0 2024-09-10T04:05:05.100 INFO:tasks.workunit.client.0.smithi046.stdout:6/33: stat l5 0 2024-09-10T04:05:05.109 INFO:tasks.workunit.client.0.smithi046.stdout:6/34: dwrite f8 [0,4194304] 0 2024-09-10T04:05:05.111 INFO:tasks.workunit.client.0.smithi046.stdout:7/64: fdatasync f4 0 2024-09-10T04:05:05.112 INFO:tasks.workunit.client.0.smithi046.stdout:6/35: creat da/fe x:0 0 0 2024-09-10T04:05:05.114 INFO:tasks.workunit.client.0.smithi046.stdout:7/65: mknod d5/c17 0 2024-09-10T04:05:05.117 INFO:tasks.workunit.client.0.smithi046.stdout:9/21: fdatasync d4/f5 0 2024-09-10T04:05:05.118 INFO:tasks.workunit.client.0.smithi046.stdout:0/28: rename d1/d2 to d1/d6 0 2024-09-10T04:05:05.118 INFO:tasks.workunit.client.0.smithi046.stdout:0/29: read - no filename 2024-09-10T04:05:05.119 INFO:tasks.workunit.client.0.smithi046.stdout:0/30: stat d1/d6/d4 0 2024-09-10T04:05:05.119 INFO:tasks.workunit.client.0.smithi046.stdout:0/31: dwrite - no filename 2024-09-10T04:05:05.119 INFO:tasks.workunit.client.0.smithi046.stdout:0/32: chown d1 77 1 2024-09-10T04:05:05.119 INFO:tasks.workunit.client.0.smithi046.stdout:0/33: rename d1 to d1/d6/d7 22 2024-09-10T04:05:05.120 INFO:tasks.workunit.client.0.smithi046.stdout:5/32: sync 2024-09-10T04:05:05.121 INFO:tasks.workunit.client.0.smithi046.stdout:7/66: mknod d5/d9/d12/c18 0 2024-09-10T04:05:05.122 INFO:tasks.workunit.client.0.smithi046.stdout:7/67: dread d5/f7 [0,4194304] 0 2024-09-10T04:05:05.122 INFO:tasks.workunit.client.0.smithi046.stdout:9/22: creat d4/d7/f8 x:0 0 0 2024-09-10T04:05:05.123 INFO:tasks.workunit.client.0.smithi046.stdout:3/26: getdents . 0 2024-09-10T04:05:05.124 INFO:tasks.workunit.client.0.smithi046.stdout:3/27: write f8 [1044832,41695] 0 2024-09-10T04:05:05.125 INFO:tasks.workunit.client.0.smithi046.stdout:5/33: rmdir d7 39 2024-09-10T04:05:05.127 INFO:tasks.workunit.client.0.smithi046.stdout:8/83: write d1/d6/d12/f1e [1107113,119755] 0 2024-09-10T04:05:05.128 INFO:tasks.workunit.client.0.smithi046.stdout:9/23: dread d4/f6 [0,4194304] 0 2024-09-10T04:05:05.128 INFO:tasks.workunit.client.0.smithi046.stdout:0/34: symlink d1/d6/d4/l8 0 2024-09-10T04:05:05.128 INFO:tasks.workunit.client.0.smithi046.stdout:0/35: fsync - no filename 2024-09-10T04:05:05.128 INFO:tasks.workunit.client.0.smithi046.stdout:0/36: truncate - no filename 2024-09-10T04:05:05.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/37: chown d1/d6/d4 14399 1 2024-09-10T04:05:05.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/38: truncate - no filename 2024-09-10T04:05:05.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/39: stat d1/d6 0 2024-09-10T04:05:05.130 INFO:tasks.workunit.client.0.smithi046.stdout:0/40: rename d1/d6 to d1/d6/d3/d9 22 2024-09-10T04:05:05.130 INFO:tasks.workunit.client.0.smithi046.stdout:0/41: dwrite - no filename 2024-09-10T04:05:05.130 INFO:tasks.workunit.client.0.smithi046.stdout:3/28: creat f9 x:0 0 0 2024-09-10T04:05:05.131 INFO:tasks.workunit.client.0.smithi046.stdout:3/29: chown f8 2444549 1 2024-09-10T04:05:05.131 INFO:tasks.workunit.client.0.smithi046.stdout:0/42: rename d1/d6 to d1/d6/d4/da 22 2024-09-10T04:05:05.132 INFO:tasks.workunit.client.0.smithi046.stdout:3/30: chown f0 53532 1 2024-09-10T04:05:05.132 INFO:tasks.workunit.client.0.smithi046.stdout:0/43: rename d1/d6/d4 to d1/d6/d4/db 22 2024-09-10T04:05:05.132 INFO:tasks.workunit.client.0.smithi046.stdout:0/44: truncate - no filename 2024-09-10T04:05:05.133 INFO:tasks.workunit.client.0.smithi046.stdout:0/45: write - no filename 2024-09-10T04:05:05.133 INFO:tasks.workunit.client.0.smithi046.stdout:0/46: readlink d1/d6/d4/l8 0 2024-09-10T04:05:05.133 INFO:tasks.workunit.client.0.smithi046.stdout:8/84: mkdir d1/d6/d17/d1f 0 2024-09-10T04:05:05.134 INFO:tasks.workunit.client.0.smithi046.stdout:9/24: mkdir d4/d9 0 2024-09-10T04:05:05.136 INFO:tasks.workunit.client.0.smithi046.stdout:9/25: write d4/d7/f8 [569256,75179] 0 2024-09-10T04:05:05.136 INFO:tasks.workunit.client.0.smithi046.stdout:5/34: symlink d7/la 0 2024-09-10T04:05:05.139 INFO:tasks.workunit.client.0.smithi046.stdout:0/47: mkdir d1/dc 0 2024-09-10T04:05:05.140 INFO:tasks.workunit.client.0.smithi046.stdout:8/85: dread d1/fd [0,4194304] 0 2024-09-10T04:05:05.142 INFO:tasks.workunit.client.0.smithi046.stdout:8/86: truncate d1/d6/f1b 900878 0 2024-09-10T04:05:05.143 INFO:tasks.workunit.client.0.smithi046.stdout:5/35: dwrite d7/f8 [0,4194304] 0 2024-09-10T04:05:05.145 INFO:tasks.workunit.client.0.smithi046.stdout:7/68: getdents d5/d6 0 2024-09-10T04:05:05.146 INFO:tasks.workunit.client.0.smithi046.stdout:7/69: chown d5/d6/db/d11/f14 832187 1 2024-09-10T04:05:05.148 INFO:tasks.workunit.client.0.smithi046.stdout:9/26: unlink d4/f5 0 2024-09-10T04:05:05.151 INFO:tasks.workunit.client.0.smithi046.stdout:5/36: mkdir d7/db 0 2024-09-10T04:05:05.152 INFO:tasks.workunit.client.0.smithi046.stdout:5/37: write f6 [1904360,74651] 0 2024-09-10T04:05:05.153 INFO:tasks.workunit.client.0.smithi046.stdout:5/38: read f1 [3457596,34474] 0 2024-09-10T04:05:05.153 INFO:tasks.workunit.client.0.smithi046.stdout:7/70: rename d5/d9/l10 to d5/d6/db/l19 0 2024-09-10T04:05:05.153 INFO:tasks.workunit.client.0.smithi046.stdout:5/39: chown d7/la 119887 1 2024-09-10T04:05:05.155 INFO:tasks.workunit.client.0.smithi046.stdout:5/40: write f6 [4884626,64876] 0 2024-09-10T04:05:05.157 INFO:tasks.workunit.client.0.smithi046.stdout:4/39: fsync d3/fc 0 2024-09-10T04:05:05.158 INFO:tasks.workunit.client.0.smithi046.stdout:9/27: mknod d4/d7/ca 0 2024-09-10T04:05:05.159 INFO:tasks.workunit.client.0.smithi046.stdout:9/28: write d4/d7/f8 [1582130,15874] 0 2024-09-10T04:05:05.160 INFO:tasks.workunit.client.0.smithi046.stdout:9/29: write d4/d7/f8 [988794,99306] 0 2024-09-10T04:05:05.164 INFO:tasks.workunit.client.0.smithi046.stdout:5/41: dread d7/f9 [0,4194304] 0 2024-09-10T04:05:05.164 INFO:tasks.workunit.client.0.smithi046.stdout:7/71: rename d5/d9/d12/c18 to d5/d9/d12/c1a 0 2024-09-10T04:05:05.169 INFO:tasks.workunit.client.0.smithi046.stdout:9/30: creat d4/fb x:0 0 0 2024-09-10T04:05:05.170 INFO:tasks.workunit.client.0.smithi046.stdout:2/136: truncate d2/d6/f18 272450 0 2024-09-10T04:05:05.172 INFO:tasks.workunit.client.0.smithi046.stdout:5/42: dwrite d7/f8 [0,4194304] 0 2024-09-10T04:05:05.173 INFO:tasks.workunit.client.0.smithi046.stdout:4/40: unlink d3/c6 0 2024-09-10T04:05:05.174 INFO:tasks.workunit.client.0.smithi046.stdout:9/31: symlink d4/lc 0 2024-09-10T04:05:05.175 INFO:tasks.workunit.client.0.smithi046.stdout:9/32: chown d4/d9 420083777 1 2024-09-10T04:05:05.177 INFO:tasks.workunit.client.0.smithi046.stdout:1/18: dwrite f1 [0,4194304] 0 2024-09-10T04:05:05.177 INFO:tasks.workunit.client.0.smithi046.stdout:1/19: readlink - no filename 2024-09-10T04:05:05.177 INFO:tasks.workunit.client.0.smithi046.stdout:1/20: readlink - no filename 2024-09-10T04:05:05.179 INFO:tasks.workunit.client.0.smithi046.stdout:1/21: truncate f0 1272596 0 2024-09-10T04:05:05.179 INFO:tasks.workunit.client.0.smithi046.stdout:9/33: write d4/d7/f8 [1058139,11202] 0 2024-09-10T04:05:05.180 INFO:tasks.workunit.client.0.smithi046.stdout:5/43: unlink f1 0 2024-09-10T04:05:05.188 INFO:tasks.workunit.client.0.smithi046.stdout:5/44: dwrite d7/f8 [0,4194304] 0 2024-09-10T04:05:05.189 INFO:tasks.workunit.client.0.smithi046.stdout:3/31: fsync f9 0 2024-09-10T04:05:05.189 INFO:tasks.workunit.client.0.smithi046.stdout:5/45: fsync f6 0 2024-09-10T04:05:05.190 INFO:tasks.workunit.client.0.smithi046.stdout:1/22: getdents d3 0 2024-09-10T04:05:05.192 INFO:tasks.workunit.client.0.smithi046.stdout:9/34: symlink d4/d9/ld 0 2024-09-10T04:05:05.195 INFO:tasks.workunit.client.0.smithi046.stdout:3/32: creat fa x:0 0 0 2024-09-10T04:05:05.196 INFO:tasks.workunit.client.0.smithi046.stdout:1/23: mknod d3/c5 0 2024-09-10T04:05:05.201 INFO:tasks.workunit.client.0.smithi046.stdout:3/33: rename f8 to fb 0 2024-09-10T04:05:05.202 INFO:tasks.workunit.client.0.smithi046.stdout:3/34: truncate fa 581611 0 2024-09-10T04:05:05.202 INFO:tasks.workunit.client.0.smithi046.stdout:1/24: creat d3/f6 x:0 0 0 2024-09-10T04:05:05.203 INFO:tasks.workunit.client.0.smithi046.stdout:7/72: readlink d5/d6/db/l19 0 2024-09-10T04:05:05.206 INFO:tasks.workunit.client.0.smithi046.stdout:9/35: dwrite d4/f6 [0,4194304] 0 2024-09-10T04:05:05.206 INFO:tasks.workunit.client.0.smithi046.stdout:8/87: truncate d1/d6/d12/f1e 1167294 0 2024-09-10T04:05:05.209 INFO:tasks.workunit.client.0.smithi046.stdout:5/46: getdents d7 0 2024-09-10T04:05:05.209 INFO:tasks.workunit.client.0.smithi046.stdout:1/25: symlink d3/l7 0 2024-09-10T04:05:05.210 INFO:tasks.workunit.client.0.smithi046.stdout:5/47: chown d7/f9 3 1 2024-09-10T04:05:05.213 INFO:tasks.workunit.client.0.smithi046.stdout:3/35: dwrite f3 [0,4194304] 0 2024-09-10T04:05:05.213 INFO:tasks.workunit.client.0.smithi046.stdout:7/73: dread f4 [0,4194304] 0 2024-09-10T04:05:05.217 INFO:tasks.workunit.client.0.smithi046.stdout:9/36: creat d4/d7/fe x:0 0 0 2024-09-10T04:05:05.218 INFO:tasks.workunit.client.0.smithi046.stdout:9/37: dread - d4/fb zero size 2024-09-10T04:05:05.220 INFO:tasks.workunit.client.0.smithi046.stdout:1/26: dread f0 [0,4194304] 0 2024-09-10T04:05:05.223 INFO:tasks.workunit.client.0.smithi046.stdout:3/36: unlink fa 0 2024-09-10T04:05:05.225 INFO:tasks.workunit.client.0.smithi046.stdout:7/74: creat d5/d9/f1b x:0 0 0 2024-09-10T04:05:05.227 INFO:tasks.workunit.client.0.smithi046.stdout:9/38: dwrite d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:05.228 INFO:tasks.workunit.client.0.smithi046.stdout:3/37: fdatasync f3 0 2024-09-10T04:05:05.229 INFO:tasks.workunit.client.0.smithi046.stdout:3/38: fdatasync f5 0 2024-09-10T04:05:05.229 INFO:tasks.workunit.client.0.smithi046.stdout:9/39: truncate d4/fb 397034 0 2024-09-10T04:05:05.231 INFO:tasks.workunit.client.0.smithi046.stdout:3/39: read f3 [1473235,130217] 0 2024-09-10T04:05:05.233 INFO:tasks.workunit.client.0.smithi046.stdout:3/40: truncate f5 4672838 0 2024-09-10T04:05:05.233 INFO:tasks.workunit.client.0.smithi046.stdout:3/41: stat f6 0 2024-09-10T04:05:05.236 INFO:tasks.workunit.client.0.smithi046.stdout:5/48: rmdir d7/db 0 2024-09-10T04:05:05.237 INFO:tasks.workunit.client.0.smithi046.stdout:5/49: chown d7/f9 1 1 2024-09-10T04:05:05.237 INFO:tasks.workunit.client.0.smithi046.stdout:3/42: symlink lc 0 2024-09-10T04:05:05.238 INFO:tasks.workunit.client.0.smithi046.stdout:9/40: dwrite d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:05.238 INFO:tasks.workunit.client.0.smithi046.stdout:3/43: readlink lc 0 2024-09-10T04:05:05.239 INFO:tasks.workunit.client.0.smithi046.stdout:3/44: readlink lc 0 2024-09-10T04:05:05.239 INFO:tasks.workunit.client.0.smithi046.stdout:9/41: write d4/fb [7693,15312] 0 2024-09-10T04:05:05.245 INFO:tasks.workunit.client.0.smithi046.stdout:3/45: dwrite f7 [0,4194304] 0 2024-09-10T04:05:05.254 INFO:tasks.workunit.client.0.smithi046.stdout:5/50: creat d7/fc x:0 0 0 2024-09-10T04:05:05.255 INFO:tasks.workunit.client.0.smithi046.stdout:5/51: truncate f6 5887428 0 2024-09-10T04:05:05.255 INFO:tasks.workunit.client.0.smithi046.stdout:5/52: chown d7/f8 0 1 2024-09-10T04:05:05.256 INFO:tasks.workunit.client.0.smithi046.stdout:9/42: creat d4/d7/ff x:0 0 0 2024-09-10T04:05:05.257 INFO:tasks.workunit.client.0.smithi046.stdout:4/41: stat d3/f4 0 2024-09-10T04:05:05.259 INFO:tasks.workunit.client.0.smithi046.stdout:4/42: write d3/fa [933307,125325] 0 2024-09-10T04:05:05.267 INFO:tasks.workunit.client.0.smithi046.stdout:9/43: dread d4/f6 [0,4194304] 0 2024-09-10T04:05:05.277 INFO:tasks.workunit.client.0.smithi046.stdout:3/46: dwrite fb [0,4194304] 0 2024-09-10T04:05:05.280 INFO:tasks.workunit.client.0.smithi046.stdout:8/88: dwrite d1/fa [0,4194304] 0 2024-09-10T04:05:05.285 INFO:tasks.workunit.client.0.smithi046.stdout:4/43: truncate f1 2830689 0 2024-09-10T04:05:05.286 INFO:tasks.workunit.client.0.smithi046.stdout:2/137: dwrite d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:05.286 INFO:tasks.workunit.client.0.smithi046.stdout:9/44: write d4/f6 [2909946,71626] 0 2024-09-10T04:05:05.287 INFO:tasks.workunit.client.0.smithi046.stdout:1/27: dwrite f0 [0,4194304] 0 2024-09-10T04:05:05.289 INFO:tasks.workunit.client.0.smithi046.stdout:3/47: mknod cd 0 2024-09-10T04:05:05.289 INFO:tasks.workunit.client.0.smithi046.stdout:9/45: read d4/f6 [2222649,77396] 0 2024-09-10T04:05:05.290 INFO:tasks.workunit.client.0.smithi046.stdout:2/138: truncate d2/f2c 924203 0 2024-09-10T04:05:05.293 INFO:tasks.workunit.client.0.smithi046.stdout:2/139: write d2/d6/d23/f33 [548334,84583] 0 2024-09-10T04:05:05.295 INFO:tasks.workunit.client.0.smithi046.stdout:4/44: unlink d3/fb 0 2024-09-10T04:05:05.296 INFO:tasks.workunit.client.0.smithi046.stdout:2/140: chown d2/c4 64 1 2024-09-10T04:05:05.297 INFO:tasks.workunit.client.1.smithi070.stdout:8/7: sync 2024-09-10T04:05:05.298 INFO:tasks.workunit.client.0.smithi046.stdout:9/46: dread d4/fb [0,4194304] 0 2024-09-10T04:05:05.302 INFO:tasks.workunit.client.0.smithi046.stdout:4/45: dread d3/f9 [0,4194304] 0 2024-09-10T04:05:05.302 INFO:tasks.workunit.client.0.smithi046.stdout:3/48: write f4 [66634,83870] 0 2024-09-10T04:05:05.305 INFO:tasks.workunit.client.0.smithi046.stdout:8/89: dwrite d1/d6/fe [4194304,4194304] 0 2024-09-10T04:05:05.307 INFO:tasks.workunit.client.0.smithi046.stdout:4/46: truncate d3/fa 1646119 0 2024-09-10T04:05:05.308 INFO:tasks.workunit.client.0.smithi046.stdout:1/28: dread f1 [0,4194304] 0 2024-09-10T04:05:05.309 INFO:tasks.workunit.client.0.smithi046.stdout:2/141: link d2/l9 d2/d6/d15/l36 0 2024-09-10T04:05:05.311 INFO:tasks.workunit.client.0.smithi046.stdout:8/90: write d1/d6/f1b [1283152,102508] 0 2024-09-10T04:05:05.312 INFO:tasks.workunit.client.0.smithi046.stdout:4/47: mknod d3/ce 0 2024-09-10T04:05:05.314 INFO:tasks.workunit.client.1.smithi070.stdout:8/8: dwrite f3 [0,4194304] 0 2024-09-10T04:05:05.315 INFO:tasks.workunit.client.0.smithi046.stdout:9/47: dwrite d4/d7/fe [0,4194304] 0 2024-09-10T04:05:05.318 INFO:tasks.workunit.client.0.smithi046.stdout:8/91: mknod d1/d6/d17/c20 0 2024-09-10T04:05:05.319 INFO:tasks.workunit.client.0.smithi046.stdout:2/142: rename d2/d6/d23/d34 to d2/d8/d37 0 2024-09-10T04:05:05.322 INFO:tasks.workunit.client.0.smithi046.stdout:9/48: creat d4/d9/f10 x:0 0 0 2024-09-10T04:05:05.323 INFO:tasks.workunit.client.0.smithi046.stdout:1/29: dwrite f0 [0,4194304] 0 2024-09-10T04:05:05.326 INFO:tasks.workunit.client.0.smithi046.stdout:4/48: write f1 [1931397,88531] 0 2024-09-10T04:05:05.326 INFO:tasks.workunit.client.0.smithi046.stdout:9/49: dread - d4/d9/f10 zero size 2024-09-10T04:05:05.328 INFO:tasks.workunit.client.0.smithi046.stdout:8/92: fdatasync d1/d6/d12/f1e 0 2024-09-10T04:05:05.330 INFO:tasks.workunit.client.1.smithi070.stdout:8/9: dwrite f3 [4194304,4194304] 0 2024-09-10T04:05:05.331 INFO:tasks.workunit.client.0.smithi046.stdout:4/49: symlink d3/lf 0 2024-09-10T04:05:05.336 INFO:tasks.workunit.client.0.smithi046.stdout:2/143: dwrite d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:05.336 INFO:tasks.workunit.client.0.smithi046.stdout:9/50: rename d4/lc to d4/l11 0 2024-09-10T04:05:05.339 INFO:tasks.workunit.client.0.smithi046.stdout:4/50: rename d3/l5 to d3/l10 0 2024-09-10T04:05:05.339 INFO:tasks.workunit.client.0.smithi046.stdout:2/144: fdatasync d2/f2c 0 2024-09-10T04:05:05.341 INFO:tasks.workunit.client.1.smithi070.stdout:8/10: write f0 [94960,8296] 0 2024-09-10T04:05:05.341 INFO:tasks.workunit.client.0.smithi046.stdout:1/30: dwrite d3/f4 [0,4194304] 0 2024-09-10T04:05:05.344 INFO:tasks.workunit.client.0.smithi046.stdout:9/51: truncate d4/fb 447095 0 2024-09-10T04:05:05.345 INFO:tasks.workunit.client.0.smithi046.stdout:8/93: dread d1/fd [0,4194304] 0 2024-09-10T04:05:05.347 INFO:tasks.workunit.client.0.smithi046.stdout:9/52: write d4/d9/f10 [607777,30987] 0 2024-09-10T04:05:05.348 INFO:tasks.workunit.client.1.smithi070.stdout:8/11: dread f0 [0,4194304] 0 2024-09-10T04:05:05.348 INFO:tasks.workunit.client.0.smithi046.stdout:2/145: getdents d2/d6/d23 0 2024-09-10T04:05:05.351 INFO:tasks.workunit.client.0.smithi046.stdout:4/51: getdents d3 0 2024-09-10T04:05:05.351 INFO:tasks.workunit.client.0.smithi046.stdout:8/94: symlink d1/d6/d19/l21 0 2024-09-10T04:05:05.352 INFO:tasks.workunit.client.0.smithi046.stdout:1/31: dread f2 [0,4194304] 0 2024-09-10T04:05:05.353 INFO:tasks.workunit.client.1.smithi070.stdout:8/12: mkdir d4 0 2024-09-10T04:05:05.353 INFO:tasks.workunit.client.0.smithi046.stdout:1/32: chown d3/l7 60 1 2024-09-10T04:05:05.354 INFO:tasks.workunit.client.0.smithi046.stdout:2/146: symlink d2/d8/l38 0 2024-09-10T04:05:05.354 INFO:tasks.workunit.client.0.smithi046.stdout:8/95: read d1/d6/d12/f16 [932475,98135] 0 2024-09-10T04:05:05.356 INFO:tasks.workunit.client.0.smithi046.stdout:9/53: unlink d4/l11 0 2024-09-10T04:05:05.358 INFO:tasks.workunit.client.0.smithi046.stdout:4/52: mknod d3/c11 0 2024-09-10T04:05:05.358 INFO:tasks.workunit.client.0.smithi046.stdout:9/54: write d4/d7/fe [1585341,42067] 0 2024-09-10T04:05:05.359 INFO:tasks.workunit.client.1.smithi070.stdout:8/13: creat d4/f5 x:0 0 0 2024-09-10T04:05:05.360 INFO:tasks.workunit.client.1.smithi070.stdout:8/14: read f0 [6824,130703] 0 2024-09-10T04:05:05.360 INFO:tasks.workunit.client.0.smithi046.stdout:1/33: mknod d3/c8 0 2024-09-10T04:05:05.360 INFO:tasks.workunit.client.0.smithi046.stdout:4/53: write f1 [3152192,33639] 0 2024-09-10T04:05:05.362 INFO:tasks.workunit.client.0.smithi046.stdout:8/96: fsync d1/d6/d12/f1e 0 2024-09-10T04:05:05.363 INFO:tasks.workunit.client.1.smithi070.stdout:8/15: dread f3 [0,4194304] 0 2024-09-10T04:05:05.366 INFO:tasks.workunit.client.0.smithi046.stdout:2/147: dwrite d2/d6/d23/f33 [0,4194304] 0 2024-09-10T04:05:05.367 INFO:tasks.workunit.client.0.smithi046.stdout:2/148: fsync d2/f25 0 2024-09-10T04:05:05.368 INFO:tasks.workunit.client.0.smithi046.stdout:9/55: symlink d4/d7/l12 0 2024-09-10T04:05:05.368 INFO:tasks.workunit.client.1.smithi070.stdout:8/16: symlink d4/l6 0 2024-09-10T04:05:05.369 INFO:tasks.workunit.client.1.smithi070.stdout:8/17: write d4/f5 [1017319,23320] 0 2024-09-10T04:05:05.370 INFO:tasks.workunit.client.0.smithi046.stdout:1/34: symlink d3/l9 0 2024-09-10T04:05:05.372 INFO:tasks.workunit.client.1.smithi070.stdout:7/10: sync 2024-09-10T04:05:05.373 INFO:tasks.workunit.client.1.smithi070.stdout:8/18: creat d4/f7 x:0 0 0 2024-09-10T04:05:05.374 INFO:tasks.workunit.client.0.smithi046.stdout:4/54: dwrite d3/fc [4194304,4194304] 0 2024-09-10T04:05:05.375 INFO:tasks.workunit.client.0.smithi046.stdout:2/149: symlink d2/d6/l39 0 2024-09-10T04:05:05.377 INFO:tasks.workunit.client.1.smithi070.stdout:7/11: symlink d2/l4 0 2024-09-10T04:05:05.378 INFO:tasks.workunit.client.0.smithi046.stdout:8/97: mknod d1/d6/d17/d1f/c22 0 2024-09-10T04:05:05.379 INFO:tasks.workunit.client.1.smithi070.stdout:8/19: creat d4/f8 x:0 0 0 2024-09-10T04:05:05.382 INFO:tasks.workunit.client.0.smithi046.stdout:4/55: mkdir d3/d12 0 2024-09-10T04:05:05.383 INFO:tasks.workunit.client.1.smithi070.stdout:7/12: dread f0 [0,4194304] 0 2024-09-10T04:05:05.383 INFO:tasks.workunit.client.1.smithi070.stdout:7/13: chown d2/d3 2 1 2024-09-10T04:05:05.385 INFO:tasks.workunit.client.0.smithi046.stdout:2/150: dwrite d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:05.385 INFO:tasks.workunit.client.0.smithi046.stdout:8/98: unlink d1/l1c 0 2024-09-10T04:05:05.388 INFO:tasks.workunit.client.1.smithi070.stdout:7/14: rename d2/l4 to d2/d3/l5 0 2024-09-10T04:05:05.389 INFO:tasks.workunit.client.0.smithi046.stdout:4/56: truncate d3/f4 1700772 0 2024-09-10T04:05:05.390 INFO:tasks.workunit.client.0.smithi046.stdout:2/151: mkdir d2/d6/d3a 0 2024-09-10T04:05:05.392 INFO:tasks.workunit.client.1.smithi070.stdout:7/15: truncate f1 2299426 0 2024-09-10T04:05:05.394 INFO:tasks.workunit.client.0.smithi046.stdout:4/57: readlink d3/l10 0 2024-09-10T04:05:05.395 INFO:tasks.workunit.client.1.smithi070.stdout:8/20: fsync d4/f8 0 2024-09-10T04:05:05.395 INFO:tasks.workunit.client.0.smithi046.stdout:4/58: rename d3/d12 to d3/d12/d13 22 2024-09-10T04:05:05.397 INFO:tasks.workunit.client.0.smithi046.stdout:2/152: mkdir d2/d6/d23/d3b 0 2024-09-10T04:05:05.401 INFO:tasks.workunit.client.0.smithi046.stdout:1/35: truncate f0 1950595 0 2024-09-10T04:05:05.402 INFO:tasks.workunit.client.0.smithi046.stdout:1/36: rename d3 to d3/da 22 2024-09-10T04:05:05.402 INFO:tasks.workunit.client.0.smithi046.stdout:1/37: stat d3 0 2024-09-10T04:05:05.405 INFO:tasks.workunit.client.0.smithi046.stdout:8/99: write d1/d6/d12/f1e [507512,17496] 0 2024-09-10T04:05:05.406 INFO:tasks.workunit.client.0.smithi046.stdout:2/153: mkdir d2/d6/d15/d16/d3c 0 2024-09-10T04:05:05.407 INFO:tasks.workunit.client.0.smithi046.stdout:4/59: getdents d3 0 2024-09-10T04:05:05.408 INFO:tasks.workunit.client.1.smithi070.stdout:8/21: truncate d4/f5 72621 0 2024-09-10T04:05:05.409 INFO:tasks.workunit.client.0.smithi046.stdout:2/154: truncate d2/d6/d23/f33 4689775 0 2024-09-10T04:05:05.409 INFO:tasks.workunit.client.0.smithi046.stdout:2/155: chown d2/d8/d37 7 1 2024-09-10T04:05:05.410 INFO:tasks.workunit.client.0.smithi046.stdout:2/156: write d2/d8/f29 [734582,82134] 0 2024-09-10T04:05:05.410 INFO:tasks.workunit.client.1.smithi070.stdout:8/22: dread - d4/f8 zero size 2024-09-10T04:05:05.410 INFO:tasks.workunit.client.1.smithi070.stdout:7/16: write f1 [980306,55052] 0 2024-09-10T04:05:05.411 INFO:tasks.workunit.client.0.smithi046.stdout:8/100: creat d1/d6/d17/f23 x:0 0 0 2024-09-10T04:05:05.413 INFO:tasks.workunit.client.1.smithi070.stdout:8/23: mknod d4/c9 0 2024-09-10T04:05:05.415 INFO:tasks.workunit.client.0.smithi046.stdout:4/60: unlink d3/f4 0 2024-09-10T04:05:05.416 INFO:tasks.workunit.client.0.smithi046.stdout:4/61: read f1 [1682517,59951] 0 2024-09-10T04:05:05.418 INFO:tasks.workunit.client.0.smithi046.stdout:8/101: dwrite d1/d6/ff [0,4194304] 0 2024-09-10T04:05:05.419 INFO:tasks.workunit.client.1.smithi070.stdout:8/24: write f0 [433805,103909] 0 2024-09-10T04:05:05.419 INFO:tasks.workunit.client.1.smithi070.stdout:8/25: stat f3 0 2024-09-10T04:05:05.423 INFO:tasks.workunit.client.1.smithi070.stdout:8/26: creat d4/fa x:0 0 0 2024-09-10T04:05:05.426 INFO:tasks.workunit.client.0.smithi046.stdout:1/38: read d3/f4 [1270974,47327] 0 2024-09-10T04:05:05.429 INFO:tasks.workunit.client.0.smithi046.stdout:6/36: sync 2024-09-10T04:05:05.430 INFO:tasks.workunit.client.0.smithi046.stdout:2/157: truncate d2/f25 186517 0 2024-09-10T04:05:05.431 INFO:tasks.workunit.client.0.smithi046.stdout:8/102: symlink d1/d6/d17/l24 0 2024-09-10T04:05:05.431 INFO:tasks.workunit.client.1.smithi070.stdout:7/17: dwrite f0 [0,4194304] 0 2024-09-10T04:05:05.431 INFO:tasks.workunit.client.1.smithi070.stdout:7/18: chown f0 78 1 2024-09-10T04:05:05.434 INFO:tasks.workunit.client.0.smithi046.stdout:4/62: write d3/f9 [851494,75489] 0 2024-09-10T04:05:05.436 INFO:tasks.workunit.client.0.smithi046.stdout:4/63: chown d3/lf 15 1 2024-09-10T04:05:05.438 INFO:tasks.workunit.client.0.smithi046.stdout:6/37: dwrite f8 [0,4194304] 0 2024-09-10T04:05:05.438 INFO:tasks.workunit.client.1.smithi070.stdout:7/19: dwrite f0 [0,4194304] 0 2024-09-10T04:05:05.439 INFO:tasks.workunit.client.0.smithi046.stdout:6/38: readlink l2 0 2024-09-10T04:05:05.439 INFO:tasks.workunit.client.0.smithi046.stdout:1/39: rename f2 to d3/fb 0 2024-09-10T04:05:05.440 INFO:tasks.workunit.client.0.smithi046.stdout:6/39: chown f9 934335 1 2024-09-10T04:05:05.440 INFO:tasks.workunit.client.1.smithi070.stdout:7/20: write f1 [700498,70726] 0 2024-09-10T04:05:05.441 INFO:tasks.workunit.client.1.smithi070.stdout:7/21: write f1 [2534282,65287] 0 2024-09-10T04:05:05.444 INFO:tasks.workunit.client.1.smithi070.stdout:8/27: stat d4/f5 0 2024-09-10T04:05:05.446 INFO:tasks.workunit.client.0.smithi046.stdout:8/103: mknod d1/d6/d17/c25 0 2024-09-10T04:05:05.450 INFO:tasks.workunit.client.1.smithi070.stdout:7/22: symlink d2/l6 0 2024-09-10T04:05:05.450 INFO:tasks.workunit.client.1.smithi070.stdout:7/23: readlink d2/l6 0 2024-09-10T04:05:05.451 INFO:tasks.workunit.client.0.smithi046.stdout:8/104: dread d1/d6/d12/f1e [0,4194304] 0 2024-09-10T04:05:05.451 INFO:tasks.workunit.client.1.smithi070.stdout:8/28: fsync f3 0 2024-09-10T04:05:05.453 INFO:tasks.workunit.client.0.smithi046.stdout:8/105: read d1/d6/f1b [291680,94380] 0 2024-09-10T04:05:05.453 INFO:tasks.workunit.client.0.smithi046.stdout:4/64: mkdir d3/d12/d14 0 2024-09-10T04:05:05.458 INFO:tasks.workunit.client.0.smithi046.stdout:2/158: rmdir d2/d6/d3a 0 2024-09-10T04:05:05.459 INFO:tasks.workunit.client.0.smithi046.stdout:2/159: dread - d2/d8/f27 zero size 2024-09-10T04:05:05.459 INFO:tasks.workunit.client.1.smithi070.stdout:8/29: dwrite d4/f8 [0,4194304] 0 2024-09-10T04:05:05.461 INFO:tasks.workunit.client.0.smithi046.stdout:8/106: rename d1/d6/l1d to d1/d6/d17/d1f/l26 0 2024-09-10T04:05:05.463 INFO:tasks.workunit.client.1.smithi070.stdout:8/30: mknod d4/cb 0 2024-09-10T04:05:05.465 INFO:tasks.workunit.client.0.smithi046.stdout:8/107: write d1/d6/f1b [1491020,95319] 0 2024-09-10T04:05:05.465 INFO:tasks.workunit.client.0.smithi046.stdout:6/40: getdents da 0 2024-09-10T04:05:05.467 INFO:tasks.workunit.client.0.smithi046.stdout:2/160: rename d2/d8/l31 to d2/d6/d23/d3b/l3d 0 2024-09-10T04:05:05.469 INFO:tasks.workunit.client.1.smithi070.stdout:7/24: truncate f1 1717151 0 2024-09-10T04:05:05.470 INFO:tasks.workunit.client.1.smithi070.stdout:8/31: dwrite f3 [4194304,4194304] 0 2024-09-10T04:05:05.470 INFO:tasks.workunit.client.0.smithi046.stdout:8/108: dread d1/d6/fe [4194304,4194304] 0 2024-09-10T04:05:05.471 INFO:tasks.workunit.client.0.smithi046.stdout:8/109: fsync d1/d6/d17/f23 0 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:5/23: sync 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:1/24: sync 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:6/106: sync 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:2/35: sync 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:9/43: sync 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:9/44: dwrite - no filename 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:9/45: read - no filename 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:3/46: sync 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:4/29: sync 2024-09-10T04:05:05.472 INFO:tasks.workunit.client.1.smithi070.stdout:0/41: sync 2024-09-10T04:05:05.473 INFO:tasks.workunit.client.0.smithi046.stdout:4/65: write f1 [682506,58516] 0 2024-09-10T04:05:05.473 INFO:tasks.workunit.client.1.smithi070.stdout:7/25: mkdir d2/d7 0 2024-09-10T04:05:05.473 INFO:tasks.workunit.client.1.smithi070.stdout:5/24: chown l1 3178 1 2024-09-10T04:05:05.475 INFO:tasks.workunit.client.1.smithi070.stdout:5/25: chown l1 0 1 2024-09-10T04:05:05.475 INFO:tasks.workunit.client.0.smithi046.stdout:1/40: write f0 [13885,67477] 0 2024-09-10T04:05:05.477 INFO:tasks.workunit.client.0.smithi046.stdout:1/41: write d3/f4 [149510,75024] 0 2024-09-10T04:05:05.477 INFO:tasks.workunit.client.1.smithi070.stdout:2/36: symlink l4 0 2024-09-10T04:05:05.479 INFO:tasks.workunit.client.0.smithi046.stdout:8/110: creat d1/d6/f27 x:0 0 0 2024-09-10T04:05:05.480 INFO:tasks.workunit.client.0.smithi046.stdout:8/111: chown d1/fd 40944 1 2024-09-10T04:05:05.481 INFO:tasks.workunit.client.1.smithi070.stdout:8/32: dwrite f0 [0,4194304] 0 2024-09-10T04:05:05.481 INFO:tasks.workunit.client.1.smithi070.stdout:0/42: creat d8/fc x:0 0 0 2024-09-10T04:05:05.482 INFO:tasks.workunit.client.0.smithi046.stdout:4/66: creat d3/f15 x:0 0 0 2024-09-10T04:05:05.482 INFO:tasks.workunit.client.0.smithi046.stdout:6/41: link c4 da/dc/cf 0 2024-09-10T04:05:05.482 INFO:tasks.workunit.client.0.smithi046.stdout:4/67: stat d3/c11 0 2024-09-10T04:05:05.483 INFO:tasks.workunit.client.0.smithi046.stdout:4/68: dread - d3/f15 zero size 2024-09-10T04:05:05.483 INFO:tasks.workunit.client.0.smithi046.stdout:4/69: chown d3 2 1 2024-09-10T04:05:05.483 INFO:tasks.workunit.client.1.smithi070.stdout:8/33: fdatasync d4/fa 0 2024-09-10T04:05:05.483 INFO:tasks.workunit.client.1.smithi070.stdout:0/43: chown f5 24401 1 2024-09-10T04:05:05.484 INFO:tasks.workunit.client.1.smithi070.stdout:1/25: rename l2 to d3/la 0 2024-09-10T04:05:05.484 INFO:tasks.workunit.client.0.smithi046.stdout:4/70: write d3/fa [206673,39005] 0 2024-09-10T04:05:05.485 INFO:tasks.workunit.client.0.smithi046.stdout:2/161: creat d2/d6/d15/f3e x:0 0 0 2024-09-10T04:05:05.486 INFO:tasks.workunit.client.1.smithi070.stdout:7/26: dwrite f0 [4194304,4194304] 0 2024-09-10T04:05:05.487 INFO:tasks.workunit.client.1.smithi070.stdout:7/27: stat d2/d7 0 2024-09-10T04:05:05.487 INFO:tasks.workunit.client.1.smithi070.stdout:5/26: unlink f0 0 2024-09-10T04:05:05.490 INFO:tasks.workunit.client.1.smithi070.stdout:2/37: rename l4 to l5 0 2024-09-10T04:05:05.493 INFO:tasks.workunit.client.0.smithi046.stdout:1/42: dwrite d3/fb [0,4194304] 0 2024-09-10T04:05:05.494 INFO:tasks.workunit.client.1.smithi070.stdout:9/46: symlink d9/dd/lf 0 2024-09-10T04:05:05.494 INFO:tasks.workunit.client.0.smithi046.stdout:6/42: getdents da/db 0 2024-09-10T04:05:05.495 INFO:tasks.workunit.client.1.smithi070.stdout:0/44: creat d8/fd x:0 0 0 2024-09-10T04:05:05.495 INFO:tasks.workunit.client.0.smithi046.stdout:6/43: chown f6 0 1 2024-09-10T04:05:05.496 INFO:tasks.workunit.client.0.smithi046.stdout:6/44: read - da/fd zero size 2024-09-10T04:05:05.496 INFO:tasks.workunit.client.0.smithi046.stdout:4/71: creat d3/d12/d14/f16 x:0 0 0 2024-09-10T04:05:05.498 INFO:tasks.workunit.client.0.smithi046.stdout:4/72: write d3/fc [2859681,44789] 0 2024-09-10T04:05:05.498 INFO:tasks.workunit.client.1.smithi070.stdout:8/34: mknod d4/cc 0 2024-09-10T04:05:05.502 INFO:tasks.workunit.client.1.smithi070.stdout:1/26: symlink d3/lb 0 2024-09-10T04:05:05.503 INFO:tasks.workunit.client.0.smithi046.stdout:1/43: mkdir d3/dc 0 2024-09-10T04:05:05.507 INFO:tasks.workunit.client.1.smithi070.stdout:5/27: creat f6 x:0 0 0 2024-09-10T04:05:05.510 INFO:tasks.workunit.client.0.smithi046.stdout:4/73: mkdir d3/d12/d17 0 2024-09-10T04:05:05.511 INFO:tasks.workunit.client.0.smithi046.stdout:4/74: write d3/fa [776231,80579] 0 2024-09-10T04:05:05.513 INFO:tasks.workunit.client.0.smithi046.stdout:8/112: getdents d1/d6/d17 0 2024-09-10T04:05:05.514 INFO:tasks.workunit.client.0.smithi046.stdout:6/45: mkdir da/db/d10 0 2024-09-10T04:05:05.516 INFO:tasks.workunit.client.1.smithi070.stdout:6/107: symlink d0/d3/d18/d1b/d17/l24 0 2024-09-10T04:05:05.517 INFO:tasks.workunit.client.1.smithi070.stdout:4/30: write f6 [615538,60382] 0 2024-09-10T04:05:05.519 INFO:tasks.workunit.client.0.smithi046.stdout:4/75: creat d3/d12/d14/f18 x:0 0 0 2024-09-10T04:05:05.520 INFO:tasks.workunit.client.1.smithi070.stdout:0/45: mknod d8/ce 0 2024-09-10T04:05:05.521 INFO:tasks.workunit.client.0.smithi046.stdout:8/113: write d1/fd [3702948,86204] 0 2024-09-10T04:05:05.522 INFO:tasks.workunit.client.1.smithi070.stdout:8/35: mknod d4/cd 0 2024-09-10T04:05:05.524 INFO:tasks.workunit.client.0.smithi046.stdout:6/46: mkdir da/db/d11 0 2024-09-10T04:05:05.525 INFO:tasks.workunit.client.0.smithi046.stdout:6/47: fsync da/fe 0 2024-09-10T04:05:05.525 INFO:tasks.workunit.client.0.smithi046.stdout:8/114: symlink d1/l28 0 2024-09-10T04:05:05.527 INFO:tasks.workunit.client.0.smithi046.stdout:6/48: truncate da/fe 599596 0 2024-09-10T04:05:05.527 INFO:tasks.workunit.client.0.smithi046.stdout:8/115: fdatasync d1/d6/f1b 0 2024-09-10T04:05:05.528 INFO:tasks.workunit.client.0.smithi046.stdout:4/76: dwrite d3/d12/d14/f16 [0,4194304] 0 2024-09-10T04:05:05.530 INFO:tasks.workunit.client.0.smithi046.stdout:4/77: write d3/fc [4858431,73534] 0 2024-09-10T04:05:05.533 INFO:tasks.workunit.client.0.smithi046.stdout:6/49: rename f8 to da/f12 0 2024-09-10T04:05:05.533 INFO:tasks.workunit.client.1.smithi070.stdout:3/47: getdents . 0 2024-09-10T04:05:05.534 INFO:tasks.workunit.client.1.smithi070.stdout:2/38: dwrite f3 [0,4194304] 0 2024-09-10T04:05:05.535 INFO:tasks.workunit.client.0.smithi046.stdout:4/78: dread f1 [0,4194304] 0 2024-09-10T04:05:05.538 INFO:tasks.workunit.client.1.smithi070.stdout:3/48: dread f3 [0,4194304] 0 2024-09-10T04:05:05.542 INFO:tasks.workunit.client.1.smithi070.stdout:5/28: creat f7 x:0 0 0 2024-09-10T04:05:05.544 INFO:tasks.workunit.client.1.smithi070.stdout:6/108: truncate d0/d3/d18/d1b/f14 580092 0 2024-09-10T04:05:05.545 INFO:tasks.workunit.client.0.smithi046.stdout:4/79: dwrite f1 [0,4194304] 0 2024-09-10T04:05:05.546 INFO:tasks.workunit.client.1.smithi070.stdout:0/46: mkdir d8/df 0 2024-09-10T04:05:05.547 INFO:tasks.workunit.client.1.smithi070.stdout:8/36: creat d4/fe x:0 0 0 2024-09-10T04:05:05.548 INFO:tasks.workunit.client.1.smithi070.stdout:8/37: stat c1 0 2024-09-10T04:05:05.549 INFO:tasks.workunit.client.1.smithi070.stdout:1/27: dwrite f0 [0,4194304] 0 2024-09-10T04:05:05.550 INFO:tasks.workunit.client.1.smithi070.stdout:8/38: write f0 [3962257,29755] 0 2024-09-10T04:05:05.550 INFO:tasks.workunit.client.1.smithi070.stdout:8/39: dread - d4/f7 zero size 2024-09-10T04:05:05.551 INFO:tasks.workunit.client.0.smithi046.stdout:4/80: rename f1 to d3/d12/f19 0 2024-09-10T04:05:05.557 INFO:tasks.workunit.client.1.smithi070.stdout:2/39: symlink l6 0 2024-09-10T04:05:05.559 INFO:tasks.workunit.client.1.smithi070.stdout:3/49: creat fa x:0 0 0 2024-09-10T04:05:05.559 INFO:tasks.workunit.client.1.smithi070.stdout:3/50: stat f6 0 2024-09-10T04:05:05.560 INFO:tasks.workunit.client.0.smithi046.stdout:4/81: dread d3/fa [0,4194304] 0 2024-09-10T04:05:05.568 INFO:tasks.workunit.client.1.smithi070.stdout:8/40: creat d4/ff x:0 0 0 2024-09-10T04:05:05.569 INFO:tasks.workunit.client.1.smithi070.stdout:9/47: getdents d9 0 2024-09-10T04:05:05.569 INFO:tasks.workunit.client.1.smithi070.stdout:9/48: truncate - no filename 2024-09-10T04:05:05.569 INFO:tasks.workunit.client.0.smithi046.stdout:4/82: symlink d3/l1a 0 2024-09-10T04:05:05.569 INFO:tasks.workunit.client.1.smithi070.stdout:4/31: creat f8 x:0 0 0 2024-09-10T04:05:05.570 INFO:tasks.workunit.client.1.smithi070.stdout:9/49: chown l4 40 1 2024-09-10T04:05:05.570 INFO:tasks.workunit.client.1.smithi070.stdout:8/41: write d4/fe [577599,73805] 0 2024-09-10T04:05:05.573 INFO:tasks.workunit.client.1.smithi070.stdout:7/28: read f1 [317655,115935] 0 2024-09-10T04:05:05.574 INFO:tasks.workunit.client.0.smithi046.stdout:4/83: creat d3/d12/d17/f1b x:0 0 0 2024-09-10T04:05:05.574 INFO:tasks.workunit.client.0.smithi046.stdout:4/84: dread - d3/d12/d14/f18 zero size 2024-09-10T04:05:05.576 INFO:tasks.workunit.client.1.smithi070.stdout:8/42: creat d4/f10 x:0 0 0 2024-09-10T04:05:05.576 INFO:tasks.workunit.client.1.smithi070.stdout:8/43: chown f3 2724603 1 2024-09-10T04:05:05.576 INFO:tasks.workunit.client.1.smithi070.stdout:8/44: readlink l2 0 2024-09-10T04:05:05.576 INFO:tasks.workunit.client.1.smithi070.stdout:9/50: mkdir d9/dd/d10 0 2024-09-10T04:05:05.577 INFO:tasks.workunit.client.1.smithi070.stdout:8/45: read - d4/fa zero size 2024-09-10T04:05:05.577 INFO:tasks.workunit.client.1.smithi070.stdout:9/51: chown d9/dd 6053 1 2024-09-10T04:05:05.577 INFO:tasks.workunit.client.1.smithi070.stdout:9/52: read - no filename 2024-09-10T04:05:05.577 INFO:tasks.workunit.client.1.smithi070.stdout:9/53: chown l4 7810 1 2024-09-10T04:05:05.578 INFO:tasks.workunit.client.1.smithi070.stdout:8/46: truncate d4/f7 117581 0 2024-09-10T04:05:05.578 INFO:tasks.workunit.client.1.smithi070.stdout:6/109: write d0/f10 [2743241,109248] 0 2024-09-10T04:05:05.578 INFO:tasks.workunit.client.1.smithi070.stdout:8/47: fdatasync d4/ff 0 2024-09-10T04:05:05.578 INFO:tasks.workunit.client.1.smithi070.stdout:4/32: fdatasync f5 0 2024-09-10T04:05:05.580 INFO:tasks.workunit.client.1.smithi070.stdout:6/110: chown d0/d3/d18/d1b/d17/l24 111666515 1 2024-09-10T04:05:05.582 INFO:tasks.workunit.client.1.smithi070.stdout:1/28: write f1 [1399957,72147] 0 2024-09-10T04:05:05.583 INFO:tasks.workunit.client.0.smithi046.stdout:4/85: link d3/l7 d3/d12/d17/l1c 0 2024-09-10T04:05:05.586 INFO:tasks.workunit.client.1.smithi070.stdout:8/48: unlink d4/f7 0 2024-09-10T04:05:05.587 INFO:tasks.workunit.client.1.smithi070.stdout:0/47: dwrite f1 [0,4194304] 0 2024-09-10T04:05:05.587 INFO:tasks.workunit.client.1.smithi070.stdout:6/111: unlink d0/l2 0 2024-09-10T04:05:05.588 INFO:tasks.workunit.client.1.smithi070.stdout:5/29: link f3 f8 0 2024-09-10T04:05:05.591 INFO:tasks.workunit.client.1.smithi070.stdout:1/29: rename d3/f6 to d3/d8/fc 0 2024-09-10T04:05:05.593 INFO:tasks.workunit.client.1.smithi070.stdout:1/30: write f0 [2773366,63846] 0 2024-09-10T04:05:05.595 INFO:tasks.workunit.client.1.smithi070.stdout:8/49: unlink d4/cb 0 2024-09-10T04:05:05.595 INFO:tasks.workunit.client.1.smithi070.stdout:7/29: dwrite f1 [0,4194304] 0 2024-09-10T04:05:05.597 INFO:tasks.workunit.client.1.smithi070.stdout:6/112: mknod d0/d3/d18/d1b/d17/c25 0 2024-09-10T04:05:05.599 INFO:tasks.workunit.client.1.smithi070.stdout:9/54: rmdir d9/de 0 2024-09-10T04:05:05.603 INFO:tasks.workunit.client.1.smithi070.stdout:4/33: write f4 [1027133,55635] 0 2024-09-10T04:05:05.605 INFO:tasks.workunit.client.1.smithi070.stdout:7/30: mknod d2/c8 0 2024-09-10T04:05:05.610 INFO:tasks.workunit.client.1.smithi070.stdout:1/31: dwrite f0 [4194304,4194304] 0 2024-09-10T04:05:05.611 INFO:tasks.workunit.client.1.smithi070.stdout:8/50: dwrite d4/f10 [0,4194304] 0 2024-09-10T04:05:05.613 INFO:tasks.workunit.client.1.smithi070.stdout:9/55: mknod d9/dd/d10/c11 0 2024-09-10T04:05:05.613 INFO:tasks.workunit.client.1.smithi070.stdout:0/48: dread f5 [0,4194304] 0 2024-09-10T04:05:05.615 INFO:tasks.workunit.client.1.smithi070.stdout:8/51: creat d4/f11 x:0 0 0 2024-09-10T04:05:05.617 INFO:tasks.workunit.client.1.smithi070.stdout:1/32: truncate d3/f5 4050511 0 2024-09-10T04:05:05.618 INFO:tasks.workunit.client.1.smithi070.stdout:6/113: dread d0/f8 [0,4194304] 0 2024-09-10T04:05:05.618 INFO:tasks.workunit.client.1.smithi070.stdout:8/52: fdatasync d4/ff 0 2024-09-10T04:05:05.619 INFO:tasks.workunit.client.1.smithi070.stdout:4/34: getdents . 0 2024-09-10T04:05:05.620 INFO:tasks.workunit.client.1.smithi070.stdout:7/31: dread f0 [0,4194304] 0 2024-09-10T04:05:05.621 INFO:tasks.workunit.client.1.smithi070.stdout:4/35: write f4 [1059633,64282] 0 2024-09-10T04:05:05.621 INFO:tasks.workunit.client.1.smithi070.stdout:8/53: write d4/ff [411497,73819] 0 2024-09-10T04:05:05.622 INFO:tasks.workunit.client.1.smithi070.stdout:1/33: creat d3/fd x:0 0 0 2024-09-10T04:05:05.623 INFO:tasks.workunit.client.1.smithi070.stdout:0/49: rmdir d8/df 0 2024-09-10T04:05:05.624 INFO:tasks.workunit.client.1.smithi070.stdout:6/114: mknod d0/d3/d18/d1b/d17/c26 0 2024-09-10T04:05:05.626 INFO:tasks.workunit.client.1.smithi070.stdout:1/34: symlink d3/d8/le 0 2024-09-10T04:05:05.628 INFO:tasks.workunit.client.1.smithi070.stdout:7/32: mkdir d2/d7/d9 0 2024-09-10T04:05:05.629 INFO:tasks.workunit.client.0.smithi046.stdout:7/75: sync 2024-09-10T04:05:05.629 INFO:tasks.workunit.client.0.smithi046.stdout:0/48: sync 2024-09-10T04:05:05.629 INFO:tasks.workunit.client.0.smithi046.stdout:5/53: sync 2024-09-10T04:05:05.629 INFO:tasks.workunit.client.1.smithi070.stdout:6/115: creat d0/d3/d18/f27 x:0 0 0 2024-09-10T04:05:05.630 INFO:tasks.workunit.client.0.smithi046.stdout:7/76: fdatasync d5/d9/fe 0 2024-09-10T04:05:05.631 INFO:tasks.workunit.client.0.smithi046.stdout:5/54: rmdir d7 39 2024-09-10T04:05:05.633 INFO:tasks.workunit.client.0.smithi046.stdout:0/49: mkdir d1/d6/d3/dd 0 2024-09-10T04:05:05.635 INFO:tasks.workunit.client.0.smithi046.stdout:7/77: creat d5/d6/f1c x:0 0 0 2024-09-10T04:05:05.635 INFO:tasks.workunit.client.1.smithi070.stdout:4/36: dwrite f8 [0,4194304] 0 2024-09-10T04:05:05.636 INFO:tasks.workunit.client.1.smithi070.stdout:8/54: dwrite d4/f8 [0,4194304] 0 2024-09-10T04:05:05.636 INFO:tasks.workunit.client.0.smithi046.stdout:5/55: readlink d7/la 0 2024-09-10T04:05:05.638 INFO:tasks.workunit.client.0.smithi046.stdout:0/50: rmdir d1/d6 39 2024-09-10T04:05:05.638 INFO:tasks.workunit.client.1.smithi070.stdout:6/116: dread d0/d3/d18/d1b/f14 [0,4194304] 0 2024-09-10T04:05:05.640 INFO:tasks.workunit.client.1.smithi070.stdout:0/50: getdents d8 0 2024-09-10T04:05:05.641 INFO:tasks.workunit.client.0.smithi046.stdout:7/78: rename d5/d9/f1b to d5/d6/db/f1d 0 2024-09-10T04:05:05.642 INFO:tasks.workunit.client.1.smithi070.stdout:4/37: symlink l9 0 2024-09-10T04:05:05.642 INFO:tasks.workunit.client.0.smithi046.stdout:7/79: chown d5/d6/db/d11/f14 13 1 2024-09-10T04:05:05.643 INFO:tasks.workunit.client.1.smithi070.stdout:1/35: dwrite d3/d8/fc [4194304,4194304] 0 2024-09-10T04:05:05.644 INFO:tasks.workunit.client.1.smithi070.stdout:8/55: creat d4/f12 x:0 0 0 2024-09-10T04:05:05.646 INFO:tasks.workunit.client.0.smithi046.stdout:0/51: symlink d1/d6/d3/le 0 2024-09-10T04:05:05.647 INFO:tasks.workunit.client.1.smithi070.stdout:7/33: rename d2/d3/l5 to d2/d7/la 0 2024-09-10T04:05:05.648 INFO:tasks.workunit.client.0.smithi046.stdout:7/80: mkdir d5/d6/db/d11/d1e 0 2024-09-10T04:05:05.649 INFO:tasks.workunit.client.1.smithi070.stdout:1/36: dread f0 [0,4194304] 0 2024-09-10T04:05:05.650 INFO:tasks.workunit.client.1.smithi070.stdout:0/51: creat d8/f10 x:0 0 0 2024-09-10T04:05:05.652 INFO:tasks.workunit.client.1.smithi070.stdout:7/34: dread f0 [0,4194304] 0 2024-09-10T04:05:05.653 INFO:tasks.workunit.client.0.smithi046.stdout:0/52: symlink d1/dc/lf 0 2024-09-10T04:05:05.655 INFO:tasks.workunit.client.1.smithi070.stdout:6/117: write d0/f15 [918799,97219] 0 2024-09-10T04:05:05.656 INFO:tasks.workunit.client.0.smithi046.stdout:0/53: creat d1/d6/d3/f10 x:0 0 0 2024-09-10T04:05:05.657 INFO:tasks.workunit.client.1.smithi070.stdout:8/56: creat d4/f13 x:0 0 0 2024-09-10T04:05:05.659 INFO:tasks.workunit.client.1.smithi070.stdout:7/35: dwrite f1 [0,4194304] 0 2024-09-10T04:05:05.660 INFO:tasks.workunit.client.1.smithi070.stdout:7/36: read f0 [1819476,72731] 0 2024-09-10T04:05:05.660 INFO:tasks.workunit.client.0.smithi046.stdout:0/54: creat d1/d6/f11 x:0 0 0 2024-09-10T04:05:05.661 INFO:tasks.workunit.client.1.smithi070.stdout:4/38: mknod ca 0 2024-09-10T04:05:05.668 INFO:tasks.workunit.client.1.smithi070.stdout:7/37: dwrite f1 [4194304,4194304] 0 2024-09-10T04:05:05.669 INFO:tasks.workunit.client.1.smithi070.stdout:8/57: symlink d4/l14 0 2024-09-10T04:05:05.670 INFO:tasks.workunit.client.1.smithi070.stdout:8/58: stat c1 0 2024-09-10T04:05:05.672 INFO:tasks.workunit.client.1.smithi070.stdout:0/52: write f5 [4027699,67504] 0 2024-09-10T04:05:05.672 INFO:tasks.workunit.client.1.smithi070.stdout:7/38: mkdir d2/d7/db 0 2024-09-10T04:05:05.673 INFO:tasks.workunit.client.1.smithi070.stdout:4/39: fsync f5 0 2024-09-10T04:05:05.676 INFO:tasks.workunit.client.1.smithi070.stdout:0/53: creat d8/f11 x:0 0 0 2024-09-10T04:05:05.676 INFO:tasks.workunit.client.1.smithi070.stdout:1/37: truncate f0 904256 0 2024-09-10T04:05:05.677 INFO:tasks.workunit.client.1.smithi070.stdout:0/54: chown d8/l9 1 1 2024-09-10T04:05:05.677 INFO:tasks.workunit.client.1.smithi070.stdout:7/39: mkdir d2/d3/dc 0 2024-09-10T04:05:05.677 INFO:tasks.workunit.client.1.smithi070.stdout:0/55: stat d8/lb 0 2024-09-10T04:05:05.790 INFO:tasks.workunit.client.1.smithi070.stdout:6/118: getdents d0/d3/d18/d1b/d17 0 2024-09-10T04:05:05.790 INFO:tasks.workunit.client.1.smithi070.stdout:8/59: link d4/ff d4/f15 0 2024-09-10T04:05:05.791 INFO:tasks.workunit.client.1.smithi070.stdout:0/56: creat d8/f12 x:0 0 0 2024-09-10T04:05:05.792 INFO:tasks.workunit.client.1.smithi070.stdout:7/40: mknod d2/d7/d9/cd 0 2024-09-10T04:05:05.792 INFO:tasks.workunit.client.1.smithi070.stdout:8/60: mkdir d4/d16 0 2024-09-10T04:05:05.793 INFO:tasks.workunit.client.1.smithi070.stdout:0/57: mkdir d8/d13 0 2024-09-10T04:05:05.804 INFO:tasks.workunit.client.0.smithi046.stdout:9/56: sync 2024-09-10T04:05:05.805 INFO:tasks.workunit.client.0.smithi046.stdout:3/49: sync 2024-09-10T04:05:05.816 INFO:tasks.workunit.client.0.smithi046.stdout:9/57: readlink d4/d7/l12 0 2024-09-10T04:05:05.817 INFO:tasks.workunit.client.0.smithi046.stdout:3/50: write fb [4898688,16291] 0 2024-09-10T04:05:05.817 INFO:tasks.workunit.client.0.smithi046.stdout:3/51: rmdir - no directory 2024-09-10T04:05:05.817 INFO:tasks.workunit.client.0.smithi046.stdout:3/52: chown f5 34 1 2024-09-10T04:05:05.818 INFO:tasks.workunit.client.0.smithi046.stdout:9/58: truncate d4/d9/f10 705514 0 2024-09-10T04:05:05.836 INFO:tasks.workunit.client.1.smithi070.stdout:0/58: read - d8/f10 zero size 2024-09-10T04:05:05.836 INFO:tasks.workunit.client.1.smithi070.stdout:2/40: sync 2024-09-10T04:05:05.836 INFO:tasks.workunit.client.1.smithi070.stdout:0/59: chown d8 135248589 1 2024-09-10T04:05:05.838 INFO:tasks.workunit.client.1.smithi070.stdout:0/60: fdatasync d8/fd 0 2024-09-10T04:05:05.838 INFO:tasks.workunit.client.1.smithi070.stdout:4/40: dwrite f4 [0,4194304] 0 2024-09-10T04:05:05.841 INFO:tasks.workunit.client.1.smithi070.stdout:1/38: link d3/la d3/d8/lf 0 2024-09-10T04:05:05.847 INFO:tasks.workunit.client.1.smithi070.stdout:7/41: dwrite f0 [0,4194304] 0 2024-09-10T04:05:05.847 INFO:tasks.workunit.client.1.smithi070.stdout:6/119: getdents d0/d3/d1f 0 2024-09-10T04:05:05.848 INFO:tasks.workunit.client.1.smithi070.stdout:2/41: dread f3 [0,4194304] 0 2024-09-10T04:05:05.850 INFO:tasks.workunit.client.1.smithi070.stdout:6/120: dread - d0/d3/d18/f27 zero size 2024-09-10T04:05:05.851 INFO:tasks.workunit.client.1.smithi070.stdout:1/39: mkdir d3/d10 0 2024-09-10T04:05:05.851 INFO:tasks.workunit.client.1.smithi070.stdout:8/61: dwrite d4/f15 [0,4194304] 0 2024-09-10T04:05:05.856 INFO:tasks.workunit.client.1.smithi070.stdout:4/41: creat fb x:0 0 0 2024-09-10T04:05:05.860 INFO:tasks.workunit.client.1.smithi070.stdout:7/42: creat d2/d7/fe x:0 0 0 2024-09-10T04:05:05.861 INFO:tasks.workunit.client.1.smithi070.stdout:6/121: mkdir d0/d3/d18/d1b/d17/d28 0 2024-09-10T04:05:05.864 INFO:tasks.workunit.client.1.smithi070.stdout:7/43: dread f0 [0,4194304] 0 2024-09-10T04:05:05.864 INFO:tasks.workunit.client.1.smithi070.stdout:1/40: rename d3/d8/lf to d3/l11 0 2024-09-10T04:05:05.865 INFO:tasks.workunit.client.1.smithi070.stdout:7/44: chown d2/d7/db 0 1 2024-09-10T04:05:05.866 INFO:tasks.workunit.client.1.smithi070.stdout:6/122: dread d0/f10 [0,4194304] 0 2024-09-10T04:05:05.866 INFO:tasks.workunit.client.1.smithi070.stdout:4/42: write f6 [1043719,58836] 0 2024-09-10T04:05:05.871 INFO:tasks.workunit.client.1.smithi070.stdout:8/62: mkdir d4/d16/d17 0 2024-09-10T04:05:05.873 INFO:tasks.workunit.client.1.smithi070.stdout:1/41: mkdir d3/d8/d12 0 2024-09-10T04:05:05.877 INFO:tasks.workunit.client.1.smithi070.stdout:8/63: dread f3 [4194304,4194304] 0 2024-09-10T04:05:05.877 INFO:tasks.workunit.client.1.smithi070.stdout:8/64: fsync d4/f10 0 2024-09-10T04:05:05.885 INFO:tasks.workunit.client.1.smithi070.stdout:7/45: symlink d2/d7/db/lf 0 2024-09-10T04:05:05.887 INFO:tasks.workunit.client.1.smithi070.stdout:8/65: symlink d4/l18 0 2024-09-10T04:05:05.887 INFO:tasks.workunit.client.1.smithi070.stdout:8/66: write d4/f11 [201418,52224] 0 2024-09-10T04:05:05.888 INFO:tasks.workunit.client.1.smithi070.stdout:8/67: readlink d4/l18 0 2024-09-10T04:05:05.888 INFO:tasks.workunit.client.1.smithi070.stdout:2/42: getdents . 0 2024-09-10T04:05:05.888 INFO:tasks.workunit.client.1.smithi070.stdout:8/68: read - d4/f12 zero size 2024-09-10T04:05:05.891 INFO:tasks.workunit.client.1.smithi070.stdout:6/123: mknod d0/c29 0 2024-09-10T04:05:05.892 INFO:tasks.workunit.client.1.smithi070.stdout:8/69: dread f0 [0,4194304] 0 2024-09-10T04:05:05.894 INFO:tasks.workunit.client.1.smithi070.stdout:7/46: symlink d2/d7/d9/l10 0 2024-09-10T04:05:05.895 INFO:tasks.workunit.client.1.smithi070.stdout:2/43: symlink l7 0 2024-09-10T04:05:05.898 INFO:tasks.workunit.client.1.smithi070.stdout:2/44: dread f3 [0,4194304] 0 2024-09-10T04:05:05.898 INFO:tasks.workunit.client.1.smithi070.stdout:4/43: creat fc x:0 0 0 2024-09-10T04:05:05.900 INFO:tasks.workunit.client.1.smithi070.stdout:6/124: mkdir d0/d3/d18/d2a 0 2024-09-10T04:05:05.901 INFO:tasks.workunit.client.1.smithi070.stdout:2/45: dread f3 [0,4194304] 0 2024-09-10T04:05:05.902 INFO:tasks.workunit.client.1.smithi070.stdout:7/47: symlink d2/l11 0 2024-09-10T04:05:05.903 INFO:tasks.workunit.client.1.smithi070.stdout:6/125: dread d0/f15 [0,4194304] 0 2024-09-10T04:05:05.907 INFO:tasks.workunit.client.1.smithi070.stdout:8/70: mkdir d4/d16/d17/d19 0 2024-09-10T04:05:05.911 INFO:tasks.workunit.client.1.smithi070.stdout:6/126: rename d0/d3/f12 to d0/d3/d18/f2b 0 2024-09-10T04:05:05.911 INFO:tasks.workunit.client.1.smithi070.stdout:7/48: dwrite d2/d7/fe [0,4194304] 0 2024-09-10T04:05:05.912 INFO:tasks.workunit.client.1.smithi070.stdout:0/61: dwrite d8/fa [0,4194304] 0 2024-09-10T04:05:05.913 INFO:tasks.workunit.client.1.smithi070.stdout:4/44: truncate f0 235861 0 2024-09-10T04:05:05.920 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:05 smithi070 ceph-mon[35495]: pgmap v256: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 239 op/s 2024-09-10T04:05:05.922 INFO:tasks.workunit.client.1.smithi070.stdout:8/71: mkdir d4/d16/d17/d1a 0 2024-09-10T04:05:05.923 INFO:tasks.workunit.client.1.smithi070.stdout:8/72: write d4/f11 [1213385,28626] 0 2024-09-10T04:05:05.925 INFO:tasks.workunit.client.1.smithi070.stdout:0/62: creat d8/f14 x:0 0 0 2024-09-10T04:05:05.925 INFO:tasks.workunit.client.1.smithi070.stdout:6/127: truncate d0/d3/f19 929809 0 2024-09-10T04:05:05.925 INFO:tasks.workunit.client.1.smithi070.stdout:0/63: read - d8/f10 zero size 2024-09-10T04:05:05.926 INFO:tasks.workunit.client.1.smithi070.stdout:0/64: dread - d8/fd zero size 2024-09-10T04:05:05.928 INFO:tasks.workunit.client.1.smithi070.stdout:6/128: readlink d0/d3/d18/d1b/d17/l24 0 2024-09-10T04:05:05.932 INFO:tasks.workunit.client.1.smithi070.stdout:8/73: dwrite d4/f8 [4194304,4194304] 0 2024-09-10T04:05:05.933 INFO:tasks.workunit.client.1.smithi070.stdout:0/65: symlink d8/l15 0 2024-09-10T04:05:05.937 INFO:tasks.workunit.client.1.smithi070.stdout:0/66: dread f1 [0,4194304] 0 2024-09-10T04:05:05.938 INFO:tasks.workunit.client.1.smithi070.stdout:7/49: link d2/d7/d9/cd d2/c12 0 2024-09-10T04:05:05.941 INFO:tasks.workunit.client.1.smithi070.stdout:8/74: fdatasync d4/f5 0 2024-09-10T04:05:05.944 INFO:tasks.workunit.client.1.smithi070.stdout:1/42: dwrite d3/f7 [0,4194304] 0 2024-09-10T04:05:05.948 INFO:tasks.workunit.client.1.smithi070.stdout:9/56: sync 2024-09-10T04:05:05.948 INFO:tasks.workunit.client.1.smithi070.stdout:3/51: sync 2024-09-10T04:05:05.948 INFO:tasks.workunit.client.1.smithi070.stdout:5/30: sync 2024-09-10T04:05:05.951 INFO:tasks.workunit.client.1.smithi070.stdout:8/75: dwrite d4/fe [0,4194304] 0 2024-09-10T04:05:05.952 INFO:tasks.workunit.client.1.smithi070.stdout:1/43: dread f1 [0,4194304] 0 2024-09-10T04:05:05.965 INFO:tasks.workunit.client.0.smithi046.stdout:2/162: sync 2024-09-10T04:05:05.966 INFO:tasks.workunit.client.1.smithi070.stdout:3/52: mkdir db 0 2024-09-10T04:05:05.967 INFO:tasks.workunit.client.1.smithi070.stdout:9/57: creat d9/dd/f12 x:0 0 0 2024-09-10T04:05:05.967 INFO:tasks.workunit.client.1.smithi070.stdout:9/58: stat l8 0 2024-09-10T04:05:05.968 INFO:tasks.workunit.client.1.smithi070.stdout:8/76: symlink d4/d16/l1b 0 2024-09-10T04:05:05.973 INFO:tasks.workunit.client.1.smithi070.stdout:6/129: link d0/l9 d0/d3/l2c 0 2024-09-10T04:05:05.977 INFO:tasks.workunit.client.1.smithi070.stdout:5/31: mknod c9 0 2024-09-10T04:05:05.980 INFO:tasks.workunit.client.1.smithi070.stdout:2/46: truncate f3 1649428 0 2024-09-10T04:05:05.981 INFO:tasks.workunit.client.1.smithi070.stdout:9/59: symlink d9/dd/d10/l13 0 2024-09-10T04:05:05.988 INFO:tasks.workunit.client.1.smithi070.stdout:9/60: dwrite d9/dd/f12 [0,4194304] 0 2024-09-10T04:05:05.989 INFO:tasks.workunit.client.1.smithi070.stdout:9/61: write d9/dd/f12 [3945708,19993] 0 2024-09-10T04:05:05.990 INFO:tasks.workunit.client.1.smithi070.stdout:7/50: getdents d2/d3 0 2024-09-10T04:05:05.991 INFO:tasks.workunit.client.1.smithi070.stdout:0/67: getdents d8 0 2024-09-10T04:05:05.991 INFO:tasks.workunit.client.1.smithi070.stdout:3/53: mkdir db/dc 0 2024-09-10T04:05:05.993 INFO:tasks.workunit.client.1.smithi070.stdout:2/47: mknod c8 0 2024-09-10T04:05:05.995 INFO:tasks.workunit.client.1.smithi070.stdout:8/77: creat d4/d16/d17/d1a/f1c x:0 0 0 2024-09-10T04:05:05.995 INFO:tasks.workunit.client.1.smithi070.stdout:8/78: chown d4/l18 3922 1 2024-09-10T04:05:05.998 INFO:tasks.workunit.client.1.smithi070.stdout:6/130: creat d0/d3/d18/d1b/d17/d28/f2d x:0 0 0 2024-09-10T04:05:05.999 INFO:tasks.workunit.client.1.smithi070.stdout:6/131: dread - d0/d3/d1f/f22 zero size 2024-09-10T04:05:05.999 INFO:tasks.workunit.client.1.smithi070.stdout:6/132: chown d0/d3/d1f/f22 222 1 2024-09-10T04:05:06.004 INFO:tasks.workunit.client.1.smithi070.stdout:9/62: creat d9/dd/f14 x:0 0 0 2024-09-10T04:05:06.009 INFO:tasks.workunit.client.1.smithi070.stdout:2/48: mknod c9 0 2024-09-10T04:05:06.011 INFO:tasks.workunit.client.1.smithi070.stdout:8/79: creat d4/d16/f1d x:0 0 0 2024-09-10T04:05:06.015 INFO:tasks.workunit.client.1.smithi070.stdout:8/80: dread d4/ff [0,4194304] 0 2024-09-10T04:05:06.015 INFO:tasks.workunit.client.1.smithi070.stdout:9/63: mknod d9/dd/d10/c15 0 2024-09-10T04:05:06.016 INFO:tasks.workunit.client.1.smithi070.stdout:9/64: chown d9/dd/d10 775 1 2024-09-10T04:05:06.019 INFO:tasks.workunit.client.1.smithi070.stdout:0/68: symlink d8/d13/l16 0 2024-09-10T04:05:06.022 INFO:tasks.workunit.client.1.smithi070.stdout:4/45: dwrite f0 [0,4194304] 0 2024-09-10T04:05:06.023 INFO:tasks.workunit.client.1.smithi070.stdout:4/46: write f0 [1839072,111342] 0 2024-09-10T04:05:06.024 INFO:tasks.workunit.client.1.smithi070.stdout:1/44: write d3/f5 [4093514,10315] 0 2024-09-10T04:05:06.025 INFO:tasks.workunit.client.1.smithi070.stdout:8/81: creat d4/f1e x:0 0 0 2024-09-10T04:05:06.026 INFO:tasks.workunit.client.1.smithi070.stdout:9/65: symlink d9/l16 0 2024-09-10T04:05:06.027 INFO:tasks.workunit.client.1.smithi070.stdout:9/66: dread - d9/dd/f14 zero size 2024-09-10T04:05:06.028 INFO:tasks.workunit.client.1.smithi070.stdout:9/67: write d9/dd/f12 [3725466,89414] 0 2024-09-10T04:05:06.029 INFO:tasks.workunit.client.1.smithi070.stdout:3/54: rmdir db/dc 0 2024-09-10T04:05:06.029 INFO:tasks.workunit.client.1.smithi070.stdout:3/55: read - f6 zero size 2024-09-10T04:05:06.033 INFO:tasks.workunit.client.1.smithi070.stdout:3/56: dread f3 [0,4194304] 0 2024-09-10T04:05:06.034 INFO:tasks.workunit.client.1.smithi070.stdout:0/69: unlink d8/f14 0 2024-09-10T04:05:06.041 INFO:tasks.workunit.client.1.smithi070.stdout:1/45: mkdir d3/d8/d13 0 2024-09-10T04:05:06.041 INFO:tasks.workunit.client.1.smithi070.stdout:5/32: write f3 [1835526,40987] 0 2024-09-10T04:05:06.045 INFO:tasks.workunit.client.1.smithi070.stdout:9/68: unlink l8 0 2024-09-10T04:05:06.046 INFO:tasks.workunit.client.1.smithi070.stdout:9/69: write d9/dd/f14 [677928,74722] 0 2024-09-10T04:05:06.046 INFO:tasks.workunit.client.1.smithi070.stdout:9/70: chown d9 1080584 1 2024-09-10T04:05:06.048 INFO:tasks.workunit.client.1.smithi070.stdout:3/57: creat db/fd x:0 0 0 2024-09-10T04:05:06.050 INFO:tasks.workunit.client.1.smithi070.stdout:0/70: symlink d8/d13/l17 0 2024-09-10T04:05:06.050 INFO:tasks.workunit.client.1.smithi070.stdout:0/71: dread - d8/f12 zero size 2024-09-10T04:05:06.055 INFO:tasks.workunit.client.1.smithi070.stdout:6/133: dwrite d0/d3/f16 [0,4194304] 0 2024-09-10T04:05:06.059 INFO:tasks.workunit.client.1.smithi070.stdout:4/47: rename f5 to fd 0 2024-09-10T04:05:06.059 INFO:tasks.workunit.client.1.smithi070.stdout:4/48: chown l9 0 1 2024-09-10T04:05:06.062 INFO:tasks.workunit.client.1.smithi070.stdout:5/33: symlink la 0 2024-09-10T04:05:06.066 INFO:tasks.workunit.client.1.smithi070.stdout:1/46: creat d3/d8/d13/f14 x:0 0 0 2024-09-10T04:05:06.071 INFO:tasks.workunit.client.1.smithi070.stdout:8/82: truncate d4/ff 5126423 0 2024-09-10T04:05:06.078 INFO:tasks.workunit.client.1.smithi070.stdout:1/47: symlink d3/d8/d13/l15 0 2024-09-10T04:05:06.079 INFO:tasks.workunit.client.1.smithi070.stdout:9/71: link d9/dd/lf d9/dd/d10/l17 0 2024-09-10T04:05:06.082 INFO:tasks.workunit.client.1.smithi070.stdout:8/83: rmdir d4/d16 39 2024-09-10T04:05:06.082 INFO:tasks.workunit.client.1.smithi070.stdout:8/84: truncate d4/f1e 44882 0 2024-09-10T04:05:06.084 INFO:tasks.workunit.client.1.smithi070.stdout:4/49: mkdir de 0 2024-09-10T04:05:06.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:05 smithi046 ceph-mon[28328]: pgmap v256: 65 pgs: 65 active+clean; 277 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 7.2 MiB/s wr, 239 op/s 2024-09-10T04:05:06.087 INFO:tasks.workunit.client.1.smithi070.stdout:5/34: mknod cb 0 2024-09-10T04:05:06.087 INFO:tasks.workunit.client.1.smithi070.stdout:5/35: fdatasync f3 0 2024-09-10T04:05:06.090 INFO:tasks.workunit.client.1.smithi070.stdout:6/134: creat d0/f2e x:0 0 0 2024-09-10T04:05:06.091 INFO:tasks.workunit.client.1.smithi070.stdout:1/48: fsync d3/d8/d13/f14 0 2024-09-10T04:05:06.091 INFO:tasks.workunit.client.1.smithi070.stdout:3/58: dwrite f7 [0,4194304] 0 2024-09-10T04:05:06.092 INFO:tasks.workunit.client.1.smithi070.stdout:3/59: chown fa 762623 1 2024-09-10T04:05:06.097 INFO:tasks.workunit.client.1.smithi070.stdout:0/72: getdents d8 0 2024-09-10T04:05:06.098 INFO:tasks.workunit.client.1.smithi070.stdout:0/73: chown d8/fa 196273908 1 2024-09-10T04:05:06.098 INFO:tasks.workunit.client.1.smithi070.stdout:0/74: dread - d8/f12 zero size 2024-09-10T04:05:06.106 INFO:tasks.workunit.client.1.smithi070.stdout:1/49: readlink d3/l11 0 2024-09-10T04:05:06.106 INFO:tasks.workunit.client.1.smithi070.stdout:1/50: chown d3/f7 56383 1 2024-09-10T04:05:06.108 INFO:tasks.workunit.client.1.smithi070.stdout:7/51: sync 2024-09-10T04:05:06.108 INFO:tasks.workunit.client.1.smithi070.stdout:6/135: read - d0/d3/d18/f2b zero size 2024-09-10T04:05:06.108 INFO:tasks.workunit.client.1.smithi070.stdout:6/136: read - d0/d3/d1f/f22 zero size 2024-09-10T04:05:06.110 INFO:tasks.workunit.client.1.smithi070.stdout:3/60: mknod db/ce 0 2024-09-10T04:05:06.111 INFO:tasks.workunit.client.1.smithi070.stdout:4/50: getdents de 0 2024-09-10T04:05:06.114 INFO:tasks.workunit.client.1.smithi070.stdout:0/75: truncate f1 4826075 0 2024-09-10T04:05:06.115 INFO:tasks.workunit.client.1.smithi070.stdout:0/76: write f5 [1541106,37837] 0 2024-09-10T04:05:06.118 INFO:tasks.workunit.client.1.smithi070.stdout:1/51: creat d3/f16 x:0 0 0 2024-09-10T04:05:06.120 INFO:tasks.workunit.client.1.smithi070.stdout:6/137: readlink d0/d3/l1c 0 2024-09-10T04:05:06.122 INFO:tasks.workunit.client.1.smithi070.stdout:2/49: dwrite f3 [0,4194304] 0 2024-09-10T04:05:06.123 INFO:tasks.workunit.client.1.smithi070.stdout:3/61: creat db/ff x:0 0 0 2024-09-10T04:05:06.123 INFO:tasks.workunit.client.1.smithi070.stdout:3/62: read - fa zero size 2024-09-10T04:05:06.123 INFO:tasks.workunit.client.1.smithi070.stdout:2/50: write f3 [3744485,128425] 0 2024-09-10T04:05:06.124 INFO:tasks.workunit.client.1.smithi070.stdout:4/51: chown f8 33950 1 2024-09-10T04:05:06.125 INFO:tasks.workunit.client.1.smithi070.stdout:4/52: chown ca 1 1 2024-09-10T04:05:06.127 INFO:tasks.workunit.client.1.smithi070.stdout:5/36: creat fc x:0 0 0 2024-09-10T04:05:06.127 INFO:tasks.workunit.client.1.smithi070.stdout:2/51: dread f3 [0,4194304] 0 2024-09-10T04:05:06.130 INFO:tasks.workunit.client.1.smithi070.stdout:9/72: getdents d9 0 2024-09-10T04:05:06.131 INFO:tasks.workunit.client.1.smithi070.stdout:7/52: getdents d2/d3/dc 0 2024-09-10T04:05:06.133 INFO:tasks.workunit.client.1.smithi070.stdout:1/52: creat d3/d8/d13/f17 x:0 0 0 2024-09-10T04:05:06.135 INFO:tasks.workunit.client.1.smithi070.stdout:8/85: dwrite d4/ff [0,4194304] 0 2024-09-10T04:05:06.138 INFO:tasks.workunit.client.1.smithi070.stdout:6/138: mknod d0/d3/c2f 0 2024-09-10T04:05:06.138 INFO:tasks.workunit.client.1.smithi070.stdout:7/53: dread d2/d7/fe [0,4194304] 0 2024-09-10T04:05:06.140 INFO:tasks.workunit.client.1.smithi070.stdout:6/139: read - d0/d3/d18/d1b/d17/d28/f2d zero size 2024-09-10T04:05:06.140 INFO:tasks.workunit.client.1.smithi070.stdout:7/54: write d2/d7/fe [3692250,38072] 0 2024-09-10T04:05:06.141 INFO:tasks.workunit.client.1.smithi070.stdout:7/55: readlink d2/l11 0 2024-09-10T04:05:06.142 INFO:tasks.workunit.client.1.smithi070.stdout:3/63: creat db/f10 x:0 0 0 2024-09-10T04:05:06.142 INFO:tasks.workunit.client.1.smithi070.stdout:7/56: write f1 [4936007,73115] 0 2024-09-10T04:05:06.143 INFO:tasks.workunit.client.1.smithi070.stdout:7/57: chown d2/l11 144017508 1 2024-09-10T04:05:06.144 INFO:tasks.workunit.client.1.smithi070.stdout:5/37: unlink la 0 2024-09-10T04:05:06.146 INFO:tasks.workunit.client.1.smithi070.stdout:2/52: creat fa x:0 0 0 2024-09-10T04:05:06.147 INFO:tasks.workunit.client.1.smithi070.stdout:0/77: link d8/fc d8/d13/f18 0 2024-09-10T04:05:06.148 INFO:tasks.workunit.client.0.smithi046.stdout:1/44: sync 2024-09-10T04:05:06.148 INFO:tasks.workunit.client.0.smithi046.stdout:8/116: sync 2024-09-10T04:05:06.148 INFO:tasks.workunit.client.0.smithi046.stdout:6/50: sync 2024-09-10T04:05:06.152 INFO:tasks.workunit.client.1.smithi070.stdout:1/53: unlink d3/d8/d13/l15 0 2024-09-10T04:05:06.154 INFO:tasks.workunit.client.1.smithi070.stdout:8/86: fdatasync d4/d16/d17/d1a/f1c 0 2024-09-10T04:05:06.158 INFO:tasks.workunit.client.1.smithi070.stdout:4/53: symlink de/lf 0 2024-09-10T04:05:06.160 INFO:tasks.workunit.client.1.smithi070.stdout:3/64: creat db/f11 x:0 0 0 2024-09-10T04:05:06.163 INFO:tasks.workunit.client.1.smithi070.stdout:2/53: creat fb x:0 0 0 2024-09-10T04:05:06.164 INFO:tasks.workunit.client.1.smithi070.stdout:0/78: creat d8/d13/f19 x:0 0 0 2024-09-10T04:05:06.165 INFO:tasks.workunit.client.1.smithi070.stdout:2/54: write fa [591922,128505] 0 2024-09-10T04:05:06.165 INFO:tasks.workunit.client.1.smithi070.stdout:0/79: chown d8 13 1 2024-09-10T04:05:06.167 INFO:tasks.workunit.client.1.smithi070.stdout:3/65: dwrite db/fd [0,4194304] 0 2024-09-10T04:05:06.168 INFO:tasks.workunit.client.1.smithi070.stdout:5/38: dread f4 [0,4194304] 0 2024-09-10T04:05:06.173 INFO:tasks.workunit.client.1.smithi070.stdout:8/87: rmdir d4/d16/d17 39 2024-09-10T04:05:06.177 INFO:tasks.workunit.client.1.smithi070.stdout:4/54: mknod de/c10 0 2024-09-10T04:05:06.177 INFO:tasks.workunit.client.1.smithi070.stdout:4/55: dread - fc zero size 2024-09-10T04:05:06.180 INFO:tasks.workunit.client.1.smithi070.stdout:2/55: write f3 [4899541,126361] 0 2024-09-10T04:05:06.182 INFO:tasks.workunit.client.1.smithi070.stdout:0/80: unlink d8/fd 0 2024-09-10T04:05:06.183 INFO:tasks.workunit.client.1.smithi070.stdout:6/140: write d0/d3/d18/d1b/f11 [2333254,61839] 0 2024-09-10T04:05:06.184 INFO:tasks.workunit.client.1.smithi070.stdout:3/66: truncate f6 919517 0 2024-09-10T04:05:06.187 INFO:tasks.workunit.client.1.smithi070.stdout:1/54: link d3/f5 d3/d8/d13/f18 0 2024-09-10T04:05:06.191 INFO:tasks.workunit.client.1.smithi070.stdout:4/56: creat de/f11 x:0 0 0 2024-09-10T04:05:06.192 INFO:tasks.workunit.client.1.smithi070.stdout:2/56: creat fc x:0 0 0 2024-09-10T04:05:06.193 INFO:tasks.workunit.client.1.smithi070.stdout:2/57: write f3 [4043880,52850] 0 2024-09-10T04:05:06.196 INFO:tasks.workunit.client.1.smithi070.stdout:0/81: unlink d8/d13/f18 0 2024-09-10T04:05:06.196 INFO:tasks.workunit.client.1.smithi070.stdout:9/73: truncate d9/dd/f12 3241447 0 2024-09-10T04:05:06.199 INFO:tasks.workunit.client.1.smithi070.stdout:9/74: dread d9/dd/f14 [0,4194304] 0 2024-09-10T04:05:06.200 INFO:tasks.workunit.client.1.smithi070.stdout:3/67: creat db/f12 x:0 0 0 2024-09-10T04:05:06.203 INFO:tasks.workunit.client.1.smithi070.stdout:4/57: chown c2 4 1 2024-09-10T04:05:06.203 INFO:tasks.workunit.client.1.smithi070.stdout:4/58: truncate f8 4571101 0 2024-09-10T04:05:06.206 INFO:tasks.workunit.client.1.smithi070.stdout:2/58: mknod cd 0 2024-09-10T04:05:06.206 INFO:tasks.workunit.client.1.smithi070.stdout:4/59: dread f8 [0,4194304] 0 2024-09-10T04:05:06.207 INFO:tasks.workunit.client.1.smithi070.stdout:0/82: mkdir d8/d13/d1a 0 2024-09-10T04:05:06.209 INFO:tasks.workunit.client.1.smithi070.stdout:8/88: write d4/f5 [10141,91325] 0 2024-09-10T04:05:06.210 INFO:tasks.workunit.client.1.smithi070.stdout:5/39: creat fd x:0 0 0 2024-09-10T04:05:06.211 INFO:tasks.workunit.client.1.smithi070.stdout:6/141: mknod d0/d3/d18/d2a/c30 0 2024-09-10T04:05:06.212 INFO:tasks.workunit.client.1.smithi070.stdout:4/60: dwrite f4 [0,4194304] 0 2024-09-10T04:05:06.213 INFO:tasks.workunit.client.1.smithi070.stdout:4/61: read f4 [1462366,5961] 0 2024-09-10T04:05:06.214 INFO:tasks.workunit.client.1.smithi070.stdout:9/75: creat d9/dd/f18 x:0 0 0 2024-09-10T04:05:06.216 INFO:tasks.workunit.client.1.smithi070.stdout:3/68: rename db/f10 to db/f13 0 2024-09-10T04:05:06.219 INFO:tasks.workunit.client.1.smithi070.stdout:1/55: creat d3/d8/d12/f19 x:0 0 0 2024-09-10T04:05:06.222 INFO:tasks.workunit.client.1.smithi070.stdout:4/62: dwrite fc [0,4194304] 0 2024-09-10T04:05:06.222 INFO:tasks.workunit.client.1.smithi070.stdout:4/63: truncate f6 1936480 0 2024-09-10T04:05:06.225 INFO:tasks.workunit.client.1.smithi070.stdout:0/83: creat d8/f1b x:0 0 0 2024-09-10T04:05:06.226 INFO:tasks.workunit.client.1.smithi070.stdout:0/84: truncate d8/f10 756156 0 2024-09-10T04:05:06.228 INFO:tasks.workunit.client.1.smithi070.stdout:8/89: unlink d4/f1e 0 2024-09-10T04:05:06.234 INFO:tasks.workunit.client.1.smithi070.stdout:8/90: dwrite d4/f10 [0,4194304] 0 2024-09-10T04:05:06.235 INFO:tasks.workunit.client.1.smithi070.stdout:9/76: creat d9/dd/d10/f19 x:0 0 0 2024-09-10T04:05:06.236 INFO:tasks.workunit.client.1.smithi070.stdout:3/69: rmdir db 39 2024-09-10T04:05:06.242 INFO:tasks.workunit.client.1.smithi070.stdout:4/64: rename c2 to de/c12 0 2024-09-10T04:05:06.246 INFO:tasks.workunit.client.1.smithi070.stdout:4/65: dread f0 [0,4194304] 0 2024-09-10T04:05:06.246 INFO:tasks.workunit.client.1.smithi070.stdout:4/66: readlink l9 0 2024-09-10T04:05:06.247 INFO:tasks.workunit.client.1.smithi070.stdout:4/67: fdatasync f6 0 2024-09-10T04:05:06.250 INFO:tasks.workunit.client.1.smithi070.stdout:8/91: rmdir d4/d16 39 2024-09-10T04:05:06.253 INFO:tasks.workunit.client.1.smithi070.stdout:3/70: write db/fd [3381611,30839] 0 2024-09-10T04:05:06.254 INFO:tasks.workunit.client.1.smithi070.stdout:3/71: write f7 [5124184,83618] 0 2024-09-10T04:05:06.257 INFO:tasks.workunit.client.1.smithi070.stdout:0/85: fdatasync d8/fc 0 2024-09-10T04:05:06.258 INFO:tasks.workunit.client.1.smithi070.stdout:3/72: dread f3 [0,4194304] 0 2024-09-10T04:05:06.259 INFO:tasks.workunit.client.1.smithi070.stdout:5/40: rename f4 to fe 0 2024-09-10T04:05:06.260 INFO:tasks.workunit.client.1.smithi070.stdout:5/41: write fd [976519,40895] 0 2024-09-10T04:05:06.262 INFO:tasks.workunit.client.1.smithi070.stdout:4/68: mkdir de/d13 0 2024-09-10T04:05:06.269 INFO:tasks.workunit.client.0.smithi046.stdout:3/53: mknod ce 0 2024-09-10T04:05:06.270 INFO:tasks.workunit.client.1.smithi070.stdout:1/56: dwrite d3/d8/d13/f18 [0,4194304] 0 2024-09-10T04:05:06.271 INFO:tasks.workunit.client.0.smithi046.stdout:9/59: rename d4/d9/f10 to d4/f13 0 2024-09-10T04:05:06.272 INFO:tasks.workunit.client.0.smithi046.stdout:5/56: sync 2024-09-10T04:05:06.272 INFO:tasks.workunit.client.0.smithi046.stdout:4/86: sync 2024-09-10T04:05:06.272 INFO:tasks.workunit.client.0.smithi046.stdout:5/57: read d7/f9 [224684,7780] 0 2024-09-10T04:05:06.273 INFO:tasks.workunit.client.0.smithi046.stdout:2/163: symlink d2/d6/l3f 0 2024-09-10T04:05:06.274 INFO:tasks.workunit.client.0.smithi046.stdout:5/58: truncate d7/f8 5209449 0 2024-09-10T04:05:06.275 INFO:tasks.workunit.client.1.smithi070.stdout:0/86: creat d8/f1c x:0 0 0 2024-09-10T04:05:06.276 INFO:tasks.workunit.client.1.smithi070.stdout:6/142: write d0/d3/f19 [960273,34089] 0 2024-09-10T04:05:06.277 INFO:tasks.workunit.client.1.smithi070.stdout:1/57: dwrite d3/d8/d13/f14 [0,4194304] 0 2024-09-10T04:05:06.277 INFO:tasks.workunit.client.1.smithi070.stdout:6/143: dread - d0/d3/d18/d1b/d17/d28/f2d zero size 2024-09-10T04:05:06.279 INFO:tasks.workunit.client.1.smithi070.stdout:3/73: creat db/f14 x:0 0 0 2024-09-10T04:05:06.281 INFO:tasks.workunit.client.0.smithi046.stdout:2/164: dwrite d2/d6/d23/f33 [0,4194304] 0 2024-09-10T04:05:06.281 INFO:tasks.workunit.client.0.smithi046.stdout:2/165: chown d2/d6 129192504 1 2024-09-10T04:05:06.282 INFO:tasks.workunit.client.1.smithi070.stdout:6/144: dread d0/f8 [0,4194304] 0 2024-09-10T04:05:06.282 INFO:tasks.workunit.client.0.smithi046.stdout:8/117: rename d1/d6 to d1/d29 0 2024-09-10T04:05:06.283 INFO:tasks.workunit.client.1.smithi070.stdout:6/145: truncate d0/f2e 720434 0 2024-09-10T04:05:06.284 INFO:tasks.workunit.client.0.smithi046.stdout:4/87: dread d3/f9 [0,4194304] 0 2024-09-10T04:05:06.289 INFO:tasks.workunit.client.0.smithi046.stdout:9/60: creat d4/f14 x:0 0 0 2024-09-10T04:05:06.291 INFO:tasks.workunit.client.0.smithi046.stdout:2/166: dwrite d2/d8/f29 [0,4194304] 0 2024-09-10T04:05:06.293 INFO:tasks.workunit.client.1.smithi070.stdout:0/87: chown f1 123635 1 2024-09-10T04:05:06.296 INFO:tasks.workunit.client.0.smithi046.stdout:2/167: dread d2/f3 [0,4194304] 0 2024-09-10T04:05:06.296 INFO:tasks.workunit.client.1.smithi070.stdout:1/58: mknod d3/c1a 0 2024-09-10T04:05:06.298 INFO:tasks.workunit.client.1.smithi070.stdout:1/59: dread - d3/d8/d13/f17 zero size 2024-09-10T04:05:06.299 INFO:tasks.workunit.client.1.smithi070.stdout:3/74: rename db/ce to db/c15 0 2024-09-10T04:05:06.303 INFO:tasks.workunit.client.0.smithi046.stdout:1/45: mknod d3/dc/cd 0 2024-09-10T04:05:06.303 INFO:tasks.workunit.client.1.smithi070.stdout:0/88: dwrite d8/f10 [0,4194304] 0 2024-09-10T04:05:06.304 INFO:tasks.workunit.client.0.smithi046.stdout:1/46: readlink d3/l9 0 2024-09-10T04:05:06.305 INFO:tasks.workunit.client.0.smithi046.stdout:2/168: dwrite d2/d6/d15/f3e [0,4194304] 0 2024-09-10T04:05:06.305 INFO:tasks.workunit.client.0.smithi046.stdout:1/47: read d3/fb [752746,127072] 0 2024-09-10T04:05:06.306 INFO:tasks.workunit.client.1.smithi070.stdout:4/69: rename l3 to de/d13/l14 0 2024-09-10T04:05:06.307 INFO:tasks.workunit.client.0.smithi046.stdout:1/48: chown d3/c5 36631 1 2024-09-10T04:05:06.307 INFO:tasks.workunit.client.0.smithi046.stdout:1/49: chown d3/f6 1 1 2024-09-10T04:05:06.310 INFO:tasks.workunit.client.1.smithi070.stdout:9/77: write d9/dd/f12 [3625095,69711] 0 2024-09-10T04:05:06.315 INFO:tasks.workunit.client.0.smithi046.stdout:8/118: write d1/d29/fe [3962664,81042] 0 2024-09-10T04:05:06.316 INFO:tasks.workunit.client.1.smithi070.stdout:1/60: rmdir d3 39 2024-09-10T04:05:06.317 INFO:tasks.workunit.client.0.smithi046.stdout:1/50: dwrite d3/fb [0,4194304] 0 2024-09-10T04:05:06.319 INFO:tasks.workunit.client.0.smithi046.stdout:3/54: link f4 ff 0 2024-09-10T04:05:06.321 INFO:tasks.workunit.client.0.smithi046.stdout:3/55: truncate ff 4520074 0 2024-09-10T04:05:06.323 INFO:tasks.workunit.client.0.smithi046.stdout:4/88: symlink d3/d12/l1d 0 2024-09-10T04:05:06.325 INFO:tasks.workunit.client.1.smithi070.stdout:3/75: mkdir db/d16 0 2024-09-10T04:05:06.325 INFO:tasks.workunit.client.1.smithi070.stdout:3/76: chown db 12742 1 2024-09-10T04:05:06.325 INFO:tasks.workunit.client.1.smithi070.stdout:3/77: chown db/f14 787029890 1 2024-09-10T04:05:06.326 INFO:tasks.workunit.client.0.smithi046.stdout:1/51: dwrite d3/fb [0,4194304] 0 2024-09-10T04:05:06.326 INFO:tasks.workunit.client.1.smithi070.stdout:3/78: write db/f12 [957558,44890] 0 2024-09-10T04:05:06.327 INFO:tasks.workunit.client.0.smithi046.stdout:9/61: creat d4/d9/f15 x:0 0 0 2024-09-10T04:05:06.328 INFO:tasks.workunit.client.0.smithi046.stdout:9/62: stat d4/d9/f15 0 2024-09-10T04:05:06.328 INFO:tasks.workunit.client.0.smithi046.stdout:9/63: chown d4/d7/fe 116448715 1 2024-09-10T04:05:06.329 INFO:tasks.workunit.client.0.smithi046.stdout:9/64: readlink d4/d7/l12 0 2024-09-10T04:05:06.330 INFO:tasks.workunit.client.0.smithi046.stdout:9/65: dread - d4/d7/ff zero size 2024-09-10T04:05:06.331 INFO:tasks.workunit.client.0.smithi046.stdout:5/59: write d7/f9 [358145,116954] 0 2024-09-10T04:05:06.333 INFO:tasks.workunit.client.1.smithi070.stdout:0/89: mknod d8/c1d 0 2024-09-10T04:05:06.335 INFO:tasks.workunit.client.0.smithi046.stdout:5/60: dread f6 [0,4194304] 0 2024-09-10T04:05:06.335 INFO:tasks.workunit.client.0.smithi046.stdout:5/61: chown l5 0 1 2024-09-10T04:05:06.337 INFO:tasks.workunit.client.0.smithi046.stdout:6/51: truncate f1 1941709 0 2024-09-10T04:05:06.338 INFO:tasks.workunit.client.0.smithi046.stdout:0/55: sync 2024-09-10T04:05:06.338 INFO:tasks.workunit.client.0.smithi046.stdout:7/81: sync 2024-09-10T04:05:06.339 INFO:tasks.workunit.client.1.smithi070.stdout:6/146: dwrite d0/d3/d18/d1b/f14 [0,4194304] 0 2024-09-10T04:05:06.343 INFO:tasks.workunit.client.1.smithi070.stdout:1/61: write d3/f16 [173203,15687] 0 2024-09-10T04:05:06.343 INFO:tasks.workunit.client.0.smithi046.stdout:8/119: mkdir d1/d29/d17/d1f/d2a 0 2024-09-10T04:05:06.346 INFO:tasks.workunit.client.1.smithi070.stdout:0/90: fdatasync f0 0 2024-09-10T04:05:06.347 INFO:tasks.workunit.client.1.smithi070.stdout:6/147: dwrite d0/d3/d18/d1b/d17/d28/f2d [0,4194304] 0 2024-09-10T04:05:06.348 INFO:tasks.workunit.client.0.smithi046.stdout:3/56: mknod c10 0 2024-09-10T04:05:06.350 INFO:tasks.workunit.client.1.smithi070.stdout:4/70: write f8 [5536119,72629] 0 2024-09-10T04:05:06.351 INFO:tasks.workunit.client.0.smithi046.stdout:4/89: fdatasync d3/d12/f19 0 2024-09-10T04:05:06.353 INFO:tasks.workunit.client.0.smithi046.stdout:1/52: creat d3/dc/fe x:0 0 0 2024-09-10T04:05:06.354 INFO:tasks.workunit.client.0.smithi046.stdout:4/90: write d3/fc [3146046,30341] 0 2024-09-10T04:05:06.354 INFO:tasks.workunit.client.1.smithi070.stdout:8/92: link d4/c9 d4/d16/c1f 0 2024-09-10T04:05:06.357 INFO:tasks.workunit.client.0.smithi046.stdout:3/57: dwrite f9 [0,4194304] 0 2024-09-10T04:05:06.357 INFO:tasks.workunit.client.0.smithi046.stdout:4/91: dread - d3/f15 zero size 2024-09-10T04:05:06.357 INFO:tasks.workunit.client.0.smithi046.stdout:3/58: fdatasync fb 0 2024-09-10T04:05:06.359 INFO:tasks.workunit.client.0.smithi046.stdout:4/92: readlink d3/l10 0 2024-09-10T04:05:06.359 INFO:tasks.workunit.client.0.smithi046.stdout:3/59: chown ce 745684 1 2024-09-10T04:05:06.360 INFO:tasks.workunit.client.0.smithi046.stdout:3/60: write f3 [5053431,12175] 0 2024-09-10T04:05:06.361 INFO:tasks.workunit.client.0.smithi046.stdout:3/61: chown ce 35505776 1 2024-09-10T04:05:06.362 INFO:tasks.workunit.client.0.smithi046.stdout:3/62: write f6 [255070,115874] 0 2024-09-10T04:05:06.363 INFO:tasks.workunit.client.0.smithi046.stdout:9/66: symlink d4/l16 0 2024-09-10T04:05:06.366 INFO:tasks.workunit.client.0.smithi046.stdout:3/63: fdatasync f6 0 2024-09-10T04:05:06.368 INFO:tasks.workunit.client.1.smithi070.stdout:1/62: creat d3/d8/d13/f1b x:0 0 0 2024-09-10T04:05:06.368 INFO:tasks.workunit.client.1.smithi070.stdout:9/78: dwrite d9/dd/f14 [0,4194304] 0 2024-09-10T04:05:06.368 INFO:tasks.workunit.client.0.smithi046.stdout:9/67: dread d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:06.369 INFO:tasks.workunit.client.1.smithi070.stdout:9/79: dread - d9/dd/f18 zero size 2024-09-10T04:05:06.369 INFO:tasks.workunit.client.0.smithi046.stdout:9/68: readlink d4/d9/ld 0 2024-09-10T04:05:06.371 INFO:tasks.workunit.client.0.smithi046.stdout:9/69: dread - d4/d9/f15 zero size 2024-09-10T04:05:06.372 INFO:tasks.workunit.client.0.smithi046.stdout:3/64: dread ff [0,4194304] 0 2024-09-10T04:05:06.373 INFO:tasks.workunit.client.0.smithi046.stdout:0/56: creat d1/dc/f12 x:0 0 0 2024-09-10T04:05:06.373 INFO:tasks.workunit.client.1.smithi070.stdout:6/148: symlink d0/d3/d18/d1b/l31 0 2024-09-10T04:05:06.375 INFO:tasks.workunit.client.0.smithi046.stdout:0/57: fsync d1/d6/f11 0 2024-09-10T04:05:06.375 INFO:tasks.workunit.client.0.smithi046.stdout:7/82: mkdir d5/d6/db/d11/d1f 0 2024-09-10T04:05:06.376 INFO:tasks.workunit.client.1.smithi070.stdout:4/71: mknod de/d13/c15 0 2024-09-10T04:05:06.377 INFO:tasks.workunit.client.0.smithi046.stdout:6/52: dwrite da/f12 [0,4194304] 0 2024-09-10T04:05:06.378 INFO:tasks.workunit.client.1.smithi070.stdout:7/58: sync 2024-09-10T04:05:06.379 INFO:tasks.workunit.client.0.smithi046.stdout:8/120: rename d1/l9 to d1/d29/d17/d1f/d2a/l2b 0 2024-09-10T04:05:06.381 INFO:tasks.workunit.client.0.smithi046.stdout:8/121: fdatasync d1/fa 0 2024-09-10T04:05:06.382 INFO:tasks.workunit.client.1.smithi070.stdout:6/149: dwrite d0/d3/d18/f27 [0,4194304] 0 2024-09-10T04:05:06.385 INFO:tasks.workunit.client.0.smithi046.stdout:8/122: chown d1/d29/d17/f23 76309 1 2024-09-10T04:05:06.385 INFO:tasks.workunit.client.0.smithi046.stdout:3/65: dwrite f3 [0,4194304] 0 2024-09-10T04:05:06.386 INFO:tasks.workunit.client.1.smithi070.stdout:0/91: symlink d8/d13/d1a/l1e 0 2024-09-10T04:05:06.388 INFO:tasks.workunit.client.0.smithi046.stdout:4/93: symlink d3/d12/l1e 0 2024-09-10T04:05:06.389 INFO:tasks.workunit.client.1.smithi070.stdout:8/93: symlink d4/d16/d17/l20 0 2024-09-10T04:05:06.390 INFO:tasks.workunit.client.0.smithi046.stdout:4/94: readlink d3/d12/l1e 0 2024-09-10T04:05:06.392 INFO:tasks.workunit.client.0.smithi046.stdout:6/53: dwrite da/f12 [4194304,4194304] 0 2024-09-10T04:05:06.393 INFO:tasks.workunit.client.1.smithi070.stdout:3/79: getdents db 0 2024-09-10T04:05:06.394 INFO:tasks.workunit.client.1.smithi070.stdout:3/80: chown c4 2139095480 1 2024-09-10T04:05:06.396 INFO:tasks.workunit.client.1.smithi070.stdout:4/72: rename f4 to de/d13/f16 0 2024-09-10T04:05:06.397 INFO:tasks.workunit.client.1.smithi070.stdout:8/94: dwrite d4/fa [0,4194304] 0 2024-09-10T04:05:06.398 INFO:tasks.workunit.client.0.smithi046.stdout:3/66: dread f3 [0,4194304] 0 2024-09-10T04:05:06.398 INFO:tasks.workunit.client.1.smithi070.stdout:8/95: chown d4/f15 202706463 1 2024-09-10T04:05:06.401 INFO:tasks.workunit.client.0.smithi046.stdout:5/62: link d7/f9 d7/fd 0 2024-09-10T04:05:06.407 INFO:tasks.workunit.client.0.smithi046.stdout:9/70: creat d4/d7/f17 x:0 0 0 2024-09-10T04:05:06.409 INFO:tasks.workunit.client.1.smithi070.stdout:6/150: symlink d0/d3/d18/d2a/l32 0 2024-09-10T04:05:06.409 INFO:tasks.workunit.client.1.smithi070.stdout:4/73: dread f6 [0,4194304] 0 2024-09-10T04:05:06.413 INFO:tasks.workunit.client.1.smithi070.stdout:3/81: symlink db/l17 0 2024-09-10T04:05:06.413 INFO:tasks.workunit.client.1.smithi070.stdout:6/151: dread d0/f8 [0,4194304] 0 2024-09-10T04:05:06.415 INFO:tasks.workunit.client.1.smithi070.stdout:8/96: rename f3 to d4/d16/d17/d1a/f21 0 2024-09-10T04:05:06.418 INFO:tasks.workunit.client.0.smithi046.stdout:2/169: rmdir d2/d6/d15/d16/d3c 0 2024-09-10T04:05:06.418 INFO:tasks.workunit.client.1.smithi070.stdout:7/59: mkdir d2/d3/dc/d13 0 2024-09-10T04:05:06.420 INFO:tasks.workunit.client.1.smithi070.stdout:9/80: truncate d9/dd/f12 1340447 0 2024-09-10T04:05:06.421 INFO:tasks.workunit.client.1.smithi070.stdout:4/74: mknod de/c17 0 2024-09-10T04:05:06.423 INFO:tasks.workunit.client.1.smithi070.stdout:0/92: write f1 [2609379,98869] 0 2024-09-10T04:05:06.425 INFO:tasks.workunit.client.1.smithi070.stdout:6/152: mkdir d0/d3/d18/d1b/d33 0 2024-09-10T04:05:06.426 INFO:tasks.workunit.client.1.smithi070.stdout:6/153: dread d0/f2e [0,4194304] 0 2024-09-10T04:05:06.427 INFO:tasks.workunit.client.1.smithi070.stdout:6/154: stat d0/d3/c13 0 2024-09-10T04:05:06.427 INFO:tasks.workunit.client.0.smithi046.stdout:8/123: creat d1/d29/d17/d1f/d2a/f2c x:0 0 0 2024-09-10T04:05:06.427 INFO:tasks.workunit.client.1.smithi070.stdout:6/155: write d0/d3/f19 [1439602,101329] 0 2024-09-10T04:05:06.428 INFO:tasks.workunit.client.1.smithi070.stdout:6/156: readlink d0/d3/d18/d1b/l31 0 2024-09-10T04:05:06.428 INFO:tasks.workunit.client.1.smithi070.stdout:7/60: mknod d2/d7/db/c14 0 2024-09-10T04:05:06.430 INFO:tasks.workunit.client.0.smithi046.stdout:4/95: unlink d3/cd 0 2024-09-10T04:05:06.431 INFO:tasks.workunit.client.1.smithi070.stdout:1/63: getdents d3/d8/d12 0 2024-09-10T04:05:06.431 INFO:tasks.workunit.client.1.smithi070.stdout:1/64: stat d3/d8/fc 0 2024-09-10T04:05:06.432 INFO:tasks.workunit.client.1.smithi070.stdout:1/65: fdatasync d3/f16 0 2024-09-10T04:05:06.433 INFO:tasks.workunit.client.1.smithi070.stdout:9/81: creat d9/dd/f1a x:0 0 0 2024-09-10T04:05:06.434 INFO:tasks.workunit.client.1.smithi070.stdout:9/82: write d9/dd/f18 [209194,119393] 0 2024-09-10T04:05:06.435 INFO:tasks.workunit.client.1.smithi070.stdout:4/75: mkdir de/d13/d18 0 2024-09-10T04:05:06.437 INFO:tasks.workunit.client.1.smithi070.stdout:3/82: unlink f6 0 2024-09-10T04:05:06.439 INFO:tasks.workunit.client.1.smithi070.stdout:0/93: creat d8/d13/f1f x:0 0 0 2024-09-10T04:05:06.440 INFO:tasks.workunit.client.1.smithi070.stdout:0/94: write d8/d13/f1f [6805,3955] 0 2024-09-10T04:05:06.440 INFO:tasks.workunit.client.0.smithi046.stdout:9/71: mkdir d4/d18 0 2024-09-10T04:05:06.440 INFO:tasks.workunit.client.0.smithi046.stdout:9/72: fsync d4/d7/ff 0 2024-09-10T04:05:06.442 INFO:tasks.workunit.client.0.smithi046.stdout:0/58: mkdir d1/d13 0 2024-09-10T04:05:06.444 INFO:tasks.workunit.client.0.smithi046.stdout:7/83: mkdir d5/d6/db/d11/d1e/d20 0 2024-09-10T04:05:06.444 INFO:tasks.workunit.client.0.smithi046.stdout:3/67: write f4 [1699993,49228] 0 2024-09-10T04:05:06.444 INFO:tasks.workunit.client.1.smithi070.stdout:2/59: sync 2024-09-10T04:05:06.445 INFO:tasks.workunit.client.1.smithi070.stdout:5/42: sync 2024-09-10T04:05:06.445 INFO:tasks.workunit.client.1.smithi070.stdout:5/43: chown cb 331478757 1 2024-09-10T04:05:06.445 INFO:tasks.workunit.client.0.smithi046.stdout:3/68: readlink lc 0 2024-09-10T04:05:06.445 INFO:tasks.workunit.client.1.smithi070.stdout:5/44: chown cb 106 1 2024-09-10T04:05:06.445 INFO:tasks.workunit.client.1.smithi070.stdout:2/60: read fa [132782,94921] 0 2024-09-10T04:05:06.446 INFO:tasks.workunit.client.1.smithi070.stdout:2/61: chown l6 8524 1 2024-09-10T04:05:06.446 INFO:tasks.workunit.client.1.smithi070.stdout:2/62: chown c9 906304 1 2024-09-10T04:05:06.448 INFO:tasks.workunit.client.1.smithi070.stdout:7/61: unlink f0 0 2024-09-10T04:05:06.450 INFO:tasks.workunit.client.1.smithi070.stdout:2/63: dread f3 [0,4194304] 0 2024-09-10T04:05:06.450 INFO:tasks.workunit.client.1.smithi070.stdout:2/64: truncate fb 630103 0 2024-09-10T04:05:06.451 INFO:tasks.workunit.client.0.smithi046.stdout:6/54: mkdir da/db/d11/d13 0 2024-09-10T04:05:06.451 INFO:tasks.workunit.client.0.smithi046.stdout:0/59: dwrite d1/d6/f11 [0,4194304] 0 2024-09-10T04:05:06.451 INFO:tasks.workunit.client.1.smithi070.stdout:2/65: write fb [1239895,69288] 0 2024-09-10T04:05:06.452 INFO:tasks.workunit.client.1.smithi070.stdout:2/66: fsync fa 0 2024-09-10T04:05:06.452 INFO:tasks.workunit.client.1.smithi070.stdout:1/66: symlink d3/d8/l1c 0 2024-09-10T04:05:06.452 INFO:tasks.workunit.client.1.smithi070.stdout:2/67: truncate fa 1557207 0 2024-09-10T04:05:06.452 INFO:tasks.workunit.client.1.smithi070.stdout:1/67: read - d3/d8/d12/f19 zero size 2024-09-10T04:05:06.454 INFO:tasks.workunit.client.1.smithi070.stdout:9/83: mkdir d9/dd/d10/d1b 0 2024-09-10T04:05:06.455 INFO:tasks.workunit.client.0.smithi046.stdout:5/63: dwrite f6 [0,4194304] 0 2024-09-10T04:05:06.455 INFO:tasks.workunit.client.1.smithi070.stdout:9/84: chown d9/dd/d10/c11 45934 1 2024-09-10T04:05:06.455 INFO:tasks.workunit.client.1.smithi070.stdout:6/157: write d0/f10 [841040,52095] 0 2024-09-10T04:05:06.457 INFO:tasks.workunit.client.1.smithi070.stdout:3/83: fsync db/f13 0 2024-09-10T04:05:06.458 INFO:tasks.workunit.client.1.smithi070.stdout:3/84: chown db/f14 113 1 2024-09-10T04:05:06.458 INFO:tasks.workunit.client.0.smithi046.stdout:1/53: getdents d3 0 2024-09-10T04:05:06.459 INFO:tasks.workunit.client.1.smithi070.stdout:3/85: fdatasync db/f13 0 2024-09-10T04:05:06.460 INFO:tasks.workunit.client.1.smithi070.stdout:2/68: dwrite fb [0,4194304] 0 2024-09-10T04:05:06.461 INFO:tasks.workunit.client.1.smithi070.stdout:3/86: read - db/f11 zero size 2024-09-10T04:05:06.461 INFO:tasks.workunit.client.0.smithi046.stdout:1/54: read - d3/dc/fe zero size 2024-09-10T04:05:06.462 INFO:tasks.workunit.client.1.smithi070.stdout:0/95: rename d8/f1c to d8/d13/f20 0 2024-09-10T04:05:06.462 INFO:tasks.workunit.client.1.smithi070.stdout:2/69: read fa [229440,107195] 0 2024-09-10T04:05:06.463 INFO:tasks.workunit.client.0.smithi046.stdout:7/84: mknod d5/c21 0 2024-09-10T04:05:06.464 INFO:tasks.workunit.client.0.smithi046.stdout:3/69: mknod c11 0 2024-09-10T04:05:06.466 INFO:tasks.workunit.client.1.smithi070.stdout:8/97: creat d4/d16/f22 x:0 0 0 2024-09-10T04:05:06.466 INFO:tasks.workunit.client.0.smithi046.stdout:0/60: dwrite d1/d6/d3/f10 [0,4194304] 0 2024-09-10T04:05:06.468 INFO:tasks.workunit.client.0.smithi046.stdout:0/61: write d1/d6/d3/f10 [1551010,10311] 0 2024-09-10T04:05:06.468 INFO:tasks.workunit.client.0.smithi046.stdout:6/55: creat da/db/f14 x:0 0 0 2024-09-10T04:05:06.470 INFO:tasks.workunit.client.0.smithi046.stdout:6/56: read - da/db/f14 zero size 2024-09-10T04:05:06.470 INFO:tasks.workunit.client.0.smithi046.stdout:9/73: creat d4/d18/f19 x:0 0 0 2024-09-10T04:05:06.470 INFO:tasks.workunit.client.1.smithi070.stdout:3/87: dwrite db/fd [4194304,4194304] 0 2024-09-10T04:05:06.471 INFO:tasks.workunit.client.1.smithi070.stdout:3/88: dread - db/f13 zero size 2024-09-10T04:05:06.479 INFO:tasks.workunit.client.0.smithi046.stdout:1/55: rename d3/f4 to d3/ff 0 2024-09-10T04:05:06.479 INFO:tasks.workunit.client.0.smithi046.stdout:1/56: readlink d3/l7 0 2024-09-10T04:05:06.480 INFO:tasks.workunit.client.0.smithi046.stdout:1/57: stat d3/dc 0 2024-09-10T04:05:06.482 INFO:tasks.workunit.client.0.smithi046.stdout:9/74: dwrite d4/d7/ff [0,4194304] 0 2024-09-10T04:05:06.482 INFO:tasks.workunit.client.0.smithi046.stdout:7/85: readlink d5/d6/db/l19 0 2024-09-10T04:05:06.482 INFO:tasks.workunit.client.0.smithi046.stdout:9/75: stat d4/d9/f15 0 2024-09-10T04:05:06.483 INFO:tasks.workunit.client.1.smithi070.stdout:3/89: fdatasync db/fd 0 2024-09-10T04:05:06.484 INFO:tasks.workunit.client.0.smithi046.stdout:3/70: mkdir d12 0 2024-09-10T04:05:06.488 INFO:tasks.workunit.client.1.smithi070.stdout:4/76: symlink de/d13/d18/l19 0 2024-09-10T04:05:06.488 INFO:tasks.workunit.client.1.smithi070.stdout:6/158: mknod d0/d3/d18/d1b/d17/d28/c34 0 2024-09-10T04:05:06.490 INFO:tasks.workunit.client.0.smithi046.stdout:0/62: creat d1/d6/d3/f14 x:0 0 0 2024-09-10T04:05:06.492 INFO:tasks.workunit.client.0.smithi046.stdout:6/57: chown da/dc/cf 50165 1 2024-09-10T04:05:06.495 INFO:tasks.workunit.client.0.smithi046.stdout:5/64: link f6 d7/fe 0 2024-09-10T04:05:06.495 INFO:tasks.workunit.client.1.smithi070.stdout:0/96: rmdir d8/d13/d1a 39 2024-09-10T04:05:06.495 INFO:tasks.workunit.client.1.smithi070.stdout:0/97: stat d8/fc 0 2024-09-10T04:05:06.499 INFO:tasks.workunit.client.0.smithi046.stdout:0/63: dwrite d1/d6/d3/f14 [0,4194304] 0 2024-09-10T04:05:06.501 INFO:tasks.workunit.client.0.smithi046.stdout:7/86: creat d5/d6/db/f22 x:0 0 0 2024-09-10T04:05:06.502 INFO:tasks.workunit.client.0.smithi046.stdout:1/58: write f1 [1436302,110755] 0 2024-09-10T04:05:06.504 INFO:tasks.workunit.client.0.smithi046.stdout:1/59: chown d3/dc/fe 900 1 2024-09-10T04:05:06.504 INFO:tasks.workunit.client.0.smithi046.stdout:0/64: dread d1/d6/f11 [0,4194304] 0 2024-09-10T04:05:06.505 INFO:tasks.workunit.client.0.smithi046.stdout:0/65: chown d1/dc 177600 1 2024-09-10T04:05:06.508 INFO:tasks.workunit.client.0.smithi046.stdout:8/124: getdents d1/d29/d17/d1f 0 2024-09-10T04:05:06.509 INFO:tasks.workunit.client.0.smithi046.stdout:9/76: dread d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:06.510 INFO:tasks.workunit.client.1.smithi070.stdout:3/90: unlink db/f13 0 2024-09-10T04:05:06.510 INFO:tasks.workunit.client.0.smithi046.stdout:1/60: dread d3/fb [0,4194304] 0 2024-09-10T04:05:06.511 INFO:tasks.workunit.client.0.smithi046.stdout:9/77: read d4/d7/fe [811635,80478] 0 2024-09-10T04:05:06.511 INFO:tasks.workunit.client.0.smithi046.stdout:4/96: getdents d3/d12/d14 0 2024-09-10T04:05:06.512 INFO:tasks.workunit.client.0.smithi046.stdout:9/78: fdatasync d4/f14 0 2024-09-10T04:05:06.513 INFO:tasks.workunit.client.1.smithi070.stdout:1/68: mkdir d3/d10/d1d 0 2024-09-10T04:05:06.513 INFO:tasks.workunit.client.0.smithi046.stdout:6/58: truncate f9 613223 0 2024-09-10T04:05:06.514 INFO:tasks.workunit.client.0.smithi046.stdout:6/59: fsync da/db/f14 0 2024-09-10T04:05:06.514 INFO:tasks.workunit.client.1.smithi070.stdout:4/77: creat de/d13/f1a x:0 0 0 2024-09-10T04:05:06.514 INFO:tasks.workunit.client.0.smithi046.stdout:6/60: read da/fe [570378,58317] 0 2024-09-10T04:05:06.515 INFO:tasks.workunit.client.0.smithi046.stdout:6/61: dread - da/db/f14 zero size 2024-09-10T04:05:06.515 INFO:tasks.workunit.client.1.smithi070.stdout:6/159: creat d0/d3/d18/f35 x:0 0 0 2024-09-10T04:05:06.516 INFO:tasks.workunit.client.0.smithi046.stdout:5/65: symlink d7/lf 0 2024-09-10T04:05:06.516 INFO:tasks.workunit.client.0.smithi046.stdout:5/66: stat d7/fe 0 2024-09-10T04:05:06.519 INFO:tasks.workunit.client.1.smithi070.stdout:8/98: fsync d4/d16/d17/d1a/f21 0 2024-09-10T04:05:06.519 INFO:tasks.workunit.client.1.smithi070.stdout:5/45: creat ff x:0 0 0 2024-09-10T04:05:06.519 INFO:tasks.workunit.client.1.smithi070.stdout:5/46: rmdir - no directory 2024-09-10T04:05:06.520 INFO:tasks.workunit.client.1.smithi070.stdout:5/47: rmdir - no directory 2024-09-10T04:05:06.521 INFO:tasks.workunit.client.0.smithi046.stdout:3/71: mkdir d12/d13 0 2024-09-10T04:05:06.523 INFO:tasks.workunit.client.0.smithi046.stdout:8/125: creat d1/d29/d17/d1f/d2a/f2d x:0 0 0 2024-09-10T04:05:06.523 INFO:tasks.workunit.client.0.smithi046.stdout:8/126: fsync d1/d29/d17/d1f/d2a/f2c 0 2024-09-10T04:05:06.525 INFO:tasks.workunit.client.0.smithi046.stdout:8/127: chown d1/d29/d17/d1f/d2a/f2c 3559 1 2024-09-10T04:05:06.527 INFO:tasks.workunit.client.1.smithi070.stdout:4/78: rename l9 to de/d13/d18/l1b 0 2024-09-10T04:05:06.528 INFO:tasks.workunit.client.1.smithi070.stdout:6/160: dwrite d0/f10 [0,4194304] 0 2024-09-10T04:05:06.529 INFO:tasks.workunit.client.1.smithi070.stdout:4/79: read - fb zero size 2024-09-10T04:05:06.530 INFO:tasks.workunit.client.0.smithi046.stdout:1/61: rename f0 to d3/dc/f10 0 2024-09-10T04:05:06.531 INFO:tasks.workunit.client.1.smithi070.stdout:6/161: stat d0/d3/d18/d1b/d17 0 2024-09-10T04:05:06.532 INFO:tasks.workunit.client.1.smithi070.stdout:4/80: read f6 [1707238,53205] 0 2024-09-10T04:05:06.532 INFO:tasks.workunit.client.1.smithi070.stdout:8/99: dwrite d4/d16/f1d [0,4194304] 0 2024-09-10T04:05:06.532 INFO:tasks.workunit.client.0.smithi046.stdout:4/97: creat d3/d12/f1f x:0 0 0 2024-09-10T04:05:06.533 INFO:tasks.workunit.client.1.smithi070.stdout:4/81: chown de/c17 441824381 1 2024-09-10T04:05:06.533 INFO:tasks.workunit.client.1.smithi070.stdout:3/91: dwrite f3 [0,4194304] 0 2024-09-10T04:05:06.534 INFO:tasks.workunit.client.1.smithi070.stdout:0/98: creat d8/d13/d1a/f21 x:0 0 0 2024-09-10T04:05:06.534 INFO:tasks.workunit.client.1.smithi070.stdout:4/82: chown de/lf 98345 1 2024-09-10T04:05:06.535 INFO:tasks.workunit.client.0.smithi046.stdout:9/79: fsync d4/f6 0 2024-09-10T04:05:06.536 INFO:tasks.workunit.client.1.smithi070.stdout:4/83: chown de/f11 472390 1 2024-09-10T04:05:06.537 INFO:tasks.workunit.client.1.smithi070.stdout:0/99: write d8/f1b [534148,123166] 0 2024-09-10T04:05:06.538 INFO:tasks.workunit.client.1.smithi070.stdout:0/100: chown d8/fc 0 1 2024-09-10T04:05:06.539 INFO:tasks.workunit.client.1.smithi070.stdout:7/62: getdents d2 0 2024-09-10T04:05:06.541 INFO:tasks.workunit.client.0.smithi046.stdout:9/80: dread d4/d7/ff [0,4194304] 0 2024-09-10T04:05:06.542 INFO:tasks.workunit.client.0.smithi046.stdout:9/81: readlink l1 0 2024-09-10T04:05:06.543 INFO:tasks.workunit.client.0.smithi046.stdout:9/82: chown d4/f13 3 1 2024-09-10T04:05:06.543 INFO:tasks.workunit.client.0.smithi046.stdout:5/67: creat d7/f10 x:0 0 0 2024-09-10T04:05:06.548 INFO:tasks.workunit.client.0.smithi046.stdout:9/83: dread d4/f6 [0,4194304] 0 2024-09-10T04:05:06.549 INFO:tasks.workunit.client.1.smithi070.stdout:8/100: mkdir d4/d16/d17/d1a/d23 0 2024-09-10T04:05:06.550 INFO:tasks.workunit.client.1.smithi070.stdout:1/69: dwrite f1 [0,4194304] 0 2024-09-10T04:05:06.551 INFO:tasks.workunit.client.1.smithi070.stdout:8/101: read d4/f5 [67873,90650] 0 2024-09-10T04:05:06.551 INFO:tasks.workunit.client.0.smithi046.stdout:7/87: truncate d5/d9/fe 231502 0 2024-09-10T04:05:06.551 INFO:tasks.workunit.client.0.smithi046.stdout:7/88: stat d5/d6/db 0 2024-09-10T04:05:06.553 INFO:tasks.workunit.client.1.smithi070.stdout:3/92: creat db/f18 x:0 0 0 2024-09-10T04:05:06.558 INFO:tasks.workunit.client.1.smithi070.stdout:4/84: stat fd 0 2024-09-10T04:05:06.558 INFO:tasks.workunit.client.0.smithi046.stdout:7/89: dwrite f2 [0,4194304] 0 2024-09-10T04:05:06.559 INFO:tasks.workunit.client.0.smithi046.stdout:7/90: stat d5/c13 0 2024-09-10T04:05:06.560 INFO:tasks.workunit.client.1.smithi070.stdout:8/102: dwrite d4/f5 [0,4194304] 0 2024-09-10T04:05:06.560 INFO:tasks.workunit.client.1.smithi070.stdout:8/103: stat d4/fe 0 2024-09-10T04:05:06.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/93: dread f3 [0,4194304] 0 2024-09-10T04:05:06.564 INFO:tasks.workunit.client.1.smithi070.stdout:7/63: creat d2/f15 x:0 0 0 2024-09-10T04:05:06.567 INFO:tasks.workunit.client.0.smithi046.stdout:7/91: dread f4 [0,4194304] 0 2024-09-10T04:05:06.572 INFO:tasks.workunit.client.0.smithi046.stdout:1/62: dwrite d3/ff [0,4194304] 0 2024-09-10T04:05:06.575 INFO:tasks.workunit.client.0.smithi046.stdout:8/128: write d1/d29/d12/f16 [38521,33163] 0 2024-09-10T04:05:06.577 INFO:tasks.workunit.client.0.smithi046.stdout:5/68: mkdir d7/d11 0 2024-09-10T04:05:06.577 INFO:tasks.workunit.client.0.smithi046.stdout:5/69: write d7/f10 [931592,832] 0 2024-09-10T04:05:06.581 INFO:tasks.workunit.client.1.smithi070.stdout:4/85: truncate fd 1241333 0 2024-09-10T04:05:06.582 INFO:tasks.workunit.client.1.smithi070.stdout:3/94: symlink db/l19 0 2024-09-10T04:05:06.582 INFO:tasks.workunit.client.1.smithi070.stdout:4/86: write de/d13/f1a [501334,5114] 0 2024-09-10T04:05:06.583 INFO:tasks.workunit.client.1.smithi070.stdout:4/87: dread - fb zero size 2024-09-10T04:05:06.583 INFO:tasks.workunit.client.1.smithi070.stdout:3/95: write fa [888192,56118] 0 2024-09-10T04:05:06.585 INFO:tasks.workunit.client.1.smithi070.stdout:5/48: creat f10 x:0 0 0 2024-09-10T04:05:06.587 INFO:tasks.workunit.client.0.smithi046.stdout:7/92: mkdir d5/d6/d23 0 2024-09-10T04:05:06.587 INFO:tasks.workunit.client.0.smithi046.stdout:7/93: readlink d5/d6/db/l19 0 2024-09-10T04:05:06.588 INFO:tasks.workunit.client.0.smithi046.stdout:7/94: readlink d5/d9/l16 0 2024-09-10T04:05:06.589 INFO:tasks.workunit.client.0.smithi046.stdout:7/95: write d5/d6/db/d11/f14 [577294,13285] 0 2024-09-10T04:05:06.589 INFO:tasks.workunit.client.1.smithi070.stdout:6/162: link d0/d3/d18/d1b/d17/c25 d0/d3/d18/c36 0 2024-09-10T04:05:06.591 INFO:tasks.workunit.client.1.smithi070.stdout:0/101: creat d8/d13/f22 x:0 0 0 2024-09-10T04:05:06.592 INFO:tasks.workunit.client.1.smithi070.stdout:8/104: creat d4/d16/d17/d1a/d23/f24 x:0 0 0 2024-09-10T04:05:06.593 INFO:tasks.workunit.client.0.smithi046.stdout:6/62: link c4 da/db/d10/c15 0 2024-09-10T04:05:06.593 INFO:tasks.workunit.client.1.smithi070.stdout:7/64: symlink d2/d7/d9/l16 0 2024-09-10T04:05:06.594 INFO:tasks.workunit.client.0.smithi046.stdout:8/129: creat d1/f2e x:0 0 0 2024-09-10T04:05:06.596 INFO:tasks.workunit.client.0.smithi046.stdout:3/72: truncate f7 207964 0 2024-09-10T04:05:06.598 INFO:tasks.workunit.client.1.smithi070.stdout:1/70: link d3/c1a d3/d8/c1e 0 2024-09-10T04:05:06.599 INFO:tasks.workunit.client.1.smithi070.stdout:1/71: fsync d3/d8/d12/f19 0 2024-09-10T04:05:06.599 INFO:tasks.workunit.client.0.smithi046.stdout:7/96: unlink f0 0 2024-09-10T04:05:06.600 INFO:tasks.workunit.client.0.smithi046.stdout:7/97: chown f2 27663 1 2024-09-10T04:05:06.600 INFO:tasks.workunit.client.1.smithi070.stdout:6/163: creat d0/d3/d1f/f37 x:0 0 0 2024-09-10T04:05:06.602 INFO:tasks.workunit.client.1.smithi070.stdout:0/102: rename f1 to d8/d13/d1a/f23 0 2024-09-10T04:05:06.602 INFO:tasks.workunit.client.1.smithi070.stdout:1/72: dread d3/d8/d13/f14 [0,4194304] 0 2024-09-10T04:05:06.603 INFO:tasks.workunit.client.1.smithi070.stdout:1/73: chown d3/f7 15 1 2024-09-10T04:05:06.604 INFO:tasks.workunit.client.0.smithi046.stdout:8/130: creat d1/d29/d17/f2f x:0 0 0 2024-09-10T04:05:06.606 INFO:tasks.workunit.client.0.smithi046.stdout:8/131: truncate d1/d29/f27 800002 0 2024-09-10T04:05:06.606 INFO:tasks.workunit.client.0.smithi046.stdout:5/70: creat d7/d11/f12 x:0 0 0 2024-09-10T04:05:06.608 INFO:tasks.workunit.client.0.smithi046.stdout:9/84: truncate d4/f6 4491923 0 2024-09-10T04:05:06.614 INFO:tasks.workunit.client.0.smithi046.stdout:6/63: truncate f1 592587 0 2024-09-10T04:05:06.616 INFO:tasks.workunit.client.0.smithi046.stdout:8/132: rename d1/c4 to d1/c30 0 2024-09-10T04:05:06.617 INFO:tasks.workunit.client.0.smithi046.stdout:5/71: creat d7/d11/f13 x:0 0 0 2024-09-10T04:05:06.618 INFO:tasks.workunit.client.1.smithi070.stdout:0/103: mkdir d8/d24 0 2024-09-10T04:05:06.619 INFO:tasks.workunit.client.0.smithi046.stdout:3/73: mkdir d12/d13/d14 0 2024-09-10T04:05:06.620 INFO:tasks.workunit.client.1.smithi070.stdout:1/74: mkdir d3/d8/d12/d1f 0 2024-09-10T04:05:06.623 INFO:tasks.workunit.client.1.smithi070.stdout:8/105: link d4/ff d4/d16/d17/d19/f25 0 2024-09-10T04:05:06.625 INFO:tasks.workunit.client.1.smithi070.stdout:6/164: write d0/d3/d18/f2b [708452,129561] 0 2024-09-10T04:05:06.627 INFO:tasks.workunit.client.1.smithi070.stdout:6/165: fsync d0/d3/d18/d1b/f14 0 2024-09-10T04:05:06.627 INFO:tasks.workunit.client.0.smithi046.stdout:7/98: mkdir d5/d6/db/dd/d24 0 2024-09-10T04:05:06.628 INFO:tasks.workunit.client.1.smithi070.stdout:3/96: link f3 db/f1a 0 2024-09-10T04:05:06.629 INFO:tasks.workunit.client.1.smithi070.stdout:5/49: creat f11 x:0 0 0 2024-09-10T04:05:06.630 INFO:tasks.workunit.client.1.smithi070.stdout:5/50: dread - f10 zero size 2024-09-10T04:05:06.630 INFO:tasks.workunit.client.0.smithi046.stdout:6/64: creat da/db/f16 x:0 0 0 2024-09-10T04:05:06.632 INFO:tasks.workunit.client.0.smithi046.stdout:8/133: creat d1/d29/d17/d1f/f31 x:0 0 0 2024-09-10T04:05:06.633 INFO:tasks.workunit.client.0.smithi046.stdout:8/134: dread - d1/d29/d17/f23 zero size 2024-09-10T04:05:06.633 INFO:tasks.workunit.client.0.smithi046.stdout:8/135: readlink d1/l28 0 2024-09-10T04:05:06.635 INFO:tasks.workunit.client.0.smithi046.stdout:5/72: fdatasync f6 0 2024-09-10T04:05:06.636 INFO:tasks.workunit.client.0.smithi046.stdout:5/73: chown l5 5 1 2024-09-10T04:05:06.636 INFO:tasks.workunit.client.1.smithi070.stdout:8/106: unlink d4/f5 0 2024-09-10T04:05:06.637 INFO:tasks.workunit.client.1.smithi070.stdout:8/107: stat d4/l18 0 2024-09-10T04:05:06.637 INFO:tasks.workunit.client.0.smithi046.stdout:5/74: chown d7/lf 21104 1 2024-09-10T04:05:06.637 INFO:tasks.workunit.client.0.smithi046.stdout:5/75: fsync d7/fc 0 2024-09-10T04:05:06.638 INFO:tasks.workunit.client.1.smithi070.stdout:8/108: write d4/f13 [627220,61525] 0 2024-09-10T04:05:06.640 INFO:tasks.workunit.client.1.smithi070.stdout:6/166: mkdir d0/d3/d18/d1b/d38 0 2024-09-10T04:05:06.643 INFO:tasks.workunit.client.1.smithi070.stdout:8/109: dread d4/d16/d17/d19/f25 [0,4194304] 0 2024-09-10T04:05:06.644 INFO:tasks.workunit.client.1.smithi070.stdout:8/110: write d4/f11 [683316,21331] 0 2024-09-10T04:05:06.646 INFO:tasks.workunit.client.0.smithi046.stdout:8/136: readlink d1/d29/d17/d1f/d2a/l2b 0 2024-09-10T04:05:06.647 INFO:tasks.workunit.client.0.smithi046.stdout:7/99: dwrite d5/d6/db/f22 [0,4194304] 0 2024-09-10T04:05:06.647 INFO:tasks.workunit.client.0.smithi046.stdout:3/74: creat d12/d13/d14/f15 x:0 0 0 2024-09-10T04:05:06.649 INFO:tasks.workunit.client.0.smithi046.stdout:5/76: mkdir d7/d11/d14 0 2024-09-10T04:05:06.649 INFO:tasks.workunit.client.1.smithi070.stdout:3/97: creat db/d16/f1b x:0 0 0 2024-09-10T04:05:06.650 INFO:tasks.workunit.client.0.smithi046.stdout:6/65: truncate f9 756601 0 2024-09-10T04:05:06.651 INFO:tasks.workunit.client.1.smithi070.stdout:6/167: dwrite d0/d3/d18/d1b/f14 [4194304,4194304] 0 2024-09-10T04:05:06.651 INFO:tasks.workunit.client.0.smithi046.stdout:8/137: write d1/d29/d17/f2f [254081,50626] 0 2024-09-10T04:05:06.655 INFO:tasks.workunit.client.1.smithi070.stdout:0/104: rename d8/l9 to d8/d13/l25 0 2024-09-10T04:05:06.655 INFO:tasks.workunit.client.0.smithi046.stdout:9/85: dwrite d4/fb [0,4194304] 0 2024-09-10T04:05:06.656 INFO:tasks.workunit.client.1.smithi070.stdout:7/65: getdents d2/d3/dc 0 2024-09-10T04:05:06.657 INFO:tasks.workunit.client.0.smithi046.stdout:8/138: dread - d1/d29/d17/d1f/d2a/f2d zero size 2024-09-10T04:05:06.658 INFO:tasks.workunit.client.1.smithi070.stdout:5/51: dwrite f3 [0,4194304] 0 2024-09-10T04:05:06.659 INFO:tasks.workunit.client.1.smithi070.stdout:5/52: chown fc 84724 1 2024-09-10T04:05:06.660 INFO:tasks.workunit.client.1.smithi070.stdout:5/53: dread - f7 zero size 2024-09-10T04:05:06.660 INFO:tasks.workunit.client.0.smithi046.stdout:7/100: write d5/d6/db/f1d [527093,67159] 0 2024-09-10T04:05:06.662 INFO:tasks.workunit.client.1.smithi070.stdout:8/111: rename d4/d16/d17/d1a/f1c to d4/d16/d17/d19/f26 0 2024-09-10T04:05:06.662 INFO:tasks.workunit.client.0.smithi046.stdout:3/75: dread f0 [0,4194304] 0 2024-09-10T04:05:06.664 INFO:tasks.workunit.client.0.smithi046.stdout:8/139: fsync d1/d29/d17/f23 0 2024-09-10T04:05:06.664 INFO:tasks.workunit.client.1.smithi070.stdout:0/105: rename d8/d24 to d8/d24/d26 22 2024-09-10T04:05:06.664 INFO:tasks.workunit.client.0.smithi046.stdout:9/86: read d4/d7/fe [285368,48223] 0 2024-09-10T04:05:06.665 INFO:tasks.workunit.client.1.smithi070.stdout:6/168: creat d0/d3/d18/d1b/d33/f39 x:0 0 0 2024-09-10T04:05:06.666 INFO:tasks.workunit.client.0.smithi046.stdout:6/66: creat da/db/d10/f17 x:0 0 0 2024-09-10T04:05:06.666 INFO:tasks.workunit.client.0.smithi046.stdout:3/76: rmdir d12/d13 39 2024-09-10T04:05:06.667 INFO:tasks.workunit.client.1.smithi070.stdout:3/98: dwrite db/fd [0,4194304] 0 2024-09-10T04:05:06.667 INFO:tasks.workunit.client.0.smithi046.stdout:5/77: dwrite d7/f10 [0,4194304] 0 2024-09-10T04:05:06.668 INFO:tasks.workunit.client.0.smithi046.stdout:3/77: fdatasync f6 0 2024-09-10T04:05:06.669 INFO:tasks.workunit.client.0.smithi046.stdout:8/140: read d1/fa [1613780,72529] 0 2024-09-10T04:05:06.669 INFO:tasks.workunit.client.0.smithi046.stdout:5/78: chown d7/f8 215106572 1 2024-09-10T04:05:06.669 INFO:tasks.workunit.client.1.smithi070.stdout:7/66: mkdir d2/d3/dc/d13/d17 0 2024-09-10T04:05:06.671 INFO:tasks.workunit.client.0.smithi046.stdout:8/141: write d1/d29/d17/d1f/f31 [197715,31989] 0 2024-09-10T04:05:06.674 INFO:tasks.workunit.client.1.smithi070.stdout:3/99: mkdir db/d1c 0 2024-09-10T04:05:06.675 INFO:tasks.workunit.client.0.smithi046.stdout:8/142: read d1/fd [1989190,20701] 0 2024-09-10T04:05:06.675 INFO:tasks.workunit.client.0.smithi046.stdout:7/101: unlink d5/d6/db/f22 0 2024-09-10T04:05:06.675 INFO:tasks.workunit.client.1.smithi070.stdout:7/67: truncate d2/f15 780456 0 2024-09-10T04:05:06.677 INFO:tasks.workunit.client.1.smithi070.stdout:3/100: dread - db/f18 zero size 2024-09-10T04:05:06.677 INFO:tasks.workunit.client.0.smithi046.stdout:8/143: truncate d1/d29/d17/d1f/d2a/f2d 245293 0 2024-09-10T04:05:06.678 INFO:tasks.workunit.client.1.smithi070.stdout:5/54: dwrite f7 [0,4194304] 0 2024-09-10T04:05:06.678 INFO:tasks.workunit.client.1.smithi070.stdout:6/169: creat d0/d3/d18/d1b/f3a x:0 0 0 2024-09-10T04:05:06.678 INFO:tasks.workunit.client.1.smithi070.stdout:7/68: fsync d2/f15 0 2024-09-10T04:05:06.678 INFO:tasks.workunit.client.0.smithi046.stdout:8/144: chown d1/l13 516 1 2024-09-10T04:05:06.680 INFO:tasks.workunit.client.1.smithi070.stdout:5/55: dread - f10 zero size 2024-09-10T04:05:06.681 INFO:tasks.workunit.client.0.smithi046.stdout:6/67: mkdir da/db/d18 0 2024-09-10T04:05:06.683 INFO:tasks.workunit.client.1.smithi070.stdout:8/112: creat d4/d16/f27 x:0 0 0 2024-09-10T04:05:06.683 INFO:tasks.workunit.client.0.smithi046.stdout:5/79: symlink d7/l15 0 2024-09-10T04:05:06.684 INFO:tasks.workunit.client.0.smithi046.stdout:5/80: readlink d7/l15 0 2024-09-10T04:05:06.685 INFO:tasks.workunit.client.0.smithi046.stdout:3/78: symlink d12/l16 0 2024-09-10T04:05:06.685 INFO:tasks.workunit.client.0.smithi046.stdout:5/81: fsync d7/d11/f13 0 2024-09-10T04:05:06.685 INFO:tasks.workunit.client.0.smithi046.stdout:3/79: fdatasync f5 0 2024-09-10T04:05:06.686 INFO:tasks.workunit.client.1.smithi070.stdout:0/106: dwrite d8/d13/f19 [0,4194304] 0 2024-09-10T04:05:06.686 INFO:tasks.workunit.client.0.smithi046.stdout:3/80: chown lc 334057 1 2024-09-10T04:05:06.686 INFO:tasks.workunit.client.0.smithi046.stdout:3/81: fdatasync f9 0 2024-09-10T04:05:06.687 INFO:tasks.workunit.client.1.smithi070.stdout:3/101: symlink db/l1d 0 2024-09-10T04:05:06.687 INFO:tasks.workunit.client.0.smithi046.stdout:3/82: stat cd 0 2024-09-10T04:05:06.692 INFO:tasks.workunit.client.1.smithi070.stdout:7/69: rename d2/d3/dc/d13/d17 to d2/d7/d9/d18 0 2024-09-10T04:05:06.693 INFO:tasks.workunit.client.0.smithi046.stdout:7/102: mknod d5/d6/db/d11/c25 0 2024-09-10T04:05:06.693 INFO:tasks.workunit.client.0.smithi046.stdout:7/103: readlink d5/d6/db/l19 0 2024-09-10T04:05:06.693 INFO:tasks.workunit.client.0.smithi046.stdout:8/145: rename d1/d29/f1b to d1/d29/d12/f32 0 2024-09-10T04:05:06.694 INFO:tasks.workunit.client.0.smithi046.stdout:6/68: mkdir da/db/d19 0 2024-09-10T04:05:06.695 INFO:tasks.workunit.client.1.smithi070.stdout:0/107: creat d8/f27 x:0 0 0 2024-09-10T04:05:06.697 INFO:tasks.workunit.client.0.smithi046.stdout:5/82: truncate d7/fe 133521 0 2024-09-10T04:05:06.697 INFO:tasks.workunit.client.1.smithi070.stdout:8/113: dwrite d4/f10 [4194304,4194304] 0 2024-09-10T04:05:06.699 INFO:tasks.workunit.client.1.smithi070.stdout:8/114: stat d4/d16/c1f 0 2024-09-10T04:05:06.701 INFO:tasks.workunit.client.1.smithi070.stdout:7/70: dwrite f1 [4194304,4194304] 0 2024-09-10T04:05:06.703 INFO:tasks.workunit.client.1.smithi070.stdout:3/102: creat db/d1c/f1e x:0 0 0 2024-09-10T04:05:06.708 INFO:tasks.workunit.client.0.smithi046.stdout:8/146: creat d1/d29/d17/d1f/d2a/f33 x:0 0 0 2024-09-10T04:05:06.710 INFO:tasks.workunit.client.0.smithi046.stdout:6/69: write da/fd [600418,87343] 0 2024-09-10T04:05:06.710 INFO:tasks.workunit.client.1.smithi070.stdout:6/170: write d0/d3/d1f/f22 [577009,107750] 0 2024-09-10T04:05:06.712 INFO:tasks.workunit.client.0.smithi046.stdout:5/83: creat d7/f16 x:0 0 0 2024-09-10T04:05:06.713 INFO:tasks.workunit.client.1.smithi070.stdout:3/103: dwrite db/d16/f1b [0,4194304] 0 2024-09-10T04:05:06.713 INFO:tasks.workunit.client.1.smithi070.stdout:0/108: unlink d8/d13/l25 0 2024-09-10T04:05:06.714 INFO:tasks.workunit.client.1.smithi070.stdout:0/109: readlink d8/lb 0 2024-09-10T04:05:06.715 INFO:tasks.workunit.client.0.smithi046.stdout:9/87: write d4/f6 [827165,67445] 0 2024-09-10T04:05:06.717 INFO:tasks.workunit.client.1.smithi070.stdout:7/71: dread d2/d7/fe [0,4194304] 0 2024-09-10T04:05:06.718 INFO:tasks.workunit.client.0.smithi046.stdout:3/83: truncate fb 3150869 0 2024-09-10T04:05:06.718 INFO:tasks.workunit.client.1.smithi070.stdout:5/56: link f6 f12 0 2024-09-10T04:05:06.718 INFO:tasks.workunit.client.1.smithi070.stdout:7/72: write d2/f15 [561023,51144] 0 2024-09-10T04:05:06.723 INFO:tasks.workunit.client.1.smithi070.stdout:3/104: dwrite db/ff [0,4194304] 0 2024-09-10T04:05:06.724 INFO:tasks.workunit.client.0.smithi046.stdout:8/147: creat d1/d29/d19/f34 x:0 0 0 2024-09-10T04:05:06.725 INFO:tasks.workunit.client.0.smithi046.stdout:9/88: dwrite d4/fb [0,4194304] 0 2024-09-10T04:05:06.726 INFO:tasks.workunit.client.0.smithi046.stdout:9/89: chown d4/d18/f19 1732045 1 2024-09-10T04:05:06.729 INFO:tasks.workunit.client.1.smithi070.stdout:6/171: creat d0/d3/d18/d1b/d17/d28/f3b x:0 0 0 2024-09-10T04:05:06.737 INFO:tasks.workunit.client.1.smithi070.stdout:7/73: mknod d2/d7/c19 0 2024-09-10T04:05:06.737 INFO:tasks.workunit.client.0.smithi046.stdout:7/104: link d5/d6/db/l19 d5/d6/d23/l26 0 2024-09-10T04:05:06.738 INFO:tasks.workunit.client.1.smithi070.stdout:6/172: dwrite d0/d3/d18/d1b/d33/f39 [0,4194304] 0 2024-09-10T04:05:06.743 INFO:tasks.workunit.client.1.smithi070.stdout:8/115: dwrite d4/d16/d17/d19/f25 [0,4194304] 0 2024-09-10T04:05:06.747 INFO:tasks.workunit.client.1.smithi070.stdout:0/110: rename d8/d13/d1a/f21 to d8/d13/f28 0 2024-09-10T04:05:06.748 INFO:tasks.workunit.client.1.smithi070.stdout:0/111: fdatasync d8/f27 0 2024-09-10T04:05:06.749 INFO:tasks.workunit.client.0.smithi046.stdout:3/84: truncate f6 7743 0 2024-09-10T04:05:06.752 INFO:tasks.workunit.client.1.smithi070.stdout:5/57: creat f13 x:0 0 0 2024-09-10T04:05:06.753 INFO:tasks.workunit.client.1.smithi070.stdout:8/116: mkdir d4/d16/d17/d19/d28 0 2024-09-10T04:05:06.754 INFO:tasks.workunit.client.1.smithi070.stdout:3/105: dwrite f3 [0,4194304] 0 2024-09-10T04:05:06.756 INFO:tasks.workunit.client.0.smithi046.stdout:6/70: rmdir da/db/d19 0 2024-09-10T04:05:06.757 INFO:tasks.workunit.client.1.smithi070.stdout:0/112: symlink d8/d13/d1a/l29 0 2024-09-10T04:05:06.760 INFO:tasks.workunit.client.1.smithi070.stdout:7/74: dwrite d2/d7/fe [0,4194304] 0 2024-09-10T04:05:06.761 INFO:tasks.workunit.client.1.smithi070.stdout:5/58: readlink l1 0 2024-09-10T04:05:06.762 INFO:tasks.workunit.client.0.smithi046.stdout:9/90: truncate d4/d7/ff 3383471 0 2024-09-10T04:05:06.765 INFO:tasks.workunit.client.0.smithi046.stdout:7/105: fdatasync d5/d9/fe 0 2024-09-10T04:05:06.766 INFO:tasks.workunit.client.0.smithi046.stdout:7/106: fsync d5/d6/f1c 0 2024-09-10T04:05:06.767 INFO:tasks.workunit.client.1.smithi070.stdout:8/117: truncate d4/d16/d17/d1a/f21 5141470 0 2024-09-10T04:05:06.768 INFO:tasks.workunit.client.1.smithi070.stdout:8/118: chown d4/d16/l1b 13439 1 2024-09-10T04:05:06.769 INFO:tasks.workunit.client.1.smithi070.stdout:6/173: link d0/c29 d0/d3/d18/c3c 0 2024-09-10T04:05:06.772 INFO:tasks.workunit.client.0.smithi046.stdout:9/91: dwrite d4/d18/f19 [0,4194304] 0 2024-09-10T04:05:06.773 INFO:tasks.workunit.client.1.smithi070.stdout:6/174: dread d0/d3/d18/d1b/f14 [4194304,4194304] 0 2024-09-10T04:05:06.774 INFO:tasks.workunit.client.1.smithi070.stdout:7/75: creat d2/d3/dc/d13/f1a x:0 0 0 2024-09-10T04:05:06.782 INFO:tasks.workunit.client.1.smithi070.stdout:8/119: unlink d4/l14 0 2024-09-10T04:05:06.782 INFO:tasks.workunit.client.1.smithi070.stdout:8/120: chown d4/ff 39939 1 2024-09-10T04:05:06.784 INFO:tasks.workunit.client.0.smithi046.stdout:3/85: rename f5 to d12/d13/f17 0 2024-09-10T04:05:06.785 INFO:tasks.workunit.client.1.smithi070.stdout:3/106: truncate db/ff 3669823 0 2024-09-10T04:05:06.788 INFO:tasks.workunit.client.1.smithi070.stdout:6/175: mknod d0/d3/d18/d1b/d17/c3d 0 2024-09-10T04:05:06.789 INFO:tasks.workunit.client.1.smithi070.stdout:7/76: mkdir d2/d3/dc/d1b 0 2024-09-10T04:05:06.790 INFO:tasks.workunit.client.0.smithi046.stdout:5/84: read f6 [98445,22845] 0 2024-09-10T04:05:06.790 INFO:tasks.workunit.client.1.smithi070.stdout:3/107: dread fa [0,4194304] 0 2024-09-10T04:05:06.791 INFO:tasks.workunit.client.0.smithi046.stdout:8/148: getdents d1 0 2024-09-10T04:05:06.795 INFO:tasks.workunit.client.1.smithi070.stdout:6/176: dwrite d0/d3/d18/d1b/d17/d28/f2d [0,4194304] 0 2024-09-10T04:05:06.798 INFO:tasks.workunit.client.1.smithi070.stdout:6/177: read d0/d3/d18/d1b/f11 [898332,8403] 0 2024-09-10T04:05:06.800 INFO:tasks.workunit.client.0.smithi046.stdout:8/149: dwrite d1/d29/d17/f2f [0,4194304] 0 2024-09-10T04:05:06.800 INFO:tasks.workunit.client.1.smithi070.stdout:6/178: write d0/d3/d1f/f37 [482177,37946] 0 2024-09-10T04:05:06.800 INFO:tasks.workunit.client.0.smithi046.stdout:8/150: readlink d1/l28 0 2024-09-10T04:05:06.800 INFO:tasks.workunit.client.1.smithi070.stdout:6/179: readlink d0/d3/d1f/l21 0 2024-09-10T04:05:06.801 INFO:tasks.workunit.client.0.smithi046.stdout:2/170: sync 2024-09-10T04:05:06.807 INFO:tasks.workunit.client.0.smithi046.stdout:2/171: dread d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:06.807 INFO:tasks.workunit.client.0.smithi046.stdout:3/86: creat d12/d13/f18 x:0 0 0 2024-09-10T04:05:06.808 INFO:tasks.workunit.client.0.smithi046.stdout:9/92: write d4/d7/f8 [3058944,69764] 0 2024-09-10T04:05:06.808 INFO:tasks.workunit.client.1.smithi070.stdout:7/77: symlink d2/d7/db/l1c 0 2024-09-10T04:05:06.809 INFO:tasks.workunit.client.1.smithi070.stdout:7/78: write f1 [9350801,50474] 0 2024-09-10T04:05:06.810 INFO:tasks.workunit.client.0.smithi046.stdout:9/93: read d4/d7/f8 [2775108,87529] 0 2024-09-10T04:05:06.810 INFO:tasks.workunit.client.1.smithi070.stdout:7/79: dread - d2/d3/dc/d13/f1a zero size 2024-09-10T04:05:06.811 INFO:tasks.workunit.client.0.smithi046.stdout:9/94: chown d4/f6 348 1 2024-09-10T04:05:06.812 INFO:tasks.workunit.client.0.smithi046.stdout:6/71: write f1 [735143,4328] 0 2024-09-10T04:05:06.812 INFO:tasks.workunit.client.0.smithi046.stdout:6/72: chown da/db/d18 6 1 2024-09-10T04:05:06.813 INFO:tasks.workunit.client.0.smithi046.stdout:7/107: creat d5/d6/db/d11/d1f/f27 x:0 0 0 2024-09-10T04:05:06.815 INFO:tasks.workunit.client.1.smithi070.stdout:8/121: rename d4/d16/f1d to d4/f29 0 2024-09-10T04:05:06.816 INFO:tasks.workunit.client.1.smithi070.stdout:8/122: chown d4/f8 3054622 1 2024-09-10T04:05:06.816 INFO:tasks.workunit.client.1.smithi070.stdout:8/123: write d4/ff [4501026,42798] 0 2024-09-10T04:05:06.818 INFO:tasks.workunit.client.1.smithi070.stdout:6/180: fsync d0/f2e 0 2024-09-10T04:05:06.827 INFO:tasks.workunit.client.1.smithi070.stdout:8/124: creat d4/d16/d17/d1a/f2a x:0 0 0 2024-09-10T04:05:06.827 INFO:tasks.workunit.client.1.smithi070.stdout:8/125: dread - d4/d16/d17/d1a/d23/f24 zero size 2024-09-10T04:05:06.831 INFO:tasks.workunit.client.1.smithi070.stdout:3/108: fdatasync db/ff 0 2024-09-10T04:05:06.837 INFO:tasks.workunit.client.1.smithi070.stdout:3/109: dwrite db/f14 [0,4194304] 0 2024-09-10T04:05:06.838 INFO:tasks.workunit.client.0.smithi046.stdout:8/151: rename d1/d29/d12/f16 to d1/d29/d17/d1f/f35 0 2024-09-10T04:05:06.839 INFO:tasks.workunit.client.1.smithi070.stdout:3/110: chown db/d16/f1b 701647 1 2024-09-10T04:05:06.839 INFO:tasks.workunit.client.0.smithi046.stdout:6/73: creat da/db/d10/f1a x:0 0 0 2024-09-10T04:05:06.840 INFO:tasks.workunit.client.1.smithi070.stdout:8/126: mknod d4/c2b 0 2024-09-10T04:05:06.841 INFO:tasks.workunit.client.0.smithi046.stdout:5/85: symlink d7/d11/d14/l17 0 2024-09-10T04:05:06.843 INFO:tasks.workunit.client.1.smithi070.stdout:3/111: creat db/f1f x:0 0 0 2024-09-10T04:05:06.845 INFO:tasks.workunit.client.1.smithi070.stdout:8/127: creat d4/d16/d17/f2c x:0 0 0 2024-09-10T04:05:06.848 INFO:tasks.workunit.client.0.smithi046.stdout:7/108: dwrite d5/d9/fe [0,4194304] 0 2024-09-10T04:05:06.848 INFO:tasks.workunit.client.1.smithi070.stdout:3/112: rename l9 to db/l20 0 2024-09-10T04:05:06.857 INFO:tasks.workunit.client.0.smithi046.stdout:4/98: sync 2024-09-10T04:05:06.857 INFO:tasks.workunit.client.0.smithi046.stdout:1/63: sync 2024-09-10T04:05:06.857 INFO:tasks.workunit.client.0.smithi046.stdout:0/66: sync 2024-09-10T04:05:06.866 INFO:tasks.workunit.client.0.smithi046.stdout:4/99: dwrite d3/d12/d14/f18 [0,4194304] 0 2024-09-10T04:05:06.867 INFO:tasks.workunit.client.0.smithi046.stdout:4/100: read - d3/f15 zero size 2024-09-10T04:05:06.875 INFO:tasks.workunit.client.0.smithi046.stdout:4/101: dwrite d3/d12/f19 [0,4194304] 0 2024-09-10T04:05:06.915 INFO:tasks.workunit.client.1.smithi070.stdout:3/113: write db/f18 [1003630,61566] 0 2024-09-10T04:05:06.916 INFO:tasks.workunit.client.1.smithi070.stdout:8/128: dwrite d4/f12 [0,4194304] 0 2024-09-10T04:05:06.916 INFO:tasks.workunit.client.1.smithi070.stdout:8/129: fsync d4/d16/f22 0 2024-09-10T04:05:06.917 INFO:tasks.workunit.client.1.smithi070.stdout:8/130: stat d4/cc 0 2024-09-10T04:05:06.917 INFO:tasks.workunit.client.1.smithi070.stdout:3/114: creat db/d1c/f21 x:0 0 0 2024-09-10T04:05:07.000 INFO:tasks.workunit.client.1.smithi070.stdout:3/115: rename db/f1a to db/f22 0 2024-09-10T04:05:07.002 INFO:tasks.workunit.client.1.smithi070.stdout:8/131: dread d4/f12 [0,4194304] 0 2024-09-10T04:05:07.002 INFO:tasks.workunit.client.1.smithi070.stdout:3/116: read f3 [3115791,46776] 0 2024-09-10T04:05:07.002 INFO:tasks.workunit.client.1.smithi070.stdout:3/117: stat db/d1c/f21 0 2024-09-10T04:05:07.004 INFO:tasks.workunit.client.1.smithi070.stdout:8/132: symlink d4/d16/d17/l2d 0 2024-09-10T04:05:07.005 INFO:tasks.workunit.client.1.smithi070.stdout:8/133: chown d4/l18 164 1 2024-09-10T04:05:07.007 INFO:tasks.workunit.client.1.smithi070.stdout:3/118: dwrite db/f12 [0,4194304] 0 2024-09-10T04:05:07.011 INFO:tasks.workunit.client.1.smithi070.stdout:3/119: symlink db/d16/l23 0 2024-09-10T04:05:07.013 INFO:tasks.workunit.client.1.smithi070.stdout:8/134: rename d4/d16/d17/d1a/f21 to d4/d16/d17/d1a/f2e 0 2024-09-10T04:05:07.015 INFO:tasks.workunit.client.1.smithi070.stdout:3/120: mknod db/d1c/c24 0 2024-09-10T04:05:07.018 INFO:tasks.workunit.client.1.smithi070.stdout:3/121: rmdir db 39 2024-09-10T04:05:07.021 INFO:tasks.workunit.client.1.smithi070.stdout:3/122: dread - db/f11 zero size 2024-09-10T04:05:07.024 INFO:tasks.workunit.client.1.smithi070.stdout:3/123: unlink db/d16/l23 0 2024-09-10T04:05:07.027 INFO:tasks.workunit.client.1.smithi070.stdout:2/70: sync 2024-09-10T04:05:07.027 INFO:tasks.workunit.client.1.smithi070.stdout:9/85: sync 2024-09-10T04:05:07.027 INFO:tasks.workunit.client.1.smithi070.stdout:8/135: getdents d4 0 2024-09-10T04:05:07.028 INFO:tasks.workunit.client.1.smithi070.stdout:9/86: write d9/dd/f1a [426070,121308] 0 2024-09-10T04:05:07.028 INFO:tasks.workunit.client.1.smithi070.stdout:3/124: unlink db/ff 0 2024-09-10T04:05:07.030 INFO:tasks.workunit.client.1.smithi070.stdout:2/71: mknod ce 0 2024-09-10T04:05:07.032 INFO:tasks.workunit.client.1.smithi070.stdout:9/87: dread d9/dd/f1a [0,4194304] 0 2024-09-10T04:05:07.033 INFO:tasks.workunit.client.1.smithi070.stdout:8/136: creat d4/d16/d17/d1a/d23/f2f x:0 0 0 2024-09-10T04:05:07.036 INFO:tasks.workunit.client.1.smithi070.stdout:3/125: mkdir db/d1c/d25 0 2024-09-10T04:05:07.040 INFO:tasks.workunit.client.1.smithi070.stdout:9/88: mkdir d9/d1c 0 2024-09-10T04:05:07.047 INFO:tasks.workunit.client.1.smithi070.stdout:9/89: rename d9/dd/d10/c11 to d9/dd/d10/d1b/c1d 0 2024-09-10T04:05:07.054 INFO:tasks.workunit.client.1.smithi070.stdout:9/90: symlink d9/dd/d10/d1b/l1e 0 2024-09-10T04:05:07.059 INFO:tasks.workunit.client.1.smithi070.stdout:9/91: rmdir d9 39 2024-09-10T04:05:07.060 INFO:tasks.workunit.client.1.smithi070.stdout:8/137: getdents d4/d16 0 2024-09-10T04:05:07.065 INFO:tasks.workunit.client.1.smithi070.stdout:8/138: dwrite d4/ff [0,4194304] 0 2024-09-10T04:05:07.067 INFO:tasks.workunit.client.1.smithi070.stdout:9/92: getdents d9/d1c 0 2024-09-10T04:05:07.068 INFO:tasks.workunit.client.1.smithi070.stdout:9/93: write d9/dd/f18 [157083,93230] 0 2024-09-10T04:05:07.070 INFO:tasks.workunit.client.1.smithi070.stdout:8/139: rename d4/f10 to d4/d16/d17/d19/d28/f30 0 2024-09-10T04:05:07.080 INFO:tasks.workunit.client.1.smithi070.stdout:8/140: creat d4/d16/d17/d19/d28/f31 x:0 0 0 2024-09-10T04:05:07.081 INFO:tasks.workunit.client.1.smithi070.stdout:8/141: write d4/d16/f22 [166234,118450] 0 2024-09-10T04:05:07.085 INFO:tasks.workunit.client.1.smithi070.stdout:8/142: rename d4/d16/d17/d19/f26 to d4/d16/d17/d1a/f32 0 2024-09-10T04:05:07.085 INFO:tasks.workunit.client.1.smithi070.stdout:8/143: stat d4/cc 0 2024-09-10T04:05:07.086 INFO:tasks.workunit.client.1.smithi070.stdout:9/94: getdents d9/dd 0 2024-09-10T04:05:07.090 INFO:tasks.workunit.client.1.smithi070.stdout:8/144: link d4/fa d4/d16/f33 0 2024-09-10T04:05:07.091 INFO:tasks.workunit.client.1.smithi070.stdout:8/145: read d4/f11 [81220,16778] 0 2024-09-10T04:05:07.097 INFO:tasks.workunit.client.1.smithi070.stdout:4/88: sync 2024-09-10T04:05:07.100 INFO:tasks.workunit.client.1.smithi070.stdout:4/89: symlink de/l1c 0 2024-09-10T04:05:07.104 INFO:tasks.workunit.client.1.smithi070.stdout:4/90: symlink de/d13/l1d 0 2024-09-10T04:05:07.105 INFO:tasks.workunit.client.1.smithi070.stdout:4/91: chown de/d13/d18/l19 1718 1 2024-09-10T04:05:07.107 INFO:tasks.workunit.client.0.smithi046.stdout:3/87: truncate f0 1775487 0 2024-09-10T04:05:07.109 INFO:tasks.workunit.client.1.smithi070.stdout:4/92: rename de/l1c to de/d13/d18/l1e 0 2024-09-10T04:05:07.112 INFO:tasks.workunit.client.1.smithi070.stdout:4/93: truncate f0 4709074 0 2024-09-10T04:05:07.113 INFO:tasks.workunit.client.0.smithi046.stdout:8/152: mkdir d1/d29/d19/d36 0 2024-09-10T04:05:07.114 INFO:tasks.workunit.client.0.smithi046.stdout:6/74: creat da/db/d11/f1b x:0 0 0 2024-09-10T04:05:07.114 INFO:tasks.workunit.client.0.smithi046.stdout:6/75: chown da/db/d10/c15 620980 1 2024-09-10T04:05:07.116 INFO:tasks.workunit.client.1.smithi070.stdout:4/94: creat de/d13/f1f x:0 0 0 2024-09-10T04:05:07.117 INFO:tasks.workunit.client.1.smithi070.stdout:4/95: rename de to de/d13/d20 22 2024-09-10T04:05:07.119 INFO:tasks.workunit.client.0.smithi046.stdout:5/86: rename l5 to d7/d11/d14/l18 0 2024-09-10T04:05:07.123 INFO:tasks.workunit.client.0.smithi046.stdout:7/109: mknod d5/d6/db/d11/d1f/c28 0 2024-09-10T04:05:07.126 INFO:tasks.workunit.client.0.smithi046.stdout:4/102: creat d3/d12/d17/f20 x:0 0 0 2024-09-10T04:05:07.127 INFO:tasks.workunit.client.0.smithi046.stdout:3/88: creat d12/d13/f19 x:0 0 0 2024-09-10T04:05:07.129 INFO:tasks.workunit.client.0.smithi046.stdout:3/89: write d12/d13/f19 [299352,125057] 0 2024-09-10T04:05:07.130 INFO:tasks.workunit.client.0.smithi046.stdout:8/153: symlink d1/d29/d17/d1f/d2a/l37 0 2024-09-10T04:05:07.132 INFO:tasks.workunit.client.0.smithi046.stdout:1/64: rename d3/c5 to d3/dc/c11 0 2024-09-10T04:05:07.132 INFO:tasks.workunit.client.0.smithi046.stdout:8/154: rename d1/d29 to d1/d29/d17/d1f/d38 22 2024-09-10T04:05:07.132 INFO:tasks.workunit.client.0.smithi046.stdout:1/65: chown d3/f6 5 1 2024-09-10T04:05:07.133 INFO:tasks.workunit.client.0.smithi046.stdout:1/66: chown d3/dc/f10 296 1 2024-09-10T04:05:07.133 INFO:tasks.workunit.client.0.smithi046.stdout:0/67: mknod d1/d6/d3/dd/c15 0 2024-09-10T04:05:07.135 INFO:tasks.workunit.client.0.smithi046.stdout:7/110: creat d5/d6/db/f29 x:0 0 0 2024-09-10T04:05:07.136 INFO:tasks.workunit.client.0.smithi046.stdout:4/103: creat d3/d12/d14/f21 x:0 0 0 2024-09-10T04:05:07.137 INFO:tasks.workunit.client.0.smithi046.stdout:7/111: chown d5/f7 529756862 1 2024-09-10T04:05:07.138 INFO:tasks.workunit.client.0.smithi046.stdout:9/95: getdents d4 0 2024-09-10T04:05:07.138 INFO:tasks.workunit.client.0.smithi046.stdout:2/172: link d2/d8/l21 d2/d6/d15/l40 0 2024-09-10T04:05:07.142 INFO:tasks.workunit.client.0.smithi046.stdout:1/67: dread d3/dc/f10 [0,4194304] 0 2024-09-10T04:05:07.142 INFO:tasks.workunit.client.0.smithi046.stdout:8/155: fsync d1/fd 0 2024-09-10T04:05:07.143 INFO:tasks.workunit.client.0.smithi046.stdout:7/112: write d5/d6/db/d11/f14 [1041515,78509] 0 2024-09-10T04:05:07.145 INFO:tasks.workunit.client.0.smithi046.stdout:3/90: symlink d12/l1a 0 2024-09-10T04:05:07.147 INFO:tasks.workunit.client.0.smithi046.stdout:1/68: rename d3/dc to d3/dc/d12 22 2024-09-10T04:05:07.147 INFO:tasks.workunit.client.0.smithi046.stdout:0/68: dwrite d1/dc/f12 [0,4194304] 0 2024-09-10T04:05:07.151 INFO:tasks.workunit.client.0.smithi046.stdout:8/156: write d1/d29/d17/d1f/d2a/f33 [962485,109275] 0 2024-09-10T04:05:07.153 INFO:tasks.workunit.client.0.smithi046.stdout:7/113: creat d5/d6/db/f2a x:0 0 0 2024-09-10T04:05:07.154 INFO:tasks.workunit.client.0.smithi046.stdout:1/69: chown d3/ff 220 1 2024-09-10T04:05:07.156 INFO:tasks.workunit.client.0.smithi046.stdout:3/91: unlink f6 0 2024-09-10T04:05:07.158 INFO:tasks.workunit.client.0.smithi046.stdout:8/157: creat d1/f39 x:0 0 0 2024-09-10T04:05:07.159 INFO:tasks.workunit.client.0.smithi046.stdout:9/96: dwrite d4/f6 [0,4194304] 0 2024-09-10T04:05:07.160 INFO:tasks.workunit.client.0.smithi046.stdout:1/70: mkdir d3/d13 0 2024-09-10T04:05:07.161 INFO:tasks.workunit.client.0.smithi046.stdout:4/104: dwrite d3/d12/d14/f21 [0,4194304] 0 2024-09-10T04:05:07.162 INFO:tasks.workunit.client.0.smithi046.stdout:2/173: dread d2/d6/d15/f3e [0,4194304] 0 2024-09-10T04:05:07.162 INFO:tasks.workunit.client.0.smithi046.stdout:1/71: chown d3/dc 409478561 1 2024-09-10T04:05:07.164 INFO:tasks.workunit.client.1.smithi070.stdout:3/126: getdents db/d16 0 2024-09-10T04:05:07.166 INFO:tasks.workunit.client.1.smithi070.stdout:7/80: sync 2024-09-10T04:05:07.166 INFO:tasks.workunit.client.1.smithi070.stdout:6/181: sync 2024-09-10T04:05:07.166 INFO:tasks.workunit.client.1.smithi070.stdout:1/75: sync 2024-09-10T04:05:07.166 INFO:tasks.workunit.client.1.smithi070.stdout:0/113: sync 2024-09-10T04:05:07.166 INFO:tasks.workunit.client.1.smithi070.stdout:5/59: sync 2024-09-10T04:05:07.167 INFO:tasks.workunit.client.0.smithi046.stdout:8/158: rmdir d1/d29 39 2024-09-10T04:05:07.167 INFO:tasks.workunit.client.1.smithi070.stdout:5/60: fdatasync ff 0 2024-09-10T04:05:07.167 INFO:tasks.workunit.client.1.smithi070.stdout:6/182: chown d0/d3/d1f 139129 1 2024-09-10T04:05:07.167 INFO:tasks.workunit.client.0.smithi046.stdout:4/105: write d3/d12/d17/f20 [451597,18240] 0 2024-09-10T04:05:07.168 INFO:tasks.workunit.client.1.smithi070.stdout:5/61: chown c9 3971821 1 2024-09-10T04:05:07.169 INFO:tasks.workunit.client.1.smithi070.stdout:6/183: dread - d0/d3/d18/d1b/d17/d28/f3b zero size 2024-09-10T04:05:07.170 INFO:tasks.workunit.client.1.smithi070.stdout:6/184: readlink d0/d3/d18/d2a/l32 0 2024-09-10T04:05:07.170 INFO:tasks.workunit.client.1.smithi070.stdout:3/127: rename db/l17 to db/d1c/l26 0 2024-09-10T04:05:07.171 INFO:tasks.workunit.client.1.smithi070.stdout:3/128: write db/d16/f1b [3810504,41575] 0 2024-09-10T04:05:07.174 INFO:tasks.workunit.client.0.smithi046.stdout:0/69: dwrite d1/d6/f11 [4194304,4194304] 0 2024-09-10T04:05:07.174 INFO:tasks.workunit.client.0.smithi046.stdout:7/114: symlink d5/d6/db/d11/d1e/d20/l2b 0 2024-09-10T04:05:07.175 INFO:tasks.workunit.client.0.smithi046.stdout:4/106: write d3/d12/d14/f18 [2533456,74841] 0 2024-09-10T04:05:07.177 INFO:tasks.workunit.client.0.smithi046.stdout:7/115: chown d5/d6/db/d11/d1e 3 1 2024-09-10T04:05:07.178 INFO:tasks.workunit.client.0.smithi046.stdout:3/92: dwrite f3 [0,4194304] 0 2024-09-10T04:05:07.179 INFO:tasks.workunit.client.1.smithi070.stdout:7/81: readlink d2/l6 0 2024-09-10T04:05:07.179 INFO:tasks.workunit.client.0.smithi046.stdout:2/174: dread d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:07.180 INFO:tasks.workunit.client.1.smithi070.stdout:1/76: mkdir d3/d20 0 2024-09-10T04:05:07.180 INFO:tasks.workunit.client.1.smithi070.stdout:2/72: getdents . 0 2024-09-10T04:05:07.181 INFO:tasks.workunit.client.0.smithi046.stdout:7/116: fsync d5/d6/db/d11/f14 0 2024-09-10T04:05:07.182 INFO:tasks.workunit.client.1.smithi070.stdout:2/73: stat l5 0 2024-09-10T04:05:07.182 INFO:tasks.workunit.client.0.smithi046.stdout:2/175: fdatasync d2/d6/d23/f33 0 2024-09-10T04:05:07.183 INFO:tasks.workunit.client.0.smithi046.stdout:4/107: dread d3/fa [0,4194304] 0 2024-09-10T04:05:07.184 INFO:tasks.workunit.client.1.smithi070.stdout:2/74: dread - fc zero size 2024-09-10T04:05:07.184 INFO:tasks.workunit.client.1.smithi070.stdout:3/129: dwrite f7 [4194304,4194304] 0 2024-09-10T04:05:07.188 INFO:tasks.workunit.client.1.smithi070.stdout:2/75: truncate fb 4965442 0 2024-09-10T04:05:07.189 INFO:tasks.workunit.client.1.smithi070.stdout:3/130: chown db/d1c/f1e 218800291 1 2024-09-10T04:05:07.191 INFO:tasks.workunit.client.1.smithi070.stdout:7/82: rmdir d2/d7 39 2024-09-10T04:05:07.194 INFO:tasks.workunit.client.1.smithi070.stdout:7/83: fsync d2/d3/dc/d13/f1a 0 2024-09-10T04:05:07.195 INFO:tasks.workunit.client.1.smithi070.stdout:9/95: sync 2024-09-10T04:05:07.196 INFO:tasks.workunit.client.1.smithi070.stdout:1/77: dwrite d3/d8/d12/f19 [0,4194304] 0 2024-09-10T04:05:07.197 INFO:tasks.workunit.client.0.smithi046.stdout:7/117: creat d5/f2c x:0 0 0 2024-09-10T04:05:07.197 INFO:tasks.workunit.client.1.smithi070.stdout:9/96: write d9/dd/f18 [769565,63851] 0 2024-09-10T04:05:07.199 INFO:tasks.workunit.client.0.smithi046.stdout:8/159: mknod d1/d29/d17/d1f/c3a 0 2024-09-10T04:05:07.200 INFO:tasks.workunit.client.0.smithi046.stdout:8/160: rename d1/d29/d19/d36 to d1/d29/d19/d36/d3b 22 2024-09-10T04:05:07.201 INFO:tasks.workunit.client.0.smithi046.stdout:8/161: readlink d1/l13 0 2024-09-10T04:05:07.201 INFO:tasks.workunit.client.1.smithi070.stdout:2/76: mknod cf 0 2024-09-10T04:05:07.201 INFO:tasks.workunit.client.1.smithi070.stdout:9/97: truncate d9/dd/d10/f19 652486 0 2024-09-10T04:05:07.201 INFO:tasks.workunit.client.1.smithi070.stdout:2/77: read - fc zero size 2024-09-10T04:05:07.202 INFO:tasks.workunit.client.0.smithi046.stdout:8/162: fsync d1/d29/d17/d1f/f31 0 2024-09-10T04:05:07.202 INFO:tasks.workunit.client.0.smithi046.stdout:4/108: mkdir d3/d12/d17/d22 0 2024-09-10T04:05:07.203 INFO:tasks.workunit.client.0.smithi046.stdout:4/109: fsync d3/d12/d14/f18 0 2024-09-10T04:05:07.204 INFO:tasks.workunit.client.0.smithi046.stdout:2/176: creat d2/d8/d37/f41 x:0 0 0 2024-09-10T04:05:07.206 INFO:tasks.workunit.client.0.smithi046.stdout:4/110: read d3/d12/d14/f18 [1075689,113476] 0 2024-09-10T04:05:07.206 INFO:tasks.workunit.client.0.smithi046.stdout:1/72: link d3/l7 d3/dc/l14 0 2024-09-10T04:05:07.207 INFO:tasks.workunit.client.0.smithi046.stdout:4/111: chown d3/ce 31083478 1 2024-09-10T04:05:07.208 INFO:tasks.workunit.client.1.smithi070.stdout:1/78: mknod d3/d8/c21 0 2024-09-10T04:05:07.209 INFO:tasks.workunit.client.0.smithi046.stdout:3/93: mkdir d12/d1b 0 2024-09-10T04:05:07.211 INFO:tasks.workunit.client.0.smithi046.stdout:7/118: mknod d5/d6/db/d11/d1e/d20/c2d 0 2024-09-10T04:05:07.212 INFO:tasks.workunit.client.0.smithi046.stdout:7/119: write d5/d6/f1c [82477,110222] 0 2024-09-10T04:05:07.212 INFO:tasks.workunit.client.1.smithi070.stdout:2/78: creat f10 x:0 0 0 2024-09-10T04:05:07.215 INFO:tasks.workunit.client.1.smithi070.stdout:9/98: creat d9/dd/d10/f1f x:0 0 0 2024-09-10T04:05:07.215 INFO:tasks.workunit.client.1.smithi070.stdout:1/79: dwrite f1 [8388608,4194304] 0 2024-09-10T04:05:07.217 INFO:tasks.workunit.client.1.smithi070.stdout:1/80: dread d3/f16 [0,4194304] 0 2024-09-10T04:05:07.218 INFO:tasks.workunit.client.1.smithi070.stdout:7/84: creat d2/d3/dc/d1b/f1d x:0 0 0 2024-09-10T04:05:07.219 INFO:tasks.workunit.client.0.smithi046.stdout:8/163: rename d1/d29/ff to d1/d29/d19/d36/f3c 0 2024-09-10T04:05:07.225 INFO:tasks.workunit.client.0.smithi046.stdout:3/94: unlink d12/l1a 0 2024-09-10T04:05:07.226 INFO:tasks.workunit.client.1.smithi070.stdout:9/99: rmdir d9 39 2024-09-10T04:05:07.227 INFO:tasks.workunit.client.0.smithi046.stdout:7/120: chown f4 12603239 1 2024-09-10T04:05:07.228 INFO:tasks.workunit.client.1.smithi070.stdout:1/81: unlink d3/lb 0 2024-09-10T04:05:07.230 INFO:tasks.workunit.client.1.smithi070.stdout:7/85: creat d2/d3/dc/f1e x:0 0 0 2024-09-10T04:05:07.232 INFO:tasks.workunit.client.0.smithi046.stdout:4/112: rename d3/l1a to d3/d12/d17/d22/l23 0 2024-09-10T04:05:07.235 INFO:tasks.workunit.client.1.smithi070.stdout:9/100: fdatasync d9/dd/d10/f19 0 2024-09-10T04:05:07.236 INFO:tasks.workunit.client.1.smithi070.stdout:9/101: dread d9/dd/d10/f19 [0,4194304] 0 2024-09-10T04:05:07.236 INFO:tasks.workunit.client.1.smithi070.stdout:1/82: rmdir d3 39 2024-09-10T04:05:07.237 INFO:tasks.workunit.client.0.smithi046.stdout:1/73: getdents d3/d13 0 2024-09-10T04:05:07.239 INFO:tasks.workunit.client.0.smithi046.stdout:1/74: stat d3/dc/f10 0 2024-09-10T04:05:07.240 INFO:tasks.workunit.client.0.smithi046.stdout:4/113: dwrite d3/f15 [0,4194304] 0 2024-09-10T04:05:07.240 INFO:tasks.workunit.client.0.smithi046.stdout:1/75: readlink d3/l9 0 2024-09-10T04:05:07.241 INFO:tasks.workunit.client.0.smithi046.stdout:1/76: chown d3 4 1 2024-09-10T04:05:07.244 INFO:tasks.workunit.client.0.smithi046.stdout:4/114: dread d3/fa [0,4194304] 0 2024-09-10T04:05:07.246 INFO:tasks.workunit.client.1.smithi070.stdout:9/102: creat d9/dd/d10/f20 x:0 0 0 2024-09-10T04:05:07.247 INFO:tasks.workunit.client.1.smithi070.stdout:7/86: mknod d2/d7/d9/d18/c1f 0 2024-09-10T04:05:07.250 INFO:tasks.workunit.client.1.smithi070.stdout:8/146: unlink d4/d16/d17/d1a/d23/f24 0 2024-09-10T04:05:07.251 INFO:tasks.workunit.client.1.smithi070.stdout:9/103: dread d9/dd/f18 [0,4194304] 0 2024-09-10T04:05:07.251 INFO:tasks.workunit.client.1.smithi070.stdout:9/104: stat d9 0 2024-09-10T04:05:07.251 INFO:tasks.workunit.client.0.smithi046.stdout:3/95: truncate f7 354507 0 2024-09-10T04:05:07.253 INFO:tasks.workunit.client.0.smithi046.stdout:4/115: dwrite d3/d12/d17/f20 [0,4194304] 0 2024-09-10T04:05:07.255 INFO:tasks.workunit.client.1.smithi070.stdout:7/87: dwrite d2/d3/dc/f1e [0,4194304] 0 2024-09-10T04:05:07.258 INFO:tasks.workunit.client.1.smithi070.stdout:7/88: write d2/d3/dc/f1e [306486,90552] 0 2024-09-10T04:05:07.259 INFO:tasks.workunit.client.0.smithi046.stdout:4/116: creat d3/d12/d17/d22/f24 x:0 0 0 2024-09-10T04:05:07.261 INFO:tasks.workunit.client.0.smithi046.stdout:7/121: getdents d5/d6 0 2024-09-10T04:05:07.263 INFO:tasks.workunit.client.1.smithi070.stdout:9/105: dwrite d9/dd/d10/f1f [0,4194304] 0 2024-09-10T04:05:07.264 INFO:tasks.workunit.client.0.smithi046.stdout:3/96: getdents d12/d1b 0 2024-09-10T04:05:07.266 INFO:tasks.workunit.client.0.smithi046.stdout:4/117: creat d3/d12/d17/d22/f25 x:0 0 0 2024-09-10T04:05:07.269 INFO:tasks.workunit.client.1.smithi070.stdout:7/89: mkdir d2/d7/db/d20 0 2024-09-10T04:05:07.270 INFO:tasks.workunit.client.1.smithi070.stdout:7/90: chown d2/d7/fe 0 1 2024-09-10T04:05:07.271 INFO:tasks.workunit.client.0.smithi046.stdout:3/97: creat d12/d13/d14/f1c x:0 0 0 2024-09-10T04:05:07.272 INFO:tasks.workunit.client.0.smithi046.stdout:3/98: dread - d12/d13/f18 zero size 2024-09-10T04:05:07.273 INFO:tasks.workunit.client.0.smithi046.stdout:4/118: unlink d3/ce 0 2024-09-10T04:05:07.275 INFO:tasks.workunit.client.1.smithi070.stdout:9/106: rename d9/dd/d10/l13 to d9/l21 0 2024-09-10T04:05:07.275 INFO:tasks.workunit.client.0.smithi046.stdout:4/119: write d3/d12/d14/f18 [4197151,100173] 0 2024-09-10T04:05:07.276 INFO:tasks.workunit.client.1.smithi070.stdout:9/107: write d9/dd/f14 [4918765,10032] 0 2024-09-10T04:05:07.278 INFO:tasks.workunit.client.1.smithi070.stdout:8/147: creat d4/d16/f34 x:0 0 0 2024-09-10T04:05:07.279 INFO:tasks.workunit.client.1.smithi070.stdout:7/91: rmdir d2/d7/d9 39 2024-09-10T04:05:07.280 INFO:tasks.workunit.client.1.smithi070.stdout:7/92: dread - d2/d3/dc/d13/f1a zero size 2024-09-10T04:05:07.280 INFO:tasks.workunit.client.1.smithi070.stdout:7/93: dread - d2/d3/dc/d13/f1a zero size 2024-09-10T04:05:07.281 INFO:tasks.workunit.client.0.smithi046.stdout:4/120: mknod d3/c26 0 2024-09-10T04:05:07.282 INFO:tasks.workunit.client.0.smithi046.stdout:4/121: chown d3/f9 1960 1 2024-09-10T04:05:07.285 INFO:tasks.workunit.client.1.smithi070.stdout:9/108: unlink d9/dd/d10/c15 0 2024-09-10T04:05:07.286 INFO:tasks.workunit.client.1.smithi070.stdout:8/148: read - d4/d16/d17/f2c zero size 2024-09-10T04:05:07.289 INFO:tasks.workunit.client.0.smithi046.stdout:4/122: rename d3/d12/d17/d22/f24 to d3/d12/d17/d22/f27 0 2024-09-10T04:05:07.289 INFO:tasks.workunit.client.0.smithi046.stdout:5/87: dwrite d7/fd [0,4194304] 0 2024-09-10T04:05:07.291 INFO:tasks.workunit.client.0.smithi046.stdout:5/88: write d7/d11/f13 [186128,631] 0 2024-09-10T04:05:07.293 INFO:tasks.workunit.client.1.smithi070.stdout:8/149: symlink d4/d16/d17/d1a/l35 0 2024-09-10T04:05:07.293 INFO:tasks.workunit.client.1.smithi070.stdout:8/150: stat d4/d16/d17/d1a/d23/f2f 0 2024-09-10T04:05:07.298 INFO:tasks.workunit.client.0.smithi046.stdout:5/89: creat d7/f19 x:0 0 0 2024-09-10T04:05:07.298 INFO:tasks.workunit.client.1.smithi070.stdout:8/151: dread d4/f12 [0,4194304] 0 2024-09-10T04:05:07.298 INFO:tasks.workunit.client.0.smithi046.stdout:5/90: readlink d7/d11/d14/l17 0 2024-09-10T04:05:07.299 INFO:tasks.workunit.client.1.smithi070.stdout:7/94: rename d2/c8 to d2/d7/d9/d18/c21 0 2024-09-10T04:05:07.301 INFO:tasks.workunit.client.0.smithi046.stdout:4/123: rename d3/d12/d17/l1c to d3/d12/d14/l28 0 2024-09-10T04:05:07.304 INFO:tasks.workunit.client.0.smithi046.stdout:9/97: write d4/d7/fe [4525548,105015] 0 2024-09-10T04:05:07.304 INFO:tasks.workunit.client.1.smithi070.stdout:3/131: getdents db 0 2024-09-10T04:05:07.305 INFO:tasks.workunit.client.1.smithi070.stdout:3/132: write f7 [6977506,62852] 0 2024-09-10T04:05:07.305 INFO:tasks.workunit.client.0.smithi046.stdout:5/91: rename f6 to d7/d11/f1a 0 2024-09-10T04:05:07.308 INFO:tasks.workunit.client.0.smithi046.stdout:9/98: mknod d4/c1a 0 2024-09-10T04:05:07.309 INFO:tasks.workunit.client.1.smithi070.stdout:7/95: symlink d2/d7/db/d20/l22 0 2024-09-10T04:05:07.311 INFO:tasks.workunit.client.1.smithi070.stdout:6/185: write d0/d3/d18/d1b/f11 [5108564,96764] 0 2024-09-10T04:05:07.311 INFO:tasks.workunit.client.0.smithi046.stdout:9/99: dread d4/fb [0,4194304] 0 2024-09-10T04:05:07.313 INFO:tasks.workunit.client.1.smithi070.stdout:3/133: creat db/f27 x:0 0 0 2024-09-10T04:05:07.314 INFO:tasks.workunit.client.1.smithi070.stdout:8/152: link d4/f12 d4/d16/f36 0 2024-09-10T04:05:07.315 INFO:tasks.workunit.client.1.smithi070.stdout:7/96: mkdir d2/d7/d9/d18/d23 0 2024-09-10T04:05:07.316 INFO:tasks.workunit.client.0.smithi046.stdout:9/100: creat d4/f1b x:0 0 0 2024-09-10T04:05:07.317 INFO:tasks.workunit.client.1.smithi070.stdout:0/114: dwrite d8/fc [0,4194304] 0 2024-09-10T04:05:07.318 INFO:tasks.workunit.client.1.smithi070.stdout:5/62: dwrite f12 [0,4194304] 0 2024-09-10T04:05:07.319 INFO:tasks.workunit.client.0.smithi046.stdout:9/101: creat d4/d9/f1c x:0 0 0 2024-09-10T04:05:07.320 INFO:tasks.workunit.client.0.smithi046.stdout:9/102: chown d4/d18 0 1 2024-09-10T04:05:07.321 INFO:tasks.workunit.client.1.smithi070.stdout:6/186: mknod d0/d3/d18/d1b/d17/c3e 0 2024-09-10T04:05:07.324 INFO:tasks.workunit.client.0.smithi046.stdout:9/103: symlink d4/l1d 0 2024-09-10T04:05:07.324 INFO:tasks.workunit.client.1.smithi070.stdout:5/63: dread fd [0,4194304] 0 2024-09-10T04:05:07.325 INFO:tasks.workunit.client.1.smithi070.stdout:5/64: stat f12 0 2024-09-10T04:05:07.327 INFO:tasks.workunit.client.1.smithi070.stdout:8/153: mknod d4/d16/d17/d1a/c37 0 2024-09-10T04:05:07.327 INFO:tasks.workunit.client.1.smithi070.stdout:0/115: dwrite d8/f10 [4194304,4194304] 0 2024-09-10T04:05:07.329 INFO:tasks.workunit.client.0.smithi046.stdout:9/104: creat d4/d7/f1e x:0 0 0 2024-09-10T04:05:07.333 INFO:tasks.workunit.client.1.smithi070.stdout:3/134: mkdir db/d1c/d25/d28 0 2024-09-10T04:05:07.335 INFO:tasks.workunit.client.1.smithi070.stdout:8/154: truncate d4/d16/d17/d19/d28/f30 8558367 0 2024-09-10T04:05:07.335 INFO:tasks.workunit.client.1.smithi070.stdout:6/187: mkdir d0/d3/d18/d1b/d17/d3f 0 2024-09-10T04:05:07.338 INFO:tasks.workunit.client.0.smithi046.stdout:9/105: dwrite d4/d9/f15 [0,4194304] 0 2024-09-10T04:05:07.339 INFO:tasks.workunit.client.0.smithi046.stdout:9/106: dread - d4/d7/f17 zero size 2024-09-10T04:05:07.339 INFO:tasks.workunit.client.1.smithi070.stdout:2/79: getdents . 0 2024-09-10T04:05:07.340 INFO:tasks.workunit.client.0.smithi046.stdout:9/107: chown d4/f14 5465 1 2024-09-10T04:05:07.340 INFO:tasks.workunit.client.1.smithi070.stdout:5/65: dwrite f13 [0,4194304] 0 2024-09-10T04:05:07.343 INFO:tasks.workunit.client.1.smithi070.stdout:8/155: chown d4/d16/f27 1 1 2024-09-10T04:05:07.343 INFO:tasks.workunit.client.1.smithi070.stdout:3/135: mknod db/d16/c29 0 2024-09-10T04:05:07.343 INFO:tasks.workunit.client.1.smithi070.stdout:2/80: write f10 [656328,49394] 0 2024-09-10T04:05:07.346 INFO:tasks.workunit.client.1.smithi070.stdout:0/116: dwrite d8/d13/f19 [0,4194304] 0 2024-09-10T04:05:07.346 INFO:tasks.workunit.client.1.smithi070.stdout:8/156: chown f0 7228 1 2024-09-10T04:05:07.346 INFO:tasks.workunit.client.1.smithi070.stdout:7/97: creat d2/f24 x:0 0 0 2024-09-10T04:05:07.346 INFO:tasks.workunit.client.1.smithi070.stdout:3/136: dread - db/f11 zero size 2024-09-10T04:05:07.347 INFO:tasks.workunit.client.1.smithi070.stdout:2/81: read fa [1081563,7354] 0 2024-09-10T04:05:07.349 INFO:tasks.workunit.client.1.smithi070.stdout:7/98: write d2/d7/fe [3157962,54649] 0 2024-09-10T04:05:07.349 INFO:tasks.workunit.client.1.smithi070.stdout:3/137: read - db/d1c/f1e zero size 2024-09-10T04:05:07.349 INFO:tasks.workunit.client.1.smithi070.stdout:2/82: write fc [662606,10840] 0 2024-09-10T04:05:07.351 INFO:tasks.workunit.client.1.smithi070.stdout:2/83: chown c9 1848 1 2024-09-10T04:05:07.351 INFO:tasks.workunit.client.0.smithi046.stdout:2/177: write d2/f25 [995769,78823] 0 2024-09-10T04:05:07.353 INFO:tasks.workunit.client.0.smithi046.stdout:8/164: truncate d1/d29/d17/d1f/d2a/f2d 187961 0 2024-09-10T04:05:07.354 INFO:tasks.workunit.client.0.smithi046.stdout:8/165: fdatasync d1/f2e 0 2024-09-10T04:05:07.355 INFO:tasks.workunit.client.0.smithi046.stdout:8/166: chown d1/d29/d17/d1f/d2a/l2b 1948370 1 2024-09-10T04:05:07.357 INFO:tasks.workunit.client.0.smithi046.stdout:2/178: mkdir d2/d6/d23/d42 0 2024-09-10T04:05:07.358 INFO:tasks.workunit.client.1.smithi070.stdout:7/99: dwrite d2/f24 [0,4194304] 0 2024-09-10T04:05:07.359 INFO:tasks.workunit.client.1.smithi070.stdout:6/188: mkdir d0/d3/d18/d1b/d17/d3f/d40 0 2024-09-10T04:05:07.360 INFO:tasks.workunit.client.0.smithi046.stdout:3/99: truncate f7 1127020 0 2024-09-10T04:05:07.360 INFO:tasks.workunit.client.1.smithi070.stdout:7/100: dread - d2/d3/dc/d13/f1a zero size 2024-09-10T04:05:07.361 INFO:tasks.workunit.client.1.smithi070.stdout:3/138: unlink db/f18 0 2024-09-10T04:05:07.362 INFO:tasks.workunit.client.1.smithi070.stdout:3/139: fdatasync db/d1c/f21 0 2024-09-10T04:05:07.363 INFO:tasks.workunit.client.1.smithi070.stdout:2/84: creat f11 x:0 0 0 2024-09-10T04:05:07.364 INFO:tasks.workunit.client.0.smithi046.stdout:2/179: chown d2/d6/d15/l28 222 1 2024-09-10T04:05:07.365 INFO:tasks.workunit.client.1.smithi070.stdout:8/157: symlink d4/d16/l38 0 2024-09-10T04:05:07.366 INFO:tasks.workunit.client.1.smithi070.stdout:0/117: creat d8/d24/f2a x:0 0 0 2024-09-10T04:05:07.366 INFO:tasks.workunit.client.0.smithi046.stdout:8/167: dwrite d1/d29/d17/d1f/f31 [0,4194304] 0 2024-09-10T04:05:07.369 INFO:tasks.workunit.client.1.smithi070.stdout:1/83: truncate d3/d8/d12/f19 2432586 0 2024-09-10T04:05:07.373 INFO:tasks.workunit.client.0.smithi046.stdout:1/77: dwrite f1 [0,4194304] 0 2024-09-10T04:05:07.373 INFO:tasks.workunit.client.1.smithi070.stdout:6/189: rename d0/d3/d1f/l21 to d0/d3/d18/d1b/d17/d28/l41 0 2024-09-10T04:05:07.375 INFO:tasks.workunit.client.0.smithi046.stdout:8/168: symlink d1/d29/d19/l3d 0 2024-09-10T04:05:07.375 INFO:tasks.workunit.client.0.smithi046.stdout:3/100: dwrite d12/d13/d14/f1c [0,4194304] 0 2024-09-10T04:05:07.376 INFO:tasks.workunit.client.1.smithi070.stdout:4/96: dwrite f0 [0,4194304] 0 2024-09-10T04:05:07.378 INFO:tasks.workunit.client.0.smithi046.stdout:3/101: stat d12/d13/f17 0 2024-09-10T04:05:07.384 INFO:tasks.workunit.client.1.smithi070.stdout:3/140: dread fa [0,4194304] 0 2024-09-10T04:05:07.384 INFO:tasks.workunit.client.0.smithi046.stdout:3/102: dread d12/d13/f17 [0,4194304] 0 2024-09-10T04:05:07.385 INFO:tasks.workunit.client.0.smithi046.stdout:3/103: readlink d12/l16 0 2024-09-10T04:05:07.385 INFO:tasks.workunit.client.1.smithi070.stdout:3/141: write db/f27 [334851,31879] 0 2024-09-10T04:05:07.391 INFO:tasks.workunit.client.1.smithi070.stdout:8/158: mkdir d4/d39 0 2024-09-10T04:05:07.394 INFO:tasks.workunit.client.1.smithi070.stdout:9/109: write d9/dd/f12 [13925,43008] 0 2024-09-10T04:05:07.394 INFO:tasks.workunit.client.0.smithi046.stdout:8/169: creat d1/d29/d17/d1f/d2a/f3e x:0 0 0 2024-09-10T04:05:07.395 INFO:tasks.workunit.client.0.smithi046.stdout:8/170: dread - d1/f39 zero size 2024-09-10T04:05:07.396 INFO:tasks.workunit.client.1.smithi070.stdout:5/66: getdents . 0 2024-09-10T04:05:07.396 INFO:tasks.workunit.client.1.smithi070.stdout:9/110: dread - d9/dd/d10/f20 zero size 2024-09-10T04:05:07.396 INFO:tasks.workunit.client.1.smithi070.stdout:5/67: write f12 [3485936,70167] 0 2024-09-10T04:05:07.399 INFO:tasks.workunit.client.0.smithi046.stdout:3/104: unlink ff 0 2024-09-10T04:05:07.400 INFO:tasks.workunit.client.1.smithi070.stdout:6/190: fsync d0/d3/d18/d1b/f14 0 2024-09-10T04:05:07.402 INFO:tasks.workunit.client.0.smithi046.stdout:1/78: link d3/f6 d3/dc/f15 0 2024-09-10T04:05:07.406 INFO:tasks.workunit.client.0.smithi046.stdout:2/180: getdents d2/d6/d15/d16 0 2024-09-10T04:05:07.407 INFO:tasks.workunit.client.0.smithi046.stdout:2/181: write d2/d8/d37/f41 [1008203,56176] 0 2024-09-10T04:05:07.410 INFO:tasks.workunit.client.0.smithi046.stdout:1/79: mkdir d3/dc/d16 0 2024-09-10T04:05:07.411 INFO:tasks.workunit.client.0.smithi046.stdout:1/80: read f1 [1120572,123408] 0 2024-09-10T04:05:07.412 INFO:tasks.workunit.client.1.smithi070.stdout:8/159: write d4/d16/d17/d1a/f32 [42090,109188] 0 2024-09-10T04:05:07.414 INFO:tasks.workunit.client.0.smithi046.stdout:9/108: rename d4/d9 to d4/d7/d1f 0 2024-09-10T04:05:07.415 INFO:tasks.workunit.client.0.smithi046.stdout:9/109: chown d4/d7/f8 926213 1 2024-09-10T04:05:07.416 INFO:tasks.workunit.client.0.smithi046.stdout:2/182: creat d2/d6/f43 x:0 0 0 2024-09-10T04:05:07.417 INFO:tasks.workunit.client.0.smithi046.stdout:2/183: chown d2/d8/l38 576784 1 2024-09-10T04:05:07.419 INFO:tasks.workunit.client.0.smithi046.stdout:1/81: creat d3/dc/f17 x:0 0 0 2024-09-10T04:05:07.421 INFO:tasks.workunit.client.1.smithi070.stdout:9/111: truncate d9/dd/f18 138242 0 2024-09-10T04:05:07.423 INFO:tasks.workunit.client.1.smithi070.stdout:6/191: creat d0/d3/d18/f42 x:0 0 0 2024-09-10T04:05:07.424 INFO:tasks.workunit.client.0.smithi046.stdout:9/110: creat d4/d7/f20 x:0 0 0 2024-09-10T04:05:07.425 INFO:tasks.workunit.client.0.smithi046.stdout:9/111: write d4/f6 [5351409,56951] 0 2024-09-10T04:05:07.432 INFO:tasks.workunit.client.0.smithi046.stdout:3/105: rename fb to d12/d1b/f1d 0 2024-09-10T04:05:07.434 INFO:tasks.workunit.client.0.smithi046.stdout:8/171: getdents d1/d29/d12 0 2024-09-10T04:05:07.434 INFO:tasks.workunit.client.1.smithi070.stdout:2/85: link cf c12 0 2024-09-10T04:05:07.437 INFO:tasks.workunit.client.1.smithi070.stdout:8/160: write d4/f29 [150205,110332] 0 2024-09-10T04:05:07.437 INFO:tasks.workunit.client.0.smithi046.stdout:9/112: rmdir d4/d18 39 2024-09-10T04:05:07.439 INFO:tasks.workunit.client.0.smithi046.stdout:2/184: mkdir d2/d6/d15/d44 0 2024-09-10T04:05:07.440 INFO:tasks.workunit.client.1.smithi070.stdout:1/84: rmdir d3/d8/d12/d1f 0 2024-09-10T04:05:07.441 INFO:tasks.workunit.client.1.smithi070.stdout:5/68: getdents . 0 2024-09-10T04:05:07.443 INFO:tasks.workunit.client.0.smithi046.stdout:1/82: link d3/dc/f17 d3/d13/f18 0 2024-09-10T04:05:07.446 INFO:tasks.workunit.client.1.smithi070.stdout:7/101: getdents d2/d7 0 2024-09-10T04:05:07.447 INFO:tasks.workunit.client.1.smithi070.stdout:2/86: mknod c13 0 2024-09-10T04:05:07.448 INFO:tasks.workunit.client.1.smithi070.stdout:8/161: mknod d4/d16/d17/d19/c3a 0 2024-09-10T04:05:07.449 INFO:tasks.workunit.client.0.smithi046.stdout:5/92: write d7/d11/f1a [408112,105801] 0 2024-09-10T04:05:07.450 INFO:tasks.workunit.client.1.smithi070.stdout:0/118: getdents d8 0 2024-09-10T04:05:07.452 INFO:tasks.workunit.client.0.smithi046.stdout:1/83: write d3/fb [2982571,40427] 0 2024-09-10T04:05:07.456 INFO:tasks.workunit.client.1.smithi070.stdout:9/112: unlink d9/dd/d10/l17 0 2024-09-10T04:05:07.457 INFO:tasks.workunit.client.0.smithi046.stdout:2/185: creat d2/d6/d23/d42/f45 x:0 0 0 2024-09-10T04:05:07.459 INFO:tasks.workunit.client.1.smithi070.stdout:7/102: mkdir d2/d3/dc/d25 0 2024-09-10T04:05:07.459 INFO:tasks.workunit.client.1.smithi070.stdout:8/162: dwrite d4/d16/d17/d1a/f32 [0,4194304] 0 2024-09-10T04:05:07.460 INFO:tasks.workunit.client.0.smithi046.stdout:5/93: creat d7/f1b x:0 0 0 2024-09-10T04:05:07.460 INFO:tasks.workunit.client.1.smithi070.stdout:7/103: write d2/d3/dc/f1e [376738,92388] 0 2024-09-10T04:05:07.460 INFO:tasks.workunit.client.1.smithi070.stdout:4/97: truncate fc 26431 0 2024-09-10T04:05:07.462 INFO:tasks.workunit.client.0.smithi046.stdout:1/84: symlink d3/l19 0 2024-09-10T04:05:07.464 INFO:tasks.workunit.client.0.smithi046.stdout:1/85: write d3/dc/fe [589717,127291] 0 2024-09-10T04:05:07.471 INFO:tasks.workunit.client.0.smithi046.stdout:5/94: mkdir d7/d11/d14/d1c 0 2024-09-10T04:05:07.472 INFO:tasks.workunit.client.0.smithi046.stdout:5/95: chown d7/d11 2024450893 1 2024-09-10T04:05:07.472 INFO:tasks.workunit.client.0.smithi046.stdout:5/96: fsync d7/f8 0 2024-09-10T04:05:07.475 INFO:tasks.workunit.client.0.smithi046.stdout:1/86: creat d3/f1a x:0 0 0 2024-09-10T04:05:07.476 INFO:tasks.workunit.client.0.smithi046.stdout:8/172: getdents d1/d29/d19/d36 0 2024-09-10T04:05:07.478 INFO:tasks.workunit.client.0.smithi046.stdout:3/106: unlink d12/d1b/f1d 0 2024-09-10T04:05:07.478 INFO:tasks.workunit.client.0.smithi046.stdout:3/107: readlink lc 0 2024-09-10T04:05:07.479 INFO:tasks.workunit.client.0.smithi046.stdout:3/108: truncate d12/d13/f18 818821 0 2024-09-10T04:05:07.479 INFO:tasks.workunit.client.0.smithi046.stdout:9/113: getdents d4/d7/d1f 0 2024-09-10T04:05:07.484 INFO:tasks.workunit.client.0.smithi046.stdout:5/97: creat d7/d11/d14/f1d x:0 0 0 2024-09-10T04:05:07.485 INFO:tasks.workunit.client.0.smithi046.stdout:5/98: chown d7/d11/f1a 0 1 2024-09-10T04:05:07.485 INFO:tasks.workunit.client.0.smithi046.stdout:1/87: rmdir d3/d13 39 2024-09-10T04:05:07.486 INFO:tasks.workunit.client.0.smithi046.stdout:5/99: fdatasync d7/d11/f13 0 2024-09-10T04:05:07.487 INFO:tasks.workunit.client.0.smithi046.stdout:5/100: write d7/d11/f1a [581829,58340] 0 2024-09-10T04:05:07.489 INFO:tasks.workunit.client.0.smithi046.stdout:9/114: creat d4/d7/d1f/f21 x:0 0 0 2024-09-10T04:05:07.490 INFO:tasks.workunit.client.0.smithi046.stdout:3/109: dread f0 [0,4194304] 0 2024-09-10T04:05:07.491 INFO:tasks.workunit.client.0.smithi046.stdout:9/115: truncate d4/d7/d1f/f1c 576783 0 2024-09-10T04:05:07.492 INFO:tasks.workunit.client.0.smithi046.stdout:9/116: stat d4/d7/d1f/ld 0 2024-09-10T04:05:07.492 INFO:tasks.workunit.client.0.smithi046.stdout:9/117: fdatasync d4/f6 0 2024-09-10T04:05:07.493 INFO:tasks.workunit.client.0.smithi046.stdout:9/118: chown d4/d7/l12 87 1 2024-09-10T04:05:07.493 INFO:tasks.workunit.client.0.smithi046.stdout:9/119: chown d4/d7/fe 0 1 2024-09-10T04:05:07.494 INFO:tasks.workunit.client.0.smithi046.stdout:8/173: link d1/d29/d12/f32 d1/d29/d19/d36/f3f 0 2024-09-10T04:05:07.494 INFO:tasks.workunit.client.0.smithi046.stdout:9/120: dread - d4/f1b zero size 2024-09-10T04:05:07.495 INFO:tasks.workunit.client.0.smithi046.stdout:8/174: write d1/d29/d17/d1f/d2a/f2c [439693,119776] 0 2024-09-10T04:05:07.497 INFO:tasks.workunit.client.0.smithi046.stdout:5/101: rename d7/f16 to d7/f1e 0 2024-09-10T04:05:07.499 INFO:tasks.workunit.client.0.smithi046.stdout:2/186: getdents d2/d6/d15 0 2024-09-10T04:05:07.501 INFO:tasks.workunit.client.0.smithi046.stdout:3/110: chown f7 15 1 2024-09-10T04:05:07.501 INFO:tasks.workunit.client.0.smithi046.stdout:1/88: mknod d3/c1b 0 2024-09-10T04:05:07.504 INFO:tasks.workunit.client.0.smithi046.stdout:8/175: dwrite d1/d29/fe [4194304,4194304] 0 2024-09-10T04:05:07.507 INFO:tasks.workunit.client.0.smithi046.stdout:5/102: dwrite d7/f10 [0,4194304] 0 2024-09-10T04:05:07.509 INFO:tasks.workunit.client.0.smithi046.stdout:1/89: mknod d3/dc/c1c 0 2024-09-10T04:05:07.512 INFO:tasks.workunit.client.0.smithi046.stdout:3/111: dread d12/d13/f17 [0,4194304] 0 2024-09-10T04:05:07.512 INFO:tasks.workunit.client.0.smithi046.stdout:2/187: creat d2/d6/d15/d44/f46 x:0 0 0 2024-09-10T04:05:07.513 INFO:tasks.workunit.client.0.smithi046.stdout:5/103: creat d7/f1f x:0 0 0 2024-09-10T04:05:07.517 INFO:tasks.workunit.client.0.smithi046.stdout:9/121: link d4/f1b d4/d18/f22 0 2024-09-10T04:05:07.517 INFO:tasks.workunit.client.0.smithi046.stdout:9/122: chown d4/d7/ca 133992 1 2024-09-10T04:05:07.519 INFO:tasks.workunit.client.0.smithi046.stdout:3/112: mkdir d12/d13/d14/d1e 0 2024-09-10T04:05:07.520 INFO:tasks.workunit.client.0.smithi046.stdout:2/188: creat d2/d6/d23/d3b/f47 x:0 0 0 2024-09-10T04:05:07.522 INFO:tasks.workunit.client.0.smithi046.stdout:2/189: chown d2/d6/l3f 51204873 1 2024-09-10T04:05:07.522 INFO:tasks.workunit.client.0.smithi046.stdout:1/90: mknod d3/dc/d16/c1d 0 2024-09-10T04:05:07.523 INFO:tasks.workunit.client.0.smithi046.stdout:5/104: rmdir d7/d11/d14 39 2024-09-10T04:05:07.526 INFO:tasks.workunit.client.0.smithi046.stdout:9/123: link d4/d7/fe d4/d7/f23 0 2024-09-10T04:05:07.527 INFO:tasks.workunit.client.0.smithi046.stdout:1/91: unlink d3/dc/f15 0 2024-09-10T04:05:07.529 INFO:tasks.workunit.client.0.smithi046.stdout:3/113: dwrite d12/d13/d14/f15 [0,4194304] 0 2024-09-10T04:05:07.529 INFO:tasks.workunit.client.0.smithi046.stdout:9/124: chown d4/c1a 652 1 2024-09-10T04:05:07.530 INFO:tasks.workunit.client.0.smithi046.stdout:1/92: chown d3/dc/f17 354243 1 2024-09-10T04:05:07.531 INFO:tasks.workunit.client.0.smithi046.stdout:2/190: rename d2/d6/d15/l19 to d2/d6/d15/l48 0 2024-09-10T04:05:07.535 INFO:tasks.workunit.client.0.smithi046.stdout:2/191: dread - d2/d6/d23/d3b/f47 zero size 2024-09-10T04:05:07.537 INFO:tasks.workunit.client.0.smithi046.stdout:5/105: dwrite d7/f1b [0,4194304] 0 2024-09-10T04:05:07.539 INFO:tasks.workunit.client.0.smithi046.stdout:9/125: link d4/l16 d4/d18/l24 0 2024-09-10T04:05:07.541 INFO:tasks.workunit.client.0.smithi046.stdout:2/192: truncate d2/d6/f1a 3865206 0 2024-09-10T04:05:07.542 INFO:tasks.workunit.client.0.smithi046.stdout:5/106: stat d7/d11/d14/d1c 0 2024-09-10T04:05:07.542 INFO:tasks.workunit.client.0.smithi046.stdout:5/107: fsync d7/f9 0 2024-09-10T04:05:07.547 INFO:tasks.workunit.client.0.smithi046.stdout:1/93: write d3/dc/f10 [3340668,107990] 0 2024-09-10T04:05:07.551 INFO:tasks.workunit.client.0.smithi046.stdout:5/108: dwrite d7/fe [0,4194304] 0 2024-09-10T04:05:07.551 INFO:tasks.workunit.client.0.smithi046.stdout:1/94: mknod d3/d13/c1e 0 2024-09-10T04:05:07.551 INFO:tasks.workunit.client.0.smithi046.stdout:9/126: write d4/f1b [708479,117935] 0 2024-09-10T04:05:07.552 INFO:tasks.workunit.client.1.smithi070.stdout:3/142: truncate db/fd 3496048 0 2024-09-10T04:05:07.552 INFO:tasks.workunit.client.1.smithi070.stdout:0/119: creat d8/f2b x:0 0 0 2024-09-10T04:05:07.553 INFO:tasks.workunit.client.1.smithi070.stdout:9/113: creat d9/f22 x:0 0 0 2024-09-10T04:05:07.553 INFO:tasks.workunit.client.0.smithi046.stdout:5/109: write d7/f8 [910438,97022] 0 2024-09-10T04:05:07.554 INFO:tasks.workunit.client.1.smithi070.stdout:2/87: rename cf to c14 0 2024-09-10T04:05:07.554 INFO:tasks.workunit.client.1.smithi070.stdout:9/114: chown d9/dd/d10/f20 105 1 2024-09-10T04:05:07.555 INFO:tasks.workunit.client.1.smithi070.stdout:5/69: truncate f7 3660862 0 2024-09-10T04:05:07.556 INFO:tasks.workunit.client.1.smithi070.stdout:9/115: fsync d9/dd/d10/f1f 0 2024-09-10T04:05:07.557 INFO:tasks.workunit.client.1.smithi070.stdout:3/143: rmdir db 39 2024-09-10T04:05:07.558 INFO:tasks.workunit.client.1.smithi070.stdout:6/192: getdents d0/d3/d18/d1b/d17/d3f 0 2024-09-10T04:05:07.560 INFO:tasks.workunit.client.1.smithi070.stdout:8/163: symlink d4/d39/l3b 0 2024-09-10T04:05:07.562 INFO:tasks.workunit.client.1.smithi070.stdout:8/164: write d4/d16/d17/d1a/f2a [61841,60390] 0 2024-09-10T04:05:07.563 INFO:tasks.workunit.client.0.smithi046.stdout:9/127: dwrite d4/d18/f22 [0,4194304] 0 2024-09-10T04:05:07.563 INFO:tasks.workunit.client.0.smithi046.stdout:5/110: creat d7/d11/d14/d1c/f20 x:0 0 0 2024-09-10T04:05:07.564 INFO:tasks.workunit.client.1.smithi070.stdout:0/120: mknod d8/d13/d1a/c2c 0 2024-09-10T04:05:07.565 INFO:tasks.workunit.client.1.smithi070.stdout:5/70: creat f14 x:0 0 0 2024-09-10T04:05:07.565 INFO:tasks.workunit.client.1.smithi070.stdout:6/193: dwrite d0/d3/d18/d1b/d17/d28/f3b [0,4194304] 0 2024-09-10T04:05:07.566 INFO:tasks.workunit.client.0.smithi046.stdout:9/128: write d4/f1b [3711392,107232] 0 2024-09-10T04:05:07.567 INFO:tasks.workunit.client.0.smithi046.stdout:5/111: write d7/f1b [867797,20460] 0 2024-09-10T04:05:07.568 INFO:tasks.workunit.client.1.smithi070.stdout:5/71: read - ff zero size 2024-09-10T04:05:07.568 INFO:tasks.workunit.client.1.smithi070.stdout:8/165: dread d4/f13 [0,4194304] 0 2024-09-10T04:05:07.568 INFO:tasks.workunit.client.1.smithi070.stdout:0/121: write d8/f10 [9040257,30039] 0 2024-09-10T04:05:07.569 INFO:tasks.workunit.client.0.smithi046.stdout:5/112: fsync d7/d11/f1a 0 2024-09-10T04:05:07.569 INFO:tasks.workunit.client.0.smithi046.stdout:1/95: dwrite d3/dc/f17 [0,4194304] 0 2024-09-10T04:05:07.570 INFO:tasks.workunit.client.0.smithi046.stdout:5/113: truncate d7/fd 4836704 0 2024-09-10T04:05:07.571 INFO:tasks.workunit.client.0.smithi046.stdout:9/129: creat d4/d18/f25 x:0 0 0 2024-09-10T04:05:07.574 INFO:tasks.workunit.client.0.smithi046.stdout:1/96: mkdir d3/dc/d16/d1f 0 2024-09-10T04:05:07.576 INFO:tasks.workunit.client.0.smithi046.stdout:5/114: unlink d7/d11/d14/l17 0 2024-09-10T04:05:07.578 INFO:tasks.workunit.client.1.smithi070.stdout:6/194: dwrite d0/d3/d1f/f22 [0,4194304] 0 2024-09-10T04:05:07.580 INFO:tasks.workunit.client.0.smithi046.stdout:9/130: rename d4/d18/f19 to d4/d7/d1f/f26 0 2024-09-10T04:05:07.584 INFO:tasks.workunit.client.0.smithi046.stdout:1/97: creat d3/d13/f20 x:0 0 0 2024-09-10T04:05:07.587 INFO:tasks.workunit.client.0.smithi046.stdout:5/115: mknod d7/d11/d14/c21 0 2024-09-10T04:05:07.589 INFO:tasks.workunit.client.1.smithi070.stdout:5/72: rename cb to c15 0 2024-09-10T04:05:07.591 INFO:tasks.workunit.client.1.smithi070.stdout:0/122: dread d8/fa [0,4194304] 0 2024-09-10T04:05:07.592 INFO:tasks.workunit.client.0.smithi046.stdout:9/131: rename d4/d7/f17 to d4/d18/f27 0 2024-09-10T04:05:07.593 INFO:tasks.workunit.client.1.smithi070.stdout:8/166: symlink d4/l3c 0 2024-09-10T04:05:07.595 INFO:tasks.workunit.client.1.smithi070.stdout:8/167: write d4/d16/d17/d1a/d23/f2f [325596,45496] 0 2024-09-10T04:05:07.595 INFO:tasks.workunit.client.0.smithi046.stdout:5/116: creat d7/d11/d14/d1c/f22 x:0 0 0 2024-09-10T04:05:07.597 INFO:tasks.workunit.client.1.smithi070.stdout:8/168: write d4/d16/d17/d1a/f2a [212008,101493] 0 2024-09-10T04:05:07.597 INFO:tasks.workunit.client.1.smithi070.stdout:4/98: getdents de/d13/d18 0 2024-09-10T04:05:07.598 INFO:tasks.workunit.client.1.smithi070.stdout:8/169: read d4/d16/f33 [1636496,44186] 0 2024-09-10T04:05:07.598 INFO:tasks.workunit.client.1.smithi070.stdout:6/195: chown d0/d3/d18/c3c 0 1 2024-09-10T04:05:07.599 INFO:tasks.workunit.client.1.smithi070.stdout:3/144: dread db/fd [0,4194304] 0 2024-09-10T04:05:07.599 INFO:tasks.workunit.client.1.smithi070.stdout:6/196: read d0/f10 [2662700,108692] 0 2024-09-10T04:05:07.601 INFO:tasks.workunit.client.1.smithi070.stdout:5/73: creat f16 x:0 0 0 2024-09-10T04:05:07.604 INFO:tasks.workunit.client.1.smithi070.stdout:0/123: mkdir d8/d24/d2d 0 2024-09-10T04:05:07.604 INFO:tasks.workunit.client.1.smithi070.stdout:0/124: fdatasync d8/f27 0 2024-09-10T04:05:07.605 INFO:tasks.workunit.client.1.smithi070.stdout:0/125: chown d8/d13/f1f 42243198 1 2024-09-10T04:05:07.606 INFO:tasks.workunit.client.0.smithi046.stdout:9/132: creat d4/f28 x:0 0 0 2024-09-10T04:05:07.607 INFO:tasks.workunit.client.0.smithi046.stdout:1/98: getdents d3/d13 0 2024-09-10T04:05:07.611 INFO:tasks.workunit.client.0.smithi046.stdout:5/117: creat d7/d11/f23 x:0 0 0 2024-09-10T04:05:07.613 INFO:tasks.workunit.client.0.smithi046.stdout:9/133: dwrite d4/d18/f25 [0,4194304] 0 2024-09-10T04:05:07.616 INFO:tasks.workunit.client.0.smithi046.stdout:1/99: getdents d3/dc/d16/d1f 0 2024-09-10T04:05:07.617 INFO:tasks.workunit.client.0.smithi046.stdout:9/134: fsync d4/d7/f8 0 2024-09-10T04:05:07.620 INFO:tasks.workunit.client.0.smithi046.stdout:5/118: creat d7/d11/f24 x:0 0 0 2024-09-10T04:05:07.621 INFO:tasks.workunit.client.0.smithi046.stdout:9/135: mkdir d4/d29 0 2024-09-10T04:05:07.622 INFO:tasks.workunit.client.0.smithi046.stdout:9/136: stat d4/fb 0 2024-09-10T04:05:07.622 INFO:tasks.workunit.client.0.smithi046.stdout:5/119: readlink d7/d11/d14/l18 0 2024-09-10T04:05:07.625 INFO:tasks.workunit.client.0.smithi046.stdout:9/137: unlink d4/f1b 0 2024-09-10T04:05:07.628 INFO:tasks.workunit.client.0.smithi046.stdout:5/120: link d7/fc d7/d11/d14/f25 0 2024-09-10T04:05:07.629 INFO:tasks.workunit.client.0.smithi046.stdout:5/121: write d7/f10 [3672573,117709] 0 2024-09-10T04:05:07.630 INFO:tasks.workunit.client.0.smithi046.stdout:9/138: dread d4/f6 [0,4194304] 0 2024-09-10T04:05:07.631 INFO:tasks.workunit.client.0.smithi046.stdout:5/122: rmdir d7/d11 39 2024-09-10T04:05:07.632 INFO:tasks.workunit.client.0.smithi046.stdout:5/123: read - d7/fc zero size 2024-09-10T04:05:07.632 INFO:tasks.workunit.client.0.smithi046.stdout:9/139: mknod d4/d7/d1f/c2a 0 2024-09-10T04:05:07.633 INFO:tasks.workunit.client.0.smithi046.stdout:9/140: chown d4/l1d 19851 1 2024-09-10T04:05:07.633 INFO:tasks.workunit.client.0.smithi046.stdout:9/141: dread - d4/f14 zero size 2024-09-10T04:05:07.635 INFO:tasks.workunit.client.0.smithi046.stdout:9/142: symlink d4/d29/l2b 0 2024-09-10T04:05:07.640 INFO:tasks.workunit.client.0.smithi046.stdout:9/143: truncate d4/f6 3346940 0 2024-09-10T04:05:07.640 INFO:tasks.workunit.client.0.smithi046.stdout:9/144: stat d4/d7/d1f/f26 0 2024-09-10T04:05:07.642 INFO:tasks.workunit.client.0.smithi046.stdout:5/124: truncate d7/fc 938240 0 2024-09-10T04:05:07.645 INFO:tasks.workunit.client.0.smithi046.stdout:5/125: truncate d7/d11/d14/d1c/f22 338328 0 2024-09-10T04:05:07.647 INFO:tasks.workunit.client.0.smithi046.stdout:5/126: truncate d7/d11/f24 401182 0 2024-09-10T04:05:07.647 INFO:tasks.workunit.client.0.smithi046.stdout:5/127: readlink d7/lf 0 2024-09-10T04:05:07.647 INFO:tasks.workunit.client.0.smithi046.stdout:9/145: truncate d4/d7/d1f/f1c 121280 0 2024-09-10T04:05:07.648 INFO:tasks.workunit.client.0.smithi046.stdout:5/128: chown d7/d11/d14/c21 7 1 2024-09-10T04:05:07.649 INFO:tasks.workunit.client.0.smithi046.stdout:5/129: symlink d7/d11/d14/d1c/l26 0 2024-09-10T04:05:07.650 INFO:tasks.workunit.client.0.smithi046.stdout:9/146: fdatasync d4/d7/d1f/f1c 0 2024-09-10T04:05:07.652 INFO:tasks.workunit.client.0.smithi046.stdout:5/130: dread d7/fc [0,4194304] 0 2024-09-10T04:05:07.654 INFO:tasks.workunit.client.0.smithi046.stdout:9/147: mknod d4/d18/c2c 0 2024-09-10T04:05:07.658 INFO:tasks.workunit.client.0.smithi046.stdout:9/148: write d4/d7/fe [2201527,30868] 0 2024-09-10T04:05:07.660 INFO:tasks.workunit.client.0.smithi046.stdout:5/131: dwrite d7/f1e [0,4194304] 0 2024-09-10T04:05:07.664 INFO:tasks.workunit.client.0.smithi046.stdout:5/132: rename d7/d11/d14/l18 to d7/d11/l27 0 2024-09-10T04:05:07.668 INFO:tasks.workunit.client.0.smithi046.stdout:5/133: creat d7/d11/f28 x:0 0 0 2024-09-10T04:05:07.668 INFO:tasks.workunit.client.0.smithi046.stdout:9/149: dwrite d4/f6 [0,4194304] 0 2024-09-10T04:05:07.670 INFO:tasks.workunit.client.0.smithi046.stdout:5/134: creat d7/d11/d14/f29 x:0 0 0 2024-09-10T04:05:07.671 INFO:tasks.workunit.client.0.smithi046.stdout:5/135: symlink d7/l2a 0 2024-09-10T04:05:07.672 INFO:tasks.workunit.client.0.smithi046.stdout:5/136: write d7/d11/f23 [707735,30131] 0 2024-09-10T04:05:07.678 INFO:tasks.workunit.client.0.smithi046.stdout:5/137: write d7/fc [187645,126027] 0 2024-09-10T04:05:07.679 INFO:tasks.workunit.client.0.smithi046.stdout:5/138: chown d7/d11/f12 0 1 2024-09-10T04:05:07.685 INFO:tasks.workunit.client.0.smithi046.stdout:5/139: dwrite d7/fe [0,4194304] 0 2024-09-10T04:05:07.686 INFO:tasks.workunit.client.0.smithi046.stdout:5/140: truncate d7/d11/f12 162023 0 2024-09-10T04:05:07.691 INFO:tasks.workunit.client.0.smithi046.stdout:5/141: truncate d7/d11/f23 314147 0 2024-09-10T04:05:07.692 INFO:tasks.workunit.client.0.smithi046.stdout:5/142: write d7/d11/f13 [643331,27541] 0 2024-09-10T04:05:07.695 INFO:tasks.workunit.client.0.smithi046.stdout:5/143: mkdir d7/d11/d14/d1c/d2b 0 2024-09-10T04:05:07.695 INFO:tasks.workunit.client.0.smithi046.stdout:5/144: chown d7/d11/d14/f1d 597 1 2024-09-10T04:05:07.696 INFO:tasks.workunit.client.0.smithi046.stdout:5/145: write d7/fd [2488491,1087] 0 2024-09-10T04:05:07.697 INFO:tasks.workunit.client.0.smithi046.stdout:5/146: write d7/fd [2610622,27564] 0 2024-09-10T04:05:07.703 INFO:tasks.workunit.client.0.smithi046.stdout:5/147: dwrite d7/f19 [0,4194304] 0 2024-09-10T04:05:07.704 INFO:tasks.workunit.client.0.smithi046.stdout:5/148: write d7/fe [744758,122443] 0 2024-09-10T04:05:07.709 INFO:tasks.workunit.client.1.smithi070.stdout:4/99: creat de/d13/d18/f21 x:0 0 0 2024-09-10T04:05:07.710 INFO:tasks.workunit.client.0.smithi046.stdout:5/149: rename d7/l15 to d7/d11/l2c 0 2024-09-10T04:05:07.710 INFO:tasks.workunit.client.1.smithi070.stdout:5/74: dwrite f10 [0,4194304] 0 2024-09-10T04:05:07.711 INFO:tasks.workunit.client.1.smithi070.stdout:4/100: dread - de/d13/d18/f21 zero size 2024-09-10T04:05:07.712 INFO:tasks.workunit.client.1.smithi070.stdout:3/145: write fa [1892559,28803] 0 2024-09-10T04:05:07.712 INFO:tasks.workunit.client.1.smithi070.stdout:5/75: read - f14 zero size 2024-09-10T04:05:07.712 INFO:tasks.workunit.client.1.smithi070.stdout:5/76: rmdir - no directory 2024-09-10T04:05:07.713 INFO:tasks.workunit.client.0.smithi046.stdout:5/150: mkdir d7/d11/d14/d1c/d2d 0 2024-09-10T04:05:07.714 INFO:tasks.workunit.client.1.smithi070.stdout:4/101: write f6 [2519194,26204] 0 2024-09-10T04:05:07.714 INFO:tasks.workunit.client.0.smithi046.stdout:5/151: chown d7/la 12897656 1 2024-09-10T04:05:07.715 INFO:tasks.workunit.client.0.smithi046.stdout:5/152: write d7/d11/f24 [851870,128792] 0 2024-09-10T04:05:07.715 INFO:tasks.workunit.client.0.smithi046.stdout:6/76: sync 2024-09-10T04:05:07.716 INFO:tasks.workunit.client.1.smithi070.stdout:0/126: dread d8/fc [0,4194304] 0 2024-09-10T04:05:07.716 INFO:tasks.workunit.client.1.smithi070.stdout:3/146: creat db/d16/f2a x:0 0 0 2024-09-10T04:05:07.717 INFO:tasks.workunit.client.1.smithi070.stdout:3/147: write db/d16/f2a [819752,104169] 0 2024-09-10T04:05:07.718 INFO:tasks.workunit.client.1.smithi070.stdout:3/148: fdatasync fa 0 2024-09-10T04:05:07.721 INFO:tasks.workunit.client.0.smithi046.stdout:5/153: unlink d7/f8 0 2024-09-10T04:05:07.723 INFO:tasks.workunit.client.1.smithi070.stdout:6/197: getdents d0/d3/d18 0 2024-09-10T04:05:07.724 INFO:tasks.workunit.client.0.smithi046.stdout:6/77: creat da/db/f1c x:0 0 0 2024-09-10T04:05:07.727 INFO:tasks.workunit.client.1.smithi070.stdout:5/77: write f7 [2230703,35344] 0 2024-09-10T04:05:07.730 INFO:tasks.workunit.client.1.smithi070.stdout:4/102: write fd [671737,4264] 0 2024-09-10T04:05:07.731 INFO:tasks.workunit.client.0.smithi046.stdout:5/154: dread d7/d11/f23 [0,4194304] 0 2024-09-10T04:05:07.731 INFO:tasks.workunit.client.1.smithi070.stdout:4/103: fdatasync f6 0 2024-09-10T04:05:07.732 INFO:tasks.workunit.client.0.smithi046.stdout:6/78: dwrite da/db/d10/f17 [0,4194304] 0 2024-09-10T04:05:07.732 INFO:tasks.workunit.client.1.smithi070.stdout:4/104: write de/d13/d18/f21 [314683,23593] 0 2024-09-10T04:05:07.733 INFO:tasks.workunit.client.1.smithi070.stdout:4/105: write f8 [1068384,125437] 0 2024-09-10T04:05:07.734 INFO:tasks.workunit.client.1.smithi070.stdout:5/78: mkdir d17 0 2024-09-10T04:05:07.734 INFO:tasks.workunit.client.1.smithi070.stdout:5/79: readlink l1 0 2024-09-10T04:05:07.736 INFO:tasks.workunit.client.1.smithi070.stdout:5/80: dread fd [0,4194304] 0 2024-09-10T04:05:07.738 INFO:tasks.workunit.client.0.smithi046.stdout:6/79: getdents da/db/d11/d13 0 2024-09-10T04:05:07.739 INFO:tasks.workunit.client.0.smithi046.stdout:5/155: dwrite d7/f1b [0,4194304] 0 2024-09-10T04:05:07.740 INFO:tasks.workunit.client.1.smithi070.stdout:6/198: unlink d0/d3/cf 0 2024-09-10T04:05:07.742 INFO:tasks.workunit.client.1.smithi070.stdout:3/149: truncate db/d16/f1b 2105376 0 2024-09-10T04:05:07.743 INFO:tasks.workunit.client.1.smithi070.stdout:0/127: link d8/d13/f1f d8/d13/f2e 0 2024-09-10T04:05:07.747 INFO:tasks.workunit.client.1.smithi070.stdout:4/106: mknod de/d13/c22 0 2024-09-10T04:05:07.747 INFO:tasks.workunit.client.0.smithi046.stdout:5/156: mknod d7/d11/d14/d1c/c2e 0 2024-09-10T04:05:07.747 INFO:tasks.workunit.client.1.smithi070.stdout:4/107: fdatasync f6 0 2024-09-10T04:05:07.748 INFO:tasks.workunit.client.0.smithi046.stdout:5/157: chown d7/fe 7 1 2024-09-10T04:05:07.748 INFO:tasks.workunit.client.1.smithi070.stdout:6/199: dwrite d0/d3/d18/d1b/d33/f39 [0,4194304] 0 2024-09-10T04:05:07.750 INFO:tasks.workunit.client.1.smithi070.stdout:6/200: chown d0/d3/d18/f42 778721284 1 2024-09-10T04:05:07.750 INFO:tasks.workunit.client.1.smithi070.stdout:3/150: creat db/d16/f2b x:0 0 0 2024-09-10T04:05:07.751 INFO:tasks.workunit.client.1.smithi070.stdout:5/81: mkdir d17/d18 0 2024-09-10T04:05:07.753 INFO:tasks.workunit.client.1.smithi070.stdout:6/201: fsync d0/f8 0 2024-09-10T04:05:07.754 INFO:tasks.workunit.client.1.smithi070.stdout:5/82: mkdir d17/d19 0 2024-09-10T04:05:07.756 INFO:tasks.workunit.client.1.smithi070.stdout:5/83: write fe [699215,54807] 0 2024-09-10T04:05:07.757 INFO:tasks.workunit.client.1.smithi070.stdout:5/84: dread - f14 zero size 2024-09-10T04:05:07.759 INFO:tasks.workunit.client.1.smithi070.stdout:0/128: dwrite d8/d13/d1a/f23 [0,4194304] 0 2024-09-10T04:05:07.763 INFO:tasks.workunit.client.1.smithi070.stdout:0/129: mkdir d8/d13/d1a/d2f 0 2024-09-10T04:05:07.763 INFO:tasks.workunit.client.1.smithi070.stdout:6/202: dread d0/d3/d18/d1b/d33/f39 [0,4194304] 0 2024-09-10T04:05:07.765 INFO:tasks.workunit.client.1.smithi070.stdout:5/85: dwrite f11 [0,4194304] 0 2024-09-10T04:05:07.767 INFO:tasks.workunit.client.0.smithi046.stdout:6/80: dwrite f9 [0,4194304] 0 2024-09-10T04:05:07.768 INFO:tasks.workunit.client.1.smithi070.stdout:0/130: write d8/d13/f28 [592819,16691] 0 2024-09-10T04:05:07.771 INFO:tasks.workunit.client.0.smithi046.stdout:6/81: creat da/db/f1d x:0 0 0 2024-09-10T04:05:07.776 INFO:tasks.workunit.client.1.smithi070.stdout:6/203: chown d0 7845 1 2024-09-10T04:05:07.776 INFO:tasks.workunit.client.1.smithi070.stdout:5/86: mknod d17/d19/c1a 0 2024-09-10T04:05:07.779 INFO:tasks.workunit.client.0.smithi046.stdout:6/82: mkdir da/db/d11/d13/d1e 0 2024-09-10T04:05:07.780 INFO:tasks.workunit.client.1.smithi070.stdout:6/204: stat d0/d3/d18/f42 0 2024-09-10T04:05:07.780 INFO:tasks.workunit.client.0.smithi046.stdout:6/83: read f9 [2852550,72565] 0 2024-09-10T04:05:07.780 INFO:tasks.workunit.client.0.smithi046.stdout:6/84: readlink l5 0 2024-09-10T04:05:07.782 INFO:tasks.workunit.client.1.smithi070.stdout:0/131: link d8/f2b d8/d13/d1a/f30 0 2024-09-10T04:05:07.782 INFO:tasks.workunit.client.0.smithi046.stdout:6/85: creat da/f1f x:0 0 0 2024-09-10T04:05:07.785 INFO:tasks.workunit.client.1.smithi070.stdout:6/205: dwrite d0/d3/d18/d1b/f3a [0,4194304] 0 2024-09-10T04:05:07.788 INFO:tasks.workunit.client.1.smithi070.stdout:5/87: rmdir d17/d18 0 2024-09-10T04:05:07.789 INFO:tasks.workunit.client.1.smithi070.stdout:6/206: creat d0/d3/d18/d1b/d17/d28/f43 x:0 0 0 2024-09-10T04:05:07.790 INFO:tasks.workunit.client.0.smithi046.stdout:6/86: dwrite da/db/d10/f17 [4194304,4194304] 0 2024-09-10T04:05:07.791 INFO:tasks.workunit.client.1.smithi070.stdout:0/132: rmdir d8/d13/d1a/d2f 0 2024-09-10T04:05:07.791 INFO:tasks.workunit.client.0.smithi046.stdout:6/87: mknod da/c20 0 2024-09-10T04:05:07.792 INFO:tasks.workunit.client.0.smithi046.stdout:6/88: write f6 [2928859,55957] 0 2024-09-10T04:05:07.795 INFO:tasks.workunit.client.1.smithi070.stdout:5/88: dwrite f10 [0,4194304] 0 2024-09-10T04:05:07.795 INFO:tasks.workunit.client.1.smithi070.stdout:5/89: readlink l1 0 2024-09-10T04:05:07.800 INFO:tasks.workunit.client.0.smithi046.stdout:6/89: getdents da/db/d11/d13/d1e 0 2024-09-10T04:05:07.805 INFO:tasks.workunit.client.1.smithi070.stdout:6/207: truncate d0/d3/f16 803446 0 2024-09-10T04:05:07.809 INFO:tasks.workunit.client.0.smithi046.stdout:6/90: dwrite da/db/d10/f1a [0,4194304] 0 2024-09-10T04:05:07.809 INFO:tasks.workunit.client.1.smithi070.stdout:6/208: dread d0/f2e [0,4194304] 0 2024-09-10T04:05:07.811 INFO:tasks.workunit.client.1.smithi070.stdout:6/209: chown d0/d3/d18/f27 8801 1 2024-09-10T04:05:07.812 INFO:tasks.workunit.client.1.smithi070.stdout:5/90: dwrite fd [0,4194304] 0 2024-09-10T04:05:07.813 INFO:tasks.workunit.client.0.smithi046.stdout:6/91: creat da/db/d11/d13/d1e/f21 x:0 0 0 2024-09-10T04:05:07.814 INFO:tasks.workunit.client.0.smithi046.stdout:6/92: write da/db/f16 [524919,107670] 0 2024-09-10T04:05:07.816 INFO:tasks.workunit.client.1.smithi070.stdout:5/91: creat d17/f1b x:0 0 0 2024-09-10T04:05:07.818 INFO:tasks.workunit.client.0.smithi046.stdout:6/93: dread f1 [0,4194304] 0 2024-09-10T04:05:07.819 INFO:tasks.workunit.client.1.smithi070.stdout:5/92: creat d17/d19/f1c x:0 0 0 2024-09-10T04:05:07.819 INFO:tasks.workunit.client.1.smithi070.stdout:6/210: link d0/d3/d18/c3c d0/d3/d18/d1b/c44 0 2024-09-10T04:05:07.820 INFO:tasks.workunit.client.1.smithi070.stdout:5/93: fsync f14 0 2024-09-10T04:05:07.821 INFO:tasks.workunit.client.0.smithi046.stdout:6/94: mkdir da/dc/d22 0 2024-09-10T04:05:07.823 INFO:tasks.workunit.client.1.smithi070.stdout:5/94: creat d17/d19/f1d x:0 0 0 2024-09-10T04:05:07.825 INFO:tasks.workunit.client.1.smithi070.stdout:5/95: creat d17/d19/f1e x:0 0 0 2024-09-10T04:05:07.826 INFO:tasks.workunit.client.1.smithi070.stdout:5/96: symlink d17/d19/l1f 0 2024-09-10T04:05:07.829 INFO:tasks.workunit.client.0.smithi046.stdout:6/95: dwrite da/db/f1c [0,4194304] 0 2024-09-10T04:05:07.833 INFO:tasks.workunit.client.1.smithi070.stdout:5/97: dwrite f16 [0,4194304] 0 2024-09-10T04:05:07.833 INFO:tasks.workunit.client.1.smithi070.stdout:5/98: fsync f10 0 2024-09-10T04:05:07.834 INFO:tasks.workunit.client.0.smithi046.stdout:6/96: dread da/fd [0,4194304] 0 2024-09-10T04:05:07.850 INFO:tasks.workunit.client.1.smithi070.stdout:5/99: truncate fe 992228 0 2024-09-10T04:05:07.853 INFO:tasks.workunit.client.0.smithi046.stdout:6/97: mkdir da/db/d11/d13/d1e/d23 0 2024-09-10T04:05:07.853 INFO:tasks.workunit.client.1.smithi070.stdout:5/100: unlink f6 0 2024-09-10T04:05:07.854 INFO:tasks.workunit.client.1.smithi070.stdout:5/101: chown d17/d19/c1a 6084 1 2024-09-10T04:05:07.859 INFO:tasks.workunit.client.1.smithi070.stdout:5/102: dwrite f14 [0,4194304] 0 2024-09-10T04:05:07.864 INFO:tasks.workunit.client.1.smithi070.stdout:5/103: unlink d17/d19/f1c 0 2024-09-10T04:05:07.866 INFO:tasks.workunit.client.1.smithi070.stdout:5/104: creat d17/f20 x:0 0 0 2024-09-10T04:05:07.908 INFO:tasks.workunit.client.1.smithi070.stdout:5/105: write f8 [118934,20294] 0 2024-09-10T04:05:07.912 INFO:tasks.workunit.client.1.smithi070.stdout:5/106: symlink d17/d19/l21 0 2024-09-10T04:05:07.913 INFO:tasks.workunit.client.1.smithi070.stdout:5/107: chown d17/f20 3200455 1 2024-09-10T04:05:07.913 INFO:tasks.workunit.client.0.smithi046.stdout:0/70: sync 2024-09-10T04:05:07.913 INFO:tasks.workunit.client.0.smithi046.stdout:0/71: chown d1/d6 258719 1 2024-09-10T04:05:07.914 INFO:tasks.workunit.client.0.smithi046.stdout:0/72: write d1/d6/f11 [8295843,62086] 0 2024-09-10T04:05:07.916 INFO:tasks.workunit.client.1.smithi070.stdout:5/108: creat d17/d19/f22 x:0 0 0 2024-09-10T04:05:07.916 INFO:tasks.workunit.client.1.smithi070.stdout:5/109: fdatasync d17/d19/f1e 0 2024-09-10T04:05:07.953 INFO:tasks.workunit.client.1.smithi070.stdout:2/88: sync 2024-09-10T04:05:07.953 INFO:tasks.workunit.client.1.smithi070.stdout:7/104: sync 2024-09-10T04:05:07.954 INFO:tasks.workunit.client.1.smithi070.stdout:9/116: sync 2024-09-10T04:05:07.954 INFO:tasks.workunit.client.1.smithi070.stdout:1/85: sync 2024-09-10T04:05:07.954 INFO:tasks.workunit.client.1.smithi070.stdout:7/105: dread - d2/d3/dc/d13/f1a zero size 2024-09-10T04:05:07.954 INFO:tasks.workunit.client.1.smithi070.stdout:7/106: chown d2/d7 51549 1 2024-09-10T04:05:07.955 INFO:tasks.workunit.client.1.smithi070.stdout:7/107: chown d2/d3/dc 100 1 2024-09-10T04:05:07.970 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:07 smithi046 ceph-mon[28328]: pgmap v257: 65 pgs: 65 active+clean; 362 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 274 op/s 2024-09-10T04:05:07.972 INFO:tasks.workunit.client.1.smithi070.stdout:5/110: creat d17/d19/f23 x:0 0 0 2024-09-10T04:05:07.973 INFO:tasks.workunit.client.1.smithi070.stdout:5/111: truncate d17/f1b 157343 0 2024-09-10T04:05:07.973 INFO:tasks.workunit.client.1.smithi070.stdout:2/89: mkdir d15 0 2024-09-10T04:05:07.978 INFO:tasks.workunit.client.1.smithi070.stdout:5/112: read f13 [2957974,76590] 0 2024-09-10T04:05:07.980 INFO:tasks.workunit.client.1.smithi070.stdout:9/117: symlink d9/d1c/l23 0 2024-09-10T04:05:07.981 INFO:tasks.workunit.client.1.smithi070.stdout:9/118: write d9/dd/f12 [1541671,41487] 0 2024-09-10T04:05:07.983 INFO:tasks.workunit.client.1.smithi070.stdout:7/108: mknod d2/d7/d9/d18/d23/c26 0 2024-09-10T04:05:07.985 INFO:tasks.workunit.client.1.smithi070.stdout:7/109: dread - d2/d3/dc/d1b/f1d zero size 2024-09-10T04:05:07.987 INFO:tasks.workunit.client.1.smithi070.stdout:5/113: write fc [6260,108407] 0 2024-09-10T04:05:07.987 INFO:tasks.workunit.client.1.smithi070.stdout:9/119: dwrite d9/dd/f14 [0,4194304] 0 2024-09-10T04:05:07.990 INFO:tasks.workunit.client.1.smithi070.stdout:1/86: link d3/l4 d3/l22 0 2024-09-10T04:05:07.992 INFO:tasks.workunit.client.1.smithi070.stdout:7/110: rmdir d2/d3/dc/d13 39 2024-09-10T04:05:07.992 INFO:tasks.workunit.client.1.smithi070.stdout:7/111: chown d2/f15 1595271 1 2024-09-10T04:05:07.993 INFO:tasks.workunit.client.1.smithi070.stdout:7/112: fdatasync d2/d3/dc/d1b/f1d 0 2024-09-10T04:05:07.996 INFO:tasks.workunit.client.1.smithi070.stdout:7/113: chown d2/c12 383349 1 2024-09-10T04:05:08.000 INFO:tasks.workunit.client.1.smithi070.stdout:7/114: rename d2/d7/db/l1c to d2/d7/l27 0 2024-09-10T04:05:08.001 INFO:tasks.workunit.client.1.smithi070.stdout:5/114: getdents d17 0 2024-09-10T04:05:08.029 INFO:tasks.workunit.client.0.smithi046.stdout:6/98: creat da/f24 x:0 0 0 2024-09-10T04:05:08.030 INFO:tasks.workunit.client.0.smithi046.stdout:0/73: creat d1/dc/f16 x:0 0 0 2024-09-10T04:05:08.030 INFO:tasks.workunit.client.0.smithi046.stdout:6/99: mknod da/dc/c25 0 2024-09-10T04:05:08.032 INFO:tasks.workunit.client.0.smithi046.stdout:6/100: dread f1 [0,4194304] 0 2024-09-10T04:05:08.033 INFO:tasks.workunit.client.0.smithi046.stdout:0/74: dread d1/d6/f11 [4194304,4194304] 0 2024-09-10T04:05:08.034 INFO:tasks.workunit.client.0.smithi046.stdout:0/75: stat d1/d6/d3 0 2024-09-10T04:05:08.036 INFO:tasks.workunit.client.0.smithi046.stdout:0/76: unlink d1/d6/d3/f10 0 2024-09-10T04:05:08.037 INFO:tasks.workunit.client.0.smithi046.stdout:0/77: write d1/dc/f12 [2377487,3388] 0 2024-09-10T04:05:08.038 INFO:tasks.workunit.client.0.smithi046.stdout:0/78: creat d1/dc/f17 x:0 0 0 2024-09-10T04:05:08.039 INFO:tasks.workunit.client.0.smithi046.stdout:0/79: dread - d1/dc/f17 zero size 2024-09-10T04:05:08.039 INFO:tasks.workunit.client.0.smithi046.stdout:0/80: stat d1/d6 0 2024-09-10T04:05:08.040 INFO:tasks.workunit.client.0.smithi046.stdout:0/81: write d1/dc/f17 [967706,103365] 0 2024-09-10T04:05:08.043 INFO:tasks.workunit.client.1.smithi070.stdout:7/115: dread d2/f24 [0,4194304] 0 2024-09-10T04:05:08.043 INFO:tasks.workunit.client.1.smithi070.stdout:5/115: truncate ff 170488 0 2024-09-10T04:05:08.044 INFO:tasks.workunit.client.0.smithi046.stdout:0/82: mknod d1/d13/c18 0 2024-09-10T04:05:08.045 INFO:tasks.workunit.client.1.smithi070.stdout:2/90: dwrite f3 [0,4194304] 0 2024-09-10T04:05:08.046 INFO:tasks.workunit.client.1.smithi070.stdout:5/116: mkdir d17/d24 0 2024-09-10T04:05:08.058 INFO:tasks.workunit.client.0.smithi046.stdout:0/83: truncate d1/d6/f11 4236550 0 2024-09-10T04:05:08.060 INFO:tasks.workunit.client.0.smithi046.stdout:0/84: rmdir d1/d6/d3/dd 39 2024-09-10T04:05:08.061 INFO:tasks.workunit.client.0.smithi046.stdout:0/85: mkdir d1/d6/d4/d19 0 2024-09-10T04:05:08.062 INFO:tasks.workunit.client.0.smithi046.stdout:0/86: write d1/d6/d3/f14 [4427976,41758] 0 2024-09-10T04:05:08.067 INFO:tasks.workunit.client.0.smithi046.stdout:0/87: write d1/d6/f11 [3735674,37339] 0 2024-09-10T04:05:08.067 INFO:tasks.workunit.client.0.smithi046.stdout:0/88: chown d1 37 1 2024-09-10T04:05:08.073 INFO:tasks.workunit.client.0.smithi046.stdout:0/89: dwrite d1/d6/d3/f14 [0,4194304] 0 2024-09-10T04:05:08.075 INFO:tasks.workunit.client.0.smithi046.stdout:0/90: mkdir d1/d6/d1a 0 2024-09-10T04:05:08.076 INFO:tasks.workunit.client.0.smithi046.stdout:0/91: stat d1/d6/d3/dd 0 2024-09-10T04:05:08.078 INFO:tasks.workunit.client.0.smithi046.stdout:0/92: creat d1/d6/f1b x:0 0 0 2024-09-10T04:05:08.080 INFO:tasks.workunit.client.0.smithi046.stdout:0/93: creat d1/d13/f1c x:0 0 0 2024-09-10T04:05:08.082 INFO:tasks.workunit.client.0.smithi046.stdout:0/94: unlink d1/dc/f12 0 2024-09-10T04:05:08.082 INFO:tasks.workunit.client.0.smithi046.stdout:0/95: readlink d1/d6/d4/l8 0 2024-09-10T04:05:08.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:07 smithi070 ceph-mon[35495]: pgmap v257: 65 pgs: 65 active+clean; 362 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 15 MiB/s wr, 274 op/s 2024-09-10T04:05:08.086 INFO:tasks.workunit.client.1.smithi070.stdout:2/91: mknod d15/c16 0 2024-09-10T04:05:08.086 INFO:tasks.workunit.client.1.smithi070.stdout:8/170: sync 2024-09-10T04:05:08.086 INFO:tasks.workunit.client.1.smithi070.stdout:4/108: sync 2024-09-10T04:05:08.087 INFO:tasks.workunit.client.1.smithi070.stdout:4/109: dread - de/d13/f1f zero size 2024-09-10T04:05:08.087 INFO:tasks.workunit.client.1.smithi070.stdout:8/171: dread - d4/d16/d17/f2c zero size 2024-09-10T04:05:08.087 INFO:tasks.workunit.client.0.smithi046.stdout:0/96: dread d1/dc/f17 [0,4194304] 0 2024-09-10T04:05:08.088 INFO:tasks.workunit.client.1.smithi070.stdout:5/117: rename ff to d17/d24/f25 0 2024-09-10T04:05:08.088 INFO:tasks.workunit.client.1.smithi070.stdout:4/110: write fb [1019823,80012] 0 2024-09-10T04:05:08.088 INFO:tasks.workunit.client.1.smithi070.stdout:5/118: stat fc 0 2024-09-10T04:05:08.089 INFO:tasks.workunit.client.1.smithi070.stdout:7/116: symlink d2/d7/d9/d18/l28 0 2024-09-10T04:05:08.090 INFO:tasks.workunit.client.1.smithi070.stdout:5/119: write d17/d19/f1d [1048077,90496] 0 2024-09-10T04:05:08.090 INFO:tasks.workunit.client.0.smithi046.stdout:0/97: creat d1/d13/f1d x:0 0 0 2024-09-10T04:05:08.091 INFO:tasks.workunit.client.1.smithi070.stdout:5/120: dread - d17/f20 zero size 2024-09-10T04:05:08.092 INFO:tasks.workunit.client.1.smithi070.stdout:5/121: write d17/d19/f22 [639334,17488] 0 2024-09-10T04:05:08.092 INFO:tasks.workunit.client.1.smithi070.stdout:8/172: mknod d4/d16/d17/d19/c3d 0 2024-09-10T04:05:08.092 INFO:tasks.workunit.client.1.smithi070.stdout:4/111: dread de/d13/d18/f21 [0,4194304] 0 2024-09-10T04:05:08.093 INFO:tasks.workunit.client.1.smithi070.stdout:5/122: readlink d17/d19/l21 0 2024-09-10T04:05:08.094 INFO:tasks.workunit.client.0.smithi046.stdout:0/98: unlink d1/d6/d4/l8 0 2024-09-10T04:05:08.099 INFO:tasks.workunit.client.0.smithi046.stdout:0/99: rmdir d1/dc 39 2024-09-10T04:05:08.103 INFO:tasks.workunit.client.0.smithi046.stdout:0/100: unlink d1/d13/f1c 0 2024-09-10T04:05:08.104 INFO:tasks.workunit.client.1.smithi070.stdout:5/123: symlink d17/l26 0 2024-09-10T04:05:08.105 INFO:tasks.workunit.client.1.smithi070.stdout:8/173: symlink d4/d16/d17/d1a/d23/l3e 0 2024-09-10T04:05:08.106 INFO:tasks.workunit.client.1.smithi070.stdout:2/92: link l7 d15/l17 0 2024-09-10T04:05:08.106 INFO:tasks.workunit.client.1.smithi070.stdout:8/174: dread - d4/d16/d17/d19/d28/f31 zero size 2024-09-10T04:05:08.107 INFO:tasks.workunit.client.0.smithi046.stdout:0/101: mkdir d1/d1e 0 2024-09-10T04:05:08.108 INFO:tasks.workunit.client.0.smithi046.stdout:0/102: write d1/d6/f1b [918836,73094] 0 2024-09-10T04:05:08.109 INFO:tasks.workunit.client.0.smithi046.stdout:0/103: rename d1 to d1/d13/d1f 22 2024-09-10T04:05:08.112 INFO:tasks.workunit.client.1.smithi070.stdout:4/112: getdents de/d13/d18 0 2024-09-10T04:05:08.113 INFO:tasks.workunit.client.0.smithi046.stdout:0/104: symlink d1/d6/d3/dd/l20 0 2024-09-10T04:05:08.114 INFO:tasks.workunit.client.1.smithi070.stdout:7/117: link d2/d7/db/d20/l22 d2/l29 0 2024-09-10T04:05:08.118 INFO:tasks.workunit.client.0.smithi046.stdout:0/105: creat d1/dc/f21 x:0 0 0 2024-09-10T04:05:08.119 INFO:tasks.workunit.client.0.smithi046.stdout:0/106: chown d1/d6/d4/d19 410 1 2024-09-10T04:05:08.121 INFO:tasks.workunit.client.0.smithi046.stdout:0/107: symlink d1/d6/l22 0 2024-09-10T04:05:08.125 INFO:tasks.workunit.client.0.smithi046.stdout:0/108: link d1/d6/f11 d1/d6/d3/dd/f23 0 2024-09-10T04:05:08.125 INFO:tasks.workunit.client.0.smithi046.stdout:0/109: chown d1/d6/l22 800148 1 2024-09-10T04:05:08.126 INFO:tasks.workunit.client.0.smithi046.stdout:0/110: write d1/d6/d3/f14 [786501,109100] 0 2024-09-10T04:05:08.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/111: link d1/d13/c18 d1/d6/c24 0 2024-09-10T04:05:08.130 INFO:tasks.workunit.client.0.smithi046.stdout:0/112: write d1/d13/f1d [549449,55506] 0 2024-09-10T04:05:08.135 INFO:tasks.workunit.client.0.smithi046.stdout:0/113: dwrite d1/d6/f1b [0,4194304] 0 2024-09-10T04:05:08.144 INFO:tasks.workunit.client.1.smithi070.stdout:5/124: dwrite f8 [0,4194304] 0 2024-09-10T04:05:08.144 INFO:tasks.workunit.client.1.smithi070.stdout:4/113: mkdir de/d13/d18/d23 0 2024-09-10T04:05:08.145 INFO:tasks.workunit.client.0.smithi046.stdout:0/114: dwrite d1/d6/f11 [4194304,4194304] 0 2024-09-10T04:05:08.148 INFO:tasks.workunit.client.1.smithi070.stdout:8/175: link d4/f13 d4/d16/f3f 0 2024-09-10T04:05:08.150 INFO:tasks.workunit.client.1.smithi070.stdout:4/114: mkdir de/d13/d18/d24 0 2024-09-10T04:05:08.152 INFO:tasks.workunit.client.1.smithi070.stdout:7/118: symlink d2/d3/dc/l2a 0 2024-09-10T04:05:08.153 INFO:tasks.workunit.client.1.smithi070.stdout:5/125: dread f12 [0,4194304] 0 2024-09-10T04:05:08.154 INFO:tasks.workunit.client.0.smithi046.stdout:0/115: dwrite d1/dc/f16 [0,4194304] 0 2024-09-10T04:05:08.157 INFO:tasks.workunit.client.1.smithi070.stdout:7/119: rmdir d2/d7/d9 39 2024-09-10T04:05:08.158 INFO:tasks.workunit.client.0.smithi046.stdout:0/116: mkdir d1/dc/d25 0 2024-09-10T04:05:08.158 INFO:tasks.workunit.client.0.smithi046.stdout:0/117: chown d1/d13 511 1 2024-09-10T04:05:08.160 INFO:tasks.workunit.client.1.smithi070.stdout:5/126: fdatasync f13 0 2024-09-10T04:05:08.162 INFO:tasks.workunit.client.1.smithi070.stdout:4/115: rmdir de/d13/d18/d24 0 2024-09-10T04:05:08.165 INFO:tasks.workunit.client.1.smithi070.stdout:7/120: dwrite d2/d3/dc/d1b/f1d [0,4194304] 0 2024-09-10T04:05:08.166 INFO:tasks.workunit.client.0.smithi046.stdout:0/118: dwrite d1/d6/d3/f14 [0,4194304] 0 2024-09-10T04:05:08.166 INFO:tasks.workunit.client.1.smithi070.stdout:5/127: creat d17/d24/f27 x:0 0 0 2024-09-10T04:05:08.169 INFO:tasks.workunit.client.1.smithi070.stdout:4/116: symlink de/d13/d18/l25 0 2024-09-10T04:05:08.170 INFO:tasks.workunit.client.0.smithi046.stdout:7/122: sync 2024-09-10T04:05:08.171 INFO:tasks.workunit.client.1.smithi070.stdout:4/117: write fd [1215531,109456] 0 2024-09-10T04:05:08.173 INFO:tasks.workunit.client.1.smithi070.stdout:7/121: chown d2/d7/la 377 1 2024-09-10T04:05:08.174 INFO:tasks.workunit.client.0.smithi046.stdout:0/119: dwrite d1/d13/f1d [0,4194304] 0 2024-09-10T04:05:08.176 INFO:tasks.workunit.client.0.smithi046.stdout:7/123: creat d5/d9/d12/f2e x:0 0 0 2024-09-10T04:05:08.176 INFO:tasks.workunit.client.1.smithi070.stdout:5/128: dwrite f3 [0,4194304] 0 2024-09-10T04:05:08.177 INFO:tasks.workunit.client.1.smithi070.stdout:4/118: truncate de/d13/d18/f21 405062 0 2024-09-10T04:05:08.180 INFO:tasks.workunit.client.0.smithi046.stdout:0/120: mknod d1/d6/d3/c26 0 2024-09-10T04:05:08.183 INFO:tasks.workunit.client.1.smithi070.stdout:7/122: dwrite f1 [0,4194304] 0 2024-09-10T04:05:08.184 INFO:tasks.workunit.client.1.smithi070.stdout:8/176: truncate d4/f15 547772 0 2024-09-10T04:05:08.186 INFO:tasks.workunit.client.0.smithi046.stdout:7/124: dwrite d5/d6/db/f2a [0,4194304] 0 2024-09-10T04:05:08.186 INFO:tasks.workunit.client.0.smithi046.stdout:0/121: creat d1/dc/f27 x:0 0 0 2024-09-10T04:05:08.186 INFO:tasks.workunit.client.1.smithi070.stdout:7/123: write d2/d3/dc/f1e [879849,89529] 0 2024-09-10T04:05:08.189 INFO:tasks.workunit.client.0.smithi046.stdout:0/122: write d1/dc/f16 [3193866,37801] 0 2024-09-10T04:05:08.190 INFO:tasks.workunit.client.0.smithi046.stdout:7/125: write d5/d6/db/d11/d1f/f27 [895175,47521] 0 2024-09-10T04:05:08.190 INFO:tasks.workunit.client.0.smithi046.stdout:0/123: stat d1/d6/l22 0 2024-09-10T04:05:08.190 INFO:tasks.workunit.client.1.smithi070.stdout:3/151: sync 2024-09-10T04:05:08.191 INFO:tasks.workunit.client.1.smithi070.stdout:0/133: sync 2024-09-10T04:05:08.191 INFO:tasks.workunit.client.1.smithi070.stdout:6/211: sync 2024-09-10T04:05:08.191 INFO:tasks.workunit.client.0.smithi046.stdout:0/124: rename d1/d6 to d1/d6/d4/d28 22 2024-09-10T04:05:08.191 INFO:tasks.workunit.client.1.smithi070.stdout:8/177: creat d4/d16/d17/f40 x:0 0 0 2024-09-10T04:05:08.192 INFO:tasks.workunit.client.0.smithi046.stdout:0/125: chown d1/d6 458 1 2024-09-10T04:05:08.195 INFO:tasks.workunit.client.1.smithi070.stdout:3/152: dread db/f27 [0,4194304] 0 2024-09-10T04:05:08.198 INFO:tasks.workunit.client.1.smithi070.stdout:4/119: truncate de/d13/f16 3608160 0 2024-09-10T04:05:08.199 INFO:tasks.workunit.client.1.smithi070.stdout:5/129: link d17/d19/l1f d17/d19/l28 0 2024-09-10T04:05:08.201 INFO:tasks.workunit.client.0.smithi046.stdout:0/126: dwrite d1/dc/f16 [0,4194304] 0 2024-09-10T04:05:08.201 INFO:tasks.workunit.client.0.smithi046.stdout:7/126: fdatasync f3 0 2024-09-10T04:05:08.203 INFO:tasks.workunit.client.0.smithi046.stdout:0/127: read - d1/dc/f21 zero size 2024-09-10T04:05:08.208 INFO:tasks.workunit.client.1.smithi070.stdout:0/134: creat d8/d13/d1a/f31 x:0 0 0 2024-09-10T04:05:08.209 INFO:tasks.workunit.client.0.smithi046.stdout:7/127: creat d5/d6/db/dd/f2f x:0 0 0 2024-09-10T04:05:08.213 INFO:tasks.workunit.client.1.smithi070.stdout:9/120: sync 2024-09-10T04:05:08.213 INFO:tasks.workunit.client.1.smithi070.stdout:1/87: sync 2024-09-10T04:05:08.214 INFO:tasks.workunit.client.0.smithi046.stdout:7/128: creat d5/d6/db/d11/d1e/f30 x:0 0 0 2024-09-10T04:05:08.216 INFO:tasks.workunit.client.1.smithi070.stdout:3/153: creat db/d16/f2c x:0 0 0 2024-09-10T04:05:08.216 INFO:tasks.workunit.client.0.smithi046.stdout:7/129: write d5/f2c [559434,59899] 0 2024-09-10T04:05:08.218 INFO:tasks.workunit.client.1.smithi070.stdout:4/120: mkdir de/d13/d18/d26 0 2024-09-10T04:05:08.219 INFO:tasks.workunit.client.1.smithi070.stdout:4/121: fdatasync fd 0 2024-09-10T04:05:08.219 INFO:tasks.workunit.client.1.smithi070.stdout:4/122: fsync de/f11 0 2024-09-10T04:05:08.220 INFO:tasks.workunit.client.1.smithi070.stdout:5/130: write fe [1913895,68069] 0 2024-09-10T04:05:08.220 INFO:tasks.workunit.client.1.smithi070.stdout:4/123: stat de/d13/d18 0 2024-09-10T04:05:08.221 INFO:tasks.workunit.client.1.smithi070.stdout:7/124: symlink d2/d3/dc/d13/l2b 0 2024-09-10T04:05:08.224 INFO:tasks.workunit.client.0.smithi046.stdout:0/128: getdents d1/d6 0 2024-09-10T04:05:08.225 INFO:tasks.workunit.client.1.smithi070.stdout:0/135: unlink d8/d13/f2e 0 2024-09-10T04:05:08.225 INFO:tasks.workunit.client.1.smithi070.stdout:5/131: dwrite d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:08.226 INFO:tasks.workunit.client.0.smithi046.stdout:7/130: creat d5/d6/db/dd/d24/f31 x:0 0 0 2024-09-10T04:05:08.229 INFO:tasks.workunit.client.0.smithi046.stdout:7/131: mkdir d5/d6/d32 0 2024-09-10T04:05:08.230 INFO:tasks.workunit.client.0.smithi046.stdout:0/129: mkdir d1/d6/d3/d29 0 2024-09-10T04:05:08.231 INFO:tasks.workunit.client.0.smithi046.stdout:0/130: write d1/d13/f1d [2150537,7445] 0 2024-09-10T04:05:08.231 INFO:tasks.workunit.client.1.smithi070.stdout:9/121: creat d9/dd/d10/f24 x:0 0 0 2024-09-10T04:05:08.232 INFO:tasks.workunit.client.0.smithi046.stdout:7/132: mkdir d5/d33 0 2024-09-10T04:05:08.233 INFO:tasks.workunit.client.1.smithi070.stdout:1/88: symlink d3/d8/d13/l23 0 2024-09-10T04:05:08.233 INFO:tasks.workunit.client.1.smithi070.stdout:1/89: fsync d3/f5 0 2024-09-10T04:05:08.234 INFO:tasks.workunit.client.1.smithi070.stdout:3/154: fdatasync db/f22 0 2024-09-10T04:05:08.234 INFO:tasks.workunit.client.1.smithi070.stdout:3/155: readlink db/l1d 0 2024-09-10T04:05:08.235 INFO:tasks.workunit.client.0.smithi046.stdout:7/133: symlink d5/d9/d12/l34 0 2024-09-10T04:05:08.238 INFO:tasks.workunit.client.1.smithi070.stdout:6/212: truncate d0/d3/d18/d1b/f3a 282490 0 2024-09-10T04:05:08.238 INFO:tasks.workunit.client.1.smithi070.stdout:6/213: chown d0/d3/d18/f27 13 1 2024-09-10T04:05:08.239 INFO:tasks.workunit.client.0.smithi046.stdout:7/134: creat d5/f35 x:0 0 0 2024-09-10T04:05:08.240 INFO:tasks.workunit.client.0.smithi046.stdout:7/135: write d5/d6/db/dd/f2f [1044707,125909] 0 2024-09-10T04:05:08.241 INFO:tasks.workunit.client.0.smithi046.stdout:0/131: creat d1/d6/d3/f2a x:0 0 0 2024-09-10T04:05:08.242 INFO:tasks.workunit.client.1.smithi070.stdout:6/214: dread d0/d3/d18/d1b/d33/f39 [0,4194304] 0 2024-09-10T04:05:08.242 INFO:tasks.workunit.client.0.smithi046.stdout:0/132: write d1/dc/f27 [778017,77285] 0 2024-09-10T04:05:08.243 INFO:tasks.workunit.client.1.smithi070.stdout:7/125: dread d2/d7/fe [0,4194304] 0 2024-09-10T04:05:08.243 INFO:tasks.workunit.client.1.smithi070.stdout:7/126: chown d2/d3/dc/f1e 734 1 2024-09-10T04:05:08.245 INFO:tasks.workunit.client.0.smithi046.stdout:7/136: rename d5/d9/d12/f2e to d5/d6/d23/f36 0 2024-09-10T04:05:08.246 INFO:tasks.workunit.client.1.smithi070.stdout:5/132: symlink d17/d19/l29 0 2024-09-10T04:05:08.247 INFO:tasks.workunit.client.0.smithi046.stdout:0/133: symlink d1/d6/d3/dd/l2b 0 2024-09-10T04:05:08.250 INFO:tasks.workunit.client.1.smithi070.stdout:8/178: creat d4/d16/f41 x:0 0 0 2024-09-10T04:05:08.252 INFO:tasks.workunit.client.0.smithi046.stdout:7/137: rmdir d5/d6/db/d11 39 2024-09-10T04:05:08.252 INFO:tasks.workunit.client.0.smithi046.stdout:7/138: dread - d5/f35 zero size 2024-09-10T04:05:08.253 INFO:tasks.workunit.client.0.smithi046.stdout:7/139: stat d5/d6/db/f2a 0 2024-09-10T04:05:08.253 INFO:tasks.workunit.client.1.smithi070.stdout:9/122: creat d9/f25 x:0 0 0 2024-09-10T04:05:08.254 INFO:tasks.workunit.client.1.smithi070.stdout:9/123: chown d9/l16 3 1 2024-09-10T04:05:08.255 INFO:tasks.workunit.client.1.smithi070.stdout:9/124: dread d9/dd/f1a [0,4194304] 0 2024-09-10T04:05:08.256 INFO:tasks.workunit.client.1.smithi070.stdout:1/90: fsync d3/d8/d13/f14 0 2024-09-10T04:05:08.257 INFO:tasks.workunit.client.0.smithi046.stdout:7/140: creat d5/d6/d23/f37 x:0 0 0 2024-09-10T04:05:08.257 INFO:tasks.workunit.client.1.smithi070.stdout:0/136: write d8/d13/d1a/f30 [564021,128422] 0 2024-09-10T04:05:08.258 INFO:tasks.workunit.client.1.smithi070.stdout:3/156: rename db/d1c/f21 to db/d16/f2d 0 2024-09-10T04:05:08.260 INFO:tasks.workunit.client.0.smithi046.stdout:0/134: rename d1/d6/c24 to d1/dc/d25/c2c 0 2024-09-10T04:05:08.260 INFO:tasks.workunit.client.1.smithi070.stdout:9/125: dread d9/dd/f14 [0,4194304] 0 2024-09-10T04:05:08.261 INFO:tasks.workunit.client.1.smithi070.stdout:4/124: unlink de/d13/l14 0 2024-09-10T04:05:08.261 INFO:tasks.workunit.client.1.smithi070.stdout:9/126: read d9/dd/f1a [479024,53218] 0 2024-09-10T04:05:08.262 INFO:tasks.workunit.client.1.smithi070.stdout:4/125: stat de/d13/l1d 0 2024-09-10T04:05:08.263 INFO:tasks.workunit.client.1.smithi070.stdout:0/137: dwrite f5 [0,4194304] 0 2024-09-10T04:05:08.264 INFO:tasks.workunit.client.1.smithi070.stdout:0/138: stat d8/d13/f22 0 2024-09-10T04:05:08.264 INFO:tasks.workunit.client.1.smithi070.stdout:0/139: fsync d8/f12 0 2024-09-10T04:05:08.265 INFO:tasks.workunit.client.1.smithi070.stdout:5/133: creat d17/d24/f2a x:0 0 0 2024-09-10T04:05:08.265 INFO:tasks.workunit.client.0.smithi046.stdout:0/135: mkdir d1/d6/d3/d29/d2d 0 2024-09-10T04:05:08.266 INFO:tasks.workunit.client.0.smithi046.stdout:0/136: fdatasync d1/d6/d3/dd/f23 0 2024-09-10T04:05:08.266 INFO:tasks.workunit.client.0.smithi046.stdout:7/141: link d5/f35 d5/d6/db/f38 0 2024-09-10T04:05:08.267 INFO:tasks.workunit.client.0.smithi046.stdout:0/137: fdatasync d1/d6/f11 0 2024-09-10T04:05:08.267 INFO:tasks.workunit.client.0.smithi046.stdout:7/142: fsync d5/d6/f1c 0 2024-09-10T04:05:08.268 INFO:tasks.workunit.client.0.smithi046.stdout:0/138: stat d1/d6/d3/dd/l20 0 2024-09-10T04:05:08.269 INFO:tasks.workunit.client.1.smithi070.stdout:2/93: sync 2024-09-10T04:05:08.270 INFO:tasks.workunit.client.1.smithi070.stdout:2/94: readlink l5 0 2024-09-10T04:05:08.270 INFO:tasks.workunit.client.1.smithi070.stdout:8/179: dread d4/d16/f22 [0,4194304] 0 2024-09-10T04:05:08.270 INFO:tasks.workunit.client.1.smithi070.stdout:8/180: readlink l2 0 2024-09-10T04:05:08.271 INFO:tasks.workunit.client.0.smithi046.stdout:7/143: mkdir d5/d6/d39 0 2024-09-10T04:05:08.274 INFO:tasks.workunit.client.0.smithi046.stdout:0/139: rename d1/d6/d3/dd/l2b to d1/dc/l2e 0 2024-09-10T04:05:08.275 INFO:tasks.workunit.client.0.smithi046.stdout:3/114: sync 2024-09-10T04:05:08.275 INFO:tasks.workunit.client.0.smithi046.stdout:8/176: sync 2024-09-10T04:05:08.275 INFO:tasks.workunit.client.0.smithi046.stdout:4/124: sync 2024-09-10T04:05:08.276 INFO:tasks.workunit.client.1.smithi070.stdout:1/91: creat d3/d8/d12/f24 x:0 0 0 2024-09-10T04:05:08.280 INFO:tasks.workunit.client.0.smithi046.stdout:7/144: rename d5/d9/l16 to d5/d6/l3a 0 2024-09-10T04:05:08.280 INFO:tasks.workunit.client.0.smithi046.stdout:0/140: dread d1/d6/f11 [4194304,4194304] 0 2024-09-10T04:05:08.283 INFO:tasks.workunit.client.0.smithi046.stdout:4/125: chown c2 57121 1 2024-09-10T04:05:08.283 INFO:tasks.workunit.client.0.smithi046.stdout:0/141: chown d1/d6/d3/d29/d2d 2 1 2024-09-10T04:05:08.285 INFO:tasks.workunit.client.0.smithi046.stdout:3/115: symlink d12/d13/l1f 0 2024-09-10T04:05:08.285 INFO:tasks.workunit.client.0.smithi046.stdout:8/177: dwrite d1/d29/d17/d1f/d2a/f33 [0,4194304] 0 2024-09-10T04:05:08.286 INFO:tasks.workunit.client.1.smithi070.stdout:3/157: mkdir db/d16/d2e 0 2024-09-10T04:05:08.289 INFO:tasks.workunit.client.1.smithi070.stdout:9/127: readlink d9/l21 0 2024-09-10T04:05:08.290 INFO:tasks.workunit.client.0.smithi046.stdout:4/126: read d3/f9 [736317,34046] 0 2024-09-10T04:05:08.292 INFO:tasks.workunit.client.1.smithi070.stdout:1/92: dread d3/f5 [0,4194304] 0 2024-09-10T04:05:08.292 INFO:tasks.workunit.client.0.smithi046.stdout:8/178: dread d1/d29/d17/d1f/f35 [0,4194304] 0 2024-09-10T04:05:08.293 INFO:tasks.workunit.client.0.smithi046.stdout:7/145: creat d5/d6/db/d11/d1f/f3b x:0 0 0 2024-09-10T04:05:08.294 INFO:tasks.workunit.client.1.smithi070.stdout:9/128: dread d9/dd/d10/f1f [0,4194304] 0 2024-09-10T04:05:08.294 INFO:tasks.workunit.client.1.smithi070.stdout:9/129: stat d9 0 2024-09-10T04:05:08.295 INFO:tasks.workunit.client.1.smithi070.stdout:0/140: unlink l7 0 2024-09-10T04:05:08.296 INFO:tasks.workunit.client.0.smithi046.stdout:8/179: write d1/d29/d17/f23 [563122,114670] 0 2024-09-10T04:05:08.297 INFO:tasks.workunit.client.1.smithi070.stdout:0/141: write d8/f10 [3658424,81523] 0 2024-09-10T04:05:08.298 INFO:tasks.workunit.client.1.smithi070.stdout:1/93: dread d3/f5 [0,4194304] 0 2024-09-10T04:05:08.298 INFO:tasks.workunit.client.0.smithi046.stdout:3/116: fsync f4 0 2024-09-10T04:05:08.299 INFO:tasks.workunit.client.1.smithi070.stdout:0/142: truncate d8/d13/f20 965678 0 2024-09-10T04:05:08.299 INFO:tasks.workunit.client.1.smithi070.stdout:5/134: mkdir d17/d24/d2b 0 2024-09-10T04:05:08.299 INFO:tasks.workunit.client.0.smithi046.stdout:4/127: dwrite d3/d12/f19 [4194304,4194304] 0 2024-09-10T04:05:08.300 INFO:tasks.workunit.client.0.smithi046.stdout:3/117: stat d12/d13/f18 0 2024-09-10T04:05:08.301 INFO:tasks.workunit.client.0.smithi046.stdout:0/142: rename d1/dc/d25/c2c to d1/dc/d25/c2f 0 2024-09-10T04:05:08.304 INFO:tasks.workunit.client.1.smithi070.stdout:2/95: mkdir d15/d18 0 2024-09-10T04:05:08.308 INFO:tasks.workunit.client.0.smithi046.stdout:4/128: rmdir d3 39 2024-09-10T04:05:08.310 INFO:tasks.workunit.client.0.smithi046.stdout:3/118: rename d12/d13/l1f to d12/d13/l20 0 2024-09-10T04:05:08.311 INFO:tasks.workunit.client.1.smithi070.stdout:4/126: symlink de/d13/d18/d23/l27 0 2024-09-10T04:05:08.311 INFO:tasks.workunit.client.0.smithi046.stdout:0/143: rmdir d1/d13 39 2024-09-10T04:05:08.313 INFO:tasks.workunit.client.1.smithi070.stdout:9/130: mknod d9/dd/c26 0 2024-09-10T04:05:08.313 INFO:tasks.workunit.client.0.smithi046.stdout:7/146: getdents d5/d6/d32 0 2024-09-10T04:05:08.313 INFO:tasks.workunit.client.1.smithi070.stdout:4/127: dread f6 [0,4194304] 0 2024-09-10T04:05:08.314 INFO:tasks.workunit.client.0.smithi046.stdout:8/180: getdents d1/d29/d12 0 2024-09-10T04:05:08.315 INFO:tasks.workunit.client.0.smithi046.stdout:2/193: sync 2024-09-10T04:05:08.316 INFO:tasks.workunit.client.0.smithi046.stdout:1/100: sync 2024-09-10T04:05:08.316 INFO:tasks.workunit.client.0.smithi046.stdout:5/158: sync 2024-09-10T04:05:08.316 INFO:tasks.workunit.client.0.smithi046.stdout:9/150: sync 2024-09-10T04:05:08.316 INFO:tasks.workunit.client.0.smithi046.stdout:6/101: sync 2024-09-10T04:05:08.317 INFO:tasks.workunit.client.1.smithi070.stdout:4/128: dread fb [0,4194304] 0 2024-09-10T04:05:08.317 INFO:tasks.workunit.client.0.smithi046.stdout:6/102: read - da/f1f zero size 2024-09-10T04:05:08.318 INFO:tasks.workunit.client.1.smithi070.stdout:7/127: link d2/d7/la d2/d7/db/l2c 0 2024-09-10T04:05:08.318 INFO:tasks.workunit.client.1.smithi070.stdout:6/215: dwrite d0/f8 [0,4194304] 0 2024-09-10T04:05:08.320 INFO:tasks.workunit.client.0.smithi046.stdout:3/119: write f4 [3937482,3633] 0 2024-09-10T04:05:08.320 INFO:tasks.workunit.client.0.smithi046.stdout:8/181: dread d1/fa [0,4194304] 0 2024-09-10T04:05:08.321 INFO:tasks.workunit.client.1.smithi070.stdout:5/135: fsync f13 0 2024-09-10T04:05:08.323 INFO:tasks.workunit.client.1.smithi070.stdout:0/143: creat d8/d13/d1a/f32 x:0 0 0 2024-09-10T04:05:08.324 INFO:tasks.workunit.client.0.smithi046.stdout:7/147: truncate f4 3097042 0 2024-09-10T04:05:08.324 INFO:tasks.workunit.client.1.smithi070.stdout:0/144: write d8/f2b [961948,92079] 0 2024-09-10T04:05:08.326 INFO:tasks.workunit.client.1.smithi070.stdout:1/94: creat d3/d8/d13/f25 x:0 0 0 2024-09-10T04:05:08.328 INFO:tasks.workunit.client.1.smithi070.stdout:1/95: write d3/d8/d13/f18 [115398,111103] 0 2024-09-10T04:05:08.329 INFO:tasks.workunit.client.1.smithi070.stdout:8/181: dwrite d4/fa [4194304,4194304] 0 2024-09-10T04:05:08.330 INFO:tasks.workunit.client.0.smithi046.stdout:1/101: mkdir d3/dc/d21 0 2024-09-10T04:05:08.331 INFO:tasks.workunit.client.1.smithi070.stdout:8/182: read - d4/d16/f34 zero size 2024-09-10T04:05:08.331 INFO:tasks.workunit.client.0.smithi046.stdout:1/102: write d3/dc/f10 [2349946,9003] 0 2024-09-10T04:05:08.334 INFO:tasks.workunit.client.0.smithi046.stdout:4/129: mkdir d3/d12/d17/d29 0 2024-09-10T04:05:08.335 INFO:tasks.workunit.client.0.smithi046.stdout:4/130: stat d3/d12/d17/d22/f27 0 2024-09-10T04:05:08.336 INFO:tasks.workunit.client.0.smithi046.stdout:6/103: fdatasync da/fe 0 2024-09-10T04:05:08.337 INFO:tasks.workunit.client.1.smithi070.stdout:3/158: truncate f3 2179426 0 2024-09-10T04:05:08.339 INFO:tasks.workunit.client.0.smithi046.stdout:3/120: dread f9 [0,4194304] 0 2024-09-10T04:05:08.340 INFO:tasks.workunit.client.0.smithi046.stdout:0/144: symlink d1/d6/d3/l30 0 2024-09-10T04:05:08.341 INFO:tasks.workunit.client.0.smithi046.stdout:0/145: fsync d1/d6/d3/dd/f23 0 2024-09-10T04:05:08.344 INFO:tasks.workunit.client.0.smithi046.stdout:5/159: rename d7/d11/d14/d1c to d7/d11/d14/d2f 0 2024-09-10T04:05:08.346 INFO:tasks.workunit.client.1.smithi070.stdout:4/129: creat de/f28 x:0 0 0 2024-09-10T04:05:08.346 INFO:tasks.workunit.client.0.smithi046.stdout:9/151: rmdir d4/d18 39 2024-09-10T04:05:08.348 INFO:tasks.workunit.client.1.smithi070.stdout:7/128: fsync d2/d7/fe 0 2024-09-10T04:05:08.348 INFO:tasks.workunit.client.0.smithi046.stdout:0/146: dwrite d1/d6/f11 [8388608,4194304] 0 2024-09-10T04:05:08.352 INFO:tasks.workunit.client.1.smithi070.stdout:9/131: write d9/dd/f14 [5901037,10734] 0 2024-09-10T04:05:08.353 INFO:tasks.workunit.client.0.smithi046.stdout:1/103: creat d3/dc/f22 x:0 0 0 2024-09-10T04:05:08.353 INFO:tasks.workunit.client.0.smithi046.stdout:1/104: stat f1 0 2024-09-10T04:05:08.355 INFO:tasks.workunit.client.1.smithi070.stdout:5/136: creat d17/d24/f2c x:0 0 0 2024-09-10T04:05:08.356 INFO:tasks.workunit.client.0.smithi046.stdout:4/131: readlink d3/d12/d14/l28 0 2024-09-10T04:05:08.357 INFO:tasks.workunit.client.0.smithi046.stdout:8/182: dwrite d1/d29/d19/d36/f3f [0,4194304] 0 2024-09-10T04:05:08.357 INFO:tasks.workunit.client.0.smithi046.stdout:4/132: readlink d3/d12/l1e 0 2024-09-10T04:05:08.359 INFO:tasks.workunit.client.1.smithi070.stdout:9/132: dwrite d9/f25 [0,4194304] 0 2024-09-10T04:05:08.362 INFO:tasks.workunit.client.1.smithi070.stdout:1/96: rmdir d3/d8/d13 39 2024-09-10T04:05:08.362 INFO:tasks.workunit.client.1.smithi070.stdout:2/96: creat d15/d18/f19 x:0 0 0 2024-09-10T04:05:08.366 INFO:tasks.workunit.client.0.smithi046.stdout:8/183: dwrite d1/d29/f27 [0,4194304] 0 2024-09-10T04:05:08.367 INFO:tasks.workunit.client.1.smithi070.stdout:3/159: symlink db/d1c/d25/l2f 0 2024-09-10T04:05:08.373 INFO:tasks.workunit.client.1.smithi070.stdout:4/130: mknod de/d13/c29 0 2024-09-10T04:05:08.375 INFO:tasks.workunit.client.0.smithi046.stdout:8/184: dwrite d1/d29/d17/f23 [0,4194304] 0 2024-09-10T04:05:08.376 INFO:tasks.workunit.client.1.smithi070.stdout:3/160: dwrite db/d1c/f1e [0,4194304] 0 2024-09-10T04:05:08.377 INFO:tasks.workunit.client.0.smithi046.stdout:7/148: truncate d5/d9/fe 3410968 0 2024-09-10T04:05:08.378 INFO:tasks.workunit.client.1.smithi070.stdout:7/129: mkdir d2/d3/dc/d1b/d2d 0 2024-09-10T04:05:08.378 INFO:tasks.workunit.client.0.smithi046.stdout:1/105: creat d3/dc/f23 x:0 0 0 2024-09-10T04:05:08.382 INFO:tasks.workunit.client.1.smithi070.stdout:6/216: link d0/d3/d18/d1b/d33/f39 d0/d3/d18/f45 0 2024-09-10T04:05:08.386 INFO:tasks.workunit.client.0.smithi046.stdout:7/149: dwrite d5/d6/db/dd/f2f [0,4194304] 0 2024-09-10T04:05:08.387 INFO:tasks.workunit.client.1.smithi070.stdout:7/130: dwrite d2/d3/dc/d13/f1a [0,4194304] 0 2024-09-10T04:05:08.388 INFO:tasks.workunit.client.0.smithi046.stdout:3/121: unlink d12/d13/l20 0 2024-09-10T04:05:08.388 INFO:tasks.workunit.client.0.smithi046.stdout:7/150: stat d5/d6/db/d11/d1e/d20/c2d 0 2024-09-10T04:05:08.388 INFO:tasks.workunit.client.0.smithi046.stdout:3/122: chown d12/d13/d14/d1e 1 1 2024-09-10T04:05:08.389 INFO:tasks.workunit.client.1.smithi070.stdout:5/137: creat d17/d24/f2d x:0 0 0 2024-09-10T04:05:08.390 INFO:tasks.workunit.client.0.smithi046.stdout:5/160: mknod d7/c30 0 2024-09-10T04:05:08.390 INFO:tasks.workunit.client.0.smithi046.stdout:5/161: dread - d7/d11/f28 zero size 2024-09-10T04:05:08.393 INFO:tasks.workunit.client.1.smithi070.stdout:0/145: mkdir d8/d13/d33 0 2024-09-10T04:05:08.393 INFO:tasks.workunit.client.1.smithi070.stdout:0/146: chown c6 90396944 1 2024-09-10T04:05:08.393 INFO:tasks.workunit.client.1.smithi070.stdout:9/133: rmdir d9/dd/d10 39 2024-09-10T04:05:08.396 INFO:tasks.workunit.client.1.smithi070.stdout:1/97: chown d3/l4 1875812231 1 2024-09-10T04:05:08.398 INFO:tasks.workunit.client.0.smithi046.stdout:0/147: creat d1/d6/d1a/f31 x:0 0 0 2024-09-10T04:05:08.403 INFO:tasks.workunit.client.0.smithi046.stdout:2/194: rename d2/d6/d15/d16 to d2/d49 0 2024-09-10T04:05:08.403 INFO:tasks.workunit.client.0.smithi046.stdout:5/162: dwrite d7/d11/d14/d2f/f22 [0,4194304] 0 2024-09-10T04:05:08.405 INFO:tasks.workunit.client.1.smithi070.stdout:2/97: creat d15/d18/f1a x:0 0 0 2024-09-10T04:05:08.406 INFO:tasks.workunit.client.1.smithi070.stdout:2/98: write f10 [1107413,4121] 0 2024-09-10T04:05:08.406 INFO:tasks.workunit.client.0.smithi046.stdout:2/195: write d2/d6/d23/d42/f45 [549109,124450] 0 2024-09-10T04:05:08.410 INFO:tasks.workunit.client.0.smithi046.stdout:1/106: mknod d3/d13/c24 0 2024-09-10T04:05:08.413 INFO:tasks.workunit.client.1.smithi070.stdout:2/99: dwrite f3 [0,4194304] 0 2024-09-10T04:05:08.413 INFO:tasks.workunit.client.1.smithi070.stdout:2/100: stat d15 0 2024-09-10T04:05:08.415 INFO:tasks.workunit.client.0.smithi046.stdout:3/123: fdatasync d12/d13/f17 0 2024-09-10T04:05:08.419 INFO:tasks.workunit.client.0.smithi046.stdout:9/152: write d4/d18/f27 [456593,125074] 0 2024-09-10T04:05:08.427 INFO:tasks.workunit.client.0.smithi046.stdout:6/104: rename da/db/f1d to da/db/d11/d13/d1e/f26 0 2024-09-10T04:05:08.427 INFO:tasks.workunit.client.0.smithi046.stdout:4/133: rename d3 to d3/d12/d17/d2a 22 2024-09-10T04:05:08.428 INFO:tasks.workunit.client.0.smithi046.stdout:4/134: chown d3/l7 115115556 1 2024-09-10T04:05:08.429 INFO:tasks.workunit.client.0.smithi046.stdout:2/196: rmdir d2/d8 39 2024-09-10T04:05:08.431 INFO:tasks.workunit.client.0.smithi046.stdout:1/107: mkdir d3/dc/d25 0 2024-09-10T04:05:08.434 INFO:tasks.workunit.client.0.smithi046.stdout:3/124: unlink c11 0 2024-09-10T04:05:08.435 INFO:tasks.workunit.client.0.smithi046.stdout:9/153: chown d4/d7/ff 59583002 1 2024-09-10T04:05:08.439 INFO:tasks.workunit.client.0.smithi046.stdout:6/105: mkdir da/db/d10/d27 0 2024-09-10T04:05:08.440 INFO:tasks.workunit.client.0.smithi046.stdout:5/163: rename d7/d11/d14/c21 to d7/d11/d14/d2f/d2b/c31 0 2024-09-10T04:05:08.441 INFO:tasks.workunit.client.0.smithi046.stdout:8/185: creat d1/f40 x:0 0 0 2024-09-10T04:05:08.443 INFO:tasks.workunit.client.0.smithi046.stdout:4/135: creat d3/d12/d14/f2b x:0 0 0 2024-09-10T04:05:08.443 INFO:tasks.workunit.client.1.smithi070.stdout:5/138: creat d17/d19/f2e x:0 0 0 2024-09-10T04:05:08.443 INFO:tasks.workunit.client.1.smithi070.stdout:6/217: chown d0/d3/d18/c36 35027 1 2024-09-10T04:05:08.444 INFO:tasks.workunit.client.0.smithi046.stdout:2/197: mknod d2/d6/d23/d42/c4a 0 2024-09-10T04:05:08.444 INFO:tasks.workunit.client.1.smithi070.stdout:8/183: link d4/c9 d4/d16/d17/d1a/c42 0 2024-09-10T04:05:08.444 INFO:tasks.workunit.client.1.smithi070.stdout:4/131: write fb [1006076,28830] 0 2024-09-10T04:05:08.445 INFO:tasks.workunit.client.1.smithi070.stdout:5/139: dread - d17/d24/f27 zero size 2024-09-10T04:05:08.445 INFO:tasks.workunit.client.0.smithi046.stdout:0/148: truncate d1/d6/d3/f14 1841253 0 2024-09-10T04:05:08.445 INFO:tasks.workunit.client.1.smithi070.stdout:3/161: fsync f3 0 2024-09-10T04:05:08.445 INFO:tasks.workunit.client.1.smithi070.stdout:2/101: mknod d15/d18/c1b 0 2024-09-10T04:05:08.446 INFO:tasks.workunit.client.1.smithi070.stdout:1/98: read - d3/d8/d13/f25 zero size 2024-09-10T04:05:08.446 INFO:tasks.workunit.client.0.smithi046.stdout:6/106: dwrite da/f12 [8388608,4194304] 0 2024-09-10T04:05:08.450 INFO:tasks.workunit.client.0.smithi046.stdout:3/125: creat d12/d1b/f21 x:0 0 0 2024-09-10T04:05:08.450 INFO:tasks.workunit.client.0.smithi046.stdout:3/126: stat f7 0 2024-09-10T04:05:08.454 INFO:tasks.workunit.client.0.smithi046.stdout:0/149: dwrite d1/d6/f1b [0,4194304] 0 2024-09-10T04:05:08.455 INFO:tasks.workunit.client.0.smithi046.stdout:0/150: stat d1/d6/d3/l30 0 2024-09-10T04:05:08.456 INFO:tasks.workunit.client.0.smithi046.stdout:0/151: truncate d1/dc/f27 1580790 0 2024-09-10T04:05:08.458 INFO:tasks.workunit.client.0.smithi046.stdout:5/164: rename d7/d11/f24 to d7/d11/d14/d2f/d2d/f32 0 2024-09-10T04:05:08.459 INFO:tasks.workunit.client.0.smithi046.stdout:5/165: write d7/f9 [996222,12936] 0 2024-09-10T04:05:08.460 INFO:tasks.workunit.client.0.smithi046.stdout:8/186: mknod d1/d29/d17/c41 0 2024-09-10T04:05:08.461 INFO:tasks.workunit.client.0.smithi046.stdout:4/136: mknod d3/d12/c2c 0 2024-09-10T04:05:08.463 INFO:tasks.workunit.client.0.smithi046.stdout:2/198: creat d2/d6/d23/f4b x:0 0 0 2024-09-10T04:05:08.464 INFO:tasks.workunit.client.0.smithi046.stdout:4/137: fdatasync d3/d12/f19 0 2024-09-10T04:05:08.464 INFO:tasks.workunit.client.0.smithi046.stdout:8/187: chown d1/d29/d17/d1f/d2a/l2b 13788349 1 2024-09-10T04:05:08.466 INFO:tasks.workunit.client.0.smithi046.stdout:2/199: fdatasync d2/d6/f43 0 2024-09-10T04:05:08.467 INFO:tasks.workunit.client.0.smithi046.stdout:6/107: unlink da/db/d11/f1b 0 2024-09-10T04:05:08.468 INFO:tasks.workunit.client.0.smithi046.stdout:3/127: symlink d12/d13/d14/l22 0 2024-09-10T04:05:08.469 INFO:tasks.workunit.client.0.smithi046.stdout:0/152: dwrite d1/d6/f11 [0,4194304] 0 2024-09-10T04:05:08.471 INFO:tasks.workunit.client.0.smithi046.stdout:0/153: chown d1/d6/d3/d29 1054 1 2024-09-10T04:05:08.472 INFO:tasks.workunit.client.0.smithi046.stdout:6/108: dread f1 [0,4194304] 0 2024-09-10T04:05:08.473 INFO:tasks.workunit.client.0.smithi046.stdout:6/109: fdatasync da/f1f 0 2024-09-10T04:05:08.475 INFO:tasks.workunit.client.0.smithi046.stdout:6/110: readlink l5 0 2024-09-10T04:05:08.475 INFO:tasks.workunit.client.0.smithi046.stdout:5/166: dwrite d7/d11/d14/d2f/f22 [0,4194304] 0 2024-09-10T04:05:08.477 INFO:tasks.workunit.client.0.smithi046.stdout:7/151: truncate d5/d9/fe 3304265 0 2024-09-10T04:05:08.480 INFO:tasks.workunit.client.0.smithi046.stdout:1/108: creat d3/f26 x:0 0 0 2024-09-10T04:05:08.480 INFO:tasks.workunit.client.0.smithi046.stdout:7/152: dread - d5/d6/db/dd/d24/f31 zero size 2024-09-10T04:05:08.482 INFO:tasks.workunit.client.0.smithi046.stdout:9/154: write d4/d18/f22 [5133,62061] 0 2024-09-10T04:05:08.483 INFO:tasks.workunit.client.0.smithi046.stdout:4/138: symlink d3/l2d 0 2024-09-10T04:05:08.485 INFO:tasks.workunit.client.0.smithi046.stdout:9/155: chown d4/d7/l12 7800 1 2024-09-10T04:05:08.485 INFO:tasks.workunit.client.0.smithi046.stdout:2/200: rmdir d2/d6 39 2024-09-10T04:05:08.486 INFO:tasks.workunit.client.0.smithi046.stdout:6/111: dwrite da/db/d11/d13/d1e/f21 [0,4194304] 0 2024-09-10T04:05:08.487 INFO:tasks.workunit.client.1.smithi070.stdout:6/218: chown d0/d3/d18/d2a/c30 7 1 2024-09-10T04:05:08.487 INFO:tasks.workunit.client.1.smithi070.stdout:9/134: truncate d9/dd/f18 429908 0 2024-09-10T04:05:08.488 INFO:tasks.workunit.client.1.smithi070.stdout:2/102: write f11 [497571,33479] 0 2024-09-10T04:05:08.488 INFO:tasks.workunit.client.1.smithi070.stdout:1/99: dread - d3/d8/d12/f24 zero size 2024-09-10T04:05:08.488 INFO:tasks.workunit.client.0.smithi046.stdout:9/156: chown d4/d7/d1f/f26 1 1 2024-09-10T04:05:08.489 INFO:tasks.workunit.client.1.smithi070.stdout:1/100: readlink d3/l11 0 2024-09-10T04:05:08.491 INFO:tasks.workunit.client.1.smithi070.stdout:1/101: write d3/f5 [4475929,33366] 0 2024-09-10T04:05:08.491 INFO:tasks.workunit.client.1.smithi070.stdout:8/184: mkdir d4/d16/d17/d19/d28/d43 0 2024-09-10T04:05:08.493 INFO:tasks.workunit.client.1.smithi070.stdout:8/185: chown d4/d16/d17 236356 1 2024-09-10T04:05:08.493 INFO:tasks.workunit.client.0.smithi046.stdout:5/167: mknod d7/d11/d14/d2f/c33 0 2024-09-10T04:05:08.494 INFO:tasks.workunit.client.0.smithi046.stdout:5/168: truncate d7/f1f 217247 0 2024-09-10T04:05:08.496 INFO:tasks.workunit.client.0.smithi046.stdout:5/169: dread d7/d11/f23 [0,4194304] 0 2024-09-10T04:05:08.496 INFO:tasks.workunit.client.1.smithi070.stdout:6/219: stat d0/f15 0 2024-09-10T04:05:08.497 INFO:tasks.workunit.client.0.smithi046.stdout:7/153: mkdir d5/d6/db/dd/d3c 0 2024-09-10T04:05:08.497 INFO:tasks.workunit.client.1.smithi070.stdout:5/140: dwrite fd [0,4194304] 0 2024-09-10T04:05:08.498 INFO:tasks.workunit.client.0.smithi046.stdout:2/201: mknod d2/d49/c4c 0 2024-09-10T04:05:08.499 INFO:tasks.workunit.client.0.smithi046.stdout:6/112: creat da/dc/f28 x:0 0 0 2024-09-10T04:05:08.500 INFO:tasks.workunit.client.1.smithi070.stdout:9/135: dread - d9/dd/d10/f24 zero size 2024-09-10T04:05:08.501 INFO:tasks.workunit.client.1.smithi070.stdout:9/136: dread - d9/dd/d10/f24 zero size 2024-09-10T04:05:08.501 INFO:tasks.workunit.client.1.smithi070.stdout:2/103: creat d15/d18/f1c x:0 0 0 2024-09-10T04:05:08.502 INFO:tasks.workunit.client.1.smithi070.stdout:9/137: stat d9/dd/d10/d1b/l1e 0 2024-09-10T04:05:08.502 INFO:tasks.workunit.client.0.smithi046.stdout:9/157: chown l3 26659950 1 2024-09-10T04:05:08.503 INFO:tasks.workunit.client.0.smithi046.stdout:6/113: dread da/db/f1c [0,4194304] 0 2024-09-10T04:05:08.503 INFO:tasks.workunit.client.1.smithi070.stdout:7/131: getdents d2/d7 0 2024-09-10T04:05:08.503 INFO:tasks.workunit.client.0.smithi046.stdout:3/128: symlink d12/d13/d14/d1e/l23 0 2024-09-10T04:05:08.505 INFO:tasks.workunit.client.0.smithi046.stdout:3/129: chown f0 4 1 2024-09-10T04:05:08.505 INFO:tasks.workunit.client.0.smithi046.stdout:6/114: chown da/dc/f28 1 1 2024-09-10T04:05:08.505 INFO:tasks.workunit.client.0.smithi046.stdout:3/130: readlink lc 0 2024-09-10T04:05:08.505 INFO:tasks.workunit.client.0.smithi046.stdout:0/154: creat d1/d6/d3/d29/d2d/f32 x:0 0 0 2024-09-10T04:05:08.506 INFO:tasks.workunit.client.0.smithi046.stdout:0/155: dread - d1/d6/d1a/f31 zero size 2024-09-10T04:05:08.507 INFO:tasks.workunit.client.1.smithi070.stdout:4/132: rename de/d13/f16 to de/d13/d18/d26/f2a 0 2024-09-10T04:05:08.507 INFO:tasks.workunit.client.0.smithi046.stdout:0/156: chown d1/dc/lf 0 1 2024-09-10T04:05:08.509 INFO:tasks.workunit.client.1.smithi070.stdout:8/186: mkdir d4/d16/d17/d1a/d23/d44 0 2024-09-10T04:05:08.509 INFO:tasks.workunit.client.1.smithi070.stdout:7/132: dwrite d2/d3/dc/f1e [0,4194304] 0 2024-09-10T04:05:08.510 INFO:tasks.workunit.client.1.smithi070.stdout:8/187: chown d4/d16/d17/d1a 580 1 2024-09-10T04:05:08.511 INFO:tasks.workunit.client.1.smithi070.stdout:6/220: rmdir d0/d3/d1f 39 2024-09-10T04:05:08.511 INFO:tasks.workunit.client.1.smithi070.stdout:8/188: read - d4/d16/d17/f40 zero size 2024-09-10T04:05:08.512 INFO:tasks.workunit.client.1.smithi070.stdout:5/141: creat d17/d24/f2f x:0 0 0 2024-09-10T04:05:08.513 INFO:tasks.workunit.client.1.smithi070.stdout:2/104: mknod d15/d18/c1d 0 2024-09-10T04:05:08.514 INFO:tasks.workunit.client.0.smithi046.stdout:0/157: dwrite d1/dc/f21 [0,4194304] 0 2024-09-10T04:05:08.515 INFO:tasks.workunit.client.1.smithi070.stdout:2/105: fsync d15/d18/f19 0 2024-09-10T04:05:08.515 INFO:tasks.workunit.client.1.smithi070.stdout:9/138: symlink d9/dd/d10/d1b/l27 0 2024-09-10T04:05:08.515 INFO:tasks.workunit.client.1.smithi070.stdout:9/139: chown d9 470727308 1 2024-09-10T04:05:08.515 INFO:tasks.workunit.client.0.smithi046.stdout:0/158: write d1/dc/f16 [1688269,30456] 0 2024-09-10T04:05:08.518 INFO:tasks.workunit.client.1.smithi070.stdout:2/106: truncate f10 1597624 0 2024-09-10T04:05:08.520 INFO:tasks.workunit.client.1.smithi070.stdout:4/133: creat de/d13/d18/d26/f2b x:0 0 0 2024-09-10T04:05:08.520 INFO:tasks.workunit.client.0.smithi046.stdout:8/188: link d1/d29/d17/c20 d1/d29/d17/c42 0 2024-09-10T04:05:08.521 INFO:tasks.workunit.client.1.smithi070.stdout:7/133: symlink d2/d7/l2e 0 2024-09-10T04:05:08.521 INFO:tasks.workunit.client.0.smithi046.stdout:8/189: stat d1/d29/d17/c25 0 2024-09-10T04:05:08.522 INFO:tasks.workunit.client.1.smithi070.stdout:4/134: read - de/f11 zero size 2024-09-10T04:05:08.523 INFO:tasks.workunit.client.0.smithi046.stdout:2/202: dwrite d2/d6/f43 [0,4194304] 0 2024-09-10T04:05:08.524 INFO:tasks.workunit.client.0.smithi046.stdout:3/131: rename d12/d13/d14/f1c to d12/d13/d14/f24 0 2024-09-10T04:05:08.525 INFO:tasks.workunit.client.1.smithi070.stdout:9/140: dwrite d9/dd/d10/f24 [0,4194304] 0 2024-09-10T04:05:08.525 INFO:tasks.workunit.client.1.smithi070.stdout:9/141: chown d9/la 21594651 1 2024-09-10T04:05:08.526 INFO:tasks.workunit.client.1.smithi070.stdout:5/142: mkdir d17/d30 0 2024-09-10T04:05:08.526 INFO:tasks.workunit.client.1.smithi070.stdout:9/142: chown d9/f22 446868 1 2024-09-10T04:05:08.526 INFO:tasks.workunit.client.1.smithi070.stdout:9/143: chown d9/dd/f1a 4068 1 2024-09-10T04:05:08.527 INFO:tasks.workunit.client.1.smithi070.stdout:9/144: chown d9/dd/d10/d1b 1986 1 2024-09-10T04:05:08.529 INFO:tasks.workunit.client.0.smithi046.stdout:1/109: rename d3/d13 to d3/d27 0 2024-09-10T04:05:08.533 INFO:tasks.workunit.client.1.smithi070.stdout:5/143: dwrite d17/d19/f1e [0,4194304] 0 2024-09-10T04:05:08.534 INFO:tasks.workunit.client.0.smithi046.stdout:5/170: mknod d7/d11/c34 0 2024-09-10T04:05:08.534 INFO:tasks.workunit.client.1.smithi070.stdout:5/144: chown fc 1964989 1 2024-09-10T04:05:08.535 INFO:tasks.workunit.client.1.smithi070.stdout:5/145: dread d17/f1b [0,4194304] 0 2024-09-10T04:05:08.538 INFO:tasks.workunit.client.1.smithi070.stdout:3/162: getdents db/d16 0 2024-09-10T04:05:08.539 INFO:tasks.workunit.client.1.smithi070.stdout:6/221: creat d0/d3/d1f/f46 x:0 0 0 2024-09-10T04:05:08.541 INFO:tasks.workunit.client.0.smithi046.stdout:0/159: chown d1/d13/f1d 1549116 1 2024-09-10T04:05:08.541 INFO:tasks.workunit.client.1.smithi070.stdout:7/134: symlink d2/d3/dc/d13/l2f 0 2024-09-10T04:05:08.545 INFO:tasks.workunit.client.0.smithi046.stdout:4/139: truncate d3/d12/d14/f16 1179490 0 2024-09-10T04:05:08.546 INFO:tasks.workunit.client.1.smithi070.stdout:9/145: mknod d9/dd/d10/d1b/c28 0 2024-09-10T04:05:08.546 INFO:tasks.workunit.client.0.smithi046.stdout:7/154: dwrite d5/f35 [0,4194304] 0 2024-09-10T04:05:08.547 INFO:tasks.workunit.client.0.smithi046.stdout:9/158: mknod d4/d18/c2d 0 2024-09-10T04:05:08.550 INFO:tasks.workunit.client.0.smithi046.stdout:6/115: creat da/db/d10/d27/f29 x:0 0 0 2024-09-10T04:05:08.551 INFO:tasks.workunit.client.1.smithi070.stdout:1/102: getdents d3/d8/d12 0 2024-09-10T04:05:08.555 INFO:tasks.workunit.client.0.smithi046.stdout:7/155: dwrite d5/d6/db/dd/d24/f31 [0,4194304] 0 2024-09-10T04:05:08.556 INFO:tasks.workunit.client.1.smithi070.stdout:2/107: unlink c14 0 2024-09-10T04:05:08.561 INFO:tasks.workunit.client.0.smithi046.stdout:8/190: symlink d1/d29/d17/d1f/d2a/l43 0 2024-09-10T04:05:08.563 INFO:tasks.workunit.client.1.smithi070.stdout:5/146: write f14 [1064302,78404] 0 2024-09-10T04:05:08.565 INFO:tasks.workunit.client.0.smithi046.stdout:3/132: rename d12/d13/f19 to d12/d13/d14/f25 0 2024-09-10T04:05:08.566 INFO:tasks.workunit.client.1.smithi070.stdout:3/163: truncate db/fd 260818 0 2024-09-10T04:05:08.566 INFO:tasks.workunit.client.1.smithi070.stdout:3/164: readlink db/l20 0 2024-09-10T04:05:08.569 INFO:tasks.workunit.client.1.smithi070.stdout:6/222: creat d0/d3/d18/d1b/d17/f47 x:0 0 0 2024-09-10T04:05:08.572 INFO:tasks.workunit.client.0.smithi046.stdout:9/159: symlink d4/d7/l2e 0 2024-09-10T04:05:08.573 INFO:tasks.workunit.client.0.smithi046.stdout:9/160: stat d4/d18/c2d 0 2024-09-10T04:05:08.574 INFO:tasks.workunit.client.0.smithi046.stdout:6/116: rmdir da 39 2024-09-10T04:05:08.574 INFO:tasks.workunit.client.1.smithi070.stdout:6/223: dwrite d0/d3/d18/f27 [0,4194304] 0 2024-09-10T04:05:08.575 INFO:tasks.workunit.client.1.smithi070.stdout:6/224: readlink d0/d3/d18/d1b/d17/l24 0 2024-09-10T04:05:08.580 INFO:tasks.workunit.client.1.smithi070.stdout:1/103: rename d3/d8/c21 to d3/d8/d12/c26 0 2024-09-10T04:05:08.580 INFO:tasks.workunit.client.1.smithi070.stdout:1/104: fsync d3/f5 0 2024-09-10T04:05:08.581 INFO:tasks.workunit.client.0.smithi046.stdout:2/203: mknod d2/c4d 0 2024-09-10T04:05:08.582 INFO:tasks.workunit.client.1.smithi070.stdout:2/108: symlink d15/d18/l1e 0 2024-09-10T04:05:08.583 INFO:tasks.workunit.client.0.smithi046.stdout:8/191: creat d1/d29/d17/d1f/f44 x:0 0 0 2024-09-10T04:05:08.585 INFO:tasks.workunit.client.1.smithi070.stdout:4/135: truncate f8 1295786 0 2024-09-10T04:05:08.587 INFO:tasks.workunit.client.0.smithi046.stdout:3/133: fsync f3 0 2024-09-10T04:05:08.590 INFO:tasks.workunit.client.0.smithi046.stdout:1/110: rename d3/dc/d25 to d3/dc/d16/d28 0 2024-09-10T04:05:08.590 INFO:tasks.workunit.client.0.smithi046.stdout:5/171: rename d7/d11 to d7/d11/d14/d2f/d2b/d35 22 2024-09-10T04:05:08.591 INFO:tasks.workunit.client.0.smithi046.stdout:5/172: dread - d7/d11/f28 zero size 2024-09-10T04:05:08.592 INFO:tasks.workunit.client.1.smithi070.stdout:3/165: readlink db/d1c/l26 0 2024-09-10T04:05:08.592 INFO:tasks.workunit.client.1.smithi070.stdout:3/166: chown db/d16/f2d 64 1 2024-09-10T04:05:08.592 INFO:tasks.workunit.client.0.smithi046.stdout:5/173: truncate d7/fd 5216665 0 2024-09-10T04:05:08.594 INFO:tasks.workunit.client.0.smithi046.stdout:9/161: rmdir d4/d7/d1f 39 2024-09-10T04:05:08.597 INFO:tasks.workunit.client.0.smithi046.stdout:5/174: dread d7/f19 [0,4194304] 0 2024-09-10T04:05:08.598 INFO:tasks.workunit.client.0.smithi046.stdout:5/175: chown d7/d11/d14/f25 953 1 2024-09-10T04:05:08.601 INFO:tasks.workunit.client.0.smithi046.stdout:7/156: creat d5/d6/d39/f3d x:0 0 0 2024-09-10T04:05:08.601 INFO:tasks.workunit.client.1.smithi070.stdout:9/146: truncate d9/dd/d10/f1f 1688422 0 2024-09-10T04:05:08.604 INFO:tasks.workunit.client.0.smithi046.stdout:6/117: dwrite da/db/f14 [0,4194304] 0 2024-09-10T04:05:08.604 INFO:tasks.workunit.client.0.smithi046.stdout:2/204: write d2/d8/d37/f41 [2026382,94982] 0 2024-09-10T04:05:08.605 INFO:tasks.workunit.client.0.smithi046.stdout:6/118: fsync da/dc/f28 0 2024-09-10T04:05:08.606 INFO:tasks.workunit.client.0.smithi046.stdout:8/192: unlink d1/d29/d17/d1f/d2a/f3e 0 2024-09-10T04:05:08.607 INFO:tasks.workunit.client.1.smithi070.stdout:9/147: dwrite d9/f25 [0,4194304] 0 2024-09-10T04:05:08.608 INFO:tasks.workunit.client.0.smithi046.stdout:8/193: chown d1/d29/d17/d1f/f44 1 1 2024-09-10T04:05:08.610 INFO:tasks.workunit.client.0.smithi046.stdout:3/134: unlink f9 0 2024-09-10T04:05:08.612 INFO:tasks.workunit.client.0.smithi046.stdout:3/135: dread f0 [0,4194304] 0 2024-09-10T04:05:08.613 INFO:tasks.workunit.client.1.smithi070.stdout:2/109: fdatasync f11 0 2024-09-10T04:05:08.614 INFO:tasks.workunit.client.0.smithi046.stdout:0/160: rename d1/d6 to d1/dc/d25/d33 0 2024-09-10T04:05:08.614 INFO:tasks.workunit.client.0.smithi046.stdout:3/136: rename d12/d13 to d12/d13/d14/d1e/d26 22 2024-09-10T04:05:08.616 INFO:tasks.workunit.client.0.smithi046.stdout:0/161: dread - d1/dc/d25/d33/d1a/f31 zero size 2024-09-10T04:05:08.617 INFO:tasks.workunit.client.0.smithi046.stdout:0/162: stat d1/dc/d25/d33/d3/dd 0 2024-09-10T04:05:08.620 INFO:tasks.workunit.client.0.smithi046.stdout:9/162: stat d4/d18/f25 0 2024-09-10T04:05:08.621 INFO:tasks.workunit.client.0.smithi046.stdout:5/176: creat d7/d11/d14/f36 x:0 0 0 2024-09-10T04:05:08.623 INFO:tasks.workunit.client.0.smithi046.stdout:5/177: chown d7/d11/d14/f1d 1861 1 2024-09-10T04:05:08.624 INFO:tasks.workunit.client.1.smithi070.stdout:7/135: rename d2/d7/l27 to d2/d3/l30 0 2024-09-10T04:05:08.626 INFO:tasks.workunit.client.0.smithi046.stdout:0/163: dwrite d1/dc/d25/d33/d3/f2a [0,4194304] 0 2024-09-10T04:05:08.627 INFO:tasks.workunit.client.0.smithi046.stdout:7/157: creat d5/d6/db/d11/d1e/f3e x:0 0 0 2024-09-10T04:05:08.629 INFO:tasks.workunit.client.0.smithi046.stdout:0/164: write d1/dc/d25/d33/d3/f2a [2876375,57832] 0 2024-09-10T04:05:08.630 INFO:tasks.workunit.client.0.smithi046.stdout:0/165: chown d1/dc/lf 5850 1 2024-09-10T04:05:08.631 INFO:tasks.workunit.client.0.smithi046.stdout:5/178: dwrite d7/f9 [0,4194304] 0 2024-09-10T04:05:08.632 INFO:tasks.workunit.client.1.smithi070.stdout:1/105: mkdir d3/d8/d27 0 2024-09-10T04:05:08.633 INFO:tasks.workunit.client.0.smithi046.stdout:2/205: stat d2/d6/d15/l40 0 2024-09-10T04:05:08.635 INFO:tasks.workunit.client.0.smithi046.stdout:6/119: creat da/f2a x:0 0 0 2024-09-10T04:05:08.637 INFO:tasks.workunit.client.1.smithi070.stdout:2/110: fdatasync f3 0 2024-09-10T04:05:08.638 INFO:tasks.workunit.client.1.smithi070.stdout:5/147: rmdir d17/d30 0 2024-09-10T04:05:08.640 INFO:tasks.workunit.client.0.smithi046.stdout:8/194: creat d1/d29/d12/f45 x:0 0 0 2024-09-10T04:05:08.643 INFO:tasks.workunit.client.0.smithi046.stdout:3/137: mknod d12/d13/c27 0 2024-09-10T04:05:08.643 INFO:tasks.workunit.client.0.smithi046.stdout:3/138: fsync f4 0 2024-09-10T04:05:08.645 INFO:tasks.workunit.client.0.smithi046.stdout:3/139: chown d12/d13/d14/d1e/l23 162000905 1 2024-09-10T04:05:08.645 INFO:tasks.workunit.client.1.smithi070.stdout:6/225: rename d0/f15 to d0/d3/d18/d1b/d17/d3f/d40/f48 0 2024-09-10T04:05:08.647 INFO:tasks.workunit.client.0.smithi046.stdout:9/163: fsync d4/f13 0 2024-09-10T04:05:08.648 INFO:tasks.workunit.client.1.smithi070.stdout:1/106: unlink d3/c9 0 2024-09-10T04:05:08.651 INFO:tasks.workunit.client.0.smithi046.stdout:7/158: stat f2 0 2024-09-10T04:05:08.651 INFO:tasks.workunit.client.0.smithi046.stdout:4/140: dwrite d3/d12/d14/f16 [0,4194304] 0 2024-09-10T04:05:08.652 INFO:tasks.workunit.client.1.smithi070.stdout:2/111: chown d15/d18/c1b 77491910 1 2024-09-10T04:05:08.653 INFO:tasks.workunit.client.0.smithi046.stdout:7/159: truncate d5/d6/db/d11/d1f/f3b 165964 0 2024-09-10T04:05:08.654 INFO:tasks.workunit.client.1.smithi070.stdout:5/148: rmdir d17/d24 39 2024-09-10T04:05:08.655 INFO:tasks.workunit.client.0.smithi046.stdout:7/160: readlink d5/d9/d12/l34 0 2024-09-10T04:05:08.657 INFO:tasks.workunit.client.0.smithi046.stdout:0/166: mknod d1/dc/d25/c34 0 2024-09-10T04:05:08.658 INFO:tasks.workunit.client.0.smithi046.stdout:0/167: chown d1/dc/d25/d33/d3/c26 4446100 1 2024-09-10T04:05:08.660 INFO:tasks.workunit.client.1.smithi070.stdout:3/167: rmdir db/d16/d2e 0 2024-09-10T04:05:08.662 INFO:tasks.workunit.client.0.smithi046.stdout:7/161: dread d5/d6/db/f1d [0,4194304] 0 2024-09-10T04:05:08.664 INFO:tasks.workunit.client.0.smithi046.stdout:2/206: creat d2/d6/d23/d42/f4e x:0 0 0 2024-09-10T04:05:08.666 INFO:tasks.workunit.client.0.smithi046.stdout:2/207: dread d2/f2c [0,4194304] 0 2024-09-10T04:05:08.666 INFO:tasks.workunit.client.1.smithi070.stdout:9/148: dwrite d9/dd/f18 [0,4194304] 0 2024-09-10T04:05:08.667 INFO:tasks.workunit.client.0.smithi046.stdout:6/120: mkdir da/db/d11/d2b 0 2024-09-10T04:05:08.668 INFO:tasks.workunit.client.1.smithi070.stdout:4/136: rename fd to de/d13/d18/d26/f2c 0 2024-09-10T04:05:08.671 INFO:tasks.workunit.client.0.smithi046.stdout:3/140: symlink d12/d1b/l28 0 2024-09-10T04:05:08.671 INFO:tasks.workunit.client.0.smithi046.stdout:9/164: stat d4/d7/d1f/ld 0 2024-09-10T04:05:08.672 INFO:tasks.workunit.client.1.smithi070.stdout:2/112: fdatasync fc 0 2024-09-10T04:05:08.673 INFO:tasks.workunit.client.1.smithi070.stdout:0/147: sync 2024-09-10T04:05:08.674 INFO:tasks.workunit.client.1.smithi070.stdout:0/148: chown d8/d13/l17 1963210 1 2024-09-10T04:05:08.675 INFO:tasks.workunit.client.1.smithi070.stdout:5/149: truncate f16 4756639 0 2024-09-10T04:05:08.676 INFO:tasks.workunit.client.0.smithi046.stdout:4/141: creat d3/f2e x:0 0 0 2024-09-10T04:05:08.679 INFO:tasks.workunit.client.1.smithi070.stdout:7/136: truncate d2/d3/dc/d1b/f1d 3766201 0 2024-09-10T04:05:08.680 INFO:tasks.workunit.client.0.smithi046.stdout:0/168: rmdir d1/dc/d25 39 2024-09-10T04:05:08.687 INFO:tasks.workunit.client.1.smithi070.stdout:3/168: dwrite f3 [0,4194304] 0 2024-09-10T04:05:08.688 INFO:tasks.workunit.client.1.smithi070.stdout:3/169: fdatasync db/d1c/f1e 0 2024-09-10T04:05:08.688 INFO:tasks.workunit.client.1.smithi070.stdout:3/170: chown db/d1c/c24 870 1 2024-09-10T04:05:08.688 INFO:tasks.workunit.client.1.smithi070.stdout:4/137: creat de/d13/d18/d26/f2d x:0 0 0 2024-09-10T04:05:08.689 INFO:tasks.workunit.client.0.smithi046.stdout:2/208: dwrite d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:08.690 INFO:tasks.workunit.client.1.smithi070.stdout:2/113: mknod d15/d18/c1f 0 2024-09-10T04:05:08.691 INFO:tasks.workunit.client.1.smithi070.stdout:2/114: read f3 [4957262,32125] 0 2024-09-10T04:05:08.693 INFO:tasks.workunit.client.0.smithi046.stdout:4/142: rename d3/c11 to d3/d12/c2f 0 2024-09-10T04:05:08.695 INFO:tasks.workunit.client.1.smithi070.stdout:5/150: fdatasync d17/f1b 0 2024-09-10T04:05:08.697 INFO:tasks.workunit.client.0.smithi046.stdout:5/179: creat d7/d11/f37 x:0 0 0 2024-09-10T04:05:08.699 INFO:tasks.workunit.client.1.smithi070.stdout:6/226: rmdir d0/d3/d18 39 2024-09-10T04:05:08.699 INFO:tasks.workunit.client.1.smithi070.stdout:6/227: dread - d0/d3/d1f/f46 zero size 2024-09-10T04:05:08.700 INFO:tasks.workunit.client.0.smithi046.stdout:2/209: dwrite d2/d8/d37/f41 [0,4194304] 0 2024-09-10T04:05:08.702 INFO:tasks.workunit.client.0.smithi046.stdout:2/210: chown d2/d8/d37/f41 3922 1 2024-09-10T04:05:08.703 INFO:tasks.workunit.client.0.smithi046.stdout:2/211: chown d2/le 79129542 1 2024-09-10T04:05:08.704 INFO:tasks.workunit.client.0.smithi046.stdout:9/165: mknod d4/d7/c2f 0 2024-09-10T04:05:08.706 INFO:tasks.workunit.client.0.smithi046.stdout:9/166: truncate d4/d7/d1f/f1c 676333 0 2024-09-10T04:05:08.706 INFO:tasks.workunit.client.0.smithi046.stdout:9/167: chown l1 13302299 1 2024-09-10T04:05:08.706 INFO:tasks.workunit.client.0.smithi046.stdout:4/143: fdatasync d3/fa 0 2024-09-10T04:05:08.707 INFO:tasks.workunit.client.1.smithi070.stdout:3/171: stat db/fd 0 2024-09-10T04:05:08.709 INFO:tasks.workunit.client.1.smithi070.stdout:2/115: creat d15/d18/f20 x:0 0 0 2024-09-10T04:05:08.710 INFO:tasks.workunit.client.1.smithi070.stdout:0/149: mkdir d8/d24/d2d/d34 0 2024-09-10T04:05:08.712 INFO:tasks.workunit.client.0.smithi046.stdout:8/195: getdents d1/d29/d17 0 2024-09-10T04:05:08.714 INFO:tasks.workunit.client.0.smithi046.stdout:3/141: creat d12/f29 x:0 0 0 2024-09-10T04:05:08.715 INFO:tasks.workunit.client.0.smithi046.stdout:4/144: dwrite d3/d12/f19 [0,4194304] 0 2024-09-10T04:05:08.715 INFO:tasks.workunit.client.0.smithi046.stdout:8/196: write d1/d29/f27 [4140061,89302] 0 2024-09-10T04:05:08.719 INFO:tasks.workunit.client.0.smithi046.stdout:2/212: creat d2/d6/f4f x:0 0 0 2024-09-10T04:05:08.720 INFO:tasks.workunit.client.0.smithi046.stdout:8/197: stat d1/d29/d17/d1f 0 2024-09-10T04:05:08.720 INFO:tasks.workunit.client.0.smithi046.stdout:3/142: write d12/d13/d14/f15 [2139503,56514] 0 2024-09-10T04:05:08.721 INFO:tasks.workunit.client.1.smithi070.stdout:9/149: link d9/dd/d10/d1b/l27 d9/dd/d10/d1b/l29 0 2024-09-10T04:05:08.722 INFO:tasks.workunit.client.0.smithi046.stdout:3/143: stat d12 0 2024-09-10T04:05:08.724 INFO:tasks.workunit.client.0.smithi046.stdout:5/180: dwrite d7/f19 [0,4194304] 0 2024-09-10T04:05:08.724 INFO:tasks.workunit.client.0.smithi046.stdout:0/169: creat d1/f35 x:0 0 0 2024-09-10T04:05:08.725 INFO:tasks.workunit.client.0.smithi046.stdout:8/198: write d1/fa [2672983,29163] 0 2024-09-10T04:05:08.726 INFO:tasks.workunit.client.0.smithi046.stdout:8/199: stat d1/f40 0 2024-09-10T04:05:08.727 INFO:tasks.workunit.client.1.smithi070.stdout:1/107: rename d3/f5 to d3/d8/f28 0 2024-09-10T04:05:08.729 INFO:tasks.workunit.client.0.smithi046.stdout:6/121: getdents da/db/d10/d27 0 2024-09-10T04:05:08.730 INFO:tasks.workunit.client.1.smithi070.stdout:3/172: creat db/d16/f30 x:0 0 0 2024-09-10T04:05:08.731 INFO:tasks.workunit.client.1.smithi070.stdout:2/116: creat d15/d18/f21 x:0 0 0 2024-09-10T04:05:08.733 INFO:tasks.workunit.client.0.smithi046.stdout:5/181: dwrite d7/f9 [0,4194304] 0 2024-09-10T04:05:08.733 INFO:tasks.workunit.client.0.smithi046.stdout:4/145: mkdir d3/d12/d17/d22/d30 0 2024-09-10T04:05:08.734 INFO:tasks.workunit.client.0.smithi046.stdout:9/168: mkdir d4/d30 0 2024-09-10T04:05:08.736 INFO:tasks.workunit.client.0.smithi046.stdout:3/144: unlink d12/d13/d14/d1e/l23 0 2024-09-10T04:05:08.740 INFO:tasks.workunit.client.1.smithi070.stdout:0/150: dwrite d8/d13/f20 [0,4194304] 0 2024-09-10T04:05:08.741 INFO:tasks.workunit.client.0.smithi046.stdout:3/145: write d12/d1b/f21 [531670,124656] 0 2024-09-10T04:05:08.741 INFO:tasks.workunit.client.0.smithi046.stdout:3/146: chown d12/d13/d14 8 1 2024-09-10T04:05:08.741 INFO:tasks.workunit.client.0.smithi046.stdout:2/213: symlink d2/d6/d23/d42/l50 0 2024-09-10T04:05:08.745 INFO:tasks.workunit.client.0.smithi046.stdout:0/170: write d1/dc/d25/d33/d3/f2a [4527600,21825] 0 2024-09-10T04:05:08.746 INFO:tasks.workunit.client.0.smithi046.stdout:0/171: chown d1/dc/f21 15796041 1 2024-09-10T04:05:08.747 INFO:tasks.workunit.client.0.smithi046.stdout:8/200: mkdir d1/d29/d17/d1f/d46 0 2024-09-10T04:05:08.749 INFO:tasks.workunit.client.0.smithi046.stdout:2/214: dwrite d2/d8/f27 [0,4194304] 0 2024-09-10T04:05:08.750 INFO:tasks.workunit.client.0.smithi046.stdout:5/182: mknod d7/d11/d14/c38 0 2024-09-10T04:05:08.751 INFO:tasks.workunit.client.1.smithi070.stdout:3/173: dread f7 [0,4194304] 0 2024-09-10T04:05:08.751 INFO:tasks.workunit.client.1.smithi070.stdout:0/151: dread d8/d13/d1a/f30 [0,4194304] 0 2024-09-10T04:05:08.752 INFO:tasks.workunit.client.1.smithi070.stdout:3/174: readlink db/l20 0 2024-09-10T04:05:08.752 INFO:tasks.workunit.client.1.smithi070.stdout:0/152: dread - d8/f27 zero size 2024-09-10T04:05:08.752 INFO:tasks.workunit.client.1.smithi070.stdout:2/117: mknod d15/c22 0 2024-09-10T04:05:08.753 INFO:tasks.workunit.client.1.smithi070.stdout:5/151: mkdir d17/d24/d2b/d31 0 2024-09-10T04:05:08.754 INFO:tasks.workunit.client.0.smithi046.stdout:3/147: fdatasync f0 0 2024-09-10T04:05:08.754 INFO:tasks.workunit.client.0.smithi046.stdout:3/148: chown d12/d1b/f21 1411 1 2024-09-10T04:05:08.757 INFO:tasks.workunit.client.0.smithi046.stdout:0/172: creat d1/dc/d25/d33/d4/f36 x:0 0 0 2024-09-10T04:05:08.758 INFO:tasks.workunit.client.1.smithi070.stdout:7/137: dread d2/d3/dc/d1b/f1d [0,4194304] 0 2024-09-10T04:05:08.759 INFO:tasks.workunit.client.0.smithi046.stdout:0/173: chown d1/dc/d25/d33/d3/d29/d2d/f32 20616 1 2024-09-10T04:05:08.759 INFO:tasks.workunit.client.1.smithi070.stdout:9/150: mkdir d9/dd/d2a 0 2024-09-10T04:05:08.761 INFO:tasks.workunit.client.0.smithi046.stdout:2/215: creat d2/d6/d23/f51 x:0 0 0 2024-09-10T04:05:08.763 INFO:tasks.workunit.client.1.smithi070.stdout:7/138: dread f1 [0,4194304] 0 2024-09-10T04:05:08.763 INFO:tasks.workunit.client.0.smithi046.stdout:6/122: write da/fe [45647,63260] 0 2024-09-10T04:05:08.763 INFO:tasks.workunit.client.0.smithi046.stdout:2/216: fsync d2/d49/f2e 0 2024-09-10T04:05:08.764 INFO:tasks.workunit.client.0.smithi046.stdout:2/217: stat d2/d8/f27 0 2024-09-10T04:05:08.765 INFO:tasks.workunit.client.1.smithi070.stdout:6/228: dwrite d0/d3/d18/d1b/f14 [4194304,4194304] 0 2024-09-10T04:05:08.765 INFO:tasks.workunit.client.0.smithi046.stdout:0/174: dread d1/dc/d25/d33/d3/dd/f23 [0,4194304] 0 2024-09-10T04:05:08.765 INFO:tasks.workunit.client.0.smithi046.stdout:5/183: symlink d7/d11/d14/d2f/l39 0 2024-09-10T04:05:08.766 INFO:tasks.workunit.client.0.smithi046.stdout:2/218: read d2/d8/d37/f41 [2297099,3264] 0 2024-09-10T04:05:08.766 INFO:tasks.workunit.client.1.smithi070.stdout:6/229: write d0/d3/d18/f35 [85096,39858] 0 2024-09-10T04:05:08.768 INFO:tasks.workunit.client.1.smithi070.stdout:3/175: unlink db/d16/f2c 0 2024-09-10T04:05:08.768 INFO:tasks.workunit.client.0.smithi046.stdout:0/175: write d1/dc/f27 [2579136,7226] 0 2024-09-10T04:05:08.770 INFO:tasks.workunit.client.1.smithi070.stdout:8/189: sync 2024-09-10T04:05:08.771 INFO:tasks.workunit.client.1.smithi070.stdout:1/108: dwrite d3/d8/d12/f19 [0,4194304] 0 2024-09-10T04:05:08.772 INFO:tasks.workunit.client.1.smithi070.stdout:1/109: stat d3/d8/d12 0 2024-09-10T04:05:08.773 INFO:tasks.workunit.client.1.smithi070.stdout:2/118: mknod d15/c23 0 2024-09-10T04:05:08.774 INFO:tasks.workunit.client.1.smithi070.stdout:0/153: creat d8/d13/d1a/f35 x:0 0 0 2024-09-10T04:05:08.774 INFO:tasks.workunit.client.0.smithi046.stdout:5/184: dwrite d7/f1f [0,4194304] 0 2024-09-10T04:05:08.775 INFO:tasks.workunit.client.1.smithi070.stdout:0/154: read - d8/d13/d1a/f32 zero size 2024-09-10T04:05:08.777 INFO:tasks.workunit.client.1.smithi070.stdout:3/176: dwrite db/f22 [0,4194304] 0 2024-09-10T04:05:08.777 INFO:tasks.workunit.client.0.smithi046.stdout:4/146: link d3/lf d3/d12/d17/d22/d30/l31 0 2024-09-10T04:05:08.779 INFO:tasks.workunit.client.0.smithi046.stdout:4/147: truncate d3/d12/d17/d22/f25 225122 0 2024-09-10T04:05:08.785 INFO:tasks.workunit.client.1.smithi070.stdout:6/230: symlink d0/d3/d18/d1b/d17/l49 0 2024-09-10T04:05:08.789 INFO:tasks.workunit.client.1.smithi070.stdout:8/190: symlink d4/d16/d17/d1a/d23/l45 0 2024-09-10T04:05:08.791 INFO:tasks.workunit.client.0.smithi046.stdout:2/219: mkdir d2/d6/d23/d52 0 2024-09-10T04:05:08.793 INFO:tasks.workunit.client.0.smithi046.stdout:9/169: rename d4/f13 to d4/d7/f31 0 2024-09-10T04:05:08.794 INFO:tasks.workunit.client.0.smithi046.stdout:9/170: dread - d4/d7/f1e zero size 2024-09-10T04:05:08.794 INFO:tasks.workunit.client.1.smithi070.stdout:5/152: mkdir d17/d24/d2b/d31/d32 0 2024-09-10T04:05:08.795 INFO:tasks.workunit.client.0.smithi046.stdout:0/176: mkdir d1/d13/d37 0 2024-09-10T04:05:08.797 INFO:tasks.workunit.client.1.smithi070.stdout:0/155: unlink c6 0 2024-09-10T04:05:08.800 INFO:tasks.workunit.client.1.smithi070.stdout:3/177: symlink db/d1c/l31 0 2024-09-10T04:05:08.800 INFO:tasks.workunit.client.1.smithi070.stdout:3/178: stat db/f1f 0 2024-09-10T04:05:08.801 INFO:tasks.workunit.client.0.smithi046.stdout:3/149: symlink d12/l2a 0 2024-09-10T04:05:08.801 INFO:tasks.workunit.client.0.smithi046.stdout:3/150: stat d12/d1b/l28 0 2024-09-10T04:05:08.804 INFO:tasks.workunit.client.1.smithi070.stdout:6/231: readlink d0/l9 0 2024-09-10T04:05:08.805 INFO:tasks.workunit.client.0.smithi046.stdout:3/151: dread d12/d13/d14/f15 [0,4194304] 0 2024-09-10T04:05:08.806 INFO:tasks.workunit.client.0.smithi046.stdout:5/185: creat d7/d11/d14/d2f/d2b/f3a x:0 0 0 2024-09-10T04:05:08.807 INFO:tasks.workunit.client.1.smithi070.stdout:8/191: creat d4/d16/d17/d19/d28/f46 x:0 0 0 2024-09-10T04:05:08.809 INFO:tasks.workunit.client.1.smithi070.stdout:8/192: fdatasync d4/d16/f41 0 2024-09-10T04:05:08.809 INFO:tasks.workunit.client.0.smithi046.stdout:6/123: creat da/db/d11/d2b/f2c x:0 0 0 2024-09-10T04:05:08.810 INFO:tasks.workunit.client.1.smithi070.stdout:6/232: dwrite d0/d3/d18/f35 [0,4194304] 0 2024-09-10T04:05:08.811 INFO:tasks.workunit.client.0.smithi046.stdout:4/148: readlink d3/d12/d17/d22/d30/l31 0 2024-09-10T04:05:08.811 INFO:tasks.workunit.client.0.smithi046.stdout:5/186: dread d7/fd [0,4194304] 0 2024-09-10T04:05:08.811 INFO:tasks.workunit.client.1.smithi070.stdout:5/153: mknod d17/d24/d2b/c33 0 2024-09-10T04:05:08.812 INFO:tasks.workunit.client.0.smithi046.stdout:4/149: chown d3/d12/d14/f2b 1 1 2024-09-10T04:05:08.814 INFO:tasks.workunit.client.1.smithi070.stdout:0/156: mkdir d8/d24/d36 0 2024-09-10T04:05:08.816 INFO:tasks.workunit.client.1.smithi070.stdout:9/151: write d9/dd/d10/f1f [1636809,108438] 0 2024-09-10T04:05:08.816 INFO:tasks.workunit.client.1.smithi070.stdout:3/179: creat db/f32 x:0 0 0 2024-09-10T04:05:08.819 INFO:tasks.workunit.client.0.smithi046.stdout:0/177: read d1/dc/d25/d33/f11 [122114,17165] 0 2024-09-10T04:05:08.820 INFO:tasks.workunit.client.0.smithi046.stdout:3/152: dread f4 [0,4194304] 0 2024-09-10T04:05:08.820 INFO:tasks.workunit.client.1.smithi070.stdout:8/193: mknod d4/d16/d17/d19/c47 0 2024-09-10T04:05:08.823 INFO:tasks.workunit.client.0.smithi046.stdout:6/124: symlink da/db/d10/d27/l2d 0 2024-09-10T04:05:08.825 INFO:tasks.workunit.client.0.smithi046.stdout:4/150: symlink d3/d12/l32 0 2024-09-10T04:05:08.826 INFO:tasks.workunit.client.1.smithi070.stdout:0/157: symlink d8/d24/l37 0 2024-09-10T04:05:08.826 INFO:tasks.workunit.client.0.smithi046.stdout:0/178: dwrite d1/f35 [0,4194304] 0 2024-09-10T04:05:08.828 INFO:tasks.workunit.client.0.smithi046.stdout:5/187: symlink d7/d11/d14/d2f/l3b 0 2024-09-10T04:05:08.828 INFO:tasks.workunit.client.0.smithi046.stdout:2/220: write d2/f3 [3704785,89560] 0 2024-09-10T04:05:08.831 INFO:tasks.workunit.client.1.smithi070.stdout:0/158: dread d8/f1b [0,4194304] 0 2024-09-10T04:05:08.832 INFO:tasks.workunit.client.1.smithi070.stdout:1/110: rename d3/d8/c1e to d3/d8/c29 0 2024-09-10T04:05:08.832 INFO:tasks.workunit.client.1.smithi070.stdout:0/159: fsync d8/d13/d1a/f23 0 2024-09-10T04:05:08.833 INFO:tasks.workunit.client.1.smithi070.stdout:8/194: rmdir d4/d16/d17/d1a 39 2024-09-10T04:05:08.835 INFO:tasks.workunit.client.0.smithi046.stdout:9/171: dwrite d4/d7/ff [0,4194304] 0 2024-09-10T04:05:08.836 INFO:tasks.workunit.client.0.smithi046.stdout:0/179: dread d1/dc/f27 [0,4194304] 0 2024-09-10T04:05:08.836 INFO:tasks.workunit.client.0.smithi046.stdout:9/172: dread - d4/d7/f20 zero size 2024-09-10T04:05:08.837 INFO:tasks.workunit.client.0.smithi046.stdout:9/173: fdatasync d4/f6 0 2024-09-10T04:05:08.839 INFO:tasks.workunit.client.1.smithi070.stdout:1/111: dwrite d3/d8/fc [8388608,4194304] 0 2024-09-10T04:05:08.842 INFO:tasks.workunit.client.0.smithi046.stdout:9/174: truncate d4/d7/f31 843848 0 2024-09-10T04:05:08.843 INFO:tasks.workunit.client.1.smithi070.stdout:9/152: mknod d9/dd/d2a/c2b 0 2024-09-10T04:05:08.844 INFO:tasks.workunit.client.1.smithi070.stdout:9/153: chown d9/dd/f14 6 1 2024-09-10T04:05:08.845 INFO:tasks.workunit.client.1.smithi070.stdout:9/154: fdatasync d9/dd/f12 0 2024-09-10T04:05:08.846 INFO:tasks.workunit.client.1.smithi070.stdout:9/155: readlink d9/dd/d10/d1b/l1e 0 2024-09-10T04:05:08.846 INFO:tasks.workunit.client.1.smithi070.stdout:9/156: chown d9/f25 228135115 1 2024-09-10T04:05:08.847 INFO:tasks.workunit.client.0.smithi046.stdout:3/153: dwrite d12/d13/d14/f15 [4194304,4194304] 0 2024-09-10T04:05:08.852 INFO:tasks.workunit.client.0.smithi046.stdout:5/188: symlink d7/d11/d14/l3c 0 2024-09-10T04:05:08.853 INFO:tasks.workunit.client.1.smithi070.stdout:8/195: read d4/d16/d17/d1a/d23/f2f [1258,6007] 0 2024-09-10T04:05:08.855 INFO:tasks.workunit.client.1.smithi070.stdout:6/233: rmdir d0/d3/d18/d1b/d38 0 2024-09-10T04:05:08.856 INFO:tasks.workunit.client.1.smithi070.stdout:6/234: chown d0/d3/d18/d1b/d17/c3e 1482 1 2024-09-10T04:05:08.856 INFO:tasks.workunit.client.0.smithi046.stdout:2/221: creat d2/d6/d23/d42/f53 x:0 0 0 2024-09-10T04:05:08.857 INFO:tasks.workunit.client.0.smithi046.stdout:5/189: dread d7/d11/f12 [0,4194304] 0 2024-09-10T04:05:08.857 INFO:tasks.workunit.client.0.smithi046.stdout:2/222: readlink d2/d6/d15/l40 0 2024-09-10T04:05:08.858 INFO:tasks.workunit.client.0.smithi046.stdout:0/180: creat d1/dc/d25/d33/d3/d29/f38 x:0 0 0 2024-09-10T04:05:08.859 INFO:tasks.workunit.client.0.smithi046.stdout:2/223: write d2/d6/d23/d42/f4e [430267,116414] 0 2024-09-10T04:05:08.860 INFO:tasks.workunit.client.1.smithi070.stdout:0/160: write d8/f2b [110715,1480] 0 2024-09-10T04:05:08.860 INFO:tasks.workunit.client.0.smithi046.stdout:9/175: rename d4/d30 to d4/d29/d32 0 2024-09-10T04:05:08.861 INFO:tasks.workunit.client.1.smithi070.stdout:0/161: chown d8/d24 202 1 2024-09-10T04:05:08.861 INFO:tasks.workunit.client.1.smithi070.stdout:1/112: write d3/d8/d13/f18 [1507480,42245] 0 2024-09-10T04:05:08.862 INFO:tasks.workunit.client.1.smithi070.stdout:0/162: write d8/d13/d1a/f32 [847938,22301] 0 2024-09-10T04:05:08.863 INFO:tasks.workunit.client.0.smithi046.stdout:6/125: link da/db/f14 da/db/d11/d13/d1e/d23/f2e 0 2024-09-10T04:05:08.865 INFO:tasks.workunit.client.1.smithi070.stdout:4/138: sync 2024-09-10T04:05:08.866 INFO:tasks.workunit.client.0.smithi046.stdout:4/151: symlink d3/d12/d17/d22/d30/l33 0 2024-09-10T04:05:08.867 INFO:tasks.workunit.client.1.smithi070.stdout:9/157: creat d9/d1c/f2c x:0 0 0 2024-09-10T04:05:08.870 INFO:tasks.workunit.client.1.smithi070.stdout:8/196: unlink d4/f8 0 2024-09-10T04:05:08.871 INFO:tasks.workunit.client.1.smithi070.stdout:5/154: getdents d17/d24 0 2024-09-10T04:05:08.873 INFO:tasks.workunit.client.1.smithi070.stdout:1/113: dread d3/d8/fc [4194304,4194304] 0 2024-09-10T04:05:08.873 INFO:tasks.workunit.client.1.smithi070.stdout:6/235: mknod d0/d3/d1f/c4a 0 2024-09-10T04:05:08.875 INFO:tasks.workunit.client.0.smithi046.stdout:5/190: symlink d7/d11/d14/d2f/d2d/l3d 0 2024-09-10T04:05:08.876 INFO:tasks.workunit.client.0.smithi046.stdout:5/191: chown d7/la 74277 1 2024-09-10T04:05:08.878 INFO:tasks.workunit.client.1.smithi070.stdout:0/163: write d8/fa [3149503,76012] 0 2024-09-10T04:05:08.880 INFO:tasks.workunit.client.1.smithi070.stdout:4/139: dread fc [0,4194304] 0 2024-09-10T04:05:08.880 INFO:tasks.workunit.client.0.smithi046.stdout:3/154: rename f0 to d12/d1b/f2b 0 2024-09-10T04:05:08.881 INFO:tasks.workunit.client.1.smithi070.stdout:9/158: creat d9/dd/d10/d1b/f2d x:0 0 0 2024-09-10T04:05:08.881 INFO:tasks.workunit.client.1.smithi070.stdout:9/159: fsync d9/dd/d10/f24 0 2024-09-10T04:05:08.882 INFO:tasks.workunit.client.0.smithi046.stdout:6/126: mkdir da/db/d11/d2b/d2f 0 2024-09-10T04:05:08.883 INFO:tasks.workunit.client.0.smithi046.stdout:4/152: fsync d3/f9 0 2024-09-10T04:05:08.886 INFO:tasks.workunit.client.0.smithi046.stdout:2/224: creat d2/d6/d23/d52/f54 x:0 0 0 2024-09-10T04:05:08.888 INFO:tasks.workunit.client.0.smithi046.stdout:3/155: creat d12/d13/f2c x:0 0 0 2024-09-10T04:05:08.888 INFO:tasks.workunit.client.1.smithi070.stdout:8/197: dwrite d4/d16/d17/d19/d28/f30 [8388608,4194304] 0 2024-09-10T04:05:08.889 INFO:tasks.workunit.client.0.smithi046.stdout:3/156: chown d12/d13/d14/f15 0 1 2024-09-10T04:05:08.889 INFO:tasks.workunit.client.0.smithi046.stdout:3/157: stat d12/l16 0 2024-09-10T04:05:08.890 INFO:tasks.workunit.client.1.smithi070.stdout:4/140: fdatasync de/d13/d18/f21 0 2024-09-10T04:05:08.893 INFO:tasks.workunit.client.1.smithi070.stdout:9/160: creat d9/dd/d2a/f2e x:0 0 0 2024-09-10T04:05:08.895 INFO:tasks.workunit.client.1.smithi070.stdout:6/236: mknod d0/d3/c4b 0 2024-09-10T04:05:08.896 INFO:tasks.workunit.client.1.smithi070.stdout:0/164: mknod d8/d24/d2d/d34/c38 0 2024-09-10T04:05:08.896 INFO:tasks.workunit.client.0.smithi046.stdout:2/225: rmdir d2/d6/d23 39 2024-09-10T04:05:08.896 INFO:tasks.workunit.client.1.smithi070.stdout:9/161: dread d9/dd/d10/f1f [0,4194304] 0 2024-09-10T04:05:08.897 INFO:tasks.workunit.client.0.smithi046.stdout:2/226: chown d2/d8/l38 10559 1 2024-09-10T04:05:08.899 INFO:tasks.workunit.client.0.smithi046.stdout:9/176: link d4/f28 d4/d18/f33 0 2024-09-10T04:05:08.901 INFO:tasks.workunit.client.0.smithi046.stdout:6/127: symlink da/db/d11/d13/d1e/d23/l30 0 2024-09-10T04:05:08.902 INFO:tasks.workunit.client.1.smithi070.stdout:9/162: dwrite d9/f25 [0,4194304] 0 2024-09-10T04:05:08.905 INFO:tasks.workunit.client.1.smithi070.stdout:5/155: getdents d17/d24 0 2024-09-10T04:05:08.906 INFO:tasks.workunit.client.0.smithi046.stdout:5/192: truncate d7/f19 3560393 0 2024-09-10T04:05:08.908 INFO:tasks.workunit.client.0.smithi046.stdout:6/128: symlink da/db/l31 0 2024-09-10T04:05:08.910 INFO:tasks.workunit.client.1.smithi070.stdout:0/165: unlink d8/f10 0 2024-09-10T04:05:08.910 INFO:tasks.workunit.client.1.smithi070.stdout:8/198: write d4/d16/f22 [278280,21369] 0 2024-09-10T04:05:08.911 INFO:tasks.workunit.client.0.smithi046.stdout:4/153: link c2 d3/d12/d14/c34 0 2024-09-10T04:05:08.911 INFO:tasks.workunit.client.1.smithi070.stdout:0/166: fdatasync d8/d13/f22 0 2024-09-10T04:05:08.911 INFO:tasks.workunit.client.1.smithi070.stdout:6/237: rmdir d0/d3/d18/d1b/d33 39 2024-09-10T04:05:08.912 INFO:tasks.workunit.client.0.smithi046.stdout:2/227: dread d2/f35 [4194304,4194304] 0 2024-09-10T04:05:08.913 INFO:tasks.workunit.client.0.smithi046.stdout:9/177: mknod d4/d7/c34 0 2024-09-10T04:05:08.914 INFO:tasks.workunit.client.1.smithi070.stdout:0/167: dread d8/fc [0,4194304] 0 2024-09-10T04:05:08.914 INFO:tasks.workunit.client.0.smithi046.stdout:5/193: symlink d7/d11/d14/l3e 0 2024-09-10T04:05:08.915 INFO:tasks.workunit.client.0.smithi046.stdout:5/194: stat d7/d11/c34 0 2024-09-10T04:05:08.915 INFO:tasks.workunit.client.0.smithi046.stdout:6/129: creat da/db/d11/f32 x:0 0 0 2024-09-10T04:05:08.916 INFO:tasks.workunit.client.1.smithi070.stdout:4/141: fsync de/d13/d18/d26/f2a 0 2024-09-10T04:05:08.917 INFO:tasks.workunit.client.0.smithi046.stdout:6/130: dread - da/db/d11/f32 zero size 2024-09-10T04:05:08.917 INFO:tasks.workunit.client.0.smithi046.stdout:2/228: dread d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:08.917 INFO:tasks.workunit.client.1.smithi070.stdout:9/163: symlink d9/dd/d10/l2f 0 2024-09-10T04:05:08.918 INFO:tasks.workunit.client.0.smithi046.stdout:4/154: symlink d3/d12/d17/d22/d30/l35 0 2024-09-10T04:05:08.920 INFO:tasks.workunit.client.0.smithi046.stdout:9/178: symlink d4/d29/l35 0 2024-09-10T04:05:08.921 INFO:tasks.workunit.client.0.smithi046.stdout:6/131: dread da/db/f14 [0,4194304] 0 2024-09-10T04:05:08.921 INFO:tasks.workunit.client.0.smithi046.stdout:9/179: write d4/f14 [546737,101579] 0 2024-09-10T04:05:08.926 INFO:tasks.workunit.client.1.smithi070.stdout:6/238: creat d0/d3/d18/f4c x:0 0 0 2024-09-10T04:05:08.928 INFO:tasks.workunit.client.1.smithi070.stdout:0/168: creat d8/d13/d1a/f39 x:0 0 0 2024-09-10T04:05:08.929 INFO:tasks.workunit.client.0.smithi046.stdout:4/155: creat d3/d12/d14/f36 x:0 0 0 2024-09-10T04:05:08.929 INFO:tasks.workunit.client.0.smithi046.stdout:4/156: chown d3 25916248 1 2024-09-10T04:05:08.931 INFO:tasks.workunit.client.1.smithi070.stdout:4/142: rmdir de/d13/d18/d26 39 2024-09-10T04:05:08.931 INFO:tasks.workunit.client.1.smithi070.stdout:4/143: fsync de/d13/f1a 0 2024-09-10T04:05:08.932 INFO:tasks.workunit.client.0.smithi046.stdout:5/195: mkdir d7/d11/d3f 0 2024-09-10T04:05:08.934 INFO:tasks.workunit.client.0.smithi046.stdout:9/180: rmdir d4/d29 39 2024-09-10T04:05:08.934 INFO:tasks.workunit.client.0.smithi046.stdout:5/196: write d7/d11/d14/f36 [688125,24613] 0 2024-09-10T04:05:08.935 INFO:tasks.workunit.client.0.smithi046.stdout:4/157: dwrite d3/f15 [0,4194304] 0 2024-09-10T04:05:08.935 INFO:tasks.workunit.client.0.smithi046.stdout:5/197: dread - d7/d11/f37 zero size 2024-09-10T04:05:08.936 INFO:tasks.workunit.client.0.smithi046.stdout:4/158: write d3/fa [1099175,62959] 0 2024-09-10T04:05:08.936 INFO:tasks.workunit.client.0.smithi046.stdout:6/132: symlink da/db/d11/l33 0 2024-09-10T04:05:08.937 INFO:tasks.workunit.client.1.smithi070.stdout:4/144: dwrite de/d13/f1a [0,4194304] 0 2024-09-10T04:05:08.938 INFO:tasks.workunit.client.0.smithi046.stdout:4/159: write d3/d12/f1f [789873,14329] 0 2024-09-10T04:05:08.939 INFO:tasks.workunit.client.1.smithi070.stdout:6/239: unlink d0/d3/d18/d1b/d17/d28/c34 0 2024-09-10T04:05:08.941 INFO:tasks.workunit.client.0.smithi046.stdout:4/160: chown d3/d12/d17/d22/f27 125 1 2024-09-10T04:05:08.941 INFO:tasks.workunit.client.0.smithi046.stdout:5/198: dread d7/d11/f13 [0,4194304] 0 2024-09-10T04:05:08.948 INFO:tasks.workunit.client.1.smithi070.stdout:6/240: dwrite d0/d3/d1f/f46 [0,4194304] 0 2024-09-10T04:05:08.949 INFO:tasks.workunit.client.1.smithi070.stdout:6/241: dread - d0/d3/d18/d1b/d17/f47 zero size 2024-09-10T04:05:08.951 INFO:tasks.workunit.client.0.smithi046.stdout:5/199: dwrite d7/fc [0,4194304] 0 2024-09-10T04:05:08.953 INFO:tasks.workunit.client.0.smithi046.stdout:5/200: chown d7/d11/d14/d2f/f22 11 1 2024-09-10T04:05:08.953 INFO:tasks.workunit.client.1.smithi070.stdout:4/145: mknod de/d13/c2e 0 2024-09-10T04:05:08.955 INFO:tasks.workunit.client.1.smithi070.stdout:9/164: write d9/dd/f1a [931915,115030] 0 2024-09-10T04:05:08.956 INFO:tasks.workunit.client.1.smithi070.stdout:9/165: fdatasync d9/f25 0 2024-09-10T04:05:08.959 INFO:tasks.workunit.client.0.smithi046.stdout:6/133: link da/f1f da/dc/d22/f34 0 2024-09-10T04:05:08.963 INFO:tasks.workunit.client.0.smithi046.stdout:5/201: dwrite d7/d11/d14/f25 [0,4194304] 0 2024-09-10T04:05:08.965 INFO:tasks.workunit.client.1.smithi070.stdout:5/156: link d17/d19/l1f d17/d24/l34 0 2024-09-10T04:05:08.968 INFO:tasks.workunit.client.0.smithi046.stdout:5/202: read d7/fd [4861481,72839] 0 2024-09-10T04:05:08.972 INFO:tasks.workunit.client.1.smithi070.stdout:5/157: dwrite d17/d24/f2a [0,4194304] 0 2024-09-10T04:05:08.973 INFO:tasks.workunit.client.0.smithi046.stdout:6/134: dwrite da/f1f [0,4194304] 0 2024-09-10T04:05:08.983 INFO:tasks.workunit.client.0.smithi046.stdout:6/135: dread da/fe [0,4194304] 0 2024-09-10T04:05:08.984 INFO:tasks.workunit.client.1.smithi070.stdout:5/158: mkdir d17/d19/d35 0 2024-09-10T04:05:08.985 INFO:tasks.workunit.client.1.smithi070.stdout:0/169: link d8/d13/f1f d8/d24/d2d/f3a 0 2024-09-10T04:05:08.985 INFO:tasks.workunit.client.0.smithi046.stdout:6/136: read da/db/f14 [3266768,28166] 0 2024-09-10T04:05:08.985 INFO:tasks.workunit.client.1.smithi070.stdout:0/170: dread - d8/d24/f2a zero size 2024-09-10T04:05:08.987 INFO:tasks.workunit.client.0.smithi046.stdout:5/203: write d7/fd [1402820,116305] 0 2024-09-10T04:05:08.989 INFO:tasks.workunit.client.1.smithi070.stdout:6/242: rename d0/d3/d18/d1b/d17/d28/f2d to d0/f4d 0 2024-09-10T04:05:08.990 INFO:tasks.workunit.client.0.smithi046.stdout:6/137: dread da/db/d10/f1a [0,4194304] 0 2024-09-10T04:05:08.992 INFO:tasks.workunit.client.1.smithi070.stdout:9/166: creat d9/dd/f30 x:0 0 0 2024-09-10T04:05:08.992 INFO:tasks.workunit.client.1.smithi070.stdout:9/167: chown d9/dd/f14 112049080 1 2024-09-10T04:05:08.992 INFO:tasks.workunit.client.0.smithi046.stdout:5/204: mknod d7/d11/d14/d2f/d2d/c40 0 2024-09-10T04:05:08.993 INFO:tasks.workunit.client.0.smithi046.stdout:5/205: readlink d7/d11/d14/d2f/l3b 0 2024-09-10T04:05:08.994 INFO:tasks.workunit.client.1.smithi070.stdout:5/159: chown d17/d19/l1f 650843 1 2024-09-10T04:05:08.994 INFO:tasks.workunit.client.1.smithi070.stdout:5/160: write fc [710188,114842] 0 2024-09-10T04:05:08.995 INFO:tasks.workunit.client.1.smithi070.stdout:5/161: write fd [4450863,35927] 0 2024-09-10T04:05:08.998 INFO:tasks.workunit.client.0.smithi046.stdout:5/206: mkdir d7/d11/d14/d2f/d2b/d41 0 2024-09-10T04:05:08.999 INFO:tasks.workunit.client.1.smithi070.stdout:4/146: rename de/d13/f1f to de/d13/d18/d26/f2f 0 2024-09-10T04:05:08.999 INFO:tasks.workunit.client.0.smithi046.stdout:6/138: creat da/db/d18/f35 x:0 0 0 2024-09-10T04:05:09.002 INFO:tasks.workunit.client.1.smithi070.stdout:9/168: unlink d9/l21 0 2024-09-10T04:05:09.004 INFO:tasks.workunit.client.1.smithi070.stdout:9/169: dread d9/dd/f14 [4194304,4194304] 0 2024-09-10T04:05:09.005 INFO:tasks.workunit.client.1.smithi070.stdout:9/170: readlink l4 0 2024-09-10T04:05:09.006 INFO:tasks.workunit.client.1.smithi070.stdout:9/171: write d9/dd/f18 [4968648,70444] 0 2024-09-10T04:05:09.007 INFO:tasks.workunit.client.1.smithi070.stdout:1/114: sync 2024-09-10T04:05:09.008 INFO:tasks.workunit.client.1.smithi070.stdout:3/180: sync 2024-09-10T04:05:09.008 INFO:tasks.workunit.client.1.smithi070.stdout:7/139: sync 2024-09-10T04:05:09.008 INFO:tasks.workunit.client.1.smithi070.stdout:2/119: sync 2024-09-10T04:05:09.008 INFO:tasks.workunit.client.0.smithi046.stdout:6/139: dwrite da/db/d10/f17 [8388608,4194304] 0 2024-09-10T04:05:09.008 INFO:tasks.workunit.client.1.smithi070.stdout:9/172: dread d9/dd/f14 [4194304,4194304] 0 2024-09-10T04:05:09.009 INFO:tasks.workunit.client.1.smithi070.stdout:9/173: fsync d9/dd/f14 0 2024-09-10T04:05:09.010 INFO:tasks.workunit.client.1.smithi070.stdout:6/243: write d0/d3/d18/f45 [121592,106765] 0 2024-09-10T04:05:09.012 INFO:tasks.workunit.client.1.smithi070.stdout:6/244: dread - d0/d3/d18/d1b/d17/d28/f43 zero size 2024-09-10T04:05:09.013 INFO:tasks.workunit.client.0.smithi046.stdout:6/140: creat da/db/d10/d27/f36 x:0 0 0 2024-09-10T04:05:09.013 INFO:tasks.workunit.client.1.smithi070.stdout:6/245: chown d0/d3/d18/f42 763 1 2024-09-10T04:05:09.013 INFO:tasks.workunit.client.1.smithi070.stdout:7/140: dread d2/d3/dc/f1e [0,4194304] 0 2024-09-10T04:05:09.014 INFO:tasks.workunit.client.0.smithi046.stdout:5/207: truncate d7/f1f 1445883 0 2024-09-10T04:05:09.014 INFO:tasks.workunit.client.1.smithi070.stdout:4/147: rename de/d13/c15 to de/d13/c30 0 2024-09-10T04:05:09.018 INFO:tasks.workunit.client.1.smithi070.stdout:0/171: write d8/d24/d2d/f3a [546551,95345] 0 2024-09-10T04:05:09.018 INFO:tasks.workunit.client.0.smithi046.stdout:6/141: rmdir da/db 39 2024-09-10T04:05:09.019 INFO:tasks.workunit.client.1.smithi070.stdout:6/246: dwrite d0/d3/d18/d1b/f14 [0,4194304] 0 2024-09-10T04:05:09.021 INFO:tasks.workunit.client.0.smithi046.stdout:6/142: read - da/db/d11/f32 zero size 2024-09-10T04:05:09.028 INFO:tasks.workunit.client.0.smithi046.stdout:6/143: fsync f1 0 2024-09-10T04:05:09.029 INFO:tasks.workunit.client.1.smithi070.stdout:9/174: mkdir d9/d31 0 2024-09-10T04:05:09.030 INFO:tasks.workunit.client.1.smithi070.stdout:9/175: chown d9/dd/d10/l2f 0 1 2024-09-10T04:05:09.037 INFO:tasks.workunit.client.1.smithi070.stdout:0/172: unlink d8/c1d 0 2024-09-10T04:05:09.038 INFO:tasks.workunit.client.1.smithi070.stdout:5/162: creat d17/f36 x:0 0 0 2024-09-10T04:05:09.039 INFO:tasks.workunit.client.1.smithi070.stdout:1/115: mknod d3/d8/d27/c2a 0 2024-09-10T04:05:09.040 INFO:tasks.workunit.client.0.smithi046.stdout:5/208: dwrite d7/f1f [0,4194304] 0 2024-09-10T04:05:09.042 INFO:tasks.workunit.client.0.smithi046.stdout:6/144: dwrite da/fe [0,4194304] 0 2024-09-10T04:05:09.043 INFO:tasks.workunit.client.1.smithi070.stdout:8/199: sync 2024-09-10T04:05:09.044 INFO:tasks.workunit.client.0.smithi046.stdout:6/145: write da/dc/f28 [813941,127057] 0 2024-09-10T04:05:09.045 INFO:tasks.workunit.client.1.smithi070.stdout:3/181: link db/d16/f2d db/f33 0 2024-09-10T04:05:09.047 INFO:tasks.workunit.client.1.smithi070.stdout:8/200: chown d4/d16/d17/f40 152006 1 2024-09-10T04:05:09.048 INFO:tasks.workunit.client.1.smithi070.stdout:7/141: creat d2/d7/d9/f31 x:0 0 0 2024-09-10T04:05:09.050 INFO:tasks.workunit.client.1.smithi070.stdout:5/163: dwrite d17/d24/f2f [0,4194304] 0 2024-09-10T04:05:09.053 INFO:tasks.workunit.client.1.smithi070.stdout:4/148: link de/f11 de/f31 0 2024-09-10T04:05:09.055 INFO:tasks.workunit.client.1.smithi070.stdout:1/116: dwrite d3/d8/d13/f25 [0,4194304] 0 2024-09-10T04:05:09.056 INFO:tasks.workunit.client.1.smithi070.stdout:6/247: fdatasync d0/d3/d18/d1b/d17/d3f/d40/f48 0 2024-09-10T04:05:09.059 INFO:tasks.workunit.client.1.smithi070.stdout:2/120: link f10 d15/d18/f24 0 2024-09-10T04:05:09.060 INFO:tasks.workunit.client.1.smithi070.stdout:9/176: creat d9/d31/f32 x:0 0 0 2024-09-10T04:05:09.060 INFO:tasks.workunit.client.1.smithi070.stdout:8/201: dwrite d4/d16/d17/f40 [0,4194304] 0 2024-09-10T04:05:09.060 INFO:tasks.workunit.client.0.smithi046.stdout:6/146: fsync da/fe 0 2024-09-10T04:05:09.062 INFO:tasks.workunit.client.1.smithi070.stdout:3/182: creat db/d1c/f34 x:0 0 0 2024-09-10T04:05:09.062 INFO:tasks.workunit.client.1.smithi070.stdout:8/202: write d4/d16/f22 [194282,83672] 0 2024-09-10T04:05:09.069 INFO:tasks.workunit.client.0.smithi046.stdout:6/147: creat da/db/d11/d2b/d2f/f37 x:0 0 0 2024-09-10T04:05:09.070 INFO:tasks.workunit.client.1.smithi070.stdout:0/173: mknod d8/d24/d36/c3b 0 2024-09-10T04:05:09.070 INFO:tasks.workunit.client.1.smithi070.stdout:8/203: dwrite d4/d16/d17/f2c [0,4194304] 0 2024-09-10T04:05:09.071 INFO:tasks.workunit.client.0.smithi046.stdout:6/148: dread - da/db/d18/f35 zero size 2024-09-10T04:05:09.075 INFO:tasks.workunit.client.0.smithi046.stdout:5/209: dwrite d7/d11/f13 [0,4194304] 0 2024-09-10T04:05:09.078 INFO:tasks.workunit.client.0.smithi046.stdout:6/149: dread da/dc/d22/f34 [0,4194304] 0 2024-09-10T04:05:09.078 INFO:tasks.workunit.client.1.smithi070.stdout:1/117: unlink d3/d8/d13/l23 0 2024-09-10T04:05:09.082 INFO:tasks.workunit.client.1.smithi070.stdout:0/174: dwrite d8/d24/f2a [0,4194304] 0 2024-09-10T04:05:09.082 INFO:tasks.workunit.client.0.smithi046.stdout:6/150: dread da/f1f [0,4194304] 0 2024-09-10T04:05:09.083 INFO:tasks.workunit.client.0.smithi046.stdout:6/151: chown l2 3651 1 2024-09-10T04:05:09.083 INFO:tasks.workunit.client.1.smithi070.stdout:2/121: mknod d15/c25 0 2024-09-10T04:05:09.084 INFO:tasks.workunit.client.1.smithi070.stdout:1/118: read d3/d8/d12/f19 [3557976,91429] 0 2024-09-10T04:05:09.085 INFO:tasks.workunit.client.1.smithi070.stdout:2/122: read f11 [56129,28585] 0 2024-09-10T04:05:09.086 INFO:tasks.workunit.client.0.smithi046.stdout:5/210: dread d7/f19 [0,4194304] 0 2024-09-10T04:05:09.087 INFO:tasks.workunit.client.1.smithi070.stdout:9/177: rename d9/dd/d10/f24 to d9/d31/f33 0 2024-09-10T04:05:09.088 INFO:tasks.workunit.client.1.smithi070.stdout:1/119: dread - d3/d8/d13/f17 zero size 2024-09-10T04:05:09.091 INFO:tasks.workunit.client.1.smithi070.stdout:5/164: mkdir d17/d37 0 2024-09-10T04:05:09.092 INFO:tasks.workunit.client.1.smithi070.stdout:4/149: write de/d13/d18/f21 [761881,4264] 0 2024-09-10T04:05:09.094 INFO:tasks.workunit.client.0.smithi046.stdout:5/211: dwrite d7/f9 [0,4194304] 0 2024-09-10T04:05:09.095 INFO:tasks.workunit.client.1.smithi070.stdout:7/142: dwrite d2/d3/dc/d1b/f1d [0,4194304] 0 2024-09-10T04:05:09.098 INFO:tasks.workunit.client.1.smithi070.stdout:0/175: dwrite d8/d13/f1f [0,4194304] 0 2024-09-10T04:05:09.098 INFO:tasks.workunit.client.0.smithi046.stdout:6/152: link da/dc/cf da/db/d11/c38 0 2024-09-10T04:05:09.098 INFO:tasks.workunit.client.1.smithi070.stdout:4/150: dread f6 [0,4194304] 0 2024-09-10T04:05:09.103 INFO:tasks.workunit.client.0.smithi046.stdout:5/212: dwrite d7/f1f [0,4194304] 0 2024-09-10T04:05:09.105 INFO:tasks.workunit.client.1.smithi070.stdout:6/248: mknod d0/d3/d18/d1b/d33/c4e 0 2024-09-10T04:05:09.106 INFO:tasks.workunit.client.1.smithi070.stdout:3/183: write db/d16/f2d [668666,22618] 0 2024-09-10T04:05:09.107 INFO:tasks.workunit.client.0.smithi046.stdout:6/153: rename da/db/f1c to da/db/d11/d13/f39 0 2024-09-10T04:05:09.108 INFO:tasks.workunit.client.0.smithi046.stdout:6/154: readlink l2 0 2024-09-10T04:05:09.116 INFO:tasks.workunit.client.0.smithi046.stdout:5/213: rename d7/c30 to d7/d11/d14/d2f/d2b/c42 0 2024-09-10T04:05:09.118 INFO:tasks.workunit.client.0.smithi046.stdout:6/155: creat da/db/d11/f3a x:0 0 0 2024-09-10T04:05:09.119 INFO:tasks.workunit.client.0.smithi046.stdout:6/156: dread - da/db/d18/f35 zero size 2024-09-10T04:05:09.120 INFO:tasks.workunit.client.0.smithi046.stdout:6/157: write da/f2a [708211,29756] 0 2024-09-10T04:05:09.121 INFO:tasks.workunit.client.1.smithi070.stdout:2/123: creat d15/d18/f26 x:0 0 0 2024-09-10T04:05:09.124 INFO:tasks.workunit.client.1.smithi070.stdout:1/120: mknod d3/c2b 0 2024-09-10T04:05:09.124 INFO:tasks.workunit.client.1.smithi070.stdout:1/121: fdatasync d3/d8/d13/f1b 0 2024-09-10T04:05:09.125 INFO:tasks.workunit.client.1.smithi070.stdout:1/122: write d3/d8/d13/f25 [4649374,28231] 0 2024-09-10T04:05:09.126 INFO:tasks.workunit.client.0.smithi046.stdout:5/214: link d7/f1f d7/f43 0 2024-09-10T04:05:09.128 INFO:tasks.workunit.client.1.smithi070.stdout:7/143: unlink d2/d3/dc/f1e 0 2024-09-10T04:05:09.133 INFO:tasks.workunit.client.0.smithi046.stdout:5/215: symlink d7/d11/d14/d2f/d2b/l44 0 2024-09-10T04:05:09.134 INFO:tasks.workunit.client.1.smithi070.stdout:4/151: rmdir de/d13 39 2024-09-10T04:05:09.136 INFO:tasks.workunit.client.1.smithi070.stdout:6/249: creat d0/d3/d18/d2a/f4f x:0 0 0 2024-09-10T04:05:09.139 INFO:tasks.workunit.client.1.smithi070.stdout:8/204: rename d4/d16/l1b to d4/d16/d17/d1a/l48 0 2024-09-10T04:05:09.139 INFO:tasks.workunit.client.1.smithi070.stdout:8/205: write d4/fa [893367,67291] 0 2024-09-10T04:05:09.140 INFO:tasks.workunit.client.1.smithi070.stdout:2/124: creat d15/d18/f27 x:0 0 0 2024-09-10T04:05:09.140 INFO:tasks.workunit.client.1.smithi070.stdout:8/206: dread - d4/d16/f41 zero size 2024-09-10T04:05:09.141 INFO:tasks.workunit.client.1.smithi070.stdout:5/165: mknod d17/d24/d2b/d31/d32/c38 0 2024-09-10T04:05:09.142 INFO:tasks.workunit.client.1.smithi070.stdout:1/123: creat d3/d10/f2c x:0 0 0 2024-09-10T04:05:09.143 INFO:tasks.workunit.client.1.smithi070.stdout:9/178: write d9/dd/d10/f1f [2354076,105642] 0 2024-09-10T04:05:09.144 INFO:tasks.workunit.client.1.smithi070.stdout:1/124: truncate d3/d8/d13/f1b 177280 0 2024-09-10T04:05:09.144 INFO:tasks.workunit.client.1.smithi070.stdout:0/176: link d8/d13/d1a/f30 d8/d13/d1a/f3c 0 2024-09-10T04:05:09.146 INFO:tasks.workunit.client.1.smithi070.stdout:3/184: mknod db/d1c/d25/d28/c35 0 2024-09-10T04:05:09.146 INFO:tasks.workunit.client.1.smithi070.stdout:0/177: truncate d8/d13/f22 856973 0 2024-09-10T04:05:09.146 INFO:tasks.workunit.client.1.smithi070.stdout:0/178: read d8/d13/f22 [637029,47749] 0 2024-09-10T04:05:09.147 INFO:tasks.workunit.client.1.smithi070.stdout:2/125: creat d15/d18/f28 x:0 0 0 2024-09-10T04:05:09.149 INFO:tasks.workunit.client.0.smithi046.stdout:5/216: truncate d7/f43 3873365 0 2024-09-10T04:05:09.151 INFO:tasks.workunit.client.1.smithi070.stdout:9/179: rename d9/d31 to d9/d1c/d34 0 2024-09-10T04:05:09.151 INFO:tasks.workunit.client.1.smithi070.stdout:0/179: dwrite d8/d13/d1a/f23 [0,4194304] 0 2024-09-10T04:05:09.153 INFO:tasks.workunit.client.1.smithi070.stdout:9/180: readlink d9/dd/d10/d1b/l29 0 2024-09-10T04:05:09.153 INFO:tasks.workunit.client.1.smithi070.stdout:1/125: unlink d3/l22 0 2024-09-10T04:05:09.156 INFO:tasks.workunit.client.0.smithi046.stdout:5/217: rename d7/d11/d14/d2f/d2d to d7/d11/d14/d45 0 2024-09-10T04:05:09.156 INFO:tasks.workunit.client.0.smithi046.stdout:5/218: write d7/f9 [1338484,97123] 0 2024-09-10T04:05:09.157 INFO:tasks.workunit.client.1.smithi070.stdout:1/126: write d3/f16 [1052307,31016] 0 2024-09-10T04:05:09.157 INFO:tasks.workunit.client.1.smithi070.stdout:1/127: chown d3/d20 5952 1 2024-09-10T04:05:09.157 INFO:tasks.workunit.client.1.smithi070.stdout:4/152: symlink de/d13/l32 0 2024-09-10T04:05:09.158 INFO:tasks.workunit.client.1.smithi070.stdout:3/185: mknod db/d1c/d25/c36 0 2024-09-10T04:05:09.159 INFO:tasks.workunit.client.1.smithi070.stdout:3/186: readlink db/l19 0 2024-09-10T04:05:09.160 INFO:tasks.workunit.client.1.smithi070.stdout:9/181: dwrite d9/d1c/f2c [0,4194304] 0 2024-09-10T04:05:09.162 INFO:tasks.workunit.client.1.smithi070.stdout:8/207: creat d4/d16/d17/d19/d28/d43/f49 x:0 0 0 2024-09-10T04:05:09.166 INFO:tasks.workunit.client.1.smithi070.stdout:2/126: mkdir d15/d18/d29 0 2024-09-10T04:05:09.167 INFO:tasks.workunit.client.1.smithi070.stdout:0/180: rmdir d8/d24/d2d 39 2024-09-10T04:05:09.169 INFO:tasks.workunit.client.1.smithi070.stdout:4/153: creat de/d13/d18/d26/f33 x:0 0 0 2024-09-10T04:05:09.170 INFO:tasks.workunit.client.1.smithi070.stdout:3/187: creat db/d1c/d25/d28/f37 x:0 0 0 2024-09-10T04:05:09.172 INFO:tasks.workunit.client.1.smithi070.stdout:0/181: chown d8/l15 33057 1 2024-09-10T04:05:09.172 INFO:tasks.workunit.client.1.smithi070.stdout:7/144: link d2/l11 d2/d7/l32 0 2024-09-10T04:05:09.174 INFO:tasks.workunit.client.1.smithi070.stdout:5/166: dread f10 [0,4194304] 0 2024-09-10T04:05:09.174 INFO:tasks.workunit.client.1.smithi070.stdout:2/127: truncate d15/d18/f27 312385 0 2024-09-10T04:05:09.175 INFO:tasks.workunit.client.1.smithi070.stdout:7/145: fdatasync d2/f15 0 2024-09-10T04:05:09.175 INFO:tasks.workunit.client.1.smithi070.stdout:1/128: rename d3/f16 to d3/d8/f2d 0 2024-09-10T04:05:09.178 INFO:tasks.workunit.client.1.smithi070.stdout:8/208: dwrite d4/d16/d17/d1a/f2a [0,4194304] 0 2024-09-10T04:05:09.178 INFO:tasks.workunit.client.1.smithi070.stdout:9/182: dwrite d9/dd/d10/f19 [0,4194304] 0 2024-09-10T04:05:09.179 INFO:tasks.workunit.client.1.smithi070.stdout:7/146: chown d2/d7/d9/f31 457 1 2024-09-10T04:05:09.182 INFO:tasks.workunit.client.1.smithi070.stdout:9/183: read - d9/dd/f30 zero size 2024-09-10T04:05:09.185 INFO:tasks.workunit.client.1.smithi070.stdout:5/167: symlink d17/d37/l39 0 2024-09-10T04:05:09.187 INFO:tasks.workunit.client.1.smithi070.stdout:7/147: readlink d2/d7/db/lf 0 2024-09-10T04:05:09.187 INFO:tasks.workunit.client.1.smithi070.stdout:3/188: dwrite db/d16/f2d [0,4194304] 0 2024-09-10T04:05:09.188 INFO:tasks.workunit.client.1.smithi070.stdout:1/129: mknod d3/d8/c2e 0 2024-09-10T04:05:09.188 INFO:tasks.workunit.client.1.smithi070.stdout:9/184: write d9/dd/f18 [3935164,97851] 0 2024-09-10T04:05:09.188 INFO:tasks.workunit.client.1.smithi070.stdout:5/168: dread - d17/d24/f2c zero size 2024-09-10T04:05:09.189 INFO:tasks.workunit.client.1.smithi070.stdout:8/209: creat d4/d16/d17/d19/d28/f4a x:0 0 0 2024-09-10T04:05:09.191 INFO:tasks.workunit.client.1.smithi070.stdout:0/182: creat d8/d13/f3d x:0 0 0 2024-09-10T04:05:09.194 INFO:tasks.workunit.client.0.smithi046.stdout:5/219: truncate d7/fd 3497981 0 2024-09-10T04:05:09.195 INFO:tasks.workunit.client.1.smithi070.stdout:7/148: mkdir d2/d7/db/d33 0 2024-09-10T04:05:09.195 INFO:tasks.workunit.client.1.smithi070.stdout:2/128: dwrite d15/d18/f28 [0,4194304] 0 2024-09-10T04:05:09.198 INFO:tasks.workunit.client.1.smithi070.stdout:2/129: fdatasync d15/d18/f1a 0 2024-09-10T04:05:09.199 INFO:tasks.workunit.client.1.smithi070.stdout:8/210: dread - d4/d16/d17/d19/d28/f31 zero size 2024-09-10T04:05:09.199 INFO:tasks.workunit.client.1.smithi070.stdout:5/169: dread - d17/d24/f2d zero size 2024-09-10T04:05:09.199 INFO:tasks.workunit.client.1.smithi070.stdout:7/149: write d2/f15 [296734,114573] 0 2024-09-10T04:05:09.199 INFO:tasks.workunit.client.0.smithi046.stdout:5/220: symlink d7/d11/d14/d2f/d2b/l46 0 2024-09-10T04:05:09.200 INFO:tasks.workunit.client.0.smithi046.stdout:5/221: write d7/d11/f28 [882650,38903] 0 2024-09-10T04:05:09.202 INFO:tasks.workunit.client.1.smithi070.stdout:9/185: creat d9/f35 x:0 0 0 2024-09-10T04:05:09.204 INFO:tasks.workunit.client.0.smithi046.stdout:5/222: creat d7/d11/d14/d2f/d2b/d41/f47 x:0 0 0 2024-09-10T04:05:09.205 INFO:tasks.workunit.client.1.smithi070.stdout:0/183: symlink d8/d13/l3e 0 2024-09-10T04:05:09.205 INFO:tasks.workunit.client.1.smithi070.stdout:8/211: chown d4/d16/d17/d19/c47 2197 1 2024-09-10T04:05:09.206 INFO:tasks.workunit.client.0.smithi046.stdout:5/223: rmdir d7/d11/d14/d45 39 2024-09-10T04:05:09.207 INFO:tasks.workunit.client.1.smithi070.stdout:5/170: chown d17/d24/f2a 85510639 1 2024-09-10T04:05:09.207 INFO:tasks.workunit.client.1.smithi070.stdout:0/184: chown d8/d13/d1a 99929130 1 2024-09-10T04:05:09.207 INFO:tasks.workunit.client.1.smithi070.stdout:7/150: creat d2/d3/dc/d1b/f34 x:0 0 0 2024-09-10T04:05:09.209 INFO:tasks.workunit.client.1.smithi070.stdout:9/186: unlink d9/d1c/l23 0 2024-09-10T04:05:09.210 INFO:tasks.workunit.client.1.smithi070.stdout:5/171: write fc [195735,44567] 0 2024-09-10T04:05:09.211 INFO:tasks.workunit.client.1.smithi070.stdout:1/130: dwrite d3/d10/f2c [0,4194304] 0 2024-09-10T04:05:09.215 INFO:tasks.workunit.client.1.smithi070.stdout:5/172: read d17/f1b [87202,43921] 0 2024-09-10T04:05:09.217 INFO:tasks.workunit.client.1.smithi070.stdout:2/130: dwrite d15/d18/f19 [0,4194304] 0 2024-09-10T04:05:09.217 INFO:tasks.workunit.client.1.smithi070.stdout:0/185: creat d8/d13/d33/f3f x:0 0 0 2024-09-10T04:05:09.217 INFO:tasks.workunit.client.1.smithi070.stdout:5/173: fsync d17/d19/f23 0 2024-09-10T04:05:09.220 INFO:tasks.workunit.client.1.smithi070.stdout:7/151: mknod d2/d7/d9/d18/d23/c35 0 2024-09-10T04:05:09.221 INFO:tasks.workunit.client.1.smithi070.stdout:1/131: mkdir d3/d8/d12/d2f 0 2024-09-10T04:05:09.221 INFO:tasks.workunit.client.0.smithi046.stdout:5/224: dwrite d7/f9 [0,4194304] 0 2024-09-10T04:05:09.222 INFO:tasks.workunit.client.1.smithi070.stdout:5/174: write d17/d19/f1d [316291,37160] 0 2024-09-10T04:05:09.225 INFO:tasks.workunit.client.1.smithi070.stdout:3/189: dwrite db/d16/f2d [4194304,4194304] 0 2024-09-10T04:05:09.229 INFO:tasks.workunit.client.0.smithi046.stdout:5/225: rename d7/f9 to d7/d11/d14/d2f/f48 0 2024-09-10T04:05:09.233 INFO:tasks.workunit.client.1.smithi070.stdout:0/186: symlink d8/d13/d33/l40 0 2024-09-10T04:05:09.235 INFO:tasks.workunit.client.1.smithi070.stdout:1/132: rmdir d3 39 2024-09-10T04:05:09.239 INFO:tasks.workunit.client.1.smithi070.stdout:9/187: getdents d9 0 2024-09-10T04:05:09.239 INFO:tasks.workunit.client.0.smithi046.stdout:5/226: creat d7/f49 x:0 0 0 2024-09-10T04:05:09.240 INFO:tasks.workunit.client.1.smithi070.stdout:0/187: dwrite d8/d13/d33/f3f [0,4194304] 0 2024-09-10T04:05:09.241 INFO:tasks.workunit.client.1.smithi070.stdout:2/131: mkdir d15/d18/d29/d2a 0 2024-09-10T04:05:09.241 INFO:tasks.workunit.client.1.smithi070.stdout:3/190: mknod db/d1c/d25/c38 0 2024-09-10T04:05:09.242 INFO:tasks.workunit.client.1.smithi070.stdout:8/212: dread d4/d16/d17/d1a/f32 [0,4194304] 0 2024-09-10T04:05:09.243 INFO:tasks.workunit.client.1.smithi070.stdout:7/152: mknod d2/d7/c36 0 2024-09-10T04:05:09.243 INFO:tasks.workunit.client.1.smithi070.stdout:3/191: truncate db/f11 898768 0 2024-09-10T04:05:09.245 INFO:tasks.workunit.client.1.smithi070.stdout:3/192: write db/f32 [606232,89652] 0 2024-09-10T04:05:09.246 INFO:tasks.workunit.client.1.smithi070.stdout:5/175: creat d17/d19/d35/f3a x:0 0 0 2024-09-10T04:05:09.248 INFO:tasks.workunit.client.1.smithi070.stdout:5/176: stat d17/d37 0 2024-09-10T04:05:09.249 INFO:tasks.workunit.client.1.smithi070.stdout:2/132: mkdir d15/d2b 0 2024-09-10T04:05:09.250 INFO:tasks.workunit.client.0.smithi046.stdout:5/227: link d7/d11/d14/d2f/d2b/l46 d7/d11/d14/d2f/d2b/d41/l4a 0 2024-09-10T04:05:09.250 INFO:tasks.workunit.client.1.smithi070.stdout:5/177: chown f3 22339 1 2024-09-10T04:05:09.250 INFO:tasks.workunit.client.0.smithi046.stdout:5/228: fsync d7/d11/d14/f29 0 2024-09-10T04:05:09.254 INFO:tasks.workunit.client.1.smithi070.stdout:0/188: dwrite d8/d13/f28 [0,4194304] 0 2024-09-10T04:05:09.256 INFO:tasks.workunit.client.1.smithi070.stdout:2/133: symlink d15/d18/l2c 0 2024-09-10T04:05:09.256 INFO:tasks.workunit.client.0.smithi046.stdout:5/229: dwrite d7/f1e [4194304,4194304] 0 2024-09-10T04:05:09.256 INFO:tasks.workunit.client.1.smithi070.stdout:8/213: mkdir d4/d16/d4b 0 2024-09-10T04:05:09.257 INFO:tasks.workunit.client.1.smithi070.stdout:3/193: dwrite db/f14 [0,4194304] 0 2024-09-10T04:05:09.258 INFO:tasks.workunit.client.1.smithi070.stdout:8/214: stat d4/d16/f36 0 2024-09-10T04:05:09.258 INFO:tasks.workunit.client.1.smithi070.stdout:5/178: mknod d17/d24/c3b 0 2024-09-10T04:05:09.259 INFO:tasks.workunit.client.1.smithi070.stdout:2/134: write d15/d18/f27 [598253,30438] 0 2024-09-10T04:05:09.260 INFO:tasks.workunit.client.1.smithi070.stdout:3/194: read db/f11 [125736,120749] 0 2024-09-10T04:05:09.262 INFO:tasks.workunit.client.1.smithi070.stdout:5/179: chown fc 154872457 1 2024-09-10T04:05:09.262 INFO:tasks.workunit.client.0.smithi046.stdout:5/230: readlink d7/d11/l2c 0 2024-09-10T04:05:09.265 INFO:tasks.workunit.client.1.smithi070.stdout:1/133: dwrite d3/d8/f2d [0,4194304] 0 2024-09-10T04:05:09.266 INFO:tasks.workunit.client.1.smithi070.stdout:1/134: readlink d3/l11 0 2024-09-10T04:05:09.268 INFO:tasks.workunit.client.1.smithi070.stdout:7/153: link d2/d3/dc/d1b/f34 d2/d7/db/f37 0 2024-09-10T04:05:09.270 INFO:tasks.workunit.client.1.smithi070.stdout:9/188: getdents d9/d1c 0 2024-09-10T04:05:09.270 INFO:tasks.workunit.client.0.smithi046.stdout:5/231: creat d7/d11/d14/d45/f4b x:0 0 0 2024-09-10T04:05:09.270 INFO:tasks.workunit.client.1.smithi070.stdout:7/154: chown d2/d3 2239302 1 2024-09-10T04:05:09.271 INFO:tasks.workunit.client.1.smithi070.stdout:2/135: creat d15/d18/d29/f2d x:0 0 0 2024-09-10T04:05:09.272 INFO:tasks.workunit.client.1.smithi070.stdout:9/189: fsync d9/dd/f12 0 2024-09-10T04:05:09.274 INFO:tasks.workunit.client.0.smithi046.stdout:5/232: fsync d7/f43 0 2024-09-10T04:05:09.276 INFO:tasks.workunit.client.1.smithi070.stdout:3/195: dwrite f3 [0,4194304] 0 2024-09-10T04:05:09.276 INFO:tasks.workunit.client.1.smithi070.stdout:2/136: truncate d15/d18/f1c 971591 0 2024-09-10T04:05:09.276 INFO:tasks.workunit.client.1.smithi070.stdout:5/180: mknod d17/d19/c3c 0 2024-09-10T04:05:09.276 INFO:tasks.workunit.client.0.smithi046.stdout:5/233: write d7/d11/d14/d2f/d2b/f3a [363518,33661] 0 2024-09-10T04:05:09.280 INFO:tasks.workunit.client.1.smithi070.stdout:0/189: symlink d8/d24/d2d/l41 0 2024-09-10T04:05:09.282 INFO:tasks.workunit.client.1.smithi070.stdout:2/137: dread - d15/d18/f26 zero size 2024-09-10T04:05:09.282 INFO:tasks.workunit.client.1.smithi070.stdout:9/190: unlink d9/dd/f14 0 2024-09-10T04:05:09.282 INFO:tasks.workunit.client.1.smithi070.stdout:1/135: write d3/d8/d12/f19 [3694868,91760] 0 2024-09-10T04:05:09.284 INFO:tasks.workunit.client.0.smithi046.stdout:5/234: dwrite d7/f1b [0,4194304] 0 2024-09-10T04:05:09.285 INFO:tasks.workunit.client.1.smithi070.stdout:5/181: unlink d17/d19/f1e 0 2024-09-10T04:05:09.285 INFO:tasks.workunit.client.1.smithi070.stdout:7/155: dwrite d2/d3/dc/d13/f1a [0,4194304] 0 2024-09-10T04:05:09.287 INFO:tasks.workunit.client.1.smithi070.stdout:1/136: write d3/d8/d13/f14 [2446098,113701] 0 2024-09-10T04:05:09.288 INFO:tasks.workunit.client.1.smithi070.stdout:0/190: symlink d8/d13/d1a/l42 0 2024-09-10T04:05:09.288 INFO:tasks.workunit.client.1.smithi070.stdout:1/137: chown f1 198559125 1 2024-09-10T04:05:09.295 INFO:tasks.workunit.client.1.smithi070.stdout:5/182: creat d17/d37/f3d x:0 0 0 2024-09-10T04:05:09.301 INFO:tasks.workunit.client.1.smithi070.stdout:9/191: link d9/dd/d10/f20 d9/f36 0 2024-09-10T04:05:09.302 INFO:tasks.workunit.client.1.smithi070.stdout:0/191: dwrite d8/d13/d1a/f32 [0,4194304] 0 2024-09-10T04:05:09.303 INFO:tasks.workunit.client.0.smithi046.stdout:5/235: truncate d7/f43 2963515 0 2024-09-10T04:05:09.303 INFO:tasks.workunit.client.1.smithi070.stdout:1/138: creat d3/d10/d1d/f30 x:0 0 0 2024-09-10T04:05:09.305 INFO:tasks.workunit.client.1.smithi070.stdout:3/196: getdents db/d1c/d25/d28 0 2024-09-10T04:05:09.306 INFO:tasks.workunit.client.1.smithi070.stdout:0/192: stat d8/f12 0 2024-09-10T04:05:09.306 INFO:tasks.workunit.client.1.smithi070.stdout:9/192: symlink d9/l37 0 2024-09-10T04:05:09.306 INFO:tasks.workunit.client.0.smithi046.stdout:5/236: symlink d7/d11/l4c 0 2024-09-10T04:05:09.307 INFO:tasks.workunit.client.1.smithi070.stdout:1/139: dread - d3/fd zero size 2024-09-10T04:05:09.307 INFO:tasks.workunit.client.1.smithi070.stdout:8/215: dread d4/d16/d17/d19/d28/f30 [4194304,4194304] 0 2024-09-10T04:05:09.308 INFO:tasks.workunit.client.1.smithi070.stdout:2/138: dwrite fb [0,4194304] 0 2024-09-10T04:05:09.311 INFO:tasks.workunit.client.1.smithi070.stdout:5/183: dwrite f14 [0,4194304] 0 2024-09-10T04:05:09.312 INFO:tasks.workunit.client.1.smithi070.stdout:1/140: rmdir d3/d8/d27 39 2024-09-10T04:05:09.314 INFO:tasks.workunit.client.1.smithi070.stdout:5/184: chown d17/d24/f2f 220326 1 2024-09-10T04:05:09.314 INFO:tasks.workunit.client.1.smithi070.stdout:7/156: dread d2/f24 [0,4194304] 0 2024-09-10T04:05:09.315 INFO:tasks.workunit.client.1.smithi070.stdout:9/193: symlink d9/dd/d10/d1b/l38 0 2024-09-10T04:05:09.315 INFO:tasks.workunit.client.1.smithi070.stdout:3/197: rename db/f32 to db/d1c/d25/f39 0 2024-09-10T04:05:09.316 INFO:tasks.workunit.client.1.smithi070.stdout:5/185: rmdir d17/d19/d35 39 2024-09-10T04:05:09.320 INFO:tasks.workunit.client.1.smithi070.stdout:0/193: creat d8/d24/f43 x:0 0 0 2024-09-10T04:05:09.320 INFO:tasks.workunit.client.0.smithi046.stdout:1/111: sync 2024-09-10T04:05:09.321 INFO:tasks.workunit.client.0.smithi046.stdout:1/112: fdatasync d3/dc/f17 0 2024-09-10T04:05:09.321 INFO:tasks.workunit.client.1.smithi070.stdout:9/194: truncate d9/f36 442391 0 2024-09-10T04:05:09.324 INFO:tasks.workunit.client.1.smithi070.stdout:2/139: mkdir d15/d18/d29/d2a/d2e 0 2024-09-10T04:05:09.324 INFO:tasks.workunit.client.0.smithi046.stdout:5/237: dread d7/f43 [0,4194304] 0 2024-09-10T04:05:09.326 INFO:tasks.workunit.client.1.smithi070.stdout:2/140: chown d15/c25 0 1 2024-09-10T04:05:09.328 INFO:tasks.workunit.client.0.smithi046.stdout:5/238: symlink d7/d11/d14/d45/l4d 0 2024-09-10T04:05:09.328 INFO:tasks.workunit.client.1.smithi070.stdout:3/198: dread db/f33 [0,4194304] 0 2024-09-10T04:05:09.328 INFO:tasks.workunit.client.0.smithi046.stdout:5/239: fdatasync d7/f10 0 2024-09-10T04:05:09.329 INFO:tasks.workunit.client.1.smithi070.stdout:2/141: dread - d15/d18/d29/f2d zero size 2024-09-10T04:05:09.329 INFO:tasks.workunit.client.1.smithi070.stdout:7/157: dwrite d2/d3/dc/d13/f1a [0,4194304] 0 2024-09-10T04:05:09.330 INFO:tasks.workunit.client.1.smithi070.stdout:5/186: dwrite d17/d24/f27 [0,4194304] 0 2024-09-10T04:05:09.330 INFO:tasks.workunit.client.0.smithi046.stdout:1/113: link d3/fb d3/dc/d16/d1f/f29 0 2024-09-10T04:05:09.330 INFO:tasks.workunit.client.1.smithi070.stdout:2/142: dread - d15/d18/f26 zero size 2024-09-10T04:05:09.334 INFO:tasks.workunit.client.0.smithi046.stdout:5/240: creat d7/d11/d14/d45/f4e x:0 0 0 2024-09-10T04:05:09.335 INFO:tasks.workunit.client.1.smithi070.stdout:8/216: link d4/d16/d17/l20 d4/d16/d17/d19/d28/d43/l4c 0 2024-09-10T04:05:09.336 INFO:tasks.workunit.client.0.smithi046.stdout:1/114: mkdir d3/d27/d2a 0 2024-09-10T04:05:09.337 INFO:tasks.workunit.client.1.smithi070.stdout:5/187: dwrite d17/d24/f2d [0,4194304] 0 2024-09-10T04:05:09.338 INFO:tasks.workunit.client.0.smithi046.stdout:1/115: truncate d3/dc/f22 124570 0 2024-09-10T04:05:09.338 INFO:tasks.workunit.client.0.smithi046.stdout:1/116: chown f1 4 1 2024-09-10T04:05:09.340 INFO:tasks.workunit.client.0.smithi046.stdout:1/117: write d3/dc/f17 [4056594,49702] 0 2024-09-10T04:05:09.341 INFO:tasks.workunit.client.0.smithi046.stdout:1/118: dread - d3/dc/f23 zero size 2024-09-10T04:05:09.341 INFO:tasks.workunit.client.0.smithi046.stdout:1/119: chown d3 446340721 1 2024-09-10T04:05:09.342 INFO:tasks.workunit.client.0.smithi046.stdout:1/120: dread - d3/d27/f20 zero size 2024-09-10T04:05:09.344 INFO:tasks.workunit.client.1.smithi070.stdout:7/158: creat d2/d7/d9/d18/d23/f38 x:0 0 0 2024-09-10T04:05:09.347 INFO:tasks.workunit.client.0.smithi046.stdout:5/241: creat d7/f4f x:0 0 0 2024-09-10T04:05:09.350 INFO:tasks.workunit.client.1.smithi070.stdout:1/141: getdents d3 0 2024-09-10T04:05:09.351 INFO:tasks.workunit.client.1.smithi070.stdout:1/142: fsync d3/d10/d1d/f30 0 2024-09-10T04:05:09.352 INFO:tasks.workunit.client.0.smithi046.stdout:5/242: dread d7/d11/d14/d2f/f22 [0,4194304] 0 2024-09-10T04:05:09.352 INFO:tasks.workunit.client.0.smithi046.stdout:5/243: readlink d7/d11/d14/d45/l4d 0 2024-09-10T04:05:09.353 INFO:tasks.workunit.client.0.smithi046.stdout:5/244: write d7/d11/d14/f1d [772010,87425] 0 2024-09-10T04:05:09.356 INFO:tasks.workunit.client.1.smithi070.stdout:5/188: symlink d17/d24/d2b/d31/l3e 0 2024-09-10T04:05:09.356 INFO:tasks.workunit.client.1.smithi070.stdout:3/199: dwrite db/fd [0,4194304] 0 2024-09-10T04:05:09.357 INFO:tasks.workunit.client.1.smithi070.stdout:2/143: unlink c13 0 2024-09-10T04:05:09.357 INFO:tasks.workunit.client.0.smithi046.stdout:1/121: truncate d3/dc/f17 4153908 0 2024-09-10T04:05:09.360 INFO:tasks.workunit.client.0.smithi046.stdout:5/245: mkdir d7/d11/d14/d50 0 2024-09-10T04:05:09.360 INFO:tasks.workunit.client.1.smithi070.stdout:7/159: write d2/f24 [1022447,17467] 0 2024-09-10T04:05:09.360 INFO:tasks.workunit.client.1.smithi070.stdout:1/143: dwrite d3/fd [0,4194304] 0 2024-09-10T04:05:09.364 INFO:tasks.workunit.client.1.smithi070.stdout:8/217: link d4/fe d4/d16/d17/d19/d28/d43/f4d 0 2024-09-10T04:05:09.364 INFO:tasks.workunit.client.0.smithi046.stdout:5/246: chown d7/d11/d14/d2f/d2b/l46 2922 1 2024-09-10T04:05:09.365 INFO:tasks.workunit.client.1.smithi070.stdout:3/200: rmdir db 39 2024-09-10T04:05:09.366 INFO:tasks.workunit.client.1.smithi070.stdout:5/189: mkdir d17/d24/d2b/d3f 0 2024-09-10T04:05:09.367 INFO:tasks.workunit.client.1.smithi070.stdout:5/190: read d17/d24/f25 [36446,88267] 0 2024-09-10T04:05:09.367 INFO:tasks.workunit.client.1.smithi070.stdout:9/195: getdents d9/dd/d2a 0 2024-09-10T04:05:09.368 INFO:tasks.workunit.client.1.smithi070.stdout:9/196: read - d9/dd/f30 zero size 2024-09-10T04:05:09.369 INFO:tasks.workunit.client.1.smithi070.stdout:9/197: write d9/dd/d10/f19 [522174,91969] 0 2024-09-10T04:05:09.370 INFO:tasks.workunit.client.0.smithi046.stdout:5/247: dread d7/d11/d14/f25 [0,4194304] 0 2024-09-10T04:05:09.373 INFO:tasks.workunit.client.1.smithi070.stdout:8/218: chown d4/f13 5 1 2024-09-10T04:05:09.375 INFO:tasks.workunit.client.0.smithi046.stdout:5/248: mkdir d7/d11/d14/d45/d51 0 2024-09-10T04:05:09.376 INFO:tasks.workunit.client.0.smithi046.stdout:5/249: truncate d7/fe 4682351 0 2024-09-10T04:05:09.378 INFO:tasks.workunit.client.1.smithi070.stdout:2/144: unlink l7 0 2024-09-10T04:05:09.379 INFO:tasks.workunit.client.0.smithi046.stdout:1/122: dread d3/dc/f10 [0,4194304] 0 2024-09-10T04:05:09.381 INFO:tasks.workunit.client.1.smithi070.stdout:5/191: symlink d17/d24/d2b/l40 0 2024-09-10T04:05:09.382 INFO:tasks.workunit.client.1.smithi070.stdout:5/192: read f7 [229934,59678] 0 2024-09-10T04:05:09.383 INFO:tasks.workunit.client.1.smithi070.stdout:5/193: rename d17 to d17/d24/d2b/d31/d41 22 2024-09-10T04:05:09.385 INFO:tasks.workunit.client.0.smithi046.stdout:1/123: link d3/d27/f20 d3/dc/d16/f2b 0 2024-09-10T04:05:09.385 INFO:tasks.workunit.client.1.smithi070.stdout:9/198: mkdir d9/d1c/d39 0 2024-09-10T04:05:09.386 INFO:tasks.workunit.client.0.smithi046.stdout:5/250: dwrite d7/d11/d14/d45/f32 [0,4194304] 0 2024-09-10T04:05:09.386 INFO:tasks.workunit.client.0.smithi046.stdout:1/124: write d3/dc/f23 [86575,6638] 0 2024-09-10T04:05:09.390 INFO:tasks.workunit.client.1.smithi070.stdout:8/219: symlink d4/d16/d17/d19/d28/d43/l4e 0 2024-09-10T04:05:09.392 INFO:tasks.workunit.client.1.smithi070.stdout:1/144: symlink d3/d8/l31 0 2024-09-10T04:05:09.393 INFO:tasks.workunit.client.1.smithi070.stdout:1/145: chown d3/d8/d13/f17 10050 1 2024-09-10T04:05:09.394 INFO:tasks.workunit.client.0.smithi046.stdout:5/251: dwrite d7/d11/f28 [0,4194304] 0 2024-09-10T04:05:09.395 INFO:tasks.workunit.client.1.smithi070.stdout:2/145: symlink d15/d18/l2f 0 2024-09-10T04:05:09.395 INFO:tasks.workunit.client.0.smithi046.stdout:5/252: fdatasync d7/fe 0 2024-09-10T04:05:09.398 INFO:tasks.workunit.client.1.smithi070.stdout:3/201: symlink db/d1c/d25/l3a 0 2024-09-10T04:05:09.398 INFO:tasks.workunit.client.1.smithi070.stdout:3/202: read db/f14 [1649310,31098] 0 2024-09-10T04:05:09.400 INFO:tasks.workunit.client.1.smithi070.stdout:5/194: rmdir d17/d24 39 2024-09-10T04:05:09.402 INFO:tasks.workunit.client.1.smithi070.stdout:7/160: rmdir d2/d3/dc/d25 0 2024-09-10T04:05:09.403 INFO:tasks.workunit.client.0.smithi046.stdout:5/253: mknod d7/c52 0 2024-09-10T04:05:09.405 INFO:tasks.workunit.client.0.smithi046.stdout:5/254: rmdir d7/d11/d14/d45 39 2024-09-10T04:05:09.411 INFO:tasks.workunit.client.0.smithi046.stdout:5/255: rename d7/d11/d14/d2f/d2b/d41 to d7/d11/d3f/d53 0 2024-09-10T04:05:09.412 INFO:tasks.workunit.client.1.smithi070.stdout:5/195: dread - d17/d24/f2c zero size 2024-09-10T04:05:09.413 INFO:tasks.workunit.client.1.smithi070.stdout:3/203: dwrite f7 [0,4194304] 0 2024-09-10T04:05:09.413 INFO:tasks.workunit.client.1.smithi070.stdout:9/199: creat d9/d1c/d39/f3a x:0 0 0 2024-09-10T04:05:09.414 INFO:tasks.workunit.client.1.smithi070.stdout:5/196: truncate d17/d19/f2e 204951 0 2024-09-10T04:05:09.414 INFO:tasks.workunit.client.1.smithi070.stdout:9/200: dread - d9/dd/d2a/f2e zero size 2024-09-10T04:05:09.415 INFO:tasks.workunit.client.1.smithi070.stdout:5/197: read f14 [1315190,39748] 0 2024-09-10T04:05:09.416 INFO:tasks.workunit.client.1.smithi070.stdout:7/161: mkdir d2/d7/db/d39 0 2024-09-10T04:05:09.417 INFO:tasks.workunit.client.1.smithi070.stdout:8/220: symlink d4/d16/d17/l4f 0 2024-09-10T04:05:09.418 INFO:tasks.workunit.client.1.smithi070.stdout:7/162: dread - d2/d7/d9/d18/d23/f38 zero size 2024-09-10T04:05:09.419 INFO:tasks.workunit.client.1.smithi070.stdout:1/146: mkdir d3/d8/d12/d2f/d32 0 2024-09-10T04:05:09.422 INFO:tasks.workunit.client.1.smithi070.stdout:3/204: creat db/f3b x:0 0 0 2024-09-10T04:05:09.424 INFO:tasks.workunit.client.1.smithi070.stdout:9/201: symlink d9/d1c/d39/l3b 0 2024-09-10T04:05:09.424 INFO:tasks.workunit.client.1.smithi070.stdout:9/202: dread - d9/f22 zero size 2024-09-10T04:05:09.426 INFO:tasks.workunit.client.1.smithi070.stdout:7/163: chown d2/d7/la 91599 1 2024-09-10T04:05:09.427 INFO:tasks.workunit.client.1.smithi070.stdout:9/203: truncate d9/dd/f30 891792 0 2024-09-10T04:05:09.427 INFO:tasks.workunit.client.1.smithi070.stdout:2/146: read d15/d18/f24 [1539931,80245] 0 2024-09-10T04:05:09.427 INFO:tasks.workunit.client.0.smithi046.stdout:5/256: dwrite d7/d11/f23 [0,4194304] 0 2024-09-10T04:05:09.428 INFO:tasks.workunit.client.1.smithi070.stdout:4/154: sync 2024-09-10T04:05:09.428 INFO:tasks.workunit.client.1.smithi070.stdout:6/250: sync 2024-09-10T04:05:09.428 INFO:tasks.workunit.client.1.smithi070.stdout:0/194: sync 2024-09-10T04:05:09.431 INFO:tasks.workunit.client.1.smithi070.stdout:4/155: truncate de/f31 154654 0 2024-09-10T04:05:09.431 INFO:tasks.workunit.client.0.smithi046.stdout:5/257: symlink d7/d11/d14/l54 0 2024-09-10T04:05:09.432 INFO:tasks.workunit.client.1.smithi070.stdout:9/204: mkdir d9/dd/d10/d3c 0 2024-09-10T04:05:09.433 INFO:tasks.workunit.client.1.smithi070.stdout:5/198: dread d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:09.433 INFO:tasks.workunit.client.0.smithi046.stdout:5/258: read d7/fe [4294933,114967] 0 2024-09-10T04:05:09.433 INFO:tasks.workunit.client.1.smithi070.stdout:9/205: readlink d9/la 0 2024-09-10T04:05:09.435 INFO:tasks.workunit.client.1.smithi070.stdout:3/205: dwrite db/d1c/d25/f39 [0,4194304] 0 2024-09-10T04:05:09.437 INFO:tasks.workunit.client.1.smithi070.stdout:3/206: write fa [973055,5408] 0 2024-09-10T04:05:09.438 INFO:tasks.workunit.client.1.smithi070.stdout:3/207: truncate db/d1c/d25/d28/f37 552433 0 2024-09-10T04:05:09.438 INFO:tasks.workunit.client.0.smithi046.stdout:5/259: mknod d7/d11/d14/d45/c55 0 2024-09-10T04:05:09.440 INFO:tasks.workunit.client.0.smithi046.stdout:5/260: fdatasync d7/d11/d14/d45/f32 0 2024-09-10T04:05:09.441 INFO:tasks.workunit.client.1.smithi070.stdout:5/199: symlink d17/d24/d2b/l42 0 2024-09-10T04:05:09.442 INFO:tasks.workunit.client.1.smithi070.stdout:5/200: dread - d17/f36 zero size 2024-09-10T04:05:09.443 INFO:tasks.workunit.client.1.smithi070.stdout:6/251: mknod d0/c50 0 2024-09-10T04:05:09.445 INFO:tasks.workunit.client.0.smithi046.stdout:5/261: dread d7/fd [0,4194304] 0 2024-09-10T04:05:09.446 INFO:tasks.workunit.client.1.smithi070.stdout:9/206: unlink d9/dd/c26 0 2024-09-10T04:05:09.447 INFO:tasks.workunit.client.1.smithi070.stdout:8/221: getdents d4/d16/d17/d1a/d23 0 2024-09-10T04:05:09.448 INFO:tasks.workunit.client.1.smithi070.stdout:0/195: dwrite d8/f1b [0,4194304] 0 2024-09-10T04:05:09.448 INFO:tasks.workunit.client.1.smithi070.stdout:0/196: readlink d8/l15 0 2024-09-10T04:05:09.449 INFO:tasks.workunit.client.1.smithi070.stdout:1/147: rename d3/d8/d27/c2a to d3/d8/c33 0 2024-09-10T04:05:09.450 INFO:tasks.workunit.client.0.smithi046.stdout:7/162: sync 2024-09-10T04:05:09.450 INFO:tasks.workunit.client.0.smithi046.stdout:5/262: mknod d7/d11/d14/d50/c56 0 2024-09-10T04:05:09.451 INFO:tasks.workunit.client.0.smithi046.stdout:7/163: mknod d5/d9/c3f 0 2024-09-10T04:05:09.452 INFO:tasks.workunit.client.0.smithi046.stdout:7/164: stat d5/d6/db/d11 0 2024-09-10T04:05:09.453 INFO:tasks.workunit.client.1.smithi070.stdout:5/201: mknod d17/d24/d2b/d31/d32/c43 0 2024-09-10T04:05:09.454 INFO:tasks.workunit.client.0.smithi046.stdout:5/263: rename d7/d11/d14/d45/f32 to d7/d11/d3f/d53/f57 0 2024-09-10T04:05:09.454 INFO:tasks.workunit.client.1.smithi070.stdout:7/164: dread d2/d7/fe [0,4194304] 0 2024-09-10T04:05:09.457 INFO:tasks.workunit.client.1.smithi070.stdout:0/197: dwrite d8/d24/f2a [0,4194304] 0 2024-09-10T04:05:09.458 INFO:tasks.workunit.client.1.smithi070.stdout:7/165: truncate d2/d7/d9/d18/d23/f38 488872 0 2024-09-10T04:05:09.458 INFO:tasks.workunit.client.1.smithi070.stdout:0/198: write f5 [648925,97076] 0 2024-09-10T04:05:09.460 INFO:tasks.workunit.client.1.smithi070.stdout:9/207: unlink d9/dd/d2a/f2e 0 2024-09-10T04:05:09.462 INFO:tasks.workunit.client.0.smithi046.stdout:5/264: dwrite d7/d11/d14/f1d [0,4194304] 0 2024-09-10T04:05:09.463 INFO:tasks.workunit.client.0.smithi046.stdout:5/265: readlink d7/d11/d3f/d53/l4a 0 2024-09-10T04:05:09.468 INFO:tasks.workunit.client.1.smithi070.stdout:7/166: dread d2/f15 [0,4194304] 0 2024-09-10T04:05:09.468 INFO:tasks.workunit.client.1.smithi070.stdout:2/147: rename d15/d18/l1e to d15/d18/d29/d2a/d2e/l30 0 2024-09-10T04:05:09.468 INFO:tasks.workunit.client.1.smithi070.stdout:1/148: stat d3/l4 0 2024-09-10T04:05:09.470 INFO:tasks.workunit.client.1.smithi070.stdout:4/156: creat de/f34 x:0 0 0 2024-09-10T04:05:09.472 INFO:tasks.workunit.client.0.smithi046.stdout:7/165: getdents d5/d6/db/d11/d1f 0 2024-09-10T04:05:09.473 INFO:tasks.workunit.client.0.smithi046.stdout:5/266: dwrite d7/d11/d14/f29 [0,4194304] 0 2024-09-10T04:05:09.474 INFO:tasks.workunit.client.1.smithi070.stdout:3/208: dwrite db/f11 [0,4194304] 0 2024-09-10T04:05:09.474 INFO:tasks.workunit.client.1.smithi070.stdout:0/199: dwrite d8/d24/d2d/f3a [0,4194304] 0 2024-09-10T04:05:09.475 INFO:tasks.workunit.client.1.smithi070.stdout:3/209: chown db/d16/f30 32158376 1 2024-09-10T04:05:09.475 INFO:tasks.workunit.client.1.smithi070.stdout:0/200: dread - d8/f12 zero size 2024-09-10T04:05:09.485 INFO:tasks.workunit.client.1.smithi070.stdout:8/222: rename d4/d16/d17/d19/d28/d43/f4d to d4/d16/d17/d1a/d23/f50 0 2024-09-10T04:05:09.485 INFO:tasks.workunit.client.1.smithi070.stdout:8/223: chown l2 6 1 2024-09-10T04:05:09.485 INFO:tasks.workunit.client.0.smithi046.stdout:5/267: dread d7/d11/d14/f36 [0,4194304] 0 2024-09-10T04:05:09.486 INFO:tasks.workunit.client.1.smithi070.stdout:1/149: rmdir d3/d10/d1d 39 2024-09-10T04:05:09.487 INFO:tasks.workunit.client.0.smithi046.stdout:5/268: dread - d7/d11/d3f/d53/f47 zero size 2024-09-10T04:05:09.488 INFO:tasks.workunit.client.1.smithi070.stdout:6/252: creat d0/d3/f51 x:0 0 0 2024-09-10T04:05:09.489 INFO:tasks.workunit.client.0.smithi046.stdout:5/269: write d7/d11/d14/d2f/d2b/f3a [1086006,58483] 0 2024-09-10T04:05:09.489 INFO:tasks.workunit.client.1.smithi070.stdout:6/253: fsync d0/d3/d18/d1b/f14 0 2024-09-10T04:05:09.491 INFO:tasks.workunit.client.1.smithi070.stdout:3/210: rename db/fd to db/f3c 0 2024-09-10T04:05:09.491 INFO:tasks.workunit.client.1.smithi070.stdout:0/201: mknod d8/d13/d1a/c44 0 2024-09-10T04:05:09.492 INFO:tasks.workunit.client.1.smithi070.stdout:2/148: mknod d15/c31 0 2024-09-10T04:05:09.494 INFO:tasks.workunit.client.1.smithi070.stdout:9/208: link d9/dd/d10/f20 d9/dd/d10/f3d 0 2024-09-10T04:05:09.494 INFO:tasks.workunit.client.0.smithi046.stdout:7/166: write d5/d6/db/f1d [618369,98143] 0 2024-09-10T04:05:09.496 INFO:tasks.workunit.client.0.smithi046.stdout:5/270: fsync d7/d11/d14/f1d 0 2024-09-10T04:05:09.497 INFO:tasks.workunit.client.1.smithi070.stdout:4/157: unlink de/c12 0 2024-09-10T04:05:09.497 INFO:tasks.workunit.client.0.smithi046.stdout:5/271: chown d7/d11/d14/d2f/l39 2028 1 2024-09-10T04:05:09.498 INFO:tasks.workunit.client.1.smithi070.stdout:5/202: creat d17/d19/f44 x:0 0 0 2024-09-10T04:05:09.498 INFO:tasks.workunit.client.1.smithi070.stdout:7/167: write f1 [7877346,88085] 0 2024-09-10T04:05:09.498 INFO:tasks.workunit.client.1.smithi070.stdout:3/211: dwrite db/f1f [0,4194304] 0 2024-09-10T04:05:09.502 INFO:tasks.workunit.client.1.smithi070.stdout:6/254: rmdir d0/d3/d18/d1b/d17/d3f/d40 39 2024-09-10T04:05:09.504 INFO:tasks.workunit.client.1.smithi070.stdout:6/255: readlink d0/d3/d18/d1b/d17/l20 0 2024-09-10T04:05:09.505 INFO:tasks.workunit.client.0.smithi046.stdout:7/167: link d5/c21 d5/d6/d32/c40 0 2024-09-10T04:05:09.505 INFO:tasks.workunit.client.1.smithi070.stdout:0/202: creat d8/d13/d1a/f45 x:0 0 0 2024-09-10T04:05:09.507 INFO:tasks.workunit.client.1.smithi070.stdout:7/168: dwrite d2/d3/dc/d1b/f1d [4194304,4194304] 0 2024-09-10T04:05:09.508 INFO:tasks.workunit.client.1.smithi070.stdout:2/149: write d15/d18/f26 [852186,103170] 0 2024-09-10T04:05:09.508 INFO:tasks.workunit.client.0.smithi046.stdout:5/272: rename d7/d11/d14/d2f/d2b/c42 to d7/d11/d14/d2f/c58 0 2024-09-10T04:05:09.509 INFO:tasks.workunit.client.1.smithi070.stdout:7/169: truncate d2/d7/db/f37 1036260 0 2024-09-10T04:05:09.510 INFO:tasks.workunit.client.1.smithi070.stdout:1/150: write d3/d8/fc [3736220,25250] 0 2024-09-10T04:05:09.511 INFO:tasks.workunit.client.1.smithi070.stdout:1/151: write d3/d8/d13/f1b [240614,97355] 0 2024-09-10T04:05:09.512 INFO:tasks.workunit.client.0.smithi046.stdout:7/168: mknod d5/d6/db/d11/d1e/c41 0 2024-09-10T04:05:09.512 INFO:tasks.workunit.client.0.smithi046.stdout:7/169: chown d5 246587 1 2024-09-10T04:05:09.514 INFO:tasks.workunit.client.0.smithi046.stdout:7/170: write d5/f2c [1649116,62193] 0 2024-09-10T04:05:09.515 INFO:tasks.workunit.client.0.smithi046.stdout:7/171: stat d5/d6/d23/f37 0 2024-09-10T04:05:09.516 INFO:tasks.workunit.client.1.smithi070.stdout:1/152: dread d3/d8/d13/f1b [0,4194304] 0 2024-09-10T04:05:09.517 INFO:tasks.workunit.client.1.smithi070.stdout:1/153: write f1 [323979,68165] 0 2024-09-10T04:05:09.517 INFO:tasks.workunit.client.1.smithi070.stdout:1/154: chown d3/d8/c2e 21893849 1 2024-09-10T04:05:09.518 INFO:tasks.workunit.client.1.smithi070.stdout:8/224: truncate d4/d16/d17/d1a/f2e 2343141 0 2024-09-10T04:05:09.521 INFO:tasks.workunit.client.0.smithi046.stdout:7/172: creat d5/d9/f42 x:0 0 0 2024-09-10T04:05:09.523 INFO:tasks.workunit.client.0.smithi046.stdout:5/273: creat d7/d11/d14/f59 x:0 0 0 2024-09-10T04:05:09.524 INFO:tasks.workunit.client.1.smithi070.stdout:3/212: creat db/d1c/d25/f3d x:0 0 0 2024-09-10T04:05:09.524 INFO:tasks.workunit.client.1.smithi070.stdout:3/213: write f3 [1387367,95527] 0 2024-09-10T04:05:09.525 INFO:tasks.workunit.client.1.smithi070.stdout:5/203: dread f14 [0,4194304] 0 2024-09-10T04:05:09.526 INFO:tasks.workunit.client.0.smithi046.stdout:5/274: dread d7/f19 [0,4194304] 0 2024-09-10T04:05:09.527 INFO:tasks.workunit.client.1.smithi070.stdout:6/256: creat d0/d3/d18/f52 x:0 0 0 2024-09-10T04:05:09.528 INFO:tasks.workunit.client.0.smithi046.stdout:5/275: truncate d7/d11/d14/d2f/f22 4203059 0 2024-09-10T04:05:09.529 INFO:tasks.workunit.client.0.smithi046.stdout:5/276: readlink d7/la 0 2024-09-10T04:05:09.529 INFO:tasks.workunit.client.1.smithi070.stdout:2/150: mkdir d15/d18/d29/d2a/d2e/d32 0 2024-09-10T04:05:09.530 INFO:tasks.workunit.client.1.smithi070.stdout:2/151: read d15/d18/f19 [3994631,14886] 0 2024-09-10T04:05:09.531 INFO:tasks.workunit.client.1.smithi070.stdout:2/152: read f10 [209683,18734] 0 2024-09-10T04:05:09.531 INFO:tasks.workunit.client.1.smithi070.stdout:7/170: rename d2/d7/db/d39 to d2/d7/db/d3a 0 2024-09-10T04:05:09.536 INFO:tasks.workunit.client.0.smithi046.stdout:5/277: dwrite d7/d11/f23 [0,4194304] 0 2024-09-10T04:05:09.538 INFO:tasks.workunit.client.0.smithi046.stdout:7/173: truncate d5/d6/db/f2a 3087381 0 2024-09-10T04:05:09.540 INFO:tasks.workunit.client.0.smithi046.stdout:7/174: chown d5/d6/d23/f37 23917 1 2024-09-10T04:05:09.545 INFO:tasks.workunit.client.1.smithi070.stdout:3/214: rename db/d16 to db/d1c/d3e 0 2024-09-10T04:05:09.546 INFO:tasks.workunit.client.0.smithi046.stdout:5/278: dwrite d7/fe [4194304,4194304] 0 2024-09-10T04:05:09.548 INFO:tasks.workunit.client.1.smithi070.stdout:6/257: mkdir d0/d3/d18/d1b/d17/d53 0 2024-09-10T04:05:09.548 INFO:tasks.workunit.client.1.smithi070.stdout:6/258: readlink d0/d3/l2c 0 2024-09-10T04:05:09.549 INFO:tasks.workunit.client.1.smithi070.stdout:6/259: chown d0/d3/d18/d1b/d17/c3e 10 1 2024-09-10T04:05:09.550 INFO:tasks.workunit.client.1.smithi070.stdout:0/203: link d8/fa d8/d24/d36/f46 0 2024-09-10T04:05:09.550 INFO:tasks.workunit.client.0.smithi046.stdout:7/175: dwrite d5/d9/f42 [0,4194304] 0 2024-09-10T04:05:09.550 INFO:tasks.workunit.client.1.smithi070.stdout:6/260: write d0/d3/d18/d1b/d17/f47 [543051,72730] 0 2024-09-10T04:05:09.551 INFO:tasks.workunit.client.1.smithi070.stdout:6/261: chown d0 494319 1 2024-09-10T04:05:09.551 INFO:tasks.workunit.client.0.smithi046.stdout:7/176: truncate d5/fc 1183618 0 2024-09-10T04:05:09.555 INFO:tasks.workunit.client.0.smithi046.stdout:5/279: mkdir d7/d11/d3f/d5a 0 2024-09-10T04:05:09.557 INFO:tasks.workunit.client.0.smithi046.stdout:5/280: dread - d7/d11/d3f/d53/f47 zero size 2024-09-10T04:05:09.557 INFO:tasks.workunit.client.0.smithi046.stdout:5/281: stat d7/f4f 0 2024-09-10T04:05:09.558 INFO:tasks.workunit.client.0.smithi046.stdout:7/177: mkdir d5/d9/d43 0 2024-09-10T04:05:09.559 INFO:tasks.workunit.client.0.smithi046.stdout:7/178: chown d5/d6/db/dd/d24/f31 166199317 1 2024-09-10T04:05:09.562 INFO:tasks.workunit.client.0.smithi046.stdout:7/179: dread d5/d6/db/f1d [0,4194304] 0 2024-09-10T04:05:09.563 INFO:tasks.workunit.client.0.smithi046.stdout:7/180: fsync d5/d6/f1c 0 2024-09-10T04:05:09.563 INFO:tasks.workunit.client.1.smithi070.stdout:5/204: dwrite f7 [0,4194304] 0 2024-09-10T04:05:09.564 INFO:tasks.workunit.client.1.smithi070.stdout:5/205: write fc [210653,25068] 0 2024-09-10T04:05:09.565 INFO:tasks.workunit.client.1.smithi070.stdout:5/206: write f10 [4043398,91092] 0 2024-09-10T04:05:09.565 INFO:tasks.workunit.client.0.smithi046.stdout:5/282: mknod d7/d11/d14/d45/d51/c5b 0 2024-09-10T04:05:09.567 INFO:tasks.workunit.client.0.smithi046.stdout:5/283: stat d7/d11/d14/d2f/d2b/f3a 0 2024-09-10T04:05:09.572 INFO:tasks.workunit.client.1.smithi070.stdout:8/225: mknod d4/d16/d17/d1a/d23/d44/c51 0 2024-09-10T04:05:09.572 INFO:tasks.workunit.client.1.smithi070.stdout:7/171: write d2/f15 [1813172,17347] 0 2024-09-10T04:05:09.575 INFO:tasks.workunit.client.0.smithi046.stdout:7/181: rmdir d5/d6/db/dd/d3c 0 2024-09-10T04:05:09.577 INFO:tasks.workunit.client.1.smithi070.stdout:6/262: mkdir d0/d3/d18/d1b/d17/d54 0 2024-09-10T04:05:09.578 INFO:tasks.workunit.client.1.smithi070.stdout:2/153: creat d15/d18/d29/d2a/d2e/d32/f33 x:0 0 0 2024-09-10T04:05:09.578 INFO:tasks.workunit.client.1.smithi070.stdout:1/155: dwrite f0 [0,4194304] 0 2024-09-10T04:05:09.579 INFO:tasks.workunit.client.0.smithi046.stdout:7/182: creat d5/d6/db/dd/f44 x:0 0 0 2024-09-10T04:05:09.579 INFO:tasks.workunit.client.1.smithi070.stdout:6/263: read - d0/d3/d18/d2a/f4f zero size 2024-09-10T04:05:09.579 INFO:tasks.workunit.client.1.smithi070.stdout:1/156: fdatasync f1 0 2024-09-10T04:05:09.580 INFO:tasks.workunit.client.0.smithi046.stdout:7/183: chown d5/d9/d12 2 1 2024-09-10T04:05:09.580 INFO:tasks.workunit.client.0.smithi046.stdout:7/184: chown d5/d9 82 1 2024-09-10T04:05:09.580 INFO:tasks.workunit.client.1.smithi070.stdout:9/209: getdents d9/d1c 0 2024-09-10T04:05:09.580 INFO:tasks.workunit.client.0.smithi046.stdout:7/185: readlink d5/d6/db/d11/d1e/d20/l2b 0 2024-09-10T04:05:09.581 INFO:tasks.workunit.client.1.smithi070.stdout:9/210: dread - d9/d1c/d34/f32 zero size 2024-09-10T04:05:09.581 INFO:tasks.workunit.client.0.smithi046.stdout:7/186: dread - d5/d6/db/d11/d1e/f30 zero size 2024-09-10T04:05:09.582 INFO:tasks.workunit.client.1.smithi070.stdout:5/207: rename d17/d24/d2b/d3f to d17/d24/d45 0 2024-09-10T04:05:09.583 INFO:tasks.workunit.client.1.smithi070.stdout:8/226: mkdir d4/d16/d17/d19/d28/d43/d52 0 2024-09-10T04:05:09.583 INFO:tasks.workunit.client.1.smithi070.stdout:9/211: chown d9/dd/d10 14337406 1 2024-09-10T04:05:09.583 INFO:tasks.workunit.client.1.smithi070.stdout:8/227: stat d4/cc 0 2024-09-10T04:05:09.584 INFO:tasks.workunit.client.1.smithi070.stdout:4/158: getdents de 0 2024-09-10T04:05:09.584 INFO:tasks.workunit.client.1.smithi070.stdout:7/172: creat d2/d3/f3b x:0 0 0 2024-09-10T04:05:09.586 INFO:tasks.workunit.client.1.smithi070.stdout:3/215: link f7 db/f3f 0 2024-09-10T04:05:09.586 INFO:tasks.workunit.client.0.smithi046.stdout:7/187: rename d5/d6/db/d11/d1f/f27 to d5/d33/f45 0 2024-09-10T04:05:09.587 INFO:tasks.workunit.client.1.smithi070.stdout:8/228: write d4/d16/f41 [864270,1376] 0 2024-09-10T04:05:09.588 INFO:tasks.workunit.client.1.smithi070.stdout:1/157: dread d3/d8/f2d [0,4194304] 0 2024-09-10T04:05:09.588 INFO:tasks.workunit.client.0.smithi046.stdout:5/284: dread d7/f10 [0,4194304] 0 2024-09-10T04:05:09.589 INFO:tasks.workunit.client.1.smithi070.stdout:0/204: mkdir d8/d13/d47 0 2024-09-10T04:05:09.590 INFO:tasks.workunit.client.0.smithi046.stdout:7/188: symlink d5/d33/l46 0 2024-09-10T04:05:09.591 INFO:tasks.workunit.client.1.smithi070.stdout:4/159: dread fb [0,4194304] 0 2024-09-10T04:05:09.591 INFO:tasks.workunit.client.1.smithi070.stdout:0/205: write f5 [1136296,108803] 0 2024-09-10T04:05:09.592 INFO:tasks.workunit.client.1.smithi070.stdout:6/264: fdatasync d0/f8 0 2024-09-10T04:05:09.593 INFO:tasks.workunit.client.0.smithi046.stdout:5/285: dread d7/f1b [0,4194304] 0 2024-09-10T04:05:09.593 INFO:tasks.workunit.client.0.smithi046.stdout:5/286: fdatasync d7/f4f 0 2024-09-10T04:05:09.594 INFO:tasks.workunit.client.1.smithi070.stdout:2/154: symlink d15/d18/d29/d2a/d2e/l34 0 2024-09-10T04:05:09.594 INFO:tasks.workunit.client.1.smithi070.stdout:5/208: dread d17/d24/f2a [0,4194304] 0 2024-09-10T04:05:09.595 INFO:tasks.workunit.client.0.smithi046.stdout:7/189: symlink d5/d6/db/d11/d1e/d20/l47 0 2024-09-10T04:05:09.595 INFO:tasks.workunit.client.0.smithi046.stdout:7/190: stat d5/f7 0 2024-09-10T04:05:09.595 INFO:tasks.workunit.client.1.smithi070.stdout:2/155: chown d15/d18/f1a 4017686 1 2024-09-10T04:05:09.596 INFO:tasks.workunit.client.1.smithi070.stdout:9/212: dwrite d9/dd/f18 [0,4194304] 0 2024-09-10T04:05:09.601 INFO:tasks.workunit.client.1.smithi070.stdout:2/156: dread d15/d18/f28 [0,4194304] 0 2024-09-10T04:05:09.603 INFO:tasks.workunit.client.1.smithi070.stdout:9/213: dwrite d9/d1c/d34/f32 [0,4194304] 0 2024-09-10T04:05:09.631 INFO:tasks.workunit.client.0.smithi046.stdout:0/181: sync 2024-09-10T04:05:09.631 INFO:tasks.workunit.client.0.smithi046.stdout:8/201: sync 2024-09-10T04:05:09.632 INFO:tasks.workunit.client.0.smithi046.stdout:5/287: symlink d7/d11/d14/d2f/d2b/l5c 0 2024-09-10T04:05:09.633 INFO:tasks.workunit.client.0.smithi046.stdout:8/202: fdatasync d1/d29/d17/f2f 0 2024-09-10T04:05:09.635 INFO:tasks.workunit.client.0.smithi046.stdout:7/191: creat d5/d6/db/d11/d1f/f48 x:0 0 0 2024-09-10T04:05:09.640 INFO:tasks.workunit.client.0.smithi046.stdout:5/288: dwrite d7/d11/f28 [0,4194304] 0 2024-09-10T04:05:09.640 INFO:tasks.workunit.client.0.smithi046.stdout:8/203: dread d1/d29/f27 [0,4194304] 0 2024-09-10T04:05:09.651 INFO:tasks.workunit.client.0.smithi046.stdout:5/289: dwrite d7/d11/d14/f1d [0,4194304] 0 2024-09-10T04:05:09.670 INFO:tasks.workunit.client.0.smithi046.stdout:0/182: creat d1/dc/d25/d33/d3/dd/f39 x:0 0 0 2024-09-10T04:05:09.671 INFO:tasks.workunit.client.0.smithi046.stdout:0/183: dread - d1/dc/d25/d33/d3/d29/f38 zero size 2024-09-10T04:05:09.673 INFO:tasks.workunit.client.0.smithi046.stdout:0/184: write d1/dc/f21 [4367757,113313] 0 2024-09-10T04:05:09.677 INFO:tasks.workunit.client.1.smithi070.stdout:1/158: creat d3/d8/d12/d2f/f34 x:0 0 0 2024-09-10T04:05:09.684 INFO:tasks.workunit.client.0.smithi046.stdout:7/192: dread d5/d6/db/f2a [0,4194304] 0 2024-09-10T04:05:09.688 INFO:tasks.workunit.client.1.smithi070.stdout:0/206: creat d8/d24/d36/f48 x:0 0 0 2024-09-10T04:05:09.689 INFO:tasks.workunit.client.0.smithi046.stdout:5/290: symlink d7/l5d 0 2024-09-10T04:05:09.690 INFO:tasks.workunit.client.1.smithi070.stdout:6/265: rmdir d0/d3/d18/d1b 39 2024-09-10T04:05:09.690 INFO:tasks.workunit.client.0.smithi046.stdout:5/291: stat d7/d11/d14/d2f/l39 0 2024-09-10T04:05:09.692 INFO:tasks.workunit.client.0.smithi046.stdout:7/193: mknod d5/d6/d39/c49 0 2024-09-10T04:05:09.693 INFO:tasks.workunit.client.0.smithi046.stdout:7/194: fdatasync d5/d6/db/d11/d1e/f30 0 2024-09-10T04:05:09.694 INFO:tasks.workunit.client.0.smithi046.stdout:5/292: creat d7/d11/d14/d2f/f5e x:0 0 0 2024-09-10T04:05:09.694 INFO:tasks.workunit.client.0.smithi046.stdout:7/195: readlink d5/d6/db/d11/d1e/d20/l2b 0 2024-09-10T04:05:09.700 INFO:tasks.workunit.client.0.smithi046.stdout:7/196: dread d5/d6/db/d11/f14 [0,4194304] 0 2024-09-10T04:05:09.701 INFO:tasks.workunit.client.0.smithi046.stdout:5/293: creat d7/d11/d14/d45/f5f x:0 0 0 2024-09-10T04:05:09.702 INFO:tasks.workunit.client.1.smithi070.stdout:2/157: creat d15/d18/d29/f35 x:0 0 0 2024-09-10T04:05:09.702 INFO:tasks.workunit.client.1.smithi070.stdout:2/158: fsync f11 0 2024-09-10T04:05:09.702 INFO:tasks.workunit.client.1.smithi070.stdout:9/214: mkdir d9/d3e 0 2024-09-10T04:05:09.703 INFO:tasks.workunit.client.1.smithi070.stdout:9/215: read - d9/f22 zero size 2024-09-10T04:05:09.705 INFO:tasks.workunit.client.0.smithi046.stdout:7/197: truncate f3 1007783 0 2024-09-10T04:05:09.708 INFO:tasks.workunit.client.0.smithi046.stdout:7/198: creat d5/d6/db/d11/d1e/d20/f4a x:0 0 0 2024-09-10T04:05:09.709 INFO:tasks.workunit.client.0.smithi046.stdout:2/229: sync 2024-09-10T04:05:09.709 INFO:tasks.workunit.client.0.smithi046.stdout:3/158: sync 2024-09-10T04:05:09.711 INFO:tasks.workunit.client.0.smithi046.stdout:5/294: creat d7/d11/f60 x:0 0 0 2024-09-10T04:05:09.713 INFO:tasks.workunit.client.0.smithi046.stdout:7/199: mkdir d5/d6/db/dd/d4b 0 2024-09-10T04:05:09.716 INFO:tasks.workunit.client.1.smithi070.stdout:8/229: rename d4/d16/d17/d1a/l48 to d4/d16/d17/d19/d28/d43/d52/l53 0 2024-09-10T04:05:09.717 INFO:tasks.workunit.client.1.smithi070.stdout:8/230: chown d4/d16/d17/d19/d28/d43/d52 209058150 1 2024-09-10T04:05:09.718 INFO:tasks.workunit.client.0.smithi046.stdout:3/159: read d12/d13/f17 [4895849,53499] 0 2024-09-10T04:05:09.718 INFO:tasks.workunit.client.1.smithi070.stdout:8/231: write d4/d16/f27 [304579,88189] 0 2024-09-10T04:05:09.718 INFO:tasks.workunit.client.1.smithi070.stdout:0/207: truncate d8/d13/f22 1680112 0 2024-09-10T04:05:09.721 INFO:tasks.workunit.client.1.smithi070.stdout:5/209: symlink d17/d19/d35/l46 0 2024-09-10T04:05:09.722 INFO:tasks.workunit.client.1.smithi070.stdout:5/210: chown d17/d37/l39 0 1 2024-09-10T04:05:09.722 INFO:tasks.workunit.client.0.smithi046.stdout:2/230: mkdir d2/d6/d23/d55 0 2024-09-10T04:05:09.722 INFO:tasks.workunit.client.1.smithi070.stdout:5/211: truncate d17/f36 341976 0 2024-09-10T04:05:09.723 INFO:tasks.workunit.client.1.smithi070.stdout:5/212: truncate f10 5077061 0 2024-09-10T04:05:09.724 INFO:tasks.workunit.client.0.smithi046.stdout:2/231: write d2/d6/d23/d3b/f47 [421623,3587] 0 2024-09-10T04:05:09.725 INFO:tasks.workunit.client.1.smithi070.stdout:2/159: stat c9 0 2024-09-10T04:05:09.726 INFO:tasks.workunit.client.1.smithi070.stdout:2/160: dread - d15/d18/f21 zero size 2024-09-10T04:05:09.729 INFO:tasks.workunit.client.1.smithi070.stdout:9/216: mkdir d9/dd/d2a/d3f 0 2024-09-10T04:05:09.732 INFO:tasks.workunit.client.1.smithi070.stdout:3/216: link c4 db/d1c/d25/d28/c40 0 2024-09-10T04:05:09.733 INFO:tasks.workunit.client.0.smithi046.stdout:3/160: fsync d12/d1b/f2b 0 2024-09-10T04:05:09.735 INFO:tasks.workunit.client.1.smithi070.stdout:3/217: dread db/d1c/d3e/f2a [0,4194304] 0 2024-09-10T04:05:09.735 INFO:tasks.workunit.client.0.smithi046.stdout:3/161: dread d12/d13/f18 [0,4194304] 0 2024-09-10T04:05:09.736 INFO:tasks.workunit.client.1.smithi070.stdout:3/218: dread db/d1c/d25/d28/f37 [0,4194304] 0 2024-09-10T04:05:09.738 INFO:tasks.workunit.client.0.smithi046.stdout:5/295: link d7/lf d7/d11/d3f/d5a/l61 0 2024-09-10T04:05:09.740 INFO:tasks.workunit.client.0.smithi046.stdout:7/200: link d5/d33/l46 d5/d9/l4c 0 2024-09-10T04:05:09.743 INFO:tasks.workunit.client.1.smithi070.stdout:6/266: stat d0/d3/d18/d1b/d17/d3f/d40/f48 0 2024-09-10T04:05:09.748 INFO:tasks.workunit.client.0.smithi046.stdout:5/296: creat d7/d11/d14/d50/f62 x:0 0 0 2024-09-10T04:05:09.758 INFO:tasks.workunit.client.0.smithi046.stdout:5/297: fdatasync d7/f43 0 2024-09-10T04:05:09.760 INFO:tasks.workunit.client.0.smithi046.stdout:1/125: sync 2024-09-10T04:05:09.760 INFO:tasks.workunit.client.0.smithi046.stdout:9/181: sync 2024-09-10T04:05:09.760 INFO:tasks.workunit.client.0.smithi046.stdout:4/161: sync 2024-09-10T04:05:09.760 INFO:tasks.workunit.client.0.smithi046.stdout:6/158: sync 2024-09-10T04:05:09.761 INFO:tasks.workunit.client.1.smithi070.stdout:2/161: mkdir d15/d18/d29/d2a/d2e/d32/d36 0 2024-09-10T04:05:09.762 INFO:tasks.workunit.client.0.smithi046.stdout:9/182: chown d4/d7/c34 3460270 1 2024-09-10T04:05:09.763 INFO:tasks.workunit.client.1.smithi070.stdout:5/213: write f11 [4583159,32358] 0 2024-09-10T04:05:09.763 INFO:tasks.workunit.client.1.smithi070.stdout:1/159: creat d3/d10/f35 x:0 0 0 2024-09-10T04:05:09.763 INFO:tasks.workunit.client.1.smithi070.stdout:8/232: dwrite d4/d16/d17/d19/d28/f30 [8388608,4194304] 0 2024-09-10T04:05:09.763 INFO:tasks.workunit.client.1.smithi070.stdout:3/219: rename db/d1c/f34 to db/d1c/d25/d28/f41 0 2024-09-10T04:05:09.766 INFO:tasks.workunit.client.0.smithi046.stdout:4/162: unlink d3/d12/c2c 0 2024-09-10T04:05:09.770 INFO:tasks.workunit.client.0.smithi046.stdout:7/201: dread d5/f7 [0,4194304] 0 2024-09-10T04:05:09.775 INFO:tasks.workunit.client.0.smithi046.stdout:4/163: mknod d3/d12/c37 0 2024-09-10T04:05:09.781 INFO:tasks.workunit.client.0.smithi046.stdout:7/202: mkdir d5/d6/db/d4d 0 2024-09-10T04:05:09.782 INFO:tasks.workunit.client.0.smithi046.stdout:4/164: creat d3/d12/d17/d29/f38 x:0 0 0 2024-09-10T04:05:09.783 INFO:tasks.workunit.client.0.smithi046.stdout:6/159: getdents da/db 0 2024-09-10T04:05:09.785 INFO:tasks.workunit.client.0.smithi046.stdout:2/232: rmdir d2/d6 39 2024-09-10T04:05:09.786 INFO:tasks.workunit.client.1.smithi070.stdout:8/233: fsync d4/d16/f34 0 2024-09-10T04:05:09.788 INFO:tasks.workunit.client.0.smithi046.stdout:2/233: truncate d2/d8/f27 4874944 0 2024-09-10T04:05:09.788 INFO:tasks.workunit.client.1.smithi070.stdout:7/173: truncate d2/d7/db/f37 327609 0 2024-09-10T04:05:09.789 INFO:tasks.workunit.client.1.smithi070.stdout:5/214: mkdir d17/d24/d47 0 2024-09-10T04:05:09.792 INFO:tasks.workunit.client.0.smithi046.stdout:7/203: dwrite d5/d6/db/dd/f44 [0,4194304] 0 2024-09-10T04:05:09.792 INFO:tasks.workunit.client.0.smithi046.stdout:8/204: rename d1/l13 to d1/d29/d17/l47 0 2024-09-10T04:05:09.792 INFO:tasks.workunit.client.0.smithi046.stdout:6/160: dread da/db/d11/d13/d1e/f21 [0,4194304] 0 2024-09-10T04:05:09.793 INFO:tasks.workunit.client.1.smithi070.stdout:4/160: dwrite de/f11 [0,4194304] 0 2024-09-10T04:05:09.793 INFO:tasks.workunit.client.1.smithi070.stdout:5/215: fsync d17/d19/f44 0 2024-09-10T04:05:09.793 INFO:tasks.workunit.client.1.smithi070.stdout:8/234: rename d4/d16/d17/d1a/d23/d44 to d4/d16/d17/d1a/d23/d44/d54 22 2024-09-10T04:05:09.793 INFO:tasks.workunit.client.1.smithi070.stdout:1/160: mknod d3/d8/d13/c36 0 2024-09-10T04:05:09.794 INFO:tasks.workunit.client.1.smithi070.stdout:6/267: getdents d0/d3/d18/d1b/d17/d54 0 2024-09-10T04:05:09.794 INFO:tasks.workunit.client.1.smithi070.stdout:4/161: dread - de/d13/d18/d26/f2f zero size 2024-09-10T04:05:09.794 INFO:tasks.workunit.client.1.smithi070.stdout:1/161: fsync d3/d8/f28 0 2024-09-10T04:05:09.795 INFO:tasks.workunit.client.1.smithi070.stdout:0/208: write d8/fa [4983667,101267] 0 2024-09-10T04:05:09.795 INFO:tasks.workunit.client.0.smithi046.stdout:4/165: getdents d3/d12/d17 0 2024-09-10T04:05:09.795 INFO:tasks.workunit.client.1.smithi070.stdout:2/162: dwrite d15/d18/d29/f2d [0,4194304] 0 2024-09-10T04:05:09.797 INFO:tasks.workunit.client.0.smithi046.stdout:4/166: readlink d3/d12/d17/d22/d30/l31 0 2024-09-10T04:05:09.797 INFO:tasks.workunit.client.1.smithi070.stdout:6/268: write d0/d3/d18/d1b/f11 [4639675,109078] 0 2024-09-10T04:05:09.801 INFO:tasks.workunit.client.0.smithi046.stdout:7/204: creat d5/d6/db/d11/f4e x:0 0 0 2024-09-10T04:05:09.801 INFO:tasks.workunit.client.1.smithi070.stdout:8/235: unlink f0 0 2024-09-10T04:05:09.804 INFO:tasks.workunit.client.0.smithi046.stdout:0/185: rename d1/dc/d25/d33/d3/c26 to d1/dc/d25/d33/d3/dd/c3a 0 2024-09-10T04:05:09.806 INFO:tasks.workunit.client.1.smithi070.stdout:5/216: symlink d17/d24/d2b/d31/l48 0 2024-09-10T04:05:09.808 INFO:tasks.workunit.client.0.smithi046.stdout:3/162: dwrite d12/d13/f17 [0,4194304] 0 2024-09-10T04:05:09.810 INFO:tasks.workunit.client.0.smithi046.stdout:8/205: unlink d1/f40 0 2024-09-10T04:05:09.810 INFO:tasks.workunit.client.0.smithi046.stdout:0/186: dread d1/dc/f27 [0,4194304] 0 2024-09-10T04:05:09.811 INFO:tasks.workunit.client.0.smithi046.stdout:0/187: chown d1/dc 1470257 1 2024-09-10T04:05:09.812 INFO:tasks.workunit.client.1.smithi070.stdout:5/217: dwrite f11 [4194304,4194304] 0 2024-09-10T04:05:09.813 INFO:tasks.workunit.client.0.smithi046.stdout:0/188: read d1/dc/d25/d33/d3/f2a [240724,127104] 0 2024-09-10T04:05:09.814 INFO:tasks.workunit.client.0.smithi046.stdout:6/161: mkdir da/db/d11/d2b/d2f/d3b 0 2024-09-10T04:05:09.815 INFO:tasks.workunit.client.1.smithi070.stdout:1/162: truncate d3/d10/d1d/f30 155276 0 2024-09-10T04:05:09.815 INFO:tasks.workunit.client.0.smithi046.stdout:0/189: write d1/dc/d25/d33/d3/d29/d2d/f32 [314479,125854] 0 2024-09-10T04:05:09.816 INFO:tasks.workunit.client.0.smithi046.stdout:2/234: mknod d2/d6/d23/d3b/c56 0 2024-09-10T04:05:09.818 INFO:tasks.workunit.client.0.smithi046.stdout:4/167: creat d3/d12/d17/d22/d30/f39 x:0 0 0 2024-09-10T04:05:09.819 INFO:tasks.workunit.client.0.smithi046.stdout:4/168: read - d3/d12/d14/f2b zero size 2024-09-10T04:05:09.819 INFO:tasks.workunit.client.1.smithi070.stdout:4/162: rmdir de/d13/d18 39 2024-09-10T04:05:09.821 INFO:tasks.workunit.client.0.smithi046.stdout:7/205: mknod d5/d6/db/dd/d24/c4f 0 2024-09-10T04:05:09.825 INFO:tasks.workunit.client.0.smithi046.stdout:3/163: mkdir d12/d13/d14/d1e/d2d 0 2024-09-10T04:05:09.826 INFO:tasks.workunit.client.0.smithi046.stdout:3/164: write f3 [3260426,82673] 0 2024-09-10T04:05:09.827 INFO:tasks.workunit.client.1.smithi070.stdout:1/163: dread d3/d8/f28 [0,4194304] 0 2024-09-10T04:05:09.828 INFO:tasks.workunit.client.1.smithi070.stdout:1/164: dread - d3/d8/d12/d2f/f34 zero size 2024-09-10T04:05:09.829 INFO:tasks.workunit.client.0.smithi046.stdout:1/126: write d3/d27/f20 [608156,22118] 0 2024-09-10T04:05:09.829 INFO:tasks.workunit.client.1.smithi070.stdout:1/165: fsync d3/d8/d12/f19 0 2024-09-10T04:05:09.830 INFO:tasks.workunit.client.1.smithi070.stdout:1/166: chown d3/d20 402065661 1 2024-09-10T04:05:09.830 INFO:tasks.workunit.client.0.smithi046.stdout:6/162: symlink da/db/d11/d2b/d2f/l3c 0 2024-09-10T04:05:09.831 INFO:tasks.workunit.client.0.smithi046.stdout:5/298: dwrite d7/f19 [0,4194304] 0 2024-09-10T04:05:09.831 INFO:tasks.workunit.client.1.smithi070.stdout:1/167: chown d3/d8/d12/f24 7308 1 2024-09-10T04:05:09.832 INFO:tasks.workunit.client.1.smithi070.stdout:1/168: chown d3/d8/d13 248698560 1 2024-09-10T04:05:09.833 INFO:tasks.workunit.client.1.smithi070.stdout:5/218: chown c15 475736 1 2024-09-10T04:05:09.833 INFO:tasks.workunit.client.1.smithi070.stdout:1/169: chown d3/d10 178903 1 2024-09-10T04:05:09.833 INFO:tasks.workunit.client.1.smithi070.stdout:8/236: dwrite d4/d16/d17/d1a/f32 [0,4194304] 0 2024-09-10T04:05:09.834 INFO:tasks.workunit.client.1.smithi070.stdout:5/219: chown d17/f20 3102 1 2024-09-10T04:05:09.836 INFO:tasks.workunit.client.0.smithi046.stdout:9/183: dwrite d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:09.836 INFO:tasks.workunit.client.0.smithi046.stdout:4/169: creat d3/d12/d17/d22/d30/f3a x:0 0 0 2024-09-10T04:05:09.838 INFO:tasks.workunit.client.0.smithi046.stdout:4/170: chown d3/d12/d17/d22/f27 27 1 2024-09-10T04:05:09.839 INFO:tasks.workunit.client.1.smithi070.stdout:0/209: link d8/d13/d1a/f35 d8/f49 0 2024-09-10T04:05:09.840 INFO:tasks.workunit.client.1.smithi070.stdout:0/210: write d8/d13/d1a/f31 [42221,90840] 0 2024-09-10T04:05:09.842 INFO:tasks.workunit.client.0.smithi046.stdout:7/206: unlink d5/f2c 0 2024-09-10T04:05:09.843 INFO:tasks.workunit.client.1.smithi070.stdout:2/163: mknod d15/d18/d29/d2a/d2e/d32/d36/c37 0 2024-09-10T04:05:09.852 INFO:tasks.workunit.client.0.smithi046.stdout:2/235: mkdir d2/d57 0 2024-09-10T04:05:09.853 INFO:tasks.workunit.client.0.smithi046.stdout:2/236: chown d2/d49 78 1 2024-09-10T04:05:09.854 INFO:tasks.workunit.client.1.smithi070.stdout:5/220: rename d17/d24/c3b to d17/d24/d2b/d31/c49 0 2024-09-10T04:05:09.855 INFO:tasks.workunit.client.0.smithi046.stdout:9/184: dread d4/d18/f22 [0,4194304] 0 2024-09-10T04:05:09.856 INFO:tasks.workunit.client.0.smithi046.stdout:9/185: dread - d4/d7/d1f/f21 zero size 2024-09-10T04:05:09.856 INFO:tasks.workunit.client.0.smithi046.stdout:4/171: mknod d3/d12/d17/d29/c3b 0 2024-09-10T04:05:09.857 INFO:tasks.workunit.client.1.smithi070.stdout:2/164: mknod d15/d18/d29/d2a/c38 0 2024-09-10T04:05:09.859 INFO:tasks.workunit.client.0.smithi046.stdout:9/186: truncate d4/d18/f22 4720132 0 2024-09-10T04:05:09.860 INFO:tasks.workunit.client.0.smithi046.stdout:3/165: creat d12/d13/d14/d1e/d2d/f2e x:0 0 0 2024-09-10T04:05:09.860 INFO:tasks.workunit.client.1.smithi070.stdout:0/211: dwrite d8/d13/d1a/f35 [0,4194304] 0 2024-09-10T04:05:09.861 INFO:tasks.workunit.client.0.smithi046.stdout:1/127: link d3/dc/f17 d3/dc/d16/d28/f2c 0 2024-09-10T04:05:09.861 INFO:tasks.workunit.client.1.smithi070.stdout:0/212: truncate d8/d13/d1a/f3c 1724774 0 2024-09-10T04:05:09.862 INFO:tasks.workunit.client.1.smithi070.stdout:5/221: unlink d17/d19/l21 0 2024-09-10T04:05:09.862 INFO:tasks.workunit.client.1.smithi070.stdout:0/213: chown d8/d13/d1a/f30 203 1 2024-09-10T04:05:09.863 INFO:tasks.workunit.client.0.smithi046.stdout:2/237: rmdir d2/d8/d37 39 2024-09-10T04:05:09.863 INFO:tasks.workunit.client.1.smithi070.stdout:0/214: write d8/d24/d36/f46 [5615452,49393] 0 2024-09-10T04:05:09.864 INFO:tasks.workunit.client.1.smithi070.stdout:4/163: creat de/d13/d18/f35 x:0 0 0 2024-09-10T04:05:09.865 INFO:tasks.workunit.client.1.smithi070.stdout:5/222: mknod d17/d19/d35/c4a 0 2024-09-10T04:05:09.871 INFO:tasks.workunit.client.0.smithi046.stdout:4/172: creat d3/f3c x:0 0 0 2024-09-10T04:05:09.871 INFO:tasks.workunit.client.0.smithi046.stdout:8/206: getdents d1/d29/d19/d36 0 2024-09-10T04:05:09.875 INFO:tasks.workunit.client.0.smithi046.stdout:3/166: symlink d12/d13/d14/l2f 0 2024-09-10T04:05:09.875 INFO:tasks.workunit.client.0.smithi046.stdout:4/173: truncate d3/d12/d17/d22/d30/f39 584345 0 2024-09-10T04:05:09.875 INFO:tasks.workunit.client.0.smithi046.stdout:1/128: mknod d3/dc/d16/c2d 0 2024-09-10T04:05:09.878 INFO:tasks.workunit.client.0.smithi046.stdout:2/238: dwrite d2/d8/f29 [0,4194304] 0 2024-09-10T04:05:09.888 INFO:tasks.workunit.client.1.smithi070.stdout:0/215: rename d8/d13/f20 to d8/d24/f4a 0 2024-09-10T04:05:09.889 INFO:tasks.workunit.client.1.smithi070.stdout:5/223: rename d17 to d17/d24/d2b/d4b 22 2024-09-10T04:05:09.889 INFO:tasks.workunit.client.1.smithi070.stdout:0/216: stat d8/d13/f19 0 2024-09-10T04:05:09.890 INFO:tasks.workunit.client.1.smithi070.stdout:0/217: readlink d8/l15 0 2024-09-10T04:05:09.894 INFO:tasks.workunit.client.0.smithi046.stdout:6/163: dread f9 [0,4194304] 0 2024-09-10T04:05:09.894 INFO:tasks.workunit.client.1.smithi070.stdout:5/224: dwrite d17/d19/d35/f3a [0,4194304] 0 2024-09-10T04:05:09.895 INFO:tasks.workunit.client.0.smithi046.stdout:5/299: getdents d7/d11 0 2024-09-10T04:05:09.895 INFO:tasks.workunit.client.1.smithi070.stdout:5/225: dread d17/f36 [0,4194304] 0 2024-09-10T04:05:09.896 INFO:tasks.workunit.client.0.smithi046.stdout:1/129: creat d3/dc/d16/f2e x:0 0 0 2024-09-10T04:05:09.898 INFO:tasks.workunit.client.0.smithi046.stdout:3/167: mknod d12/d13/d14/d1e/d2d/c30 0 2024-09-10T04:05:09.904 INFO:tasks.workunit.client.0.smithi046.stdout:7/207: rename d5/d9/fe to d5/d6/db/f50 0 2024-09-10T04:05:09.906 INFO:tasks.workunit.client.1.smithi070.stdout:6/269: truncate d0/d3/d1f/f22 2945560 0 2024-09-10T04:05:09.908 INFO:tasks.workunit.client.1.smithi070.stdout:6/270: stat d0/ce 0 2024-09-10T04:05:09.908 INFO:tasks.workunit.client.0.smithi046.stdout:6/164: rmdir da 39 2024-09-10T04:05:09.910 INFO:tasks.workunit.client.0.smithi046.stdout:1/130: symlink d3/dc/d16/d28/l2f 0 2024-09-10T04:05:09.912 INFO:tasks.workunit.client.1.smithi070.stdout:5/226: dwrite d17/d24/f25 [0,4194304] 0 2024-09-10T04:05:09.914 INFO:tasks.workunit.client.0.smithi046.stdout:2/239: mknod d2/d6/d23/d55/c58 0 2024-09-10T04:05:09.915 INFO:tasks.workunit.client.1.smithi070.stdout:1/170: truncate f0 3393071 0 2024-09-10T04:05:09.915 INFO:tasks.workunit.client.1.smithi070.stdout:8/237: dwrite d4/d16/d17/d1a/d23/f2f [0,4194304] 0 2024-09-10T04:05:09.917 INFO:tasks.workunit.client.0.smithi046.stdout:7/208: fdatasync d5/f7 0 2024-09-10T04:05:09.918 INFO:tasks.workunit.client.1.smithi070.stdout:1/171: read - d3/d8/d13/f17 zero size 2024-09-10T04:05:09.919 INFO:tasks.workunit.client.1.smithi070.stdout:6/271: dwrite d0/d3/d18/f45 [0,4194304] 0 2024-09-10T04:05:09.920 INFO:tasks.workunit.client.0.smithi046.stdout:6/165: stat da/dc 0 2024-09-10T04:05:09.920 INFO:tasks.workunit.client.0.smithi046.stdout:6/166: chown da/db/d11 0 1 2024-09-10T04:05:09.921 INFO:tasks.workunit.client.0.smithi046.stdout:5/300: mkdir d7/d11/d3f/d5a/d63 0 2024-09-10T04:05:09.921 INFO:tasks.workunit.client.0.smithi046.stdout:6/167: truncate da/fd 1328807 0 2024-09-10T04:05:09.923 INFO:tasks.workunit.client.0.smithi046.stdout:5/301: read d7/d11/d14/d2f/f48 [1057605,109748] 0 2024-09-10T04:05:09.924 INFO:tasks.workunit.client.1.smithi070.stdout:2/165: write d15/d18/f28 [428715,78297] 0 2024-09-10T04:05:09.925 INFO:tasks.workunit.client.1.smithi070.stdout:5/227: rename d17/d24/d2b/l40 to d17/d24/d45/l4c 0 2024-09-10T04:05:09.927 INFO:tasks.workunit.client.1.smithi070.stdout:8/238: fdatasync d4/f15 0 2024-09-10T04:05:09.930 INFO:tasks.workunit.client.1.smithi070.stdout:2/166: rmdir d15/d18/d29/d2a 39 2024-09-10T04:05:09.934 INFO:tasks.workunit.client.1.smithi070.stdout:5/228: dwrite d17/d24/f2d [0,4194304] 0 2024-09-10T04:05:09.935 INFO:tasks.workunit.client.1.smithi070.stdout:4/164: dwrite de/d13/d18/d26/f2a [0,4194304] 0 2024-09-10T04:05:09.936 INFO:tasks.workunit.client.1.smithi070.stdout:1/172: rename d3/fd to d3/d8/d12/d2f/f37 0 2024-09-10T04:05:09.936 INFO:tasks.workunit.client.0.smithi046.stdout:4/174: write d3/f9 [82538,78131] 0 2024-09-10T04:05:09.938 INFO:tasks.workunit.client.1.smithi070.stdout:4/165: truncate de/d13/d18/d26/f2d 609891 0 2024-09-10T04:05:09.939 INFO:tasks.workunit.client.0.smithi046.stdout:8/207: dwrite d1/fd [0,4194304] 0 2024-09-10T04:05:09.940 INFO:tasks.workunit.client.0.smithi046.stdout:5/302: write d7/fd [3259501,39059] 0 2024-09-10T04:05:09.942 INFO:tasks.workunit.client.0.smithi046.stdout:6/168: dread da/db/d10/f17 [4194304,4194304] 0 2024-09-10T04:05:09.942 INFO:tasks.workunit.client.0.smithi046.stdout:5/303: chown d7/la 1924681603 1 2024-09-10T04:05:09.943 INFO:tasks.workunit.client.1.smithi070.stdout:1/173: dwrite d3/d8/fc [4194304,4194304] 0 2024-09-10T04:05:09.944 INFO:tasks.workunit.client.0.smithi046.stdout:9/187: rename d4/d7/d1f/f1c to d4/d18/f36 0 2024-09-10T04:05:09.945 INFO:tasks.workunit.client.1.smithi070.stdout:1/174: dread - d3/d8/d12/d2f/f34 zero size 2024-09-10T04:05:09.946 INFO:tasks.workunit.client.0.smithi046.stdout:9/188: write d4/d7/f20 [850448,22470] 0 2024-09-10T04:05:09.949 INFO:tasks.workunit.client.0.smithi046.stdout:7/209: unlink d5/d6/db/l19 0 2024-09-10T04:05:09.950 INFO:tasks.workunit.client.0.smithi046.stdout:7/210: chown d5/f7 1 1 2024-09-10T04:05:09.950 INFO:tasks.workunit.client.1.smithi070.stdout:8/239: mkdir d4/d16/d17/d19/d28/d43/d55 0 2024-09-10T04:05:09.952 INFO:tasks.workunit.client.1.smithi070.stdout:8/240: chown d4/d39/l3b 1834427 1 2024-09-10T04:05:09.954 INFO:tasks.workunit.client.1.smithi070.stdout:2/167: readlink d15/d18/d29/d2a/d2e/l34 0 2024-09-10T04:05:09.958 INFO:tasks.workunit.client.0.smithi046.stdout:4/175: rename d3/d12/l1d to d3/d12/l3d 0 2024-09-10T04:05:09.960 INFO:tasks.workunit.client.0.smithi046.stdout:4/176: fdatasync d3/d12/d14/f16 0 2024-09-10T04:05:09.963 INFO:tasks.workunit.client.1.smithi070.stdout:4/166: chown f8 2037846 1 2024-09-10T04:05:09.963 INFO:tasks.workunit.client.0.smithi046.stdout:3/168: dwrite f7 [0,4194304] 0 2024-09-10T04:05:09.965 INFO:tasks.workunit.client.0.smithi046.stdout:3/169: chown d12/d1b/f21 353 1 2024-09-10T04:05:09.966 INFO:tasks.workunit.client.1.smithi070.stdout:1/175: symlink d3/d8/d27/l38 0 2024-09-10T04:05:09.966 INFO:tasks.workunit.client.0.smithi046.stdout:3/170: dread - d12/d13/d14/d1e/d2d/f2e zero size 2024-09-10T04:05:09.967 INFO:tasks.workunit.client.0.smithi046.stdout:3/171: stat c10 0 2024-09-10T04:05:09.967 INFO:tasks.workunit.client.0.smithi046.stdout:5/304: mkdir d7/d11/d14/d2f/d64 0 2024-09-10T04:05:09.969 INFO:tasks.workunit.client.0.smithi046.stdout:5/305: write d7/d11/d14/d2f/f5e [238651,117564] 0 2024-09-10T04:05:09.969 INFO:tasks.workunit.client.1.smithi070.stdout:8/241: rename d4/l6 to d4/d16/d17/d1a/d23/d44/l56 0 2024-09-10T04:05:09.970 INFO:tasks.workunit.client.0.smithi046.stdout:6/169: rmdir da/db/d11/d13/d1e/d23 39 2024-09-10T04:05:09.972 INFO:tasks.workunit.client.1.smithi070.stdout:5/229: symlink d17/d24/d47/l4d 0 2024-09-10T04:05:09.973 INFO:tasks.workunit.client.0.smithi046.stdout:2/240: rename d2/c4d to d2/d6/d23/d3b/c59 0 2024-09-10T04:05:09.974 INFO:tasks.workunit.client.1.smithi070.stdout:1/176: symlink d3/d8/d12/l39 0 2024-09-10T04:05:09.975 INFO:tasks.workunit.client.0.smithi046.stdout:3/172: dwrite d12/d13/d14/f15 [4194304,4194304] 0 2024-09-10T04:05:09.976 INFO:tasks.workunit.client.1.smithi070.stdout:1/177: write d3/d8/d13/f18 [268927,90365] 0 2024-09-10T04:05:09.976 INFO:tasks.workunit.client.0.smithi046.stdout:3/173: write f4 [3632968,35397] 0 2024-09-10T04:05:09.978 INFO:tasks.workunit.client.1.smithi070.stdout:5/230: dwrite d17/d19/f2e [0,4194304] 0 2024-09-10T04:05:09.978 INFO:tasks.workunit.client.0.smithi046.stdout:3/174: write d12/f29 [816687,98459] 0 2024-09-10T04:05:09.979 INFO:tasks.workunit.client.1.smithi070.stdout:1/178: read d3/d8/d12/d2f/f37 [2471120,39664] 0 2024-09-10T04:05:09.980 INFO:tasks.workunit.client.0.smithi046.stdout:3/175: write d12/d13/f2c [255630,70968] 0 2024-09-10T04:05:09.980 INFO:tasks.workunit.client.1.smithi070.stdout:1/179: write d3/d10/f2c [3898269,3932] 0 2024-09-10T04:05:09.982 INFO:tasks.workunit.client.1.smithi070.stdout:2/168: rename d15/d18/d29/d2a/d2e/l34 to d15/d18/d29/d2a/l39 0 2024-09-10T04:05:09.983 INFO:tasks.workunit.client.1.smithi070.stdout:4/167: rename de/d13 to de/d13/d18/d23/d36 22 2024-09-10T04:05:09.985 INFO:tasks.workunit.client.1.smithi070.stdout:4/168: stat de/d13/d18/d23 0 2024-09-10T04:05:09.985 INFO:tasks.workunit.client.1.smithi070.stdout:2/169: stat d15/d18/d29/d2a/d2e/d32/d36 0 2024-09-10T04:05:09.986 INFO:tasks.workunit.client.1.smithi070.stdout:2/170: fdatasync d15/d18/f1a 0 2024-09-10T04:05:09.987 INFO:tasks.workunit.client.1.smithi070.stdout:6/272: truncate d0/d3/d18/d1b/d17/d28/f3b 2524323 0 2024-09-10T04:05:09.988 INFO:tasks.workunit.client.0.smithi046.stdout:1/131: getdents d3/dc/d16/d28 0 2024-09-10T04:05:09.988 INFO:tasks.workunit.client.1.smithi070.stdout:1/180: dwrite d3/d8/d13/f17 [0,4194304] 0 2024-09-10T04:05:09.989 INFO:tasks.workunit.client.1.smithi070.stdout:6/273: chown d0/d3/d18/d1b/d17/c26 128426 1 2024-09-10T04:05:09.990 INFO:tasks.workunit.client.1.smithi070.stdout:6/274: read d0/f8 [2037170,52375] 0 2024-09-10T04:05:09.991 INFO:tasks.workunit.client.1.smithi070.stdout:6/275: fdatasync d0/f8 0 2024-09-10T04:05:09.994 INFO:tasks.workunit.client.1.smithi070.stdout:5/231: mkdir d17/d24/d4e 0 2024-09-10T04:05:09.994 INFO:tasks.workunit.client.1.smithi070.stdout:5/232: read - d17/d24/f2c zero size 2024-09-10T04:05:09.996 INFO:tasks.workunit.client.1.smithi070.stdout:1/181: read d3/d8/d13/f25 [761885,86549] 0 2024-09-10T04:05:10.001 INFO:tasks.workunit.client.1.smithi070.stdout:8/242: rename d4/d16/d17/d19/d28/f4a to d4/d39/f57 0 2024-09-10T04:05:10.002 INFO:tasks.workunit.client.1.smithi070.stdout:1/182: dwrite d3/d10/f35 [0,4194304] 0 2024-09-10T04:05:10.005 INFO:tasks.workunit.client.0.smithi046.stdout:5/306: unlink d7/d11/d14/d50/c56 0 2024-09-10T04:05:10.006 INFO:tasks.workunit.client.0.smithi046.stdout:6/170: rmdir da/db/d11/d13/d1e 39 2024-09-10T04:05:10.008 INFO:tasks.workunit.client.1.smithi070.stdout:2/171: symlink d15/d18/d29/d2a/d2e/l3a 0 2024-09-10T04:05:10.009 INFO:tasks.workunit.client.0.smithi046.stdout:4/177: rename d3/d12/d17/d22/d30/f39 to d3/d12/d17/d29/f3e 0 2024-09-10T04:05:10.011 INFO:tasks.workunit.client.0.smithi046.stdout:9/189: symlink d4/d29/d32/l37 0 2024-09-10T04:05:10.013 INFO:tasks.workunit.client.1.smithi070.stdout:6/276: mkdir d0/d3/d1f/d55 0 2024-09-10T04:05:10.016 INFO:tasks.workunit.client.1.smithi070.stdout:6/277: dread d0/d3/d18/d1b/d33/f39 [0,4194304] 0 2024-09-10T04:05:10.017 INFO:tasks.workunit.client.0.smithi046.stdout:7/211: dwrite d5/d6/db/f1d [0,4194304] 0 2024-09-10T04:05:10.018 INFO:tasks.workunit.client.0.smithi046.stdout:1/132: dread - d3/f6 zero size 2024-09-10T04:05:10.024 INFO:tasks.workunit.client.1.smithi070.stdout:8/243: creat d4/d16/d17/d1a/f58 x:0 0 0 2024-09-10T04:05:10.027 INFO:tasks.workunit.client.0.smithi046.stdout:8/208: dwrite d1/d29/d17/d1f/d2a/f2d [0,4194304] 0 2024-09-10T04:05:10.027 INFO:tasks.workunit.client.1.smithi070.stdout:1/183: read d3/d8/d13/f1b [247925,3545] 0 2024-09-10T04:05:10.028 INFO:tasks.workunit.client.0.smithi046.stdout:8/209: stat d1/d29/d17/d1f/d2a/f2d 0 2024-09-10T04:05:10.032 INFO:tasks.workunit.client.1.smithi070.stdout:2/172: mknod d15/d18/c3b 0 2024-09-10T04:05:10.035 INFO:tasks.workunit.client.0.smithi046.stdout:6/171: dwrite f9 [4194304,4194304] 0 2024-09-10T04:05:10.036 INFO:tasks.workunit.client.0.smithi046.stdout:6/172: chown da/db/d11/d2b 5 1 2024-09-10T04:05:10.042 INFO:tasks.workunit.client.0.smithi046.stdout:5/307: write d7/f43 [1766432,28285] 0 2024-09-10T04:05:10.046 INFO:tasks.workunit.client.0.smithi046.stdout:7/212: chown d5/d9/d12/c1a 12 1 2024-09-10T04:05:10.046 INFO:tasks.workunit.client.1.smithi070.stdout:1/184: rmdir d3/d8/d27 39 2024-09-10T04:05:10.046 INFO:tasks.workunit.client.1.smithi070.stdout:4/169: link de/d13/d18/f21 de/d13/d18/f37 0 2024-09-10T04:05:10.052 INFO:tasks.workunit.client.1.smithi070.stdout:1/185: dwrite d3/d10/f35 [0,4194304] 0 2024-09-10T04:05:10.055 INFO:tasks.workunit.client.0.smithi046.stdout:7/213: dwrite d5/d6/d39/f3d [0,4194304] 0 2024-09-10T04:05:10.060 INFO:tasks.workunit.client.1.smithi070.stdout:2/173: rename l6 to d15/d18/d29/d2a/l3c 0 2024-09-10T04:05:10.060 INFO:tasks.workunit.client.1.smithi070.stdout:2/174: write fc [762985,123765] 0 2024-09-10T04:05:10.063 INFO:tasks.workunit.client.0.smithi046.stdout:6/173: write da/fd [1029885,2210] 0 2024-09-10T04:05:10.064 INFO:tasks.workunit.client.0.smithi046.stdout:8/210: dwrite d1/d29/d17/d1f/f35 [0,4194304] 0 2024-09-10T04:05:10.065 INFO:tasks.workunit.client.0.smithi046.stdout:1/133: truncate d3/dc/d16/d1f/f29 1298956 0 2024-09-10T04:05:10.065 INFO:tasks.workunit.client.0.smithi046.stdout:6/174: read da/fe [3756719,44340] 0 2024-09-10T04:05:10.067 INFO:tasks.workunit.client.0.smithi046.stdout:6/175: write f1 [329803,112946] 0 2024-09-10T04:05:10.067 INFO:tasks.workunit.client.0.smithi046.stdout:1/134: write d3/dc/d16/f2e [75680,84392] 0 2024-09-10T04:05:10.068 INFO:tasks.workunit.client.0.smithi046.stdout:2/241: rename d2/c1e to d2/d6/d15/c5a 0 2024-09-10T04:05:10.069 INFO:tasks.workunit.client.0.smithi046.stdout:1/135: stat d3/dc 0 2024-09-10T04:05:10.074 INFO:tasks.workunit.client.0.smithi046.stdout:3/176: getdents d12/d13/d14 0 2024-09-10T04:05:10.076 INFO:tasks.workunit.client.1.smithi070.stdout:8/244: truncate d4/d16/f33 3834482 0 2024-09-10T04:05:10.077 INFO:tasks.workunit.client.0.smithi046.stdout:5/308: link d7/d11/f23 d7/d11/d3f/d5a/d63/f65 0 2024-09-10T04:05:10.078 INFO:tasks.workunit.client.0.smithi046.stdout:4/178: getdents d3/d12/d17/d22/d30 0 2024-09-10T04:05:10.078 INFO:tasks.workunit.client.0.smithi046.stdout:5/309: chown d7/d11/d3f 369 1 2024-09-10T04:05:10.080 INFO:tasks.workunit.client.0.smithi046.stdout:2/242: dwrite d2/d6/f43 [0,4194304] 0 2024-09-10T04:05:10.081 INFO:tasks.workunit.client.1.smithi070.stdout:6/278: dwrite d0/f2e [0,4194304] 0 2024-09-10T04:05:10.083 INFO:tasks.workunit.client.0.smithi046.stdout:4/179: dread d3/d12/d17/d22/f25 [0,4194304] 0 2024-09-10T04:05:10.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:09 smithi070 ceph-mon[35495]: pgmap v258: 65 pgs: 65 active+clean; 489 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 29 MiB/s wr, 234 op/s 2024-09-10T04:05:10.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:09 smithi046 ceph-mon[28328]: pgmap v258: 65 pgs: 65 active+clean; 489 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 29 MiB/s wr, 234 op/s 2024-09-10T04:05:10.088 INFO:tasks.workunit.client.0.smithi046.stdout:1/136: mknod d3/d27/c30 0 2024-09-10T04:05:10.088 INFO:tasks.workunit.client.1.smithi070.stdout:2/175: creat d15/d18/d29/f3d x:0 0 0 2024-09-10T04:05:10.089 INFO:tasks.workunit.client.1.smithi070.stdout:2/176: truncate d15/d18/f21 831047 0 2024-09-10T04:05:10.090 INFO:tasks.workunit.client.1.smithi070.stdout:2/177: write d15/d18/f28 [1831775,86321] 0 2024-09-10T04:05:10.091 INFO:tasks.workunit.client.0.smithi046.stdout:3/177: mkdir d12/d13/d14/d31 0 2024-09-10T04:05:10.091 INFO:tasks.workunit.client.1.smithi070.stdout:6/279: mknod d0/d3/d18/d2a/c56 0 2024-09-10T04:05:10.093 INFO:tasks.workunit.client.0.smithi046.stdout:6/176: stat da/db/d11/d13/d1e/d23 0 2024-09-10T04:05:10.093 INFO:tasks.workunit.client.1.smithi070.stdout:8/245: truncate d4/d16/f36 3017313 0 2024-09-10T04:05:10.101 INFO:tasks.workunit.client.0.smithi046.stdout:5/310: creat d7/d11/d14/d50/f66 x:0 0 0 2024-09-10T04:05:10.102 INFO:tasks.workunit.client.0.smithi046.stdout:5/311: dread - d7/d11/f37 zero size 2024-09-10T04:05:10.102 INFO:tasks.workunit.client.1.smithi070.stdout:2/178: truncate d15/d18/f24 2593449 0 2024-09-10T04:05:10.103 INFO:tasks.workunit.client.0.smithi046.stdout:4/180: mknod d3/d12/d14/c3f 0 2024-09-10T04:05:10.105 INFO:tasks.workunit.client.0.smithi046.stdout:4/181: write d3/d12/d17/d22/d30/f3a [181420,120838] 0 2024-09-10T04:05:10.110 INFO:tasks.workunit.client.1.smithi070.stdout:2/179: symlink d15/d18/d29/d2a/l3e 0 2024-09-10T04:05:10.112 INFO:tasks.workunit.client.0.smithi046.stdout:7/214: rename d5/d6/d23/f36 to d5/d9/d43/f51 0 2024-09-10T04:05:10.117 INFO:tasks.workunit.client.1.smithi070.stdout:6/280: creat d0/f57 x:0 0 0 2024-09-10T04:05:10.120 INFO:tasks.workunit.client.1.smithi070.stdout:2/180: mknod d15/d18/c3f 0 2024-09-10T04:05:10.124 INFO:tasks.workunit.client.0.smithi046.stdout:5/312: creat d7/d11/d14/d45/d51/f67 x:0 0 0 2024-09-10T04:05:10.124 INFO:tasks.workunit.client.0.smithi046.stdout:2/243: dwrite d2/d8/d37/f41 [0,4194304] 0 2024-09-10T04:05:10.125 INFO:tasks.workunit.client.1.smithi070.stdout:6/281: rmdir d0/d3/d18/d2a 39 2024-09-10T04:05:10.126 INFO:tasks.workunit.client.0.smithi046.stdout:1/137: creat d3/dc/d16/d1f/f31 x:0 0 0 2024-09-10T04:05:10.127 INFO:tasks.workunit.client.1.smithi070.stdout:2/181: rmdir d15/d18/d29 39 2024-09-10T04:05:10.130 INFO:tasks.workunit.client.0.smithi046.stdout:2/244: dread d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:10.130 INFO:tasks.workunit.client.0.smithi046.stdout:3/178: mknod d12/d13/d14/d1e/c32 0 2024-09-10T04:05:10.131 INFO:tasks.workunit.client.1.smithi070.stdout:6/282: dwrite d0/d3/d18/d1b/f11 [4194304,4194304] 0 2024-09-10T04:05:10.133 INFO:tasks.workunit.client.1.smithi070.stdout:8/246: truncate d4/d16/d17/d1a/f32 2120516 0 2024-09-10T04:05:10.137 INFO:tasks.workunit.client.0.smithi046.stdout:7/215: rmdir d5/d6/db/d11/d1e/d20 39 2024-09-10T04:05:10.138 INFO:tasks.workunit.client.0.smithi046.stdout:6/177: symlink da/db/d11/d2b/d2f/d3b/l3d 0 2024-09-10T04:05:10.140 INFO:tasks.workunit.client.1.smithi070.stdout:8/247: creat d4/d16/d17/d1a/d23/d44/f59 x:0 0 0 2024-09-10T04:05:10.142 INFO:tasks.workunit.client.1.smithi070.stdout:2/182: symlink d15/d18/d29/d2a/d2e/l40 0 2024-09-10T04:05:10.146 INFO:tasks.workunit.client.0.smithi046.stdout:5/313: dwrite d7/f1b [0,4194304] 0 2024-09-10T04:05:10.149 INFO:tasks.workunit.client.1.smithi070.stdout:2/183: unlink fb 0 2024-09-10T04:05:10.149 INFO:tasks.workunit.client.1.smithi070.stdout:2/184: fdatasync fc 0 2024-09-10T04:05:10.150 INFO:tasks.workunit.client.0.smithi046.stdout:2/245: creat d2/d6/d23/f5b x:0 0 0 2024-09-10T04:05:10.156 INFO:tasks.workunit.client.0.smithi046.stdout:7/216: fdatasync f3 0 2024-09-10T04:05:10.160 INFO:tasks.workunit.client.0.smithi046.stdout:1/138: rename d3/dc/d16 to d3/d27/d32 0 2024-09-10T04:05:10.168 INFO:tasks.workunit.client.0.smithi046.stdout:5/314: dwrite d7/d11/d3f/d53/f57 [0,4194304] 0 2024-09-10T04:05:10.168 INFO:tasks.workunit.client.1.smithi070.stdout:8/248: rmdir d4/d16/d4b 0 2024-09-10T04:05:10.172 INFO:tasks.workunit.client.0.smithi046.stdout:5/315: dread d7/d11/d14/d2f/f5e [0,4194304] 0 2024-09-10T04:05:10.174 INFO:tasks.workunit.client.0.smithi046.stdout:7/217: rename d5/d33/f45 to d5/d6/db/dd/d4b/f52 0 2024-09-10T04:05:10.175 INFO:tasks.workunit.client.0.smithi046.stdout:7/218: stat d5/c13 0 2024-09-10T04:05:10.175 INFO:tasks.workunit.client.0.smithi046.stdout:7/219: fsync d5/d6/f1c 0 2024-09-10T04:05:10.176 INFO:tasks.workunit.client.1.smithi070.stdout:8/249: unlink d4/d16/d17/d19/d28/d43/f49 0 2024-09-10T04:05:10.176 INFO:tasks.workunit.client.0.smithi046.stdout:4/182: truncate d3/f9 820804 0 2024-09-10T04:05:10.180 INFO:tasks.workunit.client.1.smithi070.stdout:6/283: write d0/d3/d18/d1b/f3a [785863,22437] 0 2024-09-10T04:05:10.180 INFO:tasks.workunit.client.1.smithi070.stdout:6/284: chown d0/d3/c7 1 1 2024-09-10T04:05:10.181 INFO:tasks.workunit.client.0.smithi046.stdout:4/183: dread d3/d12/d17/d29/f3e [0,4194304] 0 2024-09-10T04:05:10.181 INFO:tasks.workunit.client.0.smithi046.stdout:4/184: fsync d3/d12/d14/f36 0 2024-09-10T04:05:10.182 INFO:tasks.workunit.client.0.smithi046.stdout:3/179: link d12/d13/d14/d1e/d2d/f2e d12/d13/d14/f33 0 2024-09-10T04:05:10.183 INFO:tasks.workunit.client.0.smithi046.stdout:3/180: dread - d12/d13/d14/d1e/d2d/f2e zero size 2024-09-10T04:05:10.186 INFO:tasks.workunit.client.1.smithi070.stdout:6/285: symlink d0/d3/d18/d1b/d17/d28/l58 0 2024-09-10T04:05:10.186 INFO:tasks.workunit.client.1.smithi070.stdout:9/217: sync 2024-09-10T04:05:10.187 INFO:tasks.workunit.client.1.smithi070.stdout:3/220: sync 2024-09-10T04:05:10.188 INFO:tasks.workunit.client.1.smithi070.stdout:8/250: mknod d4/d16/d17/c5a 0 2024-09-10T04:05:10.189 INFO:tasks.workunit.client.1.smithi070.stdout:2/185: link ce d15/d18/d29/c41 0 2024-09-10T04:05:10.192 INFO:tasks.workunit.client.0.smithi046.stdout:1/139: write d3/dc/f17 [4742718,101287] 0 2024-09-10T04:05:10.193 INFO:tasks.workunit.client.1.smithi070.stdout:9/218: dread d9/d1c/d34/f32 [0,4194304] 0 2024-09-10T04:05:10.193 INFO:tasks.workunit.client.0.smithi046.stdout:6/178: creat da/db/d11/d13/d1e/f3e x:0 0 0 2024-09-10T04:05:10.194 INFO:tasks.workunit.client.1.smithi070.stdout:3/221: dwrite db/d1c/d25/f3d [0,4194304] 0 2024-09-10T04:05:10.195 INFO:tasks.workunit.client.1.smithi070.stdout:9/219: stat d9/dd/d10/d1b 0 2024-09-10T04:05:10.196 INFO:tasks.workunit.client.1.smithi070.stdout:8/251: dwrite d4/d16/d17/f2c [0,4194304] 0 2024-09-10T04:05:10.199 INFO:tasks.workunit.client.1.smithi070.stdout:8/252: write d4/d16/d17/d19/d28/f30 [2199684,115305] 0 2024-09-10T04:05:10.201 INFO:tasks.workunit.client.0.smithi046.stdout:6/179: dwrite f1 [0,4194304] 0 2024-09-10T04:05:10.202 INFO:tasks.workunit.client.0.smithi046.stdout:6/180: readlink l5 0 2024-09-10T04:05:10.208 INFO:tasks.workunit.client.1.smithi070.stdout:6/286: creat d0/d3/d18/d2a/f59 x:0 0 0 2024-09-10T04:05:10.210 INFO:tasks.workunit.client.0.smithi046.stdout:5/316: symlink d7/d11/d3f/d5a/d63/l68 0 2024-09-10T04:05:10.210 INFO:tasks.workunit.client.1.smithi070.stdout:3/222: mknod db/d1c/c42 0 2024-09-10T04:05:10.212 INFO:tasks.workunit.client.0.smithi046.stdout:2/246: rename d2/d6/d15/l28 to d2/d6/d15/d44/l5c 0 2024-09-10T04:05:10.216 INFO:tasks.workunit.client.1.smithi070.stdout:8/253: rmdir d4/d16 39 2024-09-10T04:05:10.217 INFO:tasks.workunit.client.0.smithi046.stdout:5/317: dread d7/f10 [0,4194304] 0 2024-09-10T04:05:10.218 INFO:tasks.workunit.client.0.smithi046.stdout:4/185: dwrite d3/d12/d17/d22/f25 [0,4194304] 0 2024-09-10T04:05:10.218 INFO:tasks.workunit.client.1.smithi070.stdout:2/186: mkdir d15/d18/d42 0 2024-09-10T04:05:10.218 INFO:tasks.workunit.client.1.smithi070.stdout:2/187: write d15/d18/f20 [12236,73432] 0 2024-09-10T04:05:10.219 INFO:tasks.workunit.client.1.smithi070.stdout:6/287: mknod d0/d3/d18/d1b/d17/c5a 0 2024-09-10T04:05:10.221 INFO:tasks.workunit.client.1.smithi070.stdout:3/223: dread db/d1c/d3e/f2a [0,4194304] 0 2024-09-10T04:05:10.221 INFO:tasks.workunit.client.0.smithi046.stdout:3/181: write d12/d13/d14/f25 [1144811,109552] 0 2024-09-10T04:05:10.222 INFO:tasks.workunit.client.1.smithi070.stdout:9/220: creat d9/dd/d10/d3c/f40 x:0 0 0 2024-09-10T04:05:10.222 INFO:tasks.workunit.client.0.smithi046.stdout:5/318: chown d7/d11/d14/d2f/d2b/l44 285 1 2024-09-10T04:05:10.224 INFO:tasks.workunit.client.0.smithi046.stdout:1/140: dwrite d3/dc/f10 [0,4194304] 0 2024-09-10T04:05:10.226 INFO:tasks.workunit.client.0.smithi046.stdout:6/181: mkdir da/d3f 0 2024-09-10T04:05:10.228 INFO:tasks.workunit.client.1.smithi070.stdout:9/221: dwrite d9/dd/f12 [0,4194304] 0 2024-09-10T04:05:10.228 INFO:tasks.workunit.client.1.smithi070.stdout:3/224: mkdir db/d1c/d25/d28/d43 0 2024-09-10T04:05:10.230 INFO:tasks.workunit.client.1.smithi070.stdout:9/222: write d9/f35 [769865,90872] 0 2024-09-10T04:05:10.231 INFO:tasks.workunit.client.0.smithi046.stdout:5/319: dread d7/d11/f1a [4194304,4194304] 0 2024-09-10T04:05:10.231 INFO:tasks.workunit.client.1.smithi070.stdout:8/254: rename d4/ff to d4/d16/d17/d19/f5b 0 2024-09-10T04:05:10.232 INFO:tasks.workunit.client.1.smithi070.stdout:2/188: mkdir d15/d18/d42/d43 0 2024-09-10T04:05:10.235 INFO:tasks.workunit.client.1.smithi070.stdout:2/189: write d15/d18/f28 [1127331,31465] 0 2024-09-10T04:05:10.238 INFO:tasks.workunit.client.1.smithi070.stdout:6/288: symlink d0/d3/d1f/d55/l5b 0 2024-09-10T04:05:10.239 INFO:tasks.workunit.client.1.smithi070.stdout:6/289: stat d0/ce 0 2024-09-10T04:05:10.239 INFO:tasks.workunit.client.0.smithi046.stdout:5/320: dwrite d7/d11/d14/f1d [0,4194304] 0 2024-09-10T04:05:10.240 INFO:tasks.workunit.client.1.smithi070.stdout:3/225: creat db/d1c/d3e/f44 x:0 0 0 2024-09-10T04:05:10.240 INFO:tasks.workunit.client.0.smithi046.stdout:5/321: stat d7/d11/d14/d50 0 2024-09-10T04:05:10.241 INFO:tasks.workunit.client.0.smithi046.stdout:4/186: stat d3/f9 0 2024-09-10T04:05:10.242 INFO:tasks.workunit.client.0.smithi046.stdout:4/187: readlink d3/d12/l32 0 2024-09-10T04:05:10.243 INFO:tasks.workunit.client.1.smithi070.stdout:9/223: mknod d9/d1c/c41 0 2024-09-10T04:05:10.244 INFO:tasks.workunit.client.1.smithi070.stdout:2/190: dwrite d15/d18/f27 [0,4194304] 0 2024-09-10T04:05:10.244 INFO:tasks.workunit.client.1.smithi070.stdout:9/224: chown d9/dd/d10/d1b/f2d 16570238 1 2024-09-10T04:05:10.245 INFO:tasks.workunit.client.1.smithi070.stdout:9/225: chown d9/d1c/c41 228653 1 2024-09-10T04:05:10.250 INFO:tasks.workunit.client.0.smithi046.stdout:2/247: getdents d2 0 2024-09-10T04:05:10.251 INFO:tasks.workunit.client.0.smithi046.stdout:2/248: chown d2/d6/f4f 95839411 1 2024-09-10T04:05:10.261 INFO:tasks.workunit.client.0.smithi046.stdout:0/190: sync 2024-09-10T04:05:10.262 INFO:tasks.workunit.client.1.smithi070.stdout:2/191: chown cd 3698 1 2024-09-10T04:05:10.263 INFO:tasks.workunit.client.1.smithi070.stdout:8/255: mkdir d4/d16/d17/d5c 0 2024-09-10T04:05:10.264 INFO:tasks.workunit.client.0.smithi046.stdout:3/182: truncate f7 2045924 0 2024-09-10T04:05:10.265 INFO:tasks.workunit.client.0.smithi046.stdout:6/182: creat da/f40 x:0 0 0 2024-09-10T04:05:10.267 INFO:tasks.workunit.client.0.smithi046.stdout:2/249: mkdir d2/d6/d23/d5d 0 2024-09-10T04:05:10.268 INFO:tasks.workunit.client.1.smithi070.stdout:6/290: mkdir d0/d3/d5c 0 2024-09-10T04:05:10.270 INFO:tasks.workunit.client.0.smithi046.stdout:0/191: creat d1/d13/f3b x:0 0 0 2024-09-10T04:05:10.273 INFO:tasks.workunit.client.0.smithi046.stdout:1/141: truncate d3/dc/f17 1815121 0 2024-09-10T04:05:10.273 INFO:tasks.workunit.client.1.smithi070.stdout:6/291: dwrite d0/d3/d18/f42 [0,4194304] 0 2024-09-10T04:05:10.275 INFO:tasks.workunit.client.1.smithi070.stdout:6/292: write d0/d3/d18/f4c [790252,80853] 0 2024-09-10T04:05:10.276 INFO:tasks.workunit.client.1.smithi070.stdout:6/293: stat d0/d3/d18/d1b/d17/d28 0 2024-09-10T04:05:10.276 INFO:tasks.workunit.client.0.smithi046.stdout:4/188: unlink d3/d12/d17/d22/f25 0 2024-09-10T04:05:10.277 INFO:tasks.workunit.client.0.smithi046.stdout:4/189: write d3/d12/d14/f16 [1989494,105521] 0 2024-09-10T04:05:10.278 INFO:tasks.workunit.client.0.smithi046.stdout:4/190: chown d3/l7 5 1 2024-09-10T04:05:10.279 INFO:tasks.workunit.client.1.smithi070.stdout:2/192: unlink d15/d18/l2f 0 2024-09-10T04:05:10.279 INFO:tasks.workunit.client.0.smithi046.stdout:4/191: read d3/f15 [2219603,66958] 0 2024-09-10T04:05:10.280 INFO:tasks.workunit.client.1.smithi070.stdout:2/193: chown d15/d18/f27 1540425347 1 2024-09-10T04:05:10.280 INFO:tasks.workunit.client.0.smithi046.stdout:4/192: write d3/fa [1870084,42635] 0 2024-09-10T04:05:10.281 INFO:tasks.workunit.client.1.smithi070.stdout:2/194: dread - d15/d18/d29/f35 zero size 2024-09-10T04:05:10.281 INFO:tasks.workunit.client.1.smithi070.stdout:2/195: stat d15/d18/l2c 0 2024-09-10T04:05:10.282 INFO:tasks.workunit.client.0.smithi046.stdout:4/193: write d3/d12/d14/f36 [182393,48655] 0 2024-09-10T04:05:10.282 INFO:tasks.workunit.client.1.smithi070.stdout:8/256: unlink d4/d16/d17/l4f 0 2024-09-10T04:05:10.285 INFO:tasks.workunit.client.1.smithi070.stdout:9/226: symlink d9/dd/l42 0 2024-09-10T04:05:10.285 INFO:tasks.workunit.client.1.smithi070.stdout:3/226: truncate db/f12 3255307 0 2024-09-10T04:05:10.286 INFO:tasks.workunit.client.0.smithi046.stdout:2/250: chown d2/d8/l11 50 1 2024-09-10T04:05:10.287 INFO:tasks.workunit.client.1.smithi070.stdout:8/257: dread d4/d16/f22 [0,4194304] 0 2024-09-10T04:05:10.287 INFO:tasks.workunit.client.0.smithi046.stdout:2/251: fdatasync d2/d8/f27 0 2024-09-10T04:05:10.289 INFO:tasks.workunit.client.0.smithi046.stdout:1/142: creat d3/d27/f33 x:0 0 0 2024-09-10T04:05:10.291 INFO:tasks.workunit.client.1.smithi070.stdout:6/294: mknod d0/d3/d18/d1b/d17/d28/c5d 0 2024-09-10T04:05:10.292 INFO:tasks.workunit.client.1.smithi070.stdout:2/196: symlink d15/d18/d29/d2a/l44 0 2024-09-10T04:05:10.294 INFO:tasks.workunit.client.0.smithi046.stdout:3/183: symlink d12/d13/d14/d31/l34 0 2024-09-10T04:05:10.296 INFO:tasks.workunit.client.1.smithi070.stdout:4/170: sync 2024-09-10T04:05:10.296 INFO:tasks.workunit.client.1.smithi070.stdout:0/218: sync 2024-09-10T04:05:10.296 INFO:tasks.workunit.client.1.smithi070.stdout:7/174: sync 2024-09-10T04:05:10.296 INFO:tasks.workunit.client.1.smithi070.stdout:1/186: sync 2024-09-10T04:05:10.296 INFO:tasks.workunit.client.1.smithi070.stdout:5/233: sync 2024-09-10T04:05:10.297 INFO:tasks.workunit.client.0.smithi046.stdout:4/194: symlink d3/d12/d17/l40 0 2024-09-10T04:05:10.298 INFO:tasks.workunit.client.1.smithi070.stdout:2/197: dwrite f3 [0,4194304] 0 2024-09-10T04:05:10.300 INFO:tasks.workunit.client.0.smithi046.stdout:6/183: truncate da/db/d11/d13/f39 3136399 0 2024-09-10T04:05:10.301 INFO:tasks.workunit.client.1.smithi070.stdout:0/219: read d8/d24/f4a [819719,95073] 0 2024-09-10T04:05:10.301 INFO:tasks.workunit.client.1.smithi070.stdout:5/234: read d17/d19/f1d [1000345,18590] 0 2024-09-10T04:05:10.302 INFO:tasks.workunit.client.1.smithi070.stdout:4/171: write de/d13/d18/d26/f2a [4347463,69667] 0 2024-09-10T04:05:10.302 INFO:tasks.workunit.client.1.smithi070.stdout:8/258: creat d4/d16/d17/d19/d28/d43/f5d x:0 0 0 2024-09-10T04:05:10.304 INFO:tasks.workunit.client.1.smithi070.stdout:0/220: chown d8/d13/d1a/f30 2 1 2024-09-10T04:05:10.305 INFO:tasks.workunit.client.1.smithi070.stdout:6/295: creat d0/d3/d18/d1b/d17/d28/f5e x:0 0 0 2024-09-10T04:05:10.307 INFO:tasks.workunit.client.1.smithi070.stdout:0/221: read d8/f2b [1573318,100252] 0 2024-09-10T04:05:10.308 INFO:tasks.workunit.client.0.smithi046.stdout:2/252: rename d2/d6/d15/d44 to d2/d6/d23/d5d/d5e 0 2024-09-10T04:05:10.308 INFO:tasks.workunit.client.1.smithi070.stdout:9/227: dwrite d9/d1c/d34/f32 [4194304,4194304] 0 2024-09-10T04:05:10.308 INFO:tasks.workunit.client.0.smithi046.stdout:6/184: dwrite da/fd [0,4194304] 0 2024-09-10T04:05:10.309 INFO:tasks.workunit.client.0.smithi046.stdout:2/253: chown d2/d6/d23/d42/c4a 119259 1 2024-09-10T04:05:10.310 INFO:tasks.workunit.client.1.smithi070.stdout:9/228: read d9/dd/d10/f19 [2839416,75704] 0 2024-09-10T04:05:10.312 INFO:tasks.workunit.client.0.smithi046.stdout:1/143: mkdir d3/dc/d34 0 2024-09-10T04:05:10.313 INFO:tasks.workunit.client.0.smithi046.stdout:1/144: truncate d3/d27/f20 1629105 0 2024-09-10T04:05:10.319 INFO:tasks.workunit.client.1.smithi070.stdout:9/229: dwrite d9/f22 [0,4194304] 0 2024-09-10T04:05:10.320 INFO:tasks.workunit.client.1.smithi070.stdout:9/230: stat d9/dd/d10/d1b 0 2024-09-10T04:05:10.321 INFO:tasks.workunit.client.1.smithi070.stdout:1/187: fdatasync d3/d8/d13/f25 0 2024-09-10T04:05:10.321 INFO:tasks.workunit.client.1.smithi070.stdout:7/175: dread f1 [0,4194304] 0 2024-09-10T04:05:10.322 INFO:tasks.workunit.client.0.smithi046.stdout:4/195: creat d3/d12/d14/f41 x:0 0 0 2024-09-10T04:05:10.323 INFO:tasks.workunit.client.1.smithi070.stdout:3/227: write db/f3f [8772818,30591] 0 2024-09-10T04:05:10.328 INFO:tasks.workunit.client.1.smithi070.stdout:9/231: dread d9/d1c/d34/f32 [0,4194304] 0 2024-09-10T04:05:10.329 INFO:tasks.workunit.client.0.smithi046.stdout:4/196: dwrite d3/f3c [0,4194304] 0 2024-09-10T04:05:10.330 INFO:tasks.workunit.client.1.smithi070.stdout:2/198: creat d15/d18/d29/d2a/f45 x:0 0 0 2024-09-10T04:05:10.330 INFO:tasks.workunit.client.1.smithi070.stdout:9/232: write d9/dd/f18 [3677803,106954] 0 2024-09-10T04:05:10.331 INFO:tasks.workunit.client.1.smithi070.stdout:9/233: chown d9/dd/d10/d1b/c28 38 1 2024-09-10T04:05:10.332 INFO:tasks.workunit.client.1.smithi070.stdout:9/234: fsync d9/dd/f30 0 2024-09-10T04:05:10.339 INFO:tasks.workunit.client.1.smithi070.stdout:9/235: dwrite d9/dd/f30 [0,4194304] 0 2024-09-10T04:05:10.343 INFO:tasks.workunit.client.0.smithi046.stdout:4/197: symlink d3/l42 0 2024-09-10T04:05:10.345 INFO:tasks.workunit.client.1.smithi070.stdout:6/296: rename d0/d3/c4b to d0/d3/d18/d1b/d17/c5f 0 2024-09-10T04:05:10.348 INFO:tasks.workunit.client.0.smithi046.stdout:2/254: mknod d2/d57/c5f 0 2024-09-10T04:05:10.351 INFO:tasks.workunit.client.1.smithi070.stdout:6/297: dwrite d0/d3/d18/d1b/f14 [0,4194304] 0 2024-09-10T04:05:10.351 INFO:tasks.workunit.client.0.smithi046.stdout:1/145: rename d3/fb to d3/d27/d32/d1f/f35 0 2024-09-10T04:05:10.354 INFO:tasks.workunit.client.0.smithi046.stdout:4/198: mkdir d3/d43 0 2024-09-10T04:05:10.356 INFO:tasks.workunit.client.1.smithi070.stdout:3/228: symlink db/d1c/l45 0 2024-09-10T04:05:10.358 INFO:tasks.workunit.client.1.smithi070.stdout:6/298: dwrite d0/d3/f51 [0,4194304] 0 2024-09-10T04:05:10.358 INFO:tasks.workunit.client.1.smithi070.stdout:1/188: symlink d3/d10/d1d/l3a 0 2024-09-10T04:05:10.360 INFO:tasks.workunit.client.1.smithi070.stdout:6/299: chown d0/d3/d18/d1b/d17/d28/l58 464 1 2024-09-10T04:05:10.361 INFO:tasks.workunit.client.1.smithi070.stdout:6/300: chown d0/f57 1 1 2024-09-10T04:05:10.364 INFO:tasks.workunit.client.1.smithi070.stdout:3/229: dwrite db/d1c/d25/f39 [0,4194304] 0 2024-09-10T04:05:10.364 INFO:tasks.workunit.client.0.smithi046.stdout:3/184: getdents d12/d13/d14/d31 0 2024-09-10T04:05:10.365 INFO:tasks.workunit.client.0.smithi046.stdout:3/185: readlink d12/l16 0 2024-09-10T04:05:10.366 INFO:tasks.workunit.client.0.smithi046.stdout:4/199: dread d3/fa [0,4194304] 0 2024-09-10T04:05:10.369 INFO:tasks.workunit.client.0.smithi046.stdout:6/185: getdents da/db/d11 0 2024-09-10T04:05:10.372 INFO:tasks.workunit.client.1.smithi070.stdout:5/235: link d17/d19/f22 d17/d24/d45/f4f 0 2024-09-10T04:05:10.374 INFO:tasks.workunit.client.0.smithi046.stdout:6/186: chown da/db/d11/d2b/d2f/d3b/l3d 3397 1 2024-09-10T04:05:10.379 INFO:tasks.workunit.client.0.smithi046.stdout:3/186: dwrite d12/d13/f2c [0,4194304] 0 2024-09-10T04:05:10.380 INFO:tasks.workunit.client.0.smithi046.stdout:4/200: dwrite d3/d12/d14/f41 [0,4194304] 0 2024-09-10T04:05:10.381 INFO:tasks.workunit.client.1.smithi070.stdout:7/176: rename d2/d7/db/d33 to d2/d7/d9/d18/d3c 0 2024-09-10T04:05:10.387 INFO:tasks.workunit.client.0.smithi046.stdout:4/201: dread d3/f3c [0,4194304] 0 2024-09-10T04:05:10.388 INFO:tasks.workunit.client.1.smithi070.stdout:7/177: dwrite d2/f24 [0,4194304] 0 2024-09-10T04:05:10.389 INFO:tasks.workunit.client.1.smithi070.stdout:7/178: stat d2/d7/d9/d18/d23 0 2024-09-10T04:05:10.398 INFO:tasks.workunit.client.1.smithi070.stdout:6/301: dread d0/d3/d18/f27 [0,4194304] 0 2024-09-10T04:05:10.399 INFO:tasks.workunit.client.0.smithi046.stdout:2/255: truncate d2/f3 1465446 0 2024-09-10T04:05:10.400 INFO:tasks.workunit.client.1.smithi070.stdout:2/199: fsync f10 0 2024-09-10T04:05:10.401 INFO:tasks.workunit.client.1.smithi070.stdout:4/172: creat de/d13/f38 x:0 0 0 2024-09-10T04:05:10.403 INFO:tasks.workunit.client.1.smithi070.stdout:8/259: link d4/fe d4/d16/d17/d1a/d23/f5e 0 2024-09-10T04:05:10.404 INFO:tasks.workunit.client.1.smithi070.stdout:8/260: stat d4/d16/d17/d19/d28/d43/d52 0 2024-09-10T04:05:10.404 INFO:tasks.workunit.client.0.smithi046.stdout:4/202: dwrite d3/f3c [4194304,4194304] 0 2024-09-10T04:05:10.405 INFO:tasks.workunit.client.0.smithi046.stdout:3/187: dread d12/d13/d14/f25 [0,4194304] 0 2024-09-10T04:05:10.406 INFO:tasks.workunit.client.0.smithi046.stdout:1/146: write d3/d27/f18 [2136087,110938] 0 2024-09-10T04:05:10.407 INFO:tasks.workunit.client.0.smithi046.stdout:1/147: readlink d3/l9 0 2024-09-10T04:05:10.409 INFO:tasks.workunit.client.1.smithi070.stdout:7/179: mknod d2/d7/db/c3d 0 2024-09-10T04:05:10.416 INFO:tasks.workunit.client.0.smithi046.stdout:4/203: dwrite d3/d12/d17/d22/f27 [0,4194304] 0 2024-09-10T04:05:10.419 INFO:tasks.workunit.client.0.smithi046.stdout:2/256: link l0 d2/d6/d23/d52/l60 0 2024-09-10T04:05:10.420 INFO:tasks.workunit.client.1.smithi070.stdout:6/302: dread d0/f8 [0,4194304] 0 2024-09-10T04:05:10.424 INFO:tasks.workunit.client.1.smithi070.stdout:4/173: mkdir de/d13/d18/d23/d39 0 2024-09-10T04:05:10.424 INFO:tasks.workunit.client.1.smithi070.stdout:4/174: dread - de/f28 zero size 2024-09-10T04:05:10.425 INFO:tasks.workunit.client.0.smithi046.stdout:6/187: dread da/db/d11/d13/d1e/d23/f2e [0,4194304] 0 2024-09-10T04:05:10.428 INFO:tasks.workunit.client.0.smithi046.stdout:1/148: dread d3/d27/d32/d28/f2c [0,4194304] 0 2024-09-10T04:05:10.429 INFO:tasks.workunit.client.0.smithi046.stdout:1/149: fsync d3/d27/f33 0 2024-09-10T04:05:10.435 INFO:tasks.workunit.client.0.smithi046.stdout:3/188: dread d12/d13/d14/f15 [0,4194304] 0 2024-09-10T04:05:10.484 INFO:tasks.workunit.client.1.smithi070.stdout:0/222: sync 2024-09-10T04:05:10.501 INFO:tasks.workunit.client.1.smithi070.stdout:8/261: unlink d4/d16/d17/d1a/d23/f2f 0 2024-09-10T04:05:10.502 INFO:tasks.workunit.client.1.smithi070.stdout:8/262: chown d4/d16/d17/d1a/d23/d44/c51 190 1 2024-09-10T04:05:10.502 INFO:tasks.workunit.client.0.smithi046.stdout:4/204: mknod d3/c44 0 2024-09-10T04:05:10.503 INFO:tasks.workunit.client.0.smithi046.stdout:4/205: fsync d3/d12/f19 0 2024-09-10T04:05:10.504 INFO:tasks.workunit.client.0.smithi046.stdout:2/257: readlink d2/d6/d23/d5d/d5e/l5c 0 2024-09-10T04:05:10.505 INFO:tasks.workunit.client.1.smithi070.stdout:7/180: dread d2/d3/dc/d1b/f34 [0,4194304] 0 2024-09-10T04:05:10.507 INFO:tasks.workunit.client.1.smithi070.stdout:1/189: creat d3/d10/f3b x:0 0 0 2024-09-10T04:05:10.508 INFO:tasks.workunit.client.1.smithi070.stdout:3/230: write db/f12 [799287,41212] 0 2024-09-10T04:05:10.511 INFO:tasks.workunit.client.0.smithi046.stdout:3/189: symlink d12/d13/l35 0 2024-09-10T04:05:10.513 INFO:tasks.workunit.client.1.smithi070.stdout:5/236: creat d17/d19/f50 x:0 0 0 2024-09-10T04:05:10.514 INFO:tasks.workunit.client.0.smithi046.stdout:1/150: dread d3/d27/d32/d1f/f35 [0,4194304] 0 2024-09-10T04:05:10.516 INFO:tasks.workunit.client.0.smithi046.stdout:4/206: truncate d3/f15 1070410 0 2024-09-10T04:05:10.518 INFO:tasks.workunit.client.1.smithi070.stdout:4/175: rename ca to de/d13/d18/d26/c3a 0 2024-09-10T04:05:10.518 INFO:tasks.workunit.client.0.smithi046.stdout:9/190: sync 2024-09-10T04:05:10.519 INFO:tasks.workunit.client.1.smithi070.stdout:4/176: read de/f31 [453915,127910] 0 2024-09-10T04:05:10.519 INFO:tasks.workunit.client.0.smithi046.stdout:2/258: readlink d2/d6/d23/d3b/l3d 0 2024-09-10T04:05:10.519 INFO:tasks.workunit.client.1.smithi070.stdout:4/177: fdatasync de/d13/d18/f35 0 2024-09-10T04:05:10.519 INFO:tasks.workunit.client.0.smithi046.stdout:9/191: write d4/d7/f20 [609296,42004] 0 2024-09-10T04:05:10.522 INFO:tasks.workunit.client.1.smithi070.stdout:9/236: getdents d9/dd/d10/d1b 0 2024-09-10T04:05:10.523 INFO:tasks.workunit.client.1.smithi070.stdout:7/181: creat d2/d7/d9/d18/d23/f3e x:0 0 0 2024-09-10T04:05:10.524 INFO:tasks.workunit.client.0.smithi046.stdout:6/188: dwrite da/db/d11/d13/d1e/d23/f2e [4194304,4194304] 0 2024-09-10T04:05:10.526 INFO:tasks.workunit.client.0.smithi046.stdout:6/189: write da/fd [2420588,44916] 0 2024-09-10T04:05:10.526 INFO:tasks.workunit.client.1.smithi070.stdout:3/231: symlink db/d1c/l46 0 2024-09-10T04:05:10.527 INFO:tasks.workunit.client.1.smithi070.stdout:6/303: truncate d0/d3/d18/d1b/d17/d28/f3b 1429709 0 2024-09-10T04:05:10.528 INFO:tasks.workunit.client.0.smithi046.stdout:6/190: truncate da/db/d10/d27/f29 428856 0 2024-09-10T04:05:10.531 INFO:tasks.workunit.client.1.smithi070.stdout:0/223: creat d8/d13/d47/f4b x:0 0 0 2024-09-10T04:05:10.532 INFO:tasks.workunit.client.0.smithi046.stdout:4/207: creat d3/d43/f45 x:0 0 0 2024-09-10T04:05:10.532 INFO:tasks.workunit.client.1.smithi070.stdout:4/178: chown de/d13/c30 22842 1 2024-09-10T04:05:10.534 INFO:tasks.workunit.client.1.smithi070.stdout:8/263: mkdir d4/d5f 0 2024-09-10T04:05:10.535 INFO:tasks.workunit.client.1.smithi070.stdout:8/264: readlink d4/d39/l3b 0 2024-09-10T04:05:10.536 INFO:tasks.workunit.client.0.smithi046.stdout:6/191: dwrite da/db/d11/f3a [0,4194304] 0 2024-09-10T04:05:10.537 INFO:tasks.workunit.client.1.smithi070.stdout:8/265: chown d4/d16/d17/d1a/d23/l3e 2312 1 2024-09-10T04:05:10.538 INFO:tasks.workunit.client.1.smithi070.stdout:9/237: rmdir d9/d1c/d39 39 2024-09-10T04:05:10.538 INFO:tasks.workunit.client.0.smithi046.stdout:6/192: dread - da/db/d11/d2b/f2c zero size 2024-09-10T04:05:10.538 INFO:tasks.workunit.client.1.smithi070.stdout:3/232: dread f3 [0,4194304] 0 2024-09-10T04:05:10.539 INFO:tasks.workunit.client.1.smithi070.stdout:9/238: write d9/f35 [1659930,52415] 0 2024-09-10T04:05:10.540 INFO:tasks.workunit.client.1.smithi070.stdout:9/239: stat d9/dd/d10/d3c/f40 0 2024-09-10T04:05:10.541 INFO:tasks.workunit.client.1.smithi070.stdout:7/182: creat d2/d7/d9/d18/f3f x:0 0 0 2024-09-10T04:05:10.545 INFO:tasks.workunit.client.1.smithi070.stdout:1/190: symlink d3/d8/d27/l3c 0 2024-09-10T04:05:10.546 INFO:tasks.workunit.client.0.smithi046.stdout:6/193: dwrite f6 [0,4194304] 0 2024-09-10T04:05:10.548 INFO:tasks.workunit.client.0.smithi046.stdout:6/194: chown da/d3f 314812219 1 2024-09-10T04:05:10.550 INFO:tasks.workunit.client.1.smithi070.stdout:1/191: dwrite d3/d10/f35 [0,4194304] 0 2024-09-10T04:05:10.554 INFO:tasks.workunit.client.0.smithi046.stdout:3/190: creat d12/d13/d14/f36 x:0 0 0 2024-09-10T04:05:10.555 INFO:tasks.workunit.client.0.smithi046.stdout:6/195: dread f9 [0,4194304] 0 2024-09-10T04:05:10.556 INFO:tasks.workunit.client.1.smithi070.stdout:5/237: truncate d17/d19/f23 231247 0 2024-09-10T04:05:10.557 INFO:tasks.workunit.client.1.smithi070.stdout:1/192: dread d3/d8/d13/f18 [0,4194304] 0 2024-09-10T04:05:10.557 INFO:tasks.workunit.client.0.smithi046.stdout:6/196: stat da/db/d11/d2b/d2f/f37 0 2024-09-10T04:05:10.557 INFO:tasks.workunit.client.1.smithi070.stdout:4/179: dread f0 [0,4194304] 0 2024-09-10T04:05:10.558 INFO:tasks.workunit.client.1.smithi070.stdout:5/238: chown d17/d37/l39 6961084 1 2024-09-10T04:05:10.559 INFO:tasks.workunit.client.1.smithi070.stdout:1/193: read - d3/d8/d12/d2f/f34 zero size 2024-09-10T04:05:10.559 INFO:tasks.workunit.client.0.smithi046.stdout:4/208: symlink d3/d12/d14/l46 0 2024-09-10T04:05:10.560 INFO:tasks.workunit.client.0.smithi046.stdout:1/151: truncate d3/d27/d32/f2b 745187 0 2024-09-10T04:05:10.561 INFO:tasks.workunit.client.0.smithi046.stdout:1/152: chown d3/d27/d32/c1d 2027 1 2024-09-10T04:05:10.562 INFO:tasks.workunit.client.0.smithi046.stdout:1/153: dread - d3/d27/f33 zero size 2024-09-10T04:05:10.562 INFO:tasks.workunit.client.1.smithi070.stdout:1/194: dread d3/d10/f35 [0,4194304] 0 2024-09-10T04:05:10.562 INFO:tasks.workunit.client.1.smithi070.stdout:0/224: rename d8/f12 to d8/d24/d36/f4c 0 2024-09-10T04:05:10.564 INFO:tasks.workunit.client.1.smithi070.stdout:0/225: write d8/d13/d47/f4b [176664,47107] 0 2024-09-10T04:05:10.565 INFO:tasks.workunit.client.0.smithi046.stdout:9/192: creat d4/d18/f38 x:0 0 0 2024-09-10T04:05:10.566 INFO:tasks.workunit.client.1.smithi070.stdout:8/266: write d4/d39/f57 [682726,19196] 0 2024-09-10T04:05:10.568 INFO:tasks.workunit.client.0.smithi046.stdout:2/259: link d2/d6/d23/d52/f54 d2/d6/f61 0 2024-09-10T04:05:10.571 INFO:tasks.workunit.client.0.smithi046.stdout:3/191: rename d12/d13/d14/d31 to d12/d13/d37 0 2024-09-10T04:05:10.571 INFO:tasks.workunit.client.0.smithi046.stdout:3/192: chown lc 18 1 2024-09-10T04:05:10.572 INFO:tasks.workunit.client.0.smithi046.stdout:3/193: fdatasync d12/d13/f2c 0 2024-09-10T04:05:10.572 INFO:tasks.workunit.client.1.smithi070.stdout:9/240: write d9/d1c/d34/f33 [1718621,30513] 0 2024-09-10T04:05:10.573 INFO:tasks.workunit.client.1.smithi070.stdout:9/241: chown d9/d1c/c41 53019 1 2024-09-10T04:05:10.573 INFO:tasks.workunit.client.0.smithi046.stdout:3/194: stat d12/d13/d14/d1e/d2d/f2e 0 2024-09-10T04:05:10.573 INFO:tasks.workunit.client.0.smithi046.stdout:4/209: mknod d3/d12/d17/c47 0 2024-09-10T04:05:10.577 INFO:tasks.workunit.client.1.smithi070.stdout:4/180: fsync f8 0 2024-09-10T04:05:10.579 INFO:tasks.workunit.client.1.smithi070.stdout:2/200: sync 2024-09-10T04:05:10.581 INFO:tasks.workunit.client.0.smithi046.stdout:4/210: creat d3/d43/f48 x:0 0 0 2024-09-10T04:05:10.586 INFO:tasks.workunit.client.0.smithi046.stdout:9/193: mknod d4/d18/c39 0 2024-09-10T04:05:10.591 INFO:tasks.workunit.client.0.smithi046.stdout:4/211: unlink d3/d12/d14/c3f 0 2024-09-10T04:05:10.591 INFO:tasks.workunit.client.1.smithi070.stdout:8/267: rmdir d4/d39 39 2024-09-10T04:05:10.592 INFO:tasks.workunit.client.0.smithi046.stdout:4/212: chown d3/d12/d17/d29 48 1 2024-09-10T04:05:10.592 INFO:tasks.workunit.client.1.smithi070.stdout:8/268: write d4/d16/d17/d1a/f58 [111107,102402] 0 2024-09-10T04:05:10.593 INFO:tasks.workunit.client.1.smithi070.stdout:8/269: read - d4/d16/d17/d19/d28/f31 zero size 2024-09-10T04:05:10.596 INFO:tasks.workunit.client.1.smithi070.stdout:9/242: creat d9/f43 x:0 0 0 2024-09-10T04:05:10.597 INFO:tasks.workunit.client.0.smithi046.stdout:4/213: dwrite d3/d12/d14/f18 [0,4194304] 0 2024-09-10T04:05:10.599 INFO:tasks.workunit.client.1.smithi070.stdout:7/183: mkdir d2/d40 0 2024-09-10T04:05:10.603 INFO:tasks.workunit.client.0.smithi046.stdout:1/154: getdents d3/d27 0 2024-09-10T04:05:10.606 INFO:tasks.workunit.client.0.smithi046.stdout:3/195: link d12/d13/d14/l2f d12/d13/l38 0 2024-09-10T04:05:10.609 INFO:tasks.workunit.client.0.smithi046.stdout:1/155: rename d3/f1a to d3/d27/f36 0 2024-09-10T04:05:10.609 INFO:tasks.workunit.client.0.smithi046.stdout:2/260: write d2/d6/f1a [167380,9558] 0 2024-09-10T04:05:10.612 INFO:tasks.workunit.client.0.smithi046.stdout:1/156: dread d3/d27/d32/d1f/f35 [0,4194304] 0 2024-09-10T04:05:10.613 INFO:tasks.workunit.client.0.smithi046.stdout:3/196: mkdir d12/d13/d14/d1e/d2d/d39 0 2024-09-10T04:05:10.613 INFO:tasks.workunit.client.0.smithi046.stdout:1/157: truncate d3/ff 4882391 0 2024-09-10T04:05:10.614 INFO:tasks.workunit.client.0.smithi046.stdout:1/158: stat d3/d27/d32/d1f/f29 0 2024-09-10T04:05:10.614 INFO:tasks.workunit.client.0.smithi046.stdout:1/159: stat d3/dc/c1c 0 2024-09-10T04:05:10.617 INFO:tasks.workunit.client.0.smithi046.stdout:9/194: getdents d4/d7 0 2024-09-10T04:05:10.618 INFO:tasks.workunit.client.0.smithi046.stdout:9/195: read d4/d7/f8 [3695830,68200] 0 2024-09-10T04:05:10.625 INFO:tasks.workunit.client.0.smithi046.stdout:1/160: rename d3/dc/l14 to d3/d27/d32/l37 0 2024-09-10T04:05:10.627 INFO:tasks.workunit.client.0.smithi046.stdout:1/161: dread - d3/d27/f33 zero size 2024-09-10T04:05:10.627 INFO:tasks.workunit.client.0.smithi046.stdout:1/162: write d3/d27/d32/f2e [438258,56767] 0 2024-09-10T04:05:10.629 INFO:tasks.workunit.client.0.smithi046.stdout:3/197: creat d12/d13/d14/f3a x:0 0 0 2024-09-10T04:05:10.630 INFO:tasks.workunit.client.0.smithi046.stdout:2/261: link d2/d6/d15/l2d d2/d49/l62 0 2024-09-10T04:05:10.630 INFO:tasks.workunit.client.0.smithi046.stdout:1/163: fsync d3/f6 0 2024-09-10T04:05:10.631 INFO:tasks.workunit.client.0.smithi046.stdout:9/196: link d4/d29/l35 d4/d7/d1f/l3a 0 2024-09-10T04:05:10.635 INFO:tasks.workunit.client.0.smithi046.stdout:9/197: write d4/d7/d1f/f26 [3978995,72321] 0 2024-09-10T04:05:10.636 INFO:tasks.workunit.client.0.smithi046.stdout:1/164: dread d3/dc/fe [0,4194304] 0 2024-09-10T04:05:10.636 INFO:tasks.workunit.client.0.smithi046.stdout:9/198: chown d4/d7/c2f 1637 1 2024-09-10T04:05:10.643 INFO:tasks.workunit.client.0.smithi046.stdout:2/262: truncate d2/d6/f43 1874075 0 2024-09-10T04:05:10.643 INFO:tasks.workunit.client.1.smithi070.stdout:7/184: rmdir d2/d7/d9/d18 39 2024-09-10T04:05:10.643 INFO:tasks.workunit.client.1.smithi070.stdout:8/270: chown d4/d16/d17/d1a/d23/d44/l56 28468424 1 2024-09-10T04:05:10.644 INFO:tasks.workunit.client.0.smithi046.stdout:1/165: getdents d3/d27/d32 0 2024-09-10T04:05:10.645 INFO:tasks.workunit.client.1.smithi070.stdout:7/185: readlink d2/d7/db/lf 0 2024-09-10T04:05:10.645 INFO:tasks.workunit.client.0.smithi046.stdout:2/263: mkdir d2/d8/d37/d63 0 2024-09-10T04:05:10.646 INFO:tasks.workunit.client.1.smithi070.stdout:3/233: getdents db/d1c 0 2024-09-10T04:05:10.646 INFO:tasks.workunit.client.1.smithi070.stdout:5/239: dread d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:10.647 INFO:tasks.workunit.client.0.smithi046.stdout:2/264: chown d2/d8/l30 18 1 2024-09-10T04:05:10.647 INFO:tasks.workunit.client.0.smithi046.stdout:1/166: creat d3/d27/f38 x:0 0 0 2024-09-10T04:05:10.648 INFO:tasks.workunit.client.0.smithi046.stdout:9/199: dwrite d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:10.651 INFO:tasks.workunit.client.0.smithi046.stdout:9/200: symlink d4/d7/l3b 0 2024-09-10T04:05:10.651 INFO:tasks.workunit.client.1.smithi070.stdout:2/201: link d15/d18/f19 d15/d18/d29/f46 0 2024-09-10T04:05:10.651 INFO:tasks.workunit.client.1.smithi070.stdout:3/234: chown db/d1c/d25/c38 443 1 2024-09-10T04:05:10.653 INFO:tasks.workunit.client.1.smithi070.stdout:1/195: dwrite d3/d8/d13/f25 [0,4194304] 0 2024-09-10T04:05:10.653 INFO:tasks.workunit.client.0.smithi046.stdout:1/167: dread d3/dc/f17 [0,4194304] 0 2024-09-10T04:05:10.654 INFO:tasks.workunit.client.1.smithi070.stdout:7/186: mkdir d2/d3/dc/d1b/d41 0 2024-09-10T04:05:10.657 INFO:tasks.workunit.client.1.smithi070.stdout:0/226: dwrite d8/d13/f22 [0,4194304] 0 2024-09-10T04:05:10.657 INFO:tasks.workunit.client.0.smithi046.stdout:9/201: write d4/d18/f22 [3761492,42452] 0 2024-09-10T04:05:10.657 INFO:tasks.workunit.client.0.smithi046.stdout:1/168: mknod d3/dc/c39 0 2024-09-10T04:05:10.660 INFO:tasks.workunit.client.0.smithi046.stdout:1/169: write d3/d27/f38 [781707,104484] 0 2024-09-10T04:05:10.660 INFO:tasks.workunit.client.0.smithi046.stdout:9/202: chown d4/d18/c2d 8471 1 2024-09-10T04:05:10.661 INFO:tasks.workunit.client.1.smithi070.stdout:9/243: dwrite d9/dd/d10/f20 [0,4194304] 0 2024-09-10T04:05:10.661 INFO:tasks.workunit.client.1.smithi070.stdout:7/187: dread - d2/d3/f3b zero size 2024-09-10T04:05:10.662 INFO:tasks.workunit.client.0.smithi046.stdout:1/170: chown d3/d27/c24 19 1 2024-09-10T04:05:10.663 INFO:tasks.workunit.client.1.smithi070.stdout:0/227: dread - d8/f11 zero size 2024-09-10T04:05:10.663 INFO:tasks.workunit.client.1.smithi070.stdout:1/196: mknod d3/d8/d27/c3d 0 2024-09-10T04:05:10.664 INFO:tasks.workunit.client.0.smithi046.stdout:1/171: write d3/f26 [19644,124626] 0 2024-09-10T04:05:10.668 INFO:tasks.workunit.client.1.smithi070.stdout:5/240: dwrite d17/d24/f25 [0,4194304] 0 2024-09-10T04:05:10.673 INFO:tasks.workunit.client.0.smithi046.stdout:2/265: dread d2/d8/f27 [0,4194304] 0 2024-09-10T04:05:10.674 INFO:tasks.workunit.client.0.smithi046.stdout:2/266: chown d2/d6/d23/l24 0 1 2024-09-10T04:05:10.674 INFO:tasks.workunit.client.0.smithi046.stdout:2/267: readlink d2/d8/l11 0 2024-09-10T04:05:10.676 INFO:tasks.workunit.client.0.smithi046.stdout:2/268: read d2/d8/f27 [1622455,103511] 0 2024-09-10T04:05:10.681 INFO:tasks.workunit.client.1.smithi070.stdout:6/304: sync 2024-09-10T04:05:10.681 INFO:tasks.workunit.client.1.smithi070.stdout:5/241: creat d17/d24/d45/f51 x:0 0 0 2024-09-10T04:05:10.682 INFO:tasks.workunit.client.1.smithi070.stdout:3/235: dread db/f1f [0,4194304] 0 2024-09-10T04:05:10.682 INFO:tasks.workunit.client.0.smithi046.stdout:2/269: mknod d2/d6/d23/d3b/c64 0 2024-09-10T04:05:10.683 INFO:tasks.workunit.client.0.smithi046.stdout:1/172: dread d3/d27/f20 [0,4194304] 0 2024-09-10T04:05:10.685 INFO:tasks.workunit.client.1.smithi070.stdout:8/271: truncate d4/d16/d17/f2c 3989598 0 2024-09-10T04:05:10.686 INFO:tasks.workunit.client.0.smithi046.stdout:9/203: dwrite d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:10.687 INFO:tasks.workunit.client.1.smithi070.stdout:1/197: mknod d3/d10/c3e 0 2024-09-10T04:05:10.687 INFO:tasks.workunit.client.1.smithi070.stdout:2/202: dread f10 [0,4194304] 0 2024-09-10T04:05:10.687 INFO:tasks.workunit.client.1.smithi070.stdout:6/305: symlink d0/d3/d18/d1b/d33/l60 0 2024-09-10T04:05:10.688 INFO:tasks.workunit.client.0.smithi046.stdout:2/270: write d2/f35 [7449324,118335] 0 2024-09-10T04:05:10.691 INFO:tasks.workunit.client.1.smithi070.stdout:7/188: dwrite d2/d7/d9/d18/d23/f38 [0,4194304] 0 2024-09-10T04:05:10.691 INFO:tasks.workunit.client.1.smithi070.stdout:6/306: chown d0/d3/d18/d1b/d33/l60 1195326 1 2024-09-10T04:05:10.691 INFO:tasks.workunit.client.1.smithi070.stdout:3/236: unlink db/d1c/d3e/f44 0 2024-09-10T04:05:10.691 INFO:tasks.workunit.client.0.smithi046.stdout:1/173: fdatasync d3/d27/d32/d1f/f29 0 2024-09-10T04:05:10.694 INFO:tasks.workunit.client.0.smithi046.stdout:9/204: chown d4/d7/f31 287 1 2024-09-10T04:05:10.695 INFO:tasks.workunit.client.1.smithi070.stdout:7/189: write d2/d7/d9/d18/f3f [549100,47116] 0 2024-09-10T04:05:10.695 INFO:tasks.workunit.client.1.smithi070.stdout:3/237: read - db/d1c/d3e/f2b zero size 2024-09-10T04:05:10.700 INFO:tasks.workunit.client.1.smithi070.stdout:6/307: dread d0/f2e [0,4194304] 0 2024-09-10T04:05:10.701 INFO:tasks.workunit.client.1.smithi070.stdout:8/272: readlink d4/d16/d17/l20 0 2024-09-10T04:05:10.702 INFO:tasks.workunit.client.0.smithi046.stdout:9/205: dwrite d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:10.703 INFO:tasks.workunit.client.1.smithi070.stdout:9/244: getdents d9/d1c 0 2024-09-10T04:05:10.706 INFO:tasks.workunit.client.1.smithi070.stdout:1/198: mkdir d3/d8/d13/d3f 0 2024-09-10T04:05:10.709 INFO:tasks.workunit.client.0.smithi046.stdout:2/271: mkdir d2/d8/d65 0 2024-09-10T04:05:10.712 INFO:tasks.workunit.client.0.smithi046.stdout:2/272: dread - d2/d6/d23/f4b zero size 2024-09-10T04:05:10.712 INFO:tasks.workunit.client.0.smithi046.stdout:1/174: dwrite f1 [0,4194304] 0 2024-09-10T04:05:10.714 INFO:tasks.workunit.client.0.smithi046.stdout:9/206: mkdir d4/d3c 0 2024-09-10T04:05:10.714 INFO:tasks.workunit.client.0.smithi046.stdout:1/175: stat d3/d27/d32/f2b 0 2024-09-10T04:05:10.720 INFO:tasks.workunit.client.1.smithi070.stdout:3/238: symlink db/d1c/l47 0 2024-09-10T04:05:10.722 INFO:tasks.workunit.client.0.smithi046.stdout:1/176: unlink d3/dc/f22 0 2024-09-10T04:05:10.722 INFO:tasks.workunit.client.0.smithi046.stdout:1/177: stat d3/dc/f23 0 2024-09-10T04:05:10.724 INFO:tasks.workunit.client.1.smithi070.stdout:5/242: truncate d17/d24/f2f 2413044 0 2024-09-10T04:05:10.724 INFO:tasks.workunit.client.1.smithi070.stdout:5/243: chown d17/d24/f2a 31685850 1 2024-09-10T04:05:10.727 INFO:tasks.workunit.client.0.smithi046.stdout:2/273: creat d2/d8/d37/f66 x:0 0 0 2024-09-10T04:05:10.727 INFO:tasks.workunit.client.0.smithi046.stdout:1/178: dread d3/ff [0,4194304] 0 2024-09-10T04:05:10.731 INFO:tasks.workunit.client.1.smithi070.stdout:7/190: dwrite d2/d3/dc/d1b/f34 [0,4194304] 0 2024-09-10T04:05:10.731 INFO:tasks.workunit.client.0.smithi046.stdout:9/207: link d4/d18/f38 d4/d29/f3d 0 2024-09-10T04:05:10.733 INFO:tasks.workunit.client.0.smithi046.stdout:2/274: creat d2/d6/d23/d55/f67 x:0 0 0 2024-09-10T04:05:10.738 INFO:tasks.workunit.client.1.smithi070.stdout:3/239: rmdir db 39 2024-09-10T04:05:10.738 INFO:tasks.workunit.client.0.smithi046.stdout:2/275: dread d2/d6/d23/d3b/f47 [0,4194304] 0 2024-09-10T04:05:10.741 INFO:tasks.workunit.client.0.smithi046.stdout:9/208: dread - d4/d7/f1e zero size 2024-09-10T04:05:10.742 INFO:tasks.workunit.client.1.smithi070.stdout:6/308: creat d0/d3/d18/d1b/d17/d53/f61 x:0 0 0 2024-09-10T04:05:10.746 INFO:tasks.workunit.client.1.smithi070.stdout:0/228: sync 2024-09-10T04:05:10.746 INFO:tasks.workunit.client.1.smithi070.stdout:5/244: rename d17/f1b to d17/d24/d2b/d31/f52 0 2024-09-10T04:05:10.747 INFO:tasks.workunit.client.1.smithi070.stdout:0/229: truncate d8/d24/f2a 4532929 0 2024-09-10T04:05:10.747 INFO:tasks.workunit.client.1.smithi070.stdout:5/245: dread - d17/d19/f44 zero size 2024-09-10T04:05:10.747 INFO:tasks.workunit.client.1.smithi070.stdout:0/230: readlink d8/d13/l17 0 2024-09-10T04:05:10.748 INFO:tasks.workunit.client.0.smithi046.stdout:1/179: mkdir d3/dc/d21/d3a 0 2024-09-10T04:05:10.749 INFO:tasks.workunit.client.1.smithi070.stdout:9/245: link d9/dd/d10/d1b/f2d d9/d1c/d39/f44 0 2024-09-10T04:05:10.749 INFO:tasks.workunit.client.0.smithi046.stdout:2/276: mknod d2/c68 0 2024-09-10T04:05:10.750 INFO:tasks.workunit.client.1.smithi070.stdout:1/199: mknod d3/d20/c40 0 2024-09-10T04:05:10.754 INFO:tasks.workunit.client.1.smithi070.stdout:0/231: dwrite d8/f1b [4194304,4194304] 0 2024-09-10T04:05:10.754 INFO:tasks.workunit.client.0.smithi046.stdout:1/180: rename d3/c8 to d3/d27/d32/d28/c3b 0 2024-09-10T04:05:10.755 INFO:tasks.workunit.client.1.smithi070.stdout:7/191: mkdir d2/d3/dc/d1b/d42 0 2024-09-10T04:05:10.756 INFO:tasks.workunit.client.1.smithi070.stdout:0/232: dread - d8/d24/f43 zero size 2024-09-10T04:05:10.756 INFO:tasks.workunit.client.1.smithi070.stdout:8/273: dwrite d4/d16/d17/f2c [0,4194304] 0 2024-09-10T04:05:10.757 INFO:tasks.workunit.client.1.smithi070.stdout:0/233: stat f0 0 2024-09-10T04:05:10.759 INFO:tasks.workunit.client.0.smithi046.stdout:1/181: dread d3/d27/d32/d28/f2c [0,4194304] 0 2024-09-10T04:05:10.759 INFO:tasks.workunit.client.1.smithi070.stdout:0/234: fsync d8/d13/f19 0 2024-09-10T04:05:10.761 INFO:tasks.workunit.client.1.smithi070.stdout:1/200: dwrite d3/d8/d12/f19 [0,4194304] 0 2024-09-10T04:05:10.765 INFO:tasks.workunit.client.0.smithi046.stdout:9/209: dread d4/d7/ff [0,4194304] 0 2024-09-10T04:05:10.765 INFO:tasks.workunit.client.1.smithi070.stdout:1/201: write d3/d8/d13/f25 [2088898,49729] 0 2024-09-10T04:05:10.767 INFO:tasks.workunit.client.1.smithi070.stdout:5/246: creat d17/d24/d47/f53 x:0 0 0 2024-09-10T04:05:10.770 INFO:tasks.workunit.client.0.smithi046.stdout:9/210: symlink d4/d29/d32/l3e 0 2024-09-10T04:05:10.770 INFO:tasks.workunit.client.1.smithi070.stdout:8/274: dwrite d4/d16/d17/d19/d28/f46 [0,4194304] 0 2024-09-10T04:05:10.772 INFO:tasks.workunit.client.0.smithi046.stdout:9/211: truncate d4/d7/f1e 1005630 0 2024-09-10T04:05:10.773 INFO:tasks.workunit.client.0.smithi046.stdout:2/277: rename d2/d6/d23/d3b/f47 to d2/d6/d23/d52/f69 0 2024-09-10T04:05:10.775 INFO:tasks.workunit.client.1.smithi070.stdout:2/203: getdents d15/d18/d29/d2a 0 2024-09-10T04:05:10.776 INFO:tasks.workunit.client.0.smithi046.stdout:9/212: readlink d4/d7/l2e 0 2024-09-10T04:05:10.778 INFO:tasks.workunit.client.1.smithi070.stdout:3/240: symlink db/l48 0 2024-09-10T04:05:10.779 INFO:tasks.workunit.client.1.smithi070.stdout:6/309: dread d0/d3/d18/d1b/d17/d28/f3b [0,4194304] 0 2024-09-10T04:05:10.780 INFO:tasks.workunit.client.1.smithi070.stdout:0/235: mkdir d8/d13/d1a/d4d 0 2024-09-10T04:05:10.780 INFO:tasks.workunit.client.1.smithi070.stdout:0/236: chown d8/d13/f3d 383 1 2024-09-10T04:05:10.782 INFO:tasks.workunit.client.0.smithi046.stdout:9/213: symlink d4/d7/l3f 0 2024-09-10T04:05:10.782 INFO:tasks.workunit.client.1.smithi070.stdout:2/204: dread fc [0,4194304] 0 2024-09-10T04:05:10.784 INFO:tasks.workunit.client.0.smithi046.stdout:2/278: creat d2/d6/d15/f6a x:0 0 0 2024-09-10T04:05:10.784 INFO:tasks.workunit.client.1.smithi070.stdout:5/247: mkdir d17/d24/d47/d54 0 2024-09-10T04:05:10.785 INFO:tasks.workunit.client.0.smithi046.stdout:9/214: write d4/d7/f23 [2349413,12916] 0 2024-09-10T04:05:10.786 INFO:tasks.workunit.client.1.smithi070.stdout:8/275: mkdir d4/d16/d17/d1a/d60 0 2024-09-10T04:05:10.788 INFO:tasks.workunit.client.0.smithi046.stdout:2/279: mknod d2/d6/d23/d3b/c6b 0 2024-09-10T04:05:10.789 INFO:tasks.workunit.client.1.smithi070.stdout:9/246: link d9/dd/f18 d9/dd/d10/d1b/f45 0 2024-09-10T04:05:10.789 INFO:tasks.workunit.client.1.smithi070.stdout:9/247: fdatasync d9/d1c/f2c 0 2024-09-10T04:05:10.791 INFO:tasks.workunit.client.0.smithi046.stdout:9/215: write d4/fb [1867266,106392] 0 2024-09-10T04:05:10.792 INFO:tasks.workunit.client.1.smithi070.stdout:3/241: mknod db/d1c/c49 0 2024-09-10T04:05:10.793 INFO:tasks.workunit.client.1.smithi070.stdout:6/310: fsync d0/d3/d18/f27 0 2024-09-10T04:05:10.794 INFO:tasks.workunit.client.1.smithi070.stdout:0/237: fsync d8/d24/d36/f4c 0 2024-09-10T04:05:10.797 INFO:tasks.workunit.client.1.smithi070.stdout:9/248: dread d9/dd/d10/f1f [0,4194304] 0 2024-09-10T04:05:10.798 INFO:tasks.workunit.client.1.smithi070.stdout:6/311: dread d0/d3/d18/d1b/f11 [4194304,4194304] 0 2024-09-10T04:05:10.798 INFO:tasks.workunit.client.0.smithi046.stdout:9/216: creat d4/d3c/f40 x:0 0 0 2024-09-10T04:05:10.798 INFO:tasks.workunit.client.1.smithi070.stdout:9/249: write d9/dd/f1a [627357,74363] 0 2024-09-10T04:05:10.799 INFO:tasks.workunit.client.0.smithi046.stdout:2/280: dwrite d2/f35 [4194304,4194304] 0 2024-09-10T04:05:10.800 INFO:tasks.workunit.client.1.smithi070.stdout:7/192: creat d2/d7/f43 x:0 0 0 2024-09-10T04:05:10.800 INFO:tasks.workunit.client.1.smithi070.stdout:7/193: fsync d2/d3/dc/d1b/f1d 0 2024-09-10T04:05:10.805 INFO:tasks.workunit.client.1.smithi070.stdout:3/242: truncate db/f22 2971353 0 2024-09-10T04:05:10.806 INFO:tasks.workunit.client.1.smithi070.stdout:3/243: read db/d1c/d25/f39 [1121099,1269] 0 2024-09-10T04:05:10.806 INFO:tasks.workunit.client.0.smithi046.stdout:9/217: symlink d4/d7/l41 0 2024-09-10T04:05:10.809 INFO:tasks.workunit.client.0.smithi046.stdout:2/281: creat d2/d8/d37/f6c x:0 0 0 2024-09-10T04:05:10.810 INFO:tasks.workunit.client.1.smithi070.stdout:0/238: creat d8/d24/d2d/d34/f4e x:0 0 0 2024-09-10T04:05:10.811 INFO:tasks.workunit.client.1.smithi070.stdout:0/239: chown d8/d24/d36/c3b 15752 1 2024-09-10T04:05:10.813 INFO:tasks.workunit.client.1.smithi070.stdout:2/205: write f10 [1219655,94408] 0 2024-09-10T04:05:10.813 INFO:tasks.workunit.client.0.smithi046.stdout:2/282: dread d2/f2c [0,4194304] 0 2024-09-10T04:05:10.814 INFO:tasks.workunit.client.1.smithi070.stdout:5/248: write f13 [2898476,56997] 0 2024-09-10T04:05:10.815 INFO:tasks.workunit.client.1.smithi070.stdout:8/276: mknod d4/c61 0 2024-09-10T04:05:10.816 INFO:tasks.workunit.client.1.smithi070.stdout:8/277: dread - d4/d16/d17/d19/d28/f31 zero size 2024-09-10T04:05:10.817 INFO:tasks.workunit.client.0.smithi046.stdout:9/218: rename d4/c1a to d4/c42 0 2024-09-10T04:05:10.820 INFO:tasks.workunit.client.0.smithi046.stdout:2/283: rename d2/d8/l11 to d2/d6/d23/d55/l6d 0 2024-09-10T04:05:10.821 INFO:tasks.workunit.client.1.smithi070.stdout:9/250: rmdir d9/d1c 39 2024-09-10T04:05:10.822 INFO:tasks.workunit.client.0.smithi046.stdout:9/219: truncate d4/d7/f20 1704462 0 2024-09-10T04:05:10.823 INFO:tasks.workunit.client.0.smithi046.stdout:9/220: dread - d4/d3c/f40 zero size 2024-09-10T04:05:10.824 INFO:tasks.workunit.client.0.smithi046.stdout:2/284: symlink d2/d6/d23/d52/l6e 0 2024-09-10T04:05:10.826 INFO:tasks.workunit.client.1.smithi070.stdout:7/194: chown d2/l11 48 1 2024-09-10T04:05:10.826 INFO:tasks.workunit.client.1.smithi070.stdout:7/195: dread - d2/d3/f3b zero size 2024-09-10T04:05:10.826 INFO:tasks.workunit.client.1.smithi070.stdout:7/196: read - d2/d3/f3b zero size 2024-09-10T04:05:10.829 INFO:tasks.workunit.client.0.smithi046.stdout:9/221: write d4/d7/ff [4634897,69031] 0 2024-09-10T04:05:10.829 INFO:tasks.workunit.client.1.smithi070.stdout:3/244: truncate db/f27 406681 0 2024-09-10T04:05:10.831 INFO:tasks.workunit.client.1.smithi070.stdout:0/240: mkdir d8/d13/d33/d4f 0 2024-09-10T04:05:10.832 INFO:tasks.workunit.client.0.smithi046.stdout:2/285: creat d2/d8/f6f x:0 0 0 2024-09-10T04:05:10.833 INFO:tasks.workunit.client.1.smithi070.stdout:2/206: unlink d15/d18/d29/f35 0 2024-09-10T04:05:10.834 INFO:tasks.workunit.client.0.smithi046.stdout:9/222: creat d4/d29/d32/f43 x:0 0 0 2024-09-10T04:05:10.834 INFO:tasks.workunit.client.1.smithi070.stdout:8/278: mkdir d4/d16/d17/d19/d62 0 2024-09-10T04:05:10.835 INFO:tasks.workunit.client.1.smithi070.stdout:6/312: mkdir d0/d3/d18/d62 0 2024-09-10T04:05:10.836 INFO:tasks.workunit.client.0.smithi046.stdout:9/223: read d4/d7/f8 [1581147,68478] 0 2024-09-10T04:05:10.839 INFO:tasks.workunit.client.0.smithi046.stdout:2/286: creat d2/d6/d15/f70 x:0 0 0 2024-09-10T04:05:10.840 INFO:tasks.workunit.client.1.smithi070.stdout:7/197: rename d2/d3/dc/l2a to d2/d3/dc/d1b/d42/l44 0 2024-09-10T04:05:10.840 INFO:tasks.workunit.client.1.smithi070.stdout:7/198: dread - d2/d3/f3b zero size 2024-09-10T04:05:10.841 INFO:tasks.workunit.client.1.smithi070.stdout:7/199: chown d2 83 1 2024-09-10T04:05:10.842 INFO:tasks.workunit.client.1.smithi070.stdout:7/200: write d2/d7/d9/f31 [339466,36114] 0 2024-09-10T04:05:10.842 INFO:tasks.workunit.client.0.smithi046.stdout:9/224: dwrite d4/d18/f22 [0,4194304] 0 2024-09-10T04:05:10.843 INFO:tasks.workunit.client.1.smithi070.stdout:0/241: unlink f0 0 2024-09-10T04:05:10.844 INFO:tasks.workunit.client.1.smithi070.stdout:0/242: fsync d8/d24/d2d/d34/f4e 0 2024-09-10T04:05:10.846 INFO:tasks.workunit.client.0.smithi046.stdout:2/287: mknod d2/d6/d15/c71 0 2024-09-10T04:05:10.847 INFO:tasks.workunit.client.1.smithi070.stdout:9/251: symlink d9/d3e/l46 0 2024-09-10T04:05:10.849 INFO:tasks.workunit.client.1.smithi070.stdout:7/201: mknod d2/d7/d9/d18/c45 0 2024-09-10T04:05:10.851 INFO:tasks.workunit.client.1.smithi070.stdout:8/279: rename d4/d16/d17/d1a/d23/l45 to d4/d39/l63 0 2024-09-10T04:05:10.852 INFO:tasks.workunit.client.1.smithi070.stdout:0/243: dwrite d8/f27 [0,4194304] 0 2024-09-10T04:05:10.853 INFO:tasks.workunit.client.0.smithi046.stdout:2/288: link d2/d6/d23/f33 d2/d6/d23/d5d/d5e/f72 0 2024-09-10T04:05:10.853 INFO:tasks.workunit.client.1.smithi070.stdout:0/244: dread - d8/d13/f3d zero size 2024-09-10T04:05:10.854 INFO:tasks.workunit.client.0.smithi046.stdout:9/225: getdents d4/d7/d1f 0 2024-09-10T04:05:10.856 INFO:tasks.workunit.client.1.smithi070.stdout:2/207: getdents d15/d18/d29/d2a 0 2024-09-10T04:05:10.858 INFO:tasks.workunit.client.1.smithi070.stdout:7/202: creat d2/d3/dc/d1b/f46 x:0 0 0 2024-09-10T04:05:10.860 INFO:tasks.workunit.client.1.smithi070.stdout:0/245: mknod d8/d24/d2d/c50 0 2024-09-10T04:05:10.862 INFO:tasks.workunit.client.1.smithi070.stdout:3/245: getdents db/d1c/d25 0 2024-09-10T04:05:10.862 INFO:tasks.workunit.client.0.smithi046.stdout:9/226: mkdir d4/d44 0 2024-09-10T04:05:10.862 INFO:tasks.workunit.client.1.smithi070.stdout:6/313: dread d0/f10 [0,4194304] 0 2024-09-10T04:05:10.865 INFO:tasks.workunit.client.0.smithi046.stdout:2/289: dwrite d2/d6/d23/d42/f4e [0,4194304] 0 2024-09-10T04:05:10.865 INFO:tasks.workunit.client.1.smithi070.stdout:7/203: unlink d2/f15 0 2024-09-10T04:05:10.866 INFO:tasks.workunit.client.1.smithi070.stdout:7/204: readlink d2/d7/d9/d18/l28 0 2024-09-10T04:05:10.868 INFO:tasks.workunit.client.0.smithi046.stdout:2/290: truncate d2/d6/f1a 4724488 0 2024-09-10T04:05:10.869 INFO:tasks.workunit.client.1.smithi070.stdout:6/314: dwrite d0/d3/d18/d2a/f4f [0,4194304] 0 2024-09-10T04:05:10.869 INFO:tasks.workunit.client.1.smithi070.stdout:0/246: creat d8/d24/d2d/f51 x:0 0 0 2024-09-10T04:05:10.871 INFO:tasks.workunit.client.0.smithi046.stdout:9/227: dwrite d4/d29/d32/f43 [0,4194304] 0 2024-09-10T04:05:10.876 INFO:tasks.workunit.client.1.smithi070.stdout:0/247: dwrite d8/f49 [0,4194304] 0 2024-09-10T04:05:10.878 INFO:tasks.workunit.client.1.smithi070.stdout:8/280: rename d4/d16/d17/d19/c3d to d4/c64 0 2024-09-10T04:05:10.880 INFO:tasks.workunit.client.0.smithi046.stdout:9/228: symlink d4/l45 0 2024-09-10T04:05:10.881 INFO:tasks.workunit.client.0.smithi046.stdout:9/229: chown d4/d18/f33 1802 1 2024-09-10T04:05:10.883 INFO:tasks.workunit.client.1.smithi070.stdout:7/205: creat d2/d7/d9/d18/d23/f47 x:0 0 0 2024-09-10T04:05:10.884 INFO:tasks.workunit.client.1.smithi070.stdout:6/315: symlink d0/d3/d18/d1b/d17/l63 0 2024-09-10T04:05:10.886 INFO:tasks.workunit.client.1.smithi070.stdout:6/316: read d0/d3/d18/d1b/d17/f47 [194672,30882] 0 2024-09-10T04:05:10.889 INFO:tasks.workunit.client.1.smithi070.stdout:9/252: getdents d9/d3e 0 2024-09-10T04:05:10.890 INFO:tasks.workunit.client.0.smithi046.stdout:9/230: write d4/d7/f31 [1053997,1012] 0 2024-09-10T04:05:10.890 INFO:tasks.workunit.client.1.smithi070.stdout:2/208: link d15/d18/d29/d2a/d2e/d32/d36/c37 d15/d18/d29/d2a/d2e/d32/d36/c47 0 2024-09-10T04:05:10.891 INFO:tasks.workunit.client.1.smithi070.stdout:2/209: read d15/d18/f19 [1551346,88410] 0 2024-09-10T04:05:10.894 INFO:tasks.workunit.client.1.smithi070.stdout:0/248: creat d8/d24/d2d/f52 x:0 0 0 2024-09-10T04:05:10.895 INFO:tasks.workunit.client.1.smithi070.stdout:0/249: chown d8/d24 1 1 2024-09-10T04:05:10.897 INFO:tasks.workunit.client.0.smithi046.stdout:9/231: creat d4/d3c/f46 x:0 0 0 2024-09-10T04:05:10.900 INFO:tasks.workunit.client.1.smithi070.stdout:3/246: link db/c15 db/d1c/d25/d28/c4a 0 2024-09-10T04:05:10.906 INFO:tasks.workunit.client.1.smithi070.stdout:3/247: dwrite db/d1c/d3e/f2b [0,4194304] 0 2024-09-10T04:05:10.906 INFO:tasks.workunit.client.0.smithi046.stdout:9/232: link d4/d7/d1f/c2a d4/d7/d1f/c47 0 2024-09-10T04:05:10.908 INFO:tasks.workunit.client.1.smithi070.stdout:0/250: write d8/fc [962776,124870] 0 2024-09-10T04:05:10.911 INFO:tasks.workunit.client.1.smithi070.stdout:8/281: dread d4/d16/d17/f40 [0,4194304] 0 2024-09-10T04:05:10.915 INFO:tasks.workunit.client.1.smithi070.stdout:3/248: creat db/d1c/f4b x:0 0 0 2024-09-10T04:05:10.916 INFO:tasks.workunit.client.1.smithi070.stdout:3/249: write db/d1c/d3e/f30 [418795,5909] 0 2024-09-10T04:05:10.918 INFO:tasks.workunit.client.1.smithi070.stdout:9/253: link d9/dd/d10/d1b/f45 d9/d3e/f47 0 2024-09-10T04:05:10.920 INFO:tasks.workunit.client.0.smithi046.stdout:9/233: link d4/d7/c34 d4/d7/c48 0 2024-09-10T04:05:10.921 INFO:tasks.workunit.client.1.smithi070.stdout:2/210: symlink d15/d2b/l48 0 2024-09-10T04:05:10.922 INFO:tasks.workunit.client.0.smithi046.stdout:9/234: fsync d4/d29/f3d 0 2024-09-10T04:05:10.924 INFO:tasks.workunit.client.1.smithi070.stdout:9/254: dwrite d9/f22 [0,4194304] 0 2024-09-10T04:05:10.925 INFO:tasks.workunit.client.1.smithi070.stdout:9/255: read d9/dd/d10/f1f [1205808,83194] 0 2024-09-10T04:05:10.925 INFO:tasks.workunit.client.1.smithi070.stdout:7/206: rename d2/d3/l30 to d2/d3/dc/l48 0 2024-09-10T04:05:10.927 INFO:tasks.workunit.client.1.smithi070.stdout:3/250: mknod db/d1c/d3e/c4c 0 2024-09-10T04:05:10.929 INFO:tasks.workunit.client.1.smithi070.stdout:0/251: dwrite d8/d13/d1a/f30 [0,4194304] 0 2024-09-10T04:05:10.930 INFO:tasks.workunit.client.1.smithi070.stdout:8/282: symlink d4/d16/d17/d5c/l65 0 2024-09-10T04:05:10.930 INFO:tasks.workunit.client.0.smithi046.stdout:0/192: sync 2024-09-10T04:05:10.930 INFO:tasks.workunit.client.0.smithi046.stdout:8/211: sync 2024-09-10T04:05:10.930 INFO:tasks.workunit.client.0.smithi046.stdout:5/322: sync 2024-09-10T04:05:10.931 INFO:tasks.workunit.client.0.smithi046.stdout:7/220: sync 2024-09-10T04:05:10.931 INFO:tasks.workunit.client.1.smithi070.stdout:0/252: fdatasync d8/d24/f2a 0 2024-09-10T04:05:10.931 INFO:tasks.workunit.client.0.smithi046.stdout:9/235: creat d4/d44/f49 x:0 0 0 2024-09-10T04:05:10.933 INFO:tasks.workunit.client.0.smithi046.stdout:8/212: chown d1/d29/d19/l3d 124 1 2024-09-10T04:05:10.933 INFO:tasks.workunit.client.1.smithi070.stdout:2/211: creat d15/d18/d29/d2a/d2e/d32/f49 x:0 0 0 2024-09-10T04:05:10.934 INFO:tasks.workunit.client.0.smithi046.stdout:9/236: write d4/d18/f22 [3807568,62216] 0 2024-09-10T04:05:10.934 INFO:tasks.workunit.client.1.smithi070.stdout:2/212: chown d15/d18/f26 955990673 1 2024-09-10T04:05:10.935 INFO:tasks.workunit.client.1.smithi070.stdout:0/253: dread d8/f1b [4194304,4194304] 0 2024-09-10T04:05:10.936 INFO:tasks.workunit.client.1.smithi070.stdout:2/213: dread - d15/d18/d29/d2a/d2e/d32/f33 zero size 2024-09-10T04:05:10.936 INFO:tasks.workunit.client.0.smithi046.stdout:0/193: read d1/f35 [406874,4832] 0 2024-09-10T04:05:10.937 INFO:tasks.workunit.client.1.smithi070.stdout:0/254: fdatasync d8/f11 0 2024-09-10T04:05:10.938 INFO:tasks.workunit.client.1.smithi070.stdout:0/255: readlink d8/d13/l17 0 2024-09-10T04:05:10.939 INFO:tasks.workunit.client.1.smithi070.stdout:6/317: rename d0/d3/l1a to d0/d3/d18/d62/l64 0 2024-09-10T04:05:10.941 INFO:tasks.workunit.client.0.smithi046.stdout:7/221: dread d5/d6/d39/f3d [0,4194304] 0 2024-09-10T04:05:10.942 INFO:tasks.workunit.client.1.smithi070.stdout:3/251: rename db/d1c/d25/f3d to db/d1c/d25/f4d 0 2024-09-10T04:05:10.943 INFO:tasks.workunit.client.0.smithi046.stdout:5/323: dwrite d7/d11/d14/d50/f66 [0,4194304] 0 2024-09-10T04:05:10.943 INFO:tasks.workunit.client.1.smithi070.stdout:2/214: dwrite d15/d18/f1c [0,4194304] 0 2024-09-10T04:05:10.944 INFO:tasks.workunit.client.0.smithi046.stdout:8/213: creat d1/d29/d19/f48 x:0 0 0 2024-09-10T04:05:10.947 INFO:tasks.workunit.client.0.smithi046.stdout:7/222: dread f3 [0,4194304] 0 2024-09-10T04:05:10.951 INFO:tasks.workunit.client.1.smithi070.stdout:8/283: rmdir d4/d16/d17/d5c 39 2024-09-10T04:05:10.953 INFO:tasks.workunit.client.1.smithi070.stdout:9/256: creat d9/dd/d2a/d3f/f48 x:0 0 0 2024-09-10T04:05:10.958 INFO:tasks.workunit.client.0.smithi046.stdout:0/194: dread d1/dc/f21 [0,4194304] 0 2024-09-10T04:05:10.960 INFO:tasks.workunit.client.1.smithi070.stdout:0/256: mkdir d8/d13/d33/d53 0 2024-09-10T04:05:10.960 INFO:tasks.workunit.client.1.smithi070.stdout:0/257: chown f5 28 1 2024-09-10T04:05:10.962 INFO:tasks.workunit.client.1.smithi070.stdout:7/207: unlink d2/d3/dc/d1b/d42/l44 0 2024-09-10T04:05:10.963 INFO:tasks.workunit.client.1.smithi070.stdout:7/208: chown d2/l11 289 1 2024-09-10T04:05:10.963 INFO:tasks.workunit.client.1.smithi070.stdout:7/209: fdatasync d2/d3/f3b 0 2024-09-10T04:05:10.963 INFO:tasks.workunit.client.1.smithi070.stdout:7/210: stat d2/d7/db/d20 0 2024-09-10T04:05:10.965 INFO:tasks.workunit.client.1.smithi070.stdout:6/318: creat d0/d3/d18/d1b/d33/f65 x:0 0 0 2024-09-10T04:05:10.966 INFO:tasks.workunit.client.0.smithi046.stdout:7/223: rmdir d5/d6/db/d11/d1f 39 2024-09-10T04:05:10.971 INFO:tasks.workunit.client.0.smithi046.stdout:0/195: creat d1/dc/d25/d33/d3/d29/f3c x:0 0 0 2024-09-10T04:05:10.973 INFO:tasks.workunit.client.0.smithi046.stdout:0/196: chown d1/dc/f16 2 1 2024-09-10T04:05:10.973 INFO:tasks.workunit.client.0.smithi046.stdout:5/324: dwrite d7/d11/f23 [0,4194304] 0 2024-09-10T04:05:10.974 INFO:tasks.workunit.client.0.smithi046.stdout:5/325: read d7/d11/f12 [87944,11393] 0 2024-09-10T04:05:10.975 INFO:tasks.workunit.client.0.smithi046.stdout:9/237: rename d4/d7/f23 to d4/d7/f4a 0 2024-09-10T04:05:10.976 INFO:tasks.workunit.client.0.smithi046.stdout:9/238: chown d4/fb 3976104 1 2024-09-10T04:05:10.976 INFO:tasks.workunit.client.0.smithi046.stdout:0/197: dread - d1/dc/d25/d33/d3/dd/f39 zero size 2024-09-10T04:05:10.978 INFO:tasks.workunit.client.0.smithi046.stdout:0/198: chown d1/dc/d25/d33/d3/dd/f23 6014463 1 2024-09-10T04:05:10.980 INFO:tasks.workunit.client.1.smithi070.stdout:0/258: unlink d8/d13/d1a/c2c 0 2024-09-10T04:05:10.981 INFO:tasks.workunit.client.1.smithi070.stdout:8/284: dwrite d4/f11 [0,4194304] 0 2024-09-10T04:05:10.982 INFO:tasks.workunit.client.0.smithi046.stdout:8/214: creat d1/f49 x:0 0 0 2024-09-10T04:05:10.983 INFO:tasks.workunit.client.1.smithi070.stdout:7/211: mkdir d2/d7/d9/d18/d49 0 2024-09-10T04:05:10.984 INFO:tasks.workunit.client.0.smithi046.stdout:5/326: fsync d7/d11/d14/d2f/f5e 0 2024-09-10T04:05:10.988 INFO:tasks.workunit.client.0.smithi046.stdout:9/239: creat d4/d29/f4b x:0 0 0 2024-09-10T04:05:10.996 INFO:tasks.workunit.client.1.smithi070.stdout:3/252: write db/f27 [484922,73245] 0 2024-09-10T04:05:10.997 INFO:tasks.workunit.client.0.smithi046.stdout:5/327: creat d7/d11/d3f/d5a/f69 x:0 0 0 2024-09-10T04:05:10.998 INFO:tasks.workunit.client.0.smithi046.stdout:9/240: mkdir d4/d29/d4c 0 2024-09-10T04:05:10.998 INFO:tasks.workunit.client.0.smithi046.stdout:9/241: dread - d4/f28 zero size 2024-09-10T04:05:11.000 INFO:tasks.workunit.client.1.smithi070.stdout:3/253: dwrite db/f27 [0,4194304] 0 2024-09-10T04:05:11.001 INFO:tasks.workunit.client.1.smithi070.stdout:9/257: truncate d9/dd/f30 3690630 0 2024-09-10T04:05:11.001 INFO:tasks.workunit.client.0.smithi046.stdout:7/224: creat d5/d6/d23/f53 x:0 0 0 2024-09-10T04:05:11.003 INFO:tasks.workunit.client.0.smithi046.stdout:0/199: mknod d1/d1e/c3d 0 2024-09-10T04:05:11.005 INFO:tasks.workunit.client.1.smithi070.stdout:2/215: link d15/d18/d29/d2a/l39 d15/d18/d29/d2a/d2e/l4a 0 2024-09-10T04:05:11.006 INFO:tasks.workunit.client.0.smithi046.stdout:5/328: mkdir d7/d11/d14/d2f/d2b/d6a 0 2024-09-10T04:05:11.007 INFO:tasks.workunit.client.0.smithi046.stdout:5/329: chown d7/d11/d14/f29 640375 1 2024-09-10T04:05:11.008 INFO:tasks.workunit.client.0.smithi046.stdout:7/225: dwrite d5/d6/db/d11/f4e [0,4194304] 0 2024-09-10T04:05:11.020 INFO:tasks.workunit.client.1.smithi070.stdout:4/181: sync 2024-09-10T04:05:11.021 INFO:tasks.workunit.client.0.smithi046.stdout:6/197: sync 2024-09-10T04:05:11.021 INFO:tasks.workunit.client.0.smithi046.stdout:4/214: sync 2024-09-10T04:05:11.025 INFO:tasks.workunit.client.0.smithi046.stdout:6/198: dread f6 [0,4194304] 0 2024-09-10T04:05:11.026 INFO:tasks.workunit.client.0.smithi046.stdout:9/242: mkdir d4/d3c/d4d 0 2024-09-10T04:05:11.026 INFO:tasks.workunit.client.1.smithi070.stdout:4/182: dread de/d13/d18/f21 [0,4194304] 0 2024-09-10T04:05:11.027 INFO:tasks.workunit.client.1.smithi070.stdout:3/254: creat db/f4e x:0 0 0 2024-09-10T04:05:11.029 INFO:tasks.workunit.client.1.smithi070.stdout:9/258: chown d9/d1c/f2c 0 1 2024-09-10T04:05:11.031 INFO:tasks.workunit.client.1.smithi070.stdout:4/183: dwrite de/d13/d18/d26/f2a [0,4194304] 0 2024-09-10T04:05:11.031 INFO:tasks.workunit.client.1.smithi070.stdout:6/319: link d0/ce d0/d3/d18/d1b/d17/d3f/d40/c66 0 2024-09-10T04:05:11.033 INFO:tasks.workunit.client.0.smithi046.stdout:9/243: dwrite d4/d29/f4b [0,4194304] 0 2024-09-10T04:05:11.035 INFO:tasks.workunit.client.0.smithi046.stdout:7/226: chown d5/d33/l46 3506 1 2024-09-10T04:05:11.036 INFO:tasks.workunit.client.1.smithi070.stdout:9/259: dread d9/dd/f1a [0,4194304] 0 2024-09-10T04:05:11.036 INFO:tasks.workunit.client.1.smithi070.stdout:7/212: symlink d2/d40/l4a 0 2024-09-10T04:05:11.042 INFO:tasks.workunit.client.0.smithi046.stdout:4/215: rename d3/l2d to d3/l49 0 2024-09-10T04:05:11.044 INFO:tasks.workunit.client.0.smithi046.stdout:4/216: chown d3/l7 5974 1 2024-09-10T04:05:11.044 INFO:tasks.workunit.client.0.smithi046.stdout:9/244: dwrite d4/d3c/f40 [0,4194304] 0 2024-09-10T04:05:11.045 INFO:tasks.workunit.client.0.smithi046.stdout:6/199: fsync da/db/d11/d13/d1e/f26 0 2024-09-10T04:05:11.046 INFO:tasks.workunit.client.0.smithi046.stdout:4/217: write d3/d12/d17/d29/f38 [790100,290] 0 2024-09-10T04:05:11.049 INFO:tasks.workunit.client.0.smithi046.stdout:8/215: getdents d1/d29/d12 0 2024-09-10T04:05:11.052 INFO:tasks.workunit.client.0.smithi046.stdout:8/216: chown d1/d29/d17/d1f/d2a/f2c 31360533 1 2024-09-10T04:05:11.053 INFO:tasks.workunit.client.0.smithi046.stdout:8/217: fsync d1/d29/d12/f45 0 2024-09-10T04:05:11.055 INFO:tasks.workunit.client.0.smithi046.stdout:9/245: dwrite d4/d44/f49 [0,4194304] 0 2024-09-10T04:05:11.057 INFO:tasks.workunit.client.0.smithi046.stdout:7/227: symlink d5/d6/db/dd/d4b/l54 0 2024-09-10T04:05:11.058 INFO:tasks.workunit.client.0.smithi046.stdout:4/218: read d3/d12/d17/d29/f38 [742139,79820] 0 2024-09-10T04:05:11.065 INFO:tasks.workunit.client.0.smithi046.stdout:5/330: link d7/d11/d14/d2f/c2e d7/d11/d14/d50/c6b 0 2024-09-10T04:05:11.067 INFO:tasks.workunit.client.0.smithi046.stdout:8/218: creat d1/d29/f4a x:0 0 0 2024-09-10T04:05:11.067 INFO:tasks.workunit.client.0.smithi046.stdout:5/331: write d7/d11/d3f/d5a/f69 [941475,27798] 0 2024-09-10T04:05:11.070 INFO:tasks.workunit.client.0.smithi046.stdout:8/219: truncate d1/d29/f4a 28831 0 2024-09-10T04:05:11.070 INFO:tasks.workunit.client.0.smithi046.stdout:8/220: dread - d1/f49 zero size 2024-09-10T04:05:11.072 INFO:tasks.workunit.client.0.smithi046.stdout:4/219: dread d3/f15 [0,4194304] 0 2024-09-10T04:05:11.074 INFO:tasks.workunit.client.0.smithi046.stdout:5/332: symlink d7/d11/d3f/d53/l6c 0 2024-09-10T04:05:11.074 INFO:tasks.workunit.client.0.smithi046.stdout:7/228: fsync f4 0 2024-09-10T04:05:11.076 INFO:tasks.workunit.client.0.smithi046.stdout:6/200: rmdir da/d3f 0 2024-09-10T04:05:11.078 INFO:tasks.workunit.client.0.smithi046.stdout:9/246: write d4/d29/f3d [102966,2922] 0 2024-09-10T04:05:11.080 INFO:tasks.workunit.client.0.smithi046.stdout:5/333: creat d7/d11/d14/d45/f6d x:0 0 0 2024-09-10T04:05:11.083 INFO:tasks.workunit.client.0.smithi046.stdout:7/229: write d5/d6/db/dd/d4b/f52 [1738461,43193] 0 2024-09-10T04:05:11.084 INFO:tasks.workunit.client.0.smithi046.stdout:6/201: creat da/db/d18/f41 x:0 0 0 2024-09-10T04:05:11.086 INFO:tasks.workunit.client.0.smithi046.stdout:8/221: creat d1/d29/d17/d1f/d46/f4b x:0 0 0 2024-09-10T04:05:11.087 INFO:tasks.workunit.client.0.smithi046.stdout:4/220: write d3/fa [2935113,31798] 0 2024-09-10T04:05:11.087 INFO:tasks.workunit.client.0.smithi046.stdout:9/247: dwrite d4/f14 [0,4194304] 0 2024-09-10T04:05:11.088 INFO:tasks.workunit.client.0.smithi046.stdout:6/202: symlink da/db/d18/l42 0 2024-09-10T04:05:11.089 INFO:tasks.workunit.client.0.smithi046.stdout:4/221: fdatasync d3/d12/d14/f41 0 2024-09-10T04:05:11.090 INFO:tasks.workunit.client.0.smithi046.stdout:5/334: link d7/d11/f60 d7/d11/f6e 0 2024-09-10T04:05:11.091 INFO:tasks.workunit.client.0.smithi046.stdout:9/248: read - d4/d18/f33 zero size 2024-09-10T04:05:11.092 INFO:tasks.workunit.client.0.smithi046.stdout:8/222: mkdir d1/d29/d19/d36/d4c 0 2024-09-10T04:05:11.095 INFO:tasks.workunit.client.0.smithi046.stdout:8/223: chown d1/d29/d19/d36 53616 1 2024-09-10T04:05:11.096 INFO:tasks.workunit.client.0.smithi046.stdout:9/249: write d4/d29/f3d [601120,118988] 0 2024-09-10T04:05:11.096 INFO:tasks.workunit.client.0.smithi046.stdout:6/203: truncate da/db/d10/d27/f36 758454 0 2024-09-10T04:05:11.100 INFO:tasks.workunit.client.0.smithi046.stdout:4/222: getdents d3/d12/d14 0 2024-09-10T04:05:11.100 INFO:tasks.workunit.client.0.smithi046.stdout:7/230: dread d5/d6/db/f2a [0,4194304] 0 2024-09-10T04:05:11.101 INFO:tasks.workunit.client.0.smithi046.stdout:8/224: mknod d1/d29/d17/d1f/d46/c4d 0 2024-09-10T04:05:11.104 INFO:tasks.workunit.client.0.smithi046.stdout:6/204: mknod da/db/d11/d2b/d2f/c43 0 2024-09-10T04:05:11.106 INFO:tasks.workunit.client.0.smithi046.stdout:7/231: dread d5/d6/db/f2a [0,4194304] 0 2024-09-10T04:05:11.106 INFO:tasks.workunit.client.0.smithi046.stdout:8/225: mknod d1/d29/d12/c4e 0 2024-09-10T04:05:11.107 INFO:tasks.workunit.client.0.smithi046.stdout:7/232: chown d5/d6/d39 2693 1 2024-09-10T04:05:11.108 INFO:tasks.workunit.client.1.smithi070.stdout:9/260: write d9/dd/d2a/d3f/f48 [297877,8032] 0 2024-09-10T04:05:11.109 INFO:tasks.workunit.client.0.smithi046.stdout:9/250: creat d4/d29/d4c/f4e x:0 0 0 2024-09-10T04:05:11.111 INFO:tasks.workunit.client.0.smithi046.stdout:6/205: mknod da/db/d11/d2b/d2f/c44 0 2024-09-10T04:05:11.112 INFO:tasks.workunit.client.1.smithi070.stdout:0/259: creat d8/d13/f54 x:0 0 0 2024-09-10T04:05:11.112 INFO:tasks.workunit.client.1.smithi070.stdout:8/285: write d4/f15 [593834,78974] 0 2024-09-10T04:05:11.114 INFO:tasks.workunit.client.1.smithi070.stdout:6/320: mknod d0/d3/d18/d62/c67 0 2024-09-10T04:05:11.114 INFO:tasks.workunit.client.1.smithi070.stdout:9/261: dread - d9/dd/d10/d1b/f2d zero size 2024-09-10T04:05:11.114 INFO:tasks.workunit.client.1.smithi070.stdout:0/260: write d8/d24/d36/f48 [22084,127787] 0 2024-09-10T04:05:11.115 INFO:tasks.workunit.client.1.smithi070.stdout:7/213: write d2/d7/fe [1963691,129199] 0 2024-09-10T04:05:11.116 INFO:tasks.workunit.client.1.smithi070.stdout:0/261: dread - d8/f11 zero size 2024-09-10T04:05:11.117 INFO:tasks.workunit.client.0.smithi046.stdout:7/233: dread - d5/d6/d23/f37 zero size 2024-09-10T04:05:11.118 INFO:tasks.workunit.client.1.smithi070.stdout:0/262: write d8/fa [6119647,23882] 0 2024-09-10T04:05:11.121 INFO:tasks.workunit.client.0.smithi046.stdout:9/251: rmdir d4/d29/d32 39 2024-09-10T04:05:11.124 INFO:tasks.workunit.client.0.smithi046.stdout:4/223: truncate d3/d12/d17/d22/f27 3125755 0 2024-09-10T04:05:11.127 INFO:tasks.workunit.client.0.smithi046.stdout:7/234: rename d5/d6/db/d4d to d5/d6/d55 0 2024-09-10T04:05:11.127 INFO:tasks.workunit.client.1.smithi070.stdout:8/286: symlink d4/d16/d17/d1a/d23/l66 0 2024-09-10T04:05:11.128 INFO:tasks.workunit.client.0.smithi046.stdout:8/226: link d1/d29/d17/d1f/d2a/f2c d1/d29/d19/d36/f4f 0 2024-09-10T04:05:11.129 INFO:tasks.workunit.client.1.smithi070.stdout:2/216: dwrite fa [0,4194304] 0 2024-09-10T04:05:11.129 INFO:tasks.workunit.client.1.smithi070.stdout:9/262: symlink d9/d1c/l49 0 2024-09-10T04:05:11.131 INFO:tasks.workunit.client.1.smithi070.stdout:9/263: fdatasync d9/dd/d10/f20 0 2024-09-10T04:05:11.131 INFO:tasks.workunit.client.0.smithi046.stdout:6/206: mkdir da/d45 0 2024-09-10T04:05:11.133 INFO:tasks.workunit.client.0.smithi046.stdout:6/207: fsync da/db/d10/d27/f36 0 2024-09-10T04:05:11.134 INFO:tasks.workunit.client.1.smithi070.stdout:6/321: rename d0/d3/d18/f2b to d0/d3/d18/d62/f68 0 2024-09-10T04:05:11.135 INFO:tasks.workunit.client.0.smithi046.stdout:6/208: readlink da/db/d11/d13/d1e/d23/l30 0 2024-09-10T04:05:11.136 INFO:tasks.workunit.client.0.smithi046.stdout:5/335: symlink d7/d11/d14/d45/d51/l6f 0 2024-09-10T04:05:11.136 INFO:tasks.workunit.client.0.smithi046.stdout:6/209: fsync da/db/d11/f3a 0 2024-09-10T04:05:11.136 INFO:tasks.workunit.client.1.smithi070.stdout:7/214: dwrite f1 [4194304,4194304] 0 2024-09-10T04:05:11.138 INFO:tasks.workunit.client.0.smithi046.stdout:6/210: truncate da/dc/f28 1882525 0 2024-09-10T04:05:11.141 INFO:tasks.workunit.client.1.smithi070.stdout:4/184: creat de/d13/d18/f3b x:0 0 0 2024-09-10T04:05:11.141 INFO:tasks.workunit.client.1.smithi070.stdout:4/185: dread - de/d13/d18/f35 zero size 2024-09-10T04:05:11.142 INFO:tasks.workunit.client.0.smithi046.stdout:7/235: unlink d5/d33/l46 0 2024-09-10T04:05:11.145 INFO:tasks.workunit.client.0.smithi046.stdout:8/227: readlink d1/d29/d17/l47 0 2024-09-10T04:05:11.146 INFO:tasks.workunit.client.0.smithi046.stdout:8/228: write d1/d29/d17/f2f [4545612,84540] 0 2024-09-10T04:05:11.148 INFO:tasks.workunit.client.0.smithi046.stdout:9/252: write d4/d7/fe [4975442,119072] 0 2024-09-10T04:05:11.149 INFO:tasks.workunit.client.1.smithi070.stdout:2/217: symlink d15/d2b/l4b 0 2024-09-10T04:05:11.155 INFO:tasks.workunit.client.0.smithi046.stdout:4/224: dwrite d3/f15 [0,4194304] 0 2024-09-10T04:05:11.156 INFO:tasks.workunit.client.1.smithi070.stdout:8/287: dread d4/d16/d17/d1a/f32 [0,4194304] 0 2024-09-10T04:05:11.159 INFO:tasks.workunit.client.1.smithi070.stdout:0/263: rename d8/d13/l17 to d8/l55 0 2024-09-10T04:05:11.160 INFO:tasks.workunit.client.1.smithi070.stdout:0/264: read - d8/d24/d2d/f51 zero size 2024-09-10T04:05:11.160 INFO:tasks.workunit.client.1.smithi070.stdout:1/202: sync 2024-09-10T04:05:11.161 INFO:tasks.workunit.client.1.smithi070.stdout:5/249: sync 2024-09-10T04:05:11.161 INFO:tasks.workunit.client.0.smithi046.stdout:6/211: symlink da/db/d11/d2b/l46 0 2024-09-10T04:05:11.162 INFO:tasks.workunit.client.0.smithi046.stdout:6/212: write da/db/d18/f35 [452164,112260] 0 2024-09-10T04:05:11.163 INFO:tasks.workunit.client.0.smithi046.stdout:7/236: creat d5/f56 x:0 0 0 2024-09-10T04:05:11.165 INFO:tasks.workunit.client.0.smithi046.stdout:8/229: symlink d1/d29/l50 0 2024-09-10T04:05:11.166 INFO:tasks.workunit.client.1.smithi070.stdout:0/265: dwrite d8/d13/f28 [0,4194304] 0 2024-09-10T04:05:11.167 INFO:tasks.workunit.client.1.smithi070.stdout:2/218: unlink d15/d18/d29/d2a/f45 0 2024-09-10T04:05:11.169 INFO:tasks.workunit.client.1.smithi070.stdout:9/264: mknod d9/c4a 0 2024-09-10T04:05:11.171 INFO:tasks.workunit.client.1.smithi070.stdout:8/288: mkdir d4/d16/d17/d19/d28/d43/d67 0 2024-09-10T04:05:11.174 INFO:tasks.workunit.client.0.smithi046.stdout:7/237: dwrite d5/fc [0,4194304] 0 2024-09-10T04:05:11.176 INFO:tasks.workunit.client.0.smithi046.stdout:9/253: dread d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:11.177 INFO:tasks.workunit.client.1.smithi070.stdout:0/266: dread d8/d13/d33/f3f [0,4194304] 0 2024-09-10T04:05:11.180 INFO:tasks.workunit.client.1.smithi070.stdout:5/250: dread f12 [0,4194304] 0 2024-09-10T04:05:11.181 INFO:tasks.workunit.client.1.smithi070.stdout:4/186: rename de/f28 to de/d13/d18/d23/f3c 0 2024-09-10T04:05:11.182 INFO:tasks.workunit.client.1.smithi070.stdout:6/322: write d0/d3/d18/d1b/d17/f47 [724280,35193] 0 2024-09-10T04:05:11.183 INFO:tasks.workunit.client.1.smithi070.stdout:6/323: chown d0/d3/d1f/c4a 0 1 2024-09-10T04:05:11.188 INFO:tasks.workunit.client.0.smithi046.stdout:4/225: dread d3/d12/d17/d29/f38 [0,4194304] 0 2024-09-10T04:05:11.188 INFO:tasks.workunit.client.0.smithi046.stdout:6/213: symlink da/db/d11/l47 0 2024-09-10T04:05:11.188 INFO:tasks.workunit.client.0.smithi046.stdout:4/226: readlink d3/lf 0 2024-09-10T04:05:11.190 INFO:tasks.workunit.client.0.smithi046.stdout:4/227: fsync d3/d12/d14/f18 0 2024-09-10T04:05:11.190 INFO:tasks.workunit.client.0.smithi046.stdout:4/228: readlink d3/l7 0 2024-09-10T04:05:11.196 INFO:tasks.workunit.client.0.smithi046.stdout:6/214: dwrite da/db/d10/d27/f36 [0,4194304] 0 2024-09-10T04:05:11.198 INFO:tasks.workunit.client.0.smithi046.stdout:3/198: sync 2024-09-10T04:05:11.199 INFO:tasks.workunit.client.0.smithi046.stdout:3/199: chown d12/d13/d14 6 1 2024-09-10T04:05:11.199 INFO:tasks.workunit.client.0.smithi046.stdout:9/254: write d4/d29/d32/f43 [3215240,68863] 0 2024-09-10T04:05:11.200 INFO:tasks.workunit.client.1.smithi070.stdout:8/289: rmdir d4/d16/d17 39 2024-09-10T04:05:11.201 INFO:tasks.workunit.client.0.smithi046.stdout:3/200: dread - d12/d13/d14/d1e/d2d/f2e zero size 2024-09-10T04:05:11.202 INFO:tasks.workunit.client.0.smithi046.stdout:9/255: write d4/d18/f38 [412358,4538] 0 2024-09-10T04:05:11.203 INFO:tasks.workunit.client.0.smithi046.stdout:3/201: chown d12/d1b/f2b 0 1 2024-09-10T04:05:11.204 INFO:tasks.workunit.client.0.smithi046.stdout:5/336: creat d7/f70 x:0 0 0 2024-09-10T04:05:11.205 INFO:tasks.workunit.client.0.smithi046.stdout:9/256: write d4/fb [3288197,128793] 0 2024-09-10T04:05:11.206 INFO:tasks.workunit.client.1.smithi070.stdout:9/265: dread d9/dd/d10/d1b/f45 [0,4194304] 0 2024-09-10T04:05:11.209 INFO:tasks.workunit.client.0.smithi046.stdout:6/215: dread da/db/d10/f17 [4194304,4194304] 0 2024-09-10T04:05:11.210 INFO:tasks.workunit.client.1.smithi070.stdout:0/267: write d8/d24/f4a [1270589,112665] 0 2024-09-10T04:05:11.213 INFO:tasks.workunit.client.0.smithi046.stdout:8/230: rename d1/d29/d17/c20 to d1/d29/d19/d36/c51 0 2024-09-10T04:05:11.214 INFO:tasks.workunit.client.1.smithi070.stdout:3/255: sync 2024-09-10T04:05:11.215 INFO:tasks.workunit.client.1.smithi070.stdout:3/256: fdatasync db/d1c/f4b 0 2024-09-10T04:05:11.217 INFO:tasks.workunit.client.1.smithi070.stdout:2/219: rename fc to d15/d2b/f4c 0 2024-09-10T04:05:11.220 INFO:tasks.workunit.client.0.smithi046.stdout:7/238: chown d5/d9/d12 685933 1 2024-09-10T04:05:11.221 INFO:tasks.workunit.client.0.smithi046.stdout:4/229: write d3/d12/d17/f1b [980379,54169] 0 2024-09-10T04:05:11.221 INFO:tasks.workunit.client.1.smithi070.stdout:7/215: link d2/d40/l4a d2/d7/db/d20/l4b 0 2024-09-10T04:05:11.222 INFO:tasks.workunit.client.0.smithi046.stdout:4/230: fdatasync d3/d12/f1f 0 2024-09-10T04:05:11.224 INFO:tasks.workunit.client.1.smithi070.stdout:1/203: truncate d3/d8/d12/f19 1462806 0 2024-09-10T04:05:11.227 INFO:tasks.workunit.client.1.smithi070.stdout:9/266: symlink d9/dd/d2a/l4b 0 2024-09-10T04:05:11.227 INFO:tasks.workunit.client.0.smithi046.stdout:3/202: rmdir d12/d13/d14/d1e 39 2024-09-10T04:05:11.229 INFO:tasks.workunit.client.1.smithi070.stdout:5/251: creat d17/d24/d4e/f55 x:0 0 0 2024-09-10T04:05:11.229 INFO:tasks.workunit.client.1.smithi070.stdout:5/252: stat f13 0 2024-09-10T04:05:11.229 INFO:tasks.workunit.client.0.smithi046.stdout:3/203: read d12/d1b/f2b [928410,39374] 0 2024-09-10T04:05:11.229 INFO:tasks.workunit.client.0.smithi046.stdout:6/216: unlink da/db/d18/f41 0 2024-09-10T04:05:11.230 INFO:tasks.workunit.client.1.smithi070.stdout:0/268: mknod d8/d13/d33/c56 0 2024-09-10T04:05:11.231 INFO:tasks.workunit.client.0.smithi046.stdout:6/217: fdatasync da/db/d10/d27/f36 0 2024-09-10T04:05:11.231 INFO:tasks.workunit.client.1.smithi070.stdout:4/187: dread - de/f34 zero size 2024-09-10T04:05:11.231 INFO:tasks.workunit.client.0.smithi046.stdout:8/231: mkdir d1/d29/d17/d1f/d46/d52 0 2024-09-10T04:05:11.232 INFO:tasks.workunit.client.1.smithi070.stdout:3/257: creat db/d1c/f4f x:0 0 0 2024-09-10T04:05:11.233 INFO:tasks.workunit.client.0.smithi046.stdout:6/218: chown da/db/d11/d2b/l46 257021569 1 2024-09-10T04:05:11.234 INFO:tasks.workunit.client.0.smithi046.stdout:4/231: creat d3/d12/d17/d22/f4a x:0 0 0 2024-09-10T04:05:11.238 INFO:tasks.workunit.client.0.smithi046.stdout:3/204: creat d12/d13/f3b x:0 0 0 2024-09-10T04:05:11.241 INFO:tasks.workunit.client.0.smithi046.stdout:7/239: mknod d5/d6/db/d11/c57 0 2024-09-10T04:05:11.241 INFO:tasks.workunit.client.1.smithi070.stdout:5/253: dread d17/f36 [0,4194304] 0 2024-09-10T04:05:11.241 INFO:tasks.workunit.client.1.smithi070.stdout:5/254: chown d17/d37 4236535 1 2024-09-10T04:05:11.242 INFO:tasks.workunit.client.1.smithi070.stdout:5/255: dread - d17/d24/d47/f53 zero size 2024-09-10T04:05:11.244 INFO:tasks.workunit.client.0.smithi046.stdout:6/219: rename c7 to da/db/d11/d2b/c48 0 2024-09-10T04:05:11.246 INFO:tasks.workunit.client.1.smithi070.stdout:4/188: dwrite de/d13/d18/f3b [0,4194304] 0 2024-09-10T04:05:11.246 INFO:tasks.workunit.client.0.smithi046.stdout:5/337: creat d7/d11/d14/f71 x:0 0 0 2024-09-10T04:05:11.249 INFO:tasks.workunit.client.1.smithi070.stdout:2/220: symlink d15/d18/d42/d43/l4d 0 2024-09-10T04:05:11.250 INFO:tasks.workunit.client.0.smithi046.stdout:4/232: creat d3/d12/d17/d22/f4b x:0 0 0 2024-09-10T04:05:11.251 INFO:tasks.workunit.client.1.smithi070.stdout:6/324: link d0/d3/d18/d2a/c30 d0/d3/d18/d1b/d17/c69 0 2024-09-10T04:05:11.252 INFO:tasks.workunit.client.1.smithi070.stdout:6/325: truncate d0/d3/d18/d1b/d17/d28/f5e 796698 0 2024-09-10T04:05:11.253 INFO:tasks.workunit.client.0.smithi046.stdout:9/257: creat d4/d18/f4f x:0 0 0 2024-09-10T04:05:11.253 INFO:tasks.workunit.client.1.smithi070.stdout:6/326: dread - d0/d3/d18/d2a/f59 zero size 2024-09-10T04:05:11.253 INFO:tasks.workunit.client.0.smithi046.stdout:9/258: fdatasync d4/d29/f3d 0 2024-09-10T04:05:11.255 INFO:tasks.workunit.client.0.smithi046.stdout:9/259: truncate d4/d18/f25 4540631 0 2024-09-10T04:05:11.255 INFO:tasks.workunit.client.0.smithi046.stdout:3/205: unlink d12/d13/f3b 0 2024-09-10T04:05:11.257 INFO:tasks.workunit.client.1.smithi070.stdout:1/204: creat d3/d8/d12/d2f/d32/f41 x:0 0 0 2024-09-10T04:05:11.258 INFO:tasks.workunit.client.1.smithi070.stdout:1/205: write d3/d8/d12/d2f/f34 [833097,35667] 0 2024-09-10T04:05:11.260 INFO:tasks.workunit.client.1.smithi070.stdout:1/206: read d3/d8/d13/f1b [88309,26843] 0 2024-09-10T04:05:11.261 INFO:tasks.workunit.client.1.smithi070.stdout:1/207: chown d3/d8/d13/c36 2960 1 2024-09-10T04:05:11.263 INFO:tasks.workunit.client.1.smithi070.stdout:0/269: link d8/f27 d8/d13/d33/d53/f57 0 2024-09-10T04:05:11.264 INFO:tasks.workunit.client.0.smithi046.stdout:7/240: dwrite d5/d6/d39/f3d [0,4194304] 0 2024-09-10T04:05:11.265 INFO:tasks.workunit.client.0.smithi046.stdout:4/233: creat d3/d12/d17/d22/f4c x:0 0 0 2024-09-10T04:05:11.267 INFO:tasks.workunit.client.1.smithi070.stdout:5/256: symlink d17/d19/d35/l56 0 2024-09-10T04:05:11.269 INFO:tasks.workunit.client.1.smithi070.stdout:3/258: write db/d1c/d3e/f2a [936670,17623] 0 2024-09-10T04:05:11.272 INFO:tasks.workunit.client.1.smithi070.stdout:5/257: dwrite f13 [0,4194304] 0 2024-09-10T04:05:11.273 INFO:tasks.workunit.client.0.smithi046.stdout:4/234: dwrite d3/f15 [0,4194304] 0 2024-09-10T04:05:11.274 INFO:tasks.workunit.client.1.smithi070.stdout:4/189: unlink de/d13/d18/f35 0 2024-09-10T04:05:11.275 INFO:tasks.workunit.client.1.smithi070.stdout:2/221: rmdir d15/d18 39 2024-09-10T04:05:11.278 INFO:tasks.workunit.client.1.smithi070.stdout:7/216: creat d2/f4c x:0 0 0 2024-09-10T04:05:11.285 INFO:tasks.workunit.client.1.smithi070.stdout:8/290: rename d4/d16/f41 to d4/d16/d17/f68 0 2024-09-10T04:05:11.287 INFO:tasks.workunit.client.1.smithi070.stdout:9/267: symlink d9/d1c/l4c 0 2024-09-10T04:05:11.288 INFO:tasks.workunit.client.1.smithi070.stdout:9/268: write d9/dd/d10/f20 [3859479,105103] 0 2024-09-10T04:05:11.291 INFO:tasks.workunit.client.1.smithi070.stdout:0/270: mknod d8/d13/d1a/c58 0 2024-09-10T04:05:11.292 INFO:tasks.workunit.client.0.smithi046.stdout:8/232: getdents d1/d29/d17/d1f/d2a 0 2024-09-10T04:05:11.293 INFO:tasks.workunit.client.0.smithi046.stdout:8/233: fsync d1/d29/d17/d1f/d2a/f33 0 2024-09-10T04:05:11.294 INFO:tasks.workunit.client.0.smithi046.stdout:8/234: readlink d1/d29/d17/l24 0 2024-09-10T04:05:11.299 INFO:tasks.workunit.client.1.smithi070.stdout:3/259: dwrite db/f33 [4194304,4194304] 0 2024-09-10T04:05:11.299 INFO:tasks.workunit.client.0.smithi046.stdout:7/241: creat d5/d9/d43/f58 x:0 0 0 2024-09-10T04:05:11.300 INFO:tasks.workunit.client.0.smithi046.stdout:7/242: write d5/d6/d39/f3d [3447324,71373] 0 2024-09-10T04:05:11.301 INFO:tasks.workunit.client.0.smithi046.stdout:9/260: link d4/d7/f31 d4/f50 0 2024-09-10T04:05:11.302 INFO:tasks.workunit.client.1.smithi070.stdout:3/260: dread db/f33 [4194304,4194304] 0 2024-09-10T04:05:11.303 INFO:tasks.workunit.client.0.smithi046.stdout:4/235: link d3/d12/d14/f18 d3/d12/f4d 0 2024-09-10T04:05:11.303 INFO:tasks.workunit.client.0.smithi046.stdout:4/236: fsync d3/d12/f19 0 2024-09-10T04:05:11.304 INFO:tasks.workunit.client.0.smithi046.stdout:4/237: write d3/f2e [566325,13027] 0 2024-09-10T04:05:11.307 INFO:tasks.workunit.client.0.smithi046.stdout:3/206: link d12/d13/d14/f33 d12/d13/d14/d1e/d2d/f3c 0 2024-09-10T04:05:11.308 INFO:tasks.workunit.client.0.smithi046.stdout:3/207: fdatasync d12/d13/f17 0 2024-09-10T04:05:11.309 INFO:tasks.workunit.client.0.smithi046.stdout:3/208: write d12/d1b/f21 [550436,116427] 0 2024-09-10T04:05:11.311 INFO:tasks.workunit.client.1.smithi070.stdout:6/327: mkdir d0/d3/d6a 0 2024-09-10T04:05:11.315 INFO:tasks.workunit.client.1.smithi070.stdout:6/328: dwrite d0/d3/d18/f42 [0,4194304] 0 2024-09-10T04:05:11.316 INFO:tasks.workunit.client.1.smithi070.stdout:6/329: chown d0/d3/d18/d1b/f3a 85038861 1 2024-09-10T04:05:11.317 INFO:tasks.workunit.client.1.smithi070.stdout:6/330: read d0/d3/d18/d2a/f4f [145085,50301] 0 2024-09-10T04:05:11.320 INFO:tasks.workunit.client.0.smithi046.stdout:4/238: stat d3/d12/l3d 0 2024-09-10T04:05:11.322 INFO:tasks.workunit.client.1.smithi070.stdout:0/271: rename d8/d13/d1a/d4d to d8/d13/d33/d53/d59 0 2024-09-10T04:05:11.323 INFO:tasks.workunit.client.1.smithi070.stdout:4/190: dwrite de/d13/d18/f37 [0,4194304] 0 2024-09-10T04:05:11.323 INFO:tasks.workunit.client.1.smithi070.stdout:0/272: chown d8/f49 338066738 1 2024-09-10T04:05:11.324 INFO:tasks.workunit.client.0.smithi046.stdout:8/235: write d1/d29/d17/d1f/d2a/f2c [951290,56180] 0 2024-09-10T04:05:11.325 INFO:tasks.workunit.client.1.smithi070.stdout:5/258: mkdir d17/d24/d47/d54/d57 0 2024-09-10T04:05:11.326 INFO:tasks.workunit.client.1.smithi070.stdout:5/259: read d17/f36 [42665,93145] 0 2024-09-10T04:05:11.327 INFO:tasks.workunit.client.1.smithi070.stdout:3/261: unlink db/f3b 0 2024-09-10T04:05:11.328 INFO:tasks.workunit.client.0.smithi046.stdout:5/338: getdents d7/d11/d3f/d5a/d63 0 2024-09-10T04:05:11.329 INFO:tasks.workunit.client.1.smithi070.stdout:3/262: read db/d1c/d3e/f30 [56215,125684] 0 2024-09-10T04:05:11.329 INFO:tasks.workunit.client.1.smithi070.stdout:2/222: symlink d15/d18/d42/l4e 0 2024-09-10T04:05:11.330 INFO:tasks.workunit.client.0.smithi046.stdout:7/243: mkdir d5/d6/d32/d59 0 2024-09-10T04:05:11.332 INFO:tasks.workunit.client.0.smithi046.stdout:9/261: rename d4/d7/c2f to d4/d29/d4c/c51 0 2024-09-10T04:05:11.332 INFO:tasks.workunit.client.0.smithi046.stdout:7/244: truncate d5/d6/d23/f37 221755 0 2024-09-10T04:05:11.335 INFO:tasks.workunit.client.0.smithi046.stdout:3/209: unlink d12/d13/d14/d1e/d2d/f3c 0 2024-09-10T04:05:11.336 INFO:tasks.workunit.client.1.smithi070.stdout:0/273: dread d8/d13/f19 [0,4194304] 0 2024-09-10T04:05:11.337 INFO:tasks.workunit.client.0.smithi046.stdout:5/339: creat d7/d11/d3f/d5a/f72 x:0 0 0 2024-09-10T04:05:11.341 INFO:tasks.workunit.client.0.smithi046.stdout:4/239: rename d3/f2e to d3/d12/d17/d22/f4e 0 2024-09-10T04:05:11.341 INFO:tasks.workunit.client.0.smithi046.stdout:3/210: dread f3 [4194304,4194304] 0 2024-09-10T04:05:11.342 INFO:tasks.workunit.client.1.smithi070.stdout:0/274: dwrite d8/d13/d1a/f31 [0,4194304] 0 2024-09-10T04:05:11.342 INFO:tasks.workunit.client.0.smithi046.stdout:7/245: dwrite d5/d6/db/d11/f4e [0,4194304] 0 2024-09-10T04:05:11.342 INFO:tasks.workunit.client.1.smithi070.stdout:0/275: chown d8/d13/d1a/l1e 0 1 2024-09-10T04:05:11.343 INFO:tasks.workunit.client.1.smithi070.stdout:6/331: unlink d0/d3/f19 0 2024-09-10T04:05:11.343 INFO:tasks.workunit.client.1.smithi070.stdout:0/276: stat d8/d24/d2d/f52 0 2024-09-10T04:05:11.344 INFO:tasks.workunit.client.0.smithi046.stdout:7/246: chown d5/d6/db/f1d 5 1 2024-09-10T04:05:11.344 INFO:tasks.workunit.client.1.smithi070.stdout:1/208: link d3/f7 d3/d8/d13/d3f/f42 0 2024-09-10T04:05:11.346 INFO:tasks.workunit.client.0.smithi046.stdout:7/247: dread d5/f7 [0,4194304] 0 2024-09-10T04:05:11.352 INFO:tasks.workunit.client.0.smithi046.stdout:5/340: mkdir d7/d11/d14/d2f/d2b/d73 0 2024-09-10T04:05:11.356 INFO:tasks.workunit.client.1.smithi070.stdout:4/191: mknod de/d13/d18/d26/c3d 0 2024-09-10T04:05:11.357 INFO:tasks.workunit.client.0.smithi046.stdout:4/240: creat d3/d12/d17/d29/f4f x:0 0 0 2024-09-10T04:05:11.357 INFO:tasks.workunit.client.1.smithi070.stdout:4/192: write de/d13/d18/f37 [3379249,52022] 0 2024-09-10T04:05:11.361 INFO:tasks.workunit.client.0.smithi046.stdout:9/262: symlink d4/d29/l52 0 2024-09-10T04:05:11.362 INFO:tasks.workunit.client.1.smithi070.stdout:5/260: symlink d17/d24/d2b/d31/l58 0 2024-09-10T04:05:11.365 INFO:tasks.workunit.client.1.smithi070.stdout:4/193: dwrite de/d13/d18/f21 [0,4194304] 0 2024-09-10T04:05:11.366 INFO:tasks.workunit.client.0.smithi046.stdout:8/236: link d1/l28 d1/d29/d19/l53 0 2024-09-10T04:05:11.366 INFO:tasks.workunit.client.1.smithi070.stdout:8/291: dwrite d4/d16/f33 [4194304,4194304] 0 2024-09-10T04:05:11.367 INFO:tasks.workunit.client.1.smithi070.stdout:2/223: rmdir d15/d2b 39 2024-09-10T04:05:11.368 INFO:tasks.workunit.client.1.smithi070.stdout:4/194: dread de/d13/d18/d26/f2d [0,4194304] 0 2024-09-10T04:05:11.374 INFO:tasks.workunit.client.0.smithi046.stdout:4/241: unlink d3/f15 0 2024-09-10T04:05:11.375 INFO:tasks.workunit.client.1.smithi070.stdout:9/269: mknod d9/d1c/d39/c4d 0 2024-09-10T04:05:11.376 INFO:tasks.workunit.client.1.smithi070.stdout:9/270: chown d9/d1c/l49 20948 1 2024-09-10T04:05:11.378 INFO:tasks.workunit.client.1.smithi070.stdout:3/263: write db/f14 [3274906,56370] 0 2024-09-10T04:05:11.379 INFO:tasks.workunit.client.1.smithi070.stdout:3/264: chown db 8825797 1 2024-09-10T04:05:11.379 INFO:tasks.workunit.client.0.smithi046.stdout:9/263: symlink d4/d7/d1f/l53 0 2024-09-10T04:05:11.380 INFO:tasks.workunit.client.0.smithi046.stdout:8/237: creat d1/d29/d17/d1f/f54 x:0 0 0 2024-09-10T04:05:11.383 INFO:tasks.workunit.client.0.smithi046.stdout:5/341: link d7/d11/d14/d45/d51/f67 d7/d11/d14/d2f/f74 0 2024-09-10T04:05:11.384 INFO:tasks.workunit.client.1.smithi070.stdout:3/265: dwrite db/f4e [0,4194304] 0 2024-09-10T04:05:11.384 INFO:tasks.workunit.client.1.smithi070.stdout:3/266: stat fa 0 2024-09-10T04:05:11.384 INFO:tasks.workunit.client.1.smithi070.stdout:6/332: creat d0/d3/d18/d1b/d17/d28/f6b x:0 0 0 2024-09-10T04:05:11.386 INFO:tasks.workunit.client.1.smithi070.stdout:1/209: rmdir d3/d8/d12/d2f/d32 39 2024-09-10T04:05:11.386 INFO:tasks.workunit.client.0.smithi046.stdout:5/342: dread - d7/d11/d14/d45/f4b zero size 2024-09-10T04:05:11.386 INFO:tasks.workunit.client.0.smithi046.stdout:9/264: dwrite d4/d44/f49 [0,4194304] 0 2024-09-10T04:05:11.391 INFO:tasks.workunit.client.0.smithi046.stdout:3/211: write d12/d13/d14/d1e/d2d/f2e [71596,53321] 0 2024-09-10T04:05:11.396 INFO:tasks.workunit.client.0.smithi046.stdout:9/265: mkdir d4/d7/d1f/d54 0 2024-09-10T04:05:11.398 INFO:tasks.workunit.client.0.smithi046.stdout:4/242: rename d3/d12/c2f to d3/c50 0 2024-09-10T04:05:11.399 INFO:tasks.workunit.client.0.smithi046.stdout:5/343: mkdir d7/d11/d3f/d53/d75 0 2024-09-10T04:05:11.401 INFO:tasks.workunit.client.0.smithi046.stdout:3/212: unlink d12/d13/d14/d1e/c32 0 2024-09-10T04:05:11.403 INFO:tasks.workunit.client.0.smithi046.stdout:8/238: symlink d1/l55 0 2024-09-10T04:05:11.405 INFO:tasks.workunit.client.1.smithi070.stdout:2/224: symlink d15/d18/d29/d2a/d2e/l4f 0 2024-09-10T04:05:11.406 INFO:tasks.workunit.client.1.smithi070.stdout:7/217: getdents d2/d7/d9/d18/d23 0 2024-09-10T04:05:11.408 INFO:tasks.workunit.client.0.smithi046.stdout:9/266: truncate d4/d7/f8 3144991 0 2024-09-10T04:05:11.410 INFO:tasks.workunit.client.1.smithi070.stdout:9/271: unlink d9/f36 0 2024-09-10T04:05:11.411 INFO:tasks.workunit.client.0.smithi046.stdout:4/243: mkdir d3/d12/d17/d22/d51 0 2024-09-10T04:05:11.411 INFO:tasks.workunit.client.1.smithi070.stdout:9/272: write d9/dd/d10/d3c/f40 [394311,205] 0 2024-09-10T04:05:11.415 INFO:tasks.workunit.client.0.smithi046.stdout:3/213: symlink d12/d13/l3d 0 2024-09-10T04:05:11.416 INFO:tasks.workunit.client.1.smithi070.stdout:3/267: rename db/d1c/d3e/f2d to db/d1c/d3e/f50 0 2024-09-10T04:05:11.417 INFO:tasks.workunit.client.0.smithi046.stdout:3/214: read d12/d13/f2c [927036,93709] 0 2024-09-10T04:05:11.417 INFO:tasks.workunit.client.0.smithi046.stdout:8/239: creat d1/d29/d19/f56 x:0 0 0 2024-09-10T04:05:11.419 INFO:tasks.workunit.client.0.smithi046.stdout:4/244: dwrite d3/d12/d17/d22/f4a [0,4194304] 0 2024-09-10T04:05:11.419 INFO:tasks.workunit.client.0.smithi046.stdout:7/248: getdents d5/d6/db/d11/d1f 0 2024-09-10T04:05:11.420 INFO:tasks.workunit.client.0.smithi046.stdout:8/240: write d1/d29/d19/f48 [812502,15665] 0 2024-09-10T04:05:11.421 INFO:tasks.workunit.client.1.smithi070.stdout:1/210: write d3/d8/d12/d2f/f37 [4465198,123610] 0 2024-09-10T04:05:11.421 INFO:tasks.workunit.client.0.smithi046.stdout:4/245: read d3/d12/f4d [453601,17991] 0 2024-09-10T04:05:11.422 INFO:tasks.workunit.client.0.smithi046.stdout:8/241: dread d1/d29/f4a [0,4194304] 0 2024-09-10T04:05:11.422 INFO:tasks.workunit.client.0.smithi046.stdout:1/182: sync 2024-09-10T04:05:11.423 INFO:tasks.workunit.client.0.smithi046.stdout:4/246: dread - d3/d12/d17/d29/f4f zero size 2024-09-10T04:05:11.425 INFO:tasks.workunit.client.0.smithi046.stdout:9/267: creat d4/d29/d32/f55 x:0 0 0 2024-09-10T04:05:11.425 INFO:tasks.workunit.client.0.smithi046.stdout:8/242: chown d1/d29/d17/d1f/d46 81 1 2024-09-10T04:05:11.425 INFO:tasks.workunit.client.0.smithi046.stdout:1/183: chown d3/d27/c24 445 1 2024-09-10T04:05:11.427 INFO:tasks.workunit.client.0.smithi046.stdout:5/344: mknod d7/d11/c76 0 2024-09-10T04:05:11.429 INFO:tasks.workunit.client.0.smithi046.stdout:5/345: write d7/f1b [1545398,64489] 0 2024-09-10T04:05:11.432 INFO:tasks.workunit.client.0.smithi046.stdout:3/215: symlink d12/d13/l3e 0 2024-09-10T04:05:11.436 INFO:tasks.workunit.client.0.smithi046.stdout:7/249: mkdir d5/d6/d39/d5a 0 2024-09-10T04:05:11.440 INFO:tasks.workunit.client.0.smithi046.stdout:3/216: dwrite d12/d13/d14/f36 [0,4194304] 0 2024-09-10T04:05:11.441 INFO:tasks.workunit.client.0.smithi046.stdout:3/217: write d12/d13/f2c [1782266,103646] 0 2024-09-10T04:05:11.441 INFO:tasks.workunit.client.1.smithi070.stdout:0/277: creat d8/d13/f5a x:0 0 0 2024-09-10T04:05:11.443 INFO:tasks.workunit.client.0.smithi046.stdout:8/243: creat d1/d29/d12/f57 x:0 0 0 2024-09-10T04:05:11.443 INFO:tasks.workunit.client.0.smithi046.stdout:8/244: fsync d1/d29/d19/f34 0 2024-09-10T04:05:11.445 INFO:tasks.workunit.client.1.smithi070.stdout:3/268: unlink db/l20 0 2024-09-10T04:05:11.445 INFO:tasks.workunit.client.0.smithi046.stdout:6/220: sync 2024-09-10T04:05:11.446 INFO:tasks.workunit.client.0.smithi046.stdout:0/200: sync 2024-09-10T04:05:11.446 INFO:tasks.workunit.client.0.smithi046.stdout:2/291: sync 2024-09-10T04:05:11.446 INFO:tasks.workunit.client.1.smithi070.stdout:3/269: chown db/d1c/d25/f39 61200364 1 2024-09-10T04:05:11.447 INFO:tasks.workunit.client.0.smithi046.stdout:1/184: creat d3/d27/d32/d1f/f3c x:0 0 0 2024-09-10T04:05:11.447 INFO:tasks.workunit.client.1.smithi070.stdout:0/278: dread d8/d13/f22 [0,4194304] 0 2024-09-10T04:05:11.448 INFO:tasks.workunit.client.1.smithi070.stdout:6/333: creat d0/d3/d18/d1b/d17/d54/f6c x:0 0 0 2024-09-10T04:05:11.450 INFO:tasks.workunit.client.1.smithi070.stdout:1/211: truncate d3/d8/f28 2256369 0 2024-09-10T04:05:11.450 INFO:tasks.workunit.client.1.smithi070.stdout:5/261: link d17/d24/d2b/d31/l58 d17/d24/l59 0 2024-09-10T04:05:11.451 INFO:tasks.workunit.client.1.smithi070.stdout:1/212: write d3/d10/f3b [56316,48852] 0 2024-09-10T04:05:11.454 INFO:tasks.workunit.client.1.smithi070.stdout:4/195: creat de/d13/f3e x:0 0 0 2024-09-10T04:05:11.455 INFO:tasks.workunit.client.0.smithi046.stdout:0/201: dwrite d1/dc/d25/d33/d4/f36 [0,4194304] 0 2024-09-10T04:05:11.455 INFO:tasks.workunit.client.0.smithi046.stdout:7/250: unlink d5/d9/d43/f58 0 2024-09-10T04:05:11.457 INFO:tasks.workunit.client.0.smithi046.stdout:7/251: write d5/fc [3345567,24475] 0 2024-09-10T04:05:11.458 INFO:tasks.workunit.client.0.smithi046.stdout:7/252: chown d5/d9 488 1 2024-09-10T04:05:11.458 INFO:tasks.workunit.client.1.smithi070.stdout:6/334: dwrite d0/d3/d1f/f37 [0,4194304] 0 2024-09-10T04:05:11.462 INFO:tasks.workunit.client.0.smithi046.stdout:3/218: write d12/d13/d14/f25 [942232,109349] 0 2024-09-10T04:05:11.462 INFO:tasks.workunit.client.1.smithi070.stdout:7/218: mknod d2/d3/dc/d1b/d2d/c4d 0 2024-09-10T04:05:11.465 INFO:tasks.workunit.client.0.smithi046.stdout:8/245: creat d1/d29/d17/d1f/d2a/f58 x:0 0 0 2024-09-10T04:05:11.466 INFO:tasks.workunit.client.1.smithi070.stdout:4/196: dwrite de/d13/d18/f21 [0,4194304] 0 2024-09-10T04:05:11.467 INFO:tasks.workunit.client.1.smithi070.stdout:0/279: dread d8/d24/f2a [0,4194304] 0 2024-09-10T04:05:11.467 INFO:tasks.workunit.client.0.smithi046.stdout:9/268: mkdir d4/d7/d1f/d54/d56 0 2024-09-10T04:05:11.469 INFO:tasks.workunit.client.1.smithi070.stdout:4/197: readlink de/d13/l1d 0 2024-09-10T04:05:11.472 INFO:tasks.workunit.client.0.smithi046.stdout:9/269: dread d4/d18/f22 [0,4194304] 0 2024-09-10T04:05:11.473 INFO:tasks.workunit.client.1.smithi070.stdout:8/292: link d4/d16/d17/d1a/d23/l3e d4/d16/l69 0 2024-09-10T04:05:11.475 INFO:tasks.workunit.client.0.smithi046.stdout:9/270: write d4/fb [4206230,61791] 0 2024-09-10T04:05:11.478 INFO:tasks.workunit.client.1.smithi070.stdout:5/262: rename d17/f36 to d17/d24/d47/d54/f5a 0 2024-09-10T04:05:11.480 INFO:tasks.workunit.client.0.smithi046.stdout:1/185: rmdir d3/d27/d32 39 2024-09-10T04:05:11.480 INFO:tasks.workunit.client.1.smithi070.stdout:2/225: truncate d15/d18/f27 1251431 0 2024-09-10T04:05:11.482 INFO:tasks.workunit.client.0.smithi046.stdout:5/346: symlink d7/d11/d14/d2f/l77 0 2024-09-10T04:05:11.483 INFO:tasks.workunit.client.0.smithi046.stdout:5/347: write d7/f1e [355629,33970] 0 2024-09-10T04:05:11.485 INFO:tasks.workunit.client.1.smithi070.stdout:6/335: truncate d0/d3/d18/d1b/d33/f39 3816890 0 2024-09-10T04:05:11.489 INFO:tasks.workunit.client.1.smithi070.stdout:6/336: dread d0/d3/d18/f42 [0,4194304] 0 2024-09-10T04:05:11.490 INFO:tasks.workunit.client.1.smithi070.stdout:3/270: creat db/d1c/d25/d28/d43/f51 x:0 0 0 2024-09-10T04:05:11.491 INFO:tasks.workunit.client.1.smithi070.stdout:3/271: write db/f4e [897307,15165] 0 2024-09-10T04:05:11.494 INFO:tasks.workunit.client.0.smithi046.stdout:7/253: unlink d5/d6/d39/f3d 0 2024-09-10T04:05:11.496 INFO:tasks.workunit.client.0.smithi046.stdout:3/219: rename d12/f29 to d12/d13/d14/d1e/f3f 0 2024-09-10T04:05:11.500 INFO:tasks.workunit.client.1.smithi070.stdout:5/263: mkdir d17/d24/d5b 0 2024-09-10T04:05:11.500 INFO:tasks.workunit.client.1.smithi070.stdout:5/264: read - d17/d24/d47/f53 zero size 2024-09-10T04:05:11.501 INFO:tasks.workunit.client.1.smithi070.stdout:5/265: write d17/d24/d45/f51 [632220,61688] 0 2024-09-10T04:05:11.502 INFO:tasks.workunit.client.1.smithi070.stdout:5/266: read d17/d24/f27 [1550851,60576] 0 2024-09-10T04:05:11.505 INFO:tasks.workunit.client.0.smithi046.stdout:4/247: link d3/d12/d17/l40 d3/d43/l52 0 2024-09-10T04:05:11.505 INFO:tasks.workunit.client.0.smithi046.stdout:4/248: readlink d3/d43/l52 0 2024-09-10T04:05:11.509 INFO:tasks.workunit.client.0.smithi046.stdout:0/202: mkdir d1/d13/d37/d3e 0 2024-09-10T04:05:11.510 INFO:tasks.workunit.client.1.smithi070.stdout:6/337: chown d0/d3/d18/d1b/d17/c5f 60233790 1 2024-09-10T04:05:11.513 INFO:tasks.workunit.client.1.smithi070.stdout:3/272: rename db/d1c/l26 to db/d1c/d3e/l52 0 2024-09-10T04:05:11.514 INFO:tasks.workunit.client.0.smithi046.stdout:1/186: rename d3/d27/c24 to d3/dc/d21/d3a/c3d 0 2024-09-10T04:05:11.516 INFO:tasks.workunit.client.1.smithi070.stdout:7/219: truncate f1 21935 0 2024-09-10T04:05:11.517 INFO:tasks.workunit.client.0.smithi046.stdout:8/246: symlink d1/d29/d19/d36/d4c/l59 0 2024-09-10T04:05:11.518 INFO:tasks.workunit.client.1.smithi070.stdout:8/293: symlink d4/d16/d17/d1a/d60/l6a 0 2024-09-10T04:05:11.521 INFO:tasks.workunit.client.0.smithi046.stdout:3/220: dwrite d12/d13/d14/d1e/f3f [0,4194304] 0 2024-09-10T04:05:11.521 INFO:tasks.workunit.client.1.smithi070.stdout:7/220: dwrite d2/f4c [0,4194304] 0 2024-09-10T04:05:11.524 INFO:tasks.workunit.client.0.smithi046.stdout:0/203: creat d1/dc/d25/d33/f3f x:0 0 0 2024-09-10T04:05:11.531 INFO:tasks.workunit.client.0.smithi046.stdout:2/292: getdents d2/d6/d15 0 2024-09-10T04:05:11.537 INFO:tasks.workunit.client.0.smithi046.stdout:4/249: write d3/d12/d17/d29/f38 [674779,36416] 0 2024-09-10T04:05:11.538 INFO:tasks.workunit.client.0.smithi046.stdout:5/348: creat d7/d11/f78 x:0 0 0 2024-09-10T04:05:11.539 INFO:tasks.workunit.client.0.smithi046.stdout:4/250: read d3/d12/f4d [1094000,47382] 0 2024-09-10T04:05:11.539 INFO:tasks.workunit.client.1.smithi070.stdout:2/226: truncate d15/d18/f24 224108 0 2024-09-10T04:05:11.540 INFO:tasks.workunit.client.1.smithi070.stdout:8/294: rmdir d4/d16/d17/d1a/d60 39 2024-09-10T04:05:11.541 INFO:tasks.workunit.client.1.smithi070.stdout:8/295: write d4/d16/d17/d1a/f2a [1628290,46012] 0 2024-09-10T04:05:11.544 INFO:tasks.workunit.client.0.smithi046.stdout:1/187: rename d3/dc/d21/d3a/c3d to d3/dc/d21/d3a/c3e 0 2024-09-10T04:05:11.545 INFO:tasks.workunit.client.1.smithi070.stdout:1/213: getdents d3/d8 0 2024-09-10T04:05:11.548 INFO:tasks.workunit.client.0.smithi046.stdout:3/221: mknod d12/d13/d14/c40 0 2024-09-10T04:05:11.549 INFO:tasks.workunit.client.1.smithi070.stdout:3/273: dwrite db/d1c/d25/f39 [0,4194304] 0 2024-09-10T04:05:11.549 INFO:tasks.workunit.client.0.smithi046.stdout:5/349: write d7/fc [2602465,53837] 0 2024-09-10T04:05:11.550 INFO:tasks.workunit.client.1.smithi070.stdout:5/267: symlink d17/l5c 0 2024-09-10T04:05:11.553 INFO:tasks.workunit.client.0.smithi046.stdout:4/251: mknod d3/d12/d17/d22/d30/c53 0 2024-09-10T04:05:11.556 INFO:tasks.workunit.client.0.smithi046.stdout:7/254: rename d5/d6/d23/f53 to d5/d9/f5b 0 2024-09-10T04:05:11.557 INFO:tasks.workunit.client.1.smithi070.stdout:0/280: getdents d8 0 2024-09-10T04:05:11.558 INFO:tasks.workunit.client.1.smithi070.stdout:4/198: getdents de 0 2024-09-10T04:05:11.559 INFO:tasks.workunit.client.0.smithi046.stdout:7/255: dread d5/f7 [0,4194304] 0 2024-09-10T04:05:11.562 INFO:tasks.workunit.client.1.smithi070.stdout:9/273: sync 2024-09-10T04:05:11.564 INFO:tasks.workunit.client.1.smithi070.stdout:0/281: dwrite d8/fc [0,4194304] 0 2024-09-10T04:05:11.565 INFO:tasks.workunit.client.1.smithi070.stdout:0/282: chown d8/d13/d1a/f30 282 1 2024-09-10T04:05:11.566 INFO:tasks.workunit.client.1.smithi070.stdout:7/221: mkdir d2/d4e 0 2024-09-10T04:05:11.567 INFO:tasks.workunit.client.0.smithi046.stdout:1/188: write d3/f6 [245099,53468] 0 2024-09-10T04:05:11.571 INFO:tasks.workunit.client.0.smithi046.stdout:8/247: rename d1/d29/d19/c1a to d1/d29/c5a 0 2024-09-10T04:05:11.574 INFO:tasks.workunit.client.0.smithi046.stdout:3/222: dwrite d12/d1b/f2b [0,4194304] 0 2024-09-10T04:05:11.574 INFO:tasks.workunit.client.0.smithi046.stdout:2/293: creat d2/f73 x:0 0 0 2024-09-10T04:05:11.576 INFO:tasks.workunit.client.1.smithi070.stdout:4/199: write de/d13/d18/d23/f3c [122293,69171] 0 2024-09-10T04:05:11.578 INFO:tasks.workunit.client.1.smithi070.stdout:8/296: dread d4/d16/f36 [0,4194304] 0 2024-09-10T04:05:11.578 INFO:tasks.workunit.client.1.smithi070.stdout:4/200: stat de/d13/d18/d26/c3d 0 2024-09-10T04:05:11.580 INFO:tasks.workunit.client.1.smithi070.stdout:2/227: rename d15/d2b/l48 to d15/d18/d29/d2a/l50 0 2024-09-10T04:05:11.581 INFO:tasks.workunit.client.0.smithi046.stdout:0/204: getdents d1/dc/d25/d33 0 2024-09-10T04:05:11.581 INFO:tasks.workunit.client.1.smithi070.stdout:4/201: truncate de/d13/d18/d26/f2b 47537 0 2024-09-10T04:05:11.582 INFO:tasks.workunit.client.1.smithi070.stdout:0/283: unlink d8/d13/d47/f4b 0 2024-09-10T04:05:11.583 INFO:tasks.workunit.client.0.smithi046.stdout:4/252: mknod d3/d12/d17/d22/d51/c54 0 2024-09-10T04:05:11.583 INFO:tasks.workunit.client.1.smithi070.stdout:7/222: symlink d2/d40/l4f 0 2024-09-10T04:05:11.584 INFO:tasks.workunit.client.0.smithi046.stdout:0/205: chown d1/dc/d25/d33/d4/f36 1 1 2024-09-10T04:05:11.584 INFO:tasks.workunit.client.1.smithi070.stdout:8/297: dwrite d4/fa [0,4194304] 0 2024-09-10T04:05:11.584 INFO:tasks.workunit.client.0.smithi046.stdout:7/256: dwrite d5/d9/f5b [0,4194304] 0 2024-09-10T04:05:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:11 smithi070 ceph-mon[35495]: pgmap v259: 65 pgs: 65 active+clean; 689 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s rd, 55 MiB/s wr, 216 op/s 2024-09-10T04:05:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:11 smithi070 ceph-mon[35495]: Upgrade: Updating mgr.smithi070.tezbzn 2024-09-10T04:05:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:05:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T04:05:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:11 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:11 smithi070 ceph-mon[35495]: Deploying daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T04:05:11.585 INFO:tasks.workunit.client.1.smithi070.stdout:7/223: write d2/d7/d9/d18/d23/f38 [2575307,77042] 0 2024-09-10T04:05:11.586 INFO:tasks.workunit.client.1.smithi070.stdout:8/298: chown d4/d16/d17/d19/d28/d43/d67 4556 1 2024-09-10T04:05:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:11 smithi046 ceph-mon[28328]: pgmap v259: 65 pgs: 65 active+clean; 689 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 3.9 MiB/s rd, 55 MiB/s wr, 216 op/s 2024-09-10T04:05:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:11 smithi046 ceph-mon[28328]: Upgrade: Updating mgr.smithi070.tezbzn 2024-09-10T04:05:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:05:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T04:05:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:11 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:11 smithi046 ceph-mon[28328]: Deploying daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T04:05:11.587 INFO:tasks.workunit.client.0.smithi046.stdout:0/206: write d1/dc/f16 [4425581,98032] 0 2024-09-10T04:05:11.588 INFO:tasks.workunit.client.1.smithi070.stdout:8/299: stat d4/d16/d17/d19/d28 0 2024-09-10T04:05:11.589 INFO:tasks.workunit.client.0.smithi046.stdout:8/248: creat d1/d29/d19/d36/d4c/f5b x:0 0 0 2024-09-10T04:05:11.589 INFO:tasks.workunit.client.0.smithi046.stdout:0/207: fdatasync d1/dc/d25/d33/f1b 0 2024-09-10T04:05:11.590 INFO:tasks.workunit.client.1.smithi070.stdout:5/268: fdatasync d17/d24/f2f 0 2024-09-10T04:05:11.590 INFO:tasks.workunit.client.0.smithi046.stdout:7/257: write d5/d6/db/dd/d4b/f52 [1822683,115663] 0 2024-09-10T04:05:11.593 INFO:tasks.workunit.client.1.smithi070.stdout:6/338: getdents d0/d3/d18/d1b/d17/d28 0 2024-09-10T04:05:11.593 INFO:tasks.workunit.client.1.smithi070.stdout:7/224: dwrite d2/d3/dc/d1b/f34 [0,4194304] 0 2024-09-10T04:05:11.594 INFO:tasks.workunit.client.0.smithi046.stdout:8/249: write d1/d29/d17/d1f/d2a/f2c [1313560,98919] 0 2024-09-10T04:05:11.594 INFO:tasks.workunit.client.0.smithi046.stdout:7/258: write d5/f56 [34403,58306] 0 2024-09-10T04:05:11.600 INFO:tasks.workunit.client.1.smithi070.stdout:4/202: rmdir de/d13/d18/d26 39 2024-09-10T04:05:11.600 INFO:tasks.workunit.client.0.smithi046.stdout:5/350: link d7/d11/d14/c38 d7/d11/d14/d2f/d2b/d6a/c79 0 2024-09-10T04:05:11.600 INFO:tasks.workunit.client.0.smithi046.stdout:1/189: symlink d3/d27/d32/l3f 0 2024-09-10T04:05:11.601 INFO:tasks.workunit.client.1.smithi070.stdout:0/284: creat d8/d24/d2d/d34/f5b x:0 0 0 2024-09-10T04:05:11.606 INFO:tasks.workunit.client.0.smithi046.stdout:7/259: chown d5/d6/db/dd/f2f 0 1 2024-09-10T04:05:11.606 INFO:tasks.workunit.client.0.smithi046.stdout:8/250: dread d1/d29/d19/f48 [0,4194304] 0 2024-09-10T04:05:11.607 INFO:tasks.workunit.client.0.smithi046.stdout:4/253: dwrite d3/d12/d14/f16 [0,4194304] 0 2024-09-10T04:05:11.610 INFO:tasks.workunit.client.0.smithi046.stdout:4/254: fdatasync d3/d12/d14/f36 0 2024-09-10T04:05:11.612 INFO:tasks.workunit.client.0.smithi046.stdout:3/223: mknod d12/c41 0 2024-09-10T04:05:11.613 INFO:tasks.workunit.client.0.smithi046.stdout:0/208: dwrite d1/dc/d25/d33/d1a/f31 [0,4194304] 0 2024-09-10T04:05:11.615 INFO:tasks.workunit.client.0.smithi046.stdout:4/255: read - d3/d12/d14/f2b zero size 2024-09-10T04:05:11.616 INFO:tasks.workunit.client.0.smithi046.stdout:3/224: chown d12/l2a 46 1 2024-09-10T04:05:11.616 INFO:tasks.workunit.client.0.smithi046.stdout:8/251: write d1/d29/d17/d1f/f44 [940342,110408] 0 2024-09-10T04:05:11.619 INFO:tasks.workunit.client.0.smithi046.stdout:8/252: chown d1/d29 123221 1 2024-09-10T04:05:11.620 INFO:tasks.workunit.client.0.smithi046.stdout:1/190: write d3/dc/fe [197789,118833] 0 2024-09-10T04:05:11.622 INFO:tasks.workunit.client.0.smithi046.stdout:1/191: truncate d3/f26 370966 0 2024-09-10T04:05:11.624 INFO:tasks.workunit.client.1.smithi070.stdout:1/214: dwrite d3/d8/d12/f19 [0,4194304] 0 2024-09-10T04:05:11.626 INFO:tasks.workunit.client.1.smithi070.stdout:6/339: creat d0/d3/d1f/d55/f6d x:0 0 0 2024-09-10T04:05:11.629 INFO:tasks.workunit.client.1.smithi070.stdout:1/215: dread d3/d8/d12/f19 [0,4194304] 0 2024-09-10T04:05:11.629 INFO:tasks.workunit.client.0.smithi046.stdout:5/351: rename d7/fd to d7/d11/d14/d2f/d2b/f7a 0 2024-09-10T04:05:11.630 INFO:tasks.workunit.client.1.smithi070.stdout:1/216: write d3/d8/d12/f19 [119820,130349] 0 2024-09-10T04:05:11.630 INFO:tasks.workunit.client.1.smithi070.stdout:2/228: mknod d15/d18/c51 0 2024-09-10T04:05:11.632 INFO:tasks.workunit.client.1.smithi070.stdout:1/217: write d3/d8/fc [8503900,28309] 0 2024-09-10T04:05:11.633 INFO:tasks.workunit.client.0.smithi046.stdout:0/209: creat d1/dc/d25/d33/d3/d29/f40 x:0 0 0 2024-09-10T04:05:11.634 INFO:tasks.workunit.client.1.smithi070.stdout:0/285: creat d8/d13/d47/f5c x:0 0 0 2024-09-10T04:05:11.636 INFO:tasks.workunit.client.1.smithi070.stdout:3/274: getdents db/d1c 0 2024-09-10T04:05:11.636 INFO:tasks.workunit.client.1.smithi070.stdout:3/275: readlink db/d1c/l47 0 2024-09-10T04:05:11.637 INFO:tasks.workunit.client.0.smithi046.stdout:3/225: mkdir d12/d13/d42 0 2024-09-10T04:05:11.637 INFO:tasks.workunit.client.0.smithi046.stdout:3/226: rename d12 to d12/d13/d43 22 2024-09-10T04:05:11.639 INFO:tasks.workunit.client.0.smithi046.stdout:2/294: write d2/d6/f61 [205540,51954] 0 2024-09-10T04:05:11.639 INFO:tasks.workunit.client.0.smithi046.stdout:4/256: mknod d3/d12/d17/d29/c55 0 2024-09-10T04:05:11.641 INFO:tasks.workunit.client.0.smithi046.stdout:3/227: read d12/d1b/f21 [177149,103421] 0 2024-09-10T04:05:11.641 INFO:tasks.workunit.client.1.smithi070.stdout:3/276: dwrite fa [0,4194304] 0 2024-09-10T04:05:11.644 INFO:tasks.workunit.client.0.smithi046.stdout:8/253: mknod d1/d29/d19/c5c 0 2024-09-10T04:05:11.645 INFO:tasks.workunit.client.0.smithi046.stdout:7/260: write d5/d6/db/f2a [491937,129255] 0 2024-09-10T04:05:11.646 INFO:tasks.workunit.client.0.smithi046.stdout:8/254: chown d1/d29/d17/d1f/d46 137713 1 2024-09-10T04:05:11.647 INFO:tasks.workunit.client.0.smithi046.stdout:3/228: dread d12/d13/d14/d1e/f3f [0,4194304] 0 2024-09-10T04:05:11.648 INFO:tasks.workunit.client.0.smithi046.stdout:1/192: chown d3/d27/d32/d28/c3b 1638296 1 2024-09-10T04:05:11.649 INFO:tasks.workunit.client.0.smithi046.stdout:8/255: write d1/d29/fe [665233,19020] 0 2024-09-10T04:05:11.651 INFO:tasks.workunit.client.1.smithi070.stdout:9/274: getdents d9/d1c 0 2024-09-10T04:05:11.652 INFO:tasks.workunit.client.1.smithi070.stdout:9/275: read d9/f22 [3300746,44638] 0 2024-09-10T04:05:11.652 INFO:tasks.workunit.client.1.smithi070.stdout:3/277: dread db/d1c/f1e [0,4194304] 0 2024-09-10T04:05:11.653 INFO:tasks.workunit.client.1.smithi070.stdout:9/276: rename d9/d1c/d34 to d9/d1c/d34/d4e 22 2024-09-10T04:05:11.654 INFO:tasks.workunit.client.1.smithi070.stdout:9/277: read d9/d3e/f47 [3049305,111775] 0 2024-09-10T04:05:11.655 INFO:tasks.workunit.client.1.smithi070.stdout:5/269: truncate f11 6488823 0 2024-09-10T04:05:11.655 INFO:tasks.workunit.client.0.smithi046.stdout:3/229: dwrite d12/d1b/f2b [0,4194304] 0 2024-09-10T04:05:11.656 INFO:tasks.workunit.client.0.smithi046.stdout:3/230: chown d12/d1b/f21 145851 1 2024-09-10T04:05:11.660 INFO:tasks.workunit.client.1.smithi070.stdout:2/229: symlink d15/d18/d42/d43/l52 0 2024-09-10T04:05:11.661 INFO:tasks.workunit.client.0.smithi046.stdout:3/231: dread d12/d1b/f2b [0,4194304] 0 2024-09-10T04:05:11.663 INFO:tasks.workunit.client.0.smithi046.stdout:4/257: symlink d3/d12/d17/d22/d51/l56 0 2024-09-10T04:05:11.664 INFO:tasks.workunit.client.1.smithi070.stdout:0/286: unlink d8/d24/d2d/f52 0 2024-09-10T04:05:11.667 INFO:tasks.workunit.client.1.smithi070.stdout:8/300: creat d4/d16/f6b x:0 0 0 2024-09-10T04:05:11.671 INFO:tasks.workunit.client.0.smithi046.stdout:1/193: write d3/d27/f18 [1260335,477] 0 2024-09-10T04:05:11.673 INFO:tasks.workunit.client.1.smithi070.stdout:3/278: mkdir db/d1c/d53 0 2024-09-10T04:05:11.674 INFO:tasks.workunit.client.1.smithi070.stdout:1/218: dread d3/d10/f35 [0,4194304] 0 2024-09-10T04:05:11.676 INFO:tasks.workunit.client.1.smithi070.stdout:6/340: getdents d0/d3/d5c 0 2024-09-10T04:05:11.677 INFO:tasks.workunit.client.0.smithi046.stdout:3/232: dread f4 [0,4194304] 0 2024-09-10T04:05:11.678 INFO:tasks.workunit.client.0.smithi046.stdout:5/352: symlink d7/l7b 0 2024-09-10T04:05:11.679 INFO:tasks.workunit.client.1.smithi070.stdout:9/278: symlink d9/d1c/l4f 0 2024-09-10T04:05:11.681 INFO:tasks.workunit.client.1.smithi070.stdout:8/301: dread d4/f29 [0,4194304] 0 2024-09-10T04:05:11.681 INFO:tasks.workunit.client.1.smithi070.stdout:8/302: chown d4/l3c 2584 1 2024-09-10T04:05:11.682 INFO:tasks.workunit.client.1.smithi070.stdout:2/230: write d15/d2b/f4c [1812770,120658] 0 2024-09-10T04:05:11.683 INFO:tasks.workunit.client.1.smithi070.stdout:2/231: write d15/d2b/f4c [1754849,30205] 0 2024-09-10T04:05:11.684 INFO:tasks.workunit.client.0.smithi046.stdout:5/353: dwrite d7/f19 [0,4194304] 0 2024-09-10T04:05:11.685 INFO:tasks.workunit.client.0.smithi046.stdout:5/354: dread - d7/d11/d14/d45/f4b zero size 2024-09-10T04:05:11.687 INFO:tasks.workunit.client.0.smithi046.stdout:2/295: creat d2/d49/f74 x:0 0 0 2024-09-10T04:05:11.688 INFO:tasks.workunit.client.0.smithi046.stdout:0/210: truncate d1/d13/f1d 534662 0 2024-09-10T04:05:11.692 INFO:tasks.workunit.client.0.smithi046.stdout:4/258: chown d3/c50 8826 1 2024-09-10T04:05:11.694 INFO:tasks.workunit.client.1.smithi070.stdout:1/219: creat d3/d8/d13/f43 x:0 0 0 2024-09-10T04:05:11.695 INFO:tasks.workunit.client.0.smithi046.stdout:7/261: creat d5/d9/d43/f5c x:0 0 0 2024-09-10T04:05:11.695 INFO:tasks.workunit.client.0.smithi046.stdout:2/296: dwrite d2/d6/d15/f6a [0,4194304] 0 2024-09-10T04:05:11.696 INFO:tasks.workunit.client.1.smithi070.stdout:9/279: mknod d9/dd/d10/d1b/c50 0 2024-09-10T04:05:11.696 INFO:tasks.workunit.client.1.smithi070.stdout:9/280: chown d9/f35 0 1 2024-09-10T04:05:11.697 INFO:tasks.workunit.client.1.smithi070.stdout:1/220: dread d3/d8/d12/f19 [0,4194304] 0 2024-09-10T04:05:11.697 INFO:tasks.workunit.client.0.smithi046.stdout:1/194: creat d3/dc/d21/f40 x:0 0 0 2024-09-10T04:05:11.699 INFO:tasks.workunit.client.0.smithi046.stdout:8/256: getdents d1/d29/d17/d1f/d46/d52 0 2024-09-10T04:05:11.699 INFO:tasks.workunit.client.1.smithi070.stdout:4/203: getdents de/d13/d18 0 2024-09-10T04:05:11.699 INFO:tasks.workunit.client.1.smithi070.stdout:4/204: stat de/f11 0 2024-09-10T04:05:11.700 INFO:tasks.workunit.client.0.smithi046.stdout:8/257: fsync d1/d29/d12/f45 0 2024-09-10T04:05:11.700 INFO:tasks.workunit.client.1.smithi070.stdout:4/205: dread - de/f34 zero size 2024-09-10T04:05:11.706 INFO:tasks.workunit.client.0.smithi046.stdout:8/258: dwrite d1/d29/d12/f45 [0,4194304] 0 2024-09-10T04:05:11.716 INFO:tasks.workunit.client.0.smithi046.stdout:6/221: sync 2024-09-10T04:05:11.717 INFO:tasks.workunit.client.1.smithi070.stdout:3/279: write db/d1c/d25/d28/f37 [32933,101703] 0 2024-09-10T04:05:11.717 INFO:tasks.workunit.client.1.smithi070.stdout:3/280: fdatasync db/f12 0 2024-09-10T04:05:11.719 INFO:tasks.workunit.client.0.smithi046.stdout:8/259: dread d1/d29/d17/f2f [4194304,4194304] 0 2024-09-10T04:05:11.728 INFO:tasks.workunit.client.1.smithi070.stdout:9/281: chown d9/dd/f30 41391533 1 2024-09-10T04:05:11.728 INFO:tasks.workunit.client.1.smithi070.stdout:9/282: stat d9/dd/d10/d1b/f2d 0 2024-09-10T04:05:11.732 INFO:tasks.workunit.client.0.smithi046.stdout:0/211: dwrite d1/dc/d25/d33/d3/dd/f23 [8388608,4194304] 0 2024-09-10T04:05:11.735 INFO:tasks.workunit.client.0.smithi046.stdout:3/233: mknod d12/c44 0 2024-09-10T04:05:11.736 INFO:tasks.workunit.client.1.smithi070.stdout:8/303: write d4/d16/d17/d1a/f2e [1043390,119203] 0 2024-09-10T04:05:11.738 INFO:tasks.workunit.client.1.smithi070.stdout:3/281: rename db/d1c/d25/f39 to db/d1c/d3e/f54 0 2024-09-10T04:05:11.739 INFO:tasks.workunit.client.1.smithi070.stdout:2/232: truncate d15/d18/f24 980024 0 2024-09-10T04:05:11.741 INFO:tasks.workunit.client.0.smithi046.stdout:6/222: creat da/db/d18/f49 x:0 0 0 2024-09-10T04:05:11.741 INFO:tasks.workunit.client.1.smithi070.stdout:0/287: link d8/lb d8/d24/d2d/d34/l5d 0 2024-09-10T04:05:11.743 INFO:tasks.workunit.client.0.smithi046.stdout:3/234: dwrite d12/d13/f17 [0,4194304] 0 2024-09-10T04:05:11.748 INFO:tasks.workunit.client.0.smithi046.stdout:7/262: creat d5/d9/d12/f5d x:0 0 0 2024-09-10T04:05:11.751 INFO:tasks.workunit.client.1.smithi070.stdout:8/304: rmdir d4/d16/d17/d1a 39 2024-09-10T04:05:11.751 INFO:tasks.workunit.client.0.smithi046.stdout:0/212: stat d1/dc/d25/c2f 0 2024-09-10T04:05:11.754 INFO:tasks.workunit.client.1.smithi070.stdout:3/282: rename db/d1c/f1e to db/d1c/d25/f55 0 2024-09-10T04:05:11.758 INFO:tasks.workunit.client.0.smithi046.stdout:6/223: mknod da/db/d11/d13/d1e/d23/c4a 0 2024-09-10T04:05:11.759 INFO:tasks.workunit.client.0.smithi046.stdout:6/224: dread - da/db/d11/d13/d1e/f26 zero size 2024-09-10T04:05:11.759 INFO:tasks.workunit.client.0.smithi046.stdout:5/355: rmdir d7/d11/d14/d2f/d2b/d73 0 2024-09-10T04:05:11.760 INFO:tasks.workunit.client.0.smithi046.stdout:5/356: chown d7/d11/c76 82 1 2024-09-10T04:05:11.761 INFO:tasks.workunit.client.0.smithi046.stdout:6/225: write da/db/d10/d27/f29 [1015036,66885] 0 2024-09-10T04:05:11.762 INFO:tasks.workunit.client.1.smithi070.stdout:1/221: dwrite d3/d8/d13/f18 [0,4194304] 0 2024-09-10T04:05:11.765 INFO:tasks.workunit.client.0.smithi046.stdout:2/297: creat d2/d8/d37/d63/f75 x:0 0 0 2024-09-10T04:05:11.765 INFO:tasks.workunit.client.0.smithi046.stdout:7/263: dread d5/d6/db/dd/f2f [0,4194304] 0 2024-09-10T04:05:11.766 INFO:tasks.workunit.client.1.smithi070.stdout:4/206: link de/d13/d18/l19 de/l3f 0 2024-09-10T04:05:11.768 INFO:tasks.workunit.client.1.smithi070.stdout:8/305: creat d4/d16/d17/d19/d28/d43/d52/f6c x:0 0 0 2024-09-10T04:05:11.769 INFO:tasks.workunit.client.0.smithi046.stdout:7/264: dread - d5/d6/db/f29 zero size 2024-09-10T04:05:11.770 INFO:tasks.workunit.client.1.smithi070.stdout:2/233: dwrite d15/d18/d29/f46 [4194304,4194304] 0 2024-09-10T04:05:11.771 INFO:tasks.workunit.client.0.smithi046.stdout:1/195: creat d3/d27/f41 x:0 0 0 2024-09-10T04:05:11.772 INFO:tasks.workunit.client.1.smithi070.stdout:3/283: fdatasync db/d1c/d3e/f30 0 2024-09-10T04:05:11.773 INFO:tasks.workunit.client.0.smithi046.stdout:0/213: mkdir d1/dc/d25/d33/d1a/d41 0 2024-09-10T04:05:11.774 INFO:tasks.workunit.client.0.smithi046.stdout:6/226: dwrite da/db/d11/d2b/f2c [0,4194304] 0 2024-09-10T04:05:11.776 INFO:tasks.workunit.client.0.smithi046.stdout:0/214: chown d1/dc/d25/d33/f11 1005 1 2024-09-10T04:05:11.777 INFO:tasks.workunit.client.0.smithi046.stdout:3/235: getdents d12/d13/d14/d1e/d2d/d39 0 2024-09-10T04:05:11.778 INFO:tasks.workunit.client.1.smithi070.stdout:0/288: mkdir d8/d24/d5e 0 2024-09-10T04:05:11.778 INFO:tasks.workunit.client.1.smithi070.stdout:9/283: rmdir d9/d1c/d34 39 2024-09-10T04:05:11.779 INFO:tasks.workunit.client.0.smithi046.stdout:0/215: dread - d1/dc/d25/d33/f3f zero size 2024-09-10T04:05:11.785 INFO:tasks.workunit.client.0.smithi046.stdout:7/265: fdatasync f3 0 2024-09-10T04:05:11.787 INFO:tasks.workunit.client.1.smithi070.stdout:8/306: read d4/d16/d17/d1a/f58 [13124,84043] 0 2024-09-10T04:05:11.788 INFO:tasks.workunit.client.1.smithi070.stdout:2/234: rename d15/d18/d42/l4e to d15/d18/d29/d2a/l53 0 2024-09-10T04:05:11.789 INFO:tasks.workunit.client.0.smithi046.stdout:3/236: dread d12/d13/d14/f25 [0,4194304] 0 2024-09-10T04:05:11.789 INFO:tasks.workunit.client.0.smithi046.stdout:2/298: dread d2/d6/d23/f33 [0,4194304] 0 2024-09-10T04:05:11.789 INFO:tasks.workunit.client.1.smithi070.stdout:8/307: write d4/d16/d17/d19/d28/d43/f5d [153449,35603] 0 2024-09-10T04:05:11.792 INFO:tasks.workunit.client.0.smithi046.stdout:2/299: chown d2/d6/d15/c26 16337 1 2024-09-10T04:05:11.794 INFO:tasks.workunit.client.1.smithi070.stdout:3/284: chown db/d1c/d3e/f50 6339650 1 2024-09-10T04:05:11.794 INFO:tasks.workunit.client.0.smithi046.stdout:0/216: dwrite d1/dc/d25/d33/d1a/f31 [4194304,4194304] 0 2024-09-10T04:05:11.796 INFO:tasks.workunit.client.1.smithi070.stdout:9/284: read - d9/d1c/d39/f3a zero size 2024-09-10T04:05:11.800 INFO:tasks.workunit.client.0.smithi046.stdout:6/227: mknod da/dc/d22/c4b 0 2024-09-10T04:05:11.801 INFO:tasks.workunit.client.1.smithi070.stdout:1/222: link d3/d8/d12/d2f/f34 d3/d20/f44 0 2024-09-10T04:05:11.804 INFO:tasks.workunit.client.0.smithi046.stdout:5/357: creat d7/d11/d3f/d53/d75/f7c x:0 0 0 2024-09-10T04:05:11.804 INFO:tasks.workunit.client.1.smithi070.stdout:0/289: dread d8/d13/d1a/f23 [0,4194304] 0 2024-09-10T04:05:11.806 INFO:tasks.workunit.client.1.smithi070.stdout:2/235: chown f10 1944494 1 2024-09-10T04:05:11.807 INFO:tasks.workunit.client.0.smithi046.stdout:7/266: rename d5/d6/d23/f37 to d5/d6/d32/f5e 0 2024-09-10T04:05:11.808 INFO:tasks.workunit.client.0.smithi046.stdout:7/267: chown d5/d6/db 13 1 2024-09-10T04:05:11.809 INFO:tasks.workunit.client.1.smithi070.stdout:3/285: rename db/d1c/l31 to db/d1c/d53/l56 0 2024-09-10T04:05:11.810 INFO:tasks.workunit.client.1.smithi070.stdout:9/285: unlink d9/d1c/l49 0 2024-09-10T04:05:11.812 INFO:tasks.workunit.client.1.smithi070.stdout:0/290: dwrite d8/d13/f1f [0,4194304] 0 2024-09-10T04:05:11.813 INFO:tasks.workunit.client.0.smithi046.stdout:2/300: creat d2/d6/d23/d3b/f76 x:0 0 0 2024-09-10T04:05:11.813 INFO:tasks.workunit.client.0.smithi046.stdout:2/301: write d2/d8/f6f [931934,11609] 0 2024-09-10T04:05:11.814 INFO:tasks.workunit.client.0.smithi046.stdout:2/302: write d2/d8/d37/d63/f75 [968156,57389] 0 2024-09-10T04:05:11.815 INFO:tasks.workunit.client.0.smithi046.stdout:1/196: creat d3/d27/f42 x:0 0 0 2024-09-10T04:05:11.816 INFO:tasks.workunit.client.1.smithi070.stdout:4/207: truncate de/d13/d18/d26/f2d 412605 0 2024-09-10T04:05:11.817 INFO:tasks.workunit.client.0.smithi046.stdout:2/303: truncate d2/d8/d37/f66 9178 0 2024-09-10T04:05:11.817 INFO:tasks.workunit.client.1.smithi070.stdout:4/208: dread - de/d13/f3e zero size 2024-09-10T04:05:11.817 INFO:tasks.workunit.client.1.smithi070.stdout:9/286: dwrite d9/dd/f12 [0,4194304] 0 2024-09-10T04:05:11.819 INFO:tasks.workunit.client.0.smithi046.stdout:7/268: mkdir d5/d6/d39/d5f 0 2024-09-10T04:05:11.820 INFO:tasks.workunit.client.1.smithi070.stdout:2/236: symlink d15/d18/d29/d2a/d2e/l54 0 2024-09-10T04:05:11.820 INFO:tasks.workunit.client.1.smithi070.stdout:1/223: dread d3/d8/d13/f14 [0,4194304] 0 2024-09-10T04:05:11.820 INFO:tasks.workunit.client.1.smithi070.stdout:3/286: creat db/d1c/d53/f57 x:0 0 0 2024-09-10T04:05:11.821 INFO:tasks.workunit.client.0.smithi046.stdout:3/237: fsync d12/d13/d14/f24 0 2024-09-10T04:05:11.822 INFO:tasks.workunit.client.1.smithi070.stdout:1/224: chown d3/d8/d12/d2f/f37 32 1 2024-09-10T04:05:11.824 INFO:tasks.workunit.client.0.smithi046.stdout:3/238: read d12/d13/d14/d1e/d2d/f2e [33913,78703] 0 2024-09-10T04:05:11.825 INFO:tasks.workunit.client.1.smithi070.stdout:0/291: mkdir d8/d24/d2d/d5f 0 2024-09-10T04:05:11.826 INFO:tasks.workunit.client.1.smithi070.stdout:0/292: read d8/d13/d1a/f31 [3573055,42513] 0 2024-09-10T04:05:11.826 INFO:tasks.workunit.client.0.smithi046.stdout:6/228: mknod da/db/d11/d13/d1e/c4c 0 2024-09-10T04:05:11.828 INFO:tasks.workunit.client.0.smithi046.stdout:5/358: creat d7/d11/d14/d2f/d64/f7d x:0 0 0 2024-09-10T04:05:11.830 INFO:tasks.workunit.client.1.smithi070.stdout:4/209: mknod de/d13/d18/d23/c40 0 2024-09-10T04:05:11.831 INFO:tasks.workunit.client.0.smithi046.stdout:1/197: creat d3/d27/d32/f43 x:0 0 0 2024-09-10T04:05:11.835 INFO:tasks.workunit.client.1.smithi070.stdout:2/237: mkdir d15/d18/d42/d43/d55 0 2024-09-10T04:05:11.836 INFO:tasks.workunit.client.0.smithi046.stdout:7/269: truncate f4 2137299 0 2024-09-10T04:05:11.837 INFO:tasks.workunit.client.1.smithi070.stdout:3/287: creat db/d1c/d25/d28/d43/f58 x:0 0 0 2024-09-10T04:05:11.840 INFO:tasks.workunit.client.0.smithi046.stdout:2/304: dwrite d2/d6/d15/f3e [4194304,4194304] 0 2024-09-10T04:05:11.840 INFO:tasks.workunit.client.1.smithi070.stdout:1/225: rename d3/d8/d12/d2f/f37 to d3/d8/d12/f45 0 2024-09-10T04:05:11.840 INFO:tasks.workunit.client.1.smithi070.stdout:2/238: dwrite fa [0,4194304] 0 2024-09-10T04:05:11.841 INFO:tasks.workunit.client.0.smithi046.stdout:2/305: chown d2/d6/d15 42 1 2024-09-10T04:05:11.847 INFO:tasks.workunit.client.1.smithi070.stdout:2/239: dwrite d15/d18/f21 [0,4194304] 0 2024-09-10T04:05:11.848 INFO:tasks.workunit.client.1.smithi070.stdout:2/240: chown d15/d18/d29/d2a/d2e 589 1 2024-09-10T04:05:11.851 INFO:tasks.workunit.client.1.smithi070.stdout:2/241: write d15/d18/d29/d2a/d2e/d32/f33 [651521,105436] 0 2024-09-10T04:05:11.851 INFO:tasks.workunit.client.0.smithi046.stdout:5/359: rename d7/d11/d14/d2f/l39 to d7/d11/d3f/d53/d75/l7e 0 2024-09-10T04:05:11.855 INFO:tasks.workunit.client.0.smithi046.stdout:1/198: mknod d3/d27/d32/d28/c44 0 2024-09-10T04:05:11.861 INFO:tasks.workunit.client.0.smithi046.stdout:7/270: dread d5/f35 [0,4194304] 0 2024-09-10T04:05:11.865 INFO:tasks.workunit.client.1.smithi070.stdout:9/287: dwrite d9/dd/d10/d1b/f45 [4194304,4194304] 0 2024-09-10T04:05:11.866 INFO:tasks.workunit.client.0.smithi046.stdout:7/271: dread - d5/d6/db/d11/d1f/f48 zero size 2024-09-10T04:05:11.866 INFO:tasks.workunit.client.1.smithi070.stdout:3/288: dwrite db/d1c/d25/f55 [4194304,4194304] 0 2024-09-10T04:05:11.867 INFO:tasks.workunit.client.1.smithi070.stdout:3/289: chown db/f3c 41902913 1 2024-09-10T04:05:11.867 INFO:tasks.workunit.client.0.smithi046.stdout:0/217: getdents d1/dc/d25/d33/d3/dd 0 2024-09-10T04:05:11.868 INFO:tasks.workunit.client.0.smithi046.stdout:7/272: fsync d5/d6/db/dd/f44 0 2024-09-10T04:05:11.869 INFO:tasks.workunit.client.0.smithi046.stdout:0/218: dread - d1/dc/d25/d33/d3/d29/f40 zero size 2024-09-10T04:05:11.871 INFO:tasks.workunit.client.0.smithi046.stdout:3/239: link d12/d13/d14/f3a d12/d13/d37/f45 0 2024-09-10T04:05:11.872 INFO:tasks.workunit.client.1.smithi070.stdout:3/290: dwrite db/d1c/f4f [0,4194304] 0 2024-09-10T04:05:11.877 INFO:tasks.workunit.client.0.smithi046.stdout:2/306: rename d2/d6/d23/d52 to d2/d6/d23/d5d/d77 0 2024-09-10T04:05:11.880 INFO:tasks.workunit.client.1.smithi070.stdout:0/293: creat d8/d24/d2d/d5f/f60 x:0 0 0 2024-09-10T04:05:11.886 INFO:tasks.workunit.client.0.smithi046.stdout:2/307: dwrite d2/d6/d15/f6a [0,4194304] 0 2024-09-10T04:05:11.889 INFO:tasks.workunit.client.1.smithi070.stdout:3/291: rmdir db/d1c/d25 39 2024-09-10T04:05:11.890 INFO:tasks.workunit.client.1.smithi070.stdout:3/292: chown db/f11 439504958 1 2024-09-10T04:05:11.891 INFO:tasks.workunit.client.1.smithi070.stdout:3/293: chown db/f11 21102303 1 2024-09-10T04:05:11.891 INFO:tasks.workunit.client.0.smithi046.stdout:7/273: write d5/d6/d32/f5e [595435,125556] 0 2024-09-10T04:05:11.894 INFO:tasks.workunit.client.1.smithi070.stdout:1/226: dwrite d3/d8/d12/d2f/f34 [0,4194304] 0 2024-09-10T04:05:11.897 INFO:tasks.workunit.client.0.smithi046.stdout:6/229: creat da/db/d11/d13/d1e/f4d x:0 0 0 2024-09-10T04:05:11.899 INFO:tasks.workunit.client.0.smithi046.stdout:6/230: chown da/db/d11/d13/d1e/f21 3512 1 2024-09-10T04:05:11.899 INFO:tasks.workunit.client.1.smithi070.stdout:2/242: mkdir d15/d18/d56 0 2024-09-10T04:05:11.900 INFO:tasks.workunit.client.1.smithi070.stdout:9/288: write d9/dd/d10/d1b/f2d [703755,62304] 0 2024-09-10T04:05:11.901 INFO:tasks.workunit.client.1.smithi070.stdout:9/289: write d9/dd/f18 [8071590,18858] 0 2024-09-10T04:05:11.902 INFO:tasks.workunit.client.0.smithi046.stdout:9/271: sync 2024-09-10T04:05:11.905 INFO:tasks.workunit.client.1.smithi070.stdout:9/290: dread d9/d3e/f47 [0,4194304] 0 2024-09-10T04:05:11.906 INFO:tasks.workunit.client.1.smithi070.stdout:9/291: fdatasync d9/f25 0 2024-09-10T04:05:11.908 INFO:tasks.workunit.client.1.smithi070.stdout:0/294: mknod d8/d13/c61 0 2024-09-10T04:05:11.909 INFO:tasks.workunit.client.1.smithi070.stdout:0/295: chown d8/l15 1356613 1 2024-09-10T04:05:11.911 INFO:tasks.workunit.client.1.smithi070.stdout:9/292: dwrite d9/dd/d10/d1b/f2d [0,4194304] 0 2024-09-10T04:05:11.913 INFO:tasks.workunit.client.1.smithi070.stdout:1/227: creat d3/d10/d1d/f46 x:0 0 0 2024-09-10T04:05:11.913 INFO:tasks.workunit.client.0.smithi046.stdout:0/219: dread d1/dc/d25/d33/d3/f2a [0,4194304] 0 2024-09-10T04:05:11.913 INFO:tasks.workunit.client.0.smithi046.stdout:2/308: mknod d2/d6/d15/c78 0 2024-09-10T04:05:11.914 INFO:tasks.workunit.client.0.smithi046.stdout:1/199: truncate d3/d27/d32/d28/f2c 365811 0 2024-09-10T04:05:11.915 INFO:tasks.workunit.client.0.smithi046.stdout:0/220: chown d1/dc/d25/d33/d3/d29/d2d 4304014 1 2024-09-10T04:05:11.915 INFO:tasks.workunit.client.0.smithi046.stdout:1/200: readlink d3/d27/d32/l3f 0 2024-09-10T04:05:11.916 INFO:tasks.workunit.client.0.smithi046.stdout:2/309: truncate d2/d6/d23/f5b 246510 0 2024-09-10T04:05:11.917 INFO:tasks.workunit.client.0.smithi046.stdout:2/310: chown d2/d6/d23/l2b 0 1 2024-09-10T04:05:11.918 INFO:tasks.workunit.client.0.smithi046.stdout:7/274: creat d5/f60 x:0 0 0 2024-09-10T04:05:11.919 INFO:tasks.workunit.client.0.smithi046.stdout:2/311: truncate d2/d8/f29 4545542 0 2024-09-10T04:05:11.920 INFO:tasks.workunit.client.1.smithi070.stdout:2/243: creat d15/d18/d29/d2a/d2e/d32/f57 x:0 0 0 2024-09-10T04:05:11.920 INFO:tasks.workunit.client.1.smithi070.stdout:2/244: write d15/d18/f1c [4112035,88557] 0 2024-09-10T04:05:11.923 INFO:tasks.workunit.client.0.smithi046.stdout:6/231: creat da/dc/f4e x:0 0 0 2024-09-10T04:05:11.925 INFO:tasks.workunit.client.1.smithi070.stdout:4/210: truncate de/d13/d18/d26/f2d 974075 0 2024-09-10T04:05:11.928 INFO:tasks.workunit.client.0.smithi046.stdout:1/201: symlink d3/d27/d32/d28/l45 0 2024-09-10T04:05:11.928 INFO:tasks.workunit.client.0.smithi046.stdout:1/202: chown d3/d27/d32/d1f 105122 1 2024-09-10T04:05:11.929 INFO:tasks.workunit.client.1.smithi070.stdout:0/296: mkdir d8/d13/d1a/d62 0 2024-09-10T04:05:11.930 INFO:tasks.workunit.client.1.smithi070.stdout:0/297: chown d8/d24/d2d/d34/c38 10476 1 2024-09-10T04:05:11.930 INFO:tasks.workunit.client.1.smithi070.stdout:2/245: dread d15/d18/d29/f2d [0,4194304] 0 2024-09-10T04:05:11.930 INFO:tasks.workunit.client.0.smithi046.stdout:0/221: rename d1/dc/d25/d33/d3/dd/c3a to d1/dc/d25/d33/d4/d19/c42 0 2024-09-10T04:05:11.931 INFO:tasks.workunit.client.1.smithi070.stdout:0/298: chown d8/d24/d36/f46 395153 1 2024-09-10T04:05:11.933 INFO:tasks.workunit.client.1.smithi070.stdout:1/228: creat d3/f47 x:0 0 0 2024-09-10T04:05:11.935 INFO:tasks.workunit.client.0.smithi046.stdout:5/360: getdents d7/d11/d14/d45 0 2024-09-10T04:05:11.936 INFO:tasks.workunit.client.1.smithi070.stdout:0/299: dwrite d8/d13/f5a [0,4194304] 0 2024-09-10T04:05:11.937 INFO:tasks.workunit.client.1.smithi070.stdout:0/300: dread - d8/d24/d2d/d34/f5b zero size 2024-09-10T04:05:11.940 INFO:tasks.workunit.client.1.smithi070.stdout:0/301: dread d8/d13/d1a/f30 [0,4194304] 0 2024-09-10T04:05:11.943 INFO:tasks.workunit.client.1.smithi070.stdout:9/293: rename d9/d1c/d34/f32 to d9/d1c/d39/f51 0 2024-09-10T04:05:11.948 INFO:tasks.workunit.client.0.smithi046.stdout:3/240: getdents d12/d13 0 2024-09-10T04:05:11.949 INFO:tasks.workunit.client.0.smithi046.stdout:3/241: read - d12/d13/d37/f45 zero size 2024-09-10T04:05:11.950 INFO:tasks.workunit.client.0.smithi046.stdout:6/232: mkdir da/db/d11/d13/d4f 0 2024-09-10T04:05:11.952 INFO:tasks.workunit.client.0.smithi046.stdout:5/361: creat d7/d11/d3f/d5a/d63/f7f x:0 0 0 2024-09-10T04:05:11.954 INFO:tasks.workunit.client.0.smithi046.stdout:5/362: chown d7/d11/d14/f29 13382 1 2024-09-10T04:05:11.956 INFO:tasks.workunit.client.1.smithi070.stdout:2/246: link d15/d18/d29/f46 d15/d18/d29/d2a/f58 0 2024-09-10T04:05:11.956 INFO:tasks.workunit.client.0.smithi046.stdout:5/363: write d7/d11/d14/d2f/d2b/f3a [1349113,39337] 0 2024-09-10T04:05:11.957 INFO:tasks.workunit.client.0.smithi046.stdout:1/203: mknod d3/d27/c46 0 2024-09-10T04:05:11.958 INFO:tasks.workunit.client.0.smithi046.stdout:6/233: dwrite da/db/d11/d13/d1e/f26 [0,4194304] 0 2024-09-10T04:05:11.958 INFO:tasks.workunit.client.0.smithi046.stdout:2/312: truncate d2/d6/d23/f33 948346 0 2024-09-10T04:05:11.959 INFO:tasks.workunit.client.1.smithi070.stdout:0/302: mknod d8/d13/d33/c63 0 2024-09-10T04:05:11.959 INFO:tasks.workunit.client.1.smithi070.stdout:0/303: stat d8/d13/d33/d4f 0 2024-09-10T04:05:11.961 INFO:tasks.workunit.client.0.smithi046.stdout:0/222: creat d1/d13/d37/d3e/f43 x:0 0 0 2024-09-10T04:05:11.962 INFO:tasks.workunit.client.1.smithi070.stdout:9/294: write d9/dd/d10/f1f [2419777,81130] 0 2024-09-10T04:05:11.964 INFO:tasks.workunit.client.1.smithi070.stdout:9/295: truncate d9/d1c/d39/f3a 665018 0 2024-09-10T04:05:11.964 INFO:tasks.workunit.client.0.smithi046.stdout:4/259: sync 2024-09-10T04:05:11.964 INFO:tasks.workunit.client.0.smithi046.stdout:8/260: sync 2024-09-10T04:05:11.964 INFO:tasks.workunit.client.0.smithi046.stdout:7/275: creat d5/d6/db/f61 x:0 0 0 2024-09-10T04:05:11.966 INFO:tasks.workunit.client.0.smithi046.stdout:7/276: dread - d5/d6/db/d11/d1e/f3e zero size 2024-09-10T04:05:11.966 INFO:tasks.workunit.client.1.smithi070.stdout:1/229: unlink d3/d8/d12/c26 0 2024-09-10T04:05:11.967 INFO:tasks.workunit.client.1.smithi070.stdout:3/294: getdents db/d1c/d25/d28 0 2024-09-10T04:05:11.968 INFO:tasks.workunit.client.1.smithi070.stdout:4/211: link de/d13/d18/l1b de/d13/d18/d23/l41 0 2024-09-10T04:05:11.968 INFO:tasks.workunit.client.1.smithi070.stdout:3/295: stat db/d1c/d25/f55 0 2024-09-10T04:05:11.969 INFO:tasks.workunit.client.0.smithi046.stdout:3/242: unlink d12/d13/l3d 0 2024-09-10T04:05:11.971 INFO:tasks.workunit.client.0.smithi046.stdout:9/272: getdents d4/d7 0 2024-09-10T04:05:11.974 INFO:tasks.workunit.client.0.smithi046.stdout:3/243: dread d12/d1b/f2b [0,4194304] 0 2024-09-10T04:05:11.974 INFO:tasks.workunit.client.1.smithi070.stdout:9/296: dread d9/dd/f30 [0,4194304] 0 2024-09-10T04:05:11.976 INFO:tasks.workunit.client.1.smithi070.stdout:0/304: creat d8/d13/d33/d53/d59/f64 x:0 0 0 2024-09-10T04:05:11.976 INFO:tasks.workunit.client.1.smithi070.stdout:0/305: stat f5 0 2024-09-10T04:05:11.977 INFO:tasks.workunit.client.1.smithi070.stdout:0/306: fdatasync d8/d13/f5a 0 2024-09-10T04:05:11.977 INFO:tasks.workunit.client.1.smithi070.stdout:1/230: mknod d3/d8/d12/c48 0 2024-09-10T04:05:11.979 INFO:tasks.workunit.client.1.smithi070.stdout:3/296: rmdir db/d1c/d53 39 2024-09-10T04:05:11.982 INFO:tasks.workunit.client.1.smithi070.stdout:4/212: creat de/d13/d18/d26/f42 x:0 0 0 2024-09-10T04:05:11.983 INFO:tasks.workunit.client.0.smithi046.stdout:6/234: creat da/db/f50 x:0 0 0 2024-09-10T04:05:11.984 INFO:tasks.workunit.client.1.smithi070.stdout:2/247: mkdir d15/d59 0 2024-09-10T04:05:11.984 INFO:tasks.workunit.client.0.smithi046.stdout:6/235: write da/db/f16 [830260,15340] 0 2024-09-10T04:05:11.986 INFO:tasks.workunit.client.0.smithi046.stdout:2/313: rename d2/d49/c2f to d2/d8/d37/c79 0 2024-09-10T04:05:11.987 INFO:tasks.workunit.client.1.smithi070.stdout:9/297: creat d9/dd/d10/d3c/f52 x:0 0 0 2024-09-10T04:05:11.989 INFO:tasks.workunit.client.0.smithi046.stdout:0/223: symlink d1/dc/d25/d33/d4/d19/l44 0 2024-09-10T04:05:11.991 INFO:tasks.workunit.client.1.smithi070.stdout:9/298: dwrite d9/f35 [0,4194304] 0 2024-09-10T04:05:11.992 INFO:tasks.workunit.client.0.smithi046.stdout:6/236: dwrite da/db/d11/d13/d1e/f4d [0,4194304] 0 2024-09-10T04:05:11.992 INFO:tasks.workunit.client.0.smithi046.stdout:8/261: symlink d1/d29/d17/d1f/d46/l5d 0 2024-09-10T04:05:11.993 INFO:tasks.workunit.client.1.smithi070.stdout:0/307: creat d8/d13/d33/f65 x:0 0 0 2024-09-10T04:05:11.994 INFO:tasks.workunit.client.0.smithi046.stdout:8/262: stat d1/d29/d17/d1f/f54 0 2024-09-10T04:05:11.994 INFO:tasks.workunit.client.0.smithi046.stdout:4/260: symlink d3/d12/d17/d22/d51/l57 0 2024-09-10T04:05:11.997 INFO:tasks.workunit.client.1.smithi070.stdout:7/225: sync 2024-09-10T04:05:11.997 INFO:tasks.workunit.client.0.smithi046.stdout:1/204: dwrite d3/ff [0,4194304] 0 2024-09-10T04:05:11.998 INFO:tasks.workunit.client.1.smithi070.stdout:7/226: readlink d2/d7/d9/l10 0 2024-09-10T04:05:12.000 INFO:tasks.workunit.client.1.smithi070.stdout:3/297: creat db/d1c/d3e/f59 x:0 0 0 2024-09-10T04:05:12.001 INFO:tasks.workunit.client.0.smithi046.stdout:7/277: write d5/d9/f42 [4553933,53724] 0 2024-09-10T04:05:12.005 INFO:tasks.workunit.client.1.smithi070.stdout:1/231: dwrite d3/d8/d12/d2f/d32/f41 [0,4194304] 0 2024-09-10T04:05:12.008 INFO:tasks.workunit.client.1.smithi070.stdout:7/227: dwrite d2/f4c [0,4194304] 0 2024-09-10T04:05:12.008 INFO:tasks.workunit.client.0.smithi046.stdout:5/364: link d7/d11/d14/d45/f4b d7/d11/d3f/f80 0 2024-09-10T04:05:12.012 INFO:tasks.workunit.client.1.smithi070.stdout:9/299: write d9/dd/d10/d1b/f45 [7293871,43915] 0 2024-09-10T04:05:12.012 INFO:tasks.workunit.client.1.smithi070.stdout:0/308: symlink d8/l66 0 2024-09-10T04:05:12.015 INFO:tasks.workunit.client.1.smithi070.stdout:1/232: dread d3/d8/d13/f14 [0,4194304] 0 2024-09-10T04:05:12.016 INFO:tasks.workunit.client.1.smithi070.stdout:1/233: truncate d3/d10/d1d/f30 750064 0 2024-09-10T04:05:12.016 INFO:tasks.workunit.client.1.smithi070.stdout:1/234: fdatasync d3/d10/d1d/f46 0 2024-09-10T04:05:12.017 INFO:tasks.workunit.client.0.smithi046.stdout:2/314: readlink d2/l20 0 2024-09-10T04:05:12.017 INFO:tasks.workunit.client.0.smithi046.stdout:3/244: write d12/d13/d37/f45 [542088,48403] 0 2024-09-10T04:05:12.017 INFO:tasks.workunit.client.1.smithi070.stdout:3/298: rmdir db/d1c/d3e 39 2024-09-10T04:05:12.023 INFO:tasks.workunit.client.1.smithi070.stdout:4/213: mkdir de/d13/d43 0 2024-09-10T04:05:12.024 INFO:tasks.workunit.client.0.smithi046.stdout:6/237: chown da/db/d11/d2b/c48 171558519 1 2024-09-10T04:05:12.025 INFO:tasks.workunit.client.1.smithi070.stdout:2/248: dwrite d15/d18/f24 [0,4194304] 0 2024-09-10T04:05:12.027 INFO:tasks.workunit.client.0.smithi046.stdout:4/261: mkdir d3/d12/d17/d29/d58 0 2024-09-10T04:05:12.027 INFO:tasks.workunit.client.0.smithi046.stdout:4/262: dread - d3/d43/f48 zero size 2024-09-10T04:05:12.029 INFO:tasks.workunit.client.0.smithi046.stdout:1/205: write f1 [4640944,73378] 0 2024-09-10T04:05:12.030 INFO:tasks.workunit.client.1.smithi070.stdout:2/249: dwrite d15/d2b/f4c [0,4194304] 0 2024-09-10T04:05:12.031 INFO:tasks.workunit.client.0.smithi046.stdout:1/206: write d3/d27/d32/d1f/f29 [4271724,47878] 0 2024-09-10T04:05:12.031 INFO:tasks.workunit.client.1.smithi070.stdout:9/300: rename d9/dd/d10/d1b/l1e to d9/d1c/d39/l53 0 2024-09-10T04:05:12.034 INFO:tasks.workunit.client.1.smithi070.stdout:1/235: creat d3/d8/f49 x:0 0 0 2024-09-10T04:05:12.035 INFO:tasks.workunit.client.0.smithi046.stdout:9/273: mkdir d4/d57 0 2024-09-10T04:05:12.036 INFO:tasks.workunit.client.0.smithi046.stdout:7/278: unlink d5/d6/db/f61 0 2024-09-10T04:05:12.040 INFO:tasks.workunit.client.1.smithi070.stdout:7/228: creat d2/d3/dc/d1b/d41/f50 x:0 0 0 2024-09-10T04:05:12.043 INFO:tasks.workunit.client.1.smithi070.stdout:0/309: symlink d8/d24/d5e/l67 0 2024-09-10T04:05:12.044 INFO:tasks.workunit.client.0.smithi046.stdout:0/224: symlink d1/dc/l45 0 2024-09-10T04:05:12.048 INFO:tasks.workunit.client.0.smithi046.stdout:6/238: mkdir da/db/d10/d51 0 2024-09-10T04:05:12.049 INFO:tasks.workunit.client.1.smithi070.stdout:2/250: write f10 [4467837,53542] 0 2024-09-10T04:05:12.049 INFO:tasks.workunit.client.1.smithi070.stdout:2/251: stat d15/d18/c1f 0 2024-09-10T04:05:12.049 INFO:tasks.workunit.client.1.smithi070.stdout:9/301: mknod d9/dd/d10/d3c/c54 0 2024-09-10T04:05:12.055 INFO:tasks.workunit.client.0.smithi046.stdout:5/365: dwrite d7/d11/d14/d45/f4b [0,4194304] 0 2024-09-10T04:05:12.057 INFO:tasks.workunit.client.1.smithi070.stdout:7/229: dread f1 [0,4194304] 0 2024-09-10T04:05:12.057 INFO:tasks.workunit.client.1.smithi070.stdout:7/230: chown d2/c12 519061 1 2024-09-10T04:05:12.064 INFO:tasks.workunit.client.1.smithi070.stdout:3/299: dwrite db/d1c/d3e/f50 [8388608,4194304] 0 2024-09-10T04:05:12.064 INFO:tasks.workunit.client.0.smithi046.stdout:3/245: mknod d12/d13/d14/d1e/d2d/d39/c46 0 2024-09-10T04:05:12.065 INFO:tasks.workunit.client.1.smithi070.stdout:5/270: sync 2024-09-10T04:05:12.068 INFO:tasks.workunit.client.1.smithi070.stdout:7/231: rename d2/d7/db/c3d to d2/d3/dc/d13/c51 0 2024-09-10T04:05:12.069 INFO:tasks.workunit.client.0.smithi046.stdout:2/315: write d2/d6/d23/f33 [215478,95413] 0 2024-09-10T04:05:12.069 INFO:tasks.workunit.client.1.smithi070.stdout:0/310: symlink d8/d13/l68 0 2024-09-10T04:05:12.070 INFO:tasks.workunit.client.1.smithi070.stdout:0/311: chown d8/ce 91 1 2024-09-10T04:05:12.072 INFO:tasks.workunit.client.0.smithi046.stdout:9/274: dwrite d4/d18/f22 [0,4194304] 0 2024-09-10T04:05:12.073 INFO:tasks.workunit.client.1.smithi070.stdout:0/312: dread d8/d13/d1a/f31 [0,4194304] 0 2024-09-10T04:05:12.074 INFO:tasks.workunit.client.0.smithi046.stdout:2/316: dread - d2/d49/f74 zero size 2024-09-10T04:05:12.074 INFO:tasks.workunit.client.1.smithi070.stdout:2/252: symlink d15/d18/d42/d43/d55/l5a 0 2024-09-10T04:05:12.075 INFO:tasks.workunit.client.1.smithi070.stdout:1/236: creat d3/d8/d12/f4a x:0 0 0 2024-09-10T04:05:12.077 INFO:tasks.workunit.client.0.smithi046.stdout:5/366: mkdir d7/d11/d3f/d53/d81 0 2024-09-10T04:05:12.077 INFO:tasks.workunit.client.0.smithi046.stdout:7/279: dwrite d5/f7 [0,4194304] 0 2024-09-10T04:05:12.078 INFO:tasks.workunit.client.1.smithi070.stdout:3/300: creat db/d1c/d25/d28/d43/f5a x:0 0 0 2024-09-10T04:05:12.080 INFO:tasks.workunit.client.1.smithi070.stdout:4/214: getdents de 0 2024-09-10T04:05:12.084 INFO:tasks.workunit.client.1.smithi070.stdout:5/271: symlink d17/d24/d4e/l5d 0 2024-09-10T04:05:12.086 INFO:tasks.workunit.client.1.smithi070.stdout:7/232: mkdir d2/d7/d9/d18/d3c/d52 0 2024-09-10T04:05:12.087 INFO:tasks.workunit.client.1.smithi070.stdout:9/302: creat d9/d1c/f55 x:0 0 0 2024-09-10T04:05:12.087 INFO:tasks.workunit.client.0.smithi046.stdout:8/263: getdents d1/d29/d17/d1f 0 2024-09-10T04:05:12.088 INFO:tasks.workunit.client.1.smithi070.stdout:9/303: read d9/dd/d10/d1b/f45 [1279712,29664] 0 2024-09-10T04:05:12.089 INFO:tasks.workunit.client.1.smithi070.stdout:0/313: rename d8/d13/f22 to d8/d24/d2d/d34/f69 0 2024-09-10T04:05:12.090 INFO:tasks.workunit.client.1.smithi070.stdout:2/253: creat d15/d18/d29/d2a/d2e/d32/d36/f5b x:0 0 0 2024-09-10T04:05:12.092 INFO:tasks.workunit.client.0.smithi046.stdout:6/239: truncate da/db/d10/d27/f36 3435514 0 2024-09-10T04:05:12.093 INFO:tasks.workunit.client.1.smithi070.stdout:3/301: symlink db/l5b 0 2024-09-10T04:05:12.093 INFO:tasks.workunit.client.0.smithi046.stdout:6/240: write da/fd [2209818,50088] 0 2024-09-10T04:05:12.093 INFO:tasks.workunit.client.0.smithi046.stdout:4/263: link d3/f9 d3/d12/d17/d22/f59 0 2024-09-10T04:05:12.094 INFO:tasks.workunit.client.0.smithi046.stdout:4/264: chown d3/d12/d17 621644645 1 2024-09-10T04:05:12.094 INFO:tasks.workunit.client.0.smithi046.stdout:9/275: rmdir d4 39 2024-09-10T04:05:12.095 INFO:tasks.workunit.client.1.smithi070.stdout:8/308: sync 2024-09-10T04:05:12.095 INFO:tasks.workunit.client.1.smithi070.stdout:6/341: sync 2024-09-10T04:05:12.095 INFO:tasks.workunit.client.0.smithi046.stdout:4/265: write d3/fa [935412,53408] 0 2024-09-10T04:05:12.096 INFO:tasks.workunit.client.1.smithi070.stdout:5/272: creat d17/d24/d47/f5e x:0 0 0 2024-09-10T04:05:12.096 INFO:tasks.workunit.client.0.smithi046.stdout:4/266: write d3/d43/f45 [349352,28781] 0 2024-09-10T04:05:12.098 INFO:tasks.workunit.client.0.smithi046.stdout:1/207: link d3/c1b d3/d27/d32/d1f/c47 0 2024-09-10T04:05:12.099 INFO:tasks.workunit.client.1.smithi070.stdout:7/233: chown d2/d3/dc/l48 52 1 2024-09-10T04:05:12.099 INFO:tasks.workunit.client.0.smithi046.stdout:1/208: chown d3/dc/f10 50 1 2024-09-10T04:05:12.100 INFO:tasks.workunit.client.0.smithi046.stdout:1/209: chown d3/d27/f33 76303 1 2024-09-10T04:05:12.100 INFO:tasks.workunit.client.0.smithi046.stdout:1/210: chown d3 9383170 1 2024-09-10T04:05:12.103 INFO:tasks.workunit.client.0.smithi046.stdout:7/280: mkdir d5/d6/d62 0 2024-09-10T04:05:12.103 INFO:tasks.workunit.client.1.smithi070.stdout:6/342: dwrite d0/d3/d18/f52 [0,4194304] 0 2024-09-10T04:05:12.108 INFO:tasks.workunit.client.0.smithi046.stdout:0/225: rename d1/d13/f3b to d1/dc/d25/d33/d3/f46 0 2024-09-10T04:05:12.110 INFO:tasks.workunit.client.1.smithi070.stdout:8/309: dread d4/d16/d17/f40 [0,4194304] 0 2024-09-10T04:05:12.111 INFO:tasks.workunit.client.0.smithi046.stdout:8/264: mkdir d1/d29/d19/d36/d4c/d5e 0 2024-09-10T04:05:12.115 INFO:tasks.workunit.client.1.smithi070.stdout:3/302: fsync db/d1c/d25/f4d 0 2024-09-10T04:05:12.115 INFO:tasks.workunit.client.1.smithi070.stdout:8/310: chown d4/d16/d17/d19/d62 423 1 2024-09-10T04:05:12.117 INFO:tasks.workunit.client.1.smithi070.stdout:0/314: dwrite d8/f1b [0,4194304] 0 2024-09-10T04:05:12.118 INFO:tasks.workunit.client.1.smithi070.stdout:4/215: link de/d13/d18/f21 de/d13/d18/d23/f44 0 2024-09-10T04:05:12.120 INFO:tasks.workunit.client.0.smithi046.stdout:6/241: creat da/dc/d22/f52 x:0 0 0 2024-09-10T04:05:12.121 INFO:tasks.workunit.client.1.smithi070.stdout:0/315: write d8/d13/f5a [4137824,41271] 0 2024-09-10T04:05:12.122 INFO:tasks.workunit.client.1.smithi070.stdout:1/237: dwrite d3/d8/f2d [0,4194304] 0 2024-09-10T04:05:12.123 INFO:tasks.workunit.client.1.smithi070.stdout:4/216: write de/d13/d18/d26/f42 [128841,27140] 0 2024-09-10T04:05:12.128 INFO:tasks.workunit.client.1.smithi070.stdout:8/311: dwrite d4/fa [4194304,4194304] 0 2024-09-10T04:05:12.128 INFO:tasks.workunit.client.1.smithi070.stdout:7/234: creat d2/d3/dc/d1b/d41/f53 x:0 0 0 2024-09-10T04:05:12.130 INFO:tasks.workunit.client.0.smithi046.stdout:4/267: creat d3/d12/f5a x:0 0 0 2024-09-10T04:05:12.132 INFO:tasks.workunit.client.0.smithi046.stdout:3/246: truncate d12/d13/d37/f45 330699 0 2024-09-10T04:05:12.135 INFO:tasks.workunit.client.1.smithi070.stdout:7/235: dwrite d2/d7/d9/d18/d23/f3e [0,4194304] 0 2024-09-10T04:05:12.136 INFO:tasks.workunit.client.0.smithi046.stdout:7/281: readlink d5/d6/db/d11/d1e/d20/l2b 0 2024-09-10T04:05:12.136 INFO:tasks.workunit.client.0.smithi046.stdout:5/367: write d7/d11/f1a [4076643,84479] 0 2024-09-10T04:05:12.138 INFO:tasks.workunit.client.0.smithi046.stdout:7/282: dread - d5/d6/db/d11/d1e/f3e zero size 2024-09-10T04:05:12.140 INFO:tasks.workunit.client.0.smithi046.stdout:0/226: creat d1/dc/d25/d33/d1a/f47 x:0 0 0 2024-09-10T04:05:12.140 INFO:tasks.workunit.client.0.smithi046.stdout:7/283: chown d5/d6/db/dd/d24/c4f 91615516 1 2024-09-10T04:05:12.142 INFO:tasks.workunit.client.1.smithi070.stdout:3/303: write db/d1c/d3e/f54 [2680937,74856] 0 2024-09-10T04:05:12.147 INFO:tasks.workunit.client.1.smithi070.stdout:5/273: rename d17/d24/d47/d54/f5a to d17/d19/d35/f5f 0 2024-09-10T04:05:12.148 INFO:tasks.workunit.client.1.smithi070.stdout:5/274: write d17/d24/d4e/f55 [272674,18441] 0 2024-09-10T04:05:12.151 INFO:tasks.workunit.client.1.smithi070.stdout:0/316: creat d8/d13/d1a/f6a x:0 0 0 2024-09-10T04:05:12.152 INFO:tasks.workunit.client.0.smithi046.stdout:4/268: creat d3/d12/d17/d22/f5b x:0 0 0 2024-09-10T04:05:12.154 INFO:tasks.workunit.client.0.smithi046.stdout:1/211: mknod d3/d27/d2a/c48 0 2024-09-10T04:05:12.157 INFO:tasks.workunit.client.1.smithi070.stdout:9/304: creat d9/dd/f56 x:0 0 0 2024-09-10T04:05:12.158 INFO:tasks.workunit.client.0.smithi046.stdout:0/227: creat d1/d13/f48 x:0 0 0 2024-09-10T04:05:12.158 INFO:tasks.workunit.client.1.smithi070.stdout:6/343: dwrite d0/d3/d18/f52 [4194304,4194304] 0 2024-09-10T04:05:12.159 INFO:tasks.workunit.client.1.smithi070.stdout:0/317: dwrite d8/d24/d2d/d34/f4e [0,4194304] 0 2024-09-10T04:05:12.160 INFO:tasks.workunit.client.0.smithi046.stdout:7/284: readlink d5/d6/l3a 0 2024-09-10T04:05:12.161 INFO:tasks.workunit.client.1.smithi070.stdout:2/254: link d15/d18/d29/d2a/c38 d15/d18/d29/d2a/d2e/c5c 0 2024-09-10T04:05:12.163 INFO:tasks.workunit.client.0.smithi046.stdout:8/265: creat d1/d29/d19/d36/d4c/d5e/f5f x:0 0 0 2024-09-10T04:05:12.165 INFO:tasks.workunit.client.1.smithi070.stdout:3/304: creat db/d1c/d25/f5c x:0 0 0 2024-09-10T04:05:12.166 INFO:tasks.workunit.client.1.smithi070.stdout:5/275: truncate d17/d19/f1d 1841026 0 2024-09-10T04:05:12.167 INFO:tasks.workunit.client.0.smithi046.stdout:6/242: getdents da/d45 0 2024-09-10T04:05:12.170 INFO:tasks.workunit.client.1.smithi070.stdout:3/305: dwrite db/d1c/d3e/f2b [0,4194304] 0 2024-09-10T04:05:12.173 INFO:tasks.workunit.client.0.smithi046.stdout:4/269: write d3/d12/d17/d22/f4e [413658,95] 0 2024-09-10T04:05:12.174 INFO:tasks.workunit.client.1.smithi070.stdout:8/312: creat d4/d16/d17/d5c/f6d x:0 0 0 2024-09-10T04:05:12.175 INFO:tasks.workunit.client.0.smithi046.stdout:1/212: unlink d3/dc/fe 0 2024-09-10T04:05:12.175 INFO:tasks.workunit.client.0.smithi046.stdout:6/243: dwrite da/db/d11/d13/d1e/f26 [0,4194304] 0 2024-09-10T04:05:12.177 INFO:tasks.workunit.client.0.smithi046.stdout:6/244: write da/fe [3666713,68470] 0 2024-09-10T04:05:12.178 INFO:tasks.workunit.client.0.smithi046.stdout:2/317: rename d2/d6/d23/d55/c58 to d2/d6/c7a 0 2024-09-10T04:05:12.181 INFO:tasks.workunit.client.1.smithi070.stdout:6/344: symlink d0/d3/d18/d62/l6e 0 2024-09-10T04:05:12.183 INFO:tasks.workunit.client.1.smithi070.stdout:6/345: write d0/d3/d18/d1b/d17/f47 [677893,127221] 0 2024-09-10T04:05:12.184 INFO:tasks.workunit.client.1.smithi070.stdout:6/346: chown d0/d3/d18/d1b/d17/c26 101532 1 2024-09-10T04:05:12.184 INFO:tasks.workunit.client.0.smithi046.stdout:3/247: creat d12/d13/d14/f47 x:0 0 0 2024-09-10T04:05:12.185 INFO:tasks.workunit.client.1.smithi070.stdout:6/347: chown d0/d3/d5c 884499083 1 2024-09-10T04:05:12.185 INFO:tasks.workunit.client.1.smithi070.stdout:2/255: chown d15/d18/d29/d2a/c38 755 1 2024-09-10T04:05:12.187 INFO:tasks.workunit.client.1.smithi070.stdout:2/256: dread d15/d18/f20 [0,4194304] 0 2024-09-10T04:05:12.189 INFO:tasks.workunit.client.0.smithi046.stdout:4/270: write d3/d12/d17/d29/f3e [580889,109825] 0 2024-09-10T04:05:12.190 INFO:tasks.workunit.client.1.smithi070.stdout:5/276: mknod d17/d24/d47/d54/c60 0 2024-09-10T04:05:12.191 INFO:tasks.workunit.client.1.smithi070.stdout:5/277: readlink d17/l26 0 2024-09-10T04:05:12.194 INFO:tasks.workunit.client.1.smithi070.stdout:6/348: dwrite d0/d3/d18/d1b/d17/d53/f61 [0,4194304] 0 2024-09-10T04:05:12.194 INFO:tasks.workunit.client.1.smithi070.stdout:4/217: creat de/d13/f45 x:0 0 0 2024-09-10T04:05:12.194 INFO:tasks.workunit.client.0.smithi046.stdout:2/318: mkdir d2/d6/d23/d5d/d77/d7b 0 2024-09-10T04:05:12.195 INFO:tasks.workunit.client.0.smithi046.stdout:2/319: stat d2/d6/d23/d5d/d77/f54 0 2024-09-10T04:05:12.195 INFO:tasks.workunit.client.0.smithi046.stdout:2/320: chown d2/d6/d23 624591 1 2024-09-10T04:05:12.196 INFO:tasks.workunit.client.1.smithi070.stdout:0/318: write d8/d13/d1a/f32 [4865698,32424] 0 2024-09-10T04:05:12.198 INFO:tasks.workunit.client.0.smithi046.stdout:0/228: mkdir d1/dc/d49 0 2024-09-10T04:05:12.199 INFO:tasks.workunit.client.1.smithi070.stdout:0/319: chown d8/f1b 14153 1 2024-09-10T04:05:12.200 INFO:tasks.workunit.client.1.smithi070.stdout:2/257: dwrite d15/d18/d29/d2a/d2e/d32/d36/f5b [0,4194304] 0 2024-09-10T04:05:12.201 INFO:tasks.workunit.client.0.smithi046.stdout:7/285: mknod d5/d6/db/d11/c63 0 2024-09-10T04:05:12.203 INFO:tasks.workunit.client.0.smithi046.stdout:8/266: creat d1/d29/d17/d1f/d46/d52/f60 x:0 0 0 2024-09-10T04:05:12.203 INFO:tasks.workunit.client.1.smithi070.stdout:2/258: chown d15/d18/d42/d43/d55/l5a 13135287 1 2024-09-10T04:05:12.203 INFO:tasks.workunit.client.1.smithi070.stdout:9/305: dwrite d9/d1c/d39/f44 [4194304,4194304] 0 2024-09-10T04:05:12.203 INFO:tasks.workunit.client.0.smithi046.stdout:3/248: rmdir d12/d1b 39 2024-09-10T04:05:12.204 INFO:tasks.workunit.client.1.smithi070.stdout:9/306: readlink d9/dd/d10/d1b/l29 0 2024-09-10T04:05:12.205 INFO:tasks.workunit.client.0.smithi046.stdout:4/271: mkdir d3/d12/d17/d22/d51/d5c 0 2024-09-10T04:05:12.205 INFO:tasks.workunit.client.0.smithi046.stdout:8/267: chown d1/d29/d19/d36/f3f 339925 1 2024-09-10T04:05:12.206 INFO:tasks.workunit.client.1.smithi070.stdout:1/238: getdents d3/d20 0 2024-09-10T04:05:12.208 INFO:tasks.workunit.client.0.smithi046.stdout:8/268: chown d1/d29/d17 28787155 1 2024-09-10T04:05:12.211 INFO:tasks.workunit.client.0.smithi046.stdout:7/286: dwrite d5/d6/d32/f5e [0,4194304] 0 2024-09-10T04:05:12.213 INFO:tasks.workunit.client.0.smithi046.stdout:9/276: rename d4/d7/c48 to d4/d3c/d4d/c58 0 2024-09-10T04:05:12.214 INFO:tasks.workunit.client.0.smithi046.stdout:7/287: chown d5/d6/db/d11 6 1 2024-09-10T04:05:12.215 INFO:tasks.workunit.client.1.smithi070.stdout:5/278: mknod d17/d24/d2b/d31/c61 0 2024-09-10T04:05:12.216 INFO:tasks.workunit.client.1.smithi070.stdout:3/306: creat db/d1c/d53/f5d x:0 0 0 2024-09-10T04:05:12.217 INFO:tasks.workunit.client.1.smithi070.stdout:8/313: creat d4/d16/d17/d19/d62/f6e x:0 0 0 2024-09-10T04:05:12.218 INFO:tasks.workunit.client.1.smithi070.stdout:8/314: write d4/d16/f33 [5577379,25252] 0 2024-09-10T04:05:12.220 INFO:tasks.workunit.client.0.smithi046.stdout:2/321: dread d2/d6/d23/d5d/d77/f69 [0,4194304] 0 2024-09-10T04:05:12.220 INFO:tasks.workunit.client.1.smithi070.stdout:6/349: mknod d0/d3/d18/d1b/d17/d54/c6f 0 2024-09-10T04:05:12.224 INFO:tasks.workunit.client.0.smithi046.stdout:4/272: dwrite d3/d12/d17/f1b [0,4194304] 0 2024-09-10T04:05:12.224 INFO:tasks.workunit.client.0.smithi046.stdout:7/288: truncate d5/d6/db/d11/d1e/d20/f4a 693432 0 2024-09-10T04:05:12.225 INFO:tasks.workunit.client.0.smithi046.stdout:2/322: dread - d2/d6/d23/d42/f53 zero size 2024-09-10T04:05:12.228 INFO:tasks.workunit.client.0.smithi046.stdout:4/273: stat d3/d12/l1e 0 2024-09-10T04:05:12.229 INFO:tasks.workunit.client.0.smithi046.stdout:7/289: chown d5/d6/db/f38 209909821 1 2024-09-10T04:05:12.229 INFO:tasks.workunit.client.1.smithi070.stdout:9/307: mknod d9/dd/d10/d3c/c57 0 2024-09-10T04:05:12.230 INFO:tasks.workunit.client.0.smithi046.stdout:5/368: rename d7/d11/d14/d50/f66 to d7/d11/d14/d2f/d2b/d6a/f82 0 2024-09-10T04:05:12.230 INFO:tasks.workunit.client.1.smithi070.stdout:9/308: stat d9/dd/d10/d1b/c28 0 2024-09-10T04:05:12.231 INFO:tasks.workunit.client.0.smithi046.stdout:6/245: link da/db/d11/d2b/d2f/c43 da/dc/d22/c53 0 2024-09-10T04:05:12.232 INFO:tasks.workunit.client.1.smithi070.stdout:1/239: mkdir d3/d8/d12/d2f/d32/d4b 0 2024-09-10T04:05:12.232 INFO:tasks.workunit.client.1.smithi070.stdout:1/240: chown d3/d10/d1d/f46 24617825 1 2024-09-10T04:05:12.234 INFO:tasks.workunit.client.0.smithi046.stdout:9/277: mkdir d4/d29/d4c/d59 0 2024-09-10T04:05:12.234 INFO:tasks.workunit.client.0.smithi046.stdout:0/229: dwrite d1/dc/d25/d33/d3/f46 [0,4194304] 0 2024-09-10T04:05:12.235 INFO:tasks.workunit.client.0.smithi046.stdout:6/246: read - da/db/d11/f32 zero size 2024-09-10T04:05:12.240 INFO:tasks.workunit.client.1.smithi070.stdout:3/307: rename db/d1c/l47 to db/d1c/d25/l5e 0 2024-09-10T04:05:12.240 INFO:tasks.workunit.client.0.smithi046.stdout:3/249: truncate d12/d13/d14/f3a 1223477 0 2024-09-10T04:05:12.243 INFO:tasks.workunit.client.0.smithi046.stdout:4/274: dwrite d3/d43/f45 [0,4194304] 0 2024-09-10T04:05:12.247 INFO:tasks.workunit.client.0.smithi046.stdout:6/247: dwrite da/db/f14 [0,4194304] 0 2024-09-10T04:05:12.248 INFO:tasks.workunit.client.1.smithi070.stdout:6/350: mkdir d0/d3/d18/d62/d70 0 2024-09-10T04:05:12.256 INFO:tasks.workunit.client.1.smithi070.stdout:9/309: mknod d9/dd/d10/d1b/c58 0 2024-09-10T04:05:12.256 INFO:tasks.workunit.client.0.smithi046.stdout:6/248: dwrite da/db/d11/d2b/f2c [0,4194304] 0 2024-09-10T04:05:12.261 INFO:tasks.workunit.client.1.smithi070.stdout:4/218: dwrite de/d13/d18/f21 [4194304,4194304] 0 2024-09-10T04:05:12.263 INFO:tasks.workunit.client.0.smithi046.stdout:7/290: rename d5/f7 to d5/d6/d39/d5a/f64 0 2024-09-10T04:05:12.266 INFO:tasks.workunit.client.1.smithi070.stdout:5/279: fsync d17/d19/d35/f5f 0 2024-09-10T04:05:12.267 INFO:tasks.workunit.client.0.smithi046.stdout:8/269: dwrite d1/d29/d17/f2f [0,4194304] 0 2024-09-10T04:05:12.269 INFO:tasks.workunit.client.0.smithi046.stdout:8/270: fsync d1/f39 0 2024-09-10T04:05:12.271 INFO:tasks.workunit.client.1.smithi070.stdout:8/315: mkdir d4/d16/d17/d6f 0 2024-09-10T04:05:12.272 INFO:tasks.workunit.client.1.smithi070.stdout:7/236: sync 2024-09-10T04:05:12.274 INFO:tasks.workunit.client.1.smithi070.stdout:6/351: unlink d0/d3/d18/d1b/d17/c25 0 2024-09-10T04:05:12.276 INFO:tasks.workunit.client.1.smithi070.stdout:6/352: dread - d0/d3/d18/d1b/d17/d28/f6b zero size 2024-09-10T04:05:12.277 INFO:tasks.workunit.client.1.smithi070.stdout:2/259: truncate d15/d2b/f4c 3556907 0 2024-09-10T04:05:12.277 INFO:tasks.workunit.client.1.smithi070.stdout:6/353: fsync d0/d3/d18/f52 0 2024-09-10T04:05:12.280 INFO:tasks.workunit.client.1.smithi070.stdout:1/241: dread d3/d8/fc [0,4194304] 0 2024-09-10T04:05:12.282 INFO:tasks.workunit.client.1.smithi070.stdout:1/242: truncate d3/d8/d12/f24 892759 0 2024-09-10T04:05:12.282 INFO:tasks.workunit.client.0.smithi046.stdout:6/249: mkdir da/db/d11/d13/d1e/d23/d54 0 2024-09-10T04:05:12.283 INFO:tasks.workunit.client.1.smithi070.stdout:6/354: dwrite d0/d3/f51 [0,4194304] 0 2024-09-10T04:05:12.283 INFO:tasks.workunit.client.0.smithi046.stdout:6/250: chown da/db/f14 1 1 2024-09-10T04:05:12.284 INFO:tasks.workunit.client.0.smithi046.stdout:6/251: stat da/db/d18 0 2024-09-10T04:05:12.284 INFO:tasks.workunit.client.1.smithi070.stdout:4/219: mkdir de/d13/d18/d26/d46 0 2024-09-10T04:05:12.287 INFO:tasks.workunit.client.1.smithi070.stdout:4/220: dread de/d13/d18/d23/f3c [0,4194304] 0 2024-09-10T04:05:12.292 INFO:tasks.workunit.client.1.smithi070.stdout:5/280: creat d17/d24/d47/d54/f62 x:0 0 0 2024-09-10T04:05:12.295 INFO:tasks.workunit.client.0.smithi046.stdout:9/278: rename l3 to d4/d29/d4c/d59/l5a 0 2024-09-10T04:05:12.298 INFO:tasks.workunit.client.1.smithi070.stdout:8/316: mknod d4/d16/d17/d19/d28/d43/d52/c70 0 2024-09-10T04:05:12.302 INFO:tasks.workunit.client.0.smithi046.stdout:8/271: symlink d1/d29/d19/d36/l61 0 2024-09-10T04:05:12.305 INFO:tasks.workunit.client.0.smithi046.stdout:0/230: creat d1/dc/d49/f4a x:0 0 0 2024-09-10T04:05:12.307 INFO:tasks.workunit.client.0.smithi046.stdout:5/369: truncate d7/d11/f1a 7514607 0 2024-09-10T04:05:12.310 INFO:tasks.workunit.client.0.smithi046.stdout:4/275: getdents d3/d12/d17/d29/d58 0 2024-09-10T04:05:12.310 INFO:tasks.workunit.client.1.smithi070.stdout:2/260: chown d15/d18/d29/d2a/d2e/l30 1 1 2024-09-10T04:05:12.310 INFO:tasks.workunit.client.1.smithi070.stdout:2/261: truncate f3 5769844 0 2024-09-10T04:05:12.312 INFO:tasks.workunit.client.1.smithi070.stdout:1/243: creat d3/d8/d12/d2f/d32/f4c x:0 0 0 2024-09-10T04:05:12.313 INFO:tasks.workunit.client.1.smithi070.stdout:1/244: write d3/d8/f49 [999371,77848] 0 2024-09-10T04:05:12.313 INFO:tasks.workunit.client.1.smithi070.stdout:2/262: dread f10 [4194304,4194304] 0 2024-09-10T04:05:12.319 INFO:tasks.workunit.client.0.smithi046.stdout:9/279: readlink d4/d7/d1f/l3a 0 2024-09-10T04:05:12.321 INFO:tasks.workunit.client.1.smithi070.stdout:4/221: mkdir de/d13/d18/d23/d47 0 2024-09-10T04:05:12.321 INFO:tasks.workunit.client.1.smithi070.stdout:4/222: dread - de/d13/f38 zero size 2024-09-10T04:05:12.321 INFO:tasks.workunit.client.0.smithi046.stdout:7/291: mknod d5/d6/d39/d5f/c65 0 2024-09-10T04:05:12.322 INFO:tasks.workunit.client.1.smithi070.stdout:5/281: read d17/d19/d35/f5f [175656,129447] 0 2024-09-10T04:05:12.322 INFO:tasks.workunit.client.0.smithi046.stdout:7/292: chown d5/d6/d39/d5f 114176377 1 2024-09-10T04:05:12.323 INFO:tasks.workunit.client.0.smithi046.stdout:8/272: mkdir d1/d29/d19/d36/d62 0 2024-09-10T04:05:12.327 INFO:tasks.workunit.client.1.smithi070.stdout:3/308: creat db/d1c/f5f x:0 0 0 2024-09-10T04:05:12.329 INFO:tasks.workunit.client.0.smithi046.stdout:5/370: creat d7/d11/d14/d50/f83 x:0 0 0 2024-09-10T04:05:12.329 INFO:tasks.workunit.client.0.smithi046.stdout:5/371: dread - d7/d11/d14/d50/f83 zero size 2024-09-10T04:05:12.330 INFO:tasks.workunit.client.0.smithi046.stdout:2/323: getdents d2/d6/d23 0 2024-09-10T04:05:12.333 INFO:tasks.workunit.client.0.smithi046.stdout:4/276: symlink d3/d12/d17/d22/d30/l5d 0 2024-09-10T04:05:12.333 INFO:tasks.workunit.client.0.smithi046.stdout:4/277: readlink d3/d12/d17/d22/d30/l31 0 2024-09-10T04:05:12.335 INFO:tasks.workunit.client.1.smithi070.stdout:4/223: dread de/d13/d18/f21 [0,4194304] 0 2024-09-10T04:05:12.337 INFO:tasks.workunit.client.1.smithi070.stdout:9/310: rename d9/dd to d9/d59 0 2024-09-10T04:05:12.337 INFO:tasks.workunit.client.1.smithi070.stdout:9/311: dread - d9/d1c/f55 zero size 2024-09-10T04:05:12.340 INFO:tasks.workunit.client.0.smithi046.stdout:3/250: rename d12/d13/d14/l2f to d12/d13/l48 0 2024-09-10T04:05:12.340 INFO:tasks.workunit.client.0.smithi046.stdout:6/252: write da/db/d10/d27/f36 [2409616,126869] 0 2024-09-10T04:05:12.342 INFO:tasks.workunit.client.0.smithi046.stdout:3/251: write d12/d13/f17 [5602515,58640] 0 2024-09-10T04:05:12.342 INFO:tasks.workunit.client.1.smithi070.stdout:2/263: mknod d15/d18/d42/c5d 0 2024-09-10T04:05:12.343 INFO:tasks.workunit.client.0.smithi046.stdout:3/252: stat d12/d13/f2c 0 2024-09-10T04:05:12.343 INFO:tasks.workunit.client.0.smithi046.stdout:7/293: creat d5/d9/d43/f66 x:0 0 0 2024-09-10T04:05:12.345 INFO:tasks.workunit.client.0.smithi046.stdout:8/273: mknod d1/d29/d17/d1f/d46/d52/c63 0 2024-09-10T04:05:12.346 INFO:tasks.workunit.client.0.smithi046.stdout:8/274: write d1/d29/d19/d36/f3f [2094977,62573] 0 2024-09-10T04:05:12.351 INFO:tasks.workunit.client.0.smithi046.stdout:4/278: creat d3/d12/d17/f5e x:0 0 0 2024-09-10T04:05:12.351 INFO:tasks.workunit.client.1.smithi070.stdout:5/282: fsync d17/d24/f27 0 2024-09-10T04:05:12.352 INFO:tasks.workunit.client.1.smithi070.stdout:3/309: mknod db/c60 0 2024-09-10T04:05:12.353 INFO:tasks.workunit.client.1.smithi070.stdout:3/310: dread - db/d1c/d53/f57 zero size 2024-09-10T04:05:12.353 INFO:tasks.workunit.client.0.smithi046.stdout:7/294: dread d5/d6/db/dd/d4b/f52 [0,4194304] 0 2024-09-10T04:05:12.354 INFO:tasks.workunit.client.0.smithi046.stdout:7/295: readlink d5/d6/l3a 0 2024-09-10T04:05:12.354 INFO:tasks.workunit.client.1.smithi070.stdout:3/311: write db/d1c/d3e/f59 [909461,124739] 0 2024-09-10T04:05:12.356 INFO:tasks.workunit.client.0.smithi046.stdout:6/253: dread da/db/d10/f1a [0,4194304] 0 2024-09-10T04:05:12.357 INFO:tasks.workunit.client.0.smithi046.stdout:6/254: read da/db/d11/d13/d1e/f26 [98961,20701] 0 2024-09-10T04:05:12.358 INFO:tasks.workunit.client.0.smithi046.stdout:9/280: rename d4/d29/d4c/c51 to d4/d29/d4c/c5b 0 2024-09-10T04:05:12.362 INFO:tasks.workunit.client.1.smithi070.stdout:6/355: truncate d0/d3/d18/d1b/d17/f47 29777 0 2024-09-10T04:05:12.363 INFO:tasks.workunit.client.1.smithi070.stdout:1/245: mknod d3/d10/c4d 0 2024-09-10T04:05:12.364 INFO:tasks.workunit.client.1.smithi070.stdout:1/246: readlink d3/la 0 2024-09-10T04:05:12.365 INFO:tasks.workunit.client.1.smithi070.stdout:2/264: dread d15/d18/f20 [0,4194304] 0 2024-09-10T04:05:12.367 INFO:tasks.workunit.client.0.smithi046.stdout:0/231: dwrite d1/dc/d25/d33/d3/f14 [0,4194304] 0 2024-09-10T04:05:12.368 INFO:tasks.workunit.client.1.smithi070.stdout:8/317: truncate d4/d16/d17/f68 611387 0 2024-09-10T04:05:12.369 INFO:tasks.workunit.client.0.smithi046.stdout:8/275: mknod d1/d29/d19/d36/d4c/c64 0 2024-09-10T04:05:12.370 INFO:tasks.workunit.client.0.smithi046.stdout:8/276: chown d1/d29/d17/d1f/d2a/l2b 7687256 1 2024-09-10T04:05:12.372 INFO:tasks.workunit.client.0.smithi046.stdout:8/277: truncate d1/f2e 315958 0 2024-09-10T04:05:12.373 INFO:tasks.workunit.client.1.smithi070.stdout:3/312: unlink db/d1c/d3e/f2b 0 2024-09-10T04:05:12.373 INFO:tasks.workunit.client.0.smithi046.stdout:8/278: read d1/d29/d12/f45 [3399691,64124] 0 2024-09-10T04:05:12.374 INFO:tasks.workunit.client.1.smithi070.stdout:3/313: chown db/d1c/c49 97584 1 2024-09-10T04:05:12.374 INFO:tasks.workunit.client.0.smithi046.stdout:4/279: mkdir d3/d43/d5f 0 2024-09-10T04:05:12.375 INFO:tasks.workunit.client.0.smithi046.stdout:0/232: dwrite d1/d13/f48 [0,4194304] 0 2024-09-10T04:05:12.377 INFO:tasks.workunit.client.1.smithi070.stdout:7/237: getdents d2/d7/db/d20 0 2024-09-10T04:05:12.378 INFO:tasks.workunit.client.1.smithi070.stdout:7/238: readlink d2/d3/dc/l48 0 2024-09-10T04:05:12.378 INFO:tasks.workunit.client.1.smithi070.stdout:7/239: readlink d2/d7/db/lf 0 2024-09-10T04:05:12.380 INFO:tasks.workunit.client.0.smithi046.stdout:7/296: mknod d5/d6/db/d11/d1e/d20/c67 0 2024-09-10T04:05:12.381 INFO:tasks.workunit.client.0.smithi046.stdout:7/297: readlink d5/d9/d12/l34 0 2024-09-10T04:05:12.382 INFO:tasks.workunit.client.1.smithi070.stdout:4/224: mkdir de/d13/d18/d23/d39/d48 0 2024-09-10T04:05:12.382 INFO:tasks.workunit.client.1.smithi070.stdout:7/240: dread d2/d7/db/f37 [0,4194304] 0 2024-09-10T04:05:12.384 INFO:tasks.workunit.client.0.smithi046.stdout:6/255: mkdir da/db/d11/d2b/d55 0 2024-09-10T04:05:12.385 INFO:tasks.workunit.client.0.smithi046.stdout:6/256: readlink l2 0 2024-09-10T04:05:12.386 INFO:tasks.workunit.client.1.smithi070.stdout:1/247: chown d3/d8/d13/c36 0 1 2024-09-10T04:05:12.387 INFO:tasks.workunit.client.1.smithi070.stdout:1/248: write d3/d20/f44 [620412,111332] 0 2024-09-10T04:05:12.387 INFO:tasks.workunit.client.0.smithi046.stdout:9/281: rename d4/d29/d4c to d4/d3c/d4d/d5c 0 2024-09-10T04:05:12.390 INFO:tasks.workunit.client.0.smithi046.stdout:3/253: mkdir d12/d13/d42/d49 0 2024-09-10T04:05:12.390 INFO:tasks.workunit.client.0.smithi046.stdout:6/257: dread da/db/d10/f17 [4194304,4194304] 0 2024-09-10T04:05:12.393 INFO:tasks.workunit.client.1.smithi070.stdout:1/249: dwrite d3/d8/d13/f43 [0,4194304] 0 2024-09-10T04:05:12.394 INFO:tasks.workunit.client.0.smithi046.stdout:9/282: dread d4/d18/f22 [0,4194304] 0 2024-09-10T04:05:12.394 INFO:tasks.workunit.client.1.smithi070.stdout:1/250: write d3/d8/d13/f17 [3014589,84287] 0 2024-09-10T04:05:12.397 INFO:tasks.workunit.client.0.smithi046.stdout:5/372: link d7/d11/d14/d45/c40 d7/d11/d3f/d53/d81/c84 0 2024-09-10T04:05:12.400 INFO:tasks.workunit.client.0.smithi046.stdout:2/324: link d2/d6/d23/d42/c4a d2/d6/d23/d42/c7c 0 2024-09-10T04:05:12.400 INFO:tasks.workunit.client.0.smithi046.stdout:7/298: dread d5/d6/db/dd/f44 [0,4194304] 0 2024-09-10T04:05:12.401 INFO:tasks.workunit.client.0.smithi046.stdout:7/299: chown d5/d6/db/f1d 2242079 1 2024-09-10T04:05:12.402 INFO:tasks.workunit.client.1.smithi070.stdout:3/314: unlink db/d1c/d25/f4d 0 2024-09-10T04:05:12.403 INFO:tasks.workunit.client.1.smithi070.stdout:3/315: write db/f11 [5105409,127014] 0 2024-09-10T04:05:12.407 INFO:tasks.workunit.client.1.smithi070.stdout:6/356: write d0/d3/d18/f27 [2212893,95887] 0 2024-09-10T04:05:12.408 INFO:tasks.workunit.client.1.smithi070.stdout:7/241: mknod d2/d3/c54 0 2024-09-10T04:05:12.410 INFO:tasks.workunit.client.1.smithi070.stdout:7/242: fdatasync d2/d7/d9/d18/d23/f38 0 2024-09-10T04:05:12.412 INFO:tasks.workunit.client.1.smithi070.stdout:7/243: truncate d2/f4c 4434669 0 2024-09-10T04:05:12.414 INFO:tasks.workunit.client.1.smithi070.stdout:5/283: dwrite d17/d19/f1d [0,4194304] 0 2024-09-10T04:05:12.414 INFO:tasks.workunit.client.1.smithi070.stdout:7/244: dread - d2/d7/d9/d18/d23/f47 zero size 2024-09-10T04:05:12.414 INFO:tasks.workunit.client.1.smithi070.stdout:3/316: dread db/f14 [0,4194304] 0 2024-09-10T04:05:12.415 INFO:tasks.workunit.client.1.smithi070.stdout:5/284: chown d17/d24/d47 1644362 1 2024-09-10T04:05:12.415 INFO:tasks.workunit.client.0.smithi046.stdout:4/280: rename d3/d12/d17/d29/c55 to d3/d12/d17/c60 0 2024-09-10T04:05:12.415 INFO:tasks.workunit.client.0.smithi046.stdout:0/233: dwrite d1/dc/f21 [0,4194304] 0 2024-09-10T04:05:12.417 INFO:tasks.workunit.client.0.smithi046.stdout:4/281: write d3/d12/d14/f16 [2685274,74556] 0 2024-09-10T04:05:12.419 INFO:tasks.workunit.client.0.smithi046.stdout:3/254: mknod d12/d13/d42/c4a 0 2024-09-10T04:05:12.424 INFO:tasks.workunit.client.0.smithi046.stdout:9/283: creat d4/d3c/d4d/d5c/d59/f5d x:0 0 0 2024-09-10T04:05:12.426 INFO:tasks.workunit.client.0.smithi046.stdout:9/284: truncate d4/d18/f4f 292770 0 2024-09-10T04:05:12.427 INFO:tasks.workunit.client.0.smithi046.stdout:9/285: chown d4/d29/d32/f55 28 1 2024-09-10T04:05:12.428 INFO:tasks.workunit.client.1.smithi070.stdout:8/318: rename d4/d16/d17/d1a/d23/f5e to d4/d16/d17/d19/d62/f71 0 2024-09-10T04:05:12.428 INFO:tasks.workunit.client.0.smithi046.stdout:9/286: fdatasync d4/d7/f1e 0 2024-09-10T04:05:12.429 INFO:tasks.workunit.client.1.smithi070.stdout:8/319: chown d4/d16/d17/d19/d62/f6e 120 1 2024-09-10T04:05:12.432 INFO:tasks.workunit.client.0.smithi046.stdout:7/300: rename d5/d6/db/d11/c57 to d5/d6/db/d11/d1f/c68 0 2024-09-10T04:05:12.434 INFO:tasks.workunit.client.1.smithi070.stdout:8/320: dwrite d4/d16/d17/f2c [0,4194304] 0 2024-09-10T04:05:12.434 INFO:tasks.workunit.client.0.smithi046.stdout:7/301: write d5/d6/db/d11/d1e/f3e [229429,130608] 0 2024-09-10T04:05:12.434 INFO:tasks.workunit.client.0.smithi046.stdout:6/258: dread da/db/d11/d13/f39 [0,4194304] 0 2024-09-10T04:05:12.439 INFO:tasks.workunit.client.0.smithi046.stdout:4/282: rmdir d3 39 2024-09-10T04:05:12.445 INFO:tasks.workunit.client.1.smithi070.stdout:0/320: sync 2024-09-10T04:05:12.445 INFO:tasks.workunit.client.0.smithi046.stdout:2/325: truncate d2/d6/d15/f3e 8902154 0 2024-09-10T04:05:12.448 INFO:tasks.workunit.client.1.smithi070.stdout:1/251: rename d3/d8/d27/c3d to d3/d8/d12/c4e 0 2024-09-10T04:05:12.449 INFO:tasks.workunit.client.1.smithi070.stdout:1/252: read d3/d8/d13/f17 [1216481,102001] 0 2024-09-10T04:05:12.450 INFO:tasks.workunit.client.1.smithi070.stdout:1/253: chown d3/d8/d13/d3f 0 1 2024-09-10T04:05:12.452 INFO:tasks.workunit.client.1.smithi070.stdout:6/357: symlink d0/d3/d18/d1b/d17/d54/l71 0 2024-09-10T04:05:12.454 INFO:tasks.workunit.client.1.smithi070.stdout:6/358: read d0/d3/d18/f27 [190224,80689] 0 2024-09-10T04:05:12.455 INFO:tasks.workunit.client.0.smithi046.stdout:3/255: mkdir d12/d1b/d4b 0 2024-09-10T04:05:12.455 INFO:tasks.workunit.client.0.smithi046.stdout:5/373: link d7/d11/d14/d2f/l3b d7/d11/d3f/d5a/l85 0 2024-09-10T04:05:12.455 INFO:tasks.workunit.client.1.smithi070.stdout:2/265: link d15/d18/d29/d2a/l50 d15/d18/d29/d2a/d2e/l5e 0 2024-09-10T04:05:12.456 INFO:tasks.workunit.client.1.smithi070.stdout:5/285: mkdir d17/d24/d47/d54/d57/d63 0 2024-09-10T04:05:12.458 INFO:tasks.workunit.client.1.smithi070.stdout:0/321: mkdir d8/d24/d2d/d34/d6b 0 2024-09-10T04:05:12.458 INFO:tasks.workunit.client.0.smithi046.stdout:2/326: truncate d2/d6/d23/d5d/d77/f69 1394532 0 2024-09-10T04:05:12.458 INFO:tasks.workunit.client.1.smithi070.stdout:0/322: chown d8/d13/d33/d4f 5451213 1 2024-09-10T04:05:12.459 INFO:tasks.workunit.client.0.smithi046.stdout:8/279: getdents d1/d29/d19/d36/d4c/d5e 0 2024-09-10T04:05:12.459 INFO:tasks.workunit.client.1.smithi070.stdout:2/266: dread f11 [0,4194304] 0 2024-09-10T04:05:12.460 INFO:tasks.workunit.client.0.smithi046.stdout:8/280: readlink d1/d29/d17/d1f/d2a/l37 0 2024-09-10T04:05:12.461 INFO:tasks.workunit.client.0.smithi046.stdout:7/302: link d5/f56 d5/d6/d39/f69 0 2024-09-10T04:05:12.462 INFO:tasks.workunit.client.0.smithi046.stdout:6/259: link da/f24 da/d45/f56 0 2024-09-10T04:05:12.465 INFO:tasks.workunit.client.0.smithi046.stdout:4/283: creat d3/d12/f61 x:0 0 0 2024-09-10T04:05:12.466 INFO:tasks.workunit.client.1.smithi070.stdout:7/245: creat d2/f55 x:0 0 0 2024-09-10T04:05:12.467 INFO:tasks.workunit.client.0.smithi046.stdout:3/256: fsync f4 0 2024-09-10T04:05:12.468 INFO:tasks.workunit.client.0.smithi046.stdout:3/257: stat d12 0 2024-09-10T04:05:12.470 INFO:tasks.workunit.client.0.smithi046.stdout:5/374: creat d7/d11/d14/d45/d51/f86 x:0 0 0 2024-09-10T04:05:12.471 INFO:tasks.workunit.client.0.smithi046.stdout:9/287: rmdir d4/d57 0 2024-09-10T04:05:12.472 INFO:tasks.workunit.client.0.smithi046.stdout:9/288: chown d4/d29/f3d 130513 1 2024-09-10T04:05:12.476 INFO:tasks.workunit.client.1.smithi070.stdout:8/321: truncate d4/d16/d17/d1a/f2a 680540 0 2024-09-10T04:05:12.476 INFO:tasks.workunit.client.0.smithi046.stdout:8/281: rmdir d1/d29/d19/d36/d4c/d5e 39 2024-09-10T04:05:12.479 INFO:tasks.workunit.client.0.smithi046.stdout:5/375: dread d7/f1f [0,4194304] 0 2024-09-10T04:05:12.479 INFO:tasks.workunit.client.0.smithi046.stdout:7/303: rename d5/d6/db/dd/d24/c4f to d5/d6/db/dd/d24/c6a 0 2024-09-10T04:05:12.481 INFO:tasks.workunit.client.1.smithi070.stdout:1/254: unlink d3/f7 0 2024-09-10T04:05:12.481 INFO:tasks.workunit.client.1.smithi070.stdout:2/267: rmdir d15/d18/d42/d43 39 2024-09-10T04:05:12.482 INFO:tasks.workunit.client.0.smithi046.stdout:0/234: getdents d1/d13/d37/d3e 0 2024-09-10T04:05:12.483 INFO:tasks.workunit.client.0.smithi046.stdout:7/304: write d5/d6/db/d11/d1f/f48 [261461,87944] 0 2024-09-10T04:05:12.484 INFO:tasks.workunit.client.1.smithi070.stdout:7/246: mknod d2/d3/dc/d1b/d42/c56 0 2024-09-10T04:05:12.485 INFO:tasks.workunit.client.0.smithi046.stdout:8/282: dwrite d1/d29/d17/d1f/d2a/f58 [0,4194304] 0 2024-09-10T04:05:12.485 INFO:tasks.workunit.client.0.smithi046.stdout:4/284: creat d3/d12/d14/f62 x:0 0 0 2024-09-10T04:05:12.486 INFO:tasks.workunit.client.1.smithi070.stdout:3/317: getdents db/d1c 0 2024-09-10T04:05:12.487 INFO:tasks.workunit.client.0.smithi046.stdout:3/258: rmdir d12/d13/d14/d1e 39 2024-09-10T04:05:12.491 INFO:tasks.workunit.client.0.smithi046.stdout:2/327: creat d2/d8/d65/f7d x:0 0 0 2024-09-10T04:05:12.492 INFO:tasks.workunit.client.1.smithi070.stdout:8/322: truncate d4/d16/f22 279589 0 2024-09-10T04:05:12.492 INFO:tasks.workunit.client.0.smithi046.stdout:4/285: dread d3/f9 [0,4194304] 0 2024-09-10T04:05:12.494 INFO:tasks.workunit.client.0.smithi046.stdout:4/286: write d3/d12/d17/d22/f5b [748534,76338] 0 2024-09-10T04:05:12.496 INFO:tasks.workunit.client.1.smithi070.stdout:7/247: symlink d2/d3/l57 0 2024-09-10T04:05:12.497 INFO:tasks.workunit.client.0.smithi046.stdout:5/376: symlink d7/d11/d14/d45/l87 0 2024-09-10T04:05:12.498 INFO:tasks.workunit.client.0.smithi046.stdout:5/377: chown d7/d11/d14/d2f/d2b 7288 1 2024-09-10T04:05:12.501 INFO:tasks.workunit.client.1.smithi070.stdout:5/286: rename d17/d19/f22 to d17/f64 0 2024-09-10T04:05:12.503 INFO:tasks.workunit.client.1.smithi070.stdout:3/318: symlink db/l61 0 2024-09-10T04:05:12.503 INFO:tasks.workunit.client.1.smithi070.stdout:7/248: dwrite d2/d7/d9/d18/d23/f38 [4194304,4194304] 0 2024-09-10T04:05:12.504 INFO:tasks.workunit.client.0.smithi046.stdout:0/235: mknod d1/dc/d25/d33/d3/dd/c4b 0 2024-09-10T04:05:12.505 INFO:tasks.workunit.client.1.smithi070.stdout:8/323: dread d4/f11 [0,4194304] 0 2024-09-10T04:05:12.506 INFO:tasks.workunit.client.1.smithi070.stdout:0/323: rmdir d8/d13/d1a/d62 0 2024-09-10T04:05:12.507 INFO:tasks.workunit.client.1.smithi070.stdout:0/324: dread - d8/d24/d2d/d5f/f60 zero size 2024-09-10T04:05:12.508 INFO:tasks.workunit.client.1.smithi070.stdout:1/255: write d3/d10/f35 [5086475,86953] 0 2024-09-10T04:05:12.510 INFO:tasks.workunit.client.1.smithi070.stdout:0/325: write d8/d13/d1a/f6a [872952,41509] 0 2024-09-10T04:05:12.511 INFO:tasks.workunit.client.1.smithi070.stdout:7/249: dwrite d2/d7/d9/d18/d23/f47 [0,4194304] 0 2024-09-10T04:05:12.512 INFO:tasks.workunit.client.0.smithi046.stdout:0/236: dwrite d1/dc/f16 [0,4194304] 0 2024-09-10T04:05:12.514 INFO:tasks.workunit.client.0.smithi046.stdout:1/213: sync 2024-09-10T04:05:12.516 INFO:tasks.workunit.client.0.smithi046.stdout:3/259: dread d12/d13/d14/f25 [0,4194304] 0 2024-09-10T04:05:12.518 INFO:tasks.workunit.client.1.smithi070.stdout:2/268: link d15/d18/d29/d2a/d2e/d32/f49 d15/d18/d56/f5f 0 2024-09-10T04:05:12.520 INFO:tasks.workunit.client.1.smithi070.stdout:9/312: sync 2024-09-10T04:05:12.521 INFO:tasks.workunit.client.1.smithi070.stdout:0/326: dwrite d8/d13/f5a [4194304,4194304] 0 2024-09-10T04:05:12.530 INFO:tasks.workunit.client.0.smithi046.stdout:2/328: rename d2/d6/d23/d5d/d5e/f46 to d2/d6/d23/d55/f7e 0 2024-09-10T04:05:12.531 INFO:tasks.workunit.client.0.smithi046.stdout:2/329: fsync d2/d49/f2e 0 2024-09-10T04:05:12.531 INFO:tasks.workunit.client.1.smithi070.stdout:3/319: creat db/d1c/f62 x:0 0 0 2024-09-10T04:05:12.535 INFO:tasks.workunit.client.0.smithi046.stdout:5/378: creat d7/d11/d3f/d5a/d63/f88 x:0 0 0 2024-09-10T04:05:12.538 INFO:tasks.workunit.client.0.smithi046.stdout:6/260: link da/dc/d22/f52 da/db/d11/d13/f57 0 2024-09-10T04:05:12.539 INFO:tasks.workunit.client.1.smithi070.stdout:1/256: symlink d3/d20/l4f 0 2024-09-10T04:05:12.540 INFO:tasks.workunit.client.1.smithi070.stdout:5/287: write f12 [4211299,3829] 0 2024-09-10T04:05:12.542 INFO:tasks.workunit.client.1.smithi070.stdout:1/257: dread d3/d8/d13/f1b [0,4194304] 0 2024-09-10T04:05:12.544 INFO:tasks.workunit.client.1.smithi070.stdout:7/250: creat d2/d7/d9/d18/d3c/f58 x:0 0 0 2024-09-10T04:05:12.545 INFO:tasks.workunit.client.1.smithi070.stdout:8/324: write d4/f29 [2579521,8305] 0 2024-09-10T04:05:12.546 INFO:tasks.workunit.client.0.smithi046.stdout:0/237: creat d1/d13/d37/d3e/f4c x:0 0 0 2024-09-10T04:05:12.547 INFO:tasks.workunit.client.0.smithi046.stdout:0/238: write d1/dc/d25/d33/f1b [1211487,72298] 0 2024-09-10T04:05:12.548 INFO:tasks.workunit.client.1.smithi070.stdout:1/258: dwrite d3/d10/d1d/f30 [0,4194304] 0 2024-09-10T04:05:12.551 INFO:tasks.workunit.client.1.smithi070.stdout:9/313: fdatasync d9/d59/d10/d1b/f45 0 2024-09-10T04:05:12.552 INFO:tasks.workunit.client.0.smithi046.stdout:7/305: truncate d5/d9/f42 3944014 0 2024-09-10T04:05:12.556 INFO:tasks.workunit.client.0.smithi046.stdout:8/283: truncate d1/d29/d17/d1f/d2a/f33 2032234 0 2024-09-10T04:05:12.559 INFO:tasks.workunit.client.0.smithi046.stdout:8/284: write d1/d29/d19/d36/f3f [2023911,25480] 0 2024-09-10T04:05:12.561 INFO:tasks.workunit.client.0.smithi046.stdout:7/306: dwrite d5/d6/db/f1d [0,4194304] 0 2024-09-10T04:05:12.563 INFO:tasks.workunit.client.0.smithi046.stdout:2/330: mknod d2/d8/d65/c7f 0 2024-09-10T04:05:12.564 INFO:tasks.workunit.client.1.smithi070.stdout:3/320: dwrite db/d1c/f62 [0,4194304] 0 2024-09-10T04:05:12.564 INFO:tasks.workunit.client.1.smithi070.stdout:4/225: sync 2024-09-10T04:05:12.564 INFO:tasks.workunit.client.1.smithi070.stdout:6/359: sync 2024-09-10T04:05:12.565 INFO:tasks.workunit.client.1.smithi070.stdout:4/226: readlink de/d13/d18/d23/l27 0 2024-09-10T04:05:12.566 INFO:tasks.workunit.client.1.smithi070.stdout:5/288: rmdir d17/d24/d2b 39 2024-09-10T04:05:12.570 INFO:tasks.workunit.client.0.smithi046.stdout:6/261: symlink da/db/d11/d13/d1e/d23/l58 0 2024-09-10T04:05:12.570 INFO:tasks.workunit.client.0.smithi046.stdout:7/307: dwrite d5/d6/db/d11/f4e [0,4194304] 0 2024-09-10T04:05:12.571 INFO:tasks.workunit.client.0.smithi046.stdout:1/214: symlink d3/d27/l49 0 2024-09-10T04:05:12.573 INFO:tasks.workunit.client.0.smithi046.stdout:6/262: read - da/db/d11/d13/d1e/f3e zero size 2024-09-10T04:05:12.574 INFO:tasks.workunit.client.0.smithi046.stdout:5/379: dread d7/f1f [0,4194304] 0 2024-09-10T04:05:12.574 INFO:tasks.workunit.client.0.smithi046.stdout:1/215: dread - d3/d27/d32/f43 zero size 2024-09-10T04:05:12.576 INFO:tasks.workunit.client.1.smithi070.stdout:1/259: write d3/d8/d13/f14 [4147619,72423] 0 2024-09-10T04:05:12.576 INFO:tasks.workunit.client.0.smithi046.stdout:6/263: chown da/db/d10/d27/l2d 178 1 2024-09-10T04:05:12.577 INFO:tasks.workunit.client.0.smithi046.stdout:5/380: stat d7/d11/d14/d45/f5f 0 2024-09-10T04:05:12.580 INFO:tasks.workunit.client.1.smithi070.stdout:9/314: write d9/d3e/f47 [2163308,117250] 0 2024-09-10T04:05:12.583 INFO:tasks.workunit.client.1.smithi070.stdout:0/327: symlink d8/d13/l6c 0 2024-09-10T04:05:12.584 INFO:tasks.workunit.client.0.smithi046.stdout:9/289: getdents d4/d7/d1f/d54 0 2024-09-10T04:05:12.584 INFO:tasks.workunit.client.0.smithi046.stdout:1/216: dwrite d3/d27/d32/d1f/f35 [0,4194304] 0 2024-09-10T04:05:12.586 INFO:tasks.workunit.client.1.smithi070.stdout:0/328: dread d8/d13/d1a/f6a [0,4194304] 0 2024-09-10T04:05:12.586 INFO:tasks.workunit.client.0.smithi046.stdout:9/290: truncate d4/d29/d32/f55 381417 0 2024-09-10T04:05:12.587 INFO:tasks.workunit.client.1.smithi070.stdout:0/329: fsync d8/d24/f2a 0 2024-09-10T04:05:12.587 INFO:tasks.workunit.client.0.smithi046.stdout:8/285: unlink d1/d29/l50 0 2024-09-10T04:05:12.588 INFO:tasks.workunit.client.0.smithi046.stdout:9/291: write d4/d3c/f40 [228923,67955] 0 2024-09-10T04:05:12.590 INFO:tasks.workunit.client.0.smithi046.stdout:9/292: stat d4/d7/d1f/f21 0 2024-09-10T04:05:12.594 INFO:tasks.workunit.client.1.smithi070.stdout:5/289: creat d17/d24/d47/d54/d57/f65 x:0 0 0 2024-09-10T04:05:12.596 INFO:tasks.workunit.client.0.smithi046.stdout:4/287: dwrite d3/d12/d17/d22/f27 [0,4194304] 0 2024-09-10T04:05:12.596 INFO:tasks.workunit.client.1.smithi070.stdout:7/251: getdents d2/d7/d9/d18/d23 0 2024-09-10T04:05:12.597 INFO:tasks.workunit.client.1.smithi070.stdout:8/325: getdents d4/d16/d17/d19/d28/d43/d55 0 2024-09-10T04:05:12.599 INFO:tasks.workunit.client.1.smithi070.stdout:4/227: dwrite de/d13/d18/d26/f2c [0,4194304] 0 2024-09-10T04:05:12.600 INFO:tasks.workunit.client.1.smithi070.stdout:8/326: chown d4/d16/d17/f2c 52815184 1 2024-09-10T04:05:12.607 INFO:tasks.workunit.client.1.smithi070.stdout:2/269: dwrite d15/d18/f27 [0,4194304] 0 2024-09-10T04:05:12.610 INFO:tasks.workunit.client.1.smithi070.stdout:3/321: mknod db/d1c/d53/c63 0 2024-09-10T04:05:12.611 INFO:tasks.workunit.client.1.smithi070.stdout:7/252: dwrite d2/d7/d9/d18/d3c/f58 [0,4194304] 0 2024-09-10T04:05:12.614 INFO:tasks.workunit.client.1.smithi070.stdout:8/327: dwrite d4/fa [4194304,4194304] 0 2024-09-10T04:05:12.621 INFO:tasks.workunit.client.0.smithi046.stdout:4/288: write d3/fa [3294580,102007] 0 2024-09-10T04:05:12.624 INFO:tasks.workunit.client.1.smithi070.stdout:2/270: dwrite fa [4194304,4194304] 0 2024-09-10T04:05:12.625 INFO:tasks.workunit.client.0.smithi046.stdout:0/239: truncate d1/dc/d25/d33/d3/d29/d2d/f32 264739 0 2024-09-10T04:05:12.625 INFO:tasks.workunit.client.0.smithi046.stdout:7/308: write d5/d6/d39/f69 [294813,102358] 0 2024-09-10T04:05:12.625 INFO:tasks.workunit.client.0.smithi046.stdout:8/286: unlink d1/d29/d17/d1f/l26 0 2024-09-10T04:05:12.626 INFO:tasks.workunit.client.0.smithi046.stdout:2/331: truncate d2/d49/f2e 851905 0 2024-09-10T04:05:12.626 INFO:tasks.workunit.client.1.smithi070.stdout:7/253: dwrite d2/d7/d9/f31 [0,4194304] 0 2024-09-10T04:05:12.629 INFO:tasks.workunit.client.0.smithi046.stdout:3/260: getdents d12/d13/d37 0 2024-09-10T04:05:12.629 INFO:tasks.workunit.client.1.smithi070.stdout:8/328: dwrite d4/d16/d17/d1a/f2e [0,4194304] 0 2024-09-10T04:05:12.631 INFO:tasks.workunit.client.1.smithi070.stdout:8/329: dread - d4/d16/d17/d1a/d23/d44/f59 zero size 2024-09-10T04:05:12.636 INFO:tasks.workunit.client.1.smithi070.stdout:6/360: creat d0/d3/d18/d1b/f72 x:0 0 0 2024-09-10T04:05:12.639 INFO:tasks.workunit.client.1.smithi070.stdout:3/322: unlink db/f27 0 2024-09-10T04:05:12.641 INFO:tasks.workunit.client.1.smithi070.stdout:1/260: write f0 [3202789,51541] 0 2024-09-10T04:05:12.642 INFO:tasks.workunit.client.1.smithi070.stdout:5/290: mknod d17/d24/c66 0 2024-09-10T04:05:12.648 INFO:tasks.workunit.client.1.smithi070.stdout:5/291: dwrite d17/d24/d4e/f55 [0,4194304] 0 2024-09-10T04:05:12.649 INFO:tasks.workunit.client.1.smithi070.stdout:9/315: link d9/l16 d9/d59/d10/d3c/l5a 0 2024-09-10T04:05:12.649 INFO:tasks.workunit.client.0.smithi046.stdout:6/264: mkdir da/d59 0 2024-09-10T04:05:12.650 INFO:tasks.workunit.client.0.smithi046.stdout:0/240: rmdir d1/dc/d25/d33/d4 39 2024-09-10T04:05:12.650 INFO:tasks.workunit.client.0.smithi046.stdout:7/309: creat d5/f6b x:0 0 0 2024-09-10T04:05:12.651 INFO:tasks.workunit.client.1.smithi070.stdout:7/254: symlink d2/d40/l59 0 2024-09-10T04:05:12.651 INFO:tasks.workunit.client.0.smithi046.stdout:6/265: chown da/f40 1479424 1 2024-09-10T04:05:12.651 INFO:tasks.workunit.client.0.smithi046.stdout:8/287: creat d1/d29/d17/d1f/d46/d52/f65 x:0 0 0 2024-09-10T04:05:12.651 INFO:tasks.workunit.client.0.smithi046.stdout:9/293: dread d4/d7/d1f/f15 [0,4194304] 0 2024-09-10T04:05:12.653 INFO:tasks.workunit.client.0.smithi046.stdout:2/332: chown d2/d6/d23/d5d/d77/l60 3 1 2024-09-10T04:05:12.653 INFO:tasks.workunit.client.1.smithi070.stdout:8/330: unlink d4/f13 0 2024-09-10T04:05:12.653 INFO:tasks.workunit.client.0.smithi046.stdout:4/289: symlink d3/d12/d17/d22/d51/d5c/l63 0 2024-09-10T04:05:12.654 INFO:tasks.workunit.client.0.smithi046.stdout:8/288: truncate d1/d29/d19/f56 667421 0 2024-09-10T04:05:12.655 INFO:tasks.workunit.client.1.smithi070.stdout:0/330: creat d8/d24/f6d x:0 0 0 2024-09-10T04:05:12.658 INFO:tasks.workunit.client.1.smithi070.stdout:6/361: unlink d0/d3/c7 0 2024-09-10T04:05:12.659 INFO:tasks.workunit.client.1.smithi070.stdout:6/362: write d0/d3/f51 [4009730,116529] 0 2024-09-10T04:05:12.659 INFO:tasks.workunit.client.0.smithi046.stdout:3/261: creat d12/d1b/f4c x:0 0 0 2024-09-10T04:05:12.660 INFO:tasks.workunit.client.0.smithi046.stdout:2/333: dwrite d2/d6/d23/f4b [0,4194304] 0 2024-09-10T04:05:12.662 INFO:tasks.workunit.client.0.smithi046.stdout:5/381: link d7/d11/d14/f71 d7/d11/d14/f89 0 2024-09-10T04:05:12.662 INFO:tasks.workunit.client.0.smithi046.stdout:5/382: readlink d7/l5d 0 2024-09-10T04:05:12.663 INFO:tasks.workunit.client.1.smithi070.stdout:3/323: symlink db/l64 0 2024-09-10T04:05:12.665 INFO:tasks.workunit.client.1.smithi070.stdout:1/261: unlink d3/d20/f44 0 2024-09-10T04:05:12.668 INFO:tasks.workunit.client.0.smithi046.stdout:7/310: mkdir d5/d6/d39/d6c 0 2024-09-10T04:05:12.669 INFO:tasks.workunit.client.1.smithi070.stdout:3/324: dwrite db/f4e [0,4194304] 0 2024-09-10T04:05:12.671 INFO:tasks.workunit.client.0.smithi046.stdout:7/311: read d5/d6/d32/f5e [2510798,126141] 0 2024-09-10T04:05:12.673 INFO:tasks.workunit.client.0.smithi046.stdout:9/294: creat d4/d7/f5e x:0 0 0 2024-09-10T04:05:12.673 INFO:tasks.workunit.client.1.smithi070.stdout:2/271: mknod d15/d18/d42/c60 0 2024-09-10T04:05:12.674 INFO:tasks.workunit.client.0.smithi046.stdout:7/312: write d5/d9/f5b [4228841,27963] 0 2024-09-10T04:05:12.676 INFO:tasks.workunit.client.0.smithi046.stdout:0/241: dwrite d1/dc/d25/d33/d4/f36 [4194304,4194304] 0 2024-09-10T04:05:12.680 INFO:tasks.workunit.client.0.smithi046.stdout:0/242: truncate d1/dc/f17 1219781 0 2024-09-10T04:05:12.684 INFO:tasks.workunit.client.0.smithi046.stdout:2/334: dread d2/d8/d37/f41 [0,4194304] 0 2024-09-10T04:05:12.685 INFO:tasks.workunit.client.0.smithi046.stdout:9/295: dwrite d4/d29/d32/f43 [0,4194304] 0 2024-09-10T04:05:12.686 INFO:tasks.workunit.client.1.smithi070.stdout:7/255: rename d2/d3/dc/d13/f1a to d2/d7/d9/d18/d23/f5a 0 2024-09-10T04:05:12.689 INFO:tasks.workunit.client.1.smithi070.stdout:8/331: creat d4/d16/d17/d19/f72 x:0 0 0 2024-09-10T04:05:12.690 INFO:tasks.workunit.client.1.smithi070.stdout:8/332: write d4/d16/d17/d5c/f6d [748344,25901] 0 2024-09-10T04:05:12.701 INFO:tasks.workunit.client.1.smithi070.stdout:6/363: dwrite d0/d3/d18/d2a/f59 [0,4194304] 0 2024-09-10T04:05:12.701 INFO:tasks.workunit.client.0.smithi046.stdout:6/266: write da/db/d11/d13/f57 [541226,30664] 0 2024-09-10T04:05:12.702 INFO:tasks.workunit.client.1.smithi070.stdout:1/262: rmdir d3/d8/d12 39 2024-09-10T04:05:12.705 INFO:tasks.workunit.client.0.smithi046.stdout:4/290: write d3/f9 [1423986,65968] 0 2024-09-10T04:05:12.705 INFO:tasks.workunit.client.1.smithi070.stdout:4/228: getdents de/d13/d18/d26 0 2024-09-10T04:05:12.707 INFO:tasks.workunit.client.0.smithi046.stdout:4/291: write d3/d12/d17/d22/f4c [962772,95355] 0 2024-09-10T04:05:12.707 INFO:tasks.workunit.client.0.smithi046.stdout:7/313: unlink f2 0 2024-09-10T04:05:12.709 INFO:tasks.workunit.client.0.smithi046.stdout:7/314: dread - d5/d6/db/f29 zero size 2024-09-10T04:05:12.711 INFO:tasks.workunit.client.0.smithi046.stdout:3/262: write d12/d13/d14/f25 [852647,13937] 0 2024-09-10T04:05:12.711 INFO:tasks.workunit.client.0.smithi046.stdout:3/263: readlink lc 0 2024-09-10T04:05:12.714 INFO:tasks.workunit.client.0.smithi046.stdout:0/243: rename d1/d1e to d1/d13/d37/d3e/d4d 0 2024-09-10T04:05:12.716 INFO:tasks.workunit.client.0.smithi046.stdout:0/244: write d1/dc/d25/d33/f11 [7183273,52719] 0 2024-09-10T04:05:12.718 INFO:tasks.workunit.client.1.smithi070.stdout:7/256: unlink d2/d7/d9/d18/d23/f38 0 2024-09-10T04:05:12.719 INFO:tasks.workunit.client.1.smithi070.stdout:0/331: creat d8/d13/d33/d4f/f6e x:0 0 0 2024-09-10T04:05:12.721 INFO:tasks.workunit.client.0.smithi046.stdout:5/383: symlink d7/l8a 0 2024-09-10T04:05:12.724 INFO:tasks.workunit.client.0.smithi046.stdout:6/267: mkdir da/db/d11/d2b/d5a 0 2024-09-10T04:05:12.724 INFO:tasks.workunit.client.1.smithi070.stdout:6/364: symlink d0/d3/d18/d1b/d17/l73 0 2024-09-10T04:05:12.725 INFO:tasks.workunit.client.0.smithi046.stdout:7/315: mkdir d5/d6d 0 2024-09-10T04:05:12.727 INFO:tasks.workunit.client.1.smithi070.stdout:1/263: symlink d3/d8/d13/l50 0 2024-09-10T04:05:12.729 INFO:tasks.workunit.client.1.smithi070.stdout:4/229: symlink de/d13/d18/l49 0 2024-09-10T04:05:12.730 INFO:tasks.workunit.client.0.smithi046.stdout:0/245: dread d1/dc/d25/d33/d4/f36 [0,4194304] 0 2024-09-10T04:05:12.730 INFO:tasks.workunit.client.0.smithi046.stdout:8/289: link d1/d29/d17/l47 d1/d29/d19/l66 0 2024-09-10T04:05:12.731 INFO:tasks.workunit.client.1.smithi070.stdout:3/325: mknod db/d1c/d25/d28/d43/c65 0 2024-09-10T04:05:12.733 INFO:tasks.workunit.client.0.smithi046.stdout:4/292: rename d3/d12/d14/f41 to d3/d12/d17/d22/d51/f64 0 2024-09-10T04:05:12.734 INFO:tasks.workunit.client.1.smithi070.stdout:5/292: rmdir d17/d24/d5b 0 2024-09-10T04:05:12.737 INFO:tasks.workunit.client.0.smithi046.stdout:9/296: mknod d4/c5f 0 2024-09-10T04:05:12.738 INFO:tasks.workunit.client.0.smithi046.stdout:2/335: truncate d2/d49/f2e 974914 0 2024-09-10T04:05:12.740 INFO:tasks.workunit.client.0.smithi046.stdout:4/293: dwrite d3/d12/d17/d22/d30/f3a [0,4194304] 0 2024-09-10T04:05:12.742 INFO:tasks.workunit.client.1.smithi070.stdout:0/332: mkdir d8/d13/d33/d6f 0 2024-09-10T04:05:12.742 INFO:tasks.workunit.client.1.smithi070.stdout:0/333: fdatasync f5 0 2024-09-10T04:05:12.742 INFO:tasks.workunit.client.0.smithi046.stdout:2/336: dread - d2/d6/d23/d55/f7e zero size 2024-09-10T04:05:12.744 INFO:tasks.workunit.client.1.smithi070.stdout:8/333: mkdir d4/d16/d73 0 2024-09-10T04:05:12.744 INFO:tasks.workunit.client.1.smithi070.stdout:8/334: chown d4/cc 3065 1 2024-09-10T04:05:12.747 INFO:tasks.workunit.client.1.smithi070.stdout:6/365: fdatasync d0/d3/d18/f4c 0 2024-09-10T04:05:12.753 INFO:tasks.workunit.client.1.smithi070.stdout:3/326: chown db/d1c/d25/d28/c40 218 1 2024-09-10T04:05:12.754 INFO:tasks.workunit.client.0.smithi046.stdout:9/297: symlink d4/d7/l60 0 2024-09-10T04:05:12.758 INFO:tasks.workunit.client.1.smithi070.stdout:9/316: getdents d9/d1c 0 2024-09-10T04:05:12.759 INFO:tasks.workunit.client.0.smithi046.stdout:3/264: write f7 [970545,22476] 0 2024-09-10T04:05:12.761 INFO:tasks.workunit.client.1.smithi070.stdout:7/257: dwrite d2/d7/db/f37 [0,4194304] 0 2024-09-10T04:05:12.761 INFO:tasks.workunit.client.0.smithi046.stdout:7/316: fdatasync f4 0 2024-09-10T04:05:12.765 INFO:tasks.workunit.client.0.smithi046.stdout:5/384: rename d7/d11/d3f/d53/d75/l7e to d7/d11/d14/d2f/d2b/l8b 0 2024-09-10T04:05:12.767 INFO:tasks.workunit.client.0.smithi046.stdout:3/265: dwrite d12/d13/d14/f24 [0,4194304] 0 2024-09-10T04:05:12.773 INFO:tasks.workunit.client.0.smithi046.stdout:6/268: link da/db/d18/l42 da/d59/l5b 0 2024-09-10T04:05:12.775 INFO:tasks.workunit.client.1.smithi070.stdout:4/230: mknod de/d13/c4a 0 2024-09-10T04:05:12.776 INFO:tasks.workunit.client.0.smithi046.stdout:9/298: write d4/f28 [517751,107558] 0 2024-09-10T04:05:12.777 INFO:tasks.workunit.client.0.smithi046.stdout:4/294: getdents d3/d43/d5f 0 2024-09-10T04:05:12.778 INFO:tasks.workunit.client.1.smithi070.stdout:5/293: mkdir d17/d19/d67 0 2024-09-10T04:05:12.779 INFO:tasks.workunit.client.1.smithi070.stdout:2/272: getdents d15/d18/d29/d2a/d2e/d32 0 2024-09-10T04:05:12.780 INFO:tasks.workunit.client.1.smithi070.stdout:9/317: chown d9/d59/d10/f3d 1 1 2024-09-10T04:05:12.781 INFO:tasks.workunit.client.1.smithi070.stdout:7/258: rmdir d2 39 2024-09-10T04:05:12.782 INFO:tasks.workunit.client.1.smithi070.stdout:0/334: mknod d8/d13/c70 0 2024-09-10T04:05:12.783 INFO:tasks.workunit.client.0.smithi046.stdout:6/269: write da/f1f [1914162,4663] 0 2024-09-10T04:05:12.784 INFO:tasks.workunit.client.1.smithi070.stdout:6/366: symlink d0/d3/d18/d62/l74 0 2024-09-10T04:05:12.785 INFO:tasks.workunit.client.0.smithi046.stdout:2/337: creat d2/d6/f80 x:0 0 0 2024-09-10T04:05:12.786 INFO:tasks.workunit.client.0.smithi046.stdout:2/338: stat d2/d8/d65 0 2024-09-10T04:05:12.786 INFO:tasks.workunit.client.1.smithi070.stdout:1/264: rename d3/l11 to d3/d8/l51 0 2024-09-10T04:05:12.787 INFO:tasks.workunit.client.0.smithi046.stdout:9/299: mkdir d4/d7/d61 0 2024-09-10T04:05:12.788 INFO:tasks.workunit.client.0.smithi046.stdout:4/295: mknod d3/d12/d17/d22/d51/c65 0 2024-09-10T04:05:12.790 INFO:tasks.workunit.client.0.smithi046.stdout:6/270: dwrite da/f40 [0,4194304] 0 2024-09-10T04:05:12.791 INFO:tasks.workunit.client.1.smithi070.stdout:9/318: mknod d9/d59/d10/c5b 0 2024-09-10T04:05:12.792 INFO:tasks.workunit.client.0.smithi046.stdout:7/317: truncate d5/d6/db/d11/f14 1105626 0 2024-09-10T04:05:12.793 INFO:tasks.workunit.client.1.smithi070.stdout:7/259: dread - d2/f55 zero size 2024-09-10T04:05:12.795 INFO:tasks.workunit.client.0.smithi046.stdout:6/271: dread da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:12.797 INFO:tasks.workunit.client.1.smithi070.stdout:0/335: mkdir d8/d13/d33/d71 0 2024-09-10T04:05:12.797 INFO:tasks.workunit.client.0.smithi046.stdout:6/272: write da/db/d11/d13/f57 [272990,28590] 0 2024-09-10T04:05:12.800 INFO:tasks.workunit.client.1.smithi070.stdout:8/335: mkdir d4/d16/d17/d5c/d74 0 2024-09-10T04:05:12.801 INFO:tasks.workunit.client.1.smithi070.stdout:8/336: write d4/d16/d17/d19/f25 [346316,124176] 0 2024-09-10T04:05:12.801 INFO:tasks.workunit.client.0.smithi046.stdout:2/339: creat d2/d6/d23/d55/f81 x:0 0 0 2024-09-10T04:05:12.802 INFO:tasks.workunit.client.1.smithi070.stdout:8/337: fsync d4/d16/d17/d5c/f6d 0 2024-09-10T04:05:12.805 INFO:tasks.workunit.client.1.smithi070.stdout:4/231: rename de/d13/c2e to de/d13/d18/d23/c4b 0 2024-09-10T04:05:12.806 INFO:tasks.workunit.client.0.smithi046.stdout:9/300: mknod d4/d44/c62 0 2024-09-10T04:05:12.808 INFO:tasks.workunit.client.0.smithi046.stdout:4/296: symlink d3/d12/d17/d22/d51/l66 0 2024-09-10T04:05:12.814 INFO:tasks.workunit.client.0.smithi046.stdout:3/266: dwrite d12/d13/d14/d1e/d2d/f2e [0,4194304] 0 2024-09-10T04:05:12.815 INFO:tasks.workunit.client.0.smithi046.stdout:4/297: dwrite d3/d12/d17/d22/f4e [0,4194304] 0 2024-09-10T04:05:12.846 INFO:tasks.workunit.client.1.smithi070.stdout:2/273: symlink d15/d18/d42/d43/d55/l61 0 2024-09-10T04:05:12.847 INFO:tasks.workunit.client.1.smithi070.stdout:2/274: read d15/d18/d29/d2a/d2e/d32/d36/f5b [2488864,37525] 0 2024-09-10T04:05:12.847 INFO:tasks.workunit.client.1.smithi070.stdout:2/275: fdatasync d15/d18/f21 0 2024-09-10T04:05:12.848 INFO:tasks.workunit.client.1.smithi070.stdout:2/276: readlink d15/d18/d29/d2a/l3c 0 2024-09-10T04:05:12.848 INFO:tasks.workunit.client.1.smithi070.stdout:9/319: unlink d9/d1c/c41 0 2024-09-10T04:05:12.850 INFO:tasks.workunit.client.0.smithi046.stdout:5/385: link d7/d11/d14/d2f/f74 d7/d11/d14/d2f/f8c 0 2024-09-10T04:05:12.851 INFO:tasks.workunit.client.1.smithi070.stdout:6/367: fdatasync d0/d3/d18/f42 0 2024-09-10T04:05:12.853 INFO:tasks.workunit.client.1.smithi070.stdout:6/368: write d0/d3/d18/d1b/d17/d28/f6b [862439,25083] 0 2024-09-10T04:05:12.854 INFO:tasks.workunit.client.1.smithi070.stdout:7/260: rename d2/d3/dc/l48 to d2/d7/d9/d18/d49/l5b 0 2024-09-10T04:05:12.857 INFO:tasks.workunit.client.0.smithi046.stdout:5/386: dwrite d7/d11/d3f/d5a/d63/f88 [0,4194304] 0 2024-09-10T04:05:12.859 INFO:tasks.workunit.client.0.smithi046.stdout:5/387: write d7/f1e [7762830,76570] 0 2024-09-10T04:05:12.859 INFO:tasks.workunit.client.1.smithi070.stdout:3/327: getdents db/d1c 0 2024-09-10T04:05:12.860 INFO:tasks.workunit.client.1.smithi070.stdout:8/338: dwrite d4/d16/d17/f40 [4194304,4194304] 0 2024-09-10T04:05:12.861 INFO:tasks.workunit.client.0.smithi046.stdout:4/298: symlink d3/d12/d17/d22/d51/l67 0 2024-09-10T04:05:12.862 INFO:tasks.workunit.client.1.smithi070.stdout:2/277: dread d15/d18/d29/d2a/f58 [4194304,4194304] 0 2024-09-10T04:05:12.863 INFO:tasks.workunit.client.1.smithi070.stdout:5/294: link d17/f20 d17/d24/d2b/d31/d32/f68 0 2024-09-10T04:05:12.865 INFO:tasks.workunit.client.1.smithi070.stdout:5/295: fsync d17/d24/d47/d54/f62 0 2024-09-10T04:05:12.867 INFO:tasks.workunit.client.1.smithi070.stdout:7/261: dread d2/d7/d9/d18/d23/f47 [0,4194304] 0 2024-09-10T04:05:12.869 INFO:tasks.workunit.client.0.smithi046.stdout:7/318: rename d5/d6/db/d11/d1f/c68 to d5/d6/d39/c6e 0 2024-09-10T04:05:12.869 INFO:tasks.workunit.client.0.smithi046.stdout:6/273: creat da/db/d11/d2b/d5a/f5c x:0 0 0 2024-09-10T04:05:12.869 INFO:tasks.workunit.client.1.smithi070.stdout:1/265: dwrite d3/d8/d12/f19 [4194304,4194304] 0 2024-09-10T04:05:12.871 INFO:tasks.workunit.client.0.smithi046.stdout:7/319: fsync d5/d9/d43/f5c 0 2024-09-10T04:05:12.872 INFO:tasks.workunit.client.0.smithi046.stdout:7/320: write d5/f60 [982825,83459] 0 2024-09-10T04:05:12.874 INFO:tasks.workunit.client.1.smithi070.stdout:7/262: dread d2/d7/d9/d18/f3f [0,4194304] 0 2024-09-10T04:05:12.876 INFO:tasks.workunit.client.1.smithi070.stdout:9/320: creat d9/d59/d10/d1b/f5c x:0 0 0 2024-09-10T04:05:12.882 INFO:tasks.workunit.client.1.smithi070.stdout:1/266: dread d3/d10/f35 [0,4194304] 0 2024-09-10T04:05:12.883 INFO:tasks.workunit.client.0.smithi046.stdout:9/301: truncate d4/d18/f33 305681 0 2024-09-10T04:05:12.886 INFO:tasks.workunit.client.0.smithi046.stdout:5/388: unlink d7/d11/d14/d2f/f74 0 2024-09-10T04:05:12.886 INFO:tasks.workunit.client.0.smithi046.stdout:4/299: rmdir d3/d12/d17/d22/d30 39 2024-09-10T04:05:12.888 INFO:tasks.workunit.client.1.smithi070.stdout:3/328: unlink db/d1c/d53/c63 0 2024-09-10T04:05:12.890 INFO:tasks.workunit.client.1.smithi070.stdout:8/339: symlink d4/d16/d17/d1a/d23/l75 0 2024-09-10T04:05:12.892 INFO:tasks.workunit.client.1.smithi070.stdout:5/296: write d17/d24/f2a [4439549,43368] 0 2024-09-10T04:05:12.897 INFO:tasks.workunit.client.0.smithi046.stdout:2/340: rename d2/d6/d23/d55/f81 to d2/d8/d37/d63/f82 0 2024-09-10T04:05:12.899 INFO:tasks.workunit.client.0.smithi046.stdout:6/274: creat da/db/d10/f5d x:0 0 0 2024-09-10T04:05:12.900 INFO:tasks.workunit.client.0.smithi046.stdout:7/321: creat d5/d9/d12/f6f x:0 0 0 2024-09-10T04:05:12.901 INFO:tasks.workunit.client.1.smithi070.stdout:7/263: creat d2/d7/db/f5c x:0 0 0 2024-09-10T04:05:12.903 INFO:tasks.workunit.client.0.smithi046.stdout:5/389: mknod d7/d11/d3f/d5a/c8d 0 2024-09-10T04:05:12.906 INFO:tasks.workunit.client.1.smithi070.stdout:3/329: mknod db/d1c/c66 0 2024-09-10T04:05:12.907 INFO:tasks.workunit.client.1.smithi070.stdout:8/340: dread - d4/d16/f34 zero size 2024-09-10T04:05:12.907 INFO:tasks.workunit.client.1.smithi070.stdout:8/341: chown d4/f11 501 1 2024-09-10T04:05:12.908 INFO:tasks.workunit.client.0.smithi046.stdout:6/275: read f9 [6257382,107016] 0 2024-09-10T04:05:12.909 INFO:tasks.workunit.client.1.smithi070.stdout:5/297: mknod d17/d19/d35/c69 0 2024-09-10T04:05:12.909 INFO:tasks.workunit.client.1.smithi070.stdout:5/298: dread - d17/d24/f2c zero size 2024-09-10T04:05:12.910 INFO:tasks.workunit.client.1.smithi070.stdout:5/299: dread - d17/d19/f50 zero size 2024-09-10T04:05:12.915 INFO:tasks.workunit.client.1.smithi070.stdout:5/300: dwrite d17/d24/f25 [0,4194304] 0 2024-09-10T04:05:12.915 INFO:tasks.workunit.client.0.smithi046.stdout:3/267: link f4 d12/f4d 0 2024-09-10T04:05:12.916 INFO:tasks.workunit.client.1.smithi070.stdout:0/336: getdents d8/d13/d33 0 2024-09-10T04:05:12.917 INFO:tasks.workunit.client.0.smithi046.stdout:4/300: dread d3/d12/d17/f20 [0,4194304] 0 2024-09-10T04:05:12.919 INFO:tasks.workunit.client.1.smithi070.stdout:7/264: read d2/d7/d9/d18/f3f [376447,67251] 0 2024-09-10T04:05:12.920 INFO:tasks.workunit.client.0.smithi046.stdout:5/390: rmdir d7/d11/d3f/d5a 39 2024-09-10T04:05:12.920 INFO:tasks.workunit.client.1.smithi070.stdout:9/321: mkdir d9/d59/d5d 0 2024-09-10T04:05:12.920 INFO:tasks.workunit.client.0.smithi046.stdout:5/391: stat d7/c52 0 2024-09-10T04:05:12.921 INFO:tasks.workunit.client.0.smithi046.stdout:2/341: symlink d2/l83 0 2024-09-10T04:05:12.921 INFO:tasks.workunit.client.0.smithi046.stdout:2/342: dread - d2/d49/f74 zero size 2024-09-10T04:05:12.924 INFO:tasks.workunit.client.1.smithi070.stdout:6/369: dwrite d0/d3/f16 [0,4194304] 0 2024-09-10T04:05:12.924 INFO:tasks.workunit.client.0.smithi046.stdout:6/276: rename da/dc/f4e to da/db/d11/d2b/d2f/d3b/f5e 0 2024-09-10T04:05:12.928 INFO:tasks.workunit.client.0.smithi046.stdout:9/302: truncate d4/d29/d32/f55 184742 0 2024-09-10T04:05:12.932 INFO:tasks.workunit.client.0.smithi046.stdout:7/322: write d5/d6/db/d11/f14 [297906,40564] 0 2024-09-10T04:05:12.935 INFO:tasks.workunit.client.0.smithi046.stdout:2/343: creat d2/d6/d23/d5d/d5e/f84 x:0 0 0 2024-09-10T04:05:12.936 INFO:tasks.workunit.client.1.smithi070.stdout:9/322: symlink d9/d3e/l5e 0 2024-09-10T04:05:12.937 INFO:tasks.workunit.client.1.smithi070.stdout:0/337: dwrite d8/d13/d1a/f23 [0,4194304] 0 2024-09-10T04:05:12.937 INFO:tasks.workunit.client.0.smithi046.stdout:9/303: dwrite d4/d18/f4f [0,4194304] 0 2024-09-10T04:05:12.939 INFO:tasks.workunit.client.0.smithi046.stdout:9/304: chown d4/d7/d1f/ld 1 1 2024-09-10T04:05:12.940 INFO:tasks.workunit.client.0.smithi046.stdout:3/268: creat d12/d13/d42/d49/f4e x:0 0 0 2024-09-10T04:05:12.941 INFO:tasks.workunit.client.1.smithi070.stdout:8/342: symlink d4/d16/d17/d19/d28/d43/d55/l76 0 2024-09-10T04:05:12.941 INFO:tasks.workunit.client.1.smithi070.stdout:6/370: mknod d0/d3/d18/d2a/c75 0 2024-09-10T04:05:12.945 INFO:tasks.workunit.client.1.smithi070.stdout:3/330: dwrite db/d1c/f4b [0,4194304] 0 2024-09-10T04:05:12.950 INFO:tasks.workunit.client.1.smithi070.stdout:0/338: dread d8/d13/f1f [0,4194304] 0 2024-09-10T04:05:12.950 INFO:tasks.workunit.client.0.smithi046.stdout:7/323: rename d5/c17 to d5/d6/db/d11/d1e/d20/c70 0 2024-09-10T04:05:12.955 INFO:tasks.workunit.client.0.smithi046.stdout:3/269: dwrite d12/d13/d42/d49/f4e [0,4194304] 0 2024-09-10T04:05:12.955 INFO:tasks.workunit.client.1.smithi070.stdout:5/301: rename d17/d24/d47 to d17/d19/d6a 0 2024-09-10T04:05:12.957 INFO:tasks.workunit.client.0.smithi046.stdout:4/301: dwrite d3/d12/d14/f21 [0,4194304] 0 2024-09-10T04:05:12.959 INFO:tasks.workunit.client.1.smithi070.stdout:0/339: dwrite d8/fa [0,4194304] 0 2024-09-10T04:05:12.961 INFO:tasks.workunit.client.1.smithi070.stdout:6/371: creat d0/d3/d18/d1b/d17/d28/f76 x:0 0 0 2024-09-10T04:05:12.963 INFO:tasks.workunit.client.0.smithi046.stdout:2/344: rename d2/d6/d23/d3b to d2/d6/d23/d5d/d77/d7b/d85 0 2024-09-10T04:05:12.965 INFO:tasks.workunit.client.0.smithi046.stdout:6/277: link da/db/d11/d13/d1e/c4c da/d45/c5f 0 2024-09-10T04:05:12.965 INFO:tasks.workunit.client.1.smithi070.stdout:3/331: chown db/d1c/l45 16 1 2024-09-10T04:05:12.966 INFO:tasks.workunit.client.0.smithi046.stdout:6/278: chown da/db/d11/d2b/d2f/c44 78131 1 2024-09-10T04:05:12.967 INFO:tasks.workunit.client.0.smithi046.stdout:6/279: chown da/db/d18 1341 1 2024-09-10T04:05:12.973 INFO:tasks.workunit.client.1.smithi070.stdout:9/323: rename d9/d59/d10/f1f to d9/d3e/f5f 0 2024-09-10T04:05:12.973 INFO:tasks.workunit.client.0.smithi046.stdout:3/270: dread d12/d13/d14/f15 [4194304,4194304] 0 2024-09-10T04:05:12.975 INFO:tasks.workunit.client.0.smithi046.stdout:4/302: dread d3/d12/d14/f36 [0,4194304] 0 2024-09-10T04:05:12.976 INFO:tasks.workunit.client.1.smithi070.stdout:5/302: stat d17/d24/d2b/d31/c49 0 2024-09-10T04:05:12.977 INFO:tasks.workunit.client.1.smithi070.stdout:0/340: rmdir d8/d24/d2d/d34 39 2024-09-10T04:05:12.979 INFO:tasks.workunit.client.0.smithi046.stdout:2/345: mkdir d2/d86 0 2024-09-10T04:05:12.980 INFO:tasks.workunit.client.1.smithi070.stdout:8/343: creat d4/f77 x:0 0 0 2024-09-10T04:05:12.980 INFO:tasks.workunit.client.0.smithi046.stdout:2/346: write d2/d6/d23/d5d/d77/f54 [910807,6044] 0 2024-09-10T04:05:12.982 INFO:tasks.workunit.client.0.smithi046.stdout:3/271: fsync d12/d13/d42/d49/f4e 0 2024-09-10T04:05:12.983 INFO:tasks.workunit.client.0.smithi046.stdout:6/280: creat da/f60 x:0 0 0 2024-09-10T04:05:12.984 INFO:tasks.workunit.client.0.smithi046.stdout:6/281: write da/f60 [667258,18882] 0 2024-09-10T04:05:12.986 INFO:tasks.workunit.client.0.smithi046.stdout:6/282: truncate da/db/d11/f32 104129 0 2024-09-10T04:05:12.986 INFO:tasks.workunit.client.0.smithi046.stdout:6/283: dread - da/db/d10/f5d zero size 2024-09-10T04:05:12.988 INFO:tasks.workunit.client.0.smithi046.stdout:6/284: read da/db/d10/f17 [647837,43493] 0 2024-09-10T04:05:12.989 INFO:tasks.workunit.client.0.smithi046.stdout:7/324: truncate d5/d6/db/dd/f2f 4079861 0 2024-09-10T04:05:12.990 INFO:tasks.workunit.client.1.smithi070.stdout:8/344: dread d4/d16/d17/d19/d28/f30 [8388608,4194304] 0 2024-09-10T04:05:12.993 INFO:tasks.workunit.client.1.smithi070.stdout:3/332: symlink db/d1c/d3e/l67 0 2024-09-10T04:05:12.993 INFO:tasks.workunit.client.0.smithi046.stdout:6/285: dread da/db/d11/d13/d1e/f4d [0,4194304] 0 2024-09-10T04:05:12.995 INFO:tasks.workunit.client.0.smithi046.stdout:4/303: truncate d3/d12/f19 8111604 0 2024-09-10T04:05:12.995 INFO:tasks.workunit.client.0.smithi046.stdout:6/286: fsync da/db/d18/f35 0 2024-09-10T04:05:12.997 INFO:tasks.workunit.client.0.smithi046.stdout:4/304: read d3/f3c [4556456,106047] 0 2024-09-10T04:05:12.997 INFO:tasks.workunit.client.0.smithi046.stdout:3/272: creat d12/d13/d14/d1e/d2d/f4f x:0 0 0 2024-09-10T04:05:12.998 INFO:tasks.workunit.client.0.smithi046.stdout:6/287: readlink da/db/d11/d2b/d2f/d3b/l3d 0 2024-09-10T04:05:12.998 INFO:tasks.workunit.client.0.smithi046.stdout:7/325: dwrite d5/d9/d43/f66 [0,4194304] 0 2024-09-10T04:05:12.999 INFO:tasks.workunit.client.1.smithi070.stdout:9/324: mkdir d9/d60 0 2024-09-10T04:05:12.999 INFO:tasks.workunit.client.1.smithi070.stdout:5/303: write d17/d24/d45/f4f [706123,29370] 0 2024-09-10T04:05:12.999 INFO:tasks.workunit.client.1.smithi070.stdout:5/304: fdatasync d17/d24/f2c 0 2024-09-10T04:05:12.999 INFO:tasks.workunit.client.0.smithi046.stdout:4/305: fdatasync d3/d12/f1f 0 2024-09-10T04:05:13.000 INFO:tasks.workunit.client.1.smithi070.stdout:5/305: truncate f12 4965989 0 2024-09-10T04:05:13.001 INFO:tasks.workunit.client.0.smithi046.stdout:6/288: readlink da/db/d11/d13/d1e/d23/l58 0 2024-09-10T04:05:13.004 INFO:tasks.workunit.client.1.smithi070.stdout:6/372: link d0/d3/d18/d1b/d17/d28/c5d d0/d3/d18/d1b/d17/d28/c77 0 2024-09-10T04:05:13.007 INFO:tasks.workunit.client.1.smithi070.stdout:3/333: chown db/d1c/d25/d28/d43/c65 6283 1 2024-09-10T04:05:13.007 INFO:tasks.workunit.client.1.smithi070.stdout:3/334: dread - db/d1c/d25/d28/d43/f5a zero size 2024-09-10T04:05:13.008 INFO:tasks.workunit.client.1.smithi070.stdout:3/335: stat db/d1c/d3e 0 2024-09-10T04:05:13.012 INFO:tasks.workunit.client.1.smithi070.stdout:0/341: mknod d8/d24/d2d/d34/d6b/c72 0 2024-09-10T04:05:13.013 INFO:tasks.workunit.client.1.smithi070.stdout:0/342: dread - d8/d13/d33/f65 zero size 2024-09-10T04:05:13.018 INFO:tasks.workunit.client.1.smithi070.stdout:3/336: rename db/d1c/d3e/l67 to db/d1c/d25/d28/l68 0 2024-09-10T04:05:13.021 INFO:tasks.workunit.client.0.smithi046.stdout:4/306: rename d3/d12/d17/c47 to d3/d12/d17/d22/d30/c68 0 2024-09-10T04:05:13.023 INFO:tasks.workunit.client.1.smithi070.stdout:0/343: mkdir d8/d13/d47/d73 0 2024-09-10T04:05:13.023 INFO:tasks.workunit.client.0.smithi046.stdout:3/273: write d12/d1b/f21 [1673390,73365] 0 2024-09-10T04:05:13.025 INFO:tasks.workunit.client.1.smithi070.stdout:6/373: symlink d0/l78 0 2024-09-10T04:05:13.026 INFO:tasks.workunit.client.0.smithi046.stdout:7/326: mknod d5/d6/d32/d59/c71 0 2024-09-10T04:05:13.027 INFO:tasks.workunit.client.1.smithi070.stdout:8/345: creat d4/d16/d17/d1a/f78 x:0 0 0 2024-09-10T04:05:13.027 INFO:tasks.workunit.client.1.smithi070.stdout:3/337: truncate db/d1c/d3e/f1b 1492843 0 2024-09-10T04:05:13.027 INFO:tasks.workunit.client.0.smithi046.stdout:6/289: dread f6 [0,4194304] 0 2024-09-10T04:05:13.028 INFO:tasks.workunit.client.1.smithi070.stdout:9/325: dwrite d9/d59/f1a [0,4194304] 0 2024-09-10T04:05:13.029 INFO:tasks.workunit.client.1.smithi070.stdout:5/306: link f13 d17/f6b 0 2024-09-10T04:05:13.029 INFO:tasks.workunit.client.1.smithi070.stdout:3/338: chown db/d1c/d25/d28/f41 887595390 1 2024-09-10T04:05:13.030 INFO:tasks.workunit.client.1.smithi070.stdout:0/344: mkdir d8/d13/d33/d74 0 2024-09-10T04:05:13.030 INFO:tasks.workunit.client.0.smithi046.stdout:4/307: dwrite d3/d12/d17/d22/f4b [0,4194304] 0 2024-09-10T04:05:13.031 INFO:tasks.workunit.client.1.smithi070.stdout:6/374: dread d0/d3/f51 [0,4194304] 0 2024-09-10T04:05:13.031 INFO:tasks.workunit.client.0.smithi046.stdout:3/274: mkdir d12/d1b/d50 0 2024-09-10T04:05:13.031 INFO:tasks.workunit.client.1.smithi070.stdout:5/307: dread d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:13.032 INFO:tasks.workunit.client.0.smithi046.stdout:2/347: getdents d2/d57 0 2024-09-10T04:05:13.032 INFO:tasks.workunit.client.0.smithi046.stdout:3/275: dread - d12/d13/d14/d1e/d2d/f4f zero size 2024-09-10T04:05:13.035 INFO:tasks.workunit.client.0.smithi046.stdout:7/327: mkdir d5/d9/d12/d72 0 2024-09-10T04:05:13.037 INFO:tasks.workunit.client.0.smithi046.stdout:7/328: write d5/d6/db/d11/d1e/f30 [166201,31513] 0 2024-09-10T04:05:13.039 INFO:tasks.workunit.client.1.smithi070.stdout:3/339: symlink db/d1c/d3e/l69 0 2024-09-10T04:05:13.039 INFO:tasks.workunit.client.0.smithi046.stdout:4/308: rmdir d3/d12/d14 39 2024-09-10T04:05:13.039 INFO:tasks.workunit.client.1.smithi070.stdout:3/340: readlink db/l64 0 2024-09-10T04:05:13.040 INFO:tasks.workunit.client.0.smithi046.stdout:4/309: write d3/d12/d17/f1b [3346941,114828] 0 2024-09-10T04:05:13.041 INFO:tasks.workunit.client.1.smithi070.stdout:0/345: mkdir d8/d13/d1a/d75 0 2024-09-10T04:05:13.041 INFO:tasks.workunit.client.0.smithi046.stdout:3/276: mkdir d12/d13/d37/d51 0 2024-09-10T04:05:13.042 INFO:tasks.workunit.client.1.smithi070.stdout:4/232: sync 2024-09-10T04:05:13.042 INFO:tasks.workunit.client.0.smithi046.stdout:4/310: read d3/d12/d17/d22/f27 [2079063,42610] 0 2024-09-10T04:05:13.043 INFO:tasks.workunit.client.1.smithi070.stdout:5/308: dread fd [0,4194304] 0 2024-09-10T04:05:13.044 INFO:tasks.workunit.client.1.smithi070.stdout:5/309: write d17/d19/d6a/f53 [282954,118609] 0 2024-09-10T04:05:13.046 INFO:tasks.workunit.client.1.smithi070.stdout:6/375: rename d0/f2e to d0/d3/d18/d62/f79 0 2024-09-10T04:05:13.046 INFO:tasks.workunit.client.0.smithi046.stdout:7/329: fsync d5/d6/d32/f5e 0 2024-09-10T04:05:13.047 INFO:tasks.workunit.client.1.smithi070.stdout:6/376: chown d0/d3/d18/d1b/d17/d54/f6c 2 1 2024-09-10T04:05:13.048 INFO:tasks.workunit.client.1.smithi070.stdout:6/377: dread - d0/d3/d18/d1b/d33/f65 zero size 2024-09-10T04:05:13.049 INFO:tasks.workunit.client.1.smithi070.stdout:6/378: chown d0/d3/d1f/f37 40 1 2024-09-10T04:05:13.050 INFO:tasks.workunit.client.1.smithi070.stdout:9/326: write d9/d1c/d39/f44 [7058390,60384] 0 2024-09-10T04:05:13.054 INFO:tasks.workunit.client.0.smithi046.stdout:7/330: dwrite d5/d6/db/f29 [0,4194304] 0 2024-09-10T04:05:13.055 INFO:tasks.workunit.client.1.smithi070.stdout:4/233: truncate f8 986781 0 2024-09-10T04:05:13.056 INFO:tasks.workunit.client.0.smithi046.stdout:7/331: write d5/d9/d12/f5d [823988,100879] 0 2024-09-10T04:05:13.057 INFO:tasks.workunit.client.0.smithi046.stdout:7/332: dread - d5/d9/d12/f6f zero size 2024-09-10T04:05:13.060 INFO:tasks.workunit.client.0.smithi046.stdout:6/290: creat da/f61 x:0 0 0 2024-09-10T04:05:13.061 INFO:tasks.workunit.client.0.smithi046.stdout:6/291: stat da/db/d18/f49 0 2024-09-10T04:05:13.062 INFO:tasks.workunit.client.0.smithi046.stdout:2/348: truncate d2/d6/d23/d5d/d77/f69 1281829 0 2024-09-10T04:05:13.064 INFO:tasks.workunit.client.1.smithi070.stdout:8/346: creat d4/d16/d17/f79 x:0 0 0 2024-09-10T04:05:13.064 INFO:tasks.workunit.client.0.smithi046.stdout:7/333: dwrite d5/d9/d43/f66 [0,4194304] 0 2024-09-10T04:05:13.067 INFO:tasks.workunit.client.0.smithi046.stdout:7/334: readlink d5/d6/db/d11/d1e/d20/l2b 0 2024-09-10T04:05:13.069 INFO:tasks.workunit.client.0.smithi046.stdout:4/311: mknod d3/d12/d14/c69 0 2024-09-10T04:05:13.070 INFO:tasks.workunit.client.1.smithi070.stdout:6/379: dwrite d0/d3/d18/d62/f68 [0,4194304] 0 2024-09-10T04:05:13.071 INFO:tasks.workunit.client.1.smithi070.stdout:9/327: symlink d9/d1c/d39/l61 0 2024-09-10T04:05:13.072 INFO:tasks.workunit.client.0.smithi046.stdout:3/277: mkdir d12/d1b/d50/d52 0 2024-09-10T04:05:13.073 INFO:tasks.workunit.client.1.smithi070.stdout:3/341: mknod db/d1c/d53/c6a 0 2024-09-10T04:05:13.077 INFO:tasks.workunit.client.0.smithi046.stdout:2/349: creat d2/d6/d23/f87 x:0 0 0 2024-09-10T04:05:13.082 INFO:tasks.workunit.client.0.smithi046.stdout:7/335: mknod d5/d6/db/dd/d24/c73 0 2024-09-10T04:05:13.087 INFO:tasks.workunit.client.1.smithi070.stdout:6/380: fdatasync d0/d3/d18/f42 0 2024-09-10T04:05:13.087 INFO:tasks.workunit.client.1.smithi070.stdout:8/347: dwrite d4/d16/f27 [0,4194304] 0 2024-09-10T04:05:13.089 INFO:tasks.workunit.client.0.smithi046.stdout:3/278: symlink d12/d13/d42/d49/l53 0 2024-09-10T04:05:13.090 INFO:tasks.workunit.client.1.smithi070.stdout:9/328: mkdir d9/d59/d10/d3c/d62 0 2024-09-10T04:05:13.091 INFO:tasks.workunit.client.0.smithi046.stdout:6/292: symlink da/db/d11/d13/d1e/d23/d54/l62 0 2024-09-10T04:05:13.092 INFO:tasks.workunit.client.1.smithi070.stdout:3/342: chown db/d1c/d25/d28/l68 5296 1 2024-09-10T04:05:13.093 INFO:tasks.workunit.client.1.smithi070.stdout:4/234: symlink de/d13/d18/d23/d39/d48/l4c 0 2024-09-10T04:05:13.093 INFO:tasks.workunit.client.1.smithi070.stdout:8/348: dwrite d4/d16/f33 [0,4194304] 0 2024-09-10T04:05:13.094 INFO:tasks.workunit.client.0.smithi046.stdout:7/336: creat d5/d6/d32/f74 x:0 0 0 2024-09-10T04:05:13.095 INFO:tasks.workunit.client.1.smithi070.stdout:8/349: write d4/d16/d17/d19/d28/f31 [965836,19475] 0 2024-09-10T04:05:13.097 INFO:tasks.workunit.client.1.smithi070.stdout:0/346: rename d8/l66 to d8/d24/l76 0 2024-09-10T04:05:13.099 INFO:tasks.workunit.client.1.smithi070.stdout:0/347: write d8/d13/f5a [7713753,98047] 0 2024-09-10T04:05:13.102 INFO:tasks.workunit.client.1.smithi070.stdout:0/348: write d8/d24/d2d/d34/f5b [252815,51054] 0 2024-09-10T04:05:13.103 INFO:tasks.workunit.client.1.smithi070.stdout:4/235: dwrite de/d13/f38 [0,4194304] 0 2024-09-10T04:05:13.103 INFO:tasks.workunit.client.0.smithi046.stdout:2/350: dread d2/d6/d15/f3e [4194304,4194304] 0 2024-09-10T04:05:13.105 INFO:tasks.workunit.client.0.smithi046.stdout:7/337: dwrite d5/d9/d12/f6f [0,4194304] 0 2024-09-10T04:05:13.105 INFO:tasks.workunit.client.0.smithi046.stdout:6/293: mkdir da/db/d63 0 2024-09-10T04:05:13.109 INFO:tasks.workunit.client.0.smithi046.stdout:6/294: write da/db/f50 [217728,59515] 0 2024-09-10T04:05:13.109 INFO:tasks.workunit.client.0.smithi046.stdout:5/392: sync 2024-09-10T04:05:13.109 INFO:tasks.workunit.client.0.smithi046.stdout:8/290: sync 2024-09-10T04:05:13.109 INFO:tasks.workunit.client.0.smithi046.stdout:1/217: sync 2024-09-10T04:05:13.109 INFO:tasks.workunit.client.0.smithi046.stdout:0/246: sync 2024-09-10T04:05:13.110 INFO:tasks.workunit.client.0.smithi046.stdout:6/295: truncate f1 4223576 0 2024-09-10T04:05:13.112 INFO:tasks.workunit.client.0.smithi046.stdout:7/338: read d5/d6/db/d11/d1e/f30 [68667,6655] 0 2024-09-10T04:05:13.116 INFO:tasks.workunit.client.0.smithi046.stdout:3/279: dread d12/d13/d14/f3a [0,4194304] 0 2024-09-10T04:05:13.118 INFO:tasks.workunit.client.0.smithi046.stdout:4/312: link d3/d12/d17/d29/c3b d3/d12/d17/d22/d51/c6a 0 2024-09-10T04:05:13.121 INFO:tasks.workunit.client.0.smithi046.stdout:4/313: chown d3/d12/d14 57131380 1 2024-09-10T04:05:13.123 INFO:tasks.workunit.client.0.smithi046.stdout:6/296: dwrite da/db/f50 [0,4194304] 0 2024-09-10T04:05:13.127 INFO:tasks.workunit.client.1.smithi070.stdout:4/236: creat de/d13/d18/f4d x:0 0 0 2024-09-10T04:05:13.129 INFO:tasks.workunit.client.1.smithi070.stdout:4/237: dread de/d13/d18/d26/f42 [0,4194304] 0 2024-09-10T04:05:13.129 INFO:tasks.workunit.client.0.smithi046.stdout:6/297: dread da/db/d10/f1a [0,4194304] 0 2024-09-10T04:05:13.131 INFO:tasks.workunit.client.1.smithi070.stdout:0/349: rename d8/d13/f54 to d8/d24/d2d/d34/f77 0 2024-09-10T04:05:13.132 INFO:tasks.workunit.client.1.smithi070.stdout:0/350: write d8/d24/f6d [6603,83622] 0 2024-09-10T04:05:13.137 INFO:tasks.workunit.client.1.smithi070.stdout:0/351: dwrite d8/d24/f2a [0,4194304] 0 2024-09-10T04:05:13.139 INFO:tasks.workunit.client.1.smithi070.stdout:9/329: mkdir d9/d1c/d34/d63 0 2024-09-10T04:05:13.142 INFO:tasks.workunit.client.1.smithi070.stdout:8/350: mkdir d4/d16/d17/d1a/d60/d7a 0 2024-09-10T04:05:13.144 INFO:tasks.workunit.client.0.smithi046.stdout:3/280: creat d12/d13/d37/f54 x:0 0 0 2024-09-10T04:05:13.145 INFO:tasks.workunit.client.0.smithi046.stdout:3/281: write d12/d13/d14/f25 [1380980,125214] 0 2024-09-10T04:05:13.146 INFO:tasks.workunit.client.1.smithi070.stdout:8/351: dread d4/d16/d17/f40 [4194304,4194304] 0 2024-09-10T04:05:13.148 INFO:tasks.workunit.client.0.smithi046.stdout:2/351: rename d2/d6/d15/l2d to d2/d8/d37/d63/l88 0 2024-09-10T04:05:13.152 INFO:tasks.workunit.client.0.smithi046.stdout:5/393: mkdir d7/d11/d14/d8e 0 2024-09-10T04:05:13.156 INFO:tasks.workunit.client.1.smithi070.stdout:9/330: creat d9/d1c/d39/f64 x:0 0 0 2024-09-10T04:05:13.157 INFO:tasks.workunit.client.1.smithi070.stdout:9/331: write d9/d1c/d39/f64 [985242,53537] 0 2024-09-10T04:05:13.158 INFO:tasks.workunit.client.1.smithi070.stdout:9/332: fsync d9/f35 0 2024-09-10T04:05:13.159 INFO:tasks.workunit.client.0.smithi046.stdout:7/339: getdents d5/d6/d55 0 2024-09-10T04:05:13.162 INFO:tasks.workunit.client.0.smithi046.stdout:0/247: link d1/dc/d25/d33/f3f d1/d13/d37/d3e/d4d/f4e 0 2024-09-10T04:05:13.166 INFO:tasks.workunit.client.0.smithi046.stdout:3/282: creat d12/d1b/d50/f55 x:0 0 0 2024-09-10T04:05:13.167 INFO:tasks.workunit.client.0.smithi046.stdout:3/283: dread - d12/d1b/d50/f55 zero size 2024-09-10T04:05:13.167 INFO:tasks.workunit.client.1.smithi070.stdout:6/381: getdents d0/d3 0 2024-09-10T04:05:13.168 INFO:tasks.workunit.client.1.smithi070.stdout:6/382: chown d0/d3/d18/d62/f68 3 1 2024-09-10T04:05:13.168 INFO:tasks.workunit.client.0.smithi046.stdout:4/314: rename d3/d12/d17/d22/d51/l56 to d3/d43/l6b 0 2024-09-10T04:05:13.169 INFO:tasks.workunit.client.0.smithi046.stdout:3/284: read d12/d13/d14/d1e/f3f [338960,127109] 0 2024-09-10T04:05:13.169 INFO:tasks.workunit.client.1.smithi070.stdout:0/352: mkdir d8/d13/d47/d73/d78 0 2024-09-10T04:05:13.170 INFO:tasks.workunit.client.0.smithi046.stdout:2/352: mknod d2/d6/d23/d42/c89 0 2024-09-10T04:05:13.172 INFO:tasks.workunit.client.0.smithi046.stdout:5/394: mknod d7/d11/d14/d45/d51/c8f 0 2024-09-10T04:05:13.176 INFO:tasks.workunit.client.0.smithi046.stdout:0/248: creat d1/dc/d25/d33/d3/d29/d2d/f4f x:0 0 0 2024-09-10T04:05:13.177 INFO:tasks.workunit.client.1.smithi070.stdout:8/352: getdents d4/d16/d17/d5c/d74 0 2024-09-10T04:05:13.177 INFO:tasks.workunit.client.1.smithi070.stdout:8/353: fdatasync d4/d16/f33 0 2024-09-10T04:05:13.178 INFO:tasks.workunit.client.0.smithi046.stdout:2/353: dwrite d2/d6/f80 [0,4194304] 0 2024-09-10T04:05:13.178 INFO:tasks.workunit.client.1.smithi070.stdout:2/278: sync 2024-09-10T04:05:13.178 INFO:tasks.workunit.client.0.smithi046.stdout:4/315: mkdir d3/d12/d17/d22/d51/d5c/d6c 0 2024-09-10T04:05:13.179 INFO:tasks.workunit.client.1.smithi070.stdout:2/279: chown d15/d18/d29/d2a/d2e/d32/f33 13660 1 2024-09-10T04:05:13.179 INFO:tasks.workunit.client.1.smithi070.stdout:2/280: dread f11 [0,4194304] 0 2024-09-10T04:05:13.180 INFO:tasks.workunit.client.0.smithi046.stdout:4/316: read d3/d12/d17/d22/d30/f3a [828856,72621] 0 2024-09-10T04:05:13.181 INFO:tasks.workunit.client.0.smithi046.stdout:3/285: symlink d12/d13/d14/d1e/l56 0 2024-09-10T04:05:13.185 INFO:tasks.workunit.client.0.smithi046.stdout:7/340: creat d5/d6/d39/d6c/f75 x:0 0 0 2024-09-10T04:05:13.186 INFO:tasks.workunit.client.1.smithi070.stdout:0/353: creat d8/d24/d5e/f79 x:0 0 0 2024-09-10T04:05:13.188 INFO:tasks.workunit.client.1.smithi070.stdout:0/354: truncate d8/d24/d2d/d5f/f60 842692 0 2024-09-10T04:05:13.190 INFO:tasks.workunit.client.0.smithi046.stdout:8/291: truncate d1/d29/d17/d1f/f35 1418635 0 2024-09-10T04:05:13.190 INFO:tasks.workunit.client.0.smithi046.stdout:8/292: stat d1/d29/d19/c5c 0 2024-09-10T04:05:13.191 INFO:tasks.workunit.client.0.smithi046.stdout:8/293: chown d1/d29/d19/d36/f3f 310711 1 2024-09-10T04:05:13.192 INFO:tasks.workunit.client.1.smithi070.stdout:4/238: link c7 de/d13/d18/d23/d39/d48/c4e 0 2024-09-10T04:05:13.193 INFO:tasks.workunit.client.0.smithi046.stdout:8/294: dread d1/f2e [0,4194304] 0 2024-09-10T04:05:13.194 INFO:tasks.workunit.client.1.smithi070.stdout:0/355: dwrite d8/d13/d1a/f23 [4194304,4194304] 0 2024-09-10T04:05:13.198 INFO:tasks.workunit.client.0.smithi046.stdout:7/341: readlink d5/d9/l4c 0 2024-09-10T04:05:13.200 INFO:tasks.workunit.client.1.smithi070.stdout:9/333: creat d9/d59/f65 x:0 0 0 2024-09-10T04:05:13.202 INFO:tasks.workunit.client.0.smithi046.stdout:2/354: mknod d2/d6/c8a 0 2024-09-10T04:05:13.203 INFO:tasks.workunit.client.0.smithi046.stdout:8/295: dwrite d1/d29/d19/d36/d4c/f5b [0,4194304] 0 2024-09-10T04:05:13.205 INFO:tasks.workunit.client.0.smithi046.stdout:8/296: write d1/f49 [163527,68886] 0 2024-09-10T04:05:13.206 INFO:tasks.workunit.client.0.smithi046.stdout:0/249: symlink d1/dc/d25/d33/d1a/d41/l50 0 2024-09-10T04:05:13.211 INFO:tasks.workunit.client.1.smithi070.stdout:9/334: dread d9/d59/f12 [0,4194304] 0 2024-09-10T04:05:13.212 INFO:tasks.workunit.client.1.smithi070.stdout:9/335: read - d9/f43 zero size 2024-09-10T04:05:13.213 INFO:tasks.workunit.client.0.smithi046.stdout:4/317: creat d3/d12/d17/d29/d58/f6d x:0 0 0 2024-09-10T04:05:13.216 INFO:tasks.workunit.client.1.smithi070.stdout:6/383: write d0/d3/d18/f27 [4078009,51481] 0 2024-09-10T04:05:13.221 INFO:tasks.workunit.client.0.smithi046.stdout:7/342: rename d5/d6d to d5/d6/db/dd/d24/d76 0 2024-09-10T04:05:13.222 INFO:tasks.workunit.client.1.smithi070.stdout:4/239: mkdir de/d13/d4f 0 2024-09-10T04:05:13.222 INFO:tasks.workunit.client.1.smithi070.stdout:6/384: dwrite d0/d3/d1f/d55/f6d [0,4194304] 0 2024-09-10T04:05:13.222 INFO:tasks.workunit.client.0.smithi046.stdout:7/343: truncate d5/d6/db/f29 4930993 0 2024-09-10T04:05:13.224 INFO:tasks.workunit.client.0.smithi046.stdout:9/305: sync 2024-09-10T04:05:13.224 INFO:tasks.workunit.client.0.smithi046.stdout:7/344: truncate d5/d9/d43/f51 87604 0 2024-09-10T04:05:13.225 INFO:tasks.workunit.client.1.smithi070.stdout:2/281: rename d15/d18/d42/d43 to d15/d18/d42/d62 0 2024-09-10T04:05:13.226 INFO:tasks.workunit.client.0.smithi046.stdout:3/286: write d12/d13/d37/f45 [273862,30297] 0 2024-09-10T04:05:13.235 INFO:tasks.workunit.client.0.smithi046.stdout:4/318: creat d3/d12/d17/f6e x:0 0 0 2024-09-10T04:05:13.238 INFO:tasks.workunit.client.1.smithi070.stdout:8/354: dwrite d4/d16/f22 [0,4194304] 0 2024-09-10T04:05:13.245 INFO:tasks.workunit.client.0.smithi046.stdout:0/250: mknod d1/c51 0 2024-09-10T04:05:13.246 INFO:tasks.workunit.client.1.smithi070.stdout:0/356: link d8/d13/d33/c63 d8/d13/d33/d53/d59/c7a 0 2024-09-10T04:05:13.246 INFO:tasks.workunit.client.1.smithi070.stdout:9/336: write d9/d59/d10/f3d [4685788,61579] 0 2024-09-10T04:05:13.247 INFO:tasks.workunit.client.1.smithi070.stdout:9/337: fsync d9/f25 0 2024-09-10T04:05:13.248 INFO:tasks.workunit.client.1.smithi070.stdout:9/338: readlink d9/d59/d2a/l4b 0 2024-09-10T04:05:13.249 INFO:tasks.workunit.client.1.smithi070.stdout:9/339: truncate d9/d59/f65 966237 0 2024-09-10T04:05:13.256 INFO:tasks.workunit.client.0.smithi046.stdout:2/355: creat d2/f8b x:0 0 0 2024-09-10T04:05:13.256 INFO:tasks.workunit.client.0.smithi046.stdout:2/356: stat d2/d6/d23/d55 0 2024-09-10T04:05:13.257 INFO:tasks.workunit.client.1.smithi070.stdout:4/240: dwrite f0 [4194304,4194304] 0 2024-09-10T04:05:13.257 INFO:tasks.workunit.client.0.smithi046.stdout:8/297: write d1/d29/d17/d1f/d2a/f33 [1708152,43439] 0 2024-09-10T04:05:13.258 INFO:tasks.workunit.client.1.smithi070.stdout:2/282: getdents d15/d59 0 2024-09-10T04:05:13.259 INFO:tasks.workunit.client.0.smithi046.stdout:7/345: mkdir d5/d9/d12/d72/d77 0 2024-09-10T04:05:13.260 INFO:tasks.workunit.client.1.smithi070.stdout:0/357: mkdir d8/d13/d47/d7b 0 2024-09-10T04:05:13.262 INFO:tasks.workunit.client.1.smithi070.stdout:6/385: write d0/d3/d18/d1b/d33/f39 [3244212,2619] 0 2024-09-10T04:05:13.264 INFO:tasks.workunit.client.1.smithi070.stdout:9/340: rename d9/d59/f30 to d9/d59/d5d/f66 0 2024-09-10T04:05:13.265 INFO:tasks.workunit.client.0.smithi046.stdout:4/319: dwrite d3/d12/d17/f20 [4194304,4194304] 0 2024-09-10T04:05:13.266 INFO:tasks.workunit.client.1.smithi070.stdout:9/341: read d9/d59/f1a [858627,68879] 0 2024-09-10T04:05:13.266 INFO:tasks.workunit.client.1.smithi070.stdout:0/358: dwrite f5 [0,4194304] 0 2024-09-10T04:05:13.268 INFO:tasks.workunit.client.1.smithi070.stdout:0/359: truncate d8/d13/d33/f65 492630 0 2024-09-10T04:05:13.268 INFO:tasks.workunit.client.1.smithi070.stdout:8/355: link d4/fa d4/d16/d17/d1a/d60/d7a/f7b 0 2024-09-10T04:05:13.268 INFO:tasks.workunit.client.0.smithi046.stdout:4/320: write d3/d12/d17/f5e [12890,102364] 0 2024-09-10T04:05:13.269 INFO:tasks.workunit.client.0.smithi046.stdout:3/287: link d12/d1b/f4c d12/d13/d37/d51/f57 0 2024-09-10T04:05:13.270 INFO:tasks.workunit.client.1.smithi070.stdout:8/356: truncate d4/f77 627126 0 2024-09-10T04:05:13.270 INFO:tasks.workunit.client.0.smithi046.stdout:4/321: read d3/d12/d14/f21 [2031940,110781] 0 2024-09-10T04:05:13.277 INFO:tasks.workunit.client.0.smithi046.stdout:9/306: dwrite d4/d18/f33 [0,4194304] 0 2024-09-10T04:05:13.278 INFO:tasks.workunit.client.1.smithi070.stdout:6/386: dread d0/d3/d18/d1b/d17/d53/f61 [0,4194304] 0 2024-09-10T04:05:13.280 INFO:tasks.workunit.client.1.smithi070.stdout:4/241: mknod de/d13/d4f/c50 0 2024-09-10T04:05:13.281 INFO:tasks.workunit.client.1.smithi070.stdout:8/357: mknod d4/d16/d17/d19/c7c 0 2024-09-10T04:05:13.282 INFO:tasks.workunit.client.0.smithi046.stdout:3/288: mknod d12/d1b/c58 0 2024-09-10T04:05:13.283 INFO:tasks.workunit.client.0.smithi046.stdout:2/357: truncate d2/d6/f43 383740 0 2024-09-10T04:05:13.283 INFO:tasks.workunit.client.1.smithi070.stdout:7/265: sync 2024-09-10T04:05:13.283 INFO:tasks.workunit.client.1.smithi070.stdout:1/267: sync 2024-09-10T04:05:13.284 INFO:tasks.workunit.client.0.smithi046.stdout:2/358: stat d2/d6/d23/d5d/d77/d7b/d85/c6b 0 2024-09-10T04:05:13.285 INFO:tasks.workunit.client.0.smithi046.stdout:2/359: dread - d2/d6/f4f zero size 2024-09-10T04:05:13.287 INFO:tasks.workunit.client.1.smithi070.stdout:2/283: rename d15/d18/d56/f5f to d15/d18/f63 0 2024-09-10T04:05:13.289 INFO:tasks.workunit.client.1.smithi070.stdout:4/242: read de/d13/f1a [1981254,109534] 0 2024-09-10T04:05:13.289 INFO:tasks.workunit.client.1.smithi070.stdout:1/268: dread - d3/d8/d12/d2f/d32/f4c zero size 2024-09-10T04:05:13.291 INFO:tasks.workunit.client.1.smithi070.stdout:9/342: dwrite d9/d1c/f55 [0,4194304] 0 2024-09-10T04:05:13.292 INFO:tasks.workunit.client.1.smithi070.stdout:0/360: dwrite d8/d13/f1f [0,4194304] 0 2024-09-10T04:05:13.293 INFO:tasks.workunit.client.0.smithi046.stdout:0/251: creat d1/dc/d25/d33/f52 x:0 0 0 2024-09-10T04:05:13.300 INFO:tasks.workunit.client.0.smithi046.stdout:3/289: creat d12/d1b/f59 x:0 0 0 2024-09-10T04:05:13.300 INFO:tasks.workunit.client.0.smithi046.stdout:8/298: write d1/d29/f4a [415540,102001] 0 2024-09-10T04:05:13.303 INFO:tasks.workunit.client.0.smithi046.stdout:7/346: write d5/d6/db/d11/d1e/f30 [830010,76046] 0 2024-09-10T04:05:13.303 INFO:tasks.workunit.client.0.smithi046.stdout:4/322: getdents d3/d12/d17/d22/d51/d5c/d6c 0 2024-09-10T04:05:13.306 INFO:tasks.workunit.client.1.smithi070.stdout:2/284: chown d15/c23 91 1 2024-09-10T04:05:13.309 INFO:tasks.workunit.client.0.smithi046.stdout:3/290: dwrite d12/d1b/d50/f55 [0,4194304] 0 2024-09-10T04:05:13.311 INFO:tasks.workunit.client.1.smithi070.stdout:1/269: chown d3/d8/c33 490 1 2024-09-10T04:05:13.313 INFO:tasks.workunit.client.0.smithi046.stdout:8/299: mkdir d1/d29/d17/d1f/d46/d67 0 2024-09-10T04:05:13.315 INFO:tasks.workunit.client.0.smithi046.stdout:9/307: write d4/d7/f31 [180838,22227] 0 2024-09-10T04:05:13.316 INFO:tasks.workunit.client.0.smithi046.stdout:9/308: truncate d4/d7/fe 5512387 0 2024-09-10T04:05:13.316 INFO:tasks.workunit.client.1.smithi070.stdout:0/361: dread d8/d13/f5a [0,4194304] 0 2024-09-10T04:05:13.318 INFO:tasks.workunit.client.0.smithi046.stdout:2/360: fdatasync d2/d49/f2e 0 2024-09-10T04:05:13.319 INFO:tasks.workunit.client.1.smithi070.stdout:6/387: write d0/d3/d1f/f22 [1617845,113876] 0 2024-09-10T04:05:13.325 INFO:tasks.workunit.client.1.smithi070.stdout:2/285: write d15/d18/d29/d2a/d2e/d32/d36/f5b [2761353,70560] 0 2024-09-10T04:05:13.329 INFO:tasks.workunit.client.0.smithi046.stdout:7/347: mkdir d5/d6/d23/d78 0 2024-09-10T04:05:13.331 INFO:tasks.workunit.client.0.smithi046.stdout:3/291: write d12/f4d [4587220,66599] 0 2024-09-10T04:05:13.332 INFO:tasks.workunit.client.1.smithi070.stdout:8/358: truncate d4/f11 2780310 0 2024-09-10T04:05:13.332 INFO:tasks.workunit.client.1.smithi070.stdout:2/286: dread d15/d18/d29/f2d [0,4194304] 0 2024-09-10T04:05:13.332 INFO:tasks.workunit.client.0.smithi046.stdout:9/309: unlink d4/d7/f20 0 2024-09-10T04:05:13.333 INFO:tasks.workunit.client.0.smithi046.stdout:9/310: stat d4/d29/l52 0 2024-09-10T04:05:13.335 INFO:tasks.workunit.client.0.smithi046.stdout:2/361: rename d2/d6/d23/d42/c4a to d2/d6/d23/d5d/d77/d7b/d85/c8c 0 2024-09-10T04:05:13.338 INFO:tasks.workunit.client.0.smithi046.stdout:4/323: dwrite d3/d12/f19 [4194304,4194304] 0 2024-09-10T04:05:13.340 INFO:tasks.workunit.client.1.smithi070.stdout:3/343: sync 2024-09-10T04:05:13.340 INFO:tasks.workunit.client.1.smithi070.stdout:5/310: sync 2024-09-10T04:05:13.341 INFO:tasks.workunit.client.1.smithi070.stdout:5/311: fdatasync d17/d19/d6a/f53 0 2024-09-10T04:05:13.344 INFO:tasks.workunit.client.0.smithi046.stdout:2/362: dwrite d2/d6/f61 [0,4194304] 0 2024-09-10T04:05:13.410 INFO:tasks.workunit.client.1.smithi070.stdout:7/266: rename d2/d3/dc/d1b to d2/d3/dc/d5d 0 2024-09-10T04:05:13.411 INFO:tasks.workunit.client.1.smithi070.stdout:9/343: truncate d9/d3e/f5f 795461 0 2024-09-10T04:05:13.412 INFO:tasks.workunit.client.1.smithi070.stdout:9/344: chown d9/d59/d10/d3c/f40 87037 1 2024-09-10T04:05:13.413 INFO:tasks.workunit.client.1.smithi070.stdout:8/359: truncate d4/f12 2471560 0 2024-09-10T04:05:13.414 INFO:tasks.workunit.client.1.smithi070.stdout:8/360: chown d4/d16/d17/d19/d28/d43/d52/f6c 5 1 2024-09-10T04:05:13.414 INFO:tasks.workunit.client.0.smithi046.stdout:8/300: symlink d1/d29/d19/d36/d4c/l68 0 2024-09-10T04:05:13.416 INFO:tasks.workunit.client.1.smithi070.stdout:3/344: chown db/d1c/d25/l5e 3724147 1 2024-09-10T04:05:13.418 INFO:tasks.workunit.client.0.smithi046.stdout:9/311: creat d4/d29/d32/f63 x:0 0 0 2024-09-10T04:05:13.419 INFO:tasks.workunit.client.1.smithi070.stdout:6/388: mknod d0/d3/d5c/c7a 0 2024-09-10T04:05:13.419 INFO:tasks.workunit.client.1.smithi070.stdout:2/287: dwrite d15/d18/d29/f46 [4194304,4194304] 0 2024-09-10T04:05:13.424 INFO:tasks.workunit.client.0.smithi046.stdout:4/324: mknod d3/d12/d17/d29/c6f 0 2024-09-10T04:05:13.424 INFO:tasks.workunit.client.0.smithi046.stdout:4/325: chown d3/l42 880894598 1 2024-09-10T04:05:13.425 INFO:tasks.workunit.client.1.smithi070.stdout:4/243: getdents de/d13/d18/d23/d39/d48 0 2024-09-10T04:05:13.427 INFO:tasks.workunit.client.0.smithi046.stdout:2/363: write d2/d6/f18 [3445326,44453] 0 2024-09-10T04:05:13.429 INFO:tasks.workunit.client.0.smithi046.stdout:0/252: getdents d1/dc/d25/d33/d4 0 2024-09-10T04:05:13.433 INFO:tasks.workunit.client.0.smithi046.stdout:9/312: dread d4/d29/f4b [0,4194304] 0 2024-09-10T04:05:13.434 INFO:tasks.workunit.client.0.smithi046.stdout:3/292: mknod d12/d13/d14/c5a 0 2024-09-10T04:05:13.436 INFO:tasks.workunit.client.1.smithi070.stdout:0/362: rename d8/d24/d2d/d34/l5d to d8/d13/l7c 0 2024-09-10T04:05:13.437 INFO:tasks.workunit.client.0.smithi046.stdout:2/364: dwrite d2/d6/d23/d5d/d77/d7b/d85/f76 [0,4194304] 0 2024-09-10T04:05:13.437 INFO:tasks.workunit.client.0.smithi046.stdout:3/293: chown d12/d13/d14/d1e/d2d/f2e 16354 1 2024-09-10T04:05:13.441 INFO:tasks.workunit.client.0.smithi046.stdout:3/294: truncate d12/d13/d14/f47 457645 0 2024-09-10T04:05:13.442 INFO:tasks.workunit.client.0.smithi046.stdout:8/301: link d1/d29/d19/f34 d1/d29/d17/f69 0 2024-09-10T04:05:13.443 INFO:tasks.workunit.client.1.smithi070.stdout:6/389: write d0/d3/d18/d1b/d17/d53/f61 [1946693,104852] 0 2024-09-10T04:05:13.444 INFO:tasks.workunit.client.0.smithi046.stdout:9/313: dwrite d4/d18/f4f [0,4194304] 0 2024-09-10T04:05:13.447 INFO:tasks.workunit.client.0.smithi046.stdout:9/314: write d4/d7/f5e [764911,40136] 0 2024-09-10T04:05:13.447 INFO:tasks.workunit.client.1.smithi070.stdout:9/345: dwrite d9/d59/d10/f19 [0,4194304] 0 2024-09-10T04:05:13.448 INFO:tasks.workunit.client.1.smithi070.stdout:3/345: dwrite db/f3c [0,4194304] 0 2024-09-10T04:05:13.449 INFO:tasks.workunit.client.0.smithi046.stdout:9/315: dread - d4/d3c/d4d/d5c/d59/f5d zero size 2024-09-10T04:05:13.450 INFO:tasks.workunit.client.1.smithi070.stdout:9/346: write d9/d3e/f47 [1061074,97690] 0 2024-09-10T04:05:13.451 INFO:tasks.workunit.client.1.smithi070.stdout:8/361: mkdir d4/d16/d7d 0 2024-09-10T04:05:13.451 INFO:tasks.workunit.client.0.smithi046.stdout:2/365: mkdir d2/d6/d23/d5d/d77/d7b/d85/d8d 0 2024-09-10T04:05:13.452 INFO:tasks.workunit.client.1.smithi070.stdout:9/347: stat d9/d1c/f55 0 2024-09-10T04:05:13.452 INFO:tasks.workunit.client.1.smithi070.stdout:3/346: read db/f3f [102398,28185] 0 2024-09-10T04:05:13.454 INFO:tasks.workunit.client.0.smithi046.stdout:3/295: rename d12/d1b/f59 to d12/d13/d42/f5b 0 2024-09-10T04:05:13.455 INFO:tasks.workunit.client.0.smithi046.stdout:3/296: readlink d12/d13/l35 0 2024-09-10T04:05:13.457 INFO:tasks.workunit.client.1.smithi070.stdout:0/363: mkdir d8/d13/d33/d7d 0 2024-09-10T04:05:13.461 INFO:tasks.workunit.client.1.smithi070.stdout:0/364: write d8/d13/d33/d4f/f6e [130269,104027] 0 2024-09-10T04:05:13.461 INFO:tasks.workunit.client.0.smithi046.stdout:8/302: mkdir d1/d29/d17/d1f/d46/d6a 0 2024-09-10T04:05:13.461 INFO:tasks.workunit.client.0.smithi046.stdout:1/218: sync 2024-09-10T04:05:13.461 INFO:tasks.workunit.client.0.smithi046.stdout:6/298: sync 2024-09-10T04:05:13.461 INFO:tasks.workunit.client.0.smithi046.stdout:5/395: sync 2024-09-10T04:05:13.461 INFO:tasks.workunit.client.0.smithi046.stdout:4/326: creat d3/d12/d17/d22/f70 x:0 0 0 2024-09-10T04:05:13.462 INFO:tasks.workunit.client.0.smithi046.stdout:4/327: readlink d3/d12/l3d 0 2024-09-10T04:05:13.462 INFO:tasks.workunit.client.0.smithi046.stdout:2/366: mknod d2/d6/d23/d5d/d5e/c8e 0 2024-09-10T04:05:13.463 INFO:tasks.workunit.client.1.smithi070.stdout:6/390: mknod d0/d3/d1f/c7b 0 2024-09-10T04:05:13.468 INFO:tasks.workunit.client.0.smithi046.stdout:3/297: mkdir d12/d13/d42/d49/d5c 0 2024-09-10T04:05:13.468 INFO:tasks.workunit.client.1.smithi070.stdout:4/244: creat de/d13/d18/d23/d47/f51 x:0 0 0 2024-09-10T04:05:13.469 INFO:tasks.workunit.client.0.smithi046.stdout:1/219: dread d3/d27/f38 [0,4194304] 0 2024-09-10T04:05:13.470 INFO:tasks.workunit.client.0.smithi046.stdout:8/303: creat d1/d29/d12/f6b x:0 0 0 2024-09-10T04:05:13.470 INFO:tasks.workunit.client.0.smithi046.stdout:9/316: dread d4/d3c/f40 [0,4194304] 0 2024-09-10T04:05:13.471 INFO:tasks.workunit.client.0.smithi046.stdout:8/304: read - d1/d29/d17/d1f/d46/d52/f65 zero size 2024-09-10T04:05:13.473 INFO:tasks.workunit.client.1.smithi070.stdout:8/362: symlink d4/d16/d17/d5c/l7e 0 2024-09-10T04:05:13.474 INFO:tasks.workunit.client.0.smithi046.stdout:4/328: creat d3/d12/d17/d29/d58/f71 x:0 0 0 2024-09-10T04:05:13.478 INFO:tasks.workunit.client.0.smithi046.stdout:2/367: creat d2/d6/d23/d5d/f8f x:0 0 0 2024-09-10T04:05:13.479 INFO:tasks.workunit.client.1.smithi070.stdout:0/365: readlink d8/d24/l76 0 2024-09-10T04:05:13.480 INFO:tasks.workunit.client.1.smithi070.stdout:5/312: getdents d17/d37 0 2024-09-10T04:05:13.480 INFO:tasks.workunit.client.0.smithi046.stdout:9/317: dwrite d4/d18/f38 [0,4194304] 0 2024-09-10T04:05:13.481 INFO:tasks.workunit.client.0.smithi046.stdout:1/220: dwrite d3/d27/d32/d1f/f29 [4194304,4194304] 0 2024-09-10T04:05:13.482 INFO:tasks.workunit.client.0.smithi046.stdout:0/253: getdents d1/dc/d25/d33/d3/dd 0 2024-09-10T04:05:13.483 INFO:tasks.workunit.client.1.smithi070.stdout:2/288: truncate d15/d18/d29/d2a/f58 4912457 0 2024-09-10T04:05:13.485 INFO:tasks.workunit.client.1.smithi070.stdout:7/267: getdents d2/d7/d9/d18 0 2024-09-10T04:05:13.486 INFO:tasks.workunit.client.1.smithi070.stdout:2/289: read - d15/d18/d29/d2a/d2e/d32/f57 zero size 2024-09-10T04:05:13.487 INFO:tasks.workunit.client.1.smithi070.stdout:0/366: dread d8/d13/f19 [0,4194304] 0 2024-09-10T04:05:13.487 INFO:tasks.workunit.client.1.smithi070.stdout:5/313: dwrite f10 [0,4194304] 0 2024-09-10T04:05:13.488 INFO:tasks.workunit.client.1.smithi070.stdout:4/245: creat de/d13/d18/d23/f52 x:0 0 0 2024-09-10T04:05:13.488 INFO:tasks.workunit.client.1.smithi070.stdout:0/367: readlink d8/d13/d1a/l29 0 2024-09-10T04:05:13.490 INFO:tasks.workunit.client.0.smithi046.stdout:0/254: dwrite d1/d13/f48 [0,4194304] 0 2024-09-10T04:05:13.493 INFO:tasks.workunit.client.0.smithi046.stdout:5/396: mknod d7/d11/d14/d8e/c90 0 2024-09-10T04:05:13.495 INFO:tasks.workunit.client.0.smithi046.stdout:0/255: read d1/dc/d25/d33/d3/dd/f23 [6853653,92506] 0 2024-09-10T04:05:13.501 INFO:tasks.workunit.client.0.smithi046.stdout:0/256: fsync d1/dc/d25/d33/d3/dd/f23 0 2024-09-10T04:05:13.501 INFO:tasks.workunit.client.0.smithi046.stdout:6/299: mknod da/db/d11/d13/d4f/c64 0 2024-09-10T04:05:13.501 INFO:tasks.workunit.client.0.smithi046.stdout:4/329: chown d3/d12/d17/d22/d30/c68 676 1 2024-09-10T04:05:13.505 INFO:tasks.workunit.client.1.smithi070.stdout:6/391: creat d0/d3/d18/d62/d70/f7c x:0 0 0 2024-09-10T04:05:13.508 INFO:tasks.workunit.client.1.smithi070.stdout:7/268: unlink d2/f4c 0 2024-09-10T04:05:13.509 INFO:tasks.workunit.client.0.smithi046.stdout:2/368: mkdir d2/d8/d65/d90 0 2024-09-10T04:05:13.512 INFO:tasks.workunit.client.0.smithi046.stdout:4/330: dwrite d3/d12/d17/d22/f59 [0,4194304] 0 2024-09-10T04:05:13.513 INFO:tasks.workunit.client.0.smithi046.stdout:4/331: write d3/d12/d17/d22/f59 [1908202,26337] 0 2024-09-10T04:05:13.515 INFO:tasks.workunit.client.1.smithi070.stdout:5/314: creat d17/d19/d35/f6c x:0 0 0 2024-09-10T04:05:13.516 INFO:tasks.workunit.client.0.smithi046.stdout:3/298: mkdir d12/d5d 0 2024-09-10T04:05:13.518 INFO:tasks.workunit.client.0.smithi046.stdout:3/299: write d12/f4d [1730328,3748] 0 2024-09-10T04:05:13.519 INFO:tasks.workunit.client.0.smithi046.stdout:8/305: symlink d1/d29/d19/d36/d4c/d5e/l6c 0 2024-09-10T04:05:13.522 INFO:tasks.workunit.client.1.smithi070.stdout:8/363: truncate d4/fa 3761395 0 2024-09-10T04:05:13.526 INFO:tasks.workunit.client.0.smithi046.stdout:5/397: creat d7/d11/d14/d45/d51/f91 x:0 0 0 2024-09-10T04:05:13.526 INFO:tasks.workunit.client.0.smithi046.stdout:7/348: sync 2024-09-10T04:05:13.527 INFO:tasks.workunit.client.1.smithi070.stdout:1/270: rename d3/d8/f28 to d3/d8/d13/d3f/f52 0 2024-09-10T04:05:13.528 INFO:tasks.workunit.client.0.smithi046.stdout:7/349: readlink d5/d9/l4c 0 2024-09-10T04:05:13.528 INFO:tasks.workunit.client.0.smithi046.stdout:3/300: dread d12/d13/d42/d49/f4e [0,4194304] 0 2024-09-10T04:05:13.529 INFO:tasks.workunit.client.0.smithi046.stdout:8/306: dwrite d1/d29/d17/d1f/d46/d52/f65 [0,4194304] 0 2024-09-10T04:05:13.530 INFO:tasks.workunit.client.0.smithi046.stdout:0/257: rename d1/d13/f48 to d1/d13/f53 0 2024-09-10T04:05:13.530 INFO:tasks.workunit.client.1.smithi070.stdout:6/392: rmdir d0/d3/d18/d1b/d17/d53 39 2024-09-10T04:05:13.534 INFO:tasks.workunit.client.0.smithi046.stdout:8/307: chown d1/d29/d17/d1f/d46/d52 129736 1 2024-09-10T04:05:13.534 INFO:tasks.workunit.client.0.smithi046.stdout:3/301: write d12/d13/d42/d49/f4e [1176096,121899] 0 2024-09-10T04:05:13.534 INFO:tasks.workunit.client.0.smithi046.stdout:3/302: stat c10 0 2024-09-10T04:05:13.534 INFO:tasks.workunit.client.0.smithi046.stdout:8/308: chown d1/d29/d17/d1f/c3a 0 1 2024-09-10T04:05:13.535 INFO:tasks.workunit.client.1.smithi070.stdout:3/347: truncate db/d1c/d25/d28/f37 257914 0 2024-09-10T04:05:13.539 INFO:tasks.workunit.client.0.smithi046.stdout:6/300: creat da/db/d11/d2b/d2f/f65 x:0 0 0 2024-09-10T04:05:13.540 INFO:tasks.workunit.client.1.smithi070.stdout:2/290: mkdir d15/d18/d29/d64 0 2024-09-10T04:05:13.542 INFO:tasks.workunit.client.1.smithi070.stdout:4/246: creat de/d13/d43/f53 x:0 0 0 2024-09-10T04:05:13.545 INFO:tasks.workunit.client.0.smithi046.stdout:6/301: chown da/db/d11/d2b/d2f/f37 0 1 2024-09-10T04:05:13.545 INFO:tasks.workunit.client.1.smithi070.stdout:1/271: dread d3/d8/d13/f18 [0,4194304] 0 2024-09-10T04:05:13.545 INFO:tasks.workunit.client.1.smithi070.stdout:0/368: link d8/d13/f3d d8/d13/d1a/f7e 0 2024-09-10T04:05:13.546 INFO:tasks.workunit.client.0.smithi046.stdout:9/318: write d4/d7/f8 [1591987,34917] 0 2024-09-10T04:05:13.548 INFO:tasks.workunit.client.0.smithi046.stdout:8/309: dwrite d1/d29/d17/d1f/f44 [0,4194304] 0 2024-09-10T04:05:13.552 INFO:tasks.workunit.client.0.smithi046.stdout:3/303: dwrite d12/d1b/f21 [0,4194304] 0 2024-09-10T04:05:13.553 INFO:tasks.workunit.client.0.smithi046.stdout:4/332: chown d3/d12/d17/d29/c3b 77 1 2024-09-10T04:05:13.553 INFO:tasks.workunit.client.1.smithi070.stdout:9/348: rename d9/d59/f56 to d9/d59/d5d/f67 0 2024-09-10T04:05:13.554 INFO:tasks.workunit.client.0.smithi046.stdout:6/302: chown da/db/f16 229534709 1 2024-09-10T04:05:13.555 INFO:tasks.workunit.client.1.smithi070.stdout:9/349: truncate d9/d1c/d34/f33 5032202 0 2024-09-10T04:05:13.556 INFO:tasks.workunit.client.0.smithi046.stdout:9/319: chown d4/d7/l60 51185528 1 2024-09-10T04:05:13.558 INFO:tasks.workunit.client.0.smithi046.stdout:6/303: fdatasync da/f60 0 2024-09-10T04:05:13.558 INFO:tasks.workunit.client.0.smithi046.stdout:9/320: chown d4/d44 1264410224 1 2024-09-10T04:05:13.558 INFO:tasks.workunit.client.1.smithi070.stdout:3/348: rmdir db/d1c/d25/d28 39 2024-09-10T04:05:13.559 INFO:tasks.workunit.client.0.smithi046.stdout:2/369: dwrite d2/d6/d15/f70 [0,4194304] 0 2024-09-10T04:05:13.584 INFO:tasks.workunit.client.0.smithi046.stdout:7/350: rename d5/d6/db/d11/d1e/d20/f4a to d5/d6/d39/d5a/f79 0 2024-09-10T04:05:13.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:13 smithi070 ceph-mon[35495]: pgmap v260: 65 pgs: 65 active+clean; 1.0 GiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 12 MiB/s rd, 116 MiB/s wr, 311 op/s 2024-09-10T04:05:13.585 INFO:tasks.workunit.client.1.smithi070.stdout:2/291: creat d15/d18/d42/d62/d55/f65 x:0 0 0 2024-09-10T04:05:13.586 INFO:tasks.workunit.client.0.smithi046.stdout:0/258: truncate d1/dc/d25/d33/d3/f2a 161683 0 2024-09-10T04:05:13.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:13 smithi046 ceph-mon[28328]: pgmap v260: 65 pgs: 65 active+clean; 1.0 GiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 12 MiB/s rd, 116 MiB/s wr, 311 op/s 2024-09-10T04:05:13.587 INFO:tasks.workunit.client.1.smithi070.stdout:4/247: mknod de/d13/d18/d23/d39/c54 0 2024-09-10T04:05:13.590 INFO:tasks.workunit.client.1.smithi070.stdout:0/369: creat d8/d13/d33/d4f/f7f x:0 0 0 2024-09-10T04:05:13.590 INFO:tasks.workunit.client.1.smithi070.stdout:0/370: chown d8/d13/c61 223 1 2024-09-10T04:05:13.591 INFO:tasks.workunit.client.0.smithi046.stdout:1/221: creat d3/f4a x:0 0 0 2024-09-10T04:05:13.591 INFO:tasks.workunit.client.1.smithi070.stdout:0/371: truncate d8/d24/d36/f46 6807100 0 2024-09-10T04:05:13.592 INFO:tasks.workunit.client.1.smithi070.stdout:8/364: mknod d4/d16/d73/c7f 0 2024-09-10T04:05:13.592 INFO:tasks.workunit.client.1.smithi070.stdout:0/372: readlink d8/l15 0 2024-09-10T04:05:13.592 INFO:tasks.workunit.client.0.smithi046.stdout:1/222: read - d3/d27/d32/d1f/f3c zero size 2024-09-10T04:05:13.593 INFO:tasks.workunit.client.1.smithi070.stdout:8/365: write d4/d16/d17/d19/d28/d43/f5d [309484,108524] 0 2024-09-10T04:05:13.595 INFO:tasks.workunit.client.0.smithi046.stdout:3/304: mkdir d12/d13/d42/d49/d5e 0 2024-09-10T04:05:13.597 INFO:tasks.workunit.client.1.smithi070.stdout:7/269: truncate d2/d7/db/f37 141760 0 2024-09-10T04:05:13.599 INFO:tasks.workunit.client.0.smithi046.stdout:3/305: dread d12/d13/d14/f24 [0,4194304] 0 2024-09-10T04:05:13.600 INFO:tasks.workunit.client.0.smithi046.stdout:5/398: chown d7/d11/d3f/d5a/l61 259263627 1 2024-09-10T04:05:13.603 INFO:tasks.workunit.client.1.smithi070.stdout:4/248: write de/d13/f1a [2931729,2317] 0 2024-09-10T04:05:13.604 INFO:tasks.workunit.client.0.smithi046.stdout:6/304: creat da/db/d11/d2b/d2f/f66 x:0 0 0 2024-09-10T04:05:13.605 INFO:tasks.workunit.client.0.smithi046.stdout:6/305: dread - da/f24 zero size 2024-09-10T04:05:13.605 INFO:tasks.workunit.client.1.smithi070.stdout:1/272: unlink d3/d8/d12/c4e 0 2024-09-10T04:05:13.606 INFO:tasks.workunit.client.1.smithi070.stdout:1/273: stat d3/d8/d12/f45 0 2024-09-10T04:05:13.607 INFO:tasks.workunit.client.1.smithi070.stdout:1/274: truncate d3/d8/d12/f4a 333870 0 2024-09-10T04:05:13.608 INFO:tasks.workunit.client.0.smithi046.stdout:6/306: dread da/db/d18/f35 [0,4194304] 0 2024-09-10T04:05:13.609 INFO:tasks.workunit.client.0.smithi046.stdout:2/370: rename d2/l83 to d2/d6/d23/d5d/d77/d7b/d85/d8d/l91 0 2024-09-10T04:05:13.611 INFO:tasks.workunit.client.0.smithi046.stdout:6/307: write da/db/d11/d2b/d2f/f65 [636907,34368] 0 2024-09-10T04:05:13.611 INFO:tasks.workunit.client.1.smithi070.stdout:0/373: rename d8/d13/d1a/l29 to d8/d13/d33/d74/l80 0 2024-09-10T04:05:13.612 INFO:tasks.workunit.client.0.smithi046.stdout:7/351: mkdir d5/d6/db/d11/d1f/d7a 0 2024-09-10T04:05:13.613 INFO:tasks.workunit.client.1.smithi070.stdout:8/366: mknod d4/d16/d17/d1a/d23/c80 0 2024-09-10T04:05:13.615 INFO:tasks.workunit.client.0.smithi046.stdout:0/259: symlink d1/d13/d37/d3e/l54 0 2024-09-10T04:05:13.620 INFO:tasks.workunit.client.0.smithi046.stdout:8/310: symlink d1/d29/d17/d1f/d46/d67/l6d 0 2024-09-10T04:05:13.621 INFO:tasks.workunit.client.0.smithi046.stdout:6/308: dread da/db/d10/f17 [8388608,4194304] 0 2024-09-10T04:05:13.622 INFO:tasks.workunit.client.0.smithi046.stdout:5/399: mkdir d7/d11/d14/d2f/d2b/d92 0 2024-09-10T04:05:13.624 INFO:tasks.workunit.client.0.smithi046.stdout:9/321: mknod d4/d7/d1f/d54/d56/c64 0 2024-09-10T04:05:13.625 INFO:tasks.workunit.client.1.smithi070.stdout:6/393: creat d0/d3/d18/d1b/f7d x:0 0 0 2024-09-10T04:05:13.628 INFO:tasks.workunit.client.1.smithi070.stdout:3/349: link db/d1c/d3e/f50 db/d1c/d53/f6b 0 2024-09-10T04:05:13.629 INFO:tasks.workunit.client.1.smithi070.stdout:4/249: symlink de/d13/d18/d23/l55 0 2024-09-10T04:05:13.634 INFO:tasks.workunit.client.1.smithi070.stdout:0/374: unlink d8/d13/d1a/f6a 0 2024-09-10T04:05:13.634 INFO:tasks.workunit.client.1.smithi070.stdout:1/275: dwrite d3/d8/fc [0,4194304] 0 2024-09-10T04:05:13.638 INFO:tasks.workunit.client.1.smithi070.stdout:9/350: creat d9/f68 x:0 0 0 2024-09-10T04:05:13.641 INFO:tasks.workunit.client.1.smithi070.stdout:6/394: creat d0/d3/d18/d62/d70/f7e x:0 0 0 2024-09-10T04:05:13.642 INFO:tasks.workunit.client.0.smithi046.stdout:6/309: write da/db/d10/f17 [935373,51632] 0 2024-09-10T04:05:13.644 INFO:tasks.workunit.client.0.smithi046.stdout:5/400: creat d7/d11/d3f/d53/d75/f93 x:0 0 0 2024-09-10T04:05:13.645 INFO:tasks.workunit.client.1.smithi070.stdout:9/351: dwrite d9/f43 [0,4194304] 0 2024-09-10T04:05:13.646 INFO:tasks.workunit.client.1.smithi070.stdout:4/250: creat de/d13/d4f/f56 x:0 0 0 2024-09-10T04:05:13.646 INFO:tasks.workunit.client.1.smithi070.stdout:0/375: rmdir d8/d24/d36 39 2024-09-10T04:05:13.647 INFO:tasks.workunit.client.1.smithi070.stdout:4/251: write de/d13/d18/d26/f2c [15437,90622] 0 2024-09-10T04:05:13.652 INFO:tasks.workunit.client.1.smithi070.stdout:8/367: unlink d4/d16/d17/d1a/d23/f50 0 2024-09-10T04:05:13.653 INFO:tasks.workunit.client.0.smithi046.stdout:7/352: mknod d5/d6/d23/c7b 0 2024-09-10T04:05:13.660 INFO:tasks.workunit.client.1.smithi070.stdout:6/395: fsync d0/d3/d18/f42 0 2024-09-10T04:05:13.661 INFO:tasks.workunit.client.0.smithi046.stdout:9/322: mkdir d4/d29/d65 0 2024-09-10T04:05:13.662 INFO:tasks.workunit.client.1.smithi070.stdout:3/350: symlink db/d1c/d25/d28/d43/l6c 0 2024-09-10T04:05:13.664 INFO:tasks.workunit.client.0.smithi046.stdout:3/306: rename d12/d1b/f2b to d12/f5f 0 2024-09-10T04:05:13.667 INFO:tasks.workunit.client.0.smithi046.stdout:2/371: write d2/d6/d23/d5d/d77/f69 [900890,80952] 0 2024-09-10T04:05:13.670 INFO:tasks.workunit.client.1.smithi070.stdout:0/376: rename d8/d13/d47/d73 to d8/d13/d33/d81 0 2024-09-10T04:05:13.671 INFO:tasks.workunit.client.1.smithi070.stdout:7/270: dread d2/d7/db/f37 [0,4194304] 0 2024-09-10T04:05:13.671 INFO:tasks.workunit.client.1.smithi070.stdout:0/377: write d8/fc [2625490,64731] 0 2024-09-10T04:05:13.671 INFO:tasks.workunit.client.0.smithi046.stdout:8/311: truncate d1/d29/d12/f45 3117282 0 2024-09-10T04:05:13.672 INFO:tasks.workunit.client.0.smithi046.stdout:6/310: write f9 [2560941,50479] 0 2024-09-10T04:05:13.674 INFO:tasks.workunit.client.0.smithi046.stdout:8/312: write d1/d29/d17/d1f/d2a/f58 [5177134,96711] 0 2024-09-10T04:05:13.674 INFO:tasks.workunit.client.1.smithi070.stdout:8/368: unlink d4/d16/d17/d5c/l65 0 2024-09-10T04:05:13.676 INFO:tasks.workunit.client.0.smithi046.stdout:8/313: stat d1/d29/d17/d1f/d46/d67 0 2024-09-10T04:05:13.677 INFO:tasks.workunit.client.0.smithi046.stdout:0/260: rename d1/d13/d37/d3e to d1/dc/d25/d33/d3/dd/d55 0 2024-09-10T04:05:13.679 INFO:tasks.workunit.client.0.smithi046.stdout:0/261: truncate d1/dc/d25/d33/f3f 30781 0 2024-09-10T04:05:13.679 INFO:tasks.workunit.client.0.smithi046.stdout:6/311: dwrite da/f40 [0,4194304] 0 2024-09-10T04:05:13.681 INFO:tasks.workunit.client.0.smithi046.stdout:0/262: chown d1/dc/f21 184651179 1 2024-09-10T04:05:13.681 INFO:tasks.workunit.client.0.smithi046.stdout:6/312: chown da/db/d11/l33 3528429 1 2024-09-10T04:05:13.683 INFO:tasks.workunit.client.0.smithi046.stdout:5/401: creat d7/f94 x:0 0 0 2024-09-10T04:05:13.684 INFO:tasks.workunit.client.1.smithi070.stdout:9/352: creat d9/d1c/d34/d63/f69 x:0 0 0 2024-09-10T04:05:13.684 INFO:tasks.workunit.client.0.smithi046.stdout:9/323: mkdir d4/d66 0 2024-09-10T04:05:13.685 INFO:tasks.workunit.client.0.smithi046.stdout:9/324: write d4/d18/f27 [1040275,23924] 0 2024-09-10T04:05:13.688 INFO:tasks.workunit.client.1.smithi070.stdout:4/252: rename de/d13/d18/d23/f44 to de/d13/d18/d23/d39/f57 0 2024-09-10T04:05:13.688 INFO:tasks.workunit.client.1.smithi070.stdout:9/353: dread d9/d59/d10/d3c/f40 [0,4194304] 0 2024-09-10T04:05:13.689 INFO:tasks.workunit.client.0.smithi046.stdout:8/314: rmdir d1/d29/d19/d36 39 2024-09-10T04:05:13.689 INFO:tasks.workunit.client.1.smithi070.stdout:9/354: write d9/d1c/f55 [4076538,46076] 0 2024-09-10T04:05:13.691 INFO:tasks.workunit.client.0.smithi046.stdout:8/315: stat d1/d29/d17/l47 0 2024-09-10T04:05:13.692 INFO:tasks.workunit.client.0.smithi046.stdout:5/402: dread d7/f43 [0,4194304] 0 2024-09-10T04:05:13.692 INFO:tasks.workunit.client.1.smithi070.stdout:0/378: mknod d8/d13/d33/d53/d59/c82 0 2024-09-10T04:05:13.692 INFO:tasks.workunit.client.0.smithi046.stdout:0/263: rmdir d1/d13 39 2024-09-10T04:05:13.693 INFO:tasks.workunit.client.1.smithi070.stdout:0/379: fdatasync d8/d24/f43 0 2024-09-10T04:05:13.693 INFO:tasks.workunit.client.0.smithi046.stdout:5/403: write d7/d11/d3f/d53/d75/f93 [347330,29030] 0 2024-09-10T04:05:13.695 INFO:tasks.workunit.client.1.smithi070.stdout:8/369: symlink d4/d16/d17/d19/d28/d43/l81 0 2024-09-10T04:05:13.698 INFO:tasks.workunit.client.1.smithi070.stdout:0/380: dwrite d8/d13/f19 [0,4194304] 0 2024-09-10T04:05:13.699 INFO:tasks.workunit.client.1.smithi070.stdout:0/381: dread - d8/d13/d1a/f7e zero size 2024-09-10T04:05:13.701 INFO:tasks.workunit.client.1.smithi070.stdout:6/396: mkdir d0/d7f 0 2024-09-10T04:05:13.701 INFO:tasks.workunit.client.1.smithi070.stdout:8/370: dread d4/d16/d17/d19/f5b [0,4194304] 0 2024-09-10T04:05:13.707 INFO:tasks.workunit.client.0.smithi046.stdout:3/307: truncate d12/f5f 4150360 0 2024-09-10T04:05:13.711 INFO:tasks.workunit.client.1.smithi070.stdout:4/253: read de/d13/d18/d23/f3c [183094,130606] 0 2024-09-10T04:05:13.712 INFO:tasks.workunit.client.1.smithi070.stdout:1/276: creat d3/d8/d13/d3f/f53 x:0 0 0 2024-09-10T04:05:13.712 INFO:tasks.workunit.client.0.smithi046.stdout:9/325: getdents d4/d66 0 2024-09-10T04:05:13.716 INFO:tasks.workunit.client.0.smithi046.stdout:0/264: symlink d1/l56 0 2024-09-10T04:05:13.719 INFO:tasks.workunit.client.0.smithi046.stdout:0/265: dread - d1/dc/d25/d33/d3/d29/f40 zero size 2024-09-10T04:05:13.720 INFO:tasks.workunit.client.0.smithi046.stdout:8/316: dwrite d1/f2e [0,4194304] 0 2024-09-10T04:05:13.720 INFO:tasks.workunit.client.0.smithi046.stdout:2/372: truncate d2/f35 32077 0 2024-09-10T04:05:13.723 INFO:tasks.workunit.client.1.smithi070.stdout:9/355: write d9/d59/f1a [135800,115411] 0 2024-09-10T04:05:13.725 INFO:tasks.workunit.client.0.smithi046.stdout:8/317: write d1/d29/d17/d1f/d2a/f58 [1972023,5854] 0 2024-09-10T04:05:13.727 INFO:tasks.workunit.client.0.smithi046.stdout:8/318: read - d1/d29/d12/f6b zero size 2024-09-10T04:05:13.727 INFO:tasks.workunit.client.0.smithi046.stdout:9/326: stat d4/d3c/d4d/c58 0 2024-09-10T04:05:13.728 INFO:tasks.workunit.client.0.smithi046.stdout:3/308: dread d12/d13/d14/f24 [0,4194304] 0 2024-09-10T04:05:13.728 INFO:tasks.workunit.client.1.smithi070.stdout:3/351: creat db/d1c/f6d x:0 0 0 2024-09-10T04:05:13.729 INFO:tasks.workunit.client.0.smithi046.stdout:3/309: write d12/d13/f17 [2807967,103150] 0 2024-09-10T04:05:13.730 INFO:tasks.workunit.client.1.smithi070.stdout:8/371: creat d4/d16/d17/d19/d28/d43/d55/f82 x:0 0 0 2024-09-10T04:05:13.731 INFO:tasks.workunit.client.0.smithi046.stdout:5/404: creat d7/d11/d14/f95 x:0 0 0 2024-09-10T04:05:13.735 INFO:tasks.workunit.client.0.smithi046.stdout:2/373: rename d2/d6/d23/d5d/d77/d7b/d85/f76 to d2/d6/d23/d5d/d77/d7b/f92 0 2024-09-10T04:05:13.737 INFO:tasks.workunit.client.0.smithi046.stdout:8/319: dwrite d1/d29/f4a [0,4194304] 0 2024-09-10T04:05:13.738 INFO:tasks.workunit.client.1.smithi070.stdout:5/315: sync 2024-09-10T04:05:13.738 INFO:tasks.workunit.client.1.smithi070.stdout:5/316: fsync f3 0 2024-09-10T04:05:13.740 INFO:tasks.workunit.client.0.smithi046.stdout:8/320: truncate d1/d29/d17/d1f/f44 4884780 0 2024-09-10T04:05:13.741 INFO:tasks.workunit.client.0.smithi046.stdout:8/321: fsync d1/d29/d17/d1f/d2a/f2c 0 2024-09-10T04:05:13.743 INFO:tasks.workunit.client.1.smithi070.stdout:6/397: dwrite d0/d3/f51 [0,4194304] 0 2024-09-10T04:05:13.743 INFO:tasks.workunit.client.1.smithi070.stdout:5/317: dread d17/d24/d45/f4f [0,4194304] 0 2024-09-10T04:05:13.744 INFO:tasks.workunit.client.1.smithi070.stdout:0/382: symlink d8/d13/d33/d7d/l83 0 2024-09-10T04:05:13.749 INFO:tasks.workunit.client.1.smithi070.stdout:3/352: chown db/d1c/c24 4417 1 2024-09-10T04:05:13.751 INFO:tasks.workunit.client.1.smithi070.stdout:8/372: readlink d4/d16/l38 0 2024-09-10T04:05:13.751 INFO:tasks.workunit.client.0.smithi046.stdout:2/374: rmdir d2/d57 39 2024-09-10T04:05:13.752 INFO:tasks.workunit.client.1.smithi070.stdout:7/271: getdents d2/d3 0 2024-09-10T04:05:13.754 INFO:tasks.workunit.client.0.smithi046.stdout:9/327: creat d4/d29/d65/f67 x:0 0 0 2024-09-10T04:05:13.755 INFO:tasks.workunit.client.1.smithi070.stdout:3/353: dwrite db/d1c/f4b [0,4194304] 0 2024-09-10T04:05:13.757 INFO:tasks.workunit.client.1.smithi070.stdout:3/354: write db/f11 [3522464,84869] 0 2024-09-10T04:05:13.757 INFO:tasks.workunit.client.1.smithi070.stdout:3/355: chown db/d1c/d25/c36 12 1 2024-09-10T04:05:13.757 INFO:tasks.workunit.client.1.smithi070.stdout:1/277: mkdir d3/d8/d12/d2f/d32/d4b/d54 0 2024-09-10T04:05:13.761 INFO:tasks.workunit.client.0.smithi046.stdout:2/375: dwrite d2/d6/f80 [0,4194304] 0 2024-09-10T04:05:13.767 INFO:tasks.workunit.client.1.smithi070.stdout:5/318: symlink d17/d37/l6d 0 2024-09-10T04:05:13.768 INFO:tasks.workunit.client.0.smithi046.stdout:9/328: dwrite d4/d3c/f46 [0,4194304] 0 2024-09-10T04:05:13.768 INFO:tasks.workunit.client.0.smithi046.stdout:0/266: link d1/dc/d25/d33/d3/dd/c15 d1/dc/d25/d33/d3/dd/d55/c57 0 2024-09-10T04:05:13.774 INFO:tasks.workunit.client.0.smithi046.stdout:5/405: dwrite d7/d11/d14/f89 [0,4194304] 0 2024-09-10T04:05:13.776 INFO:tasks.workunit.client.0.smithi046.stdout:8/322: dread d1/d29/d17/d1f/d2a/f2d [0,4194304] 0 2024-09-10T04:05:13.777 INFO:tasks.workunit.client.1.smithi070.stdout:8/373: mknod d4/d16/d17/d1a/d23/c83 0 2024-09-10T04:05:13.777 INFO:tasks.workunit.client.0.smithi046.stdout:0/267: truncate d1/dc/d25/d33/d3/d29/f40 520256 0 2024-09-10T04:05:13.778 INFO:tasks.workunit.client.1.smithi070.stdout:7/272: mknod d2/d3/dc/d5d/c5e 0 2024-09-10T04:05:13.781 INFO:tasks.workunit.client.0.smithi046.stdout:9/329: dread d4/d29/f3d [0,4194304] 0 2024-09-10T04:05:13.782 INFO:tasks.workunit.client.0.smithi046.stdout:3/310: truncate f3 4012682 0 2024-09-10T04:05:13.783 INFO:tasks.workunit.client.0.smithi046.stdout:2/376: symlink d2/d6/d23/d5d/d77/l93 0 2024-09-10T04:05:13.784 INFO:tasks.workunit.client.1.smithi070.stdout:4/254: link de/d13/c22 de/d13/d18/d26/d46/c58 0 2024-09-10T04:05:13.788 INFO:tasks.workunit.client.0.smithi046.stdout:8/323: dwrite d1/d29/f4a [0,4194304] 0 2024-09-10T04:05:13.789 INFO:tasks.workunit.client.1.smithi070.stdout:3/356: creat db/d1c/d53/f6e x:0 0 0 2024-09-10T04:05:13.792 INFO:tasks.workunit.client.0.smithi046.stdout:5/406: mkdir d7/d11/d14/d8e/d96 0 2024-09-10T04:05:13.792 INFO:tasks.workunit.client.1.smithi070.stdout:1/278: mknod d3/d8/d12/d2f/d32/d4b/c55 0 2024-09-10T04:05:13.795 INFO:tasks.workunit.client.0.smithi046.stdout:2/377: dwrite d2/d8/f29 [0,4194304] 0 2024-09-10T04:05:13.796 INFO:tasks.workunit.client.0.smithi046.stdout:3/311: dread d12/d1b/f21 [0,4194304] 0 2024-09-10T04:05:13.796 INFO:tasks.workunit.client.0.smithi046.stdout:0/268: unlink d1/dc/f16 0 2024-09-10T04:05:13.796 INFO:tasks.workunit.client.0.smithi046.stdout:2/378: readlink d2/d6/d23/l2b 0 2024-09-10T04:05:13.797 INFO:tasks.workunit.client.1.smithi070.stdout:0/383: creat d8/d13/d1a/d75/f84 x:0 0 0 2024-09-10T04:05:13.798 INFO:tasks.workunit.client.1.smithi070.stdout:0/384: write d8/d24/f43 [982091,33655] 0 2024-09-10T04:05:13.799 INFO:tasks.workunit.client.0.smithi046.stdout:9/330: mknod d4/d29/d65/c68 0 2024-09-10T04:05:13.801 INFO:tasks.workunit.client.1.smithi070.stdout:5/319: mkdir d17/d19/d6a/d54/d6e 0 2024-09-10T04:05:13.805 INFO:tasks.workunit.client.0.smithi046.stdout:3/312: dwrite d12/d13/d14/d1e/d2d/f4f [0,4194304] 0 2024-09-10T04:05:13.807 INFO:tasks.workunit.client.0.smithi046.stdout:3/313: write d12/d13/f2c [2127557,107760] 0 2024-09-10T04:05:13.811 INFO:tasks.workunit.client.1.smithi070.stdout:7/273: rename d2/d7/c36 to d2/d3/dc/d5d/d41/c5f 0 2024-09-10T04:05:13.812 INFO:tasks.workunit.client.1.smithi070.stdout:7/274: write d2/f55 [340861,44795] 0 2024-09-10T04:05:13.813 INFO:tasks.workunit.client.0.smithi046.stdout:5/407: chown d7/d11/d14/d2f/d2b/c31 11 1 2024-09-10T04:05:13.813 INFO:tasks.workunit.client.0.smithi046.stdout:5/408: stat d7/d11/d3f/f80 0 2024-09-10T04:05:13.814 INFO:tasks.workunit.client.1.smithi070.stdout:7/275: write d2/f24 [4356986,122503] 0 2024-09-10T04:05:13.815 INFO:tasks.workunit.client.1.smithi070.stdout:4/255: mkdir de/d13/d18/d23/d47/d59 0 2024-09-10T04:05:13.818 INFO:tasks.workunit.client.1.smithi070.stdout:3/357: mknod db/d1c/d53/c6f 0 2024-09-10T04:05:13.820 INFO:tasks.workunit.client.0.smithi046.stdout:2/379: chown d2/d6/c7a 12955 1 2024-09-10T04:05:13.822 INFO:tasks.workunit.client.0.smithi046.stdout:0/269: creat d1/dc/d25/d33/d4/d19/f58 x:0 0 0 2024-09-10T04:05:13.822 INFO:tasks.workunit.client.1.smithi070.stdout:0/385: mkdir d8/d85 0 2024-09-10T04:05:13.823 INFO:tasks.workunit.client.1.smithi070.stdout:0/386: chown d8/d13/d33/d4f 297683 1 2024-09-10T04:05:13.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/356: getdents d9/d59/d2a 0 2024-09-10T04:05:13.824 INFO:tasks.workunit.client.1.smithi070.stdout:0/387: write d8/d24/d2d/f3a [4374453,115521] 0 2024-09-10T04:05:13.824 INFO:tasks.workunit.client.0.smithi046.stdout:9/331: write d4/d7/d1f/f15 [4667488,21076] 0 2024-09-10T04:05:13.825 INFO:tasks.workunit.client.1.smithi070.stdout:2/292: sync 2024-09-10T04:05:13.825 INFO:tasks.workunit.client.1.smithi070.stdout:6/398: link d0/d3/d18/d2a/l32 d0/d3/d18/d1b/d17/d3f/d40/l80 0 2024-09-10T04:05:13.826 INFO:tasks.workunit.client.1.smithi070.stdout:8/374: mkdir d4/d16/d17/d1a/d60/d7a/d84 0 2024-09-10T04:05:13.828 INFO:tasks.workunit.client.0.smithi046.stdout:3/314: creat d12/d13/d37/f60 x:0 0 0 2024-09-10T04:05:13.829 INFO:tasks.workunit.client.0.smithi046.stdout:3/315: write d12/d13/d14/d1e/d2d/f2e [3776678,22805] 0 2024-09-10T04:05:13.830 INFO:tasks.workunit.client.0.smithi046.stdout:3/316: readlink d12/d13/l3e 0 2024-09-10T04:05:13.831 INFO:tasks.workunit.client.0.smithi046.stdout:8/324: mkdir d1/d29/d19/d36/d6e 0 2024-09-10T04:05:13.832 INFO:tasks.workunit.client.1.smithi070.stdout:4/256: creat de/d13/d18/d23/d39/f5a x:0 0 0 2024-09-10T04:05:13.833 INFO:tasks.workunit.client.1.smithi070.stdout:4/257: chown de/d13/d18/d23/d39/d48/l4c 179196333 1 2024-09-10T04:05:13.833 INFO:tasks.workunit.client.0.smithi046.stdout:5/409: fdatasync d7/d11/d14/f36 0 2024-09-10T04:05:13.837 INFO:tasks.workunit.client.1.smithi070.stdout:1/279: mknod d3/d8/d12/d2f/d32/d4b/d54/c56 0 2024-09-10T04:05:13.839 INFO:tasks.workunit.client.0.smithi046.stdout:9/332: creat d4/d44/f69 x:0 0 0 2024-09-10T04:05:13.839 INFO:tasks.workunit.client.0.smithi046.stdout:9/333: chown d4/d7/c34 182252 1 2024-09-10T04:05:13.840 INFO:tasks.workunit.client.1.smithi070.stdout:6/399: mkdir d0/d3/d1f/d81 0 2024-09-10T04:05:13.841 INFO:tasks.workunit.client.0.smithi046.stdout:9/334: read d4/d18/f25 [2019860,107716] 0 2024-09-10T04:05:13.842 INFO:tasks.workunit.client.1.smithi070.stdout:2/293: unlink d15/d18/d29/d2a/d2e/l40 0 2024-09-10T04:05:13.844 INFO:tasks.workunit.client.0.smithi046.stdout:8/325: mknod d1/d29/d17/c6f 0 2024-09-10T04:05:13.845 INFO:tasks.workunit.client.0.smithi046.stdout:9/335: dread d4/d18/f38 [0,4194304] 0 2024-09-10T04:05:13.845 INFO:tasks.workunit.client.1.smithi070.stdout:8/375: symlink d4/d16/d17/d1a/d23/d44/l85 0 2024-09-10T04:05:13.848 INFO:tasks.workunit.client.0.smithi046.stdout:2/380: rename d2/d6/d15/l36 to d2/d6/d23/l94 0 2024-09-10T04:05:13.849 INFO:tasks.workunit.client.1.smithi070.stdout:4/258: rmdir de/d13/d18/d23/d39 39 2024-09-10T04:05:13.855 INFO:tasks.workunit.client.1.smithi070.stdout:9/357: mknod d9/d59/c6a 0 2024-09-10T04:05:13.858 INFO:tasks.workunit.client.0.smithi046.stdout:5/410: dread d7/d11/f13 [0,4194304] 0 2024-09-10T04:05:13.859 INFO:tasks.workunit.client.1.smithi070.stdout:3/358: write db/f22 [3146404,55295] 0 2024-09-10T04:05:13.860 INFO:tasks.workunit.client.0.smithi046.stdout:5/411: stat d7/d11/d3f 0 2024-09-10T04:05:13.860 INFO:tasks.workunit.client.1.smithi070.stdout:2/294: rmdir d15/d18/d29/d2a 39 2024-09-10T04:05:13.862 INFO:tasks.workunit.client.0.smithi046.stdout:3/317: truncate d12/d13/f2c 615258 0 2024-09-10T04:05:13.862 INFO:tasks.workunit.client.0.smithi046.stdout:8/326: dread d1/d29/d19/f48 [0,4194304] 0 2024-09-10T04:05:13.863 INFO:tasks.workunit.client.0.smithi046.stdout:4/333: sync 2024-09-10T04:05:13.863 INFO:tasks.workunit.client.0.smithi046.stdout:5/412: read d7/d11/d3f/d5a/f69 [641564,64416] 0 2024-09-10T04:05:13.864 INFO:tasks.workunit.client.1.smithi070.stdout:3/359: dwrite db/f12 [0,4194304] 0 2024-09-10T04:05:13.865 INFO:tasks.workunit.client.0.smithi046.stdout:2/381: mkdir d2/d8/d37/d95 0 2024-09-10T04:05:13.867 INFO:tasks.workunit.client.0.smithi046.stdout:4/334: chown d3/d12/d17/d22/f4b 29 1 2024-09-10T04:05:13.867 INFO:tasks.workunit.client.1.smithi070.stdout:5/320: rename d17/d24/d2b/d31 to d17/d6f 0 2024-09-10T04:05:13.869 INFO:tasks.workunit.client.0.smithi046.stdout:9/336: dwrite d4/d18/f38 [0,4194304] 0 2024-09-10T04:05:13.872 INFO:tasks.workunit.client.1.smithi070.stdout:8/376: dread d4/f11 [0,4194304] 0 2024-09-10T04:05:13.874 INFO:tasks.workunit.client.0.smithi046.stdout:8/327: truncate d1/d29/d17/d1f/d2a/f2c 2337598 0 2024-09-10T04:05:13.874 INFO:tasks.workunit.client.1.smithi070.stdout:1/280: symlink d3/d8/l57 0 2024-09-10T04:05:13.875 INFO:tasks.workunit.client.0.smithi046.stdout:2/382: truncate d2/d6/d23/f87 206878 0 2024-09-10T04:05:13.875 INFO:tasks.workunit.client.1.smithi070.stdout:0/388: rename d8/d13/l6c to d8/d13/l86 0 2024-09-10T04:05:13.877 INFO:tasks.workunit.client.1.smithi070.stdout:0/389: write d8/d24/d2d/d34/f5b [523539,11534] 0 2024-09-10T04:05:13.877 INFO:tasks.workunit.client.1.smithi070.stdout:6/400: mknod d0/d3/d18/d1b/d17/d3f/c82 0 2024-09-10T04:05:13.882 INFO:tasks.workunit.client.0.smithi046.stdout:5/413: unlink d7/f1e 0 2024-09-10T04:05:13.884 INFO:tasks.workunit.client.0.smithi046.stdout:0/270: getdents d1/dc/d25/d33/d1a/d41 0 2024-09-10T04:05:13.885 INFO:tasks.workunit.client.0.smithi046.stdout:0/271: chown d1/dc/d49/f4a 13707449 1 2024-09-10T04:05:13.886 INFO:tasks.workunit.client.0.smithi046.stdout:0/272: write d1/dc/d25/d33/d4/d19/f58 [390421,86798] 0 2024-09-10T04:05:13.893 INFO:tasks.workunit.client.1.smithi070.stdout:7/276: rename f1 to d2/d7/d9/d18/d3c/f60 0 2024-09-10T04:05:13.893 INFO:tasks.workunit.client.1.smithi070.stdout:4/259: mkdir de/d13/d5b 0 2024-09-10T04:05:13.894 INFO:tasks.workunit.client.1.smithi070.stdout:8/377: mknod d4/d16/d73/c86 0 2024-09-10T04:05:13.895 INFO:tasks.workunit.client.1.smithi070.stdout:0/390: unlink d8/d13/l3e 0 2024-09-10T04:05:13.895 INFO:tasks.workunit.client.1.smithi070.stdout:2/295: mknod d15/d18/d29/c66 0 2024-09-10T04:05:13.896 INFO:tasks.workunit.client.1.smithi070.stdout:3/360: dread fa [0,4194304] 0 2024-09-10T04:05:13.896 INFO:tasks.workunit.client.1.smithi070.stdout:1/281: dwrite d3/f47 [0,4194304] 0 2024-09-10T04:05:13.897 INFO:tasks.workunit.client.1.smithi070.stdout:3/361: chown db/d1c/d25/d28 8958751 1 2024-09-10T04:05:13.897 INFO:tasks.workunit.client.1.smithi070.stdout:0/391: dread - d8/d13/d33/d4f/f7f zero size 2024-09-10T04:05:13.898 INFO:tasks.workunit.client.0.smithi046.stdout:9/337: creat d4/d3c/d4d/d5c/f6a x:0 0 0 2024-09-10T04:05:13.899 INFO:tasks.workunit.client.1.smithi070.stdout:6/401: write d0/d3/d18/f35 [3540181,26897] 0 2024-09-10T04:05:13.900 INFO:tasks.workunit.client.0.smithi046.stdout:2/383: mknod d2/d8/d37/c96 0 2024-09-10T04:05:13.903 INFO:tasks.workunit.client.1.smithi070.stdout:9/358: rename d9/d1c/d39 to d9/d59/d5d/d6b 0 2024-09-10T04:05:13.904 INFO:tasks.workunit.client.0.smithi046.stdout:9/338: dwrite d4/d7/fe [0,4194304] 0 2024-09-10T04:05:13.904 INFO:tasks.workunit.client.0.smithi046.stdout:9/339: fdatasync d4/d44/f69 0 2024-09-10T04:05:13.905 INFO:tasks.workunit.client.1.smithi070.stdout:4/260: mkdir de/d13/d18/d5c 0 2024-09-10T04:05:13.909 INFO:tasks.workunit.client.0.smithi046.stdout:9/340: dread d4/d3c/f46 [0,4194304] 0 2024-09-10T04:05:13.910 INFO:tasks.workunit.client.1.smithi070.stdout:8/378: readlink d4/d16/d17/d19/d28/d43/d52/l53 0 2024-09-10T04:05:13.910 INFO:tasks.workunit.client.0.smithi046.stdout:6/313: sync 2024-09-10T04:05:13.910 INFO:tasks.workunit.client.0.smithi046.stdout:1/223: sync 2024-09-10T04:05:13.910 INFO:tasks.workunit.client.0.smithi046.stdout:7/353: sync 2024-09-10T04:05:13.911 INFO:tasks.workunit.client.1.smithi070.stdout:1/282: chown d3/d8/l51 17737 1 2024-09-10T04:05:13.912 INFO:tasks.workunit.client.0.smithi046.stdout:7/354: write d5/d9/d43/f66 [3072992,77225] 0 2024-09-10T04:05:13.913 INFO:tasks.workunit.client.1.smithi070.stdout:0/392: fdatasync d8/d24/d36/f48 0 2024-09-10T04:05:13.914 INFO:tasks.workunit.client.0.smithi046.stdout:0/273: chown d1/dc/l2e 1 1 2024-09-10T04:05:13.916 INFO:tasks.workunit.client.0.smithi046.stdout:4/335: link d3/d12/d14/f16 d3/d12/d17/d22/d30/f72 0 2024-09-10T04:05:13.916 INFO:tasks.workunit.client.1.smithi070.stdout:3/362: rmdir db/d1c/d53 39 2024-09-10T04:05:13.918 INFO:tasks.workunit.client.1.smithi070.stdout:6/402: rmdir d0 39 2024-09-10T04:05:13.922 INFO:tasks.workunit.client.1.smithi070.stdout:7/277: mknod d2/d7/d9/d18/d3c/d52/c61 0 2024-09-10T04:05:13.923 INFO:tasks.workunit.client.0.smithi046.stdout:1/224: creat d3/dc/f4b x:0 0 0 2024-09-10T04:05:13.926 INFO:tasks.workunit.client.1.smithi070.stdout:1/283: creat d3/d8/d12/d2f/d32/f58 x:0 0 0 2024-09-10T04:05:13.926 INFO:tasks.workunit.client.1.smithi070.stdout:1/284: stat d3/d8/d12/f24 0 2024-09-10T04:05:13.929 INFO:tasks.workunit.client.1.smithi070.stdout:0/393: write d8/d24/d2d/d34/f77 [811269,31182] 0 2024-09-10T04:05:13.929 INFO:tasks.workunit.client.1.smithi070.stdout:0/394: chown d8 201849 1 2024-09-10T04:05:13.929 INFO:tasks.workunit.client.0.smithi046.stdout:7/355: creat d5/d6/db/dd/d24/d76/f7c x:0 0 0 2024-09-10T04:05:13.931 INFO:tasks.workunit.client.0.smithi046.stdout:7/356: write d5/d9/d12/f6f [296473,83162] 0 2024-09-10T04:05:13.932 INFO:tasks.workunit.client.0.smithi046.stdout:0/274: rmdir d1/dc 39 2024-09-10T04:05:13.933 INFO:tasks.workunit.client.0.smithi046.stdout:8/328: truncate d1/d29/d19/d36/f3c 2576462 0 2024-09-10T04:05:13.934 INFO:tasks.workunit.client.1.smithi070.stdout:0/395: dwrite d8/d24/d2d/d34/f77 [0,4194304] 0 2024-09-10T04:05:13.935 INFO:tasks.workunit.client.1.smithi070.stdout:3/363: symlink db/d1c/d3e/l70 0 2024-09-10T04:05:13.935 INFO:tasks.workunit.client.1.smithi070.stdout:0/396: readlink d8/d24/l76 0 2024-09-10T04:05:13.935 INFO:tasks.workunit.client.1.smithi070.stdout:3/364: readlink db/d1c/d25/l3a 0 2024-09-10T04:05:13.937 INFO:tasks.workunit.client.1.smithi070.stdout:0/397: read d8/d13/d1a/f23 [328558,41464] 0 2024-09-10T04:05:13.937 INFO:tasks.workunit.client.1.smithi070.stdout:0/398: readlink d8/d13/l16 0 2024-09-10T04:05:13.937 INFO:tasks.workunit.client.0.smithi046.stdout:2/384: unlink d2/d6/d15/f6a 0 2024-09-10T04:05:13.938 INFO:tasks.workunit.client.0.smithi046.stdout:8/329: dread d1/d29/f4a [0,4194304] 0 2024-09-10T04:05:13.939 INFO:tasks.workunit.client.1.smithi070.stdout:0/399: dread d8/d13/d33/d4f/f6e [0,4194304] 0 2024-09-10T04:05:13.943 INFO:tasks.workunit.client.0.smithi046.stdout:3/318: getdents d12/d13/d37 0 2024-09-10T04:05:13.946 INFO:tasks.workunit.client.1.smithi070.stdout:2/296: truncate d15/d18/f1c 2520089 0 2024-09-10T04:05:13.947 INFO:tasks.workunit.client.1.smithi070.stdout:2/297: chown d15/d18/f27 26235 1 2024-09-10T04:05:13.947 INFO:tasks.workunit.client.0.smithi046.stdout:7/357: dread d5/d6/d32/f5e [0,4194304] 0 2024-09-10T04:05:13.948 INFO:tasks.workunit.client.1.smithi070.stdout:7/278: rename d2/c12 to d2/d7/d9/d18/d3c/d52/c62 0 2024-09-10T04:05:13.955 INFO:tasks.workunit.client.0.smithi046.stdout:5/414: creat d7/f97 x:0 0 0 2024-09-10T04:05:13.955 INFO:tasks.workunit.client.0.smithi046.stdout:6/314: symlink da/db/d11/d2b/d55/l67 0 2024-09-10T04:05:13.955 INFO:tasks.workunit.client.0.smithi046.stdout:1/225: dwrite d3/d27/f36 [0,4194304] 0 2024-09-10T04:05:13.956 INFO:tasks.workunit.client.1.smithi070.stdout:3/365: creat db/d1c/d25/d28/d43/f71 x:0 0 0 2024-09-10T04:05:13.957 INFO:tasks.workunit.client.0.smithi046.stdout:7/358: dread d5/d9/d12/f6f [0,4194304] 0 2024-09-10T04:05:13.957 INFO:tasks.workunit.client.0.smithi046.stdout:1/226: read - d3/d27/f33 zero size 2024-09-10T04:05:13.959 INFO:tasks.workunit.client.0.smithi046.stdout:7/359: truncate d5/d6/db/f38 5185707 0 2024-09-10T04:05:13.966 INFO:tasks.workunit.client.1.smithi070.stdout:9/359: link d9/d59/d10/c5b d9/d59/d10/c6c 0 2024-09-10T04:05:13.967 INFO:tasks.workunit.client.1.smithi070.stdout:8/379: write d4/f12 [1082135,118752] 0 2024-09-10T04:05:13.967 INFO:tasks.workunit.client.0.smithi046.stdout:3/319: mknod d12/d13/d42/d49/c61 0 2024-09-10T04:05:13.972 INFO:tasks.workunit.client.1.smithi070.stdout:7/279: mkdir d2/d3/d63 0 2024-09-10T04:05:13.973 INFO:tasks.workunit.client.1.smithi070.stdout:9/360: dwrite d9/d59/d10/f19 [0,4194304] 0 2024-09-10T04:05:13.973 INFO:tasks.workunit.client.0.smithi046.stdout:6/315: unlink da/db/d18/l42 0 2024-09-10T04:05:13.975 INFO:tasks.workunit.client.0.smithi046.stdout:6/316: stat da/db/d11/d13/d1e/d23/c4a 0 2024-09-10T04:05:13.976 INFO:tasks.workunit.client.0.smithi046.stdout:4/336: creat d3/d12/f73 x:0 0 0 2024-09-10T04:05:13.978 INFO:tasks.workunit.client.0.smithi046.stdout:5/415: rename d7/d11/d14/d45/f5f to d7/d11/d3f/d5a/d63/f98 0 2024-09-10T04:05:13.980 INFO:tasks.workunit.client.0.smithi046.stdout:0/275: symlink d1/dc/d25/d33/d3/d29/l59 0 2024-09-10T04:05:13.981 INFO:tasks.workunit.client.1.smithi070.stdout:3/366: write db/d1c/d53/f57 [622369,13763] 0 2024-09-10T04:05:13.982 INFO:tasks.workunit.client.1.smithi070.stdout:3/367: chown db/d1c/f4b 0 1 2024-09-10T04:05:13.983 INFO:tasks.workunit.client.1.smithi070.stdout:3/368: chown f7 312559 1 2024-09-10T04:05:13.983 INFO:tasks.workunit.client.1.smithi070.stdout:2/298: dread d15/d2b/f4c [0,4194304] 0 2024-09-10T04:05:13.984 INFO:tasks.workunit.client.1.smithi070.stdout:3/369: dread - db/d1c/d25/d28/d43/f51 zero size 2024-09-10T04:05:13.985 INFO:tasks.workunit.client.0.smithi046.stdout:1/227: symlink d3/dc/d21/d3a/l4c 0 2024-09-10T04:05:13.985 INFO:tasks.workunit.client.1.smithi070.stdout:6/403: symlink d0/l83 0 2024-09-10T04:05:13.986 INFO:tasks.workunit.client.0.smithi046.stdout:7/360: mkdir d5/d6/d55/d7d 0 2024-09-10T04:05:13.987 INFO:tasks.workunit.client.1.smithi070.stdout:4/261: dwrite de/d13/d18/d23/d39/f57 [4194304,4194304] 0 2024-09-10T04:05:13.988 INFO:tasks.workunit.client.0.smithi046.stdout:2/385: unlink d2/f2c 0 2024-09-10T04:05:13.989 INFO:tasks.workunit.client.1.smithi070.stdout:4/262: dread fc [0,4194304] 0 2024-09-10T04:05:13.989 INFO:tasks.workunit.client.1.smithi070.stdout:4/263: truncate de/f34 337596 0 2024-09-10T04:05:13.992 INFO:tasks.workunit.client.1.smithi070.stdout:8/380: rmdir d4/d16/d17/d1a/d23/d44 39 2024-09-10T04:05:13.993 INFO:tasks.workunit.client.1.smithi070.stdout:8/381: fsync d4/d16/d17/d19/d28/d43/d55/f82 0 2024-09-10T04:05:13.994 INFO:tasks.workunit.client.1.smithi070.stdout:8/382: truncate d4/d16/d17/d5c/f6d 1698089 0 2024-09-10T04:05:13.998 INFO:tasks.workunit.client.1.smithi070.stdout:0/400: truncate f5 484718 0 2024-09-10T04:05:14.000 INFO:tasks.workunit.client.0.smithi046.stdout:4/337: rename d3/d12/d17/d22/d51/l57 to d3/d12/d17/d29/l74 0 2024-09-10T04:05:14.001 INFO:tasks.workunit.client.0.smithi046.stdout:5/416: mkdir d7/d11/d3f/d53/d99 0 2024-09-10T04:05:14.002 INFO:tasks.workunit.client.0.smithi046.stdout:4/338: truncate d3/d12/d17/d29/f38 1531386 0 2024-09-10T04:05:14.003 INFO:tasks.workunit.client.0.smithi046.stdout:5/417: truncate d7/d11/d14/d50/f62 866059 0 2024-09-10T04:05:14.006 INFO:tasks.workunit.client.1.smithi070.stdout:3/370: creat db/d1c/d25/d28/f72 x:0 0 0 2024-09-10T04:05:14.009 INFO:tasks.workunit.client.1.smithi070.stdout:6/404: symlink d0/d3/l84 0 2024-09-10T04:05:14.009 INFO:tasks.workunit.client.1.smithi070.stdout:9/361: dread d9/d59/d5d/d6b/f51 [0,4194304] 0 2024-09-10T04:05:14.009 INFO:tasks.workunit.client.1.smithi070.stdout:7/280: dwrite d2/d3/dc/d5d/f34 [0,4194304] 0 2024-09-10T04:05:14.010 INFO:tasks.workunit.client.1.smithi070.stdout:9/362: dread - d9/d59/d10/d1b/f5c zero size 2024-09-10T04:05:14.011 INFO:tasks.workunit.client.1.smithi070.stdout:9/363: dread - d9/f68 zero size 2024-09-10T04:05:14.011 INFO:tasks.workunit.client.1.smithi070.stdout:7/281: chown d2/d3/dc/d5d/d2d/c4d 243438310 1 2024-09-10T04:05:14.011 INFO:tasks.workunit.client.0.smithi046.stdout:0/276: write d1/dc/d25/d33/d4/f36 [4880379,37205] 0 2024-09-10T04:05:14.012 INFO:tasks.workunit.client.1.smithi070.stdout:5/321: sync 2024-09-10T04:05:14.012 INFO:tasks.workunit.client.0.smithi046.stdout:3/320: creat d12/d13/d37/d51/f62 x:0 0 0 2024-09-10T04:05:14.012 INFO:tasks.workunit.client.1.smithi070.stdout:4/264: mknod de/d13/d43/c5d 0 2024-09-10T04:05:14.015 INFO:tasks.workunit.client.1.smithi070.stdout:8/383: mkdir d4/d16/d17/d5c/d87 0 2024-09-10T04:05:14.016 INFO:tasks.workunit.client.0.smithi046.stdout:6/317: mknod da/db/d11/d13/c68 0 2024-09-10T04:05:14.017 INFO:tasks.workunit.client.0.smithi046.stdout:6/318: write da/fe [2606377,53849] 0 2024-09-10T04:05:14.018 INFO:tasks.workunit.client.1.smithi070.stdout:0/401: mkdir d8/d24/d36/d87 0 2024-09-10T04:05:14.019 INFO:tasks.workunit.client.1.smithi070.stdout:5/322: dwrite f10 [0,4194304] 0 2024-09-10T04:05:14.019 INFO:tasks.workunit.client.1.smithi070.stdout:1/285: getdents d3/d8/d27 0 2024-09-10T04:05:14.020 INFO:tasks.workunit.client.1.smithi070.stdout:0/402: dread d8/d13/d33/f65 [0,4194304] 0 2024-09-10T04:05:14.023 INFO:tasks.workunit.client.1.smithi070.stdout:2/299: creat d15/d18/d29/d2a/f67 x:0 0 0 2024-09-10T04:05:14.025 INFO:tasks.workunit.client.1.smithi070.stdout:3/371: fdatasync fa 0 2024-09-10T04:05:14.027 INFO:tasks.workunit.client.1.smithi070.stdout:5/323: dwrite f16 [4194304,4194304] 0 2024-09-10T04:05:14.028 INFO:tasks.workunit.client.1.smithi070.stdout:6/405: unlink d0/l78 0 2024-09-10T04:05:14.028 INFO:tasks.workunit.client.1.smithi070.stdout:3/372: chown db/d1c/d3e/f59 278916796 1 2024-09-10T04:05:14.029 INFO:tasks.workunit.client.1.smithi070.stdout:0/403: dwrite d8/f11 [0,4194304] 0 2024-09-10T04:05:14.029 INFO:tasks.workunit.client.0.smithi046.stdout:8/330: getdents d1/d29/d19/d36/d4c 0 2024-09-10T04:05:14.033 INFO:tasks.workunit.client.0.smithi046.stdout:0/277: creat d1/dc/d25/d33/d1a/d41/f5a x:0 0 0 2024-09-10T04:05:14.040 INFO:tasks.workunit.client.1.smithi070.stdout:1/286: unlink d3/d8/d13/c36 0 2024-09-10T04:05:14.040 INFO:tasks.workunit.client.1.smithi070.stdout:1/287: readlink d3/la 0 2024-09-10T04:05:14.041 INFO:tasks.workunit.client.0.smithi046.stdout:5/418: creat d7/d11/d14/d2f/d2b/d92/f9a x:0 0 0 2024-09-10T04:05:14.043 INFO:tasks.workunit.client.0.smithi046.stdout:5/419: write d7/d11/d3f/d5a/d63/f88 [2358343,51286] 0 2024-09-10T04:05:14.045 INFO:tasks.workunit.client.1.smithi070.stdout:5/324: symlink d17/d19/d35/l70 0 2024-09-10T04:05:14.045 INFO:tasks.workunit.client.1.smithi070.stdout:5/325: chown d17/d19/f1d 5228 1 2024-09-10T04:05:14.048 INFO:tasks.workunit.client.0.smithi046.stdout:1/228: rename d3/d27/f36 to d3/f4d 0 2024-09-10T04:05:14.048 INFO:tasks.workunit.client.1.smithi070.stdout:3/373: stat db/d1c/c42 0 2024-09-10T04:05:14.049 INFO:tasks.workunit.client.0.smithi046.stdout:6/319: mkdir da/db/d11/d13/d1e/d69 0 2024-09-10T04:05:14.050 INFO:tasks.workunit.client.0.smithi046.stdout:8/331: dread d1/d29/d12/f32 [0,4194304] 0 2024-09-10T04:05:14.050 INFO:tasks.workunit.client.0.smithi046.stdout:6/320: chown da/c20 1540238 1 2024-09-10T04:05:14.050 INFO:tasks.workunit.client.1.smithi070.stdout:5/326: dwrite d17/d24/f27 [0,4194304] 0 2024-09-10T04:05:14.051 INFO:tasks.workunit.client.0.smithi046.stdout:8/332: fdatasync d1/d29/d12/f6b 0 2024-09-10T04:05:14.051 INFO:tasks.workunit.client.1.smithi070.stdout:9/364: mknod d9/c6d 0 2024-09-10T04:05:14.052 INFO:tasks.workunit.client.1.smithi070.stdout:4/265: symlink de/d13/d18/d23/l5e 0 2024-09-10T04:05:14.052 INFO:tasks.workunit.client.1.smithi070.stdout:4/266: readlink de/d13/l1d 0 2024-09-10T04:05:14.053 INFO:tasks.workunit.client.0.smithi046.stdout:7/361: getdents d5/d6/d55 0 2024-09-10T04:05:14.053 INFO:tasks.workunit.client.1.smithi070.stdout:4/267: fdatasync de/f34 0 2024-09-10T04:05:14.054 INFO:tasks.workunit.client.0.smithi046.stdout:6/321: dread da/f40 [0,4194304] 0 2024-09-10T04:05:14.056 INFO:tasks.workunit.client.0.smithi046.stdout:2/386: link d2/d6/d23/d5d/d77/d7b/d85/c8c d2/d6/d23/d5d/d77/d7b/c97 0 2024-09-10T04:05:14.058 INFO:tasks.workunit.client.1.smithi070.stdout:2/300: symlink d15/d18/l68 0 2024-09-10T04:05:14.059 INFO:tasks.workunit.client.0.smithi046.stdout:5/420: creat d7/d11/d3f/d53/f9b x:0 0 0 2024-09-10T04:05:14.060 INFO:tasks.workunit.client.1.smithi070.stdout:1/288: write d3/d8/d13/f18 [2541117,14905] 0 2024-09-10T04:05:14.060 INFO:tasks.workunit.client.0.smithi046.stdout:3/321: creat d12/f63 x:0 0 0 2024-09-10T04:05:14.065 INFO:tasks.workunit.client.1.smithi070.stdout:9/365: stat d9/d59/lf 0 2024-09-10T04:05:14.066 INFO:tasks.workunit.client.1.smithi070.stdout:5/327: stat d17/d24/d45/l4c 0 2024-09-10T04:05:14.068 INFO:tasks.workunit.client.0.smithi046.stdout:4/339: getdents d3/d12/d17/d22 0 2024-09-10T04:05:14.069 INFO:tasks.workunit.client.0.smithi046.stdout:4/340: fsync d3/d12/d17/d29/f3e 0 2024-09-10T04:05:14.069 INFO:tasks.workunit.client.0.smithi046.stdout:4/341: chown d3/d12/f19 1356 1 2024-09-10T04:05:14.072 INFO:tasks.workunit.client.1.smithi070.stdout:8/384: creat d4/d16/d17/d1a/f88 x:0 0 0 2024-09-10T04:05:14.073 INFO:tasks.workunit.client.0.smithi046.stdout:6/322: write da/db/d10/f1a [1804565,50885] 0 2024-09-10T04:05:14.073 INFO:tasks.workunit.client.1.smithi070.stdout:8/385: dread d4/f77 [0,4194304] 0 2024-09-10T04:05:14.074 INFO:tasks.workunit.client.0.smithi046.stdout:8/333: write d1/d29/d17/f69 [63244,127161] 0 2024-09-10T04:05:14.076 INFO:tasks.workunit.client.1.smithi070.stdout:7/282: rename d2/d3/dc/d5d/d42/c56 to d2/c64 0 2024-09-10T04:05:14.077 INFO:tasks.workunit.client.0.smithi046.stdout:5/421: mkdir d7/d11/d14/d50/d9c 0 2024-09-10T04:05:14.079 INFO:tasks.workunit.client.0.smithi046.stdout:3/322: symlink d12/d13/d37/l64 0 2024-09-10T04:05:14.080 INFO:tasks.workunit.client.1.smithi070.stdout:6/406: creat d0/d3/d18/d62/f85 x:0 0 0 2024-09-10T04:05:14.081 INFO:tasks.workunit.client.1.smithi070.stdout:6/407: chown d0/d3/d18/d1b/d17/l63 0 1 2024-09-10T04:05:14.082 INFO:tasks.workunit.client.1.smithi070.stdout:6/408: dread - d0/d3/d18/d62/d70/f7e zero size 2024-09-10T04:05:14.082 INFO:tasks.workunit.client.1.smithi070.stdout:3/374: truncate db/d1c/d25/d28/f37 7990 0 2024-09-10T04:05:14.082 INFO:tasks.workunit.client.1.smithi070.stdout:6/409: write d0/f57 [964745,62770] 0 2024-09-10T04:05:14.084 INFO:tasks.workunit.client.0.smithi046.stdout:5/422: dwrite d7/d11/d14/d45/f4b [0,4194304] 0 2024-09-10T04:05:14.085 INFO:tasks.workunit.client.0.smithi046.stdout:5/423: rename d7 to d7/d11/d14/d2f/d2b/d6a/d9d 22 2024-09-10T04:05:14.086 INFO:tasks.workunit.client.1.smithi070.stdout:9/366: symlink d9/d3e/l6e 0 2024-09-10T04:05:14.090 INFO:tasks.workunit.client.1.smithi070.stdout:1/289: dwrite d3/d8/d13/f1b [0,4194304] 0 2024-09-10T04:05:14.091 INFO:tasks.workunit.client.1.smithi070.stdout:5/328: symlink d17/d19/d6a/d54/l71 0 2024-09-10T04:05:14.091 INFO:tasks.workunit.client.1.smithi070.stdout:5/329: chown d17/d19 0 1 2024-09-10T04:05:14.092 INFO:tasks.workunit.client.1.smithi070.stdout:5/330: readlink d17/d19/l29 0 2024-09-10T04:05:14.093 INFO:tasks.workunit.client.0.smithi046.stdout:1/229: truncate d3/dc/f10 3865559 0 2024-09-10T04:05:14.094 INFO:tasks.workunit.client.0.smithi046.stdout:3/323: dread d12/d13/d14/d1e/f3f [0,4194304] 0 2024-09-10T04:05:14.095 INFO:tasks.workunit.client.1.smithi070.stdout:4/268: symlink de/d13/d18/d23/d47/d59/l5f 0 2024-09-10T04:05:14.096 INFO:tasks.workunit.client.1.smithi070.stdout:4/269: chown de/f11 12 1 2024-09-10T04:05:14.096 INFO:tasks.workunit.client.1.smithi070.stdout:4/270: chown de/f34 79783311 1 2024-09-10T04:05:14.097 INFO:tasks.workunit.client.0.smithi046.stdout:7/362: symlink d5/d6/d62/l7e 0 2024-09-10T04:05:14.098 INFO:tasks.workunit.client.0.smithi046.stdout:6/323: unlink l5 0 2024-09-10T04:05:14.101 INFO:tasks.workunit.client.0.smithi046.stdout:6/324: write da/db/d10/f17 [10727984,15011] 0 2024-09-10T04:05:14.101 INFO:tasks.workunit.client.0.smithi046.stdout:8/334: symlink d1/d29/d12/l70 0 2024-09-10T04:05:14.104 INFO:tasks.workunit.client.1.smithi070.stdout:3/375: creat db/d1c/d25/d28/f73 x:0 0 0 2024-09-10T04:05:14.106 INFO:tasks.workunit.client.0.smithi046.stdout:5/424: rename d7/d11/f78 to d7/d11/d3f/d53/f9e 0 2024-09-10T04:05:14.109 INFO:tasks.workunit.client.0.smithi046.stdout:6/325: dwrite da/db/d18/f35 [0,4194304] 0 2024-09-10T04:05:14.116 INFO:tasks.workunit.client.1.smithi070.stdout:1/290: write d3/d8/d13/d3f/f53 [315747,62065] 0 2024-09-10T04:05:14.117 INFO:tasks.workunit.client.1.smithi070.stdout:1/291: read d3/f47 [1667390,76730] 0 2024-09-10T04:05:14.117 INFO:tasks.workunit.client.0.smithi046.stdout:4/342: link d3/d12/d14/f2b d3/d12/d17/d29/d58/f75 0 2024-09-10T04:05:14.118 INFO:tasks.workunit.client.1.smithi070.stdout:8/386: write d4/f11 [1360024,11630] 0 2024-09-10T04:05:14.121 INFO:tasks.workunit.client.1.smithi070.stdout:5/331: creat d17/d19/d35/f72 x:0 0 0 2024-09-10T04:05:14.122 INFO:tasks.workunit.client.0.smithi046.stdout:2/387: link d2/d6/d23/d5d/d5e/l5c d2/d6/d23/d5d/d77/d7b/d85/d8d/l98 0 2024-09-10T04:05:14.124 INFO:tasks.workunit.client.1.smithi070.stdout:4/271: dread de/d13/d18/d26/f42 [0,4194304] 0 2024-09-10T04:05:14.127 INFO:tasks.workunit.client.1.smithi070.stdout:7/283: mknod d2/d3/d63/c65 0 2024-09-10T04:05:14.128 INFO:tasks.workunit.client.1.smithi070.stdout:9/367: write d9/d59/d5d/d6b/f51 [2955877,82248] 0 2024-09-10T04:05:14.130 INFO:tasks.workunit.client.1.smithi070.stdout:9/368: truncate d9/d59/d10/d1b/f5c 727669 0 2024-09-10T04:05:14.134 INFO:tasks.workunit.client.1.smithi070.stdout:3/376: unlink db/d1c/d25/d28/d43/f5a 0 2024-09-10T04:05:14.135 INFO:tasks.workunit.client.1.smithi070.stdout:3/377: chown db/f12 91433 1 2024-09-10T04:05:14.135 INFO:tasks.workunit.client.1.smithi070.stdout:9/369: dread d9/d59/d10/f19 [0,4194304] 0 2024-09-10T04:05:14.136 INFO:tasks.workunit.client.1.smithi070.stdout:9/370: stat d9/d59/c6a 0 2024-09-10T04:05:14.136 INFO:tasks.workunit.client.1.smithi070.stdout:5/332: dread d17/d24/f2d [0,4194304] 0 2024-09-10T04:05:14.142 INFO:tasks.workunit.client.0.smithi046.stdout:3/324: creat d12/d1b/d50/d52/f65 x:0 0 0 2024-09-10T04:05:14.142 INFO:tasks.workunit.client.0.smithi046.stdout:3/325: read - d12/d13/d37/d51/f62 zero size 2024-09-10T04:05:14.143 INFO:tasks.workunit.client.0.smithi046.stdout:4/343: creat d3/d12/d17/d22/d51/d5c/f76 x:0 0 0 2024-09-10T04:05:14.144 INFO:tasks.workunit.client.0.smithi046.stdout:4/344: chown d3/d12/d17/d22/f4b 1959125 1 2024-09-10T04:05:14.145 INFO:tasks.workunit.client.0.smithi046.stdout:4/345: chown d3/d12/l3d 177303704 1 2024-09-10T04:05:14.147 INFO:tasks.workunit.client.0.smithi046.stdout:8/335: creat d1/d29/d19/d36/d62/f71 x:0 0 0 2024-09-10T04:05:14.152 INFO:tasks.workunit.client.0.smithi046.stdout:1/230: truncate f1 4026196 0 2024-09-10T04:05:14.153 INFO:tasks.workunit.client.0.smithi046.stdout:3/326: read f3 [3433575,58263] 0 2024-09-10T04:05:14.153 INFO:tasks.workunit.client.0.smithi046.stdout:1/231: dread - d3/f4a zero size 2024-09-10T04:05:14.154 INFO:tasks.workunit.client.1.smithi070.stdout:0/404: rename d8/d24/d2d/d34/f4e to d8/d24/d2d/f88 0 2024-09-10T04:05:14.155 INFO:tasks.workunit.client.0.smithi046.stdout:4/346: mknod d3/d12/d17/d29/d58/c77 0 2024-09-10T04:05:14.155 INFO:tasks.workunit.client.0.smithi046.stdout:5/425: dwrite d7/d11/d3f/d5a/f69 [0,4194304] 0 2024-09-10T04:05:14.157 INFO:tasks.workunit.client.1.smithi070.stdout:7/284: unlink d2/d7/l2e 0 2024-09-10T04:05:14.158 INFO:tasks.workunit.client.0.smithi046.stdout:2/388: mknod d2/d8/c99 0 2024-09-10T04:05:14.160 INFO:tasks.workunit.client.0.smithi046.stdout:2/389: chown d2/d6/d23/d5d/d77/d7b/d85 13530364 1 2024-09-10T04:05:14.162 INFO:tasks.workunit.client.0.smithi046.stdout:8/336: mknod d1/d29/d17/d1f/c72 0 2024-09-10T04:05:14.163 INFO:tasks.workunit.client.0.smithi046.stdout:7/363: truncate d5/f60 98349 0 2024-09-10T04:05:14.165 INFO:tasks.workunit.client.0.smithi046.stdout:6/326: creat da/db/d11/d13/d1e/f6a x:0 0 0 2024-09-10T04:05:14.169 INFO:tasks.workunit.client.1.smithi070.stdout:3/378: creat db/f74 x:0 0 0 2024-09-10T04:05:14.172 INFO:tasks.workunit.client.1.smithi070.stdout:5/333: creat d17/d37/f73 x:0 0 0 2024-09-10T04:05:14.172 INFO:tasks.workunit.client.1.smithi070.stdout:0/405: dread d8/d24/d2d/d34/f69 [0,4194304] 0 2024-09-10T04:05:14.173 INFO:tasks.workunit.client.1.smithi070.stdout:0/406: dread - d8/d13/f3d zero size 2024-09-10T04:05:14.173 INFO:tasks.workunit.client.1.smithi070.stdout:0/407: chown d8/d13/d33/d7d 923 1 2024-09-10T04:05:14.174 INFO:tasks.workunit.client.0.smithi046.stdout:5/426: unlink d7/d11/d14/d2f/l26 0 2024-09-10T04:05:14.174 INFO:tasks.workunit.client.1.smithi070.stdout:0/408: fsync d8/d24/d2d/d34/f77 0 2024-09-10T04:05:14.175 INFO:tasks.workunit.client.1.smithi070.stdout:9/371: mkdir d9/d59/d5d/d6b/d6f 0 2024-09-10T04:05:14.175 INFO:tasks.workunit.client.1.smithi070.stdout:0/409: dread - d8/d13/d47/f5c zero size 2024-09-10T04:05:14.176 INFO:tasks.workunit.client.0.smithi046.stdout:4/347: symlink d3/d12/d17/d22/d51/d5c/l78 0 2024-09-10T04:05:14.177 INFO:tasks.workunit.client.1.smithi070.stdout:6/410: truncate d0/d3/d1f/f46 663748 0 2024-09-10T04:05:14.178 INFO:tasks.workunit.client.1.smithi070.stdout:6/411: write d0/d3/d18/f4c [1136210,59028] 0 2024-09-10T04:05:14.178 INFO:tasks.workunit.client.1.smithi070.stdout:8/387: mknod d4/c89 0 2024-09-10T04:05:14.180 INFO:tasks.workunit.client.1.smithi070.stdout:9/372: dwrite d9/d1c/f55 [0,4194304] 0 2024-09-10T04:05:14.181 INFO:tasks.workunit.client.1.smithi070.stdout:9/373: chown d9/d59 39 1 2024-09-10T04:05:14.181 INFO:tasks.workunit.client.1.smithi070.stdout:9/374: fdatasync d9/f35 0 2024-09-10T04:05:14.185 INFO:tasks.workunit.client.0.smithi046.stdout:7/364: chown d5/d6/db/dd/d24/c6a 926668289 1 2024-09-10T04:05:14.187 INFO:tasks.workunit.client.0.smithi046.stdout:7/365: write d5/d6/db/d11/f4e [3424939,90106] 0 2024-09-10T04:05:14.188 INFO:tasks.workunit.client.1.smithi070.stdout:9/375: dwrite d9/d59/d10/f3d [4194304,4194304] 0 2024-09-10T04:05:14.189 INFO:tasks.workunit.client.0.smithi046.stdout:3/327: write d12/d13/f18 [451918,72505] 0 2024-09-10T04:05:14.192 INFO:tasks.workunit.client.0.smithi046.stdout:4/348: dread d3/d12/d14/f16 [0,4194304] 0 2024-09-10T04:05:14.193 INFO:tasks.workunit.client.0.smithi046.stdout:1/232: symlink d3/dc/d34/l4e 0 2024-09-10T04:05:14.195 INFO:tasks.workunit.client.1.smithi070.stdout:7/285: fdatasync d2/d7/d9/d18/d23/f5a 0 2024-09-10T04:05:14.195 INFO:tasks.workunit.client.0.smithi046.stdout:1/233: dread - d3/d27/d32/f43 zero size 2024-09-10T04:05:14.198 INFO:tasks.workunit.client.0.smithi046.stdout:1/234: write d3/f6 [862718,88326] 0 2024-09-10T04:05:14.199 INFO:tasks.workunit.client.0.smithi046.stdout:5/427: chown d7/fe 109206214 1 2024-09-10T04:05:14.202 INFO:tasks.workunit.client.1.smithi070.stdout:7/286: dwrite d2/f24 [0,4194304] 0 2024-09-10T04:05:14.202 INFO:tasks.workunit.client.0.smithi046.stdout:4/349: dwrite d3/f9 [0,4194304] 0 2024-09-10T04:05:14.202 INFO:tasks.workunit.client.1.smithi070.stdout:7/287: stat d2/d40/l59 0 2024-09-10T04:05:14.206 INFO:tasks.workunit.client.1.smithi070.stdout:8/388: dread d4/f29 [0,4194304] 0 2024-09-10T04:05:14.209 INFO:tasks.workunit.client.1.smithi070.stdout:0/410: symlink d8/d24/d2d/d34/l89 0 2024-09-10T04:05:14.212 INFO:tasks.workunit.client.0.smithi046.stdout:7/366: rename d5/d6/d23 to d5/d7f 0 2024-09-10T04:05:14.213 INFO:tasks.workunit.client.1.smithi070.stdout:1/292: dwrite d3/d8/d12/d2f/f34 [0,4194304] 0 2024-09-10T04:05:14.213 INFO:tasks.workunit.client.0.smithi046.stdout:9/341: sync 2024-09-10T04:05:14.214 INFO:tasks.workunit.client.0.smithi046.stdout:8/337: dwrite d1/d29/d19/f48 [0,4194304] 0 2024-09-10T04:05:14.217 INFO:tasks.workunit.client.0.smithi046.stdout:7/367: read d5/d9/d43/f66 [1674770,53207] 0 2024-09-10T04:05:14.217 INFO:tasks.workunit.client.0.smithi046.stdout:3/328: mknod d12/d13/d42/c66 0 2024-09-10T04:05:14.219 INFO:tasks.workunit.client.0.smithi046.stdout:6/327: dwrite da/f24 [0,4194304] 0 2024-09-10T04:05:14.226 INFO:tasks.workunit.client.0.smithi046.stdout:1/235: unlink d3/d27/c1e 0 2024-09-10T04:05:14.228 INFO:tasks.workunit.client.0.smithi046.stdout:0/278: sync 2024-09-10T04:05:14.228 INFO:tasks.workunit.client.0.smithi046.stdout:5/428: creat d7/d11/d3f/d53/d75/f9f x:0 0 0 2024-09-10T04:05:14.230 INFO:tasks.workunit.client.0.smithi046.stdout:0/279: chown d1/dc/d25/d33/d3/le 2614 1 2024-09-10T04:05:14.230 INFO:tasks.workunit.client.0.smithi046.stdout:4/350: mknod d3/d12/d17/d29/c79 0 2024-09-10T04:05:14.231 INFO:tasks.workunit.client.0.smithi046.stdout:4/351: chown d3/d12/f61 1746986 1 2024-09-10T04:05:14.235 INFO:tasks.workunit.client.1.smithi070.stdout:1/293: unlink d3/d8/d27/l3c 0 2024-09-10T04:05:14.237 INFO:tasks.workunit.client.1.smithi070.stdout:6/412: unlink d0/d3/d18/d1b/d17/d3f/d40/c66 0 2024-09-10T04:05:14.240 INFO:tasks.workunit.client.0.smithi046.stdout:9/342: rename d4/d3c/f46 to d4/d3c/f6b 0 2024-09-10T04:05:14.241 INFO:tasks.workunit.client.0.smithi046.stdout:7/368: dread d5/d9/f5b [0,4194304] 0 2024-09-10T04:05:14.242 INFO:tasks.workunit.client.1.smithi070.stdout:9/376: symlink d9/d59/d10/d3c/d62/l70 0 2024-09-10T04:05:14.244 INFO:tasks.workunit.client.1.smithi070.stdout:2/301: rename d15/d18/d29/f2d to d15/d18/d42/f69 0 2024-09-10T04:05:14.244 INFO:tasks.workunit.client.0.smithi046.stdout:3/329: rmdir d12/d13/d14/d1e/d2d 39 2024-09-10T04:05:14.245 INFO:tasks.workunit.client.1.smithi070.stdout:7/288: mkdir d2/d66 0 2024-09-10T04:05:14.247 INFO:tasks.workunit.client.1.smithi070.stdout:8/389: mknod d4/c8a 0 2024-09-10T04:05:14.248 INFO:tasks.workunit.client.0.smithi046.stdout:2/390: getdents d2/d6/d23/d5d/d5e 0 2024-09-10T04:05:14.249 INFO:tasks.workunit.client.1.smithi070.stdout:5/334: truncate d17/d19/f2e 740419 0 2024-09-10T04:05:14.249 INFO:tasks.workunit.client.1.smithi070.stdout:5/335: read - d17/d19/f50 zero size 2024-09-10T04:05:14.249 INFO:tasks.workunit.client.1.smithi070.stdout:1/294: mknod d3/d10/d1d/c59 0 2024-09-10T04:05:14.250 INFO:tasks.workunit.client.1.smithi070.stdout:1/295: readlink d3/d8/d12/l39 0 2024-09-10T04:05:14.252 INFO:tasks.workunit.client.1.smithi070.stdout:6/413: read d0/d3/d18/d1b/d17/f47 [23439,122098] 0 2024-09-10T04:05:14.252 INFO:tasks.workunit.client.0.smithi046.stdout:0/280: rename d1/dc/d25/d33/d3/dd/f39 to d1/dc/d25/d33/d4/f5b 0 2024-09-10T04:05:14.253 INFO:tasks.workunit.client.1.smithi070.stdout:0/411: write d8/d13/d1a/f7e [31026,17551] 0 2024-09-10T04:05:14.255 INFO:tasks.workunit.client.0.smithi046.stdout:4/352: mkdir d3/d43/d7a 0 2024-09-10T04:05:14.257 INFO:tasks.workunit.client.0.smithi046.stdout:8/338: mkdir d1/d29/d19/d73 0 2024-09-10T04:05:14.258 INFO:tasks.workunit.client.0.smithi046.stdout:8/339: readlink d1/d29/d19/l3d 0 2024-09-10T04:05:14.259 INFO:tasks.workunit.client.1.smithi070.stdout:8/390: unlink l2 0 2024-09-10T04:05:14.259 INFO:tasks.workunit.client.0.smithi046.stdout:0/281: dwrite d1/dc/d25/d33/d3/f14 [0,4194304] 0 2024-09-10T04:05:14.261 INFO:tasks.workunit.client.0.smithi046.stdout:7/369: unlink d5/d9/d12/f6f 0 2024-09-10T04:05:14.261 INFO:tasks.workunit.client.0.smithi046.stdout:0/282: dread - d1/dc/d25/d33/d1a/d41/f5a zero size 2024-09-10T04:05:14.263 INFO:tasks.workunit.client.1.smithi070.stdout:5/336: unlink d17/d19/l29 0 2024-09-10T04:05:14.263 INFO:tasks.workunit.client.1.smithi070.stdout:5/337: stat d17/d24/d2b 0 2024-09-10T04:05:14.263 INFO:tasks.workunit.client.0.smithi046.stdout:6/328: symlink da/db/d10/d51/l6b 0 2024-09-10T04:05:14.265 INFO:tasks.workunit.client.1.smithi070.stdout:6/414: rmdir d0/d3/d18/d2a 39 2024-09-10T04:05:14.267 INFO:tasks.workunit.client.0.smithi046.stdout:5/429: creat d7/d11/d3f/d53/d99/fa0 x:0 0 0 2024-09-10T04:05:14.270 INFO:tasks.workunit.client.1.smithi070.stdout:2/302: mknod d15/d59/c6a 0 2024-09-10T04:05:14.275 INFO:tasks.workunit.client.0.smithi046.stdout:8/340: unlink d1/d29/d12/f32 0 2024-09-10T04:05:14.277 INFO:tasks.workunit.client.0.smithi046.stdout:7/370: symlink d5/d6/d39/d6c/l80 0 2024-09-10T04:05:14.281 INFO:tasks.workunit.client.1.smithi070.stdout:8/391: dread d4/d16/d17/d1a/d60/d7a/f7b [0,4194304] 0 2024-09-10T04:05:14.281 INFO:tasks.workunit.client.1.smithi070.stdout:8/392: readlink d4/d16/d17/d19/d28/d43/d55/l76 0 2024-09-10T04:05:14.281 INFO:tasks.workunit.client.0.smithi046.stdout:7/371: read - d5/d6/d39/d6c/f75 zero size 2024-09-10T04:05:14.281 INFO:tasks.workunit.client.0.smithi046.stdout:7/372: read d5/d6/f1c [177016,89387] 0 2024-09-10T04:05:14.281 INFO:tasks.workunit.client.0.smithi046.stdout:0/283: mknod d1/dc/d25/d33/d3/dd/d55/d4d/c5c 0 2024-09-10T04:05:14.281 INFO:tasks.workunit.client.0.smithi046.stdout:6/329: rename da/c20 to da/db/d63/c6c 0 2024-09-10T04:05:14.283 INFO:tasks.workunit.client.1.smithi070.stdout:5/338: creat d17/d37/f74 x:0 0 0 2024-09-10T04:05:14.287 INFO:tasks.workunit.client.0.smithi046.stdout:5/430: creat d7/d11/d14/d2f/d2b/d6a/fa1 x:0 0 0 2024-09-10T04:05:14.289 INFO:tasks.workunit.client.0.smithi046.stdout:5/431: read d7/d11/d14/d2f/d2b/f7a [1895178,52685] 0 2024-09-10T04:05:14.290 INFO:tasks.workunit.client.0.smithi046.stdout:5/432: truncate d7/d11/d3f/d53/d75/f7c 671887 0 2024-09-10T04:05:14.291 INFO:tasks.workunit.client.1.smithi070.stdout:7/289: link d2/d7/db/l2c d2/d3/d63/l67 0 2024-09-10T04:05:14.291 INFO:tasks.workunit.client.1.smithi070.stdout:7/290: fsync d2/f24 0 2024-09-10T04:05:14.293 INFO:tasks.workunit.client.1.smithi070.stdout:8/393: rmdir d4/d16/d17/d1a/d60 39 2024-09-10T04:05:14.296 INFO:tasks.workunit.client.0.smithi046.stdout:0/284: symlink d1/dc/d25/d33/d3/d29/l5d 0 2024-09-10T04:05:14.296 INFO:tasks.workunit.client.1.smithi070.stdout:5/339: dread d17/d24/f2d [0,4194304] 0 2024-09-10T04:05:14.298 INFO:tasks.workunit.client.0.smithi046.stdout:6/330: creat da/db/d11/d2b/f6d x:0 0 0 2024-09-10T04:05:14.299 INFO:tasks.workunit.client.0.smithi046.stdout:7/373: dwrite d5/d6/d39/d5a/f79 [0,4194304] 0 2024-09-10T04:05:14.299 INFO:tasks.workunit.client.1.smithi070.stdout:9/377: truncate d9/d59/f18 3272697 0 2024-09-10T04:05:14.299 INFO:tasks.workunit.client.1.smithi070.stdout:2/303: getdents d15/d18/d29/d64 0 2024-09-10T04:05:14.300 INFO:tasks.workunit.client.0.smithi046.stdout:0/285: dread - d1/dc/d25/d33/d3/dd/d55/f43 zero size 2024-09-10T04:05:14.303 INFO:tasks.workunit.client.1.smithi070.stdout:0/412: write d8/f27 [4759820,103968] 0 2024-09-10T04:05:14.304 INFO:tasks.workunit.client.1.smithi070.stdout:0/413: truncate d8/d13/f1f 5243205 0 2024-09-10T04:05:14.305 INFO:tasks.workunit.client.1.smithi070.stdout:0/414: write d8/d24/d2d/d34/f77 [2920114,99023] 0 2024-09-10T04:05:14.305 INFO:tasks.workunit.client.0.smithi046.stdout:0/286: read - d1/dc/d25/d33/d3/d29/d2d/f4f zero size 2024-09-10T04:05:14.306 INFO:tasks.workunit.client.1.smithi070.stdout:0/415: write d8/fc [41462,116263] 0 2024-09-10T04:05:14.307 INFO:tasks.workunit.client.0.smithi046.stdout:5/433: symlink d7/d11/d14/d8e/la2 0 2024-09-10T04:05:14.307 INFO:tasks.workunit.client.0.smithi046.stdout:1/236: dwrite d3/dc/f17 [0,4194304] 0 2024-09-10T04:05:14.308 INFO:tasks.workunit.client.1.smithi070.stdout:7/291: creat d2/d7/db/d3a/f68 x:0 0 0 2024-09-10T04:05:14.310 INFO:tasks.workunit.client.0.smithi046.stdout:6/331: dread da/db/d11/d2b/d2f/f65 [0,4194304] 0 2024-09-10T04:05:14.312 INFO:tasks.workunit.client.0.smithi046.stdout:5/434: chown d7/d11/d3f/d53/d75 1941 1 2024-09-10T04:05:14.317 INFO:tasks.workunit.client.1.smithi070.stdout:1/296: truncate d3/d8/d12/f19 6744386 0 2024-09-10T04:05:14.318 INFO:tasks.workunit.client.0.smithi046.stdout:7/374: dwrite d5/d6/db/d11/f4e [0,4194304] 0 2024-09-10T04:05:14.319 INFO:tasks.workunit.client.0.smithi046.stdout:4/353: dwrite d3/d12/d17/d22/d30/f3a [0,4194304] 0 2024-09-10T04:05:14.324 INFO:tasks.workunit.client.1.smithi070.stdout:6/415: write d0/d3/d18/d1b/d17/d28/f3b [1837971,99571] 0 2024-09-10T04:05:14.329 INFO:tasks.workunit.client.0.smithi046.stdout:0/287: mkdir d1/dc/d25/d33/d3/d29/d2d/d5e 0 2024-09-10T04:05:14.330 INFO:tasks.workunit.client.0.smithi046.stdout:0/288: stat d1/dc/d25/d33/d3/f14 0 2024-09-10T04:05:14.332 INFO:tasks.workunit.client.0.smithi046.stdout:0/289: fsync d1/dc/d25/d33/d3/d29/f38 0 2024-09-10T04:05:14.334 INFO:tasks.workunit.client.0.smithi046.stdout:5/435: mkdir d7/d11/d14/d2f/da3 0 2024-09-10T04:05:14.337 INFO:tasks.workunit.client.0.smithi046.stdout:5/436: read - d7/d11/d3f/d53/d99/fa0 zero size 2024-09-10T04:05:14.339 INFO:tasks.workunit.client.0.smithi046.stdout:7/375: dread d5/f35 [0,4194304] 0 2024-09-10T04:05:14.341 INFO:tasks.workunit.client.0.smithi046.stdout:8/341: dwrite d1/d29/d12/f45 [0,4194304] 0 2024-09-10T04:05:14.341 INFO:tasks.workunit.client.1.smithi070.stdout:9/378: fdatasync d9/d59/d10/d1b/f45 0 2024-09-10T04:05:14.345 INFO:tasks.workunit.client.1.smithi070.stdout:2/304: rename d15/d18/d29/d2a/d2e/d32/d36/c37 to d15/d18/d56/c6b 0 2024-09-10T04:05:14.350 INFO:tasks.workunit.client.0.smithi046.stdout:4/354: rename d3/d12/d14/f21 to d3/d43/f7b 0 2024-09-10T04:05:14.351 INFO:tasks.workunit.client.1.smithi070.stdout:6/416: write d0/d3/d18/d62/f79 [2041210,91078] 0 2024-09-10T04:05:14.352 INFO:tasks.workunit.client.0.smithi046.stdout:4/355: write d3/d12/d17/d29/d58/f6d [155560,104211] 0 2024-09-10T04:05:14.352 INFO:tasks.workunit.client.0.smithi046.stdout:1/237: symlink d3/d27/l4f 0 2024-09-10T04:05:14.353 INFO:tasks.workunit.client.1.smithi070.stdout:6/417: dread - d0/d3/d18/d62/d70/f7c zero size 2024-09-10T04:05:14.354 INFO:tasks.workunit.client.0.smithi046.stdout:4/356: write d3/d12/d17/f5e [85656,129678] 0 2024-09-10T04:05:14.358 INFO:tasks.workunit.client.0.smithi046.stdout:1/238: dread d3/d27/f20 [0,4194304] 0 2024-09-10T04:05:14.359 INFO:tasks.workunit.client.1.smithi070.stdout:5/340: creat d17/f75 x:0 0 0 2024-09-10T04:05:14.359 INFO:tasks.workunit.client.0.smithi046.stdout:6/332: truncate da/db/d11/f3a 3593714 0 2024-09-10T04:05:14.360 INFO:tasks.workunit.client.1.smithi070.stdout:8/394: dwrite d4/d16/f3f [0,4194304] 0 2024-09-10T04:05:14.361 INFO:tasks.workunit.client.0.smithi046.stdout:6/333: dread - da/db/d10/f5d zero size 2024-09-10T04:05:14.361 INFO:tasks.workunit.client.1.smithi070.stdout:1/297: fdatasync d3/d8/d13/d3f/f42 0 2024-09-10T04:05:14.362 INFO:tasks.workunit.client.0.smithi046.stdout:6/334: chown da/db/d11/d2b/d2f/d3b/f5e 3294 1 2024-09-10T04:05:14.363 INFO:tasks.workunit.client.1.smithi070.stdout:1/298: dread d3/d8/d12/f4a [0,4194304] 0 2024-09-10T04:05:14.363 INFO:tasks.workunit.client.0.smithi046.stdout:5/437: symlink d7/d11/d14/d2f/d2b/d92/la4 0 2024-09-10T04:05:14.364 INFO:tasks.workunit.client.1.smithi070.stdout:9/379: symlink d9/d59/d2a/l71 0 2024-09-10T04:05:14.366 INFO:tasks.workunit.client.1.smithi070.stdout:1/299: rename d3/d8/d12/d2f/d32 to d3/d8/d12/d2f/d32/d4b/d5a 22 2024-09-10T04:05:14.367 INFO:tasks.workunit.client.1.smithi070.stdout:6/418: dread d0/d3/d18/f27 [0,4194304] 0 2024-09-10T04:05:14.373 INFO:tasks.workunit.client.0.smithi046.stdout:4/357: creat d3/d12/d14/f7c x:0 0 0 2024-09-10T04:05:14.373 INFO:tasks.workunit.client.0.smithi046.stdout:4/358: chown d3/d12/d17/d22 5408166 1 2024-09-10T04:05:14.378 INFO:tasks.workunit.client.1.smithi070.stdout:9/380: rmdir d9/d3e 39 2024-09-10T04:05:14.379 INFO:tasks.workunit.client.0.smithi046.stdout:1/239: dread d3/d27/f38 [0,4194304] 0 2024-09-10T04:05:14.379 INFO:tasks.workunit.client.0.smithi046.stdout:4/359: dwrite d3/fa [0,4194304] 0 2024-09-10T04:05:14.379 INFO:tasks.workunit.client.0.smithi046.stdout:1/240: chown d3/dc/f23 978 1 2024-09-10T04:05:14.381 INFO:tasks.workunit.client.1.smithi070.stdout:2/305: fdatasync d15/d18/d29/f46 0 2024-09-10T04:05:14.382 INFO:tasks.workunit.client.1.smithi070.stdout:2/306: fsync d15/d18/d29/d2a/f67 0 2024-09-10T04:05:14.384 INFO:tasks.workunit.client.1.smithi070.stdout:6/419: creat d0/d3/d18/d62/d70/f86 x:0 0 0 2024-09-10T04:05:14.384 INFO:tasks.workunit.client.1.smithi070.stdout:7/292: link d2/l29 d2/d3/dc/d5d/l69 0 2024-09-10T04:05:14.385 INFO:tasks.workunit.client.0.smithi046.stdout:1/241: dread d3/d27/f18 [0,4194304] 0 2024-09-10T04:05:14.386 INFO:tasks.workunit.client.1.smithi070.stdout:7/293: dread - d2/d3/dc/d5d/d41/f50 zero size 2024-09-10T04:05:14.386 INFO:tasks.workunit.client.1.smithi070.stdout:3/379: sync 2024-09-10T04:05:14.386 INFO:tasks.workunit.client.1.smithi070.stdout:4/272: sync 2024-09-10T04:05:14.387 INFO:tasks.workunit.client.1.smithi070.stdout:4/273: chown de/d13/c22 1 1 2024-09-10T04:05:14.390 INFO:tasks.workunit.client.0.smithi046.stdout:6/335: rename da/db/d11/f3a to da/db/d11/d2b/d2f/d3b/f6e 0 2024-09-10T04:05:14.392 INFO:tasks.workunit.client.0.smithi046.stdout:7/376: truncate d5/d6/db/d11/d1e/f30 678998 0 2024-09-10T04:05:14.393 INFO:tasks.workunit.client.0.smithi046.stdout:6/336: write da/db/d11/d2b/f6d [959333,116912] 0 2024-09-10T04:05:14.394 INFO:tasks.workunit.client.1.smithi070.stdout:5/341: link d17/d24/d45/f51 d17/d24/d45/f76 0 2024-09-10T04:05:14.395 INFO:tasks.workunit.client.0.smithi046.stdout:6/337: truncate da/db/d11/d2b/d2f/f66 540418 0 2024-09-10T04:05:14.395 INFO:tasks.workunit.client.0.smithi046.stdout:5/438: mkdir d7/d11/d14/d50/d9c/da5 0 2024-09-10T04:05:14.396 INFO:tasks.workunit.client.0.smithi046.stdout:0/290: write d1/d13/f1d [609191,80540] 0 2024-09-10T04:05:14.396 INFO:tasks.workunit.client.1.smithi070.stdout:5/342: read f14 [2965196,78694] 0 2024-09-10T04:05:14.397 INFO:tasks.workunit.client.1.smithi070.stdout:9/381: mkdir d9/d59/d5d/d6b/d72 0 2024-09-10T04:05:14.397 INFO:tasks.workunit.client.0.smithi046.stdout:5/439: chown d7/d11/d14/d2f/d64 2638413 1 2024-09-10T04:05:14.400 INFO:tasks.workunit.client.0.smithi046.stdout:4/360: mknod d3/d12/d17/d22/d51/d5c/c7d 0 2024-09-10T04:05:14.401 INFO:tasks.workunit.client.1.smithi070.stdout:4/274: dread de/d13/d18/f3b [0,4194304] 0 2024-09-10T04:05:14.401 INFO:tasks.workunit.client.1.smithi070.stdout:2/307: symlink d15/d18/d29/d2a/d2e/d32/d36/l6c 0 2024-09-10T04:05:14.402 INFO:tasks.workunit.client.1.smithi070.stdout:6/420: rmdir d0/d3/d18/d62/d70 39 2024-09-10T04:05:14.403 INFO:tasks.workunit.client.1.smithi070.stdout:4/275: write de/d13/d18/d26/f2a [293374,106975] 0 2024-09-10T04:05:14.404 INFO:tasks.workunit.client.1.smithi070.stdout:6/421: chown d0/d3/d1f/d55/f6d 25361 1 2024-09-10T04:05:14.406 INFO:tasks.workunit.client.0.smithi046.stdout:5/440: dwrite d7/d11/d3f/d53/f9b [0,4194304] 0 2024-09-10T04:05:14.406 INFO:tasks.workunit.client.0.smithi046.stdout:1/242: rename d3/dc/cd to d3/d27/d32/d1f/c50 0 2024-09-10T04:05:14.410 INFO:tasks.workunit.client.1.smithi070.stdout:9/382: dread d9/f25 [0,4194304] 0 2024-09-10T04:05:14.411 INFO:tasks.workunit.client.1.smithi070.stdout:3/380: unlink db/d1c/f62 0 2024-09-10T04:05:14.412 INFO:tasks.workunit.client.1.smithi070.stdout:8/395: mknod d4/d16/d17/d1a/d60/d7a/d84/c8b 0 2024-09-10T04:05:14.413 INFO:tasks.workunit.client.0.smithi046.stdout:0/291: symlink d1/dc/d25/d33/d4/d19/l5f 0 2024-09-10T04:05:14.413 INFO:tasks.workunit.client.1.smithi070.stdout:8/396: chown d4/d16/d17/d19/d28/d43/d52/c70 113743205 1 2024-09-10T04:05:14.416 INFO:tasks.workunit.client.0.smithi046.stdout:8/342: creat d1/d29/d19/d36/f74 x:0 0 0 2024-09-10T04:05:14.419 INFO:tasks.workunit.client.0.smithi046.stdout:4/361: mknod d3/d12/d14/c7e 0 2024-09-10T04:05:14.420 INFO:tasks.workunit.client.0.smithi046.stdout:4/362: chown d3/d12/d14/f2b 2 1 2024-09-10T04:05:14.425 INFO:tasks.workunit.client.0.smithi046.stdout:7/377: rename d5/d6/db/d11/d1e/d20/l47 to d5/d6/db/d11/d1f/d7a/l81 0 2024-09-10T04:05:14.426 INFO:tasks.workunit.client.0.smithi046.stdout:6/338: dwrite da/db/d11/d2b/d2f/f65 [0,4194304] 0 2024-09-10T04:05:14.427 INFO:tasks.workunit.client.1.smithi070.stdout:1/300: creat d3/d8/d12/d2f/f5b x:0 0 0 2024-09-10T04:05:14.429 INFO:tasks.workunit.client.0.smithi046.stdout:5/441: write d7/d11/d14/d2f/f48 [1881848,80806] 0 2024-09-10T04:05:14.430 INFO:tasks.workunit.client.0.smithi046.stdout:5/442: chown d7/d11/d3f/f80 133 1 2024-09-10T04:05:14.432 INFO:tasks.workunit.client.0.smithi046.stdout:1/243: mkdir d3/dc/d34/d51 0 2024-09-10T04:05:14.434 INFO:tasks.workunit.client.1.smithi070.stdout:2/308: mkdir d15/d18/d42/d62/d55/d6d 0 2024-09-10T04:05:14.434 INFO:tasks.workunit.client.0.smithi046.stdout:1/244: dread d3/d27/f20 [0,4194304] 0 2024-09-10T04:05:14.436 INFO:tasks.workunit.client.0.smithi046.stdout:0/292: creat d1/dc/d25/d33/d4/f60 x:0 0 0 2024-09-10T04:05:14.437 INFO:tasks.workunit.client.1.smithi070.stdout:6/422: fsync d0/d3/d18/f27 0 2024-09-10T04:05:14.439 INFO:tasks.workunit.client.1.smithi070.stdout:7/294: symlink d2/d3/dc/l6a 0 2024-09-10T04:05:14.440 INFO:tasks.workunit.client.1.smithi070.stdout:2/309: dwrite d15/d18/f28 [0,4194304] 0 2024-09-10T04:05:14.446 INFO:tasks.workunit.client.1.smithi070.stdout:8/397: unlink d4/d16/l38 0 2024-09-10T04:05:14.447 INFO:tasks.workunit.client.0.smithi046.stdout:7/378: fdatasync d5/d9/d43/f51 0 2024-09-10T04:05:14.450 INFO:tasks.workunit.client.1.smithi070.stdout:5/343: mkdir d17/d24/d77 0 2024-09-10T04:05:14.450 INFO:tasks.workunit.client.1.smithi070.stdout:5/344: write f8 [719864,103879] 0 2024-09-10T04:05:14.452 INFO:tasks.workunit.client.1.smithi070.stdout:4/276: creat de/d13/d18/d26/d46/f60 x:0 0 0 2024-09-10T04:05:14.453 INFO:tasks.workunit.client.0.smithi046.stdout:1/245: creat d3/dc/d21/d3a/f52 x:0 0 0 2024-09-10T04:05:14.456 INFO:tasks.workunit.client.0.smithi046.stdout:1/246: truncate d3/dc/d21/f40 28511 0 2024-09-10T04:05:14.456 INFO:tasks.workunit.client.1.smithi070.stdout:6/423: creat d0/d3/d18/d1b/d17/d54/f87 x:0 0 0 2024-09-10T04:05:14.457 INFO:tasks.workunit.client.1.smithi070.stdout:2/310: mkdir d15/d2b/d6e 0 2024-09-10T04:05:14.457 INFO:tasks.workunit.client.0.smithi046.stdout:4/363: mkdir d3/d12/d7f 0 2024-09-10T04:05:14.460 INFO:tasks.workunit.client.0.smithi046.stdout:6/339: link da/dc/d22/f34 da/db/d11/d2b/d5a/f6f 0 2024-09-10T04:05:14.461 INFO:tasks.workunit.client.1.smithi070.stdout:8/398: truncate d4/f15 1194994 0 2024-09-10T04:05:14.462 INFO:tasks.workunit.client.0.smithi046.stdout:6/340: truncate da/db/d11/d13/d1e/f3e 399094 0 2024-09-10T04:05:14.462 INFO:tasks.workunit.client.1.smithi070.stdout:5/345: dread fe [0,4194304] 0 2024-09-10T04:05:14.462 INFO:tasks.workunit.client.0.smithi046.stdout:7/379: dread d5/d6/db/f2a [0,4194304] 0 2024-09-10T04:05:14.463 INFO:tasks.workunit.client.1.smithi070.stdout:5/346: write fe [3045768,53804] 0 2024-09-10T04:05:14.463 INFO:tasks.workunit.client.0.smithi046.stdout:0/293: unlink d1/dc/d25/d33/d4/d19/c42 0 2024-09-10T04:05:14.463 INFO:tasks.workunit.client.0.smithi046.stdout:6/341: chown da/db/d11/d13/d1e/f4d 3 1 2024-09-10T04:05:14.465 INFO:tasks.workunit.client.0.smithi046.stdout:7/380: truncate d5/d6/db/d11/d1f/f48 376073 0 2024-09-10T04:05:14.466 INFO:tasks.workunit.client.0.smithi046.stdout:1/247: rmdir d3/d27/d32 39 2024-09-10T04:05:14.468 INFO:tasks.workunit.client.1.smithi070.stdout:6/424: chown d0/d3/c13 3 1 2024-09-10T04:05:14.470 INFO:tasks.workunit.client.1.smithi070.stdout:9/383: creat d9/f73 x:0 0 0 2024-09-10T04:05:14.473 INFO:tasks.workunit.client.1.smithi070.stdout:2/311: mknod d15/d18/d29/d2a/d2e/c6f 0 2024-09-10T04:05:14.474 INFO:tasks.workunit.client.1.smithi070.stdout:6/425: dwrite d0/d3/d1f/f22 [0,4194304] 0 2024-09-10T04:05:14.474 INFO:tasks.workunit.client.0.smithi046.stdout:0/294: creat d1/d13/d37/f61 x:0 0 0 2024-09-10T04:05:14.474 INFO:tasks.workunit.client.1.smithi070.stdout:6/426: fdatasync d0/d3/d18/f35 0 2024-09-10T04:05:14.476 INFO:tasks.workunit.client.1.smithi070.stdout:3/381: rename db/d1c/d3e/f30 to db/d1c/d25/f75 0 2024-09-10T04:05:14.477 INFO:tasks.workunit.client.0.smithi046.stdout:6/342: dread da/db/d11/d2b/f2c [0,4194304] 0 2024-09-10T04:05:14.477 INFO:tasks.workunit.client.1.smithi070.stdout:3/382: truncate db/f4e 4898942 0 2024-09-10T04:05:14.478 INFO:tasks.workunit.client.0.smithi046.stdout:8/343: mkdir d1/d29/d19/d36/d6e/d75 0 2024-09-10T04:05:14.480 INFO:tasks.workunit.client.0.smithi046.stdout:8/344: chown d1/d29/d17/l24 29110 1 2024-09-10T04:05:14.480 INFO:tasks.workunit.client.1.smithi070.stdout:1/301: creat d3/d8/f5c x:0 0 0 2024-09-10T04:05:14.480 INFO:tasks.workunit.client.0.smithi046.stdout:0/295: dread d1/dc/d25/d33/d3/f14 [0,4194304] 0 2024-09-10T04:05:14.484 INFO:tasks.workunit.client.0.smithi046.stdout:0/296: dread - d1/dc/d25/d33/f52 zero size 2024-09-10T04:05:14.484 INFO:tasks.workunit.client.0.smithi046.stdout:8/345: read d1/d29/d17/d1f/d2a/f2c [304837,81430] 0 2024-09-10T04:05:14.485 INFO:tasks.workunit.client.1.smithi070.stdout:3/383: dwrite db/d1c/d3e/f2a [0,4194304] 0 2024-09-10T04:05:14.485 INFO:tasks.workunit.client.1.smithi070.stdout:3/384: chown db/d1c/d25/d28/f41 6676120 1 2024-09-10T04:05:14.492 INFO:tasks.workunit.client.1.smithi070.stdout:2/312: stat l5 0 2024-09-10T04:05:14.492 INFO:tasks.workunit.client.1.smithi070.stdout:2/313: fdatasync f3 0 2024-09-10T04:05:14.493 INFO:tasks.workunit.client.0.smithi046.stdout:6/343: creat da/db/d11/d2b/f70 x:0 0 0 2024-09-10T04:05:14.494 INFO:tasks.workunit.client.0.smithi046.stdout:4/364: dwrite d3/fc [0,4194304] 0 2024-09-10T04:05:14.497 INFO:tasks.workunit.client.1.smithi070.stdout:2/314: dwrite d15/d18/f28 [0,4194304] 0 2024-09-10T04:05:14.498 INFO:tasks.workunit.client.1.smithi070.stdout:6/427: chown d0/d3/d18/d2a/c75 353 1 2024-09-10T04:05:14.500 INFO:tasks.workunit.client.0.smithi046.stdout:4/365: dread d3/fc [0,4194304] 0 2024-09-10T04:05:14.500 INFO:tasks.workunit.client.0.smithi046.stdout:7/381: write f4 [2754521,64896] 0 2024-09-10T04:05:14.503 INFO:tasks.workunit.client.0.smithi046.stdout:4/366: write d3/d12/f61 [881721,23051] 0 2024-09-10T04:05:14.503 INFO:tasks.workunit.client.0.smithi046.stdout:8/346: mkdir d1/d29/d17/d1f/d46/d76 0 2024-09-10T04:05:14.508 INFO:tasks.workunit.client.0.smithi046.stdout:5/443: link d7/d11/c34 d7/ca6 0 2024-09-10T04:05:14.509 INFO:tasks.workunit.client.0.smithi046.stdout:5/444: stat d7/d11/d14/d50/d9c/da5 0 2024-09-10T04:05:14.510 INFO:tasks.workunit.client.0.smithi046.stdout:6/344: rmdir da/db/d18 39 2024-09-10T04:05:14.511 INFO:tasks.workunit.client.1.smithi070.stdout:4/277: creat de/d13/d18/d23/f61 x:0 0 0 2024-09-10T04:05:14.514 INFO:tasks.workunit.client.1.smithi070.stdout:2/315: symlink d15/d18/d42/d62/l70 0 2024-09-10T04:05:14.514 INFO:tasks.workunit.client.1.smithi070.stdout:2/316: chown d15/d18/d29/d2a/d2e/c6f 57 1 2024-09-10T04:05:14.515 INFO:tasks.workunit.client.1.smithi070.stdout:9/384: rename d9/d3e to d9/d59/d5d/d6b/d74 0 2024-09-10T04:05:14.516 INFO:tasks.workunit.client.1.smithi070.stdout:9/385: write d9/d1c/f55 [3666815,123122] 0 2024-09-10T04:05:14.517 INFO:tasks.workunit.client.0.smithi046.stdout:4/367: creat d3/f80 x:0 0 0 2024-09-10T04:05:14.517 INFO:tasks.workunit.client.0.smithi046.stdout:4/368: stat d3/d12/d14/l28 0 2024-09-10T04:05:14.517 INFO:tasks.workunit.client.1.smithi070.stdout:9/386: rename d9/d59/d5d/d6b to d9/d59/d5d/d6b/d74/d75 22 2024-09-10T04:05:14.518 INFO:tasks.workunit.client.0.smithi046.stdout:4/369: stat d3/d12/d17/d22/d51/d5c/l78 0 2024-09-10T04:05:14.520 INFO:tasks.workunit.client.1.smithi070.stdout:1/302: mkdir d3/d8/d13/d3f/d5d 0 2024-09-10T04:05:14.522 INFO:tasks.workunit.client.0.smithi046.stdout:1/248: creat d3/d27/f53 x:0 0 0 2024-09-10T04:05:14.522 INFO:tasks.workunit.client.1.smithi070.stdout:8/399: link d4/d16/d17/d1a/l35 d4/d16/d73/l8c 0 2024-09-10T04:05:14.523 INFO:tasks.workunit.client.0.smithi046.stdout:4/370: dread d3/d12/d17/d22/f5b [0,4194304] 0 2024-09-10T04:05:14.523 INFO:tasks.workunit.client.1.smithi070.stdout:8/400: chown d4/d16/d17/d1a/l35 1512275471 1 2024-09-10T04:05:14.523 INFO:tasks.workunit.client.1.smithi070.stdout:5/347: link d17/d6f/l3e d17/d24/d4e/l78 0 2024-09-10T04:05:14.525 INFO:tasks.workunit.client.0.smithi046.stdout:5/445: mknod d7/d11/d14/d50/d9c/ca7 0 2024-09-10T04:05:14.528 INFO:tasks.workunit.client.0.smithi046.stdout:7/382: creat d5/d9/d12/d72/d77/f82 x:0 0 0 2024-09-10T04:05:14.529 INFO:tasks.workunit.client.1.smithi070.stdout:8/401: dwrite d4/d16/d17/d1a/f88 [0,4194304] 0 2024-09-10T04:05:14.529 INFO:tasks.workunit.client.0.smithi046.stdout:8/347: mknod d1/d29/d19/d36/d4c/c77 0 2024-09-10T04:05:14.530 INFO:tasks.workunit.client.0.smithi046.stdout:0/297: dwrite d1/dc/d25/d33/d3/d29/d2d/f32 [0,4194304] 0 2024-09-10T04:05:14.533 INFO:tasks.workunit.client.1.smithi070.stdout:5/348: creat d17/d24/d45/f79 x:0 0 0 2024-09-10T04:05:14.533 INFO:tasks.workunit.client.0.smithi046.stdout:0/298: chown d1/dc/d25/d33/d3/dd/d55/l54 3 1 2024-09-10T04:05:14.534 INFO:tasks.workunit.client.1.smithi070.stdout:5/349: write d17/d19/d6a/f5e [412277,20120] 0 2024-09-10T04:05:14.535 INFO:tasks.workunit.client.1.smithi070.stdout:5/350: read - d17/d24/d45/f79 zero size 2024-09-10T04:05:14.536 INFO:tasks.workunit.client.0.smithi046.stdout:6/345: mkdir da/d71 0 2024-09-10T04:05:14.537 INFO:tasks.workunit.client.1.smithi070.stdout:4/278: mknod de/d13/d5b/c62 0 2024-09-10T04:05:14.539 INFO:tasks.workunit.client.1.smithi070.stdout:6/428: link d0/d3/d18/d1b/d17/l24 d0/d3/d1f/d81/l88 0 2024-09-10T04:05:14.540 INFO:tasks.workunit.client.1.smithi070.stdout:4/279: dread de/d13/d18/d23/f3c [0,4194304] 0 2024-09-10T04:05:14.541 INFO:tasks.workunit.client.1.smithi070.stdout:4/280: chown de/d13/d18/d5c 1352635 1 2024-09-10T04:05:14.542 INFO:tasks.workunit.client.1.smithi070.stdout:9/387: link d9/d59/d5d/d6b/f51 d9/d1c/d34/f76 0 2024-09-10T04:05:14.544 INFO:tasks.workunit.client.1.smithi070.stdout:2/317: dread d15/d18/f24 [0,4194304] 0 2024-09-10T04:05:14.545 INFO:tasks.workunit.client.1.smithi070.stdout:8/402: mkdir d4/d16/d17/d19/d62/d8d 0 2024-09-10T04:05:14.549 INFO:tasks.workunit.client.0.smithi046.stdout:1/249: symlink d3/l54 0 2024-09-10T04:05:14.549 INFO:tasks.workunit.client.1.smithi070.stdout:3/385: link db/d1c/c66 db/d1c/c76 0 2024-09-10T04:05:14.551 INFO:tasks.workunit.client.0.smithi046.stdout:4/371: mknod d3/d12/c81 0 2024-09-10T04:05:14.551 INFO:tasks.workunit.client.0.smithi046.stdout:4/372: stat d3/d12/d17/d22/d30/c68 0 2024-09-10T04:05:14.552 INFO:tasks.workunit.client.1.smithi070.stdout:8/403: dwrite d4/d16/d17/d19/d62/f6e [0,4194304] 0 2024-09-10T04:05:14.552 INFO:tasks.workunit.client.1.smithi070.stdout:5/351: rename d17/d19/c3c to d17/d24/d4e/c7a 0 2024-09-10T04:05:14.555 INFO:tasks.workunit.client.0.smithi046.stdout:6/346: creat da/db/d11/d2b/d2f/f72 x:0 0 0 2024-09-10T04:05:14.556 INFO:tasks.workunit.client.1.smithi070.stdout:9/388: dread d9/d59/d5d/d6b/d74/f47 [0,4194304] 0 2024-09-10T04:05:14.557 INFO:tasks.workunit.client.1.smithi070.stdout:3/386: dwrite db/f3c [0,4194304] 0 2024-09-10T04:05:14.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/389: write d9/d59/d10/d1b/f45 [1357350,60740] 0 2024-09-10T04:05:14.560 INFO:tasks.workunit.client.0.smithi046.stdout:5/446: creat d7/d11/d14/d2f/da3/fa8 x:0 0 0 2024-09-10T04:05:14.565 INFO:tasks.workunit.client.0.smithi046.stdout:6/347: dwrite da/db/d11/f32 [0,4194304] 0 2024-09-10T04:05:14.568 INFO:tasks.workunit.client.0.smithi046.stdout:6/348: chown da/db/d11/d2b/d2f/f37 18364 1 2024-09-10T04:05:14.569 INFO:tasks.workunit.client.0.smithi046.stdout:6/349: write f9 [2397048,115503] 0 2024-09-10T04:05:14.572 INFO:tasks.workunit.client.1.smithi070.stdout:6/429: unlink d0/d3/d18/d1b/f14 0 2024-09-10T04:05:14.572 INFO:tasks.workunit.client.1.smithi070.stdout:4/281: truncate f6 844712 0 2024-09-10T04:05:14.572 INFO:tasks.workunit.client.0.smithi046.stdout:4/373: unlink d3/f80 0 2024-09-10T04:05:14.574 INFO:tasks.workunit.client.1.smithi070.stdout:6/430: chown d0/d3/d18/d1b/l31 282 1 2024-09-10T04:05:14.578 INFO:tasks.workunit.client.1.smithi070.stdout:6/431: read d0/d3/d18/d2a/f59 [3519341,93712] 0 2024-09-10T04:05:14.580 INFO:tasks.workunit.client.1.smithi070.stdout:6/432: dread d0/d3/d18/d1b/d17/f47 [0,4194304] 0 2024-09-10T04:05:14.585 INFO:tasks.workunit.client.0.smithi046.stdout:5/447: rename d7/d11/d14/d50 to d7/d11/d3f/d53/da9 0 2024-09-10T04:05:14.586 INFO:tasks.workunit.client.1.smithi070.stdout:3/387: creat db/d1c/d25/d28/d43/f77 x:0 0 0 2024-09-10T04:05:14.588 INFO:tasks.workunit.client.0.smithi046.stdout:7/383: write d5/d6/db/dd/f2f [3965333,81171] 0 2024-09-10T04:05:14.589 INFO:tasks.workunit.client.0.smithi046.stdout:0/299: dwrite d1/dc/d25/d33/f11 [4194304,4194304] 0 2024-09-10T04:05:14.590 INFO:tasks.workunit.client.0.smithi046.stdout:7/384: write d5/d6/d39/d5a/f79 [2323847,12504] 0 2024-09-10T04:05:14.598 INFO:tasks.workunit.client.0.smithi046.stdout:4/374: creat d3/d12/d17/d22/d30/f82 x:0 0 0 2024-09-10T04:05:14.599 INFO:tasks.workunit.client.1.smithi070.stdout:4/282: symlink de/d13/d18/l63 0 2024-09-10T04:05:14.600 INFO:tasks.workunit.client.1.smithi070.stdout:8/404: write d4/f77 [1529832,21752] 0 2024-09-10T04:05:14.602 INFO:tasks.workunit.client.0.smithi046.stdout:5/448: truncate d7/fe 4602043 0 2024-09-10T04:05:14.603 INFO:tasks.workunit.client.1.smithi070.stdout:1/303: link d3/d8/d13/d3f/f42 d3/d10/f5e 0 2024-09-10T04:05:14.605 INFO:tasks.workunit.client.1.smithi070.stdout:7/295: sync 2024-09-10T04:05:14.605 INFO:tasks.workunit.client.1.smithi070.stdout:0/416: sync 2024-09-10T04:05:14.607 INFO:tasks.workunit.client.1.smithi070.stdout:7/296: truncate d2/d7/d9/d18/f3f 1200523 0 2024-09-10T04:05:14.608 INFO:tasks.workunit.client.1.smithi070.stdout:7/297: readlink d2/d7/db/d20/l4b 0 2024-09-10T04:05:14.609 INFO:tasks.workunit.client.1.smithi070.stdout:5/352: mkdir d17/d19/d6a/d54/d57/d63/d7b 0 2024-09-10T04:05:14.612 INFO:tasks.workunit.client.1.smithi070.stdout:3/388: creat db/d1c/d25/d28/d43/f78 x:0 0 0 2024-09-10T04:05:14.612 INFO:tasks.workunit.client.0.smithi046.stdout:4/375: dread d3/d12/d17/d22/f5b [0,4194304] 0 2024-09-10T04:05:14.613 INFO:tasks.workunit.client.0.smithi046.stdout:8/348: getdents d1/d29/d19/d36/d4c 0 2024-09-10T04:05:14.614 INFO:tasks.workunit.client.0.smithi046.stdout:6/350: dwrite da/db/d11/d13/d1e/f26 [4194304,4194304] 0 2024-09-10T04:05:14.615 INFO:tasks.workunit.client.1.smithi070.stdout:4/283: dread de/f11 [0,4194304] 0 2024-09-10T04:05:14.615 INFO:tasks.workunit.client.1.smithi070.stdout:9/390: fdatasync d9/d59/d5d/d6b/d74/f5f 0 2024-09-10T04:05:14.615 INFO:tasks.workunit.client.0.smithi046.stdout:4/376: chown d3/d12/l3d 22 1 2024-09-10T04:05:14.616 INFO:tasks.workunit.client.0.smithi046.stdout:4/377: chown d3/d12/d17/d22/d30/l5d 3 1 2024-09-10T04:05:14.618 INFO:tasks.workunit.client.0.smithi046.stdout:5/449: mknod d7/d11/d14/d2f/d2b/d6a/caa 0 2024-09-10T04:05:14.620 INFO:tasks.workunit.client.1.smithi070.stdout:6/433: write d0/d3/d18/f42 [603583,112302] 0 2024-09-10T04:05:14.630 INFO:tasks.workunit.client.1.smithi070.stdout:0/417: mkdir d8/d13/d33/d7d/d8a 0 2024-09-10T04:05:14.630 INFO:tasks.workunit.client.1.smithi070.stdout:0/418: fsync d8/d24/d36/f46 0 2024-09-10T04:05:14.632 INFO:tasks.workunit.client.0.smithi046.stdout:7/385: link d5/d9/l4c d5/d6/d39/d5f/l83 0 2024-09-10T04:05:14.633 INFO:tasks.workunit.client.1.smithi070.stdout:7/298: rmdir d2/d3/dc/d13 39 2024-09-10T04:05:14.634 INFO:tasks.workunit.client.1.smithi070.stdout:7/299: write d2/f24 [3972926,78216] 0 2024-09-10T04:05:14.636 INFO:tasks.workunit.client.1.smithi070.stdout:0/419: dwrite d8/d13/d33/d53/d59/f64 [0,4194304] 0 2024-09-10T04:05:14.638 INFO:tasks.workunit.client.1.smithi070.stdout:7/300: truncate d2/d3/dc/d5d/d41/f50 920476 0 2024-09-10T04:05:14.638 INFO:tasks.workunit.client.1.smithi070.stdout:3/389: mkdir db/d79 0 2024-09-10T04:05:14.639 INFO:tasks.workunit.client.0.smithi046.stdout:4/378: dwrite d3/d12/f4d [0,4194304] 0 2024-09-10T04:05:14.639 INFO:tasks.workunit.client.1.smithi070.stdout:8/405: dread d4/fe [0,4194304] 0 2024-09-10T04:05:14.639 INFO:tasks.workunit.client.1.smithi070.stdout:3/390: readlink db/l19 0 2024-09-10T04:05:14.639 INFO:tasks.workunit.client.1.smithi070.stdout:4/284: creat de/f64 x:0 0 0 2024-09-10T04:05:14.640 INFO:tasks.workunit.client.0.smithi046.stdout:8/349: dread d1/d29/d17/f23 [0,4194304] 0 2024-09-10T04:05:14.640 INFO:tasks.workunit.client.0.smithi046.stdout:4/379: chown d3/f3c 52085489 1 2024-09-10T04:05:14.642 INFO:tasks.workunit.client.0.smithi046.stdout:7/386: unlink d5/d6/db/d11/f4e 0 2024-09-10T04:05:14.642 INFO:tasks.workunit.client.0.smithi046.stdout:8/350: readlink d1/d29/d17/d1f/d2a/l43 0 2024-09-10T04:05:14.647 INFO:tasks.workunit.client.1.smithi070.stdout:8/406: dwrite d4/d16/d17/d5c/f6d [0,4194304] 0 2024-09-10T04:05:14.688 INFO:tasks.workunit.client.1.smithi070.stdout:2/318: getdents d15/d18/d29/d2a/d2e/d32/d36 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:5/450: truncate d7/d11/d14/d2f/d2b/f3a 774105 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:8/351: symlink d1/d29/d19/d36/d6e/l78 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:5/451: dread d7/d11/f1a [0,4194304] 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:9/343: sync 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:7/387: unlink d5/d6/db/d11/d1e/c41 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:7/388: fdatasync d5/d6/d39/d5a/f64 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:8/352: truncate d1/d29/d19/d36/d4c/f5b 3732681 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:8/353: creat d1/f79 x:0 0 0 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:9/344: rmdir d4/d7 39 2024-09-10T04:05:14.821 INFO:tasks.workunit.client.0.smithi046.stdout:8/354: symlink d1/d29/d17/d1f/d46/d67/l7a 0 2024-09-10T04:05:14.823 INFO:tasks.workunit.client.1.smithi070.stdout:6/434: mknod d0/d3/d18/d2a/c89 0 2024-09-10T04:05:14.823 INFO:tasks.workunit.client.1.smithi070.stdout:5/353: getdents d17/d19/d6a/d54/d6e 0 2024-09-10T04:05:14.823 INFO:tasks.workunit.client.1.smithi070.stdout:0/420: write d8/d13/d1a/f30 [3879262,85089] 0 2024-09-10T04:05:14.823 INFO:tasks.workunit.client.1.smithi070.stdout:9/391: truncate d9/d59/d5d/f66 854773 0 2024-09-10T04:05:14.823 INFO:tasks.workunit.client.1.smithi070.stdout:6/435: stat d0/d3/d1f/d81/l88 0 2024-09-10T04:05:14.823 INFO:tasks.workunit.client.1.smithi070.stdout:9/392: dread d9/d59/d2a/d3f/f48 [0,4194304] 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:6/436: fdatasync d0/d3/d18/d62/d70/f7e 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/393: symlink d9/d1c/d34/l77 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:4/285: creat de/d13/d43/f65 x:0 0 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:2/319: link d15/d18/c1b d15/d59/c71 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/394: symlink d9/d59/d5d/d6b/d74/l78 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/395: stat d9/d59/f12 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:7/301: getdents d2/d7/d9/d18/d3c/d52 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/396: chown d9/d59/d5d/d6b/c4d 1179558 1 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:7/302: chown d2/d7/f43 1346259 1 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/397: chown d9/d59/d2a/l4b 3 1 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/398: dwrite d9/d1c/f2c [0,4194304] 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/399: truncate d9/d59/f1a 4227829 0 2024-09-10T04:05:14.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/400: symlink d9/d60/l79 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:7/303: creat d2/d7/d9/d18/f6b x:0 0 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:7/304: rmdir d2/d7/d9/d18/d23 39 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:7/305: fsync d2/d3/dc/d5d/d41/f53 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:7/306: fsync d2/d7/d9/d18/d3c/f58 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:7/307: dwrite d2/d7/d9/d18/d23/f47 [0,4194304] 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:7/308: mkdir d2/d40/d6c 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:5/354: write d17/d24/d45/f76 [1301489,29404] 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:5/355: creat d17/d19/d6a/f7c x:0 0 0 2024-09-10T04:05:14.825 INFO:tasks.workunit.client.1.smithi070.stdout:5/356: stat f16 0 2024-09-10T04:05:14.826 INFO:tasks.workunit.client.0.smithi046.stdout:4/380: rename d3/d12/f1f to d3/d12/d17/d29/f83 0 2024-09-10T04:05:14.826 INFO:tasks.workunit.client.1.smithi070.stdout:5/357: write f10 [2712997,63168] 0 2024-09-10T04:05:14.826 INFO:tasks.workunit.client.1.smithi070.stdout:5/358: stat d17/d19/d6a/f53 0 2024-09-10T04:05:14.826 INFO:tasks.workunit.client.1.smithi070.stdout:5/359: chown d17/d24/f27 5915 1 2024-09-10T04:05:14.829 INFO:tasks.workunit.client.1.smithi070.stdout:5/360: creat d17/d19/d6a/d54/d57/f7d x:0 0 0 2024-09-10T04:05:14.829 INFO:tasks.workunit.client.1.smithi070.stdout:5/361: write d17/d19/f44 [580122,4358] 0 2024-09-10T04:05:14.830 INFO:tasks.workunit.client.1.smithi070.stdout:5/362: chown d17/d24 1774845868 1 2024-09-10T04:05:14.832 INFO:tasks.workunit.client.0.smithi046.stdout:4/381: dwrite d3/d12/d17/d29/f3e [0,4194304] 0 2024-09-10T04:05:14.833 INFO:tasks.workunit.client.0.smithi046.stdout:4/382: truncate d3/d12/f61 1400521 0 2024-09-10T04:05:14.835 INFO:tasks.workunit.client.0.smithi046.stdout:4/383: fsync d3/d12/d17/f1b 0 2024-09-10T04:05:14.835 INFO:tasks.workunit.client.1.smithi070.stdout:7/309: link d2/d7/d9/d18/d3c/d52/c62 d2/d7/c6d 0 2024-09-10T04:05:14.837 INFO:tasks.workunit.client.1.smithi070.stdout:7/310: write d2/d7/db/f37 [4801084,58063] 0 2024-09-10T04:05:14.839 INFO:tasks.workunit.client.1.smithi070.stdout:1/304: mknod d3/d8/c5f 0 2024-09-10T04:05:14.841 INFO:tasks.workunit.client.1.smithi070.stdout:2/320: write d15/d18/f63 [1011739,116125] 0 2024-09-10T04:05:14.842 INFO:tasks.workunit.client.1.smithi070.stdout:7/311: symlink d2/d3/dc/d13/l6e 0 2024-09-10T04:05:14.844 INFO:tasks.workunit.client.1.smithi070.stdout:8/407: rename d4/d16/d17/d1a/d23 to d4/d16/d17/d5c/d87/d8e 0 2024-09-10T04:05:14.845 INFO:tasks.workunit.client.1.smithi070.stdout:6/437: dwrite d0/d3/d1f/f37 [4194304,4194304] 0 2024-09-10T04:05:14.846 INFO:tasks.workunit.client.1.smithi070.stdout:2/321: mknod d15/d18/d42/d62/c72 0 2024-09-10T04:05:14.847 INFO:tasks.workunit.client.1.smithi070.stdout:7/312: unlink d2/d3/l57 0 2024-09-10T04:05:14.848 INFO:tasks.workunit.client.1.smithi070.stdout:2/322: chown d15/d18/f19 173227515 1 2024-09-10T04:05:14.851 INFO:tasks.workunit.client.1.smithi070.stdout:7/313: write d2/d3/dc/d5d/d41/f53 [709304,37743] 0 2024-09-10T04:05:14.851 INFO:tasks.workunit.client.1.smithi070.stdout:1/305: dwrite d3/d8/d12/d2f/d32/f4c [0,4194304] 0 2024-09-10T04:05:14.852 INFO:tasks.workunit.client.0.smithi046.stdout:5/452: rename d7/d11/d14/f36 to d7/d11/d3f/d5a/d63/fab 0 2024-09-10T04:05:14.858 INFO:tasks.workunit.client.1.smithi070.stdout:2/323: dwrite d15/d18/f28 [0,4194304] 0 2024-09-10T04:05:14.859 INFO:tasks.workunit.client.1.smithi070.stdout:1/306: creat d3/f60 x:0 0 0 2024-09-10T04:05:14.860 INFO:tasks.workunit.client.1.smithi070.stdout:1/307: chown d3/d8/d13/f17 925258 1 2024-09-10T04:05:14.860 INFO:tasks.workunit.client.0.smithi046.stdout:5/453: dwrite d7/d11/d14/d2f/d2b/f7a [0,4194304] 0 2024-09-10T04:05:14.860 INFO:tasks.workunit.client.1.smithi070.stdout:6/438: mkdir d0/d3/d18/d1b/d17/d8a 0 2024-09-10T04:05:14.861 INFO:tasks.workunit.client.0.smithi046.stdout:5/454: chown d7/d11/d3f/d53/l6c 1 1 2024-09-10T04:05:14.862 INFO:tasks.workunit.client.0.smithi046.stdout:4/384: rename c2 to d3/d12/d17/d29/c84 0 2024-09-10T04:05:14.862 INFO:tasks.workunit.client.0.smithi046.stdout:5/455: chown d7/d11/d3f/d5a/d63/l68 91 1 2024-09-10T04:05:14.863 INFO:tasks.workunit.client.1.smithi070.stdout:1/308: truncate d3/d8/d12/f24 1589022 0 2024-09-10T04:05:14.864 INFO:tasks.workunit.client.1.smithi070.stdout:2/324: fdatasync f10 0 2024-09-10T04:05:14.868 INFO:tasks.workunit.client.0.smithi046.stdout:5/456: mkdir d7/d11/d3f/d53/dac 0 2024-09-10T04:05:14.873 INFO:tasks.workunit.client.1.smithi070.stdout:8/408: link d4/c61 d4/d16/d17/d5c/d87/d8e/d44/c8f 0 2024-09-10T04:05:14.875 INFO:tasks.workunit.client.1.smithi070.stdout:1/309: dwrite d3/d10/d1d/f46 [0,4194304] 0 2024-09-10T04:05:14.876 INFO:tasks.workunit.client.1.smithi070.stdout:1/310: stat d3/d8/d12/d2f/d32/d4b/c55 0 2024-09-10T04:05:14.877 INFO:tasks.workunit.client.0.smithi046.stdout:5/457: creat d7/d11/d14/d2f/d2b/fad x:0 0 0 2024-09-10T04:05:14.878 INFO:tasks.workunit.client.1.smithi070.stdout:4/286: rename f6 to de/d13/d4f/f66 0 2024-09-10T04:05:14.880 INFO:tasks.workunit.client.1.smithi070.stdout:1/311: dread d3/d8/d12/f24 [0,4194304] 0 2024-09-10T04:05:14.881 INFO:tasks.workunit.client.1.smithi070.stdout:4/287: truncate de/d13/d18/d26/f2c 4702616 0 2024-09-10T04:05:14.882 INFO:tasks.workunit.client.1.smithi070.stdout:1/312: write f0 [1542277,76681] 0 2024-09-10T04:05:14.885 INFO:tasks.workunit.client.1.smithi070.stdout:8/409: dread d4/d16/d17/d1a/f2a [0,4194304] 0 2024-09-10T04:05:14.885 INFO:tasks.workunit.client.1.smithi070.stdout:1/313: truncate d3/d8/d12/d2f/f5b 340961 0 2024-09-10T04:05:14.888 INFO:tasks.workunit.client.0.smithi046.stdout:9/345: write d4/d18/f25 [3123607,76362] 0 2024-09-10T04:05:14.888 INFO:tasks.workunit.client.1.smithi070.stdout:4/288: dwrite de/d13/d43/f65 [0,4194304] 0 2024-09-10T04:05:14.889 INFO:tasks.workunit.client.0.smithi046.stdout:9/346: write d4/d18/f4f [1047317,39831] 0 2024-09-10T04:05:14.890 INFO:tasks.workunit.client.0.smithi046.stdout:9/347: readlink d4/d29/l52 0 2024-09-10T04:05:14.896 INFO:tasks.workunit.client.0.smithi046.stdout:8/355: dwrite d1/d29/d17/d1f/f35 [0,4194304] 0 2024-09-10T04:05:14.897 INFO:tasks.workunit.client.1.smithi070.stdout:3/391: sync 2024-09-10T04:05:14.897 INFO:tasks.workunit.client.1.smithi070.stdout:6/439: symlink d0/l8b 0 2024-09-10T04:05:14.897 INFO:tasks.workunit.client.1.smithi070.stdout:6/440: chown d0/d3/d18/d1b/d33/f65 109765819 1 2024-09-10T04:05:14.898 INFO:tasks.workunit.client.1.smithi070.stdout:9/401: rename d9/d59/d5d/d6b to d9/d7a 0 2024-09-10T04:05:14.902 INFO:tasks.workunit.client.1.smithi070.stdout:4/289: creat de/d13/d18/d23/d39/f67 x:0 0 0 2024-09-10T04:05:14.906 INFO:tasks.workunit.client.0.smithi046.stdout:5/458: creat d7/d11/d3f/d53/d99/fae x:0 0 0 2024-09-10T04:05:14.908 INFO:tasks.workunit.client.0.smithi046.stdout:5/459: write d7/d11/d3f/d53/d99/fae [955107,110469] 0 2024-09-10T04:05:14.908 INFO:tasks.workunit.client.0.smithi046.stdout:5/460: readlink d7/la 0 2024-09-10T04:05:14.910 INFO:tasks.workunit.client.1.smithi070.stdout:4/290: dwrite de/d13/d18/d26/f2c [4194304,4194304] 0 2024-09-10T04:05:14.912 INFO:tasks.workunit.client.1.smithi070.stdout:5/363: rename d17/f6b to d17/d24/d45/f7e 0 2024-09-10T04:05:14.913 INFO:tasks.workunit.client.1.smithi070.stdout:5/364: readlink d17/l5c 0 2024-09-10T04:05:14.915 INFO:tasks.workunit.client.1.smithi070.stdout:5/365: write d17/d24/d45/f79 [135711,90173] 0 2024-09-10T04:05:14.916 INFO:tasks.workunit.client.0.smithi046.stdout:8/356: rmdir d1/d29/d19/d36/d4c/d5e 39 2024-09-10T04:05:14.922 INFO:tasks.workunit.client.0.smithi046.stdout:8/357: rename d1/d29/d17/d1f/f44 to d1/d29/d17/d1f/f7b 0 2024-09-10T04:05:14.923 INFO:tasks.workunit.client.1.smithi070.stdout:3/392: symlink db/d79/l7a 0 2024-09-10T04:05:14.925 INFO:tasks.workunit.client.0.smithi046.stdout:9/348: dread d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:14.925 INFO:tasks.workunit.client.1.smithi070.stdout:6/441: mkdir d0/d3/d8c 0 2024-09-10T04:05:14.926 INFO:tasks.workunit.client.1.smithi070.stdout:6/442: chown d0/d3/d1f/d55/f6d 94117 1 2024-09-10T04:05:14.927 INFO:tasks.workunit.client.1.smithi070.stdout:6/443: dread - d0/d3/d18/d62/f85 zero size 2024-09-10T04:05:14.928 INFO:tasks.workunit.client.0.smithi046.stdout:4/385: dwrite d3/f3c [0,4194304] 0 2024-09-10T04:05:14.929 INFO:tasks.workunit.client.1.smithi070.stdout:4/291: unlink de/d13/d18/d26/f33 0 2024-09-10T04:05:14.930 INFO:tasks.workunit.client.1.smithi070.stdout:4/292: dread - de/d13/d4f/f56 zero size 2024-09-10T04:05:14.931 INFO:tasks.workunit.client.1.smithi070.stdout:4/293: dread - de/d13/f3e zero size 2024-09-10T04:05:14.932 INFO:tasks.workunit.client.1.smithi070.stdout:9/402: creat d9/d7a/d72/f7b x:0 0 0 2024-09-10T04:05:14.934 INFO:tasks.workunit.client.1.smithi070.stdout:9/403: write d9/d1c/d34/f33 [2249774,43516] 0 2024-09-10T04:05:14.935 INFO:tasks.workunit.client.1.smithi070.stdout:9/404: stat d9/d59/d10/d1b/c28 0 2024-09-10T04:05:14.936 INFO:tasks.workunit.client.1.smithi070.stdout:2/325: truncate d15/d18/d29/d2a/d2e/d32/f49 710975 0 2024-09-10T04:05:14.936 INFO:tasks.workunit.client.0.smithi046.stdout:4/386: dwrite d3/d12/d17/d22/f4a [0,4194304] 0 2024-09-10T04:05:14.938 INFO:tasks.workunit.client.1.smithi070.stdout:5/366: symlink d17/d19/d35/l7f 0 2024-09-10T04:05:14.938 INFO:tasks.workunit.client.0.smithi046.stdout:2/391: sync 2024-09-10T04:05:14.938 INFO:tasks.workunit.client.0.smithi046.stdout:3/330: sync 2024-09-10T04:05:14.939 INFO:tasks.workunit.client.0.smithi046.stdout:8/358: creat d1/d29/d19/d36/d6e/d75/f7c x:0 0 0 2024-09-10T04:05:14.939 INFO:tasks.workunit.client.1.smithi070.stdout:5/367: write f7 [2411661,40640] 0 2024-09-10T04:05:14.940 INFO:tasks.workunit.client.1.smithi070.stdout:3/393: mknod db/d1c/d25/d28/c7b 0 2024-09-10T04:05:14.942 INFO:tasks.workunit.client.0.smithi046.stdout:9/349: mknod d4/d18/c6c 0 2024-09-10T04:05:14.945 INFO:tasks.workunit.client.1.smithi070.stdout:6/444: chown d0/c29 0 1 2024-09-10T04:05:14.945 INFO:tasks.workunit.client.1.smithi070.stdout:9/405: dwrite d9/d59/f18 [0,4194304] 0 2024-09-10T04:05:14.948 INFO:tasks.workunit.client.1.smithi070.stdout:2/326: dwrite d15/d18/d29/d2a/d2e/d32/f33 [0,4194304] 0 2024-09-10T04:05:14.953 INFO:tasks.workunit.client.1.smithi070.stdout:4/294: rmdir de/d13/d18/d23/d47/d59 39 2024-09-10T04:05:14.953 INFO:tasks.workunit.client.1.smithi070.stdout:2/327: stat d15/d18/d42/d62/l52 0 2024-09-10T04:05:14.955 INFO:tasks.workunit.client.1.smithi070.stdout:4/295: chown de/d13/d18/d26/f42 0 1 2024-09-10T04:05:14.956 INFO:tasks.workunit.client.1.smithi070.stdout:9/406: dwrite d9/d7a/f64 [0,4194304] 0 2024-09-10T04:05:14.956 INFO:tasks.workunit.client.1.smithi070.stdout:2/328: truncate d15/d18/f27 5065603 0 2024-09-10T04:05:14.958 INFO:tasks.workunit.client.0.smithi046.stdout:4/387: mkdir d3/d43/d5f/d85 0 2024-09-10T04:05:14.959 INFO:tasks.workunit.client.0.smithi046.stdout:4/388: stat d3/d12/d17/d29/c79 0 2024-09-10T04:05:14.960 INFO:tasks.workunit.client.0.smithi046.stdout:4/389: chown d3/d12/c81 99836622 1 2024-09-10T04:05:14.962 INFO:tasks.workunit.client.0.smithi046.stdout:3/331: mkdir d12/d13/d14/d1e/d2d/d39/d67 0 2024-09-10T04:05:14.963 INFO:tasks.workunit.client.0.smithi046.stdout:3/332: dread - d12/d13/d37/d51/f57 zero size 2024-09-10T04:05:14.964 INFO:tasks.workunit.client.0.smithi046.stdout:3/333: chown d12/d1b/d50 103005 1 2024-09-10T04:05:14.965 INFO:tasks.workunit.client.1.smithi070.stdout:5/368: rmdir d17/d19/d6a/d54/d57 39 2024-09-10T04:05:14.965 INFO:tasks.workunit.client.1.smithi070.stdout:5/369: fdatasync d17/d19/f44 0 2024-09-10T04:05:14.966 INFO:tasks.workunit.client.1.smithi070.stdout:5/370: stat d17/d24/d2b/l42 0 2024-09-10T04:05:14.967 INFO:tasks.workunit.client.1.smithi070.stdout:5/371: fdatasync d17/d37/f73 0 2024-09-10T04:05:14.972 INFO:tasks.workunit.client.1.smithi070.stdout:7/314: rename d2/d7/db/d20/l22 to d2/d7/d9/d18/l6f 0 2024-09-10T04:05:14.972 INFO:tasks.workunit.client.1.smithi070.stdout:0/421: sync 2024-09-10T04:05:14.973 INFO:tasks.workunit.client.1.smithi070.stdout:4/296: mknod de/d13/d4f/c68 0 2024-09-10T04:05:14.973 INFO:tasks.workunit.client.1.smithi070.stdout:0/422: write d8/d24/d2d/f3a [2048521,18413] 0 2024-09-10T04:05:14.974 INFO:tasks.workunit.client.1.smithi070.stdout:8/410: truncate d4/d16/f36 2197929 0 2024-09-10T04:05:14.976 INFO:tasks.workunit.client.1.smithi070.stdout:9/407: creat d9/d59/d10/d3c/d62/f7c x:0 0 0 2024-09-10T04:05:14.976 INFO:tasks.workunit.client.1.smithi070.stdout:8/411: write d4/d39/f57 [695242,129538] 0 2024-09-10T04:05:14.977 INFO:tasks.workunit.client.0.smithi046.stdout:8/359: unlink d1/d29/d17/d1f/f7b 0 2024-09-10T04:05:14.977 INFO:tasks.workunit.client.0.smithi046.stdout:2/392: link d2/d6/d23/d5d/d77/d7b/d85/c56 d2/d6/d23/d5d/c9a 0 2024-09-10T04:05:14.978 INFO:tasks.workunit.client.0.smithi046.stdout:2/393: stat d2/d6/d23/d5d/d77/l6e 0 2024-09-10T04:05:14.979 INFO:tasks.workunit.client.1.smithi070.stdout:2/329: creat d15/d18/d29/d2a/d2e/d32/f73 x:0 0 0 2024-09-10T04:05:14.982 INFO:tasks.workunit.client.0.smithi046.stdout:9/350: rename d4/d7/ff to d4/d29/f6d 0 2024-09-10T04:05:14.982 INFO:tasks.workunit.client.1.smithi070.stdout:9/408: dwrite d9/f25 [0,4194304] 0 2024-09-10T04:05:14.983 INFO:tasks.workunit.client.1.smithi070.stdout:9/409: dread - d9/f68 zero size 2024-09-10T04:05:14.984 INFO:tasks.workunit.client.0.smithi046.stdout:4/390: symlink d3/d12/l86 0 2024-09-10T04:05:14.985 INFO:tasks.workunit.client.1.smithi070.stdout:5/372: rmdir d17/d6f 39 2024-09-10T04:05:14.985 INFO:tasks.workunit.client.0.smithi046.stdout:2/394: dwrite d2/d6/d23/f4b [0,4194304] 0 2024-09-10T04:05:14.991 INFO:tasks.workunit.client.0.smithi046.stdout:4/391: dread d3/d12/d17/d29/f3e [0,4194304] 0 2024-09-10T04:05:14.992 INFO:tasks.workunit.client.1.smithi070.stdout:3/394: truncate db/d1c/d25/d28/f37 23709 0 2024-09-10T04:05:14.992 INFO:tasks.workunit.client.1.smithi070.stdout:7/315: dread d2/d7/d9/d18/d23/f3e [0,4194304] 0 2024-09-10T04:05:14.996 INFO:tasks.workunit.client.1.smithi070.stdout:6/445: rename d0/d3/d1f/c7b to d0/d3/d1f/d55/c8d 0 2024-09-10T04:05:14.997 INFO:tasks.workunit.client.1.smithi070.stdout:6/446: chown d0/d3/d18/d1b/f3a 12356067 1 2024-09-10T04:05:14.998 INFO:tasks.workunit.client.1.smithi070.stdout:6/447: write d0/d3/d18/d62/d70/f7e [139584,121685] 0 2024-09-10T04:05:15.000 INFO:tasks.workunit.client.0.smithi046.stdout:3/334: rename d12/l2a to d12/d5d/l68 0 2024-09-10T04:05:15.005 INFO:tasks.workunit.client.0.smithi046.stdout:9/351: dread d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:15.010 INFO:tasks.workunit.client.1.smithi070.stdout:9/410: creat d9/d59/d10/d3c/d62/f7d x:0 0 0 2024-09-10T04:05:15.012 INFO:tasks.workunit.client.0.smithi046.stdout:2/395: rename d2/d6/d23/f5b to d2/d49/f9b 0 2024-09-10T04:05:15.013 INFO:tasks.workunit.client.0.smithi046.stdout:3/335: symlink d12/d13/d42/d49/l69 0 2024-09-10T04:05:15.015 INFO:tasks.workunit.client.1.smithi070.stdout:7/316: creat d2/d3/f70 x:0 0 0 2024-09-10T04:05:15.018 INFO:tasks.workunit.client.1.smithi070.stdout:2/330: rename fa to d15/d18/d29/d2a/f74 0 2024-09-10T04:05:15.018 INFO:tasks.workunit.client.0.smithi046.stdout:2/396: dwrite d2/d6/d23/d42/f53 [0,4194304] 0 2024-09-10T04:05:15.026 INFO:tasks.workunit.client.0.smithi046.stdout:9/352: creat d4/d7/d1f/d54/f6e x:0 0 0 2024-09-10T04:05:15.027 INFO:tasks.workunit.client.1.smithi070.stdout:0/423: mknod d8/d24/c8b 0 2024-09-10T04:05:15.027 INFO:tasks.workunit.client.0.smithi046.stdout:2/397: dwrite d2/f73 [0,4194304] 0 2024-09-10T04:05:15.029 INFO:tasks.workunit.client.1.smithi070.stdout:6/448: dread d0/d3/d1f/f46 [0,4194304] 0 2024-09-10T04:05:15.031 INFO:tasks.workunit.client.1.smithi070.stdout:8/412: symlink d4/d16/d17/l90 0 2024-09-10T04:05:15.033 INFO:tasks.workunit.client.0.smithi046.stdout:3/336: fsync d12/d1b/f4c 0 2024-09-10T04:05:15.034 INFO:tasks.workunit.client.1.smithi070.stdout:5/373: stat d17/d6f/f52 0 2024-09-10T04:05:15.038 INFO:tasks.workunit.client.0.smithi046.stdout:8/360: creat d1/d29/d19/d36/d4c/f7d x:0 0 0 2024-09-10T04:05:15.046 INFO:tasks.workunit.client.0.smithi046.stdout:4/392: rmdir d3/d12/d7f 0 2024-09-10T04:05:15.047 INFO:tasks.workunit.client.0.smithi046.stdout:4/393: fsync d3/d12/f19 0 2024-09-10T04:05:15.050 INFO:tasks.workunit.client.0.smithi046.stdout:3/337: chown d12/d13/d14/l22 588193 1 2024-09-10T04:05:15.051 INFO:tasks.workunit.client.0.smithi046.stdout:8/361: unlink d1/d29/d19/d36/d6e/d75/f7c 0 2024-09-10T04:05:15.052 INFO:tasks.workunit.client.0.smithi046.stdout:8/362: chown d1/d29/d17/d1f/f35 2746 1 2024-09-10T04:05:15.053 INFO:tasks.workunit.client.0.smithi046.stdout:9/353: fsync d4/d29/d32/f55 0 2024-09-10T04:05:15.053 INFO:tasks.workunit.client.0.smithi046.stdout:8/363: dread - d1/f39 zero size 2024-09-10T04:05:15.055 INFO:tasks.workunit.client.0.smithi046.stdout:2/398: mkdir d2/d86/d9c 0 2024-09-10T04:05:15.058 INFO:tasks.workunit.client.1.smithi070.stdout:1/314: sync 2024-09-10T04:05:15.058 INFO:tasks.workunit.client.1.smithi070.stdout:2/331: rename d15/d18/d29/c66 to d15/d18/d42/d62/d55/d6d/c75 0 2024-09-10T04:05:15.059 INFO:tasks.workunit.client.1.smithi070.stdout:6/449: creat d0/d3/d1f/d55/f8e x:0 0 0 2024-09-10T04:05:15.059 INFO:tasks.workunit.client.1.smithi070.stdout:8/413: rmdir d4/d16 39 2024-09-10T04:05:15.059 INFO:tasks.workunit.client.0.smithi046.stdout:4/394: rename d3/d12/f61 to d3/d12/d17/d29/f87 0 2024-09-10T04:05:15.060 INFO:tasks.workunit.client.0.smithi046.stdout:2/399: dread d2/d8/f6f [0,4194304] 0 2024-09-10T04:05:15.060 INFO:tasks.workunit.client.0.smithi046.stdout:4/395: stat d3/d12/d14/f2b 0 2024-09-10T04:05:15.061 INFO:tasks.workunit.client.0.smithi046.stdout:2/400: dread - d2/d8/d37/d63/f82 zero size 2024-09-10T04:05:15.062 INFO:tasks.workunit.client.1.smithi070.stdout:2/332: dread d15/d18/f26 [0,4194304] 0 2024-09-10T04:05:15.062 INFO:tasks.workunit.client.0.smithi046.stdout:2/401: write d2/d6/f80 [1603739,35529] 0 2024-09-10T04:05:15.063 INFO:tasks.workunit.client.1.smithi070.stdout:2/333: write d15/d18/f28 [980770,98781] 0 2024-09-10T04:05:15.065 INFO:tasks.workunit.client.1.smithi070.stdout:6/450: dread d0/d3/d18/d1b/d17/d28/f6b [0,4194304] 0 2024-09-10T04:05:15.065 INFO:tasks.workunit.client.1.smithi070.stdout:0/424: creat d8/d13/d47/d7b/f8c x:0 0 0 2024-09-10T04:05:15.066 INFO:tasks.workunit.client.0.smithi046.stdout:3/338: dwrite d12/d13/d14/d1e/f3f [0,4194304] 0 2024-09-10T04:05:15.067 INFO:tasks.workunit.client.1.smithi070.stdout:0/425: dread - d8/d13/d1a/f45 zero size 2024-09-10T04:05:15.067 INFO:tasks.workunit.client.1.smithi070.stdout:4/297: getdents de/d13/d18 0 2024-09-10T04:05:15.068 INFO:tasks.workunit.client.1.smithi070.stdout:0/426: read d8/d24/d2d/d5f/f60 [79731,52694] 0 2024-09-10T04:05:15.069 INFO:tasks.workunit.client.1.smithi070.stdout:0/427: read d8/d13/d33/d53/d59/f64 [2790912,971] 0 2024-09-10T04:05:15.069 INFO:tasks.workunit.client.1.smithi070.stdout:9/411: creat d9/d59/f7e x:0 0 0 2024-09-10T04:05:15.070 INFO:tasks.workunit.client.0.smithi046.stdout:6/351: sync 2024-09-10T04:05:15.071 INFO:tasks.workunit.client.0.smithi046.stdout:1/250: sync 2024-09-10T04:05:15.071 INFO:tasks.workunit.client.0.smithi046.stdout:7/389: sync 2024-09-10T04:05:15.071 INFO:tasks.workunit.client.0.smithi046.stdout:0/300: sync 2024-09-10T04:05:15.073 INFO:tasks.workunit.client.0.smithi046.stdout:0/301: chown d1/dc/d25/d33/f3f 776 1 2024-09-10T04:05:15.074 INFO:tasks.workunit.client.0.smithi046.stdout:8/364: rename d1/d29/d17/d1f/d46/d52/c63 to d1/d29/d17/d1f/d46/d67/c7e 0 2024-09-10T04:05:15.075 INFO:tasks.workunit.client.1.smithi070.stdout:5/374: creat d17/d19/d6a/d54/d57/f80 x:0 0 0 2024-09-10T04:05:15.076 INFO:tasks.workunit.client.0.smithi046.stdout:4/396: mknod d3/d12/d17/c88 0 2024-09-10T04:05:15.078 INFO:tasks.workunit.client.1.smithi070.stdout:5/375: dread d17/d24/f2d [0,4194304] 0 2024-09-10T04:05:15.078 INFO:tasks.workunit.client.0.smithi046.stdout:4/397: truncate d3/d12/f73 316445 0 2024-09-10T04:05:15.081 INFO:tasks.workunit.client.0.smithi046.stdout:8/365: dwrite d1/d29/fe [4194304,4194304] 0 2024-09-10T04:05:15.083 INFO:tasks.workunit.client.0.smithi046.stdout:9/354: creat d4/d7/d61/f6f x:0 0 0 2024-09-10T04:05:15.084 INFO:tasks.workunit.client.0.smithi046.stdout:3/339: fdatasync d12/d1b/f4c 0 2024-09-10T04:05:15.087 INFO:tasks.workunit.client.0.smithi046.stdout:1/251: chown d3/d27/d32/f2b 323 1 2024-09-10T04:05:15.087 INFO:tasks.workunit.client.0.smithi046.stdout:1/252: fsync d3/f4a 0 2024-09-10T04:05:15.088 INFO:tasks.workunit.client.0.smithi046.stdout:1/253: write d3/f6 [1427537,15937] 0 2024-09-10T04:05:15.092 INFO:tasks.workunit.client.0.smithi046.stdout:0/302: creat d1/dc/d25/d33/d4/f62 x:0 0 0 2024-09-10T04:05:15.095 INFO:tasks.workunit.client.0.smithi046.stdout:7/390: rename d5/d6/db/dd/d24/c73 to d5/d9/d12/d72/d77/c84 0 2024-09-10T04:05:15.096 INFO:tasks.workunit.client.1.smithi070.stdout:5/376: creat d17/d24/d2b/f81 x:0 0 0 2024-09-10T04:05:15.100 INFO:tasks.workunit.client.1.smithi070.stdout:2/334: link d15/d18/d42/d62/d55/f65 d15/d18/d42/d62/f76 0 2024-09-10T04:05:15.101 INFO:tasks.workunit.client.0.smithi046.stdout:7/391: dwrite d5/d6/db/f29 [0,4194304] 0 2024-09-10T04:05:15.101 INFO:tasks.workunit.client.1.smithi070.stdout:2/335: dread d15/d2b/f4c [0,4194304] 0 2024-09-10T04:05:15.102 INFO:tasks.workunit.client.0.smithi046.stdout:7/392: write d5/d9/d43/f51 [690457,68906] 0 2024-09-10T04:05:15.102 INFO:tasks.workunit.client.0.smithi046.stdout:4/398: mknod d3/d12/d17/d22/d30/c89 0 2024-09-10T04:05:15.103 INFO:tasks.workunit.client.0.smithi046.stdout:7/393: write d5/d9/d43/f5c [766760,115539] 0 2024-09-10T04:05:15.106 INFO:tasks.workunit.client.0.smithi046.stdout:9/355: unlink d4/d7/l41 0 2024-09-10T04:05:15.109 INFO:tasks.workunit.client.0.smithi046.stdout:3/340: rmdir d12/d13/d14/d1e/d2d 39 2024-09-10T04:05:15.110 INFO:tasks.workunit.client.0.smithi046.stdout:6/352: mknod da/db/d18/c73 0 2024-09-10T04:05:15.111 INFO:tasks.workunit.client.0.smithi046.stdout:4/399: dwrite d3/d12/d17/d22/d30/f3a [0,4194304] 0 2024-09-10T04:05:15.113 INFO:tasks.workunit.client.0.smithi046.stdout:5/461: sync 2024-09-10T04:05:15.116 INFO:tasks.workunit.client.0.smithi046.stdout:1/254: creat d3/d27/d32/d1f/f55 x:0 0 0 2024-09-10T04:05:15.119 INFO:tasks.workunit.client.1.smithi070.stdout:2/336: mknod d15/d18/d42/d62/c77 0 2024-09-10T04:05:15.124 INFO:tasks.workunit.client.1.smithi070.stdout:9/412: rename d9/c4a to d9/d1c/c7f 0 2024-09-10T04:05:15.133 INFO:tasks.workunit.client.1.smithi070.stdout:5/377: dwrite d17/d24/f2f [0,4194304] 0 2024-09-10T04:05:15.134 INFO:tasks.workunit.client.1.smithi070.stdout:8/414: dwrite d4/d16/d17/f68 [0,4194304] 0 2024-09-10T04:05:15.135 INFO:tasks.workunit.client.1.smithi070.stdout:3/395: sync 2024-09-10T04:05:15.136 INFO:tasks.workunit.client.0.smithi046.stdout:9/356: symlink d4/d3c/d4d/d5c/d59/l70 0 2024-09-10T04:05:15.140 INFO:tasks.workunit.client.1.smithi070.stdout:9/413: dread d9/d59/f1a [0,4194304] 0 2024-09-10T04:05:15.140 INFO:tasks.workunit.client.0.smithi046.stdout:3/341: rename d12/d13/d14/l22 to d12/d1b/d50/l6a 0 2024-09-10T04:05:15.142 INFO:tasks.workunit.client.0.smithi046.stdout:4/400: symlink d3/d12/d17/l8a 0 2024-09-10T04:05:15.144 INFO:tasks.workunit.client.0.smithi046.stdout:6/353: creat da/db/d11/d2b/d2f/f74 x:0 0 0 2024-09-10T04:05:15.146 INFO:tasks.workunit.client.0.smithi046.stdout:5/462: readlink d7/d11/l4c 0 2024-09-10T04:05:15.146 INFO:tasks.workunit.client.0.smithi046.stdout:8/366: write d1/d29/d19/d36/f4f [3181803,87265] 0 2024-09-10T04:05:15.147 INFO:tasks.workunit.client.0.smithi046.stdout:5/463: readlink d7/d11/d14/d2f/d2b/l5c 0 2024-09-10T04:05:15.149 INFO:tasks.workunit.client.0.smithi046.stdout:5/464: chown d7/f70 626 1 2024-09-10T04:05:15.149 INFO:tasks.workunit.client.1.smithi070.stdout:4/298: rename de/l3f to de/d13/d18/d23/d47/d59/l69 0 2024-09-10T04:05:15.149 INFO:tasks.workunit.client.1.smithi070.stdout:4/299: stat de/d13/d43/c5d 0 2024-09-10T04:05:15.150 INFO:tasks.workunit.client.1.smithi070.stdout:4/300: chown de/d13/d18/l1e 68 1 2024-09-10T04:05:15.151 INFO:tasks.workunit.client.1.smithi070.stdout:4/301: fsync de/d13/d43/f53 0 2024-09-10T04:05:15.151 INFO:tasks.workunit.client.1.smithi070.stdout:2/337: symlink d15/d18/d29/l78 0 2024-09-10T04:05:15.151 INFO:tasks.workunit.client.0.smithi046.stdout:2/402: getdents d2/d86 0 2024-09-10T04:05:15.153 INFO:tasks.workunit.client.1.smithi070.stdout:5/378: creat d17/d24/d4e/f82 x:0 0 0 2024-09-10T04:05:15.156 INFO:tasks.workunit.client.1.smithi070.stdout:8/415: unlink d4/d16/d17/d5c/d87/d8e/c83 0 2024-09-10T04:05:15.157 INFO:tasks.workunit.client.0.smithi046.stdout:9/357: symlink d4/d7/d61/l71 0 2024-09-10T04:05:15.157 INFO:tasks.workunit.client.1.smithi070.stdout:3/396: creat db/d1c/d25/d28/f7c x:0 0 0 2024-09-10T04:05:15.160 INFO:tasks.workunit.client.0.smithi046.stdout:4/401: rmdir d3/d12/d17 39 2024-09-10T04:05:15.162 INFO:tasks.workunit.client.0.smithi046.stdout:6/354: fsync da/f1f 0 2024-09-10T04:05:15.163 INFO:tasks.workunit.client.1.smithi070.stdout:4/302: rmdir de 39 2024-09-10T04:05:15.164 INFO:tasks.workunit.client.0.smithi046.stdout:8/367: chown d1/d29/d17/c42 8010 1 2024-09-10T04:05:15.165 INFO:tasks.workunit.client.0.smithi046.stdout:8/368: chown d1/d29/d17/d1f/f54 3138 1 2024-09-10T04:05:15.166 INFO:tasks.workunit.client.0.smithi046.stdout:8/369: chown d1/d29/fe 3772291 1 2024-09-10T04:05:15.166 INFO:tasks.workunit.client.0.smithi046.stdout:0/303: creat d1/dc/d25/d33/d3/f63 x:0 0 0 2024-09-10T04:05:15.169 INFO:tasks.workunit.client.0.smithi046.stdout:5/465: mknod d7/d11/d14/d2f/d2b/d92/caf 0 2024-09-10T04:05:15.170 INFO:tasks.workunit.client.1.smithi070.stdout:8/416: read d4/f29 [1923874,67084] 0 2024-09-10T04:05:15.171 INFO:tasks.workunit.client.1.smithi070.stdout:8/417: chown d4/d16/d17/d19/c7c 76319 1 2024-09-10T04:05:15.172 INFO:tasks.workunit.client.1.smithi070.stdout:3/397: creat db/d1c/d25/d28/f7d x:0 0 0 2024-09-10T04:05:15.173 INFO:tasks.workunit.client.0.smithi046.stdout:0/304: dwrite d1/dc/d25/d33/d3/f46 [0,4194304] 0 2024-09-10T04:05:15.173 INFO:tasks.workunit.client.1.smithi070.stdout:3/398: chown db/d79/l7a 1 1 2024-09-10T04:05:15.173 INFO:tasks.workunit.client.0.smithi046.stdout:0/305: read - d1/d13/d37/f61 zero size 2024-09-10T04:05:15.174 INFO:tasks.workunit.client.0.smithi046.stdout:0/306: read - d1/dc/d25/d33/d4/f60 zero size 2024-09-10T04:05:15.175 INFO:tasks.workunit.client.0.smithi046.stdout:0/307: dread - d1/d13/d37/f61 zero size 2024-09-10T04:05:15.176 INFO:tasks.workunit.client.0.smithi046.stdout:7/394: write d5/d9/f42 [2291039,85456] 0 2024-09-10T04:05:15.178 INFO:tasks.workunit.client.0.smithi046.stdout:9/358: creat d4/d7/d1f/d54/d56/f72 x:0 0 0 2024-09-10T04:05:15.181 INFO:tasks.workunit.client.1.smithi070.stdout:5/379: mknod d17/d19/d67/c83 0 2024-09-10T04:05:15.181 INFO:tasks.workunit.client.0.smithi046.stdout:9/359: dread d4/d18/f25 [4194304,4194304] 0 2024-09-10T04:05:15.183 INFO:tasks.workunit.client.0.smithi046.stdout:4/402: write d3/d12/d14/f16 [501443,105773] 0 2024-09-10T04:05:15.184 INFO:tasks.workunit.client.0.smithi046.stdout:9/360: chown d4/d7/d1f/d54 1 1 2024-09-10T04:05:15.184 INFO:tasks.workunit.client.1.smithi070.stdout:2/338: write d15/d18/d42/d62/f76 [549991,122733] 0 2024-09-10T04:05:15.185 INFO:tasks.workunit.client.0.smithi046.stdout:9/361: chown d4/d7/f31 25257 1 2024-09-10T04:05:15.188 INFO:tasks.workunit.client.0.smithi046.stdout:3/342: dwrite d12/d13/f17 [0,4194304] 0 2024-09-10T04:05:15.188 INFO:tasks.workunit.client.1.smithi070.stdout:9/414: dwrite d9/d59/d10/f19 [0,4194304] 0 2024-09-10T04:05:15.189 INFO:tasks.workunit.client.0.smithi046.stdout:3/343: fsync d12/d13/d14/f33 0 2024-09-10T04:05:15.191 INFO:tasks.workunit.client.1.smithi070.stdout:7/317: sync 2024-09-10T04:05:15.195 INFO:tasks.workunit.client.1.smithi070.stdout:4/303: dwrite fb [0,4194304] 0 2024-09-10T04:05:15.195 INFO:tasks.workunit.client.0.smithi046.stdout:5/466: mknod d7/d11/d14/d2f/da3/cb0 0 2024-09-10T04:05:15.196 INFO:tasks.workunit.client.1.smithi070.stdout:8/418: mkdir d4/d16/d17/d19/d28/d43/d67/d91 0 2024-09-10T04:05:15.196 INFO:tasks.workunit.client.1.smithi070.stdout:3/399: symlink db/d1c/l7e 0 2024-09-10T04:05:15.196 INFO:tasks.workunit.client.0.smithi046.stdout:3/344: dread d12/d13/f17 [0,4194304] 0 2024-09-10T04:05:15.197 INFO:tasks.workunit.client.1.smithi070.stdout:2/339: creat d15/d18/d29/d2a/d2e/d32/f79 x:0 0 0 2024-09-10T04:05:15.198 INFO:tasks.workunit.client.1.smithi070.stdout:2/340: stat d15/d18/f28 0 2024-09-10T04:05:15.198 INFO:tasks.workunit.client.1.smithi070.stdout:5/380: mkdir d17/d24/d84 0 2024-09-10T04:05:15.201 INFO:tasks.workunit.client.0.smithi046.stdout:7/395: mknod d5/d6/d39/d5a/c85 0 2024-09-10T04:05:15.202 INFO:tasks.workunit.client.1.smithi070.stdout:5/381: chown d17/d24/c66 97064610 1 2024-09-10T04:05:15.202 INFO:tasks.workunit.client.1.smithi070.stdout:7/318: mknod d2/d7/db/c71 0 2024-09-10T04:05:15.203 INFO:tasks.workunit.client.0.smithi046.stdout:7/396: chown d5/d9 5 1 2024-09-10T04:05:15.206 INFO:tasks.workunit.client.0.smithi046.stdout:3/345: dwrite d12/d13/d37/f54 [0,4194304] 0 2024-09-10T04:05:15.206 INFO:tasks.workunit.client.0.smithi046.stdout:4/403: mknod d3/d12/d14/c8b 0 2024-09-10T04:05:15.207 INFO:tasks.workunit.client.0.smithi046.stdout:9/362: symlink d4/d3c/l73 0 2024-09-10T04:05:15.212 INFO:tasks.workunit.client.1.smithi070.stdout:9/415: read d9/d59/d5d/f66 [305652,74978] 0 2024-09-10T04:05:15.215 INFO:tasks.workunit.client.0.smithi046.stdout:1/255: link d3/d27/d32/d1f/f35 d3/dc/f56 0 2024-09-10T04:05:15.215 INFO:tasks.workunit.client.1.smithi070.stdout:9/416: dread - d9/d59/d5d/f67 zero size 2024-09-10T04:05:15.215 INFO:tasks.workunit.client.1.smithi070.stdout:7/319: symlink d2/d3/d63/l72 0 2024-09-10T04:05:15.217 INFO:tasks.workunit.client.1.smithi070.stdout:1/315: sync 2024-09-10T04:05:15.217 INFO:tasks.workunit.client.1.smithi070.stdout:6/451: sync 2024-09-10T04:05:15.217 INFO:tasks.workunit.client.1.smithi070.stdout:0/428: sync 2024-09-10T04:05:15.217 INFO:tasks.workunit.client.1.smithi070.stdout:8/419: truncate d4/f12 689188 0 2024-09-10T04:05:15.219 INFO:tasks.workunit.client.0.smithi046.stdout:7/397: fsync d5/d6/db/dd/d4b/f52 0 2024-09-10T04:05:15.220 INFO:tasks.workunit.client.1.smithi070.stdout:8/420: write d4/d16/d17/f68 [1566224,77526] 0 2024-09-10T04:05:15.221 INFO:tasks.workunit.client.1.smithi070.stdout:3/400: link db/f1f db/d1c/d3e/f7f 0 2024-09-10T04:05:15.222 INFO:tasks.workunit.client.1.smithi070.stdout:3/401: stat db/d1c/d25/l2f 0 2024-09-10T04:05:15.222 INFO:tasks.workunit.client.1.smithi070.stdout:0/429: dread - d8/d13/d1a/f45 zero size 2024-09-10T04:05:15.223 INFO:tasks.workunit.client.1.smithi070.stdout:8/421: readlink d4/d16/d17/d1a/l35 0 2024-09-10T04:05:15.225 INFO:tasks.workunit.client.1.smithi070.stdout:9/417: symlink d9/d59/d10/l80 0 2024-09-10T04:05:15.226 INFO:tasks.workunit.client.1.smithi070.stdout:9/418: stat d9/d59/d2a/l4b 0 2024-09-10T04:05:15.227 INFO:tasks.workunit.client.1.smithi070.stdout:6/452: dread d0/d3/d18/f27 [0,4194304] 0 2024-09-10T04:05:15.227 INFO:tasks.workunit.client.0.smithi046.stdout:8/370: creat d1/d29/d19/d36/d4c/d5e/f7f x:0 0 0 2024-09-10T04:05:15.228 INFO:tasks.workunit.client.0.smithi046.stdout:6/355: dwrite da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:15.228 INFO:tasks.workunit.client.1.smithi070.stdout:6/453: chown d0/d3/d18/f4c 140914742 1 2024-09-10T04:05:15.232 INFO:tasks.workunit.client.1.smithi070.stdout:4/304: dwrite de/f31 [0,4194304] 0 2024-09-10T04:05:15.233 INFO:tasks.workunit.client.0.smithi046.stdout:2/403: truncate d2/d6/d23/d5d/d77/d7b/f92 1425887 0 2024-09-10T04:05:15.234 INFO:tasks.workunit.client.1.smithi070.stdout:1/316: mkdir d3/d8/d12/d61 0 2024-09-10T04:05:15.234 INFO:tasks.workunit.client.0.smithi046.stdout:2/404: dread - d2/d6/d23/d55/f7e zero size 2024-09-10T04:05:15.235 INFO:tasks.workunit.client.0.smithi046.stdout:2/405: write d2/d8/f27 [231631,38131] 0 2024-09-10T04:05:15.238 INFO:tasks.workunit.client.0.smithi046.stdout:1/256: read d3/d27/d32/d1f/f29 [3358517,68019] 0 2024-09-10T04:05:15.238 INFO:tasks.workunit.client.0.smithi046.stdout:5/467: mknod d7/d11/d3f/cb1 0 2024-09-10T04:05:15.244 INFO:tasks.workunit.client.1.smithi070.stdout:8/422: rename d4/d16/d17/d1a/c42 to d4/d16/d73/c92 0 2024-09-10T04:05:15.245 INFO:tasks.workunit.client.0.smithi046.stdout:2/406: dwrite d2/d6/d23/d5d/f8f [0,4194304] 0 2024-09-10T04:05:15.245 INFO:tasks.workunit.client.0.smithi046.stdout:4/404: rmdir d3/d12/d17 39 2024-09-10T04:05:15.245 INFO:tasks.workunit.client.1.smithi070.stdout:8/423: fdatasync d4/f11 0 2024-09-10T04:05:15.247 INFO:tasks.workunit.client.0.smithi046.stdout:4/405: write d3/d43/f48 [601289,25835] 0 2024-09-10T04:05:15.248 INFO:tasks.workunit.client.1.smithi070.stdout:9/419: creat d9/d59/d10/f81 x:0 0 0 2024-09-10T04:05:15.248 INFO:tasks.workunit.client.0.smithi046.stdout:2/407: truncate d2/d6/d23/d5d/d5e/f84 1016186 0 2024-09-10T04:05:15.254 INFO:tasks.workunit.client.1.smithi070.stdout:5/382: dwrite d17/d6f/f52 [0,4194304] 0 2024-09-10T04:05:15.256 INFO:tasks.workunit.client.1.smithi070.stdout:4/305: mkdir de/d13/d4f/d6a 0 2024-09-10T04:05:15.256 INFO:tasks.workunit.client.1.smithi070.stdout:2/341: dwrite d15/d18/d29/f46 [4194304,4194304] 0 2024-09-10T04:05:15.256 INFO:tasks.workunit.client.1.smithi070.stdout:5/383: readlink d17/l5c 0 2024-09-10T04:05:15.258 INFO:tasks.workunit.client.1.smithi070.stdout:4/306: chown de/d13/d5b/c62 931599663 1 2024-09-10T04:05:15.258 INFO:tasks.workunit.client.1.smithi070.stdout:2/342: chown d15/d18/d42/d62/d55 979 1 2024-09-10T04:05:15.258 INFO:tasks.workunit.client.1.smithi070.stdout:5/384: write d17/d19/f1d [1423055,48467] 0 2024-09-10T04:05:15.259 INFO:tasks.workunit.client.0.smithi046.stdout:0/308: write d1/dc/d25/d33/d3/f2a [83366,35599] 0 2024-09-10T04:05:15.259 INFO:tasks.workunit.client.0.smithi046.stdout:8/371: fdatasync d1/d29/d19/d36/d62/f71 0 2024-09-10T04:05:15.259 INFO:tasks.workunit.client.1.smithi070.stdout:1/317: unlink d3/d8/d12/l39 0 2024-09-10T04:05:15.261 INFO:tasks.workunit.client.0.smithi046.stdout:8/372: readlink d1/d29/d19/l53 0 2024-09-10T04:05:15.261 INFO:tasks.workunit.client.1.smithi070.stdout:5/385: chown f7 5421463 1 2024-09-10T04:05:15.261 INFO:tasks.workunit.client.1.smithi070.stdout:4/307: dread - de/f64 zero size 2024-09-10T04:05:15.261 INFO:tasks.workunit.client.1.smithi070.stdout:7/320: link d2/d7/db/d3a/f68 d2/d7/d9/d18/d3c/f73 0 2024-09-10T04:05:15.261 INFO:tasks.workunit.client.1.smithi070.stdout:5/386: fdatasync d17/d19/d35/f6c 0 2024-09-10T04:05:15.263 INFO:tasks.workunit.client.1.smithi070.stdout:4/308: chown de/d13/d43/c5d 0 1 2024-09-10T04:05:15.263 INFO:tasks.workunit.client.1.smithi070.stdout:4/309: stat de/d13/d18/d26/f2f 0 2024-09-10T04:05:15.264 INFO:tasks.workunit.client.1.smithi070.stdout:9/420: dread d9/d59/d5d/f66 [0,4194304] 0 2024-09-10T04:05:15.266 INFO:tasks.workunit.client.0.smithi046.stdout:1/257: mknod d3/d27/d2a/c57 0 2024-09-10T04:05:15.269 INFO:tasks.workunit.client.0.smithi046.stdout:5/468: creat d7/d11/d3f/d5a/fb2 x:0 0 0 2024-09-10T04:05:15.270 INFO:tasks.workunit.client.0.smithi046.stdout:1/258: dread d3/d27/d32/f2b [0,4194304] 0 2024-09-10T04:05:15.270 INFO:tasks.workunit.client.1.smithi070.stdout:7/321: dread d2/d7/db/f37 [4194304,4194304] 0 2024-09-10T04:05:15.271 INFO:tasks.workunit.client.0.smithi046.stdout:8/373: dwrite d1/d29/d17/d1f/d2a/f58 [4194304,4194304] 0 2024-09-10T04:05:15.273 INFO:tasks.workunit.client.0.smithi046.stdout:8/374: write d1/f39 [137787,62234] 0 2024-09-10T04:05:15.278 INFO:tasks.workunit.client.1.smithi070.stdout:8/424: creat d4/d16/d73/f93 x:0 0 0 2024-09-10T04:05:15.279 INFO:tasks.workunit.client.0.smithi046.stdout:3/346: creat d12/d13/d14/d1e/d2d/d39/d67/f6b x:0 0 0 2024-09-10T04:05:15.280 INFO:tasks.workunit.client.0.smithi046.stdout:3/347: write f4 [963816,51525] 0 2024-09-10T04:05:15.281 INFO:tasks.workunit.client.0.smithi046.stdout:3/348: readlink d12/d1b/l28 0 2024-09-10T04:05:15.281 INFO:tasks.workunit.client.0.smithi046.stdout:2/408: chown d2/d6/d23/d5d/d77/d7b/d85/c8c 981 1 2024-09-10T04:05:15.282 INFO:tasks.workunit.client.0.smithi046.stdout:3/349: stat d12/d13/d14/f33 0 2024-09-10T04:05:15.286 INFO:tasks.workunit.client.1.smithi070.stdout:2/343: creat d15/d2b/f7a x:0 0 0 2024-09-10T04:05:15.288 INFO:tasks.workunit.client.0.smithi046.stdout:9/363: truncate d4/d29/f4b 3366856 0 2024-09-10T04:05:15.288 INFO:tasks.workunit.client.0.smithi046.stdout:9/364: chown d4/d7/f31 122548 1 2024-09-10T04:05:15.289 INFO:tasks.workunit.client.1.smithi070.stdout:1/318: symlink d3/d10/d1d/l62 0 2024-09-10T04:05:15.289 INFO:tasks.workunit.client.1.smithi070.stdout:0/430: truncate d8/d13/d1a/f3c 3320452 0 2024-09-10T04:05:15.292 INFO:tasks.workunit.client.1.smithi070.stdout:2/344: dwrite d15/d18/d29/d2a/f67 [0,4194304] 0 2024-09-10T04:05:15.292 INFO:tasks.workunit.client.1.smithi070.stdout:9/421: unlink d9/f43 0 2024-09-10T04:05:15.293 INFO:tasks.workunit.client.1.smithi070.stdout:2/345: fsync d15/d18/d29/f46 0 2024-09-10T04:05:15.294 INFO:tasks.workunit.client.0.smithi046.stdout:5/469: symlink d7/d11/d14/d2f/lb3 0 2024-09-10T04:05:15.296 INFO:tasks.workunit.client.1.smithi070.stdout:4/310: rename de/d13/d18/d26/f2a to de/d13/d43/f6b 0 2024-09-10T04:05:15.297 INFO:tasks.workunit.client.0.smithi046.stdout:1/259: fdatasync d3/d27/d32/f2b 0 2024-09-10T04:05:15.298 INFO:tasks.workunit.client.1.smithi070.stdout:3/402: link db/d1c/d53/c6f db/d1c/d25/d28/d43/c80 0 2024-09-10T04:05:15.300 INFO:tasks.workunit.client.0.smithi046.stdout:8/375: mkdir d1/d29/d17/d1f/d46/d67/d80 0 2024-09-10T04:05:15.304 INFO:tasks.workunit.client.0.smithi046.stdout:4/406: creat d3/d12/d17/d29/d58/f8c x:0 0 0 2024-09-10T04:05:15.306 INFO:tasks.workunit.client.0.smithi046.stdout:3/350: mknod d12/d13/d37/d51/c6c 0 2024-09-10T04:05:15.307 INFO:tasks.workunit.client.0.smithi046.stdout:3/351: dread - d12/d13/d37/d51/f62 zero size 2024-09-10T04:05:15.309 INFO:tasks.workunit.client.0.smithi046.stdout:3/352: truncate d12/d13/d14/f25 2054086 0 2024-09-10T04:05:15.313 INFO:tasks.workunit.client.1.smithi070.stdout:1/319: mknod d3/d20/c63 0 2024-09-10T04:05:15.315 INFO:tasks.workunit.client.0.smithi046.stdout:9/365: rmdir d4/d7/d1f/d54 39 2024-09-10T04:05:15.320 INFO:tasks.workunit.client.1.smithi070.stdout:9/422: mkdir d9/d59/d10/d82 0 2024-09-10T04:05:15.321 INFO:tasks.workunit.client.1.smithi070.stdout:9/423: dread - d9/d59/f7e zero size 2024-09-10T04:05:15.322 INFO:tasks.workunit.client.0.smithi046.stdout:9/366: dwrite d4/d7/f31 [0,4194304] 0 2024-09-10T04:05:15.324 INFO:tasks.workunit.client.0.smithi046.stdout:1/260: mkdir d3/dc/d34/d58 0 2024-09-10T04:05:15.326 INFO:tasks.workunit.client.0.smithi046.stdout:8/376: truncate d1/d29/f27 5060614 0 2024-09-10T04:05:15.327 INFO:tasks.workunit.client.1.smithi070.stdout:2/346: rmdir d15/d18/d29/d2a/d2e/d32 39 2024-09-10T04:05:15.328 INFO:tasks.workunit.client.0.smithi046.stdout:0/309: truncate d1/dc/d25/d33/d1a/f31 2032779 0 2024-09-10T04:05:15.331 INFO:tasks.workunit.client.1.smithi070.stdout:5/387: rename d17/d24/c66 to d17/d19/d35/c85 0 2024-09-10T04:05:15.332 INFO:tasks.workunit.client.0.smithi046.stdout:6/356: link da/dc/cf da/dc/c75 0 2024-09-10T04:05:15.333 INFO:tasks.workunit.client.1.smithi070.stdout:3/403: rmdir db/d1c/d25/d28 39 2024-09-10T04:05:15.334 INFO:tasks.workunit.client.0.smithi046.stdout:1/261: dwrite d3/d27/f42 [0,4194304] 0 2024-09-10T04:05:15.335 INFO:tasks.workunit.client.0.smithi046.stdout:3/353: mknod d12/d13/d14/d1e/d2d/c6d 0 2024-09-10T04:05:15.337 INFO:tasks.workunit.client.1.smithi070.stdout:7/322: creat d2/d4e/f74 x:0 0 0 2024-09-10T04:05:15.342 INFO:tasks.workunit.client.0.smithi046.stdout:5/470: unlink d7/d11/d14/d2f/c2e 0 2024-09-10T04:05:15.342 INFO:tasks.workunit.client.1.smithi070.stdout:6/454: link d0/d3/d18/d2a/c75 d0/d3/d18/c8f 0 2024-09-10T04:05:15.344 INFO:tasks.workunit.client.0.smithi046.stdout:9/367: dread d4/d44/f49 [0,4194304] 0 2024-09-10T04:05:15.345 INFO:tasks.workunit.client.0.smithi046.stdout:1/262: dwrite d3/d27/d32/f2e [0,4194304] 0 2024-09-10T04:05:15.346 INFO:tasks.workunit.client.0.smithi046.stdout:9/368: chown d4/d3c/l73 106 1 2024-09-10T04:05:15.346 INFO:tasks.workunit.client.1.smithi070.stdout:1/320: symlink d3/d8/d12/d2f/d32/d4b/l64 0 2024-09-10T04:05:15.347 INFO:tasks.workunit.client.0.smithi046.stdout:9/369: write d4/d7/f31 [364313,92927] 0 2024-09-10T04:05:15.348 INFO:tasks.workunit.client.1.smithi070.stdout:9/424: symlink d9/d59/d2a/d3f/l83 0 2024-09-10T04:05:15.349 INFO:tasks.workunit.client.0.smithi046.stdout:1/263: chown d3/d27/d32/d28 91 1 2024-09-10T04:05:15.354 INFO:tasks.workunit.client.1.smithi070.stdout:9/425: dwrite d9/d59/d10/d3c/f52 [0,4194304] 0 2024-09-10T04:05:15.356 INFO:tasks.workunit.client.1.smithi070.stdout:9/426: chown d9/d59/d2a/c2b 3460 1 2024-09-10T04:05:15.357 INFO:tasks.workunit.client.1.smithi070.stdout:9/427: fsync d9/d59/f65 0 2024-09-10T04:05:15.360 INFO:tasks.workunit.client.1.smithi070.stdout:3/404: dread - db/d1c/d25/d28/d43/f77 zero size 2024-09-10T04:05:15.365 INFO:tasks.workunit.client.0.smithi046.stdout:6/357: dread da/db/d11/d2b/f2c [0,4194304] 0 2024-09-10T04:05:15.366 INFO:tasks.workunit.client.1.smithi070.stdout:8/425: creat d4/d16/d17/d1a/f94 x:0 0 0 2024-09-10T04:05:15.367 INFO:tasks.workunit.client.0.smithi046.stdout:3/354: creat d12/d5d/f6e x:0 0 0 2024-09-10T04:05:15.369 INFO:tasks.workunit.client.1.smithi070.stdout:0/431: dwrite f5 [0,4194304] 0 2024-09-10T04:05:15.369 INFO:tasks.workunit.client.1.smithi070.stdout:0/432: fdatasync d8/f27 0 2024-09-10T04:05:15.370 INFO:tasks.workunit.client.0.smithi046.stdout:5/471: rename d7/l5d to d7/d11/d3f/d53/dac/lb4 0 2024-09-10T04:05:15.371 INFO:tasks.workunit.client.0.smithi046.stdout:5/472: fsync d7/f70 0 2024-09-10T04:05:15.376 INFO:tasks.workunit.client.0.smithi046.stdout:9/370: read d4/f6 [1243173,113851] 0 2024-09-10T04:05:15.377 INFO:tasks.workunit.client.0.smithi046.stdout:1/264: rmdir d3 39 2024-09-10T04:05:15.379 INFO:tasks.workunit.client.1.smithi070.stdout:5/388: mknod d17/d19/d6a/d54/d57/d63/d7b/c86 0 2024-09-10T04:05:15.380 INFO:tasks.workunit.client.1.smithi070.stdout:9/428: creat d9/d60/f84 x:0 0 0 2024-09-10T04:05:15.381 INFO:tasks.workunit.client.0.smithi046.stdout:3/355: dread d12/d13/d14/f33 [0,4194304] 0 2024-09-10T04:05:15.383 INFO:tasks.workunit.client.1.smithi070.stdout:3/405: rmdir db/d1c/d25/d28/d43 39 2024-09-10T04:05:15.384 INFO:tasks.workunit.client.1.smithi070.stdout:4/311: sync 2024-09-10T04:05:15.385 INFO:tasks.workunit.client.1.smithi070.stdout:4/312: chown fb 502 1 2024-09-10T04:05:15.385 INFO:tasks.workunit.client.1.smithi070.stdout:7/323: unlink d2/l29 0 2024-09-10T04:05:15.390 INFO:tasks.workunit.client.1.smithi070.stdout:7/324: dwrite d2/d3/f3b [0,4194304] 0 2024-09-10T04:05:15.390 INFO:tasks.workunit.client.0.smithi046.stdout:0/310: truncate d1/dc/d25/d33/f11 2929298 0 2024-09-10T04:05:15.392 INFO:tasks.workunit.client.1.smithi070.stdout:8/426: symlink d4/d16/d17/d19/d28/d43/l95 0 2024-09-10T04:05:15.396 INFO:tasks.workunit.client.0.smithi046.stdout:5/473: symlink d7/d11/d14/d2f/d2b/lb5 0 2024-09-10T04:05:15.397 INFO:tasks.workunit.client.0.smithi046.stdout:5/474: dread - d7/d11/d14/d2f/d2b/d6a/fa1 zero size 2024-09-10T04:05:15.397 INFO:tasks.workunit.client.1.smithi070.stdout:1/321: mknod d3/d10/c65 0 2024-09-10T04:05:15.398 INFO:tasks.workunit.client.1.smithi070.stdout:2/347: fdatasync d15/d18/d29/d2a/d2e/d32/f49 0 2024-09-10T04:05:15.399 INFO:tasks.workunit.client.0.smithi046.stdout:9/371: readlink d4/d18/l24 0 2024-09-10T04:05:15.399 INFO:tasks.workunit.client.0.smithi046.stdout:8/377: read d1/d29/f27 [4006339,61881] 0 2024-09-10T04:05:15.401 INFO:tasks.workunit.client.0.smithi046.stdout:8/378: chown d1/d29/d17/d1f/d46/d67 7766331 1 2024-09-10T04:05:15.401 INFO:tasks.workunit.client.0.smithi046.stdout:1/265: stat d3/d27/c30 0 2024-09-10T04:05:15.402 INFO:tasks.workunit.client.1.smithi070.stdout:9/429: symlink d9/d59/d2a/d3f/l85 0 2024-09-10T04:05:15.402 INFO:tasks.workunit.client.1.smithi070.stdout:9/430: stat d9/d1c/l4f 0 2024-09-10T04:05:15.403 INFO:tasks.workunit.client.0.smithi046.stdout:4/407: getdents d3/d12/d17/d22/d51 0 2024-09-10T04:05:15.404 INFO:tasks.workunit.client.0.smithi046.stdout:9/372: dread d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:15.404 INFO:tasks.workunit.client.0.smithi046.stdout:3/356: mkdir d12/d13/d14/d1e/d2d/d6f 0 2024-09-10T04:05:15.405 INFO:tasks.workunit.client.0.smithi046.stdout:9/373: fsync d4/d18/f4f 0 2024-09-10T04:05:15.408 INFO:tasks.workunit.client.0.smithi046.stdout:9/374: write d4/d3c/d4d/d5c/f4e [717268,45888] 0 2024-09-10T04:05:15.408 INFO:tasks.workunit.client.0.smithi046.stdout:0/311: mkdir d1/dc/d25/d33/d1a/d64 0 2024-09-10T04:05:15.410 INFO:tasks.workunit.client.1.smithi070.stdout:4/313: dread de/d13/d4f/f66 [0,4194304] 0 2024-09-10T04:05:15.412 INFO:tasks.workunit.client.1.smithi070.stdout:6/455: creat d0/d3/d18/d1b/d17/f90 x:0 0 0 2024-09-10T04:05:15.413 INFO:tasks.workunit.client.1.smithi070.stdout:7/325: readlink d2/d7/d9/d18/d49/l5b 0 2024-09-10T04:05:15.414 INFO:tasks.workunit.client.1.smithi070.stdout:5/389: write f14 [2968390,114296] 0 2024-09-10T04:05:15.415 INFO:tasks.workunit.client.1.smithi070.stdout:8/427: unlink d4/d16/d17/c5a 0 2024-09-10T04:05:15.416 INFO:tasks.workunit.client.1.smithi070.stdout:5/390: write d17/d24/d45/f79 [440845,85874] 0 2024-09-10T04:05:15.420 INFO:tasks.workunit.client.1.smithi070.stdout:5/391: truncate d17/d24/d4e/f82 811990 0 2024-09-10T04:05:15.420 INFO:tasks.workunit.client.0.smithi046.stdout:3/357: creat d12/d13/f70 x:0 0 0 2024-09-10T04:05:15.421 INFO:tasks.workunit.client.1.smithi070.stdout:7/326: dwrite d2/d7/f43 [0,4194304] 0 2024-09-10T04:05:15.422 INFO:tasks.workunit.client.0.smithi046.stdout:9/375: mknod d4/d3c/c74 0 2024-09-10T04:05:15.424 INFO:tasks.workunit.client.0.smithi046.stdout:4/408: write d3/d12/d17/d22/f27 [2255889,38462] 0 2024-09-10T04:05:15.425 INFO:tasks.workunit.client.0.smithi046.stdout:4/409: dread - d3/d12/d14/f7c zero size 2024-09-10T04:05:15.426 INFO:tasks.workunit.client.1.smithi070.stdout:9/431: creat d9/d59/d2a/d3f/f86 x:0 0 0 2024-09-10T04:05:15.427 INFO:tasks.workunit.client.0.smithi046.stdout:4/410: fsync d3/d12/d17/d29/d58/f71 0 2024-09-10T04:05:15.427 INFO:tasks.workunit.client.0.smithi046.stdout:4/411: readlink d3/d12/l1e 0 2024-09-10T04:05:15.427 INFO:tasks.workunit.client.1.smithi070.stdout:9/432: chown d9/d59/f1a 1271121 1 2024-09-10T04:05:15.428 INFO:tasks.workunit.client.0.smithi046.stdout:6/358: rename da/db/d11/d13/d1e/d23/l30 to da/db/d11/d13/d1e/l76 0 2024-09-10T04:05:15.429 INFO:tasks.workunit.client.1.smithi070.stdout:3/406: symlink db/d1c/d25/l81 0 2024-09-10T04:05:15.430 INFO:tasks.workunit.client.1.smithi070.stdout:3/407: readlink db/d1c/d25/l2f 0 2024-09-10T04:05:15.431 INFO:tasks.workunit.client.0.smithi046.stdout:1/266: creat d3/dc/d34/d58/f59 x:0 0 0 2024-09-10T04:05:15.432 INFO:tasks.workunit.client.1.smithi070.stdout:6/456: mknod d0/d3/d18/d1b/d33/c91 0 2024-09-10T04:05:15.432 INFO:tasks.workunit.client.0.smithi046.stdout:1/267: fsync d3/d27/d32/f2e 0 2024-09-10T04:05:15.433 INFO:tasks.workunit.client.0.smithi046.stdout:1/268: stat d3/dc/d21 0 2024-09-10T04:05:15.434 INFO:tasks.workunit.client.1.smithi070.stdout:8/428: rmdir d4/d16/d17/d19/d28 39 2024-09-10T04:05:15.434 INFO:tasks.workunit.client.1.smithi070.stdout:6/457: dread - d0/d3/d18/d1b/d17/d54/f6c zero size 2024-09-10T04:05:15.435 INFO:tasks.workunit.client.1.smithi070.stdout:9/433: dwrite d9/d60/f84 [0,4194304] 0 2024-09-10T04:05:15.436 INFO:tasks.workunit.client.1.smithi070.stdout:9/434: readlink d9/d59/d10/l2f 0 2024-09-10T04:05:15.436 INFO:tasks.workunit.client.0.smithi046.stdout:0/312: dwrite d1/dc/d25/d33/d3/f14 [4194304,4194304] 0 2024-09-10T04:05:15.436 INFO:tasks.workunit.client.0.smithi046.stdout:1/269: read d3/f6 [604958,74112] 0 2024-09-10T04:05:15.437 INFO:tasks.workunit.client.1.smithi070.stdout:0/433: rename d8/d24/d5e to d8/d24/d2d/d8d 0 2024-09-10T04:05:15.437 INFO:tasks.workunit.client.1.smithi070.stdout:9/435: dread - d9/d59/d5d/f67 zero size 2024-09-10T04:05:15.440 INFO:tasks.workunit.client.1.smithi070.stdout:1/322: unlink d3/d8/d12/f19 0 2024-09-10T04:05:15.441 INFO:tasks.workunit.client.0.smithi046.stdout:4/412: creat d3/d43/f8d x:0 0 0 2024-09-10T04:05:15.445 INFO:tasks.workunit.client.0.smithi046.stdout:9/376: dwrite d4/d29/f6d [0,4194304] 0 2024-09-10T04:05:15.445 INFO:tasks.workunit.client.0.smithi046.stdout:3/358: dread d12/d1b/d50/f55 [0,4194304] 0 2024-09-10T04:05:15.446 INFO:tasks.workunit.client.1.smithi070.stdout:5/392: symlink d17/d37/l87 0 2024-09-10T04:05:15.453 INFO:tasks.workunit.client.1.smithi070.stdout:7/327: symlink d2/d3/dc/d5d/d42/l75 0 2024-09-10T04:05:15.453 INFO:tasks.workunit.client.0.smithi046.stdout:6/359: dwrite da/db/d11/d2b/d2f/d3b/f5e [0,4194304] 0 2024-09-10T04:05:15.454 INFO:tasks.workunit.client.1.smithi070.stdout:4/314: mknod de/d13/d18/d5c/c6c 0 2024-09-10T04:05:15.454 INFO:tasks.workunit.client.1.smithi070.stdout:5/393: dwrite d17/d19/d6a/d54/d57/f80 [0,4194304] 0 2024-09-10T04:05:15.454 INFO:tasks.workunit.client.1.smithi070.stdout:7/328: stat d2/d40/d6c 0 2024-09-10T04:05:15.456 INFO:tasks.workunit.client.1.smithi070.stdout:3/408: write db/d1c/d25/d28/f41 [1032024,91590] 0 2024-09-10T04:05:15.458 INFO:tasks.workunit.client.1.smithi070.stdout:8/429: stat d4/d16/d17/d19/d28/f30 0 2024-09-10T04:05:15.462 INFO:tasks.workunit.client.0.smithi046.stdout:0/313: mknod d1/dc/d25/d33/d1a/c65 0 2024-09-10T04:05:15.464 INFO:tasks.workunit.client.0.smithi046.stdout:0/314: write d1/dc/d25/d33/d3/f14 [7139119,66724] 0 2024-09-10T04:05:15.465 INFO:tasks.workunit.client.1.smithi070.stdout:9/436: fdatasync d9/d59/d5d/f66 0 2024-09-10T04:05:15.465 INFO:tasks.workunit.client.0.smithi046.stdout:0/315: stat d1/dc/d25/d33/d3/d29/d2d/f4f 0 2024-09-10T04:05:15.467 INFO:tasks.workunit.client.1.smithi070.stdout:9/437: chown d9/d59/d2a/d3f/f48 2312 1 2024-09-10T04:05:15.467 INFO:tasks.workunit.client.1.smithi070.stdout:1/323: creat d3/d8/d12/f66 x:0 0 0 2024-09-10T04:05:15.469 INFO:tasks.workunit.client.0.smithi046.stdout:5/475: creat d7/d11/d3f/d53/da9/d9c/da5/fb6 x:0 0 0 2024-09-10T04:05:15.474 INFO:tasks.workunit.client.0.smithi046.stdout:3/359: rename d12/d13/d42 to d12/d5d/d71 0 2024-09-10T04:05:15.474 INFO:tasks.workunit.client.1.smithi070.stdout:5/394: rmdir d17/d19/d67 39 2024-09-10T04:05:15.475 INFO:tasks.workunit.client.1.smithi070.stdout:5/395: read - d17/d24/f2c zero size 2024-09-10T04:05:15.475 INFO:tasks.workunit.client.0.smithi046.stdout:3/360: fsync d12/f4d 0 2024-09-10T04:05:15.477 INFO:tasks.workunit.client.0.smithi046.stdout:9/377: dwrite d4/d7/d1f/d54/d56/f72 [0,4194304] 0 2024-09-10T04:05:15.477 INFO:tasks.workunit.client.1.smithi070.stdout:5/396: dread d17/d19/d6a/f5e [0,4194304] 0 2024-09-10T04:05:15.478 INFO:tasks.workunit.client.0.smithi046.stdout:0/316: dwrite d1/dc/d25/d33/d3/d29/f3c [0,4194304] 0 2024-09-10T04:05:15.480 INFO:tasks.workunit.client.0.smithi046.stdout:0/317: write d1/dc/d25/d33/d1a/d41/f5a [18928,36042] 0 2024-09-10T04:05:15.481 INFO:tasks.workunit.client.1.smithi070.stdout:4/315: rename de/f64 to de/d13/d18/d23/d39/f6d 0 2024-09-10T04:05:15.483 INFO:tasks.workunit.client.1.smithi070.stdout:3/409: fsync db/d1c/d3e/f7f 0 2024-09-10T04:05:15.484 INFO:tasks.workunit.client.1.smithi070.stdout:2/348: sync 2024-09-10T04:05:15.484 INFO:tasks.workunit.client.0.smithi046.stdout:4/413: mknod d3/d43/d5f/d85/c8e 0 2024-09-10T04:05:15.486 INFO:tasks.workunit.client.1.smithi070.stdout:6/458: write d0/d3/d18/f27 [3628796,1233] 0 2024-09-10T04:05:15.488 INFO:tasks.workunit.client.1.smithi070.stdout:0/434: unlink d8/f2b 0 2024-09-10T04:05:15.493 INFO:tasks.workunit.client.1.smithi070.stdout:9/438: rmdir d9/d60 39 2024-09-10T04:05:15.496 INFO:tasks.workunit.client.1.smithi070.stdout:1/324: mkdir d3/d8/d13/d67 0 2024-09-10T04:05:15.499 INFO:tasks.workunit.client.1.smithi070.stdout:9/439: dwrite d9/d1c/d34/f33 [0,4194304] 0 2024-09-10T04:05:15.499 INFO:tasks.workunit.client.0.smithi046.stdout:6/360: rename da/db/f14 to da/db/d10/d27/f77 0 2024-09-10T04:05:15.500 INFO:tasks.workunit.client.1.smithi070.stdout:7/329: symlink d2/d40/d6c/l76 0 2024-09-10T04:05:15.501 INFO:tasks.workunit.client.0.smithi046.stdout:9/378: rmdir d4/d3c/d4d/d5c/d59 39 2024-09-10T04:05:15.502 INFO:tasks.workunit.client.0.smithi046.stdout:5/476: write d7/d11/d14/f29 [2849364,73757] 0 2024-09-10T04:05:15.503 INFO:tasks.workunit.client.1.smithi070.stdout:5/397: creat d17/d19/d6a/d54/d57/f88 x:0 0 0 2024-09-10T04:05:15.508 INFO:tasks.workunit.client.0.smithi046.stdout:0/318: rmdir d1/dc/d25/d33/d3/dd 39 2024-09-10T04:05:15.509 INFO:tasks.workunit.client.1.smithi070.stdout:8/430: rename d4/d16/d17/d5c/d87/d8e/d44 to d4/d16/d73/d96 0 2024-09-10T04:05:15.509 INFO:tasks.workunit.client.1.smithi070.stdout:8/431: write d4/d16/d17/d19/d28/d43/d55/f82 [939980,24410] 0 2024-09-10T04:05:15.511 INFO:tasks.workunit.client.0.smithi046.stdout:6/361: dwrite da/dc/d22/f52 [0,4194304] 0 2024-09-10T04:05:15.511 INFO:tasks.workunit.client.1.smithi070.stdout:3/410: creat db/f82 x:0 0 0 2024-09-10T04:05:15.513 INFO:tasks.workunit.client.0.smithi046.stdout:3/361: rename d12/d1b/f21 to d12/d13/d37/d51/f72 0 2024-09-10T04:05:15.515 INFO:tasks.workunit.client.1.smithi070.stdout:2/349: unlink l5 0 2024-09-10T04:05:15.516 INFO:tasks.workunit.client.0.smithi046.stdout:0/319: dread d1/dc/d25/d33/d4/d19/f58 [0,4194304] 0 2024-09-10T04:05:15.516 INFO:tasks.workunit.client.0.smithi046.stdout:3/362: write d12/d13/d37/f54 [4322433,74470] 0 2024-09-10T04:05:15.517 INFO:tasks.workunit.client.0.smithi046.stdout:6/362: truncate da/db/d11/d2b/d2f/f72 402619 0 2024-09-10T04:05:15.519 INFO:tasks.workunit.client.1.smithi070.stdout:6/459: unlink d0/d3/d18/d1b/d17/l24 0 2024-09-10T04:05:15.520 INFO:tasks.workunit.client.1.smithi070.stdout:0/435: dread d8/d13/d33/f65 [0,4194304] 0 2024-09-10T04:05:15.520 INFO:tasks.workunit.client.1.smithi070.stdout:6/460: chown d0/d3/d18/d1b/d17/c3d 625150 1 2024-09-10T04:05:15.521 INFO:tasks.workunit.client.0.smithi046.stdout:1/270: getdents d3/d27/d32/d1f 0 2024-09-10T04:05:15.522 INFO:tasks.workunit.client.0.smithi046.stdout:5/477: fsync d7/f1f 0 2024-09-10T04:05:15.523 INFO:tasks.workunit.client.1.smithi070.stdout:1/325: fsync d3/d10/f5e 0 2024-09-10T04:05:15.526 INFO:tasks.workunit.client.0.smithi046.stdout:4/414: link d3/d12/d17/d22/d30/f72 d3/d12/d17/d22/d51/f8f 0 2024-09-10T04:05:15.529 INFO:tasks.workunit.client.0.smithi046.stdout:4/415: stat d3/d12/d17/d29/d58/c77 0 2024-09-10T04:05:15.529 INFO:tasks.workunit.client.0.smithi046.stdout:9/379: dwrite d4/d18/f36 [0,4194304] 0 2024-09-10T04:05:15.532 INFO:tasks.workunit.client.0.smithi046.stdout:0/320: creat d1/dc/d25/d33/d1a/f66 x:0 0 0 2024-09-10T04:05:15.536 INFO:tasks.workunit.client.0.smithi046.stdout:0/321: write d1/dc/d25/d33/d3/f63 [928021,47289] 0 2024-09-10T04:05:15.537 INFO:tasks.workunit.client.1.smithi070.stdout:4/316: getdents de/d13/d4f/d6a 0 2024-09-10T04:05:15.539 INFO:tasks.workunit.client.0.smithi046.stdout:6/363: dwrite da/db/d10/d27/f77 [4194304,4194304] 0 2024-09-10T04:05:15.539 INFO:tasks.workunit.client.0.smithi046.stdout:1/271: symlink d3/dc/d21/l5a 0 2024-09-10T04:05:15.543 INFO:tasks.workunit.client.0.smithi046.stdout:5/478: mknod d7/d11/d3f/d5a/cb7 0 2024-09-10T04:05:15.544 INFO:tasks.workunit.client.0.smithi046.stdout:5/479: chown d7/f97 1656436480 1 2024-09-10T04:05:15.548 INFO:tasks.workunit.client.1.smithi070.stdout:8/432: chown d4/d16/d17/d19/f25 1583 1 2024-09-10T04:05:15.549 INFO:tasks.workunit.client.1.smithi070.stdout:8/433: readlink d4/d39/l3b 0 2024-09-10T04:05:15.554 INFO:tasks.workunit.client.1.smithi070.stdout:2/350: chown ce 60802025 1 2024-09-10T04:05:15.557 INFO:tasks.workunit.client.0.smithi046.stdout:9/380: rmdir d4/d44 39 2024-09-10T04:05:15.560 INFO:tasks.workunit.client.1.smithi070.stdout:5/398: dwrite d17/d19/d35/f3a [0,4194304] 0 2024-09-10T04:05:15.561 INFO:tasks.workunit.client.0.smithi046.stdout:0/322: symlink d1/dc/d49/l67 0 2024-09-10T04:05:15.563 INFO:tasks.workunit.client.1.smithi070.stdout:3/411: truncate db/f1f 1767717 0 2024-09-10T04:05:15.566 INFO:tasks.workunit.client.0.smithi046.stdout:4/416: write d3/d12/d17/d29/f3e [1485549,83586] 0 2024-09-10T04:05:15.567 INFO:tasks.workunit.client.0.smithi046.stdout:5/480: rmdir d7/d11/d14/d8e 39 2024-09-10T04:05:15.573 INFO:tasks.workunit.client.1.smithi070.stdout:1/326: write d3/d10/f2c [2215644,76297] 0 2024-09-10T04:05:15.575 INFO:tasks.workunit.client.0.smithi046.stdout:9/381: read d4/d29/f4b [2334632,40553] 0 2024-09-10T04:05:15.575 INFO:tasks.workunit.client.0.smithi046.stdout:4/417: dwrite d3/d12/d17/d22/f4e [0,4194304] 0 2024-09-10T04:05:15.579 INFO:tasks.workunit.client.1.smithi070.stdout:2/351: rename d15/d18/d29/d2a/d2e/d32/f79 to d15/d18/d42/d62/d55/f7b 0 2024-09-10T04:05:15.585 INFO:tasks.workunit.client.1.smithi070.stdout:0/436: dread - d8/d13/d1a/f45 zero size 2024-09-10T04:05:15.587 INFO:tasks.workunit.client.0.smithi046.stdout:5/481: mknod d7/d11/d14/d2f/d2b/d6a/cb8 0 2024-09-10T04:05:15.587 INFO:tasks.workunit.client.0.smithi046.stdout:1/272: link d3/d27/f20 d3/d27/d32/d28/f5b 0 2024-09-10T04:05:15.589 INFO:tasks.workunit.client.1.smithi070.stdout:8/434: dwrite d4/d16/f33 [0,4194304] 0 2024-09-10T04:05:15.589 INFO:tasks.workunit.client.0.smithi046.stdout:1/273: dread - d3/dc/d34/d58/f59 zero size 2024-09-10T04:05:15.589 INFO:tasks.workunit.client.1.smithi070.stdout:3/412: readlink db/d1c/l46 0 2024-09-10T04:05:15.590 INFO:tasks.workunit.client.0.smithi046.stdout:1/274: fdatasync d3/d27/d32/d1f/f31 0 2024-09-10T04:05:15.590 INFO:tasks.workunit.client.1.smithi070.stdout:3/413: stat db/d1c/l46 0 2024-09-10T04:05:15.592 INFO:tasks.workunit.client.0.smithi046.stdout:4/418: unlink d3/d12/d17/d22/f5b 0 2024-09-10T04:05:15.593 INFO:tasks.workunit.client.0.smithi046.stdout:4/419: read d3/f3c [482238,53134] 0 2024-09-10T04:05:15.593 INFO:tasks.workunit.client.0.smithi046.stdout:9/382: dread d4/d29/f3d [0,4194304] 0 2024-09-10T04:05:15.594 INFO:tasks.workunit.client.0.smithi046.stdout:4/420: dread - d3/d12/d14/f2b zero size 2024-09-10T04:05:15.597 INFO:tasks.workunit.client.0.smithi046.stdout:6/364: rename da/db/d11/d13/d1e/f21 to da/db/f78 0 2024-09-10T04:05:15.597 INFO:tasks.workunit.client.0.smithi046.stdout:4/421: write d3/d12/d17/d22/d51/d5c/f76 [997722,124236] 0 2024-09-10T04:05:15.598 INFO:tasks.workunit.client.1.smithi070.stdout:9/440: creat d9/d1c/f87 x:0 0 0 2024-09-10T04:05:15.599 INFO:tasks.workunit.client.0.smithi046.stdout:1/275: truncate d3/f6 1758928 0 2024-09-10T04:05:15.600 INFO:tasks.workunit.client.0.smithi046.stdout:1/276: read - d3/d27/d32/d1f/f3c zero size 2024-09-10T04:05:15.601 INFO:tasks.workunit.client.1.smithi070.stdout:7/330: sync 2024-09-10T04:05:15.603 INFO:tasks.workunit.client.1.smithi070.stdout:2/352: rename d15/d18/d42/d62 to d15/d18/d56/d7c 0 2024-09-10T04:05:15.604 INFO:tasks.workunit.client.1.smithi070.stdout:6/461: mkdir d0/d3/d6a/d92 0 2024-09-10T04:05:15.605 INFO:tasks.workunit.client.0.smithi046.stdout:9/383: creat d4/d29/d65/f75 x:0 0 0 2024-09-10T04:05:15.605 INFO:tasks.workunit.client.1.smithi070.stdout:6/462: write d0/d3/d1f/f37 [1710968,93681] 0 2024-09-10T04:05:15.606 INFO:tasks.workunit.client.1.smithi070.stdout:6/463: stat d0/d3/d18/d1b/d17/d28/f3b 0 2024-09-10T04:05:15.607 INFO:tasks.workunit.client.0.smithi046.stdout:5/482: rename d7/d11/d14/d2f/lb3 to d7/d11/d14/d2f/d2b/lb9 0 2024-09-10T04:05:15.607 INFO:tasks.workunit.client.1.smithi070.stdout:6/464: dread - d0/d3/d1f/d55/f8e zero size 2024-09-10T04:05:15.608 INFO:tasks.workunit.client.1.smithi070.stdout:0/437: mknod d8/d13/d33/d74/c8e 0 2024-09-10T04:05:15.609 INFO:tasks.workunit.client.0.smithi046.stdout:6/365: rmdir da/db/d18 39 2024-09-10T04:05:15.610 INFO:tasks.workunit.client.1.smithi070.stdout:5/399: symlink d17/d24/d77/l89 0 2024-09-10T04:05:15.610 INFO:tasks.workunit.client.1.smithi070.stdout:5/400: readlink d17/d24/d4e/l5d 0 2024-09-10T04:05:15.611 INFO:tasks.workunit.client.1.smithi070.stdout:3/414: mkdir db/d83 0 2024-09-10T04:05:15.613 INFO:tasks.workunit.client.1.smithi070.stdout:9/441: mkdir d9/d59/d5d/d88 0 2024-09-10T04:05:15.614 INFO:tasks.workunit.client.1.smithi070.stdout:1/327: write d3/d8/d12/f4a [982363,53841] 0 2024-09-10T04:05:15.615 INFO:tasks.workunit.client.0.smithi046.stdout:9/384: chown d4/d3c/d4d/d5c/d59/f5d 12405977 1 2024-09-10T04:05:15.616 INFO:tasks.workunit.client.1.smithi070.stdout:9/442: truncate d9/d59/d10/d1b/f2d 8481520 0 2024-09-10T04:05:15.616 INFO:tasks.workunit.client.1.smithi070.stdout:7/331: symlink d2/d3/dc/d5d/l77 0 2024-09-10T04:05:15.619 INFO:tasks.workunit.client.1.smithi070.stdout:8/435: rename d4/d16/d17/d19/d28/d43/d55 to d4/d16/d17/d19/d28/d43/d67/d97 0 2024-09-10T04:05:15.619 INFO:tasks.workunit.client.0.smithi046.stdout:6/366: mkdir da/dc/d79 0 2024-09-10T04:05:15.621 INFO:tasks.workunit.client.1.smithi070.stdout:8/436: fsync d4/d16/d17/d19/d28/f46 0 2024-09-10T04:05:15.622 INFO:tasks.workunit.client.1.smithi070.stdout:6/465: write d0/f4d [1992936,125912] 0 2024-09-10T04:05:15.623 INFO:tasks.workunit.client.1.smithi070.stdout:3/415: symlink db/d1c/d3e/l84 0 2024-09-10T04:05:15.623 INFO:tasks.workunit.client.1.smithi070.stdout:8/437: chown d4/d16/d17/d5c/d87 83306 1 2024-09-10T04:05:15.623 INFO:tasks.workunit.client.1.smithi070.stdout:9/443: dwrite d9/d7a/f44 [4194304,4194304] 0 2024-09-10T04:05:15.626 INFO:tasks.workunit.client.0.smithi046.stdout:6/367: unlink da/db/d11/d2b/d5a/f6f 0 2024-09-10T04:05:15.628 INFO:tasks.workunit.client.1.smithi070.stdout:2/353: rename d15/d18/f19 to d15/d18/d56/d7c/d55/d6d/f7d 0 2024-09-10T04:05:15.629 INFO:tasks.workunit.client.0.smithi046.stdout:9/385: getdents d4/d66 0 2024-09-10T04:05:15.631 INFO:tasks.workunit.client.1.smithi070.stdout:1/328: dread d3/d8/d12/f45 [0,4194304] 0 2024-09-10T04:05:15.631 INFO:tasks.workunit.client.1.smithi070.stdout:0/438: mkdir d8/d13/d1a/d8f 0 2024-09-10T04:05:15.631 INFO:tasks.workunit.client.0.smithi046.stdout:5/483: link d7/ca6 d7/d11/d3f/d53/cba 0 2024-09-10T04:05:15.634 INFO:tasks.workunit.client.0.smithi046.stdout:4/422: getdents d3/d12/d17/d22/d30 0 2024-09-10T04:05:15.636 INFO:tasks.workunit.client.1.smithi070.stdout:6/466: unlink d0/d3/d18/d2a/c89 0 2024-09-10T04:05:15.637 INFO:tasks.workunit.client.1.smithi070.stdout:6/467: write d0/d3/d18/d1b/f3a [833227,61142] 0 2024-09-10T04:05:15.638 INFO:tasks.workunit.client.0.smithi046.stdout:6/368: fsync f6 0 2024-09-10T04:05:15.638 INFO:tasks.workunit.client.1.smithi070.stdout:6/468: write d0/d3/d18/d1b/f3a [1518301,115156] 0 2024-09-10T04:05:15.639 INFO:tasks.workunit.client.1.smithi070.stdout:9/444: truncate d9/f22 1913573 0 2024-09-10T04:05:15.641 INFO:tasks.workunit.client.0.smithi046.stdout:1/277: getdents d3/dc/d21 0 2024-09-10T04:05:15.641 INFO:tasks.workunit.client.0.smithi046.stdout:5/484: dwrite d7/d11/d3f/d5a/d63/f7f [0,4194304] 0 2024-09-10T04:05:15.641 INFO:tasks.workunit.client.1.smithi070.stdout:9/445: truncate d9/d1c/d34/d63/f69 112365 0 2024-09-10T04:05:15.643 INFO:tasks.workunit.client.1.smithi070.stdout:8/438: creat d4/d16/d17/d19/d28/d43/d67/d97/f98 x:0 0 0 2024-09-10T04:05:15.646 INFO:tasks.workunit.client.1.smithi070.stdout:7/332: creat d2/d66/f78 x:0 0 0 2024-09-10T04:05:15.647 INFO:tasks.workunit.client.1.smithi070.stdout:7/333: chown d2/d40/d6c/l76 1978 1 2024-09-10T04:05:15.647 INFO:tasks.workunit.client.0.smithi046.stdout:4/423: creat d3/d12/d17/d29/d58/f90 x:0 0 0 2024-09-10T04:05:15.651 INFO:tasks.workunit.client.0.smithi046.stdout:9/386: dwrite d4/d3c/f40 [0,4194304] 0 2024-09-10T04:05:15.652 INFO:tasks.workunit.client.0.smithi046.stdout:9/387: read d4/d18/f36 [1596392,61086] 0 2024-09-10T04:05:15.654 INFO:tasks.workunit.client.1.smithi070.stdout:2/354: dread - d15/d18/d29/f3d zero size 2024-09-10T04:05:15.660 INFO:tasks.workunit.client.1.smithi070.stdout:4/317: sync 2024-09-10T04:05:15.661 INFO:tasks.workunit.client.1.smithi070.stdout:0/439: dread d8/d13/d1a/f7e [0,4194304] 0 2024-09-10T04:05:15.663 INFO:tasks.workunit.client.0.smithi046.stdout:4/424: mkdir d3/d12/d17/d91 0 2024-09-10T04:05:15.670 INFO:tasks.workunit.client.0.smithi046.stdout:9/388: creat d4/d3c/d4d/f76 x:0 0 0 2024-09-10T04:05:15.671 INFO:tasks.workunit.client.0.smithi046.stdout:6/369: dread da/db/f78 [0,4194304] 0 2024-09-10T04:05:15.672 INFO:tasks.workunit.client.1.smithi070.stdout:6/469: mkdir d0/d3/d1f/d93 0 2024-09-10T04:05:15.673 INFO:tasks.workunit.client.0.smithi046.stdout:1/278: dread d3/dc/f17 [0,4194304] 0 2024-09-10T04:05:15.674 INFO:tasks.workunit.client.0.smithi046.stdout:5/485: rename d7/d11/f12 to d7/d11/d14/d45/d51/fbb 0 2024-09-10T04:05:15.675 INFO:tasks.workunit.client.0.smithi046.stdout:5/486: readlink d7/d11/d14/d2f/l77 0 2024-09-10T04:05:15.676 INFO:tasks.workunit.client.0.smithi046.stdout:5/487: write d7/d11/d3f/d53/f57 [521118,29661] 0 2024-09-10T04:05:15.677 INFO:tasks.workunit.client.0.smithi046.stdout:4/425: creat d3/d12/d17/d29/f92 x:0 0 0 2024-09-10T04:05:15.681 INFO:tasks.workunit.client.0.smithi046.stdout:9/389: creat d4/d7/f77 x:0 0 0 2024-09-10T04:05:15.686 INFO:tasks.workunit.client.0.smithi046.stdout:6/370: rename da/dc/d22/c4b to da/db/d11/d2b/d55/c7a 0 2024-09-10T04:05:15.687 INFO:tasks.workunit.client.1.smithi070.stdout:1/329: unlink d3/d8/l31 0 2024-09-10T04:05:15.689 INFO:tasks.workunit.client.0.smithi046.stdout:4/426: dread d3/d12/d17/d22/f59 [0,4194304] 0 2024-09-10T04:05:15.689 INFO:tasks.workunit.client.1.smithi070.stdout:9/446: write d9/d59/f12 [373279,86175] 0 2024-09-10T04:05:15.690 INFO:tasks.workunit.client.0.smithi046.stdout:4/427: readlink d3/d12/d17/d22/d51/l67 0 2024-09-10T04:05:15.691 INFO:tasks.workunit.client.0.smithi046.stdout:4/428: write d3/fa [2501069,43345] 0 2024-09-10T04:05:15.694 INFO:tasks.workunit.client.1.smithi070.stdout:5/401: getdents d17/d6f 0 2024-09-10T04:05:15.694 INFO:tasks.workunit.client.0.smithi046.stdout:1/279: write d3/d27/f38 [1193514,79804] 0 2024-09-10T04:05:15.695 INFO:tasks.workunit.client.1.smithi070.stdout:5/402: write d17/d24/f2c [41555,62517] 0 2024-09-10T04:05:15.696 INFO:tasks.workunit.client.1.smithi070.stdout:5/403: fsync f12 0 2024-09-10T04:05:15.697 INFO:tasks.workunit.client.0.smithi046.stdout:5/488: symlink d7/d11/d14/d8e/lbc 0 2024-09-10T04:05:15.698 INFO:tasks.workunit.client.1.smithi070.stdout:7/334: mkdir d2/d3/dc/d79 0 2024-09-10T04:05:15.703 INFO:tasks.workunit.client.1.smithi070.stdout:3/416: rename db/l1d to db/d1c/l85 0 2024-09-10T04:05:15.704 INFO:tasks.workunit.client.1.smithi070.stdout:7/335: dwrite d2/d7/d9/d18/f6b [0,4194304] 0 2024-09-10T04:05:15.705 INFO:tasks.workunit.client.0.smithi046.stdout:9/390: creat d4/d44/f78 x:0 0 0 2024-09-10T04:05:15.706 INFO:tasks.workunit.client.0.smithi046.stdout:1/280: symlink d3/dc/d21/d3a/l5c 0 2024-09-10T04:05:15.707 INFO:tasks.workunit.client.0.smithi046.stdout:4/429: write d3/d12/d14/f16 [2419169,836] 0 2024-09-10T04:05:15.709 INFO:tasks.workunit.client.1.smithi070.stdout:1/330: write d3/d8/d13/d3f/f52 [3047544,103929] 0 2024-09-10T04:05:15.709 INFO:tasks.workunit.client.0.smithi046.stdout:4/430: readlink d3/d12/d17/d22/d30/l5d 0 2024-09-10T04:05:15.709 INFO:tasks.workunit.client.1.smithi070.stdout:7/336: dread d2/d7/d9/d18/f3f [0,4194304] 0 2024-09-10T04:05:15.710 INFO:tasks.workunit.client.0.smithi046.stdout:4/431: stat d3/d12/f73 0 2024-09-10T04:05:15.710 INFO:tasks.workunit.client.0.smithi046.stdout:5/489: read d7/d11/d3f/d53/da9/f62 [111218,85241] 0 2024-09-10T04:05:15.710 INFO:tasks.workunit.client.1.smithi070.stdout:7/337: write d2/d3/f70 [272216,73834] 0 2024-09-10T04:05:15.711 INFO:tasks.workunit.client.1.smithi070.stdout:4/318: mknod de/d13/d4f/d6a/c6e 0 2024-09-10T04:05:15.711 INFO:tasks.workunit.client.0.smithi046.stdout:4/432: dread - d3/d12/d17/d22/f70 zero size 2024-09-10T04:05:15.716 INFO:tasks.workunit.client.0.smithi046.stdout:6/371: creat da/f7b x:0 0 0 2024-09-10T04:05:15.718 INFO:tasks.workunit.client.1.smithi070.stdout:8/439: rmdir d4/d16/d17/d19/d28/d43/d67/d91 0 2024-09-10T04:05:15.719 INFO:tasks.workunit.client.0.smithi046.stdout:4/433: rmdir d3/d43/d5f/d85 39 2024-09-10T04:05:15.723 INFO:tasks.workunit.client.1.smithi070.stdout:9/447: dread d9/f22 [0,4194304] 0 2024-09-10T04:05:15.723 INFO:tasks.workunit.client.0.smithi046.stdout:1/281: mkdir d3/d27/d5d 0 2024-09-10T04:05:15.723 INFO:tasks.workunit.client.1.smithi070.stdout:0/440: truncate d8/f11 1601423 0 2024-09-10T04:05:15.724 INFO:tasks.workunit.client.1.smithi070.stdout:9/448: write d9/d59/f12 [5140299,97141] 0 2024-09-10T04:05:15.726 INFO:tasks.workunit.client.1.smithi070.stdout:2/355: creat d15/d18/f7e x:0 0 0 2024-09-10T04:05:15.729 INFO:tasks.workunit.client.1.smithi070.stdout:3/417: write db/f14 [2872639,44432] 0 2024-09-10T04:05:15.730 INFO:tasks.workunit.client.1.smithi070.stdout:2/356: dread d15/d2b/f4c [0,4194304] 0 2024-09-10T04:05:15.730 INFO:tasks.workunit.client.1.smithi070.stdout:7/338: creat d2/d3/d63/f7a x:0 0 0 2024-09-10T04:05:15.731 INFO:tasks.workunit.client.0.smithi046.stdout:9/391: creat d4/f79 x:0 0 0 2024-09-10T04:05:15.734 INFO:tasks.workunit.client.0.smithi046.stdout:9/392: write d4/d7/d1f/d54/d56/f72 [1712499,56534] 0 2024-09-10T04:05:15.734 INFO:tasks.workunit.client.1.smithi070.stdout:2/357: dread d15/d18/f28 [0,4194304] 0 2024-09-10T04:05:15.734 INFO:tasks.workunit.client.0.smithi046.stdout:4/434: dread d3/f3c [0,4194304] 0 2024-09-10T04:05:15.735 INFO:tasks.workunit.client.0.smithi046.stdout:9/393: chown d4/d29/l52 106 1 2024-09-10T04:05:15.736 INFO:tasks.workunit.client.0.smithi046.stdout:1/282: mknod d3/d27/c5e 0 2024-09-10T04:05:15.739 INFO:tasks.workunit.client.0.smithi046.stdout:1/283: truncate d3/d27/d32/d1f/f31 630074 0 2024-09-10T04:05:15.742 INFO:tasks.workunit.client.0.smithi046.stdout:6/372: creat da/db/d11/d13/f7c x:0 0 0 2024-09-10T04:05:15.743 INFO:tasks.workunit.client.1.smithi070.stdout:5/404: truncate f3 1580447 0 2024-09-10T04:05:15.743 INFO:tasks.workunit.client.1.smithi070.stdout:5/405: chown d17/d6f 2613809 1 2024-09-10T04:05:15.744 INFO:tasks.workunit.client.0.smithi046.stdout:5/490: getdents d7/d11/d3f/d5a/d63 0 2024-09-10T04:05:15.745 INFO:tasks.workunit.client.1.smithi070.stdout:2/358: dread d15/d18/d29/d2a/f58 [0,4194304] 0 2024-09-10T04:05:15.751 INFO:tasks.workunit.client.1.smithi070.stdout:6/470: sync 2024-09-10T04:05:15.752 INFO:tasks.workunit.client.1.smithi070.stdout:1/331: truncate d3/d8/d12/f4a 755832 0 2024-09-10T04:05:15.753 INFO:tasks.workunit.client.0.smithi046.stdout:5/491: creat d7/d11/d3f/d53/d75/fbd x:0 0 0 2024-09-10T04:05:15.754 INFO:tasks.workunit.client.1.smithi070.stdout:7/339: chown d2/d7/c19 488856988 1 2024-09-10T04:05:15.755 INFO:tasks.workunit.client.0.smithi046.stdout:4/435: mkdir d3/d12/d93 0 2024-09-10T04:05:15.756 INFO:tasks.workunit.client.1.smithi070.stdout:4/319: mknod de/d13/d18/d23/d47/c6f 0 2024-09-10T04:05:15.757 INFO:tasks.workunit.client.1.smithi070.stdout:4/320: chown de/d13/d18/d23/l41 16379 1 2024-09-10T04:05:15.757 INFO:tasks.workunit.client.0.smithi046.stdout:4/436: read - d3/d12/d17/d22/f70 zero size 2024-09-10T04:05:15.758 INFO:tasks.workunit.client.0.smithi046.stdout:6/373: dwrite da/db/d18/f49 [0,4194304] 0 2024-09-10T04:05:15.759 INFO:tasks.workunit.client.0.smithi046.stdout:1/284: symlink d3/d27/l5f 0 2024-09-10T04:05:15.760 INFO:tasks.workunit.client.1.smithi070.stdout:0/441: creat d8/d13/d33/d6f/f90 x:0 0 0 2024-09-10T04:05:15.760 INFO:tasks.workunit.client.1.smithi070.stdout:0/442: chown d8/l55 148431041 1 2024-09-10T04:05:15.761 INFO:tasks.workunit.client.1.smithi070.stdout:9/449: mkdir d9/d59/d5d/d88/d89 0 2024-09-10T04:05:15.764 INFO:tasks.workunit.client.0.smithi046.stdout:9/394: link d4/d7/d1f/l53 d4/d29/d65/l7a 0 2024-09-10T04:05:15.764 INFO:tasks.workunit.client.0.smithi046.stdout:9/395: fsync d4/d7/f4a 0 2024-09-10T04:05:15.765 INFO:tasks.workunit.client.1.smithi070.stdout:2/359: rmdir d15/d18/d56/d7c/d55 39 2024-09-10T04:05:15.766 INFO:tasks.workunit.client.0.smithi046.stdout:6/374: symlink da/db/d63/l7d 0 2024-09-10T04:05:15.768 INFO:tasks.workunit.client.1.smithi070.stdout:9/450: dread d9/d1c/d34/f76 [4194304,4194304] 0 2024-09-10T04:05:15.771 INFO:tasks.workunit.client.0.smithi046.stdout:1/285: rmdir d3/d27/d32/d28 39 2024-09-10T04:05:15.772 INFO:tasks.workunit.client.1.smithi070.stdout:6/471: mkdir d0/d3/d18/d1b/d17/d3f/d40/d94 0 2024-09-10T04:05:15.773 INFO:tasks.workunit.client.0.smithi046.stdout:5/492: dread d7/d11/d3f/d5a/f69 [0,4194304] 0 2024-09-10T04:05:15.778 INFO:tasks.workunit.client.1.smithi070.stdout:7/340: mknod d2/d7/d9/d18/d49/c7b 0 2024-09-10T04:05:15.779 INFO:tasks.workunit.client.1.smithi070.stdout:4/321: rmdir de/d13/d18/d23/d47 39 2024-09-10T04:05:15.779 INFO:tasks.workunit.client.1.smithi070.stdout:4/322: stat de/d13/d4f/c50 0 2024-09-10T04:05:15.781 INFO:tasks.workunit.client.0.smithi046.stdout:1/286: unlink d3/dc/f56 0 2024-09-10T04:05:15.782 INFO:tasks.workunit.client.1.smithi070.stdout:8/440: link d4/d16/d17/d5c/f6d d4/f99 0 2024-09-10T04:05:15.787 INFO:tasks.workunit.client.1.smithi070.stdout:5/406: unlink f3 0 2024-09-10T04:05:15.790 INFO:tasks.workunit.client.0.smithi046.stdout:1/287: dwrite d3/dc/f4b [0,4194304] 0 2024-09-10T04:05:15.790 INFO:tasks.workunit.client.0.smithi046.stdout:4/437: dwrite d3/d12/d17/d29/f83 [0,4194304] 0 2024-09-10T04:05:15.791 INFO:tasks.workunit.client.0.smithi046.stdout:7/398: sync 2024-09-10T04:05:15.793 INFO:tasks.workunit.client.0.smithi046.stdout:5/493: link d7/fc d7/d11/d14/d8e/d96/fbe 0 2024-09-10T04:05:15.798 INFO:tasks.workunit.client.1.smithi070.stdout:3/418: creat db/d1c/d25/f86 x:0 0 0 2024-09-10T04:05:15.800 INFO:tasks.workunit.client.0.smithi046.stdout:6/375: link da/db/d11/d2b/d5a/f5c da/db/d11/d13/d1e/f7e 0 2024-09-10T04:05:15.803 INFO:tasks.workunit.client.1.smithi070.stdout:6/472: truncate d0/d3/d18/f45 4506162 0 2024-09-10T04:05:15.806 INFO:tasks.workunit.client.1.smithi070.stdout:1/332: link d3/d10/d1d/f46 d3/d8/d27/f68 0 2024-09-10T04:05:15.807 INFO:tasks.workunit.client.1.smithi070.stdout:1/333: read - d3/d8/d12/f66 zero size 2024-09-10T04:05:15.807 INFO:tasks.workunit.client.0.smithi046.stdout:9/396: truncate d4/d29/f6d 868584 0 2024-09-10T04:05:15.808 INFO:tasks.workunit.client.1.smithi070.stdout:1/334: fdatasync d3/d8/fc 0 2024-09-10T04:05:15.808 INFO:tasks.workunit.client.0.smithi046.stdout:9/397: readlink d4/d7/d1f/ld 0 2024-09-10T04:05:15.808 INFO:tasks.workunit.client.0.smithi046.stdout:4/438: mkdir d3/d12/d17/d22/d51/d94 0 2024-09-10T04:05:15.808 INFO:tasks.workunit.client.1.smithi070.stdout:1/335: truncate d3/d8/d12/d2f/d32/f41 4831809 0 2024-09-10T04:05:15.809 INFO:tasks.workunit.client.0.smithi046.stdout:4/439: write d3/d12/d14/f18 [5052806,4943] 0 2024-09-10T04:05:15.810 INFO:tasks.workunit.client.1.smithi070.stdout:7/341: unlink d2/d7/d9/d18/d23/f3e 0 2024-09-10T04:05:15.812 INFO:tasks.workunit.client.0.smithi046.stdout:7/399: fdatasync d5/d6/db/dd/d24/f31 0 2024-09-10T04:05:15.813 INFO:tasks.workunit.client.0.smithi046.stdout:7/400: fdatasync d5/d6/db/d11/f14 0 2024-09-10T04:05:15.815 INFO:tasks.workunit.client.0.smithi046.stdout:7/401: truncate d5/d6/d39/d5a/f64 4353906 0 2024-09-10T04:05:15.816 INFO:tasks.workunit.client.0.smithi046.stdout:5/494: creat d7/d11/d14/d2f/da3/fbf x:0 0 0 2024-09-10T04:05:15.819 INFO:tasks.workunit.client.0.smithi046.stdout:6/376: unlink da/db/d11/d13/d1e/f26 0 2024-09-10T04:05:15.822 INFO:tasks.workunit.client.1.smithi070.stdout:0/443: write d8/d13/d1a/f31 [2255137,121188] 0 2024-09-10T04:05:15.830 INFO:tasks.workunit.client.1.smithi070.stdout:9/451: rename d9/d59/d10 to d9/d60/d8a 0 2024-09-10T04:05:15.830 INFO:tasks.workunit.client.1.smithi070.stdout:9/452: stat d9/d59/d2a/d3f/f48 0 2024-09-10T04:05:15.832 INFO:tasks.workunit.client.1.smithi070.stdout:9/453: truncate d9/d60/d8a/f81 1004262 0 2024-09-10T04:05:15.833 INFO:tasks.workunit.client.0.smithi046.stdout:7/402: symlink d5/d9/d12/d72/l86 0 2024-09-10T04:05:15.833 INFO:tasks.workunit.client.1.smithi070.stdout:9/454: fdatasync d9/f68 0 2024-09-10T04:05:15.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:15 smithi070 ceph-mon[35495]: pgmap v261: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 16 MiB/s rd, 147 MiB/s wr, 275 op/s 2024-09-10T04:05:15.835 INFO:tasks.workunit.client.1.smithi070.stdout:4/323: dwrite de/d13/d18/d26/f42 [0,4194304] 0 2024-09-10T04:05:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:15 smithi046 ceph-mon[28328]: pgmap v261: 65 pgs: 65 active+clean; 1.2 GiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 16 MiB/s rd, 147 MiB/s wr, 275 op/s 2024-09-10T04:05:15.836 INFO:tasks.workunit.client.0.smithi046.stdout:5/495: mkdir d7/d11/d3f/d53/dac/dc0 0 2024-09-10T04:05:15.840 INFO:tasks.workunit.client.0.smithi046.stdout:1/288: write d3/d27/d32/d28/f2c [3754548,35720] 0 2024-09-10T04:05:15.841 INFO:tasks.workunit.client.1.smithi070.stdout:9/455: dwrite d9/f68 [0,4194304] 0 2024-09-10T04:05:15.842 INFO:tasks.workunit.client.1.smithi070.stdout:7/342: dread d2/d7/d9/d18/d3c/f60 [0,4194304] 0 2024-09-10T04:05:15.844 INFO:tasks.workunit.client.1.smithi070.stdout:7/343: chown d2/d40/d6c/l76 28563024 1 2024-09-10T04:05:15.846 INFO:tasks.workunit.client.1.smithi070.stdout:6/473: write d0/f10 [2968295,76374] 0 2024-09-10T04:05:15.847 INFO:tasks.workunit.client.1.smithi070.stdout:9/456: dread d9/d60/d8a/d1b/f2d [4194304,4194304] 0 2024-09-10T04:05:15.847 INFO:tasks.workunit.client.1.smithi070.stdout:7/344: dread d2/d3/dc/d5d/d41/f50 [0,4194304] 0 2024-09-10T04:05:15.850 INFO:tasks.workunit.client.1.smithi070.stdout:5/407: fdatasync d17/d19/f2e 0 2024-09-10T04:05:15.851 INFO:tasks.workunit.client.1.smithi070.stdout:5/408: read d17/d19/d35/f3a [4040297,113845] 0 2024-09-10T04:05:15.851 INFO:tasks.workunit.client.1.smithi070.stdout:5/409: stat d17/d19/d6a/f53 0 2024-09-10T04:05:15.853 INFO:tasks.workunit.client.0.smithi046.stdout:1/289: dread d3/d27/d32/f2b [0,4194304] 0 2024-09-10T04:05:15.853 INFO:tasks.workunit.client.1.smithi070.stdout:2/360: creat d15/d18/d29/f7f x:0 0 0 2024-09-10T04:05:15.853 INFO:tasks.workunit.client.1.smithi070.stdout:5/410: dread d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:15.859 INFO:tasks.workunit.client.0.smithi046.stdout:9/398: getdents d4/d3c 0 2024-09-10T04:05:15.860 INFO:tasks.workunit.client.0.smithi046.stdout:6/377: dwrite da/f40 [4194304,4194304] 0 2024-09-10T04:05:15.861 INFO:tasks.workunit.client.0.smithi046.stdout:5/496: link d7/d11/d14/d8e/lbc d7/d11/d14/d2f/d64/lc1 0 2024-09-10T04:05:15.866 INFO:tasks.workunit.client.0.smithi046.stdout:7/403: rename d5/d6/db/f50 to d5/d9/f87 0 2024-09-10T04:05:15.867 INFO:tasks.workunit.client.0.smithi046.stdout:1/290: dwrite d3/d27/d32/f2e [0,4194304] 0 2024-09-10T04:05:15.869 INFO:tasks.workunit.client.0.smithi046.stdout:1/291: fsync d3/d27/f33 0 2024-09-10T04:05:15.870 INFO:tasks.workunit.client.1.smithi070.stdout:8/441: creat d4/d16/d17/d5c/d87/d8e/f9a x:0 0 0 2024-09-10T04:05:15.870 INFO:tasks.workunit.client.0.smithi046.stdout:6/378: symlink da/db/d63/l7f 0 2024-09-10T04:05:15.871 INFO:tasks.workunit.client.1.smithi070.stdout:0/444: symlink d8/d24/d2d/l91 0 2024-09-10T04:05:15.871 INFO:tasks.workunit.client.0.smithi046.stdout:9/399: dread d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:15.872 INFO:tasks.workunit.client.1.smithi070.stdout:0/445: chown d8/d13/d47 15236758 1 2024-09-10T04:05:15.873 INFO:tasks.workunit.client.1.smithi070.stdout:6/474: creat d0/d3/d18/d1b/d17/d28/f95 x:0 0 0 2024-09-10T04:05:15.873 INFO:tasks.workunit.client.0.smithi046.stdout:6/379: stat da/db/d11/d13/d1e/d69 0 2024-09-10T04:05:15.876 INFO:tasks.workunit.client.1.smithi070.stdout:9/457: mkdir d9/d1c/d34/d8b 0 2024-09-10T04:05:15.877 INFO:tasks.workunit.client.1.smithi070.stdout:9/458: truncate d9/d1c/f87 193157 0 2024-09-10T04:05:15.877 INFO:tasks.workunit.client.1.smithi070.stdout:7/345: creat d2/d3/dc/d5d/d42/f7c x:0 0 0 2024-09-10T04:05:15.882 INFO:tasks.workunit.client.0.smithi046.stdout:7/404: read d5/d6/db/d11/d1e/f30 [233992,45772] 0 2024-09-10T04:05:15.882 INFO:tasks.workunit.client.0.smithi046.stdout:5/497: dwrite d7/d11/d3f/d53/d99/fae [0,4194304] 0 2024-09-10T04:05:15.886 INFO:tasks.workunit.client.1.smithi070.stdout:6/475: dread d0/d3/d1f/f22 [0,4194304] 0 2024-09-10T04:05:15.886 INFO:tasks.workunit.client.0.smithi046.stdout:7/405: write d5/d6/d39/d5a/f79 [1258501,35916] 0 2024-09-10T04:05:15.888 INFO:tasks.workunit.client.0.smithi046.stdout:5/498: truncate d7/d11/d3f/d53/d75/fbd 824504 0 2024-09-10T04:05:15.888 INFO:tasks.workunit.client.0.smithi046.stdout:9/400: dread d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:15.888 INFO:tasks.workunit.client.1.smithi070.stdout:3/419: creat db/d1c/d25/f87 x:0 0 0 2024-09-10T04:05:15.889 INFO:tasks.workunit.client.0.smithi046.stdout:5/499: stat d7/d11/d14/f71 0 2024-09-10T04:05:15.890 INFO:tasks.workunit.client.0.smithi046.stdout:9/401: write d4/d29/d32/f63 [934494,96479] 0 2024-09-10T04:05:15.891 INFO:tasks.workunit.client.1.smithi070.stdout:4/324: mknod de/d13/d18/d23/c70 0 2024-09-10T04:05:15.892 INFO:tasks.workunit.client.1.smithi070.stdout:4/325: write de/d13/f45 [653681,80082] 0 2024-09-10T04:05:15.894 INFO:tasks.workunit.client.1.smithi070.stdout:1/336: creat d3/d8/d13/d3f/f69 x:0 0 0 2024-09-10T04:05:15.897 INFO:tasks.workunit.client.1.smithi070.stdout:4/326: dread de/d13/f45 [0,4194304] 0 2024-09-10T04:05:15.901 INFO:tasks.workunit.client.0.smithi046.stdout:1/292: dwrite d3/dc/f10 [0,4194304] 0 2024-09-10T04:05:15.901 INFO:tasks.workunit.client.1.smithi070.stdout:9/459: mkdir d9/d7a/d74/d8c 0 2024-09-10T04:05:15.901 INFO:tasks.workunit.client.1.smithi070.stdout:9/460: stat d9/d7a/d74/l46 0 2024-09-10T04:05:15.902 INFO:tasks.workunit.client.0.smithi046.stdout:1/293: write d3/d27/f33 [336757,113213] 0 2024-09-10T04:05:15.902 INFO:tasks.workunit.client.1.smithi070.stdout:7/346: mknod d2/d40/c7d 0 2024-09-10T04:05:15.904 INFO:tasks.workunit.client.1.smithi070.stdout:7/347: write d2/d3/d63/f7a [318723,121844] 0 2024-09-10T04:05:15.906 INFO:tasks.workunit.client.0.smithi046.stdout:7/406: dread d5/d6/d39/d5a/f64 [0,4194304] 0 2024-09-10T04:05:15.908 INFO:tasks.workunit.client.0.smithi046.stdout:7/407: write d5/d9/d43/f5c [1407616,72841] 0 2024-09-10T04:05:15.909 INFO:tasks.workunit.client.0.smithi046.stdout:5/500: creat d7/d11/d14/d45/fc2 x:0 0 0 2024-09-10T04:05:15.910 INFO:tasks.workunit.client.0.smithi046.stdout:5/501: readlink d7/d11/d14/d8e/la2 0 2024-09-10T04:05:15.911 INFO:tasks.workunit.client.1.smithi070.stdout:9/461: dwrite d9/d60/d8a/f81 [0,4194304] 0 2024-09-10T04:05:15.914 INFO:tasks.workunit.client.1.smithi070.stdout:9/462: fdatasync d9/f25 0 2024-09-10T04:05:15.916 INFO:tasks.workunit.client.1.smithi070.stdout:9/463: chown d9/d59/d2a/l71 165756 1 2024-09-10T04:05:15.919 INFO:tasks.workunit.client.1.smithi070.stdout:1/337: fsync d3/d10/f5e 0 2024-09-10T04:05:15.921 INFO:tasks.workunit.client.1.smithi070.stdout:2/361: dwrite d15/d18/d56/d7c/d55/f7b [0,4194304] 0 2024-09-10T04:05:15.921 INFO:tasks.workunit.client.1.smithi070.stdout:8/442: symlink d4/d16/d17/d5c/d87/d8e/l9b 0 2024-09-10T04:05:15.922 INFO:tasks.workunit.client.0.smithi046.stdout:7/408: symlink d5/d6/d62/l88 0 2024-09-10T04:05:15.926 INFO:tasks.workunit.client.1.smithi070.stdout:4/327: creat de/d13/d18/d26/d46/f71 x:0 0 0 2024-09-10T04:05:15.927 INFO:tasks.workunit.client.0.smithi046.stdout:6/380: truncate da/fd 2192096 0 2024-09-10T04:05:15.928 INFO:tasks.workunit.client.0.smithi046.stdout:6/381: write da/dc/f28 [954050,86510] 0 2024-09-10T04:05:15.929 INFO:tasks.workunit.client.0.smithi046.stdout:6/382: chown da/db/d10 0 1 2024-09-10T04:05:15.929 INFO:tasks.workunit.client.1.smithi070.stdout:5/411: dwrite fd [0,4194304] 0 2024-09-10T04:05:15.930 INFO:tasks.workunit.client.0.smithi046.stdout:9/402: truncate d4/d18/f4f 1726188 0 2024-09-10T04:05:15.932 INFO:tasks.workunit.client.0.smithi046.stdout:1/294: mkdir d3/d27/d5d/d60 0 2024-09-10T04:05:15.934 INFO:tasks.workunit.client.1.smithi070.stdout:7/348: fdatasync d2/d3/dc/d5d/d41/f50 0 2024-09-10T04:05:15.937 INFO:tasks.workunit.client.1.smithi070.stdout:5/412: dwrite d17/d24/d4e/f82 [0,4194304] 0 2024-09-10T04:05:15.938 INFO:tasks.workunit.client.1.smithi070.stdout:5/413: dread - d17/d19/f50 zero size 2024-09-10T04:05:15.942 INFO:tasks.workunit.client.0.smithi046.stdout:1/295: unlink d3/d27/d32/l3f 0 2024-09-10T04:05:15.943 INFO:tasks.workunit.client.1.smithi070.stdout:3/420: symlink db/d1c/l88 0 2024-09-10T04:05:15.948 INFO:tasks.workunit.client.0.smithi046.stdout:5/502: creat d7/d11/d14/fc3 x:0 0 0 2024-09-10T04:05:15.951 INFO:tasks.workunit.client.1.smithi070.stdout:9/464: creat d9/d60/d8a/d3c/d62/f8d x:0 0 0 2024-09-10T04:05:15.954 INFO:tasks.workunit.client.1.smithi070.stdout:9/465: dread d9/d60/f84 [0,4194304] 0 2024-09-10T04:05:15.955 INFO:tasks.workunit.client.0.smithi046.stdout:1/296: rename d3/dc/d21/f40 to d3/d27/d32/d1f/f61 0 2024-09-10T04:05:15.955 INFO:tasks.workunit.client.1.smithi070.stdout:9/466: fsync d9/d60/d8a/d3c/d62/f7d 0 2024-09-10T04:05:15.955 INFO:tasks.workunit.client.0.smithi046.stdout:8/379: sync 2024-09-10T04:05:15.956 INFO:tasks.workunit.client.0.smithi046.stdout:3/363: sync 2024-09-10T04:05:15.956 INFO:tasks.workunit.client.0.smithi046.stdout:2/409: sync 2024-09-10T04:05:15.956 INFO:tasks.workunit.client.1.smithi070.stdout:9/467: fsync d9/f73 0 2024-09-10T04:05:15.956 INFO:tasks.workunit.client.0.smithi046.stdout:5/503: dwrite d7/d11/d3f/f80 [0,4194304] 0 2024-09-10T04:05:15.957 INFO:tasks.workunit.client.0.smithi046.stdout:3/364: stat d12/d1b/d4b 0 2024-09-10T04:05:15.958 INFO:tasks.workunit.client.0.smithi046.stdout:5/504: read - d7/f70 zero size 2024-09-10T04:05:15.959 INFO:tasks.workunit.client.0.smithi046.stdout:1/297: dread d3/d27/d32/d1f/f31 [0,4194304] 0 2024-09-10T04:05:15.960 INFO:tasks.workunit.client.0.smithi046.stdout:9/403: dread d4/d7/d1f/f26 [0,4194304] 0 2024-09-10T04:05:15.961 INFO:tasks.workunit.client.0.smithi046.stdout:7/409: getdents d5/d9 0 2024-09-10T04:05:15.964 INFO:tasks.workunit.client.1.smithi070.stdout:4/328: truncate de/d13/d18/d26/f2d 522742 0 2024-09-10T04:05:15.965 INFO:tasks.workunit.client.1.smithi070.stdout:4/329: chown de/d13/l32 88 1 2024-09-10T04:05:15.968 INFO:tasks.workunit.client.0.smithi046.stdout:3/365: readlink d12/d13/l38 0 2024-09-10T04:05:15.972 INFO:tasks.workunit.client.0.smithi046.stdout:8/380: rename d1/d29/d17/d1f/c3a to d1/d29/d17/d1f/d46/d52/c81 0 2024-09-10T04:05:15.973 INFO:tasks.workunit.client.1.smithi070.stdout:5/414: fdatasync d17/d19/d6a/f5e 0 2024-09-10T04:05:15.976 INFO:tasks.workunit.client.0.smithi046.stdout:8/381: dread d1/d29/d19/f34 [0,4194304] 0 2024-09-10T04:05:15.977 INFO:tasks.workunit.client.0.smithi046.stdout:7/410: write d5/d6/db/f2a [848224,91937] 0 2024-09-10T04:05:15.978 INFO:tasks.workunit.client.0.smithi046.stdout:7/411: fsync d5/fc 0 2024-09-10T04:05:15.980 INFO:tasks.workunit.client.0.smithi046.stdout:5/505: dread d7/d11/d3f/d53/f57 [0,4194304] 0 2024-09-10T04:05:15.980 INFO:tasks.workunit.client.1.smithi070.stdout:8/443: write d4/d16/d17/d1a/f2a [1497568,18958] 0 2024-09-10T04:05:15.981 INFO:tasks.workunit.client.0.smithi046.stdout:5/506: write d7/f70 [507016,59985] 0 2024-09-10T04:05:15.981 INFO:tasks.workunit.client.1.smithi070.stdout:8/444: fsync d4/d16/d17/d19/f72 0 2024-09-10T04:05:15.981 INFO:tasks.workunit.client.0.smithi046.stdout:2/410: mkdir d2/d6/d9d 0 2024-09-10T04:05:15.982 INFO:tasks.workunit.client.1.smithi070.stdout:8/445: fsync d4/f77 0 2024-09-10T04:05:15.983 INFO:tasks.workunit.client.1.smithi070.stdout:8/446: stat d4/d16/d17/d19/d62/f71 0 2024-09-10T04:05:15.984 INFO:tasks.workunit.client.0.smithi046.stdout:5/507: truncate d7/d11/d14/d2f/da3/fbf 530492 0 2024-09-10T04:05:15.984 INFO:tasks.workunit.client.1.smithi070.stdout:9/468: dread d9/d59/d2a/d3f/f48 [0,4194304] 0 2024-09-10T04:05:15.985 INFO:tasks.workunit.client.1.smithi070.stdout:1/338: link d3/d10/f2c d3/f6a 0 2024-09-10T04:05:15.986 INFO:tasks.workunit.client.0.smithi046.stdout:1/298: mkdir d3/d62 0 2024-09-10T04:05:15.988 INFO:tasks.workunit.client.1.smithi070.stdout:0/446: getdents d8/d13/d33/d53 0 2024-09-10T04:05:15.989 INFO:tasks.workunit.client.1.smithi070.stdout:2/362: getdents d15/d18/d29/d2a/d2e/d32/d36 0 2024-09-10T04:05:15.991 INFO:tasks.workunit.client.1.smithi070.stdout:4/330: mkdir de/d13/d18/d26/d72 0 2024-09-10T04:05:15.992 INFO:tasks.workunit.client.1.smithi070.stdout:7/349: dwrite d2/d7/db/d3a/f68 [0,4194304] 0 2024-09-10T04:05:15.992 INFO:tasks.workunit.client.1.smithi070.stdout:4/331: dread - de/d13/d18/d23/f52 zero size 2024-09-10T04:05:15.993 INFO:tasks.workunit.client.1.smithi070.stdout:7/350: fdatasync d2/d7/d9/d18/d23/f47 0 2024-09-10T04:05:15.994 INFO:tasks.workunit.client.1.smithi070.stdout:6/476: creat d0/d3/d1f/d81/f96 x:0 0 0 2024-09-10T04:05:15.994 INFO:tasks.workunit.client.1.smithi070.stdout:6/477: readlink d0/l8b 0 2024-09-10T04:05:15.994 INFO:tasks.workunit.client.0.smithi046.stdout:7/412: read d5/d6/db/d11/d1f/f3b [24299,129115] 0 2024-09-10T04:05:15.997 INFO:tasks.workunit.client.1.smithi070.stdout:6/478: truncate d0/d3/d18/d62/d70/f7e 1147316 0 2024-09-10T04:05:15.999 INFO:tasks.workunit.client.1.smithi070.stdout:6/479: write d0/d3/d18/d1b/f7d [214681,112178] 0 2024-09-10T04:05:16.000 INFO:tasks.workunit.client.1.smithi070.stdout:3/421: mkdir db/d1c/d89 0 2024-09-10T04:05:16.001 INFO:tasks.workunit.client.0.smithi046.stdout:1/299: creat d3/dc/d34/d58/f63 x:0 0 0 2024-09-10T04:05:16.002 INFO:tasks.workunit.client.1.smithi070.stdout:3/422: read - db/d1c/d25/d28/f7c zero size 2024-09-10T04:05:16.002 INFO:tasks.workunit.client.1.smithi070.stdout:4/332: dread de/d13/d18/f3b [0,4194304] 0 2024-09-10T04:05:16.002 INFO:tasks.workunit.client.0.smithi046.stdout:7/413: dread d5/d6/db/d11/f14 [0,4194304] 0 2024-09-10T04:05:16.003 INFO:tasks.workunit.client.0.smithi046.stdout:8/382: mkdir d1/d29/d17/d1f/d46/d6a/d82 0 2024-09-10T04:05:16.005 INFO:tasks.workunit.client.1.smithi070.stdout:8/447: creat d4/d16/d17/d1a/d60/f9c x:0 0 0 2024-09-10T04:05:16.007 INFO:tasks.workunit.client.0.smithi046.stdout:2/411: truncate d2/f3 403427 0 2024-09-10T04:05:16.008 INFO:tasks.workunit.client.1.smithi070.stdout:1/339: creat d3/d8/d12/d2f/d32/f6b x:0 0 0 2024-09-10T04:05:16.009 INFO:tasks.workunit.client.1.smithi070.stdout:4/333: dwrite de/d13/f1a [0,4194304] 0 2024-09-10T04:05:16.009 INFO:tasks.workunit.client.0.smithi046.stdout:7/414: dwrite d5/d6/d32/f74 [0,4194304] 0 2024-09-10T04:05:16.010 INFO:tasks.workunit.client.0.smithi046.stdout:2/412: dread d2/d6/d23/d5d/d5e/f84 [0,4194304] 0 2024-09-10T04:05:16.011 INFO:tasks.workunit.client.0.smithi046.stdout:7/415: fdatasync d5/d6/db/dd/d24/d76/f7c 0 2024-09-10T04:05:16.011 INFO:tasks.workunit.client.1.smithi070.stdout:8/448: dread d4/d16/d17/f68 [0,4194304] 0 2024-09-10T04:05:16.013 INFO:tasks.workunit.client.1.smithi070.stdout:8/449: chown d4/d16/f34 11532939 1 2024-09-10T04:05:16.013 INFO:tasks.workunit.client.1.smithi070.stdout:4/334: dread fc [0,4194304] 0 2024-09-10T04:05:16.014 INFO:tasks.workunit.client.0.smithi046.stdout:9/404: getdents d4/d3c/d4d/d5c/d59 0 2024-09-10T04:05:16.014 INFO:tasks.workunit.client.1.smithi070.stdout:8/450: write d4/d16/d17/d1a/d60/f9c [551810,126893] 0 2024-09-10T04:05:16.015 INFO:tasks.workunit.client.0.smithi046.stdout:9/405: readlink d4/d7/l2e 0 2024-09-10T04:05:16.015 INFO:tasks.workunit.client.0.smithi046.stdout:3/366: truncate d12/d13/f17 1688383 0 2024-09-10T04:05:16.016 INFO:tasks.workunit.client.0.smithi046.stdout:9/406: stat d4/d18/f36 0 2024-09-10T04:05:16.016 INFO:tasks.workunit.client.1.smithi070.stdout:2/363: truncate d15/d18/f20 711087 0 2024-09-10T04:05:16.017 INFO:tasks.workunit.client.1.smithi070.stdout:2/364: chown d15/d18/f7e 3146383 1 2024-09-10T04:05:16.019 INFO:tasks.workunit.client.1.smithi070.stdout:8/451: dread d4/d16/d17/f68 [0,4194304] 0 2024-09-10T04:05:16.020 INFO:tasks.workunit.client.1.smithi070.stdout:8/452: write d4/d39/f57 [661137,56697] 0 2024-09-10T04:05:16.021 INFO:tasks.workunit.client.1.smithi070.stdout:7/351: mkdir d2/d7/db/d3a/d7e 0 2024-09-10T04:05:16.022 INFO:tasks.workunit.client.1.smithi070.stdout:7/352: fdatasync d2/d3/dc/d5d/f1d 0 2024-09-10T04:05:16.022 INFO:tasks.workunit.client.0.smithi046.stdout:2/413: symlink d2/d8/d65/l9e 0 2024-09-10T04:05:16.025 INFO:tasks.workunit.client.1.smithi070.stdout:5/415: fsync fc 0 2024-09-10T04:05:16.026 INFO:tasks.workunit.client.0.smithi046.stdout:7/416: rmdir d5/d6/db/d11/d1e 39 2024-09-10T04:05:16.030 INFO:tasks.workunit.client.0.smithi046.stdout:3/367: symlink d12/d13/d14/d1e/d2d/d39/d67/l73 0 2024-09-10T04:05:16.031 INFO:tasks.workunit.client.0.smithi046.stdout:3/368: chown d12/d13 39 1 2024-09-10T04:05:16.033 INFO:tasks.workunit.client.1.smithi070.stdout:8/453: dread d4/d16/d17/d19/d62/f71 [0,4194304] 0 2024-09-10T04:05:16.035 INFO:tasks.workunit.client.0.smithi046.stdout:1/300: link d3/dc/d21/l5a d3/dc/d21/l64 0 2024-09-10T04:05:16.036 INFO:tasks.workunit.client.1.smithi070.stdout:1/340: creat d3/d10/d1d/f6c x:0 0 0 2024-09-10T04:05:16.038 INFO:tasks.workunit.client.0.smithi046.stdout:3/369: dwrite d12/d13/d14/f47 [0,4194304] 0 2024-09-10T04:05:16.038 INFO:tasks.workunit.client.0.smithi046.stdout:2/414: mkdir d2/d6/d23/d55/d9f 0 2024-09-10T04:05:16.039 INFO:tasks.workunit.client.1.smithi070.stdout:3/423: dwrite db/d1c/d25/d28/d43/f51 [0,4194304] 0 2024-09-10T04:05:16.039 INFO:tasks.workunit.client.1.smithi070.stdout:0/447: dwrite d8/d13/d1a/f7e [0,4194304] 0 2024-09-10T04:05:16.041 INFO:tasks.workunit.client.0.smithi046.stdout:8/383: creat d1/d29/d19/f83 x:0 0 0 2024-09-10T04:05:16.042 INFO:tasks.workunit.client.1.smithi070.stdout:4/335: symlink de/d13/d4f/d6a/l73 0 2024-09-10T04:05:16.044 INFO:tasks.workunit.client.1.smithi070.stdout:4/336: write de/d13/d43/f65 [800377,90122] 0 2024-09-10T04:05:16.045 INFO:tasks.workunit.client.0.smithi046.stdout:7/417: stat d5/d6/db/d11/d1e/d20/l2b 0 2024-09-10T04:05:16.046 INFO:tasks.workunit.client.1.smithi070.stdout:2/365: fsync d15/d18/d29/d2a/d2e/d32/f49 0 2024-09-10T04:05:16.046 INFO:tasks.workunit.client.1.smithi070.stdout:2/366: chown d15/d18/d29/d2a/d2e/d32/d36 1335018 1 2024-09-10T04:05:16.054 INFO:tasks.workunit.client.1.smithi070.stdout:7/353: creat d2/d40/d6c/f7f x:0 0 0 2024-09-10T04:05:16.058 INFO:tasks.workunit.client.0.smithi046.stdout:2/415: creat d2/d6/d23/d42/fa0 x:0 0 0 2024-09-10T04:05:16.059 INFO:tasks.workunit.client.1.smithi070.stdout:6/480: symlink d0/d3/d18/d1b/l97 0 2024-09-10T04:05:16.061 INFO:tasks.workunit.client.0.smithi046.stdout:7/418: mkdir d5/d6/d55/d89 0 2024-09-10T04:05:16.062 INFO:tasks.workunit.client.0.smithi046.stdout:7/419: write d5/d6/db/f2a [2215160,61162] 0 2024-09-10T04:05:16.064 INFO:tasks.workunit.client.0.smithi046.stdout:7/420: write d5/d6/db/f1d [1451903,102869] 0 2024-09-10T04:05:16.064 INFO:tasks.workunit.client.0.smithi046.stdout:1/301: unlink f1 0 2024-09-10T04:05:16.065 INFO:tasks.workunit.client.0.smithi046.stdout:7/421: chown d5/d6/d62/l88 3115567 1 2024-09-10T04:05:16.070 INFO:tasks.workunit.client.0.smithi046.stdout:7/422: dread d5/d9/d12/f5d [0,4194304] 0 2024-09-10T04:05:16.071 INFO:tasks.workunit.client.0.smithi046.stdout:2/416: dread d2/d6/d23/d5d/d5e/f84 [0,4194304] 0 2024-09-10T04:05:16.073 INFO:tasks.workunit.client.0.smithi046.stdout:9/407: getdents d4/d44 0 2024-09-10T04:05:16.074 INFO:tasks.workunit.client.0.smithi046.stdout:9/408: chown d4/d18/f38 21119761 1 2024-09-10T04:05:16.077 INFO:tasks.workunit.client.1.smithi070.stdout:8/454: creat d4/d16/d17/d1a/d60/d7a/f9d x:0 0 0 2024-09-10T04:05:16.077 INFO:tasks.workunit.client.1.smithi070.stdout:4/337: symlink de/d13/d18/l74 0 2024-09-10T04:05:16.078 INFO:tasks.workunit.client.1.smithi070.stdout:4/338: dread - de/d13/f3e zero size 2024-09-10T04:05:16.080 INFO:tasks.workunit.client.1.smithi070.stdout:2/367: mknod d15/d18/d29/d2a/d2e/d32/d36/c80 0 2024-09-10T04:05:16.082 INFO:tasks.workunit.client.1.smithi070.stdout:2/368: read d15/d18/f21 [166501,57191] 0 2024-09-10T04:05:16.082 INFO:tasks.workunit.client.0.smithi046.stdout:3/370: creat d12/f74 x:0 0 0 2024-09-10T04:05:16.085 INFO:tasks.workunit.client.0.smithi046.stdout:2/417: mknod d2/d6/d23/d5d/d77/d7b/d85/ca1 0 2024-09-10T04:05:16.087 INFO:tasks.workunit.client.1.smithi070.stdout:6/481: write d0/d3/d18/d2a/f4f [4555896,110796] 0 2024-09-10T04:05:16.088 INFO:tasks.workunit.client.0.smithi046.stdout:1/302: creat d3/d27/d5d/d60/f65 x:0 0 0 2024-09-10T04:05:16.088 INFO:tasks.workunit.client.1.smithi070.stdout:6/482: write d0/d3/d18/d1b/d17/d28/f5e [890962,82103] 0 2024-09-10T04:05:16.092 INFO:tasks.workunit.client.1.smithi070.stdout:7/354: dwrite d2/d7/db/f37 [0,4194304] 0 2024-09-10T04:05:16.094 INFO:tasks.workunit.client.1.smithi070.stdout:0/448: truncate d8/f27 4855247 0 2024-09-10T04:05:16.095 INFO:tasks.workunit.client.1.smithi070.stdout:1/341: mkdir d3/d8/d13/d3f/d6d 0 2024-09-10T04:05:16.096 INFO:tasks.workunit.client.0.smithi046.stdout:1/303: dwrite d3/dc/d21/d3a/f52 [0,4194304] 0 2024-09-10T04:05:16.096 INFO:tasks.workunit.client.1.smithi070.stdout:0/449: read d8/d24/d2d/f3a [5116154,1362] 0 2024-09-10T04:05:16.096 INFO:tasks.workunit.client.0.smithi046.stdout:3/371: rmdir d12/d13/d14/d1e 39 2024-09-10T04:05:16.098 INFO:tasks.workunit.client.1.smithi070.stdout:8/455: mknod d4/d16/d17/d19/d28/c9e 0 2024-09-10T04:05:16.102 INFO:tasks.workunit.client.1.smithi070.stdout:4/339: creat de/d13/d18/d26/f75 x:0 0 0 2024-09-10T04:05:16.102 INFO:tasks.workunit.client.1.smithi070.stdout:8/456: dread - d4/d16/d17/d1a/f78 zero size 2024-09-10T04:05:16.103 INFO:tasks.workunit.client.0.smithi046.stdout:9/409: mknod d4/d7/d1f/c7b 0 2024-09-10T04:05:16.104 INFO:tasks.workunit.client.0.smithi046.stdout:9/410: write d4/d7/f77 [468208,78019] 0 2024-09-10T04:05:16.105 INFO:tasks.workunit.client.1.smithi070.stdout:8/457: chown d4/d16/f34 601178442 1 2024-09-10T04:05:16.106 INFO:tasks.workunit.client.1.smithi070.stdout:1/342: dwrite d3/d10/d1d/f6c [0,4194304] 0 2024-09-10T04:05:16.106 INFO:tasks.workunit.client.1.smithi070.stdout:0/450: dwrite d8/f1b [0,4194304] 0 2024-09-10T04:05:16.108 INFO:tasks.workunit.client.0.smithi046.stdout:2/418: symlink d2/d6/d9d/la2 0 2024-09-10T04:05:16.109 INFO:tasks.workunit.client.1.smithi070.stdout:8/458: fsync d4/d16/d17/d1a/f78 0 2024-09-10T04:05:16.110 INFO:tasks.workunit.client.1.smithi070.stdout:0/451: chown d8/d13/d33/d4f/f6e 1857 1 2024-09-10T04:05:16.111 INFO:tasks.workunit.client.1.smithi070.stdout:8/459: chown d4/d16/d17/d19/d28/d43 321532315 1 2024-09-10T04:05:16.111 INFO:tasks.workunit.client.0.smithi046.stdout:3/372: symlink d12/d13/d14/l75 0 2024-09-10T04:05:16.112 INFO:tasks.workunit.client.1.smithi070.stdout:8/460: stat d4/c8a 0 2024-09-10T04:05:16.112 INFO:tasks.workunit.client.1.smithi070.stdout:0/452: chown d8/d24/d2d/d34 4083 1 2024-09-10T04:05:16.112 INFO:tasks.workunit.client.1.smithi070.stdout:2/369: dwrite d15/d18/d56/d7c/d55/d6d/f7d [8388608,4194304] 0 2024-09-10T04:05:16.113 INFO:tasks.workunit.client.1.smithi070.stdout:8/461: read - d4/d16/d73/d96/f59 zero size 2024-09-10T04:05:16.114 INFO:tasks.workunit.client.0.smithi046.stdout:3/373: chown d12/d13/d37/d51 131721 1 2024-09-10T04:05:16.117 INFO:tasks.workunit.client.0.smithi046.stdout:7/423: getdents d5/d6/db/d11/d1f/d7a 0 2024-09-10T04:05:16.120 INFO:tasks.workunit.client.0.smithi046.stdout:2/419: dwrite d2/d6/d23/d42/fa0 [0,4194304] 0 2024-09-10T04:05:16.120 INFO:tasks.workunit.client.0.smithi046.stdout:9/411: mkdir d4/d66/d7c 0 2024-09-10T04:05:16.123 INFO:tasks.workunit.client.0.smithi046.stdout:7/424: truncate d5/d6/db/dd/d24/d76/f7c 632284 0 2024-09-10T04:05:16.124 INFO:tasks.workunit.client.0.smithi046.stdout:9/412: read d4/d3c/f6b [3776038,99373] 0 2024-09-10T04:05:16.129 INFO:tasks.workunit.client.1.smithi070.stdout:5/416: getdents d17/d19/d6a/d54/d57/d63/d7b 0 2024-09-10T04:05:16.129 INFO:tasks.workunit.client.1.smithi070.stdout:4/340: read de/f34 [179492,103600] 0 2024-09-10T04:05:16.130 INFO:tasks.workunit.client.1.smithi070.stdout:4/341: stat de/d13/d18/d23/d39/d48 0 2024-09-10T04:05:16.130 INFO:tasks.workunit.client.1.smithi070.stdout:4/342: fsync de/f31 0 2024-09-10T04:05:16.131 INFO:tasks.workunit.client.1.smithi070.stdout:4/343: dread - de/d13/f3e zero size 2024-09-10T04:05:16.132 INFO:tasks.workunit.client.0.smithi046.stdout:1/304: write d3/dc/f23 [578707,40210] 0 2024-09-10T04:05:16.133 INFO:tasks.workunit.client.0.smithi046.stdout:2/420: creat d2/d6/d23/d55/fa3 x:0 0 0 2024-09-10T04:05:16.134 INFO:tasks.workunit.client.0.smithi046.stdout:7/425: symlink d5/d9/l8a 0 2024-09-10T04:05:16.135 INFO:tasks.workunit.client.1.smithi070.stdout:7/355: creat d2/d3/dc/d79/f80 x:0 0 0 2024-09-10T04:05:16.139 INFO:tasks.workunit.client.1.smithi070.stdout:0/453: creat d8/f92 x:0 0 0 2024-09-10T04:05:16.139 INFO:tasks.workunit.client.0.smithi046.stdout:3/374: truncate d12/f4d 2318097 0 2024-09-10T04:05:16.143 INFO:tasks.workunit.client.1.smithi070.stdout:7/356: dwrite d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:16.143 INFO:tasks.workunit.client.1.smithi070.stdout:8/462: creat d4/d16/d17/d5c/d87/f9f x:0 0 0 2024-09-10T04:05:16.144 INFO:tasks.workunit.client.1.smithi070.stdout:8/463: readlink d4/d16/d17/l90 0 2024-09-10T04:05:16.145 INFO:tasks.workunit.client.1.smithi070.stdout:1/343: dread d3/d8/d13/d3f/f42 [0,4194304] 0 2024-09-10T04:05:16.145 INFO:tasks.workunit.client.1.smithi070.stdout:1/344: read d3/d10/f5e [259512,90939] 0 2024-09-10T04:05:16.147 INFO:tasks.workunit.client.0.smithi046.stdout:9/413: creat d4/d66/d7c/f7d x:0 0 0 2024-09-10T04:05:16.150 INFO:tasks.workunit.client.1.smithi070.stdout:8/464: dwrite d4/d16/d17/d19/d28/f46 [0,4194304] 0 2024-09-10T04:05:16.156 INFO:tasks.workunit.client.0.smithi046.stdout:9/414: creat d4/d7/f7e x:0 0 0 2024-09-10T04:05:16.157 INFO:tasks.workunit.client.0.smithi046.stdout:9/415: chown d4/d3c/f40 577 1 2024-09-10T04:05:16.163 INFO:tasks.workunit.client.0.smithi046.stdout:1/305: creat d3/d27/f66 x:0 0 0 2024-09-10T04:05:16.164 INFO:tasks.workunit.client.1.smithi070.stdout:4/344: creat de/d13/d43/f76 x:0 0 0 2024-09-10T04:05:16.164 INFO:tasks.workunit.client.0.smithi046.stdout:9/416: dwrite d4/d3c/d4d/d5c/d59/f5d [0,4194304] 0 2024-09-10T04:05:16.165 INFO:tasks.workunit.client.1.smithi070.stdout:4/345: read - de/d13/d18/d23/f61 zero size 2024-09-10T04:05:16.167 INFO:tasks.workunit.client.0.smithi046.stdout:2/421: creat d2/d6/d15/fa4 x:0 0 0 2024-09-10T04:05:16.170 INFO:tasks.workunit.client.1.smithi070.stdout:6/483: creat d0/d3/f98 x:0 0 0 2024-09-10T04:05:16.172 INFO:tasks.workunit.client.1.smithi070.stdout:4/346: dwrite de/d13/d4f/f56 [0,4194304] 0 2024-09-10T04:05:16.174 INFO:tasks.workunit.client.1.smithi070.stdout:6/484: truncate d0/d3/d18/d1b/f11 8980535 0 2024-09-10T04:05:16.174 INFO:tasks.workunit.client.0.smithi046.stdout:7/426: mkdir d5/d9/d8b 0 2024-09-10T04:05:16.176 INFO:tasks.workunit.client.1.smithi070.stdout:6/485: fdatasync d0/f4d 0 2024-09-10T04:05:16.177 INFO:tasks.workunit.client.1.smithi070.stdout:6/486: chown d0/d3/f98 38093785 1 2024-09-10T04:05:16.178 INFO:tasks.workunit.client.1.smithi070.stdout:6/487: read - d0/d3/d18/d1b/d17/d28/f95 zero size 2024-09-10T04:05:16.179 INFO:tasks.workunit.client.0.smithi046.stdout:1/306: mknod d3/d27/d2a/c67 0 2024-09-10T04:05:16.180 INFO:tasks.workunit.client.1.smithi070.stdout:2/370: creat d15/d18/d29/d64/f81 x:0 0 0 2024-09-10T04:05:16.183 INFO:tasks.workunit.client.0.smithi046.stdout:9/417: creat d4/d3c/d4d/d5c/d59/f7f x:0 0 0 2024-09-10T04:05:16.184 INFO:tasks.workunit.client.0.smithi046.stdout:3/375: dwrite d12/d13/d37/d51/f57 [0,4194304] 0 2024-09-10T04:05:16.186 INFO:tasks.workunit.client.0.smithi046.stdout:9/418: write d4/d7/d1f/d54/f6e [91182,16279] 0 2024-09-10T04:05:16.187 INFO:tasks.workunit.client.1.smithi070.stdout:0/454: symlink d8/d24/l93 0 2024-09-10T04:05:16.187 INFO:tasks.workunit.client.0.smithi046.stdout:3/376: truncate d12/d13/d37/d51/f57 4946230 0 2024-09-10T04:05:16.188 INFO:tasks.workunit.client.1.smithi070.stdout:0/455: write d8/fa [3177813,25956] 0 2024-09-10T04:05:16.189 INFO:tasks.workunit.client.0.smithi046.stdout:1/307: symlink d3/d27/d5d/d60/l68 0 2024-09-10T04:05:16.192 INFO:tasks.workunit.client.0.smithi046.stdout:2/422: getdents d2/d6/d23/d55/d9f 0 2024-09-10T04:05:16.194 INFO:tasks.workunit.client.1.smithi070.stdout:4/347: rename de/d13/d18/d26/d72 to de/d13/d18/d23/d39/d77 0 2024-09-10T04:05:16.194 INFO:tasks.workunit.client.0.smithi046.stdout:2/423: fdatasync d2/d6/d23/d42/fa0 0 2024-09-10T04:05:16.198 INFO:tasks.workunit.client.0.smithi046.stdout:1/308: dwrite d3/d27/d32/f43 [0,4194304] 0 2024-09-10T04:05:16.198 INFO:tasks.workunit.client.1.smithi070.stdout:2/371: creat d15/d18/d29/d2a/d2e/d32/f82 x:0 0 0 2024-09-10T04:05:16.199 INFO:tasks.workunit.client.0.smithi046.stdout:1/309: dread - d3/d27/f66 zero size 2024-09-10T04:05:16.201 INFO:tasks.workunit.client.1.smithi070.stdout:1/345: mknod d3/d8/d12/d61/c6e 0 2024-09-10T04:05:16.204 INFO:tasks.workunit.client.1.smithi070.stdout:7/357: symlink d2/d7/d9/l81 0 2024-09-10T04:05:16.209 INFO:tasks.workunit.client.1.smithi070.stdout:7/358: dread d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:16.210 INFO:tasks.workunit.client.1.smithi070.stdout:0/456: symlink d8/d13/d47/l94 0 2024-09-10T04:05:16.211 INFO:tasks.workunit.client.1.smithi070.stdout:4/348: dread de/d13/f38 [0,4194304] 0 2024-09-10T04:05:16.212 INFO:tasks.workunit.client.0.smithi046.stdout:2/424: stat d2/d6/f43 0 2024-09-10T04:05:16.213 INFO:tasks.workunit.client.1.smithi070.stdout:4/349: dread de/d13/d4f/f66 [0,4194304] 0 2024-09-10T04:05:16.213 INFO:tasks.workunit.client.0.smithi046.stdout:2/425: write d2/d8/d65/f7d [740089,95175] 0 2024-09-10T04:05:16.213 INFO:tasks.workunit.client.1.smithi070.stdout:4/350: fdatasync de/d13/d18/d26/f42 0 2024-09-10T04:05:16.214 INFO:tasks.workunit.client.1.smithi070.stdout:4/351: fdatasync de/d13/d18/f37 0 2024-09-10T04:05:16.215 INFO:tasks.workunit.client.0.smithi046.stdout:3/377: mknod d12/d5d/d71/d49/d5c/c76 0 2024-09-10T04:05:16.216 INFO:tasks.workunit.client.0.smithi046.stdout:3/378: chown d12/d13/d37/d51/c6c 20334 1 2024-09-10T04:05:16.218 INFO:tasks.workunit.client.0.smithi046.stdout:7/427: getdents d5/d6/d39 0 2024-09-10T04:05:16.220 INFO:tasks.workunit.client.0.smithi046.stdout:2/426: fsync d2/d8/d37/f41 0 2024-09-10T04:05:16.221 INFO:tasks.workunit.client.0.smithi046.stdout:2/427: fdatasync d2/d6/d23/d55/fa3 0 2024-09-10T04:05:16.223 INFO:tasks.workunit.client.0.smithi046.stdout:2/428: truncate d2/d6/d23/d42/f53 4392201 0 2024-09-10T04:05:16.224 INFO:tasks.workunit.client.0.smithi046.stdout:2/429: write d2/d8/d65/f7d [1560697,109910] 0 2024-09-10T04:05:16.224 INFO:tasks.workunit.client.0.smithi046.stdout:9/419: write d4/d18/f22 [2593757,24189] 0 2024-09-10T04:05:16.226 INFO:tasks.workunit.client.1.smithi070.stdout:9/469: sync 2024-09-10T04:05:16.227 INFO:tasks.workunit.client.0.smithi046.stdout:9/420: write d4/d29/d32/f43 [193517,87525] 0 2024-09-10T04:05:16.229 INFO:tasks.workunit.client.0.smithi046.stdout:7/428: creat d5/d6/d39/d5a/f8c x:0 0 0 2024-09-10T04:05:16.234 INFO:tasks.workunit.client.0.smithi046.stdout:1/310: rename d3/d27/f38 to d3/d27/d32/f69 0 2024-09-10T04:05:16.235 INFO:tasks.workunit.client.0.smithi046.stdout:9/421: dwrite d4/d3c/f40 [0,4194304] 0 2024-09-10T04:05:16.235 INFO:tasks.workunit.client.0.smithi046.stdout:1/311: dread - d3/d27/f66 zero size 2024-09-10T04:05:16.236 INFO:tasks.workunit.client.0.smithi046.stdout:1/312: fdatasync d3/dc/f23 0 2024-09-10T04:05:16.237 INFO:tasks.workunit.client.0.smithi046.stdout:9/422: dread d4/d7/f1e [0,4194304] 0 2024-09-10T04:05:16.239 INFO:tasks.workunit.client.1.smithi070.stdout:8/465: dwrite d4/d16/f36 [0,4194304] 0 2024-09-10T04:05:16.240 INFO:tasks.workunit.client.1.smithi070.stdout:8/466: dread - d4/d16/d17/d19/d28/d43/d52/f6c zero size 2024-09-10T04:05:16.242 INFO:tasks.workunit.client.1.smithi070.stdout:4/352: creat de/d13/d5b/f78 x:0 0 0 2024-09-10T04:05:16.242 INFO:tasks.workunit.client.1.smithi070.stdout:9/470: dread d9/d7a/f64 [0,4194304] 0 2024-09-10T04:05:16.243 INFO:tasks.workunit.client.0.smithi046.stdout:3/379: write d12/d13/d14/f24 [1479727,78474] 0 2024-09-10T04:05:16.244 INFO:tasks.workunit.client.1.smithi070.stdout:6/488: creat d0/d3/f99 x:0 0 0 2024-09-10T04:05:16.246 INFO:tasks.workunit.client.1.smithi070.stdout:9/471: write d9/d60/d8a/d3c/d62/f7c [665439,33818] 0 2024-09-10T04:05:16.246 INFO:tasks.workunit.client.1.smithi070.stdout:2/372: symlink d15/d18/d56/d7c/l83 0 2024-09-10T04:05:16.247 INFO:tasks.workunit.client.0.smithi046.stdout:7/429: symlink d5/d6/db/dd/d24/d76/l8d 0 2024-09-10T04:05:16.248 INFO:tasks.workunit.client.1.smithi070.stdout:1/346: mknod d3/d8/d13/d3f/d5d/c6f 0 2024-09-10T04:05:16.249 INFO:tasks.workunit.client.1.smithi070.stdout:2/373: fsync d15/d18/d29/d2a/d2e/d32/f33 0 2024-09-10T04:05:16.250 INFO:tasks.workunit.client.1.smithi070.stdout:8/467: dwrite d4/d16/f27 [0,4194304] 0 2024-09-10T04:05:16.251 INFO:tasks.workunit.client.0.smithi046.stdout:2/430: rename d2/d6/d23/d42/fa0 to d2/d6/d9d/fa5 0 2024-09-10T04:05:16.255 INFO:tasks.workunit.client.0.smithi046.stdout:1/313: unlink d3/dc/d21/d3a/f52 0 2024-09-10T04:05:16.256 INFO:tasks.workunit.client.1.smithi070.stdout:9/472: dwrite d9/d59/f7e [0,4194304] 0 2024-09-10T04:05:16.258 INFO:tasks.workunit.client.1.smithi070.stdout:6/489: dread d0/d3/d18/d1b/d17/d28/f5e [0,4194304] 0 2024-09-10T04:05:16.258 INFO:tasks.workunit.client.1.smithi070.stdout:9/473: write d9/d59/f18 [4987070,41566] 0 2024-09-10T04:05:16.260 INFO:tasks.workunit.client.1.smithi070.stdout:9/474: write d9/d7a/d72/f7b [642597,59602] 0 2024-09-10T04:05:16.265 INFO:tasks.workunit.client.0.smithi046.stdout:7/430: mknod d5/d6/db/dd/d4b/c8e 0 2024-09-10T04:05:16.266 INFO:tasks.workunit.client.1.smithi070.stdout:1/347: dread d3/f47 [0,4194304] 0 2024-09-10T04:05:16.270 INFO:tasks.workunit.client.0.smithi046.stdout:2/431: mknod d2/d6/d23/d5d/d77/d7b/d85/ca6 0 2024-09-10T04:05:16.273 INFO:tasks.workunit.client.0.smithi046.stdout:1/314: stat d3/dc/c11 0 2024-09-10T04:05:16.274 INFO:tasks.workunit.client.0.smithi046.stdout:1/315: write d3/f4a [630534,4687] 0 2024-09-10T04:05:16.276 INFO:tasks.workunit.client.0.smithi046.stdout:9/423: write d4/d44/f49 [61224,122091] 0 2024-09-10T04:05:16.278 INFO:tasks.workunit.client.1.smithi070.stdout:0/457: dwrite d8/d13/d1a/f3c [0,4194304] 0 2024-09-10T04:05:16.280 INFO:tasks.workunit.client.1.smithi070.stdout:8/468: rename d4/d16/d17/d19/c47 to d4/d16/d17/d19/d62/ca0 0 2024-09-10T04:05:16.281 INFO:tasks.workunit.client.1.smithi070.stdout:4/353: write de/d13/d4f/f66 [555117,126188] 0 2024-09-10T04:05:16.281 INFO:tasks.workunit.client.0.smithi046.stdout:7/431: symlink d5/d6/db/dd/d24/d76/l8f 0 2024-09-10T04:05:16.285 INFO:tasks.workunit.client.0.smithi046.stdout:1/316: rmdir d3/d27 39 2024-09-10T04:05:16.287 INFO:tasks.workunit.client.0.smithi046.stdout:3/380: truncate d12/d13/d14/f36 2187318 0 2024-09-10T04:05:16.288 INFO:tasks.workunit.client.0.smithi046.stdout:9/424: mkdir d4/d3c/d4d/d80 0 2024-09-10T04:05:16.290 INFO:tasks.workunit.client.0.smithi046.stdout:7/432: write d5/d6/db/f38 [4038478,33387] 0 2024-09-10T04:05:16.291 INFO:tasks.workunit.client.1.smithi070.stdout:1/348: fsync d3/d8/d12/f24 0 2024-09-10T04:05:16.292 INFO:tasks.workunit.client.0.smithi046.stdout:2/432: symlink d2/la7 0 2024-09-10T04:05:16.293 INFO:tasks.workunit.client.1.smithi070.stdout:1/349: dread d3/d10/f3b [0,4194304] 0 2024-09-10T04:05:16.293 INFO:tasks.workunit.client.1.smithi070.stdout:1/350: chown d3/d8/d12/d2f/f34 549 1 2024-09-10T04:05:16.296 INFO:tasks.workunit.client.0.smithi046.stdout:3/381: mknod d12/d13/c77 0 2024-09-10T04:05:16.297 INFO:tasks.workunit.client.1.smithi070.stdout:3/424: sync 2024-09-10T04:05:16.297 INFO:tasks.workunit.client.1.smithi070.stdout:7/359: link d2/d7/fe d2/d7/d9/f82 0 2024-09-10T04:05:16.298 INFO:tasks.workunit.client.1.smithi070.stdout:1/351: dread d3/d8/d13/d3f/f53 [0,4194304] 0 2024-09-10T04:05:16.298 INFO:tasks.workunit.client.0.smithi046.stdout:9/425: mkdir d4/d7/d81 0 2024-09-10T04:05:16.299 INFO:tasks.workunit.client.1.smithi070.stdout:3/425: truncate db/d1c/d25/d28/d43/f71 154559 0 2024-09-10T04:05:16.299 INFO:tasks.workunit.client.0.smithi046.stdout:9/426: write d4/d3c/d4d/d5c/d59/f7f [664911,310] 0 2024-09-10T04:05:16.301 INFO:tasks.workunit.client.1.smithi070.stdout:6/490: rename d0/d3/d6a to d0/d3/d8c/d9a 0 2024-09-10T04:05:16.303 INFO:tasks.workunit.client.1.smithi070.stdout:6/491: dread d0/d3/d18/d1b/d17/f47 [0,4194304] 0 2024-09-10T04:05:16.305 INFO:tasks.workunit.client.0.smithi046.stdout:3/382: rename d12/c41 to d12/d1b/d50/c78 0 2024-09-10T04:05:16.306 INFO:tasks.workunit.client.1.smithi070.stdout:9/475: symlink d9/d1c/d34/d8b/l8e 0 2024-09-10T04:05:16.306 INFO:tasks.workunit.client.0.smithi046.stdout:3/383: write d12/f63 [333284,25250] 0 2024-09-10T04:05:16.306 INFO:tasks.workunit.client.0.smithi046.stdout:7/433: truncate d5/d9/f5b 434940 0 2024-09-10T04:05:16.307 INFO:tasks.workunit.client.0.smithi046.stdout:7/434: chown d5/d6/db/d11 94610821 1 2024-09-10T04:05:16.309 INFO:tasks.workunit.client.1.smithi070.stdout:2/374: link d15/d18/l2c d15/d59/l84 0 2024-09-10T04:05:16.310 INFO:tasks.workunit.client.1.smithi070.stdout:0/458: mkdir d8/d13/d1a/d8f/d95 0 2024-09-10T04:05:16.311 INFO:tasks.workunit.client.0.smithi046.stdout:7/435: symlink d5/d6/d32/d59/l90 0 2024-09-10T04:05:16.312 INFO:tasks.workunit.client.1.smithi070.stdout:9/476: dwrite d9/d60/d8a/d1b/f5c [0,4194304] 0 2024-09-10T04:05:16.312 INFO:tasks.workunit.client.0.smithi046.stdout:9/427: dread d4/f6 [0,4194304] 0 2024-09-10T04:05:16.312 INFO:tasks.workunit.client.0.smithi046.stdout:1/317: fdatasync d3/d27/d32/d1f/f29 0 2024-09-10T04:05:16.314 INFO:tasks.workunit.client.0.smithi046.stdout:2/433: getdents d2/d8/d37 0 2024-09-10T04:05:16.315 INFO:tasks.workunit.client.0.smithi046.stdout:3/384: dread d12/d13/f2c [0,4194304] 0 2024-09-10T04:05:16.316 INFO:tasks.workunit.client.1.smithi070.stdout:1/352: rename d3/d8/d27/l38 to d3/d8/d13/d3f/d5d/l70 0 2024-09-10T04:05:16.318 INFO:tasks.workunit.client.0.smithi046.stdout:7/436: chown d5/d9/d12/d72 2453 1 2024-09-10T04:05:16.320 INFO:tasks.workunit.client.1.smithi070.stdout:8/469: unlink d4/c9 0 2024-09-10T04:05:16.322 INFO:tasks.workunit.client.0.smithi046.stdout:0/323: sync 2024-09-10T04:05:16.322 INFO:tasks.workunit.client.0.smithi046.stdout:9/428: dwrite d4/d44/f78 [0,4194304] 0 2024-09-10T04:05:16.329 INFO:tasks.workunit.client.0.smithi046.stdout:1/318: creat d3/d27/d5d/f6a x:0 0 0 2024-09-10T04:05:16.335 INFO:tasks.workunit.client.1.smithi070.stdout:7/360: mknod d2/d7/db/d3a/d7e/c83 0 2024-09-10T04:05:16.337 INFO:tasks.workunit.client.0.smithi046.stdout:0/324: chown d1/dc/d25/d33/d3/dd/d55/c57 763 1 2024-09-10T04:05:16.343 INFO:tasks.workunit.client.1.smithi070.stdout:0/459: symlink d8/l96 0 2024-09-10T04:05:16.344 INFO:tasks.workunit.client.0.smithi046.stdout:1/319: symlink d3/d27/d2a/l6b 0 2024-09-10T04:05:16.348 INFO:tasks.workunit.client.1.smithi070.stdout:4/354: rename de/d13/d4f to de/d13/d18/d23/d47/d59/d79 0 2024-09-10T04:05:16.349 INFO:tasks.workunit.client.0.smithi046.stdout:0/325: symlink d1/dc/d25/d33/d1a/d41/l68 0 2024-09-10T04:05:16.350 INFO:tasks.workunit.client.1.smithi070.stdout:4/355: stat de/d13/d18/d23/d47/d59 0 2024-09-10T04:05:16.350 INFO:tasks.workunit.client.0.smithi046.stdout:2/434: write d2/f25 [850686,67822] 0 2024-09-10T04:05:16.351 INFO:tasks.workunit.client.1.smithi070.stdout:1/353: fsync d3/d8/d13/f25 0 2024-09-10T04:05:16.352 INFO:tasks.workunit.client.0.smithi046.stdout:0/326: fdatasync d1/dc/d25/d33/d3/d29/f40 0 2024-09-10T04:05:16.352 INFO:tasks.workunit.client.0.smithi046.stdout:2/435: write d2/d8/d37/d63/f75 [1292014,79995] 0 2024-09-10T04:05:16.352 INFO:tasks.workunit.client.1.smithi070.stdout:6/492: symlink d0/d3/d8c/d9a/d92/l9b 0 2024-09-10T04:05:16.352 INFO:tasks.workunit.client.1.smithi070.stdout:4/356: fsync de/d13/d18/d23/d39/f5a 0 2024-09-10T04:05:16.352 INFO:tasks.workunit.client.1.smithi070.stdout:1/354: fdatasync d3/d8/fc 0 2024-09-10T04:05:16.354 INFO:tasks.workunit.client.1.smithi070.stdout:2/375: dwrite d15/d18/f63 [0,4194304] 0 2024-09-10T04:05:16.354 INFO:tasks.workunit.client.1.smithi070.stdout:7/361: rename d2/d3/d63/c65 to d2/d3/d63/c84 0 2024-09-10T04:05:16.354 INFO:tasks.workunit.client.1.smithi070.stdout:1/355: chown d3/d10/f5e 816 1 2024-09-10T04:05:16.357 INFO:tasks.workunit.client.0.smithi046.stdout:1/320: symlink d3/d27/d32/d28/l6c 0 2024-09-10T04:05:16.358 INFO:tasks.workunit.client.0.smithi046.stdout:1/321: write d3/dc/f23 [966272,32757] 0 2024-09-10T04:05:16.359 INFO:tasks.workunit.client.1.smithi070.stdout:6/493: dwrite d0/f4d [0,4194304] 0 2024-09-10T04:05:16.361 INFO:tasks.workunit.client.1.smithi070.stdout:1/356: dwrite d3/f60 [0,4194304] 0 2024-09-10T04:05:16.362 INFO:tasks.workunit.client.0.smithi046.stdout:7/437: link d5/d9/d12/d72/l86 d5/d33/l91 0 2024-09-10T04:05:16.363 INFO:tasks.workunit.client.1.smithi070.stdout:1/357: read d3/d8/d12/d2f/f34 [3945802,93967] 0 2024-09-10T04:05:16.365 INFO:tasks.workunit.client.1.smithi070.stdout:3/426: creat db/d1c/d25/d28/f8a x:0 0 0 2024-09-10T04:05:16.366 INFO:tasks.workunit.client.0.smithi046.stdout:7/438: dread d5/d6/f1c [0,4194304] 0 2024-09-10T04:05:16.373 INFO:tasks.workunit.client.0.smithi046.stdout:9/429: link d4/d7/f8 d4/d3c/d4d/d5c/f82 0 2024-09-10T04:05:16.374 INFO:tasks.workunit.client.0.smithi046.stdout:2/436: mknod d2/d6/d23/d5d/ca8 0 2024-09-10T04:05:16.376 INFO:tasks.workunit.client.0.smithi046.stdout:1/322: mknod d3/dc/d34/c6d 0 2024-09-10T04:05:16.377 INFO:tasks.workunit.client.0.smithi046.stdout:3/385: getdents d12/d13/d37 0 2024-09-10T04:05:16.379 INFO:tasks.workunit.client.1.smithi070.stdout:4/357: read de/f11 [436925,60205] 0 2024-09-10T04:05:16.380 INFO:tasks.workunit.client.0.smithi046.stdout:1/323: dread d3/d27/f20 [0,4194304] 0 2024-09-10T04:05:16.382 INFO:tasks.workunit.client.0.smithi046.stdout:9/430: mknod d4/d29/d65/c83 0 2024-09-10T04:05:16.384 INFO:tasks.workunit.client.0.smithi046.stdout:2/437: dwrite d2/d6/d15/fa4 [0,4194304] 0 2024-09-10T04:05:16.387 INFO:tasks.workunit.client.0.smithi046.stdout:1/324: dread d3/f26 [0,4194304] 0 2024-09-10T04:05:16.389 INFO:tasks.workunit.client.0.smithi046.stdout:9/431: dread d4/f50 [0,4194304] 0 2024-09-10T04:05:16.389 INFO:tasks.workunit.client.1.smithi070.stdout:3/427: rename db/d1c/d25/d28/d43/f78 to db/d1c/d25/d28/d43/f8b 0 2024-09-10T04:05:16.390 INFO:tasks.workunit.client.1.smithi070.stdout:3/428: chown db/d1c/d25/l3a 454 1 2024-09-10T04:05:16.390 INFO:tasks.workunit.client.0.smithi046.stdout:9/432: truncate d4/d7/d1f/f15 4774232 0 2024-09-10T04:05:16.391 INFO:tasks.workunit.client.0.smithi046.stdout:9/433: fdatasync d4/d7/f1e 0 2024-09-10T04:05:16.392 INFO:tasks.workunit.client.1.smithi070.stdout:3/429: write db/d1c/d53/f6e [594453,55120] 0 2024-09-10T04:05:16.392 INFO:tasks.workunit.client.1.smithi070.stdout:9/477: creat d9/d1c/f8f x:0 0 0 2024-09-10T04:05:16.395 INFO:tasks.workunit.client.1.smithi070.stdout:8/470: rename d4/d16/d73/c92 to d4/d16/d17/d19/d28/d43/d52/ca1 0 2024-09-10T04:05:16.402 INFO:tasks.workunit.client.1.smithi070.stdout:8/471: dread d4/f77 [0,4194304] 0 2024-09-10T04:05:16.403 INFO:tasks.workunit.client.1.smithi070.stdout:0/460: truncate d8/d24/f2a 2721489 0 2024-09-10T04:05:16.403 INFO:tasks.workunit.client.0.smithi046.stdout:7/439: dwrite d5/d6/db/dd/d4b/f52 [0,4194304] 0 2024-09-10T04:05:16.404 INFO:tasks.workunit.client.1.smithi070.stdout:4/358: creat de/d13/d18/d23/d47/d59/d79/d6a/f7a x:0 0 0 2024-09-10T04:05:16.404 INFO:tasks.workunit.client.1.smithi070.stdout:4/359: stat de/d13/f1a 0 2024-09-10T04:05:16.406 INFO:tasks.workunit.client.0.smithi046.stdout:1/325: creat d3/dc/f6e x:0 0 0 2024-09-10T04:05:16.408 INFO:tasks.workunit.client.0.smithi046.stdout:9/434: dread d4/d18/f22 [0,4194304] 0 2024-09-10T04:05:16.411 INFO:tasks.workunit.client.0.smithi046.stdout:3/386: truncate d12/d13/f18 779488 0 2024-09-10T04:05:16.419 INFO:tasks.workunit.client.1.smithi070.stdout:2/376: dwrite d15/d18/f20 [0,4194304] 0 2024-09-10T04:05:16.419 INFO:tasks.workunit.client.1.smithi070.stdout:4/360: dread de/d13/f45 [0,4194304] 0 2024-09-10T04:05:16.420 INFO:tasks.workunit.client.0.smithi046.stdout:9/435: creat d4/d3c/f84 x:0 0 0 2024-09-10T04:05:16.421 INFO:tasks.workunit.client.0.smithi046.stdout:9/436: chown d4/d18/f33 577541136 1 2024-09-10T04:05:16.422 INFO:tasks.workunit.client.0.smithi046.stdout:9/437: truncate d4/d29/d65/f67 976671 0 2024-09-10T04:05:16.426 INFO:tasks.workunit.client.1.smithi070.stdout:0/461: mkdir d8/d24/d97 0 2024-09-10T04:05:16.427 INFO:tasks.workunit.client.1.smithi070.stdout:0/462: write d8/f1b [1730325,38092] 0 2024-09-10T04:05:16.428 INFO:tasks.workunit.client.1.smithi070.stdout:8/472: dread d4/d16/d17/d19/d62/f71 [0,4194304] 0 2024-09-10T04:05:16.431 INFO:tasks.workunit.client.0.smithi046.stdout:2/438: dwrite d2/d6/d23/d5d/d77/d7b/f92 [0,4194304] 0 2024-09-10T04:05:16.432 INFO:tasks.workunit.client.0.smithi046.stdout:1/326: write d3/d27/d32/d1f/f61 [56769,43042] 0 2024-09-10T04:05:16.433 INFO:tasks.workunit.client.1.smithi070.stdout:4/361: read f8 [867065,33058] 0 2024-09-10T04:05:16.433 INFO:tasks.workunit.client.0.smithi046.stdout:1/327: fdatasync d3/dc/d34/d58/f63 0 2024-09-10T04:05:16.434 INFO:tasks.workunit.client.1.smithi070.stdout:2/377: creat d15/d18/d56/f85 x:0 0 0 2024-09-10T04:05:16.436 INFO:tasks.workunit.client.0.smithi046.stdout:9/438: creat d4/d44/f85 x:0 0 0 2024-09-10T04:05:16.437 INFO:tasks.workunit.client.0.smithi046.stdout:9/439: fsync d4/d3c/f40 0 2024-09-10T04:05:16.438 INFO:tasks.workunit.client.1.smithi070.stdout:3/430: creat db/d1c/f8c x:0 0 0 2024-09-10T04:05:16.438 INFO:tasks.workunit.client.0.smithi046.stdout:9/440: chown d4/d29/d65/f67 3005 1 2024-09-10T04:05:16.441 INFO:tasks.workunit.client.0.smithi046.stdout:7/440: truncate d5/d6/db/f1d 4016475 0 2024-09-10T04:05:16.443 INFO:tasks.workunit.client.1.smithi070.stdout:5/417: sync 2024-09-10T04:05:16.443 INFO:tasks.workunit.client.0.smithi046.stdout:2/439: creat d2/d8/d65/fa9 x:0 0 0 2024-09-10T04:05:16.444 INFO:tasks.workunit.client.0.smithi046.stdout:3/387: symlink d12/d13/d14/d1e/l79 0 2024-09-10T04:05:16.445 INFO:tasks.workunit.client.0.smithi046.stdout:3/388: truncate d12/f74 638021 0 2024-09-10T04:05:16.449 INFO:tasks.workunit.client.1.smithi070.stdout:8/473: creat d4/d16/d17/d5c/d87/fa2 x:0 0 0 2024-09-10T04:05:16.449 INFO:tasks.workunit.client.0.smithi046.stdout:7/441: creat d5/d6/db/d11/d1e/d20/f92 x:0 0 0 2024-09-10T04:05:16.451 INFO:tasks.workunit.client.1.smithi070.stdout:4/362: mkdir de/d13/d5b/d7b 0 2024-09-10T04:05:16.451 INFO:tasks.workunit.client.1.smithi070.stdout:4/363: write f0 [1112917,108523] 0 2024-09-10T04:05:16.452 INFO:tasks.workunit.client.1.smithi070.stdout:2/378: dread - d15/d18/f1a zero size 2024-09-10T04:05:16.455 INFO:tasks.workunit.client.1.smithi070.stdout:6/494: rename d0/d3/d18/c3c to d0/d3/d18/d1b/d17/c9c 0 2024-09-10T04:05:16.455 INFO:tasks.workunit.client.0.smithi046.stdout:2/440: mknod d2/d6/d15/caa 0 2024-09-10T04:05:16.457 INFO:tasks.workunit.client.1.smithi070.stdout:6/495: write d0/d3/d18/d1b/d17/d28/f3b [995578,55335] 0 2024-09-10T04:05:16.457 INFO:tasks.workunit.client.1.smithi070.stdout:2/379: dwrite d15/d18/d29/d2a/f58 [8388608,4194304] 0 2024-09-10T04:05:16.458 INFO:tasks.workunit.client.1.smithi070.stdout:5/418: fdatasync d17/d24/d45/f7e 0 2024-09-10T04:05:16.459 INFO:tasks.workunit.client.1.smithi070.stdout:2/380: chown d15/d18/d42 2821 1 2024-09-10T04:05:16.460 INFO:tasks.workunit.client.0.smithi046.stdout:7/442: rename d5/d6/d39/d5a/f8c to d5/d6/db/dd/d24/d76/f93 0 2024-09-10T04:05:16.461 INFO:tasks.workunit.client.1.smithi070.stdout:2/381: read d15/d18/d29/d2a/d2e/d32/f33 [1931476,52511] 0 2024-09-10T04:05:16.462 INFO:tasks.workunit.client.1.smithi070.stdout:0/463: symlink d8/d13/d33/d71/l98 0 2024-09-10T04:05:16.462 INFO:tasks.workunit.client.1.smithi070.stdout:2/382: chown d15/d18/d29/d2a/d2e/d32/f82 708328 1 2024-09-10T04:05:16.463 INFO:tasks.workunit.client.1.smithi070.stdout:0/464: write d8/d13/d1a/f31 [3740252,48859] 0 2024-09-10T04:05:16.464 INFO:tasks.workunit.client.1.smithi070.stdout:8/474: truncate d4/d16/d17/d19/f5b 6415 0 2024-09-10T04:05:16.465 INFO:tasks.workunit.client.1.smithi070.stdout:7/362: sync 2024-09-10T04:05:16.465 INFO:tasks.workunit.client.1.smithi070.stdout:8/475: dread - d4/d16/d73/f93 zero size 2024-09-10T04:05:16.468 INFO:tasks.workunit.client.0.smithi046.stdout:2/441: mknod d2/d6/d9d/cab 0 2024-09-10T04:05:16.469 INFO:tasks.workunit.client.1.smithi070.stdout:4/364: mknod de/d13/d18/d23/d39/d48/c7c 0 2024-09-10T04:05:16.470 INFO:tasks.workunit.client.0.smithi046.stdout:3/389: write d12/d13/f2c [188507,25408] 0 2024-09-10T04:05:16.470 INFO:tasks.workunit.client.0.smithi046.stdout:9/441: write d4/d18/f4f [2694749,68227] 0 2024-09-10T04:05:16.472 INFO:tasks.workunit.client.0.smithi046.stdout:7/443: symlink d5/d6/d39/d5f/l94 0 2024-09-10T04:05:16.472 INFO:tasks.workunit.client.0.smithi046.stdout:3/390: stat d12/d5d/d71/d49/c61 0 2024-09-10T04:05:16.474 INFO:tasks.workunit.client.1.smithi070.stdout:1/358: rename d3/d8/l57 to d3/d8/d12/d2f/d32/d4b/l71 0 2024-09-10T04:05:16.475 INFO:tasks.workunit.client.1.smithi070.stdout:1/359: stat d3/d8/d13/d3f/d5d 0 2024-09-10T04:05:16.475 INFO:tasks.workunit.client.0.smithi046.stdout:2/442: mkdir d2/d6/d23/d5d/d77/dac 0 2024-09-10T04:05:16.477 INFO:tasks.workunit.client.0.smithi046.stdout:9/442: creat d4/d44/f86 x:0 0 0 2024-09-10T04:05:16.481 INFO:tasks.workunit.client.0.smithi046.stdout:3/391: mkdir d12/d1b/d50/d7a 0 2024-09-10T04:05:16.485 INFO:tasks.workunit.client.0.smithi046.stdout:7/444: dwrite d5/d9/d43/f5c [0,4194304] 0 2024-09-10T04:05:16.486 INFO:tasks.workunit.client.0.smithi046.stdout:9/443: unlink d4/d44/f78 0 2024-09-10T04:05:16.488 INFO:tasks.workunit.client.1.smithi070.stdout:2/383: creat d15/d18/d56/f86 x:0 0 0 2024-09-10T04:05:16.488 INFO:tasks.workunit.client.1.smithi070.stdout:2/384: chown d15/d18/d29/d2a 6456 1 2024-09-10T04:05:16.491 INFO:tasks.workunit.client.1.smithi070.stdout:0/465: creat d8/f99 x:0 0 0 2024-09-10T04:05:16.493 INFO:tasks.workunit.client.1.smithi070.stdout:7/363: mknod d2/d7/db/d3a/d7e/c85 0 2024-09-10T04:05:16.494 INFO:tasks.workunit.client.1.smithi070.stdout:7/364: fdatasync d2/d7/f43 0 2024-09-10T04:05:16.496 INFO:tasks.workunit.client.0.smithi046.stdout:7/445: fsync d5/d6/db/d11/d1f/f3b 0 2024-09-10T04:05:16.500 INFO:tasks.workunit.client.1.smithi070.stdout:7/365: dwrite d2/d7/d9/d18/f6b [0,4194304] 0 2024-09-10T04:05:16.502 INFO:tasks.workunit.client.0.smithi046.stdout:3/392: dread d12/d13/d14/f36 [0,4194304] 0 2024-09-10T04:05:16.502 INFO:tasks.workunit.client.1.smithi070.stdout:9/478: rename d9/d59/d5d/d88 to d9/d7a/d6f/d90 0 2024-09-10T04:05:16.502 INFO:tasks.workunit.client.1.smithi070.stdout:7/366: dread d2/d7/d9/d18/d3c/f60 [0,4194304] 0 2024-09-10T04:05:16.504 INFO:tasks.workunit.client.0.smithi046.stdout:7/446: creat d5/d6/d32/f95 x:0 0 0 2024-09-10T04:05:16.505 INFO:tasks.workunit.client.1.smithi070.stdout:5/419: dwrite d17/d24/d45/f4f [0,4194304] 0 2024-09-10T04:05:16.507 INFO:tasks.workunit.client.0.smithi046.stdout:9/444: mkdir d4/d3c/d4d/d5c/d87 0 2024-09-10T04:05:16.508 INFO:tasks.workunit.client.0.smithi046.stdout:9/445: readlink d4/d3c/d4d/d5c/d59/l70 0 2024-09-10T04:05:16.508 INFO:tasks.workunit.client.1.smithi070.stdout:6/496: symlink d0/d3/d18/d1b/d17/d3f/l9d 0 2024-09-10T04:05:16.509 INFO:tasks.workunit.client.0.smithi046.stdout:3/393: mknod d12/d5d/c7b 0 2024-09-10T04:05:16.511 INFO:tasks.workunit.client.0.smithi046.stdout:7/447: creat d5/d6/d39/d5f/f96 x:0 0 0 2024-09-10T04:05:16.512 INFO:tasks.workunit.client.0.smithi046.stdout:7/448: readlink d5/d6/d32/d59/l90 0 2024-09-10T04:05:16.512 INFO:tasks.workunit.client.1.smithi070.stdout:2/385: unlink d15/d18/d56/d7c/d55/f7b 0 2024-09-10T04:05:16.513 INFO:tasks.workunit.client.0.smithi046.stdout:7/449: chown d5/d6/d39/d5f/l83 16224556 1 2024-09-10T04:05:16.513 INFO:tasks.workunit.client.0.smithi046.stdout:9/446: mknod d4/d66/c88 0 2024-09-10T04:05:16.515 INFO:tasks.workunit.client.0.smithi046.stdout:3/394: symlink d12/d5d/l7c 0 2024-09-10T04:05:16.519 INFO:tasks.workunit.client.0.smithi046.stdout:3/395: mknod d12/d5d/d71/d49/c7d 0 2024-09-10T04:05:16.521 INFO:tasks.workunit.client.0.smithi046.stdout:9/447: mkdir d4/d18/d89 0 2024-09-10T04:05:16.527 INFO:tasks.workunit.client.0.smithi046.stdout:3/396: mknod d12/d1b/d50/c7e 0 2024-09-10T04:05:16.529 INFO:tasks.workunit.client.1.smithi070.stdout:3/431: rename db/d1c/d25/d28/f8a to db/d83/f8d 0 2024-09-10T04:05:16.529 INFO:tasks.workunit.client.0.smithi046.stdout:6/383: sync 2024-09-10T04:05:16.530 INFO:tasks.workunit.client.0.smithi046.stdout:4/440: sync 2024-09-10T04:05:16.530 INFO:tasks.workunit.client.0.smithi046.stdout:9/448: unlink d4/d7/d1f/l3a 0 2024-09-10T04:05:16.532 INFO:tasks.workunit.client.0.smithi046.stdout:4/441: truncate d3/d12/d17/d29/f92 56722 0 2024-09-10T04:05:16.533 INFO:tasks.workunit.client.1.smithi070.stdout:9/479: creat d9/d59/d2a/d3f/f91 x:0 0 0 2024-09-10T04:05:16.534 INFO:tasks.workunit.client.1.smithi070.stdout:7/367: mknod d2/d7/d9/d18/d3c/d52/c86 0 2024-09-10T04:05:16.535 INFO:tasks.workunit.client.0.smithi046.stdout:9/449: mknod d4/d7/d1f/d54/d56/c8a 0 2024-09-10T04:05:16.536 INFO:tasks.workunit.client.0.smithi046.stdout:9/450: stat d4/d7/d1f/ld 0 2024-09-10T04:05:16.536 INFO:tasks.workunit.client.1.smithi070.stdout:5/420: dread - d17/d37/f3d zero size 2024-09-10T04:05:16.536 INFO:tasks.workunit.client.0.smithi046.stdout:9/451: fsync d4/d18/f27 0 2024-09-10T04:05:16.538 INFO:tasks.workunit.client.0.smithi046.stdout:7/450: truncate d5/d6/db/d11/d1e/f3e 36248 0 2024-09-10T04:05:16.539 INFO:tasks.workunit.client.1.smithi070.stdout:5/421: dread d17/d19/f44 [0,4194304] 0 2024-09-10T04:05:16.539 INFO:tasks.workunit.client.0.smithi046.stdout:7/451: chown d5/d6/db/d11/d1f 2414275 1 2024-09-10T04:05:16.540 INFO:tasks.workunit.client.0.smithi046.stdout:4/442: chown d3/d43/d5f/d85 11131876 1 2024-09-10T04:05:16.540 INFO:tasks.workunit.client.1.smithi070.stdout:5/422: write d17/d19/d6a/d54/d57/f65 [142584,123966] 0 2024-09-10T04:05:16.540 INFO:tasks.workunit.client.1.smithi070.stdout:5/423: dread - d17/d37/f3d zero size 2024-09-10T04:05:16.541 INFO:tasks.workunit.client.1.smithi070.stdout:5/424: chown d17/d24/f27 186 1 2024-09-10T04:05:16.547 INFO:tasks.workunit.client.0.smithi046.stdout:3/397: link d12/d5d/d71/d49/l53 d12/d1b/d50/l7f 0 2024-09-10T04:05:16.548 INFO:tasks.workunit.client.1.smithi070.stdout:4/365: link de/d13/f45 de/d13/d18/d23/d47/d59/f7d 0 2024-09-10T04:05:16.549 INFO:tasks.workunit.client.0.smithi046.stdout:7/452: truncate d5/d6/d39/d5a/f64 5071174 0 2024-09-10T04:05:16.553 INFO:tasks.workunit.client.1.smithi070.stdout:3/432: rmdir db 39 2024-09-10T04:05:16.556 INFO:tasks.workunit.client.0.smithi046.stdout:6/384: dwrite da/f1f [0,4194304] 0 2024-09-10T04:05:16.557 INFO:tasks.workunit.client.1.smithi070.stdout:1/360: creat d3/d10/f72 x:0 0 0 2024-09-10T04:05:16.559 INFO:tasks.workunit.client.0.smithi046.stdout:4/443: mknod d3/d12/d17/d91/c95 0 2024-09-10T04:05:16.559 INFO:tasks.workunit.client.1.smithi070.stdout:7/368: rmdir d2/d7/db/d3a 39 2024-09-10T04:05:16.560 INFO:tasks.workunit.client.1.smithi070.stdout:0/466: dwrite d8/d13/d1a/f35 [0,4194304] 0 2024-09-10T04:05:16.562 INFO:tasks.workunit.client.1.smithi070.stdout:6/497: stat d0/d3/d1f/d81/l88 0 2024-09-10T04:05:16.564 INFO:tasks.workunit.client.1.smithi070.stdout:5/425: symlink d17/d24/d4e/l8a 0 2024-09-10T04:05:16.566 INFO:tasks.workunit.client.0.smithi046.stdout:3/398: truncate f4 870430 0 2024-09-10T04:05:16.566 INFO:tasks.workunit.client.0.smithi046.stdout:3/399: chown c10 2734 1 2024-09-10T04:05:16.567 INFO:tasks.workunit.client.0.smithi046.stdout:9/452: link d4/d29/f6d d4/d7/d1f/d54/d56/f8b 0 2024-09-10T04:05:16.568 INFO:tasks.workunit.client.1.smithi070.stdout:8/476: getdents d4/d16/d17/d19/d28/d43/d52 0 2024-09-10T04:05:16.569 INFO:tasks.workunit.client.1.smithi070.stdout:8/477: fsync d4/d16/d17/d19/d28/f31 0 2024-09-10T04:05:16.570 INFO:tasks.workunit.client.0.smithi046.stdout:4/444: mknod d3/d12/d17/d91/c96 0 2024-09-10T04:05:16.571 INFO:tasks.workunit.client.1.smithi070.stdout:9/480: dread d9/d7a/f44 [0,4194304] 0 2024-09-10T04:05:16.572 INFO:tasks.workunit.client.1.smithi070.stdout:4/366: creat de/d13/d18/d23/d39/d48/f7e x:0 0 0 2024-09-10T04:05:16.573 INFO:tasks.workunit.client.1.smithi070.stdout:5/426: dwrite d17/d19/d6a/f7c [0,4194304] 0 2024-09-10T04:05:16.574 INFO:tasks.workunit.client.0.smithi046.stdout:6/385: dread da/db/d11/d2b/d2f/f65 [0,4194304] 0 2024-09-10T04:05:16.575 INFO:tasks.workunit.client.1.smithi070.stdout:5/427: stat d17/d6f/d32 0 2024-09-10T04:05:16.575 INFO:tasks.workunit.client.1.smithi070.stdout:1/361: fsync d3/d8/d27/f68 0 2024-09-10T04:05:16.577 INFO:tasks.workunit.client.0.smithi046.stdout:6/386: truncate da/db/d10/d27/f36 4240727 0 2024-09-10T04:05:16.577 INFO:tasks.workunit.client.1.smithi070.stdout:5/428: truncate d17/d19/d6a/f7c 4950696 0 2024-09-10T04:05:16.578 INFO:tasks.workunit.client.0.smithi046.stdout:3/400: creat d12/d13/d14/d1e/d2d/f80 x:0 0 0 2024-09-10T04:05:16.578 INFO:tasks.workunit.client.1.smithi070.stdout:9/481: dread d9/d60/d8a/d1b/f2d [4194304,4194304] 0 2024-09-10T04:05:16.579 INFO:tasks.workunit.client.0.smithi046.stdout:6/387: write da/f7b [438160,110801] 0 2024-09-10T04:05:16.583 INFO:tasks.workunit.client.1.smithi070.stdout:3/433: dread db/f14 [0,4194304] 0 2024-09-10T04:05:16.583 INFO:tasks.workunit.client.0.smithi046.stdout:3/401: creat d12/d13/d37/f81 x:0 0 0 2024-09-10T04:05:16.585 INFO:tasks.workunit.client.0.smithi046.stdout:3/402: write d12/d13/f2c [1321271,16659] 0 2024-09-10T04:05:16.585 INFO:tasks.workunit.client.1.smithi070.stdout:9/482: dread d9/d7a/d72/f7b [0,4194304] 0 2024-09-10T04:05:16.588 INFO:tasks.workunit.client.1.smithi070.stdout:9/483: dread d9/f22 [0,4194304] 0 2024-09-10T04:05:16.589 INFO:tasks.workunit.client.1.smithi070.stdout:2/386: creat d15/d18/d29/d2a/d2e/f87 x:0 0 0 2024-09-10T04:05:16.591 INFO:tasks.workunit.client.1.smithi070.stdout:6/498: dwrite d0/d3/d18/d2a/f59 [0,4194304] 0 2024-09-10T04:05:16.592 INFO:tasks.workunit.client.1.smithi070.stdout:2/387: write d15/d18/d29/d2a/d2e/f87 [535300,36419] 0 2024-09-10T04:05:16.594 INFO:tasks.workunit.client.1.smithi070.stdout:2/388: write d15/d18/f27 [2122971,85755] 0 2024-09-10T04:05:16.597 INFO:tasks.workunit.client.1.smithi070.stdout:8/478: symlink d4/d16/d17/d19/d28/d43/d67/d97/la3 0 2024-09-10T04:05:16.599 INFO:tasks.workunit.client.1.smithi070.stdout:8/479: chown d4/d16/d73/l8c 1012 1 2024-09-10T04:05:16.601 INFO:tasks.workunit.client.0.smithi046.stdout:4/445: rmdir d3/d12/d93 0 2024-09-10T04:05:16.601 INFO:tasks.workunit.client.0.smithi046.stdout:6/388: dread da/f24 [0,4194304] 0 2024-09-10T04:05:16.602 INFO:tasks.workunit.client.1.smithi070.stdout:8/480: chown d4/d16/d17/d19/d28/f31 847824755 1 2024-09-10T04:05:16.603 INFO:tasks.workunit.client.1.smithi070.stdout:8/481: stat d4/f11 0 2024-09-10T04:05:16.603 INFO:tasks.workunit.client.1.smithi070.stdout:4/367: mkdir de/d13/d18/d5c/d7f 0 2024-09-10T04:05:16.603 INFO:tasks.workunit.client.1.smithi070.stdout:2/389: dwrite d15/d18/f21 [4194304,4194304] 0 2024-09-10T04:05:16.604 INFO:tasks.workunit.client.0.smithi046.stdout:6/389: dread da/db/d11/d2b/d2f/f66 [0,4194304] 0 2024-09-10T04:05:16.605 INFO:tasks.workunit.client.1.smithi070.stdout:9/484: dread d9/d59/f18 [0,4194304] 0 2024-09-10T04:05:16.606 INFO:tasks.workunit.client.1.smithi070.stdout:8/482: stat d4/d16/d17/d19/d28/d43/d67/d97 0 2024-09-10T04:05:16.606 INFO:tasks.workunit.client.0.smithi046.stdout:3/403: mknod d12/d13/d14/d1e/d2d/d6f/c82 0 2024-09-10T04:05:16.606 INFO:tasks.workunit.client.1.smithi070.stdout:4/368: truncate de/d13/d18/d23/d39/f6d 785018 0 2024-09-10T04:05:16.607 INFO:tasks.workunit.client.0.smithi046.stdout:3/404: read - d12/d13/f70 zero size 2024-09-10T04:05:16.607 INFO:tasks.workunit.client.1.smithi070.stdout:5/429: mknod d17/d24/d77/c8b 0 2024-09-10T04:05:16.611 INFO:tasks.workunit.client.1.smithi070.stdout:9/485: dread d9/d7a/f64 [0,4194304] 0 2024-09-10T04:05:16.611 INFO:tasks.workunit.client.1.smithi070.stdout:0/467: mkdir d8/d13/d1a/d9a 0 2024-09-10T04:05:16.613 INFO:tasks.workunit.client.0.smithi046.stdout:4/446: rename d3/d12/d17/d29/f87 to d3/d12/d17/d91/f97 0 2024-09-10T04:05:16.614 INFO:tasks.workunit.client.0.smithi046.stdout:4/447: chown d3/d12/d17/d22/d51/c65 1719576433 1 2024-09-10T04:05:16.616 INFO:tasks.workunit.client.0.smithi046.stdout:6/390: creat da/db/d10/d51/f80 x:0 0 0 2024-09-10T04:05:16.617 INFO:tasks.workunit.client.1.smithi070.stdout:7/369: link d2/d7/d9/f31 d2/d3/d63/f87 0 2024-09-10T04:05:16.618 INFO:tasks.workunit.client.1.smithi070.stdout:7/370: write d2/d40/d6c/f7f [796004,123481] 0 2024-09-10T04:05:16.619 INFO:tasks.workunit.client.0.smithi046.stdout:3/405: mknod d12/d1b/d50/c83 0 2024-09-10T04:05:16.623 INFO:tasks.workunit.client.0.smithi046.stdout:6/391: dwrite da/db/d11/f32 [0,4194304] 0 2024-09-10T04:05:16.627 INFO:tasks.workunit.client.1.smithi070.stdout:6/499: rename d0/d3/d18/c8f to d0/d3/d1f/d55/c9e 0 2024-09-10T04:05:16.630 INFO:tasks.workunit.client.0.smithi046.stdout:4/448: link d3/d12/d17/d22/f70 d3/d43/f98 0 2024-09-10T04:05:16.631 INFO:tasks.workunit.client.0.smithi046.stdout:6/392: dwrite da/db/d10/f5d [0,4194304] 0 2024-09-10T04:05:16.641 INFO:tasks.workunit.client.0.smithi046.stdout:6/393: dwrite da/db/d18/f35 [4194304,4194304] 0 2024-09-10T04:05:16.641 INFO:tasks.workunit.client.1.smithi070.stdout:2/390: symlink d15/d18/d56/d7c/d55/l88 0 2024-09-10T04:05:16.646 INFO:tasks.workunit.client.0.smithi046.stdout:6/394: dread da/dc/d22/f52 [0,4194304] 0 2024-09-10T04:05:16.646 INFO:tasks.workunit.client.0.smithi046.stdout:6/395: fdatasync da/f7b 0 2024-09-10T04:05:16.650 INFO:tasks.workunit.client.0.smithi046.stdout:4/449: creat d3/d12/d17/d29/f99 x:0 0 0 2024-09-10T04:05:16.653 INFO:tasks.workunit.client.1.smithi070.stdout:5/430: read f11 [1894873,20625] 0 2024-09-10T04:05:16.654 INFO:tasks.workunit.client.1.smithi070.stdout:9/486: truncate d9/d60/d8a/d3c/f40 666905 0 2024-09-10T04:05:16.654 INFO:tasks.workunit.client.1.smithi070.stdout:5/431: dread d17/d19/d35/f5f [0,4194304] 0 2024-09-10T04:05:16.656 INFO:tasks.workunit.client.0.smithi046.stdout:3/406: getdents d12/d13/d14/d1e/d2d/d39 0 2024-09-10T04:05:16.658 INFO:tasks.workunit.client.0.smithi046.stdout:4/450: dwrite d3/d12/d17/d22/d30/f82 [0,4194304] 0 2024-09-10T04:05:16.660 INFO:tasks.workunit.client.0.smithi046.stdout:6/396: creat da/db/d11/d2b/d55/f81 x:0 0 0 2024-09-10T04:05:16.661 INFO:tasks.workunit.client.1.smithi070.stdout:8/483: dread d4/d16/f3f [0,4194304] 0 2024-09-10T04:05:16.661 INFO:tasks.workunit.client.0.smithi046.stdout:6/397: chown da/db/d11 8673973 1 2024-09-10T04:05:16.661 INFO:tasks.workunit.client.1.smithi070.stdout:7/371: symlink d2/d7/d9/d18/d23/l88 0 2024-09-10T04:05:16.663 INFO:tasks.workunit.client.1.smithi070.stdout:8/484: write d4/d16/d17/d19/d28/d43/d52/f6c [828876,7876] 0 2024-09-10T04:05:16.664 INFO:tasks.workunit.client.1.smithi070.stdout:6/500: readlink d0/d3/l2c 0 2024-09-10T04:05:16.665 INFO:tasks.workunit.client.1.smithi070.stdout:8/485: truncate d4/d16/d17/d1a/f78 215022 0 2024-09-10T04:05:16.666 INFO:tasks.workunit.client.1.smithi070.stdout:4/369: rename de/d13/d43/f53 to de/d13/d5b/f80 0 2024-09-10T04:05:16.666 INFO:tasks.workunit.client.1.smithi070.stdout:6/501: fdatasync d0/d3/d18/d1b/d17/d28/f3b 0 2024-09-10T04:05:16.667 INFO:tasks.workunit.client.1.smithi070.stdout:8/486: truncate d4/d16/d73/f93 680427 0 2024-09-10T04:05:16.669 INFO:tasks.workunit.client.1.smithi070.stdout:1/362: creat d3/d10/f73 x:0 0 0 2024-09-10T04:05:16.669 INFO:tasks.workunit.client.1.smithi070.stdout:4/370: read de/d13/d43/f65 [1456182,7113] 0 2024-09-10T04:05:16.671 INFO:tasks.workunit.client.0.smithi046.stdout:3/407: mknod d12/d13/d14/c84 0 2024-09-10T04:05:16.674 INFO:tasks.workunit.client.1.smithi070.stdout:1/363: dwrite d3/d8/f2d [0,4194304] 0 2024-09-10T04:05:16.676 INFO:tasks.workunit.client.1.smithi070.stdout:3/434: creat db/d1c/f8e x:0 0 0 2024-09-10T04:05:16.677 INFO:tasks.workunit.client.0.smithi046.stdout:3/408: dread d12/d13/d14/f47 [0,4194304] 0 2024-09-10T04:05:16.678 INFO:tasks.workunit.client.0.smithi046.stdout:3/409: chown d12/d13/d14 2583549 1 2024-09-10T04:05:16.678 INFO:tasks.workunit.client.1.smithi070.stdout:3/435: write db/d1c/d25/d28/f73 [73416,80114] 0 2024-09-10T04:05:16.679 INFO:tasks.workunit.client.1.smithi070.stdout:7/372: mknod d2/d3/d63/c89 0 2024-09-10T04:05:16.680 INFO:tasks.workunit.client.1.smithi070.stdout:3/436: read db/f4e [1245267,72527] 0 2024-09-10T04:05:16.681 INFO:tasks.workunit.client.1.smithi070.stdout:7/373: read d2/d3/dc/d5d/f1d [2921348,77558] 0 2024-09-10T04:05:16.681 INFO:tasks.workunit.client.1.smithi070.stdout:3/437: stat db/d1c/d25/d28/l68 0 2024-09-10T04:05:16.684 INFO:tasks.workunit.client.1.smithi070.stdout:8/487: dread d4/d16/d17/d1a/f2a [0,4194304] 0 2024-09-10T04:05:16.686 INFO:tasks.workunit.client.1.smithi070.stdout:8/488: truncate d4/d16/d17/d5c/d87/fa2 177976 0 2024-09-10T04:05:16.687 INFO:tasks.workunit.client.1.smithi070.stdout:8/489: chown d4/d16/d17/d5c/l7e 7190 1 2024-09-10T04:05:16.688 INFO:tasks.workunit.client.0.smithi046.stdout:6/398: creat da/db/d11/d13/d1e/f82 x:0 0 0 2024-09-10T04:05:16.689 INFO:tasks.workunit.client.0.smithi046.stdout:6/399: readlink da/db/d10/d27/l2d 0 2024-09-10T04:05:16.690 INFO:tasks.workunit.client.0.smithi046.stdout:4/451: dwrite d3/d12/d17/d22/f59 [0,4194304] 0 2024-09-10T04:05:16.690 INFO:tasks.workunit.client.1.smithi070.stdout:2/391: link d15/d18/d56/f85 d15/d2b/d6e/f89 0 2024-09-10T04:05:16.694 INFO:tasks.workunit.client.0.smithi046.stdout:3/410: mkdir d12/d1b/d50/d7a/d85 0 2024-09-10T04:05:16.695 INFO:tasks.workunit.client.1.smithi070.stdout:6/502: dwrite d0/d3/d18/f52 [4194304,4194304] 0 2024-09-10T04:05:16.696 INFO:tasks.workunit.client.1.smithi070.stdout:3/438: dread db/f33 [4194304,4194304] 0 2024-09-10T04:05:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.695+0000 7f73edffb640 1 -- 172.21.15.46:0/3003476211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f73e0031740 con 0x7f73f0101960 2024-09-10T04:05:16.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.696+0000 7f73f527f640 1 -- 172.21.15.46:0/3003476211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 msgr2=0x7f73f0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.696+0000 7f73f527f640 1 --2- 172.21.15.46:0/3003476211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 0x7f73f0101d60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009880 tx=0x7f73e002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.696+0000 7f73f527f640 1 -- 172.21.15.46:0/3003476211 shutdown_connections 2024-09-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.696+0000 7f73f527f640 1 --2- 172.21.15.46:0/3003476211 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73f0102b60 0x7f73f0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.696+0000 7f73f527f640 1 --2- 172.21.15.46:0/3003476211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 0x7f73f0101d60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.696+0000 7f73f527f640 1 -- 172.21.15.46:0/3003476211 >> 172.21.15.46:0/3003476211 conn(0x7f73f00fd0f0 msgr2=0x7f73f00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.697+0000 7f73f527f640 1 -- 172.21.15.46:0/3003476211 shutdown_connections 2024-09-10T04:05:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.697+0000 7f73f527f640 1 -- 172.21.15.46:0/3003476211 wait complete. 2024-09-10T04:05:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.698+0000 7f73f527f640 1 Processor -- start 2024-09-10T04:05:16.700 INFO:tasks.workunit.client.1.smithi070.stdout:3/439: dread - db/d1c/f8e zero size 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.698+0000 7f73f527f640 1 -- start start 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73f527f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 0x7f73f01105d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73f527f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73f0102b60 0x7f73f0110b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73f527f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73f01085a0 con 0x7f73f0101960 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73f527f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f73f0108420 con 0x7f73f0102b60 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73ee7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73f0102b60 0x7f73f0110b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73eeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 0x7f73f01105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73ee7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73f0102b60 0x7f73f0110b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:38544/0 (socket says 172.21.15.46:38544) 2024-09-10T04:05:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.699+0000 7f73ee7fc640 1 -- 172.21.15.46:0/1206283186 learned_addr learned my addr 172.21.15.46:0/1206283186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:16.702 INFO:tasks.workunit.client.1.smithi070.stdout:2/392: dwrite d15/d18/d29/d2a/d2e/d32/f73 [0,4194304] 0 2024-09-10T04:05:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.700+0000 7f73eeffd640 1 -- 172.21.15.46:0/1206283186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73f0102b60 msgr2=0x7f73f0110b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.700+0000 7f73eeffd640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73f0102b60 0x7f73f0110b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.700+0000 7f73eeffd640 1 -- 172.21.15.46:0/1206283186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f0111190 con 0x7f73f0101960 2024-09-10T04:05:16.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.701+0000 7f73eeffd640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 0x7f73f01105d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006010 tx=0x7f73e00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:16.703 INFO:tasks.workunit.client.1.smithi070.stdout:2/393: fdatasync d15/d18/f20 0 2024-09-10T04:05:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.701+0000 7f73e7fff640 1 -- 172.21.15.46:0/1206283186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f73e003f070 con 0x7f73f0101960 2024-09-10T04:05:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.702+0000 7f73e7fff640 1 -- 172.21.15.46:0/1206283186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f73e0004a50 con 0x7f73f0101960 2024-09-10T04:05:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.702+0000 7f73e7fff640 1 -- 172.21.15.46:0/1206283186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f73e0035030 con 0x7f73f0101960 2024-09-10T04:05:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.702+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f0113b60 con 0x7f73f0101960 2024-09-10T04:05:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.702+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f01140f0 con 0x7f73f0101960 2024-09-10T04:05:16.705 INFO:tasks.workunit.client.0.smithi046.stdout:6/400: rename da/db/d63/c6c to da/db/d10/c83 0 2024-09-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.703+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f010a490 con 0x7f73f0101960 2024-09-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.705+0000 7f73e7fff640 1 -- 172.21.15.46:0/1206283186 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f73e0042070 con 0x7f73f0101960 2024-09-10T04:05:16.707 INFO:tasks.workunit.client.1.smithi070.stdout:7/374: dread d2/d7/fe [0,4194304] 0 2024-09-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.705+0000 7f73e7fff640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f73d406dfa0 0x7f73d4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.706+0000 7f73e7fff640 1 -- 172.21.15.46:0/1206283186 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f73e00ae490 con 0x7f73f0101960 2024-09-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.706+0000 7f73ee7fc640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f73d406dfa0 0x7f73d4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.707+0000 7f73ee7fc640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f73d406dfa0 0x7f73d4070460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73f0111a80 tx=0x7f73dc00b410 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:16.711 INFO:tasks.workunit.client.1.smithi070.stdout:7/375: dwrite d2/d7/d9/d18/f6b [0,4194304] 0 2024-09-10T04:05:16.711 INFO:tasks.workunit.client.1.smithi070.stdout:7/376: stat d2/d3/d63 0 2024-09-10T04:05:16.716 INFO:tasks.workunit.client.1.smithi070.stdout:4/371: creat de/d13/d5b/d7b/f81 x:0 0 0 2024-09-10T04:05:16.716 INFO:tasks.workunit.client.0.smithi046.stdout:6/401: readlink da/d59/l5b 0 2024-09-10T04:05:16.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:16.715+0000 7f73e7fff640 1 -- 172.21.15.46:0/1206283186 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73e007fd40 con 0x7f73f0101960 2024-09-10T04:05:16.718 INFO:tasks.workunit.client.1.smithi070.stdout:4/372: write de/d13/d18/d23/d39/f57 [7298995,110739] 0 2024-09-10T04:05:16.718 INFO:tasks.workunit.client.1.smithi070.stdout:5/432: rmdir d17/d19/d6a/d54/d6e 0 2024-09-10T04:05:16.718 INFO:tasks.workunit.client.0.smithi046.stdout:6/402: write da/dc/d22/f34 [3636891,53153] 0 2024-09-10T04:05:16.721 INFO:tasks.workunit.client.0.smithi046.stdout:3/411: dread d12/d13/d14/d1e/f3f [0,4194304] 0 2024-09-10T04:05:16.721 INFO:tasks.workunit.client.1.smithi070.stdout:2/394: creat d15/d18/d29/d2a/d2e/d32/d36/f8a x:0 0 0 2024-09-10T04:05:16.724 INFO:tasks.workunit.client.0.smithi046.stdout:6/403: dread da/dc/d22/f34 [0,4194304] 0 2024-09-10T04:05:16.727 INFO:tasks.workunit.client.0.smithi046.stdout:3/412: read d12/f5f [2400900,2552] 0 2024-09-10T04:05:16.727 INFO:tasks.workunit.client.1.smithi070.stdout:6/503: mknod d0/d3/d18/d1b/c9f 0 2024-09-10T04:05:16.728 INFO:tasks.workunit.client.1.smithi070.stdout:9/487: getdents d9/d1c/d34/d63 0 2024-09-10T04:05:16.731 INFO:tasks.workunit.client.1.smithi070.stdout:3/440: creat db/d1c/d89/f8f x:0 0 0 2024-09-10T04:05:16.731 INFO:tasks.workunit.client.0.smithi046.stdout:6/404: dwrite da/db/d11/d13/f7c [0,4194304] 0 2024-09-10T04:05:16.733 INFO:tasks.workunit.client.1.smithi070.stdout:1/364: rename d3/d8/d12/d61/c6e to d3/d10/c74 0 2024-09-10T04:05:16.735 INFO:tasks.workunit.client.1.smithi070.stdout:0/468: sync 2024-09-10T04:05:16.735 INFO:tasks.workunit.client.1.smithi070.stdout:0/469: chown d8/d13/d47 121 1 2024-09-10T04:05:16.736 INFO:tasks.workunit.client.1.smithi070.stdout:4/373: mknod de/d13/d5b/c82 0 2024-09-10T04:05:16.736 INFO:tasks.workunit.client.1.smithi070.stdout:0/470: chown d8/d24/d36/c3b 2 1 2024-09-10T04:05:16.736 INFO:tasks.workunit.client.1.smithi070.stdout:4/374: readlink de/d13/d18/l74 0 2024-09-10T04:05:16.740 INFO:tasks.workunit.client.0.smithi046.stdout:3/413: rename d12/d1b/d50/d7a/d85 to d12/d1b/d50/d52/d86 0 2024-09-10T04:05:16.740 INFO:tasks.workunit.client.0.smithi046.stdout:5/508: sync 2024-09-10T04:05:16.740 INFO:tasks.workunit.client.0.smithi046.stdout:8/384: sync 2024-09-10T04:05:16.741 INFO:tasks.workunit.client.1.smithi070.stdout:4/375: read f8 [25856,89625] 0 2024-09-10T04:05:16.742 INFO:tasks.workunit.client.0.smithi046.stdout:4/452: truncate d3/d12/d17/d22/f59 1658073 0 2024-09-10T04:05:16.743 INFO:tasks.workunit.client.1.smithi070.stdout:2/395: rmdir d15/d18/d42 39 2024-09-10T04:05:16.746 INFO:tasks.workunit.client.0.smithi046.stdout:6/405: rename da/db/f50 to da/db/d11/d13/d1e/d69/f84 0 2024-09-10T04:05:16.747 INFO:tasks.workunit.client.0.smithi046.stdout:8/385: dwrite d1/d29/d19/d36/f74 [0,4194304] 0 2024-09-10T04:05:16.781 INFO:tasks.workunit.client.1.smithi070.stdout:9/488: mkdir d9/d60/d8a/d3c/d92 0 2024-09-10T04:05:16.784 INFO:tasks.workunit.client.1.smithi070.stdout:3/441: read db/d1c/d3e/f7f [1080797,86667] 0 2024-09-10T04:05:16.785 INFO:tasks.workunit.client.1.smithi070.stdout:8/490: rename d4/d5f to d4/d16/d17/d5c/d74/da4 0 2024-09-10T04:05:16.786 INFO:tasks.workunit.client.1.smithi070.stdout:8/491: fsync d4/d16/d17/d19/d62/f6e 0 2024-09-10T04:05:16.787 INFO:tasks.workunit.client.1.smithi070.stdout:1/365: chown d3/d8/le 5168039 1 2024-09-10T04:05:16.788 INFO:tasks.workunit.client.0.smithi046.stdout:4/453: fdatasync d3/d12/d17/d29/d58/f75 0 2024-09-10T04:05:16.788 INFO:tasks.workunit.client.1.smithi070.stdout:1/366: write d3/d8/d13/f43 [2518883,63505] 0 2024-09-10T04:05:16.789 INFO:tasks.workunit.client.1.smithi070.stdout:0/471: mkdir d8/d24/d2d/d34/d9b 0 2024-09-10T04:05:16.791 INFO:tasks.workunit.client.0.smithi046.stdout:5/509: rename d7/f1b to d7/d11/d3f/d5a/d63/fc4 0 2024-09-10T04:05:16.793 INFO:tasks.workunit.client.0.smithi046.stdout:6/406: rmdir da/db/d11/d13/d1e/d23 39 2024-09-10T04:05:16.794 INFO:tasks.workunit.client.1.smithi070.stdout:2/396: symlink d15/d18/d29/d2a/d2e/d32/l8b 0 2024-09-10T04:05:16.794 INFO:tasks.workunit.client.1.smithi070.stdout:1/367: dwrite d3/d8/d13/f43 [4194304,4194304] 0 2024-09-10T04:05:16.798 INFO:tasks.workunit.client.1.smithi070.stdout:5/433: write d17/d24/d45/f7e [128598,941] 0 2024-09-10T04:05:16.799 INFO:tasks.workunit.client.1.smithi070.stdout:5/434: write d17/d19/d6a/f53 [1258537,30762] 0 2024-09-10T04:05:16.802 INFO:tasks.workunit.client.1.smithi070.stdout:9/489: creat d9/d7a/f93 x:0 0 0 2024-09-10T04:05:16.803 INFO:tasks.workunit.client.1.smithi070.stdout:9/490: fsync d9/d59/f7e 0 2024-09-10T04:05:16.809 INFO:tasks.workunit.client.1.smithi070.stdout:3/442: mknod db/d1c/d53/c90 0 2024-09-10T04:05:16.809 INFO:tasks.workunit.client.1.smithi070.stdout:3/443: readlink db/d1c/d3e/l69 0 2024-09-10T04:05:16.812 INFO:tasks.workunit.client.1.smithi070.stdout:7/377: rename d2/d7/d9/d18/c45 to d2/d7/db/d3a/d7e/c8a 0 2024-09-10T04:05:16.816 INFO:tasks.workunit.client.0.smithi046.stdout:8/386: write d1/d29/d19/f34 [956253,106523] 0 2024-09-10T04:05:16.818 INFO:tasks.workunit.client.0.smithi046.stdout:6/407: fsync f6 0 2024-09-10T04:05:16.818 INFO:tasks.workunit.client.0.smithi046.stdout:3/414: truncate d12/d13/d37/f54 45295 0 2024-09-10T04:05:16.823 INFO:tasks.workunit.client.1.smithi070.stdout:0/472: rmdir d8/d13/d33/d74 39 2024-09-10T04:05:16.824 INFO:tasks.workunit.client.0.smithi046.stdout:8/387: mknod d1/d29/d17/c84 0 2024-09-10T04:05:16.825 INFO:tasks.workunit.client.1.smithi070.stdout:2/397: symlink d15/d18/d56/l8c 0 2024-09-10T04:05:16.826 INFO:tasks.workunit.client.1.smithi070.stdout:2/398: dread - d15/d18/d56/f85 zero size 2024-09-10T04:05:16.827 INFO:tasks.workunit.client.1.smithi070.stdout:2/399: write d15/d18/d29/d2a/d2e/d32/f57 [600544,36251] 0 2024-09-10T04:05:16.829 INFO:tasks.workunit.client.0.smithi046.stdout:5/510: dwrite d7/d11/d14/d45/f4e [0,4194304] 0 2024-09-10T04:05:16.832 INFO:tasks.workunit.client.1.smithi070.stdout:6/504: link d0/d3/d18/d2a/f59 d0/d3/d18/d1b/fa0 0 2024-09-10T04:05:16.833 INFO:tasks.workunit.client.1.smithi070.stdout:2/400: dwrite d15/d18/d29/d2a/d2e/d32/f33 [0,4194304] 0 2024-09-10T04:05:16.834 INFO:tasks.workunit.client.1.smithi070.stdout:6/505: write d0/d3/d18/d1b/f72 [993265,85151] 0 2024-09-10T04:05:16.835 INFO:tasks.workunit.client.0.smithi046.stdout:3/415: creat d12/d13/d14/d1e/d2d/d39/f87 x:0 0 0 2024-09-10T04:05:16.836 INFO:tasks.workunit.client.0.smithi046.stdout:3/416: fsync d12/d13/d37/d51/f57 0 2024-09-10T04:05:16.837 INFO:tasks.workunit.client.1.smithi070.stdout:5/435: creat d17/d24/d77/f8c x:0 0 0 2024-09-10T04:05:16.837 INFO:tasks.workunit.client.0.smithi046.stdout:3/417: stat d12/d5d/d71/d49/d5c 0 2024-09-10T04:05:16.837 INFO:tasks.workunit.client.0.smithi046.stdout:4/454: creat d3/d12/f9a x:0 0 0 2024-09-10T04:05:16.840 INFO:tasks.workunit.client.0.smithi046.stdout:4/455: chown d3/d12/d17/d22/d51/d5c/c7d 0 1 2024-09-10T04:05:16.842 INFO:tasks.workunit.client.1.smithi070.stdout:4/376: rename de/d13/l32 to de/d13/d18/d23/d39/d48/l83 0 2024-09-10T04:05:16.849 INFO:tasks.workunit.client.1.smithi070.stdout:0/473: creat d8/d13/d1a/d8f/f9c x:0 0 0 2024-09-10T04:05:16.849 INFO:tasks.workunit.client.1.smithi070.stdout:2/401: dread d15/d18/f27 [0,4194304] 0 2024-09-10T04:05:16.850 INFO:tasks.workunit.client.1.smithi070.stdout:0/474: chown d8/d13/d33/c56 75 1 2024-09-10T04:05:16.850 INFO:tasks.workunit.client.1.smithi070.stdout:0/475: stat d8/d13/d1a/f35 0 2024-09-10T04:05:16.853 INFO:tasks.workunit.client.0.smithi046.stdout:6/408: write da/db/d11/d2b/d2f/f66 [42019,44589] 0 2024-09-10T04:05:16.855 INFO:tasks.workunit.client.1.smithi070.stdout:1/368: symlink d3/d8/d13/d3f/l75 0 2024-09-10T04:05:16.857 INFO:tasks.workunit.client.0.smithi046.stdout:3/418: creat d12/d13/d14/d1e/d2d/f88 x:0 0 0 2024-09-10T04:05:16.858 INFO:tasks.workunit.client.0.smithi046.stdout:3/419: readlink d12/d13/d14/d1e/d2d/d39/d67/l73 0 2024-09-10T04:05:16.861 INFO:tasks.workunit.client.0.smithi046.stdout:4/456: mknod d3/d12/d17/d22/d30/c9b 0 2024-09-10T04:05:16.863 INFO:tasks.workunit.client.1.smithi070.stdout:5/436: creat d17/d37/f8d x:0 0 0 2024-09-10T04:05:16.866 INFO:tasks.workunit.client.1.smithi070.stdout:9/491: rename d9/d60/d8a/d1b/l38 to d9/d60/d8a/d3c/d62/l94 0 2024-09-10T04:05:16.867 INFO:tasks.workunit.client.0.smithi046.stdout:8/388: dwrite d1/d29/f4a [0,4194304] 0 2024-09-10T04:05:16.872 INFO:tasks.workunit.client.1.smithi070.stdout:8/492: link d4/cd d4/d16/d17/d19/d28/d43/d52/ca5 0 2024-09-10T04:05:16.877 INFO:tasks.workunit.client.0.smithi046.stdout:4/457: dread d3/d12/d17/d22/d30/f72 [0,4194304] 0 2024-09-10T04:05:16.878 INFO:tasks.workunit.client.0.smithi046.stdout:8/389: rename d1/d29/d19/d36/d62/f71 to d1/d29/d17/d1f/d2a/f85 0 2024-09-10T04:05:16.879 INFO:tasks.workunit.client.0.smithi046.stdout:3/420: mknod d12/d1b/d4b/c89 0 2024-09-10T04:05:16.881 INFO:tasks.workunit.client.1.smithi070.stdout:0/476: dwrite d8/d13/d1a/f30 [0,4194304] 0 2024-09-10T04:05:16.881 INFO:tasks.workunit.client.1.smithi070.stdout:7/378: rename d2/d3/dc/d13/l6e to d2/d7/d9/d18/d49/l8b 0 2024-09-10T04:05:16.885 INFO:tasks.workunit.client.1.smithi070.stdout:4/377: link de/d13/d18/d23/d47/d59/d79/f66 de/d13/d18/d5c/d7f/f84 0 2024-09-10T04:05:16.886 INFO:tasks.workunit.client.0.smithi046.stdout:5/511: getdents d7/d11/d3f/d53/da9 0 2024-09-10T04:05:16.887 INFO:tasks.workunit.client.0.smithi046.stdout:6/409: creat da/d59/f85 x:0 0 0 2024-09-10T04:05:16.887 INFO:tasks.workunit.client.1.smithi070.stdout:4/378: stat de/d13/d18/d23/d47/d59/d79/d6a 0 2024-09-10T04:05:16.888 INFO:tasks.workunit.client.1.smithi070.stdout:8/493: symlink d4/d16/d17/d19/d62/la6 0 2024-09-10T04:05:16.892 INFO:tasks.workunit.client.1.smithi070.stdout:8/494: dread d4/f12 [0,4194304] 0 2024-09-10T04:05:16.892 INFO:tasks.workunit.client.0.smithi046.stdout:4/458: creat d3/d12/d17/f9c x:0 0 0 2024-09-10T04:05:16.895 INFO:tasks.workunit.client.0.smithi046.stdout:6/410: dwrite da/db/d10/d51/f80 [0,4194304] 0 2024-09-10T04:05:16.897 INFO:tasks.workunit.client.0.smithi046.stdout:6/411: write da/db/d10/d51/f80 [368620,40868] 0 2024-09-10T04:05:16.897 INFO:tasks.workunit.client.0.smithi046.stdout:3/421: mknod d12/d5d/d71/c8a 0 2024-09-10T04:05:16.902 INFO:tasks.workunit.client.1.smithi070.stdout:1/369: rename d3/d10/d1d/c59 to d3/d8/d12/d2f/d32/d4b/d54/c76 0 2024-09-10T04:05:16.905 INFO:tasks.workunit.client.0.smithi046.stdout:6/412: dwrite da/db/d18/f49 [0,4194304] 0 2024-09-10T04:05:16.907 INFO:tasks.workunit.client.1.smithi070.stdout:9/492: dwrite d9/d59/f1a [0,4194304] 0 2024-09-10T04:05:16.908 INFO:tasks.workunit.client.1.smithi070.stdout:7/379: symlink d2/d7/d9/d18/d3c/d52/l8c 0 2024-09-10T04:05:16.913 INFO:tasks.workunit.client.1.smithi070.stdout:9/493: dwrite d9/d59/f12 [0,4194304] 0 2024-09-10T04:05:16.914 INFO:tasks.workunit.client.1.smithi070.stdout:4/379: mkdir de/d13/d18/d26/d46/d85 0 2024-09-10T04:05:16.916 INFO:tasks.workunit.client.1.smithi070.stdout:2/402: symlink d15/d18/d42/l8d 0 2024-09-10T04:05:16.918 INFO:tasks.workunit.client.1.smithi070.stdout:8/495: symlink d4/d16/d73/d96/la7 0 2024-09-10T04:05:16.918 INFO:tasks.workunit.client.1.smithi070.stdout:4/380: write de/d13/d18/d23/d47/d59/d79/f56 [1149115,74995] 0 2024-09-10T04:05:16.921 INFO:tasks.workunit.client.1.smithi070.stdout:6/506: getdents d0/d3/d1f/d55 0 2024-09-10T04:05:16.921 INFO:tasks.workunit.client.1.smithi070.stdout:8/496: chown d4/d16/d17/d19/d62/la6 781706 1 2024-09-10T04:05:16.922 INFO:tasks.workunit.client.1.smithi070.stdout:6/507: fsync d0/f57 0 2024-09-10T04:05:16.923 INFO:tasks.workunit.client.1.smithi070.stdout:5/437: creat d17/f8e x:0 0 0 2024-09-10T04:05:16.926 INFO:tasks.workunit.client.1.smithi070.stdout:0/477: symlink d8/d13/d33/d81/d78/l9d 0 2024-09-10T04:05:16.929 INFO:tasks.workunit.client.1.smithi070.stdout:7/380: mknod d2/d3/d63/c8d 0 2024-09-10T04:05:16.930 INFO:tasks.workunit.client.1.smithi070.stdout:9/494: mknod d9/d1c/d34/d63/c95 0 2024-09-10T04:05:16.933 INFO:tasks.workunit.client.1.smithi070.stdout:2/403: symlink d15/d59/l8e 0 2024-09-10T04:05:16.934 INFO:tasks.workunit.client.0.smithi046.stdout:4/459: creat d3/d12/d17/d22/f9d x:0 0 0 2024-09-10T04:05:16.935 INFO:tasks.workunit.client.1.smithi070.stdout:9/495: dwrite d9/d60/d8a/d3c/d62/f7c [0,4194304] 0 2024-09-10T04:05:16.936 INFO:tasks.workunit.client.1.smithi070.stdout:9/496: write d9/d60/d8a/d1b/f5c [3397932,1480] 0 2024-09-10T04:05:16.937 INFO:tasks.workunit.client.1.smithi070.stdout:9/497: stat d9 0 2024-09-10T04:05:16.941 INFO:tasks.workunit.client.0.smithi046.stdout:4/460: dwrite d3/d12/d17/f1b [0,4194304] 0 2024-09-10T04:05:16.945 INFO:tasks.workunit.client.1.smithi070.stdout:5/438: fsync f12 0 2024-09-10T04:05:16.953 INFO:tasks.workunit.client.1.smithi070.stdout:0/478: mknod d8/d24/d2d/d34/d6b/c9e 0 2024-09-10T04:05:16.956 INFO:tasks.workunit.client.0.smithi046.stdout:8/390: rename d1/d29/d19/d36/f74 to d1/f86 0 2024-09-10T04:05:16.957 INFO:tasks.workunit.client.0.smithi046.stdout:8/391: dread - d1/d29/d19/d36/d4c/f7d zero size 2024-09-10T04:05:16.958 INFO:tasks.workunit.client.1.smithi070.stdout:7/381: creat d2/d3/dc/d5d/d2d/f8e x:0 0 0 2024-09-10T04:05:16.961 INFO:tasks.workunit.client.0.smithi046.stdout:6/413: mkdir da/db/d11/d13/d1e/d23/d86 0 2024-09-10T04:05:16.969 INFO:tasks.workunit.client.0.smithi046.stdout:3/422: rename d12/d5d/l7c to d12/d13/d14/d1e/d2d/d39/d67/l8b 0 2024-09-10T04:05:16.969 INFO:tasks.workunit.client.1.smithi070.stdout:4/381: write de/d13/d18/d5c/d7f/f84 [826024,5077] 0 2024-09-10T04:05:16.969 INFO:tasks.workunit.client.1.smithi070.stdout:9/498: fsync d9/d60/f84 0 2024-09-10T04:05:16.970 INFO:tasks.workunit.client.0.smithi046.stdout:8/392: dread - d1/d29/d17/d1f/f54 zero size 2024-09-10T04:05:16.970 INFO:tasks.workunit.client.1.smithi070.stdout:4/382: dread - de/d13/d5b/f80 zero size 2024-09-10T04:05:16.971 INFO:tasks.workunit.client.0.smithi046.stdout:3/423: write d12/d13/d14/d1e/d2d/f80 [893387,114585] 0 2024-09-10T04:05:16.972 INFO:tasks.workunit.client.1.smithi070.stdout:4/383: write de/d13/d18/d23/f61 [999884,109538] 0 2024-09-10T04:05:16.973 INFO:tasks.workunit.client.1.smithi070.stdout:5/439: mknod d17/d24/d4e/c8f 0 2024-09-10T04:05:16.975 INFO:tasks.workunit.client.1.smithi070.stdout:6/508: dwrite d0/d3/d18/d1b/d17/d3f/d40/f48 [0,4194304] 0 2024-09-10T04:05:16.975 INFO:tasks.workunit.client.0.smithi046.stdout:6/414: rename da/db/d11/d13/d1e/d69/f84 to da/db/d11/d13/d1e/d69/f87 0 2024-09-10T04:05:16.977 INFO:tasks.workunit.client.1.smithi070.stdout:1/370: link d3/d8/d13/d3f/l75 d3/d20/l77 0 2024-09-10T04:05:16.977 INFO:tasks.workunit.client.0.smithi046.stdout:4/461: link d3/l7 d3/d12/d17/d29/l9e 0 2024-09-10T04:05:16.980 INFO:tasks.workunit.client.1.smithi070.stdout:7/382: symlink d2/d7/db/d20/l8f 0 2024-09-10T04:05:16.982 INFO:tasks.workunit.client.1.smithi070.stdout:6/509: dwrite d0/d3/d18/d1b/d17/d3f/d40/f48 [4194304,4194304] 0 2024-09-10T04:05:16.983 INFO:tasks.workunit.client.0.smithi046.stdout:5/512: dwrite d7/d11/d14/d45/d51/fbb [0,4194304] 0 2024-09-10T04:05:16.983 INFO:tasks.workunit.client.1.smithi070.stdout:6/510: chown d0/d3/d18/f35 55496508 1 2024-09-10T04:05:16.985 INFO:tasks.workunit.client.0.smithi046.stdout:3/424: mknod d12/d1b/d50/d52/d86/c8c 0 2024-09-10T04:05:16.987 INFO:tasks.workunit.client.1.smithi070.stdout:9/499: unlink d9/d60/d8a/d3c/d62/f7c 0 2024-09-10T04:05:16.987 INFO:tasks.workunit.client.1.smithi070.stdout:9/500: readlink l4 0 2024-09-10T04:05:16.989 INFO:tasks.workunit.client.0.smithi046.stdout:0/327: sync 2024-09-10T04:05:16.990 INFO:tasks.workunit.client.0.smithi046.stdout:6/415: dwrite da/db/d11/d13/d1e/f82 [0,4194304] 0 2024-09-10T04:05:16.993 INFO:tasks.workunit.client.1.smithi070.stdout:3/444: sync 2024-09-10T04:05:16.993 INFO:tasks.workunit.client.0.smithi046.stdout:8/393: symlink d1/d29/d17/d1f/d46/d76/l87 0 2024-09-10T04:05:16.994 INFO:tasks.workunit.client.1.smithi070.stdout:4/384: symlink de/d13/d18/d26/l86 0 2024-09-10T04:05:16.998 INFO:tasks.workunit.client.0.smithi046.stdout:6/416: fdatasync da/db/d11/d13/d1e/f3e 0 2024-09-10T04:05:16.998 INFO:tasks.workunit.client.0.smithi046.stdout:5/513: unlink d7/d11/d3f/d53/da9/d9c/ca7 0 2024-09-10T04:05:16.999 INFO:tasks.workunit.client.1.smithi070.stdout:3/445: dread db/d1c/d3e/f50 [4194304,4194304] 0 2024-09-10T04:05:17.000 INFO:tasks.workunit.client.0.smithi046.stdout:3/425: truncate d12/f5f 4228688 0 2024-09-10T04:05:17.001 INFO:tasks.workunit.client.0.smithi046.stdout:6/417: stat da/db/d11/d2b/f2c 0 2024-09-10T04:05:17.002 INFO:tasks.workunit.client.1.smithi070.stdout:4/385: dwrite de/d13/d18/d26/f42 [0,4194304] 0 2024-09-10T04:05:17.004 INFO:tasks.workunit.client.0.smithi046.stdout:4/462: dwrite d3/d12/d17/d22/f59 [0,4194304] 0 2024-09-10T04:05:17.004 INFO:tasks.workunit.client.0.smithi046.stdout:3/426: chown d12/d13/d37/d51/f72 7925 1 2024-09-10T04:05:17.006 INFO:tasks.workunit.client.1.smithi070.stdout:3/446: dwrite db/d1c/f4b [0,4194304] 0 2024-09-10T04:05:17.013 INFO:tasks.workunit.client.0.smithi046.stdout:0/328: symlink d1/dc/d25/d33/d3/dd/l69 0 2024-09-10T04:05:17.016 INFO:tasks.workunit.client.0.smithi046.stdout:3/427: dwrite d12/d13/d37/d51/f62 [0,4194304] 0 2024-09-10T04:05:17.017 INFO:tasks.workunit.client.0.smithi046.stdout:3/428: chown d12/d13 0 1 2024-09-10T04:05:17.019 INFO:tasks.workunit.client.0.smithi046.stdout:8/394: dread d1/d29/d17/f2f [0,4194304] 0 2024-09-10T04:05:17.021 INFO:tasks.workunit.client.0.smithi046.stdout:5/514: fdatasync d7/f10 0 2024-09-10T04:05:17.024 INFO:tasks.workunit.client.0.smithi046.stdout:8/395: fsync d1/d29/d17/d1f/d46/d52/f65 0 2024-09-10T04:05:17.026 INFO:tasks.workunit.client.1.smithi070.stdout:2/404: link d15/d18/d56/c6b d15/d59/c8f 0 2024-09-10T04:05:17.029 INFO:tasks.workunit.client.0.smithi046.stdout:3/429: dwrite d12/d1b/d50/d52/f65 [0,4194304] 0 2024-09-10T04:05:17.029 INFO:tasks.workunit.client.0.smithi046.stdout:6/418: creat da/db/d10/d27/f88 x:0 0 0 2024-09-10T04:05:17.035 INFO:tasks.workunit.client.0.smithi046.stdout:4/463: rename d3/d12/d14/c69 to d3/d43/d5f/d85/c9f 0 2024-09-10T04:05:17.036 INFO:tasks.workunit.client.1.smithi070.stdout:6/511: mknod d0/d3/d18/d1b/ca1 0 2024-09-10T04:05:17.039 INFO:tasks.workunit.client.1.smithi070.stdout:4/386: creat de/d13/d18/d23/d47/d59/f87 x:0 0 0 2024-09-10T04:05:17.039 INFO:tasks.workunit.client.1.smithi070.stdout:4/387: readlink de/d13/d18/d26/l86 0 2024-09-10T04:05:17.041 INFO:tasks.workunit.client.1.smithi070.stdout:5/440: truncate d17/d24/f2f 869411 0 2024-09-10T04:05:17.042 INFO:tasks.workunit.client.0.smithi046.stdout:4/464: dwrite d3/d12/d17/d22/f4e [0,4194304] 0 2024-09-10T04:05:17.043 INFO:tasks.workunit.client.0.smithi046.stdout:4/465: chown d3/d12/d17/d22/d51/c6a 1677462 1 2024-09-10T04:05:17.050 INFO:tasks.workunit.client.1.smithi070.stdout:4/388: mkdir de/d13/d43/d88 0 2024-09-10T04:05:17.051 INFO:tasks.workunit.client.0.smithi046.stdout:3/430: creat d12/d5d/d71/d49/f8d x:0 0 0 2024-09-10T04:05:17.052 INFO:tasks.workunit.client.0.smithi046.stdout:4/466: dwrite d3/d12/d17/d29/f38 [0,4194304] 0 2024-09-10T04:05:17.057 INFO:tasks.workunit.client.0.smithi046.stdout:0/329: rename d1/dc/d25/d33/d3/f2a to d1/d13/d37/f6a 0 2024-09-10T04:05:17.058 INFO:tasks.workunit.client.1.smithi070.stdout:5/441: rmdir d17/d24/d4e 39 2024-09-10T04:05:17.062 INFO:tasks.workunit.client.1.smithi070.stdout:7/383: rename d2/d7/d9/d18/l6f to d2/d7/d9/l90 0 2024-09-10T04:05:17.063 INFO:tasks.workunit.client.1.smithi070.stdout:7/384: read - d2/d3/dc/d5d/d2d/f8e zero size 2024-09-10T04:05:17.065 INFO:tasks.workunit.client.1.smithi070.stdout:2/405: write d15/d2b/d6e/f89 [237630,124061] 0 2024-09-10T04:05:17.067 INFO:tasks.workunit.client.1.smithi070.stdout:2/406: write d15/d18/d29/d2a/d2e/d32/f57 [647029,109866] 0 2024-09-10T04:05:17.067 INFO:tasks.workunit.client.1.smithi070.stdout:4/389: mknod de/d13/d18/d26/c89 0 2024-09-10T04:05:17.068 INFO:tasks.workunit.client.1.smithi070.stdout:4/390: stat de/d13/d5b/c82 0 2024-09-10T04:05:17.069 INFO:tasks.workunit.client.0.smithi046.stdout:5/515: link d7/d11/d14/d2f/d2b/fad d7/d11/d14/d2f/da3/fc5 0 2024-09-10T04:05:17.070 INFO:tasks.workunit.client.0.smithi046.stdout:5/516: dread - d7/d11/d3f/d5a/d63/f98 zero size 2024-09-10T04:05:17.070 INFO:tasks.workunit.client.1.smithi070.stdout:3/447: link db/d1c/c24 db/d1c/d89/c91 0 2024-09-10T04:05:17.075 INFO:tasks.workunit.client.1.smithi070.stdout:9/501: rename d9/d59/d2a/d3f/l83 to d9/d7a/d74/l96 0 2024-09-10T04:05:17.076 INFO:tasks.workunit.client.0.smithi046.stdout:0/330: symlink d1/dc/d25/d33/d3/dd/l6b 0 2024-09-10T04:05:17.078 INFO:tasks.workunit.client.1.smithi070.stdout:9/502: dread d9/d59/f7e [0,4194304] 0 2024-09-10T04:05:17.079 INFO:tasks.workunit.client.0.smithi046.stdout:8/396: write d1/d29/d19/d36/d4c/f5b [503136,122140] 0 2024-09-10T04:05:17.079 INFO:tasks.workunit.client.0.smithi046.stdout:0/331: dread d1/d13/d37/f6a [0,4194304] 0 2024-09-10T04:05:17.080 INFO:tasks.workunit.client.1.smithi070.stdout:6/512: creat d0/d3/d18/fa2 x:0 0 0 2024-09-10T04:05:17.081 INFO:tasks.workunit.client.0.smithi046.stdout:8/397: write d1/d29/d19/f83 [717587,33904] 0 2024-09-10T04:05:17.081 INFO:tasks.workunit.client.1.smithi070.stdout:6/513: write d0/d3/d18/d1b/d17/f90 [505143,113989] 0 2024-09-10T04:05:17.081 INFO:tasks.workunit.client.0.smithi046.stdout:0/332: readlink d1/dc/d25/d33/d3/d29/l59 0 2024-09-10T04:05:17.082 INFO:tasks.workunit.client.0.smithi046.stdout:3/431: link d12/d5d/d71/d49/f4e d12/d13/d37/d51/f8e 0 2024-09-10T04:05:17.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.081+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f73f0062630 con 0x7f73d406dfa0 2024-09-10T04:05:17.084 INFO:tasks.workunit.client.0.smithi046.stdout:3/432: fsync d12/d13/d14/d1e/d2d/f88 0 2024-09-10T04:05:17.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.083+0000 7f73e7fff640 1 -- 172.21.15.46:0/1206283186 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f73f0062630 con 0x7f73d406dfa0 2024-09-10T04:05:17.085 INFO:tasks.workunit.client.1.smithi070.stdout:8/497: sync 2024-09-10T04:05:17.085 INFO:tasks.workunit.client.0.smithi046.stdout:8/398: readlink d1/d29/d19/d36/d4c/l68 0 2024-09-10T04:05:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.085+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f73d406dfa0 msgr2=0x7f73d4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.085+0000 7f73f527f640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f73d406dfa0 0x7f73d4070460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f73f0111a80 tx=0x7f73dc00b410 comp rx=0 tx=0).stop 2024-09-10T04:05:17.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.085+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 msgr2=0x7f73f01105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.085+0000 7f73f527f640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 0x7f73f01105d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f73e0006010 tx=0x7f73e00045e0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.086+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 shutdown_connections 2024-09-10T04:05:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.086+0000 7f73f527f640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f73d406dfa0 0x7f73d4070460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.086+0000 7f73f527f640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73f0102b60 0x7f73f0110b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.086+0000 7f73f527f640 1 --2- 172.21.15.46:0/1206283186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73f0101960 0x7f73f01105d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.086+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 >> 172.21.15.46:0/1206283186 conn(0x7f73f00fd0f0 msgr2=0x7f73f0106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:17.088 INFO:tasks.workunit.client.0.smithi046.stdout:8/399: write d1/d29/d17/d1f/d2a/f58 [5434140,8945] 0 2024-09-10T04:05:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.086+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 shutdown_connections 2024-09-10T04:05:17.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.086+0000 7f73f527f640 1 -- 172.21.15.46:0/1206283186 wait complete. 2024-09-10T04:05:17.091 INFO:tasks.workunit.client.0.smithi046.stdout:0/333: read d1/dc/f27 [200479,100186] 0 2024-09-10T04:05:17.091 INFO:tasks.workunit.client.1.smithi070.stdout:3/448: fdatasync db/f4e 0 2024-09-10T04:05:17.092 INFO:tasks.workunit.client.1.smithi070.stdout:5/442: dwrite d17/d19/f2e [0,4194304] 0 2024-09-10T04:05:17.092 INFO:tasks.workunit.client.0.smithi046.stdout:5/517: link d7/d11/d14/d45/d51/f91 d7/d11/d14/d2f/d64/fc6 0 2024-09-10T04:05:17.093 INFO:tasks.workunit.client.0.smithi046.stdout:0/334: write d1/dc/d25/d33/d4/f36 [48055,117190] 0 2024-09-10T04:05:17.095 INFO:tasks.workunit.client.0.smithi046.stdout:3/433: mkdir d12/d13/d14/d1e/d2d/d6f/d8f 0 2024-09-10T04:05:17.096 INFO:tasks.workunit.client.1.smithi070.stdout:8/498: dwrite d4/d16/d17/d5c/d87/f9f [0,4194304] 0 2024-09-10T04:05:17.098 INFO:tasks.workunit.client.1.smithi070.stdout:8/499: stat d4/d16/d17/d19/d28/d43/d52/ca5 0 2024-09-10T04:05:17.098 INFO:tasks.workunit.client.1.smithi070.stdout:7/385: dwrite d2/d7/fe [0,4194304] 0 2024-09-10T04:05:17.099 INFO:tasks.workunit.client.1.smithi070.stdout:8/500: stat d4/f29 0 2024-09-10T04:05:17.102 INFO:tasks.workunit.client.0.smithi046.stdout:0/335: dread d1/dc/f17 [0,4194304] 0 2024-09-10T04:05:17.105 INFO:tasks.workunit.client.0.smithi046.stdout:8/400: unlink d1/d29/d19/f48 0 2024-09-10T04:05:17.105 INFO:tasks.workunit.client.1.smithi070.stdout:4/391: write fc [220965,22614] 0 2024-09-10T04:05:17.107 INFO:tasks.workunit.client.1.smithi070.stdout:9/503: read d9/d7a/d72/f7b [336785,31255] 0 2024-09-10T04:05:17.110 INFO:tasks.workunit.client.1.smithi070.stdout:6/514: fdatasync d0/d3/d18/d1b/d17/d53/f61 0 2024-09-10T04:05:17.111 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:05:17.115 INFO:tasks.workunit.client.1.smithi070.stdout:2/407: mkdir d15/d18/d90 0 2024-09-10T04:05:17.121 INFO:tasks.workunit.client.1.smithi070.stdout:3/449: mkdir db/d83/d92 0 2024-09-10T04:05:17.121 INFO:tasks.workunit.client.1.smithi070.stdout:5/443: mkdir d17/d6f/d32/d90 0 2024-09-10T04:05:17.124 INFO:tasks.workunit.client.1.smithi070.stdout:8/501: mkdir d4/d16/d17/d19/d28/da8 0 2024-09-10T04:05:17.127 INFO:tasks.workunit.client.0.smithi046.stdout:3/434: link d12/d5d/d71/d49/c7d d12/d13/d37/c90 0 2024-09-10T04:05:17.131 INFO:tasks.workunit.client.1.smithi070.stdout:9/504: creat d9/d60/d8a/d3c/d62/f97 x:0 0 0 2024-09-10T04:05:17.131 INFO:tasks.workunit.client.0.smithi046.stdout:0/336: creat d1/f6c x:0 0 0 2024-09-10T04:05:17.132 INFO:tasks.workunit.client.0.smithi046.stdout:3/435: truncate d12/d13/d14/d1e/d2d/d39/d67/f6b 258646 0 2024-09-10T04:05:17.133 INFO:tasks.workunit.client.0.smithi046.stdout:5/518: dwrite d7/f43 [0,4194304] 0 2024-09-10T04:05:17.133 INFO:tasks.workunit.client.0.smithi046.stdout:8/401: rmdir d1/d29/d19/d36/d62 0 2024-09-10T04:05:17.135 INFO:tasks.workunit.client.1.smithi070.stdout:6/515: truncate d0/d3/f16 5067824 0 2024-09-10T04:05:17.135 INFO:tasks.workunit.client.0.smithi046.stdout:3/436: write d12/d1b/d50/d52/f65 [4500574,105001] 0 2024-09-10T04:05:17.136 INFO:tasks.workunit.client.1.smithi070.stdout:2/408: creat d15/d18/d29/d2a/d2e/d32/f91 x:0 0 0 2024-09-10T04:05:17.136 INFO:tasks.workunit.client.0.smithi046.stdout:5/519: dread - d7/d11/d3f/d53/d75/f9f zero size 2024-09-10T04:05:17.139 INFO:tasks.workunit.client.1.smithi070.stdout:3/450: symlink db/d79/l93 0 2024-09-10T04:05:17.140 INFO:tasks.workunit.client.0.smithi046.stdout:3/437: write d12/d13/f2c [672576,24437] 0 2024-09-10T04:05:17.142 INFO:tasks.workunit.client.1.smithi070.stdout:0/479: sync 2024-09-10T04:05:17.143 INFO:tasks.workunit.client.0.smithi046.stdout:0/337: creat d1/dc/d25/d33/d3/dd/d55/f6d x:0 0 0 2024-09-10T04:05:17.144 INFO:tasks.workunit.client.1.smithi070.stdout:5/444: rename d17/d19/d6a/d54/d57/f7d to d17/d37/f91 0 2024-09-10T04:05:17.145 INFO:tasks.workunit.client.1.smithi070.stdout:6/516: dread d0/d3/d18/d2a/f4f [0,4194304] 0 2024-09-10T04:05:17.147 INFO:tasks.workunit.client.0.smithi046.stdout:5/520: dwrite d7/d11/d3f/d53/f9b [0,4194304] 0 2024-09-10T04:05:17.147 INFO:tasks.workunit.client.1.smithi070.stdout:6/517: truncate d0/d3/d18/d62/f85 558159 0 2024-09-10T04:05:17.147 INFO:tasks.workunit.client.1.smithi070.stdout:7/386: unlink d2/d3/d63/c84 0 2024-09-10T04:05:17.149 INFO:tasks.workunit.client.1.smithi070.stdout:6/518: write d0/d3/d18/d1b/d17/d54/f87 [372109,111491] 0 2024-09-10T04:05:17.149 INFO:tasks.workunit.client.1.smithi070.stdout:8/502: rmdir d4/d16/d17/d19/d28/d43/d67/d97 39 2024-09-10T04:05:17.153 INFO:tasks.workunit.client.1.smithi070.stdout:6/519: truncate d0/d3/d18/d1b/d17/d28/f95 494957 0 2024-09-10T04:05:17.154 INFO:tasks.workunit.client.1.smithi070.stdout:6/520: chown d0/d3/d18/d62/f68 1371 1 2024-09-10T04:05:17.155 INFO:tasks.workunit.client.1.smithi070.stdout:6/521: chown d0/d3/d1f/f22 113 1 2024-09-10T04:05:17.156 INFO:tasks.workunit.client.0.smithi046.stdout:3/438: mkdir d12/d1b/d4b/d91 0 2024-09-10T04:05:17.160 INFO:tasks.workunit.client.0.smithi046.stdout:0/338: unlink d1/dc/d25/d33/d3/d29/f40 0 2024-09-10T04:05:17.162 INFO:tasks.workunit.client.0.smithi046.stdout:8/402: write d1/d29/d19/d36/f3f [2899122,8594] 0 2024-09-10T04:05:17.166 INFO:tasks.workunit.client.1.smithi070.stdout:7/387: dread d2/d3/dc/d5d/f34 [0,4194304] 0 2024-09-10T04:05:17.167 INFO:tasks.workunit.client.0.smithi046.stdout:3/439: rmdir d12/d5d/d71 39 2024-09-10T04:05:17.168 INFO:tasks.workunit.client.0.smithi046.stdout:3/440: stat d12/d13/d37/d51/f62 0 2024-09-10T04:05:17.173 INFO:tasks.workunit.client.1.smithi070.stdout:6/522: dread d0/d3/d18/d62/f68 [0,4194304] 0 2024-09-10T04:05:17.179 INFO:tasks.workunit.client.0.smithi046.stdout:3/441: dwrite d12/d13/d14/d1e/d2d/f88 [0,4194304] 0 2024-09-10T04:05:17.180 INFO:tasks.workunit.client.0.smithi046.stdout:8/403: creat d1/d29/d19/d73/f88 x:0 0 0 2024-09-10T04:05:17.181 INFO:tasks.workunit.client.1.smithi070.stdout:9/505: symlink d9/d1c/d34/d8b/l98 0 2024-09-10T04:05:17.182 INFO:tasks.workunit.client.0.smithi046.stdout:0/339: dwrite d1/dc/f17 [0,4194304] 0 2024-09-10T04:05:17.183 INFO:tasks.workunit.client.1.smithi070.stdout:6/523: dread d0/d3/d18/d1b/d17/d53/f61 [0,4194304] 0 2024-09-10T04:05:17.184 INFO:tasks.workunit.client.0.smithi046.stdout:0/340: chown d1/dc/d25/d33/d4/d19/l44 7159 1 2024-09-10T04:05:17.185 INFO:tasks.workunit.client.0.smithi046.stdout:5/521: link d7/f10 d7/d11/d3f/d53/fc7 0 2024-09-10T04:05:17.189 INFO:tasks.workunit.client.1.smithi070.stdout:0/480: symlink d8/d13/d33/d4f/l9f 0 2024-09-10T04:05:17.191 INFO:tasks.workunit.client.0.smithi046.stdout:0/341: read d1/dc/f21 [3932586,96521] 0 2024-09-10T04:05:17.192 INFO:tasks.workunit.client.0.smithi046.stdout:0/342: readlink d1/dc/d25/d33/d3/l30 0 2024-09-10T04:05:17.194 INFO:tasks.workunit.client.0.smithi046.stdout:0/343: write d1/dc/d25/d33/d4/f60 [69683,4603] 0 2024-09-10T04:05:17.195 INFO:tasks.workunit.client.0.smithi046.stdout:5/522: fsync d7/d11/d14/d2f/da3/fc5 0 2024-09-10T04:05:17.196 INFO:tasks.workunit.client.1.smithi070.stdout:1/371: sync 2024-09-10T04:05:17.200 INFO:tasks.workunit.client.0.smithi046.stdout:8/404: mknod d1/d29/d19/d36/c89 0 2024-09-10T04:05:17.201 INFO:tasks.workunit.client.1.smithi070.stdout:1/372: dwrite d3/f60 [0,4194304] 0 2024-09-10T04:05:17.201 INFO:tasks.workunit.client.1.smithi070.stdout:1/373: stat d3/d10/d1d/l3a 0 2024-09-10T04:05:17.204 INFO:tasks.workunit.client.1.smithi070.stdout:7/388: creat d2/d7/d9/d18/d23/f91 x:0 0 0 2024-09-10T04:05:17.206 INFO:tasks.workunit.client.1.smithi070.stdout:7/389: chown d2/d40/l59 26 1 2024-09-10T04:05:17.208 INFO:tasks.workunit.client.0.smithi046.stdout:0/344: creat d1/dc/d25/d33/d1a/d41/f6e x:0 0 0 2024-09-10T04:05:17.208 INFO:tasks.workunit.client.1.smithi070.stdout:7/390: write d2/d7/d9/d18/d3c/f58 [186665,113019] 0 2024-09-10T04:05:17.209 INFO:tasks.workunit.client.1.smithi070.stdout:1/374: dwrite d3/f60 [4194304,4194304] 0 2024-09-10T04:05:17.209 INFO:tasks.workunit.client.0.smithi046.stdout:0/345: readlink d1/dc/d25/d33/d3/d29/l59 0 2024-09-10T04:05:17.210 INFO:tasks.workunit.client.0.smithi046.stdout:0/346: chown d1/dc/d49/f4a 71036 1 2024-09-10T04:05:17.212 INFO:tasks.workunit.client.1.smithi070.stdout:6/524: creat d0/d3/d18/d1b/d17/d53/fa3 x:0 0 0 2024-09-10T04:05:17.217 INFO:tasks.workunit.client.1.smithi070.stdout:3/451: dwrite db/f4e [0,4194304] 0 2024-09-10T04:05:17.220 INFO:tasks.workunit.client.1.smithi070.stdout:6/525: dwrite d0/d3/f98 [0,4194304] 0 2024-09-10T04:05:17.220 INFO:tasks.workunit.client.1.smithi070.stdout:5/445: mknod d17/d24/c92 0 2024-09-10T04:05:17.221 INFO:tasks.workunit.client.0.smithi046.stdout:8/405: mknod d1/d29/d17/d1f/d46/d6a/d82/c8a 0 2024-09-10T04:05:17.224 INFO:tasks.workunit.client.0.smithi046.stdout:8/406: chown d1/d29/d17/d1f/d2a/f2d 350097 1 2024-09-10T04:05:17.224 INFO:tasks.workunit.client.0.smithi046.stdout:3/442: dwrite f3 [0,4194304] 0 2024-09-10T04:05:17.228 INFO:tasks.workunit.client.0.smithi046.stdout:3/443: write d12/d13/d37/d51/f57 [5382128,46839] 0 2024-09-10T04:05:17.228 INFO:tasks.workunit.client.0.smithi046.stdout:8/407: stat d1/d29/d17/d1f/d46/d52 0 2024-09-10T04:05:17.231 INFO:tasks.workunit.client.0.smithi046.stdout:5/523: write d7/d11/d14/d2f/d2b/f3a [638468,73235] 0 2024-09-10T04:05:17.238 INFO:tasks.workunit.client.1.smithi070.stdout:0/481: mkdir d8/d13/d1a/da0 0 2024-09-10T04:05:17.239 INFO:tasks.workunit.client.0.smithi046.stdout:0/347: dread d1/dc/d25/d33/d3/d29/d2d/f32 [0,4194304] 0 2024-09-10T04:05:17.240 INFO:tasks.workunit.client.0.smithi046.stdout:3/444: dwrite d12/d13/d37/f81 [0,4194304] 0 2024-09-10T04:05:17.241 INFO:tasks.workunit.client.1.smithi070.stdout:2/409: rename d15/d18/d29/d2a/d2e/d32/f91 to d15/d18/d56/d7c/f92 0 2024-09-10T04:05:17.245 INFO:tasks.workunit.client.0.smithi046.stdout:8/408: fdatasync d1/d29/d19/d36/f3c 0 2024-09-10T04:05:17.246 INFO:tasks.workunit.client.1.smithi070.stdout:3/452: mkdir db/d79/d94 0 2024-09-10T04:05:17.247 INFO:tasks.workunit.client.0.smithi046.stdout:1/328: sync 2024-09-10T04:05:17.247 INFO:tasks.workunit.client.0.smithi046.stdout:2/443: sync 2024-09-10T04:05:17.249 INFO:tasks.workunit.client.1.smithi070.stdout:5/446: mkdir d17/d37/d93 0 2024-09-10T04:05:17.249 INFO:tasks.workunit.client.0.smithi046.stdout:2/444: dread d2/d6/d23/f87 [0,4194304] 0 2024-09-10T04:05:17.251 INFO:tasks.workunit.client.0.smithi046.stdout:0/348: creat d1/dc/d25/d33/d4/f6f x:0 0 0 2024-09-10T04:05:17.251 INFO:tasks.workunit.client.0.smithi046.stdout:2/445: chown d2/d6/d23/d55 283898217 1 2024-09-10T04:05:17.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.252+0000 7f2ba259c640 1 -- 172.21.15.46:0/2648584417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2b94031740 con 0x7f2ba4072fe0 2024-09-10T04:05:17.254 INFO:tasks.workunit.client.1.smithi070.stdout:5/447: dwrite d17/d19/d6a/f7c [0,4194304] 0 2024-09-10T04:05:17.254 INFO:tasks.workunit.client.0.smithi046.stdout:2/446: read d2/d6/d23/d5d/f8f [1539025,125765] 0 2024-09-10T04:05:17.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.252+0000 7f2ba8a03640 1 -- 172.21.15.46:0/2648584417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 msgr2=0x7f2ba40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.252+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/2648584417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba40733e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009880 tx=0x7f2b9402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.253+0000 7f2ba8a03640 1 -- 172.21.15.46:0/2648584417 shutdown_connections 2024-09-10T04:05:17.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.253+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/2648584417 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba40739b0 0x7f2ba410d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.253+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/2648584417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba40733e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.253+0000 7f2ba8a03640 1 -- 172.21.15.46:0/2648584417 >> 172.21.15.46:0/2648584417 conn(0x7f2ba406e8b0 msgr2=0x7f2ba4070cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:17.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.253+0000 7f2ba8a03640 1 -- 172.21.15.46:0/2648584417 shutdown_connections 2024-09-10T04:05:17.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.253+0000 7f2ba8a03640 1 -- 172.21.15.46:0/2648584417 wait complete. 2024-09-10T04:05:17.256 INFO:tasks.workunit.client.0.smithi046.stdout:3/445: mkdir d12/d13/d14/d1e/d2d/d39/d67/d92 0 2024-09-10T04:05:17.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.254+0000 7f2ba8a03640 1 Processor -- start 2024-09-10T04:05:17.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.254+0000 7f2ba8a03640 1 -- start start 2024-09-10T04:05:17.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.255+0000 7f2ba8a03640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba41a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:17.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.255+0000 7f2ba8a03640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba40739b0 0x7f2ba41a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:17.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.255+0000 7f2ba8a03640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41193c0 con 0x7f2ba4072fe0 2024-09-10T04:05:17.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.255+0000 7f2ba8a03640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4119240 con 0x7f2ba40739b0 2024-09-10T04:05:17.257 INFO:tasks.workunit.client.1.smithi070.stdout:8/503: creat d4/d16/d17/d1a/fa9 x:0 0 0 2024-09-10T04:05:17.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.255+0000 7f2ba2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba41a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:17.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.255+0000 7f2ba2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba41a60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39588/0 (socket says 172.21.15.46:39588) 2024-09-10T04:05:17.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.255+0000 7f2ba2d9d640 1 -- 172.21.15.46:0/3127532265 learned_addr learned my addr 172.21.15.46:0/3127532265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.256+0000 7f2b9bfff640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba40739b0 0x7f2ba41a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.256+0000 7f2ba2d9d640 1 -- 172.21.15.46:0/3127532265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba40739b0 msgr2=0x7f2ba41a65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.256+0000 7f2ba2d9d640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba40739b0 0x7f2ba41a65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.256+0000 7f2ba2d9d640 1 -- 172.21.15.46:0/3127532265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba41a6c60 con 0x7f2ba4072fe0 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.256+0000 7f2ba2d9d640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba41a60a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009230 tx=0x7f2b94004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.257+0000 7f2ba0d99640 1 -- 172.21.15.46:0/3127532265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2b9403f070 con 0x7f2ba4072fe0 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.257+0000 7f2ba0d99640 1 -- 172.21.15.46:0/3127532265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2b94004a90 con 0x7f2ba4072fe0 2024-09-10T04:05:17.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.257+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba41ab640 con 0x7f2ba4072fe0 2024-09-10T04:05:17.262 INFO:tasks.workunit.client.0.smithi046.stdout:0/349: dwrite d1/dc/d49/f4a [0,4194304] 0 2024-09-10T04:05:17.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.257+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba41abbd0 con 0x7f2ba4072fe0 2024-09-10T04:05:17.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.257+0000 7f2ba0d99640 1 -- 172.21.15.46:0/3127532265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2b94035030 con 0x7f2ba4072fe0 2024-09-10T04:05:17.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.260+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b840052c0 con 0x7f2ba4072fe0 2024-09-10T04:05:17.265 INFO:tasks.workunit.client.0.smithi046.stdout:0/350: fsync d1/dc/d25/d33/d3/f46 0 2024-09-10T04:05:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.264+0000 7f2ba0d99640 1 -- 172.21.15.46:0/3127532265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f2b94042070 con 0x7f2ba4072fe0 2024-09-10T04:05:17.266 INFO:tasks.workunit.client.1.smithi070.stdout:2/410: unlink d15/d18/d56/d7c/d55/l5a 0 2024-09-10T04:05:17.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.265+0000 7f2ba0d99640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2b9006ded0 0x7f2b90070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:17.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.266+0000 7f2ba0d99640 1 -- 172.21.15.46:0/3127532265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f2b940aeaf0 con 0x7f2ba4072fe0 2024-09-10T04:05:17.268 INFO:tasks.workunit.client.0.smithi046.stdout:0/351: truncate d1/dc/d25/d33/d1a/d41/f6e 587978 0 2024-09-10T04:05:17.270 INFO:tasks.workunit.client.1.smithi070.stdout:6/526: mkdir d0/d3/da4 0 2024-09-10T04:05:17.271 INFO:tasks.workunit.client.0.smithi046.stdout:0/352: write d1/dc/d25/d33/d3/d29/d2d/f4f [918043,45606] 0 2024-09-10T04:05:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.270+0000 7f2ba0d99640 1 -- 172.21.15.46:0/3127532265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b9407c6f0 con 0x7f2ba4072fe0 2024-09-10T04:05:17.274 INFO:tasks.workunit.client.1.smithi070.stdout:9/506: getdents d9/d7a/d6f 0 2024-09-10T04:05:17.276 INFO:tasks.workunit.client.0.smithi046.stdout:5/524: creat d7/d11/d3f/fc8 x:0 0 0 2024-09-10T04:05:17.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.275+0000 7f2b9bfff640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2b9006ded0 0x7f2b90070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:17.277 INFO:tasks.workunit.client.1.smithi070.stdout:8/504: rename d4/d16/d17/d5c/d87/d8e/c80 to d4/d16/d17/d1a/d60/caa 0 2024-09-10T04:05:17.278 INFO:tasks.workunit.client.0.smithi046.stdout:8/409: dread d1/d29/d17/d1f/d2a/f58 [4194304,4194304] 0 2024-09-10T04:05:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.277+0000 7f2b9bfff640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2b9006ded0 0x7f2b90070390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ba41a7550 tx=0x7f2b8c005f90 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:17.279 INFO:tasks.workunit.client.1.smithi070.stdout:7/391: creat d2/d7/db/f92 x:0 0 0 2024-09-10T04:05:17.280 INFO:tasks.workunit.client.1.smithi070.stdout:7/392: read - d2/d66/f78 zero size 2024-09-10T04:05:17.280 INFO:tasks.workunit.client.1.smithi070.stdout:7/393: stat d2/d7/d9/d18/d3c 0 2024-09-10T04:05:17.281 INFO:tasks.workunit.client.0.smithi046.stdout:1/329: fdatasync d3/d27/d32/d1f/f31 0 2024-09-10T04:05:17.282 INFO:tasks.workunit.client.1.smithi070.stdout:8/505: dread d4/d16/d17/d1a/d60/f9c [0,4194304] 0 2024-09-10T04:05:17.283 INFO:tasks.workunit.client.0.smithi046.stdout:1/330: write d3/d27/d5d/f6a [1015143,21601] 0 2024-09-10T04:05:17.283 INFO:tasks.workunit.client.0.smithi046.stdout:1/331: truncate d3/ff 4899532 0 2024-09-10T04:05:17.284 INFO:tasks.workunit.client.0.smithi046.stdout:1/332: dread - d3/dc/d34/d58/f59 zero size 2024-09-10T04:05:17.285 INFO:tasks.workunit.client.1.smithi070.stdout:1/375: creat d3/d8/d13/d3f/f78 x:0 0 0 2024-09-10T04:05:17.287 INFO:tasks.workunit.client.1.smithi070.stdout:8/506: dwrite d4/f77 [0,4194304] 0 2024-09-10T04:05:17.288 INFO:tasks.workunit.client.1.smithi070.stdout:0/482: creat d8/d13/d33/d74/fa1 x:0 0 0 2024-09-10T04:05:17.291 INFO:tasks.workunit.client.1.smithi070.stdout:0/483: write d8/d13/d1a/f31 [379745,112539] 0 2024-09-10T04:05:17.292 INFO:tasks.workunit.client.1.smithi070.stdout:1/376: dwrite d3/d10/f72 [0,4194304] 0 2024-09-10T04:05:17.294 INFO:tasks.workunit.client.0.smithi046.stdout:2/447: write d2/d6/f1a [2080304,33991] 0 2024-09-10T04:05:17.297 INFO:tasks.workunit.client.1.smithi070.stdout:5/448: symlink d17/d19/d6a/d54/l94 0 2024-09-10T04:05:17.297 INFO:tasks.workunit.client.0.smithi046.stdout:0/353: symlink d1/dc/d25/d33/d4/d19/l70 0 2024-09-10T04:05:17.303 INFO:tasks.workunit.client.1.smithi070.stdout:7/394: creat d2/d7/db/d3a/d7e/f93 x:0 0 0 2024-09-10T04:05:17.304 INFO:tasks.workunit.client.1.smithi070.stdout:7/395: write d2/d7/d9/d18/d23/f91 [198672,28945] 0 2024-09-10T04:05:17.305 INFO:tasks.workunit.client.0.smithi046.stdout:8/410: creat d1/d29/d17/d1f/d46/f8b x:0 0 0 2024-09-10T04:05:17.306 INFO:tasks.workunit.client.1.smithi070.stdout:8/507: fdatasync d4/d16/d17/d19/d28/f30 0 2024-09-10T04:05:17.314 INFO:tasks.workunit.client.1.smithi070.stdout:0/484: creat d8/d13/d33/d81/d78/fa2 x:0 0 0 2024-09-10T04:05:17.316 INFO:tasks.workunit.client.0.smithi046.stdout:8/411: write d1/d29/d17/d1f/d2a/f85 [835606,29154] 0 2024-09-10T04:05:17.317 INFO:tasks.workunit.client.1.smithi070.stdout:1/377: unlink d3/d8/d13/f1b 0 2024-09-10T04:05:17.321 INFO:tasks.workunit.client.1.smithi070.stdout:9/507: mkdir d9/d60/d8a/d82/d99 0 2024-09-10T04:05:17.321 INFO:tasks.workunit.client.1.smithi070.stdout:6/527: write d0/d3/d18/d1b/fa0 [1409747,27754] 0 2024-09-10T04:05:17.322 INFO:tasks.workunit.client.1.smithi070.stdout:5/449: fsync d17/d24/d4e/f82 0 2024-09-10T04:05:17.323 INFO:tasks.workunit.client.0.smithi046.stdout:0/354: link d1/dc/d25/d33/d4/f5b d1/dc/d25/d33/d3/dd/d55/d4d/f71 0 2024-09-10T04:05:17.324 INFO:tasks.workunit.client.0.smithi046.stdout:5/525: write d7/fe [2491298,38516] 0 2024-09-10T04:05:17.325 INFO:tasks.workunit.client.0.smithi046.stdout:8/412: symlink d1/d29/d19/d36/d4c/d5e/l8c 0 2024-09-10T04:05:17.327 INFO:tasks.workunit.client.0.smithi046.stdout:8/413: readlink d1/d29/d17/d1f/d2a/l2b 0 2024-09-10T04:05:17.328 INFO:tasks.workunit.client.1.smithi070.stdout:7/396: rename d2/d7/d9/d18/d23/c35 to d2/d40/d6c/c94 0 2024-09-10T04:05:17.329 INFO:tasks.workunit.client.0.smithi046.stdout:3/446: truncate d12/d13/f17 500151 0 2024-09-10T04:05:17.337 INFO:tasks.workunit.client.0.smithi046.stdout:2/448: write d2/d6/d23/d42/f45 [881304,95712] 0 2024-09-10T04:05:17.339 INFO:tasks.workunit.client.0.smithi046.stdout:2/449: write d2/d6/f18 [2522735,68498] 0 2024-09-10T04:05:17.339 INFO:tasks.workunit.client.1.smithi070.stdout:4/392: sync 2024-09-10T04:05:17.340 INFO:tasks.workunit.client.1.smithi070.stdout:4/393: write de/d13/d18/f37 [113272,3523] 0 2024-09-10T04:05:17.343 INFO:tasks.workunit.client.1.smithi070.stdout:1/378: rmdir d3/d8/d12/d2f/d32/d4b 39 2024-09-10T04:05:17.347 INFO:tasks.workunit.client.0.smithi046.stdout:8/414: mkdir d1/d29/d19/d36/d8d 0 2024-09-10T04:05:17.348 INFO:tasks.workunit.client.0.smithi046.stdout:1/333: dwrite d3/d27/d32/d1f/f35 [0,4194304] 0 2024-09-10T04:05:17.355 INFO:tasks.workunit.client.1.smithi070.stdout:5/450: creat d17/d19/d6a/d54/d57/d63/d7b/f95 x:0 0 0 2024-09-10T04:05:17.356 INFO:tasks.workunit.client.0.smithi046.stdout:2/450: rename d2/d8/d65/l9e to d2/d6/d23/d5d/d77/dac/lad 0 2024-09-10T04:05:17.359 INFO:tasks.workunit.client.1.smithi070.stdout:0/485: write d8/d13/d33/f65 [799369,99643] 0 2024-09-10T04:05:17.360 INFO:tasks.workunit.client.0.smithi046.stdout:5/526: write d7/d11/d14/d2f/f8c [22651,48348] 0 2024-09-10T04:05:17.362 INFO:tasks.workunit.client.0.smithi046.stdout:5/527: stat d7/d11/d3f/d5a/fb2 0 2024-09-10T04:05:17.363 INFO:tasks.workunit.client.1.smithi070.stdout:8/508: mkdir d4/d16/d17/dab 0 2024-09-10T04:05:17.364 INFO:tasks.workunit.client.0.smithi046.stdout:0/355: dwrite d1/f35 [0,4194304] 0 2024-09-10T04:05:17.365 INFO:tasks.workunit.client.1.smithi070.stdout:0/486: dwrite d8/d24/f43 [0,4194304] 0 2024-09-10T04:05:17.366 INFO:tasks.workunit.client.1.smithi070.stdout:2/411: getdents d15/d18/d56/d7c/d55 0 2024-09-10T04:05:17.366 INFO:tasks.workunit.client.0.smithi046.stdout:5/528: dread - d7/d11/d3f/d53/d75/f9f zero size 2024-09-10T04:05:17.366 INFO:tasks.workunit.client.0.smithi046.stdout:8/415: chown d1/d29/d19/d36/l61 1592612 1 2024-09-10T04:05:17.367 INFO:tasks.workunit.client.1.smithi070.stdout:0/487: write d8/f1b [8133020,96688] 0 2024-09-10T04:05:17.367 INFO:tasks.workunit.client.0.smithi046.stdout:3/447: creat d12/d5d/d71/d49/d5e/f93 x:0 0 0 2024-09-10T04:05:17.370 INFO:tasks.workunit.client.1.smithi070.stdout:7/397: write d2/d3/dc/d5d/d41/f50 [104027,12207] 0 2024-09-10T04:05:17.374 INFO:tasks.workunit.client.0.smithi046.stdout:1/334: rename d3/l54 to d3/dc/d34/d51/l6f 0 2024-09-10T04:05:17.375 INFO:tasks.workunit.client.1.smithi070.stdout:2/412: dwrite d15/d18/d29/d2a/f58 [0,4194304] 0 2024-09-10T04:05:17.375 INFO:tasks.workunit.client.1.smithi070.stdout:4/394: rmdir de/d13/d18/d5c/d7f 39 2024-09-10T04:05:17.376 INFO:tasks.workunit.client.1.smithi070.stdout:2/413: chown d15/d18/d56/l8c 85491 1 2024-09-10T04:05:17.378 INFO:tasks.workunit.client.1.smithi070.stdout:3/453: sync 2024-09-10T04:05:17.378 INFO:tasks.workunit.client.1.smithi070.stdout:4/395: truncate de/d13/d18/d23/d47/f51 336757 0 2024-09-10T04:05:17.379 INFO:tasks.workunit.client.0.smithi046.stdout:0/356: unlink d1/dc/d25/d33/d4/f5b 0 2024-09-10T04:05:17.380 INFO:tasks.workunit.client.1.smithi070.stdout:1/379: dread d3/d8/d12/f24 [0,4194304] 0 2024-09-10T04:05:17.380 INFO:tasks.workunit.client.1.smithi070.stdout:3/454: chown db/d1c/d53/c6f 24 1 2024-09-10T04:05:17.381 INFO:tasks.workunit.client.1.smithi070.stdout:3/455: chown db/d1c/c42 15441 1 2024-09-10T04:05:17.382 INFO:tasks.workunit.client.0.smithi046.stdout:8/416: unlink d1/d29/d17/f69 0 2024-09-10T04:05:17.386 INFO:tasks.workunit.client.0.smithi046.stdout:3/448: chown d12/d1b/d50/c78 105816997 1 2024-09-10T04:05:17.387 INFO:tasks.workunit.client.0.smithi046.stdout:8/417: write d1/d29/d19/f56 [1056438,85785] 0 2024-09-10T04:05:17.389 INFO:tasks.workunit.client.0.smithi046.stdout:0/357: dwrite d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:17.390 INFO:tasks.workunit.client.1.smithi070.stdout:0/488: fdatasync d8/d13/d1a/d75/f84 0 2024-09-10T04:05:17.390 INFO:tasks.workunit.client.1.smithi070.stdout:0/489: chown d8/f49 148614 1 2024-09-10T04:05:17.390 INFO:tasks.workunit.client.0.smithi046.stdout:2/451: link d2/d6/f1a d2/d6/d23/d5d/fae 0 2024-09-10T04:05:17.391 INFO:tasks.workunit.client.0.smithi046.stdout:8/418: write d1/d29/d12/f45 [636919,128328] 0 2024-09-10T04:05:17.396 INFO:tasks.workunit.client.0.smithi046.stdout:0/358: dread d1/dc/d25/d33/d3/dd/d55/d4d/f4e [0,4194304] 0 2024-09-10T04:05:17.397 INFO:tasks.workunit.client.0.smithi046.stdout:0/359: stat d1/dc/d25/d33/d3/dd/d55/c57 0 2024-09-10T04:05:17.397 INFO:tasks.workunit.client.1.smithi070.stdout:9/508: creat d9/f9a x:0 0 0 2024-09-10T04:05:17.398 INFO:tasks.workunit.client.1.smithi070.stdout:9/509: chown d9/d60/d8a/l80 1627606 1 2024-09-10T04:05:17.399 INFO:tasks.workunit.client.1.smithi070.stdout:2/414: symlink d15/d18/d29/d2a/d2e/d32/d36/l93 0 2024-09-10T04:05:17.402 INFO:tasks.workunit.client.0.smithi046.stdout:3/449: mkdir d12/d13/d37/d94 0 2024-09-10T04:05:17.402 INFO:tasks.workunit.client.1.smithi070.stdout:4/396: creat de/d13/d5b/d7b/f8a x:0 0 0 2024-09-10T04:05:17.404 INFO:tasks.workunit.client.1.smithi070.stdout:1/380: mkdir d3/d10/d1d/d79 0 2024-09-10T04:05:17.404 INFO:tasks.workunit.client.0.smithi046.stdout:2/452: chown d2/d6/d23/l94 3485 1 2024-09-10T04:05:17.406 INFO:tasks.workunit.client.0.smithi046.stdout:5/529: link d7/d11/d3f/d5a/l85 d7/d11/d14/d2f/d64/lc9 0 2024-09-10T04:05:17.407 INFO:tasks.workunit.client.1.smithi070.stdout:3/456: creat db/d1c/d53/f95 x:0 0 0 2024-09-10T04:05:17.407 INFO:tasks.workunit.client.1.smithi070.stdout:4/397: dread de/d13/d18/d23/f61 [0,4194304] 0 2024-09-10T04:05:17.407 INFO:tasks.workunit.client.1.smithi070.stdout:3/457: fsync db/d1c/f4b 0 2024-09-10T04:05:17.409 INFO:tasks.workunit.client.1.smithi070.stdout:3/458: write db/d1c/d3e/f59 [658951,124461] 0 2024-09-10T04:05:17.410 INFO:tasks.workunit.client.0.smithi046.stdout:2/453: truncate d2/d6/d23/d5d/d77/f54 4997241 0 2024-09-10T04:05:17.410 INFO:tasks.workunit.client.0.smithi046.stdout:8/419: rename d1/d29/d19/f83 to d1/d29/d17/f8e 0 2024-09-10T04:05:17.410 INFO:tasks.workunit.client.1.smithi070.stdout:5/451: link d17/d24/d77/f8c d17/d24/d45/f96 0 2024-09-10T04:05:17.411 INFO:tasks.workunit.client.0.smithi046.stdout:3/450: dwrite d12/d1b/d50/d52/f65 [4194304,4194304] 0 2024-09-10T04:05:17.414 INFO:tasks.workunit.client.1.smithi070.stdout:8/509: mknod d4/d16/d17/d19/d28/d43/d67/cac 0 2024-09-10T04:05:17.414 INFO:tasks.workunit.client.0.smithi046.stdout:3/451: write d12/d13/d14/f25 [1242259,72013] 0 2024-09-10T04:05:17.415 INFO:tasks.workunit.client.0.smithi046.stdout:3/452: write d12/f63 [120917,87129] 0 2024-09-10T04:05:17.419 INFO:tasks.workunit.client.1.smithi070.stdout:0/490: creat d8/d24/d2d/d34/fa3 x:0 0 0 2024-09-10T04:05:17.420 INFO:tasks.workunit.client.1.smithi070.stdout:0/491: fsync d8/d13/d47/f5c 0 2024-09-10T04:05:17.420 INFO:tasks.workunit.client.1.smithi070.stdout:0/492: read d8/d24/f43 [1536335,71580] 0 2024-09-10T04:05:17.422 INFO:tasks.workunit.client.0.smithi046.stdout:0/360: mkdir d1/dc/d25/d72 0 2024-09-10T04:05:17.423 INFO:tasks.workunit.client.0.smithi046.stdout:3/453: dwrite d12/d13/d14/f25 [0,4194304] 0 2024-09-10T04:05:17.427 INFO:tasks.workunit.client.1.smithi070.stdout:9/510: symlink d9/d60/l9b 0 2024-09-10T04:05:17.427 INFO:tasks.workunit.client.1.smithi070.stdout:9/511: fdatasync d9/f68 0 2024-09-10T04:05:17.429 INFO:tasks.workunit.client.1.smithi070.stdout:2/415: mkdir d15/d2b/d94 0 2024-09-10T04:05:17.431 INFO:tasks.workunit.client.1.smithi070.stdout:6/528: getdents d0/d3/d18/d1b/d17/d54 0 2024-09-10T04:05:17.433 INFO:tasks.workunit.client.0.smithi046.stdout:0/361: dwrite d1/dc/d25/d33/d3/dd/d55/f4c [0,4194304] 0 2024-09-10T04:05:17.433 INFO:tasks.workunit.client.0.smithi046.stdout:7/453: sync 2024-09-10T04:05:17.433 INFO:tasks.workunit.client.1.smithi070.stdout:6/529: write d0/d3/d18/d1b/f3a [748785,74096] 0 2024-09-10T04:05:17.435 INFO:tasks.workunit.client.1.smithi070.stdout:7/398: dwrite d2/d3/dc/d5d/f46 [0,4194304] 0 2024-09-10T04:05:17.437 INFO:tasks.workunit.client.1.smithi070.stdout:1/381: mkdir d3/d8/d27/d7a 0 2024-09-10T04:05:17.444 INFO:tasks.workunit.client.0.smithi046.stdout:0/362: dwrite d1/dc/d25/d33/d4/f6f [0,4194304] 0 2024-09-10T04:05:17.445 INFO:tasks.workunit.client.1.smithi070.stdout:6/530: dread d0/d3/d1f/f46 [0,4194304] 0 2024-09-10T04:05:17.446 INFO:tasks.workunit.client.0.smithi046.stdout:7/454: write d5/d9/d43/f51 [266808,54625] 0 2024-09-10T04:05:17.448 INFO:tasks.workunit.client.0.smithi046.stdout:7/455: fsync d5/d9/d12/d72/d77/f82 0 2024-09-10T04:05:17.449 INFO:tasks.workunit.client.1.smithi070.stdout:5/452: creat d17/d24/d4e/f97 x:0 0 0 2024-09-10T04:05:17.451 INFO:tasks.workunit.client.0.smithi046.stdout:1/335: link d3/l7 d3/d27/d5d/d60/l70 0 2024-09-10T04:05:17.453 INFO:tasks.workunit.client.1.smithi070.stdout:8/510: creat d4/d16/d17/d5c/d74/da4/fad x:0 0 0 2024-09-10T04:05:17.453 INFO:tasks.workunit.client.0.smithi046.stdout:0/363: dread d1/dc/d25/d33/d3/dd/d55/f4c [0,4194304] 0 2024-09-10T04:05:17.457 INFO:tasks.workunit.client.0.smithi046.stdout:3/454: rename d12/d13/d14/c5a to d12/d13/d14/d1e/d2d/d39/d67/d92/c95 0 2024-09-10T04:05:17.457 INFO:tasks.workunit.client.0.smithi046.stdout:0/364: write d1/dc/d25/d33/d4/f6f [1183625,2239] 0 2024-09-10T04:05:17.459 INFO:tasks.workunit.client.1.smithi070.stdout:8/511: dwrite d4/d16/d73/f93 [0,4194304] 0 2024-09-10T04:05:17.460 INFO:tasks.workunit.client.1.smithi070.stdout:8/512: write d4/d16/d73/d96/f59 [419516,96193] 0 2024-09-10T04:05:17.460 INFO:tasks.workunit.client.0.smithi046.stdout:7/456: dwrite d5/d6/db/d11/d1f/f48 [0,4194304] 0 2024-09-10T04:05:17.462 INFO:tasks.workunit.client.0.smithi046.stdout:0/365: chown d1/dc/d25/d33/d1a/d41/l50 34450 1 2024-09-10T04:05:17.462 INFO:tasks.workunit.client.1.smithi070.stdout:0/493: truncate d8/d24/d2d/f3a 5021286 0 2024-09-10T04:05:17.463 INFO:tasks.workunit.client.0.smithi046.stdout:7/457: write d5/d6/d39/d6c/f75 [146896,17266] 0 2024-09-10T04:05:17.465 INFO:tasks.workunit.client.1.smithi070.stdout:0/494: read - d8/d24/d2d/d34/fa3 zero size 2024-09-10T04:05:17.465 INFO:tasks.workunit.client.0.smithi046.stdout:2/454: getdents d2/d86/d9c 0 2024-09-10T04:05:17.468 INFO:tasks.workunit.client.0.smithi046.stdout:5/530: write d7/d11/d3f/d53/f47 [993962,59481] 0 2024-09-10T04:05:17.468 INFO:tasks.workunit.client.0.smithi046.stdout:8/420: mknod d1/d29/d19/d36/c8f 0 2024-09-10T04:05:17.469 INFO:tasks.workunit.client.1.smithi070.stdout:8/513: dwrite d4/d16/f33 [0,4194304] 0 2024-09-10T04:05:17.471 INFO:tasks.workunit.client.1.smithi070.stdout:2/416: unlink d15/d18/d42/f69 0 2024-09-10T04:05:17.473 INFO:tasks.workunit.client.1.smithi070.stdout:2/417: write d15/d18/d56/f85 [1059972,86120] 0 2024-09-10T04:05:17.477 INFO:tasks.workunit.client.0.smithi046.stdout:7/458: dwrite d5/d6/d39/d6c/f75 [0,4194304] 0 2024-09-10T04:05:17.478 INFO:tasks.workunit.client.0.smithi046.stdout:7/459: dread - d5/d6/d32/f95 zero size 2024-09-10T04:05:17.479 INFO:tasks.workunit.client.1.smithi070.stdout:4/398: dwrite de/d13/d18/d23/d47/d59/f7d [0,4194304] 0 2024-09-10T04:05:17.481 INFO:tasks.workunit.client.0.smithi046.stdout:7/460: stat d5/d6/d62 0 2024-09-10T04:05:17.481 INFO:tasks.workunit.client.0.smithi046.stdout:7/461: chown d5/d6 7 1 2024-09-10T04:05:17.481 INFO:tasks.workunit.client.1.smithi070.stdout:2/418: dwrite d15/d18/d29/f46 [4194304,4194304] 0 2024-09-10T04:05:17.482 INFO:tasks.workunit.client.0.smithi046.stdout:7/462: readlink d5/d33/l91 0 2024-09-10T04:05:17.483 INFO:tasks.workunit.client.0.smithi046.stdout:7/463: write d5/d6/db/d11/d1e/d20/f92 [848284,97474] 0 2024-09-10T04:05:17.492 INFO:tasks.workunit.client.0.smithi046.stdout:9/453: sync 2024-09-10T04:05:17.493 INFO:tasks.workunit.client.0.smithi046.stdout:6/419: sync 2024-09-10T04:05:17.495 INFO:tasks.workunit.client.1.smithi070.stdout:3/459: creat db/d83/d92/f96 x:0 0 0 2024-09-10T04:05:17.496 INFO:tasks.workunit.client.1.smithi070.stdout:9/512: dwrite d9/d59/d2a/d3f/f48 [0,4194304] 0 2024-09-10T04:05:17.496 INFO:tasks.workunit.client.0.smithi046.stdout:1/336: rename d3/dc/d34/l4e to d3/d62/l71 0 2024-09-10T04:05:17.497 INFO:tasks.workunit.client.0.smithi046.stdout:3/455: mkdir d12/d5d/d71/d49/d5e/d96 0 2024-09-10T04:05:17.498 INFO:tasks.workunit.client.0.smithi046.stdout:1/337: chown d3/d27/d2a 0 1 2024-09-10T04:05:17.498 INFO:tasks.workunit.client.1.smithi070.stdout:5/453: creat d17/d24/d45/f98 x:0 0 0 2024-09-10T04:05:17.500 INFO:tasks.workunit.client.0.smithi046.stdout:1/338: read d3/f4d [3269551,85786] 0 2024-09-10T04:05:17.501 INFO:tasks.workunit.client.0.smithi046.stdout:1/339: write d3/d27/f42 [2750328,81291] 0 2024-09-10T04:05:17.503 INFO:tasks.workunit.client.1.smithi070.stdout:5/454: dread d17/d19/d6a/f7c [0,4194304] 0 2024-09-10T04:05:17.506 INFO:tasks.workunit.client.0.smithi046.stdout:5/531: write d7/d11/d3f/d53/f9e [553830,7052] 0 2024-09-10T04:05:17.507 INFO:tasks.workunit.client.0.smithi046.stdout:1/340: dread d3/d27/d32/d1f/f29 [0,4194304] 0 2024-09-10T04:05:17.508 INFO:tasks.workunit.client.1.smithi070.stdout:0/495: rmdir d8/d13/d1a/d75 39 2024-09-10T04:05:17.510 INFO:tasks.workunit.client.0.smithi046.stdout:0/366: mknod d1/dc/d25/d33/d3/c73 0 2024-09-10T04:05:17.512 INFO:tasks.workunit.client.0.smithi046.stdout:3/456: creat d12/d5d/d71/d49/d5e/f97 x:0 0 0 2024-09-10T04:05:17.514 INFO:tasks.workunit.client.1.smithi070.stdout:4/399: rename de/d13/d18/d23/d47/d59/d79/f56 to de/d13/d18/d26/f8b 0 2024-09-10T04:05:17.515 INFO:tasks.workunit.client.1.smithi070.stdout:2/419: creat d15/d18/d29/d64/f95 x:0 0 0 2024-09-10T04:05:17.515 INFO:tasks.workunit.client.1.smithi070.stdout:2/420: stat d15/c31 0 2024-09-10T04:05:17.516 INFO:tasks.workunit.client.1.smithi070.stdout:7/399: symlink d2/d7/d9/l95 0 2024-09-10T04:05:17.516 INFO:tasks.workunit.client.0.smithi046.stdout:8/421: symlink d1/d29/d19/d36/d8d/l90 0 2024-09-10T04:05:17.516 INFO:tasks.workunit.client.1.smithi070.stdout:2/421: dread - d15/d18/d56/d7c/f92 zero size 2024-09-10T04:05:17.517 INFO:tasks.workunit.client.1.smithi070.stdout:1/382: mkdir d3/d8/d12/d2f/d32/d7b 0 2024-09-10T04:05:17.518 INFO:tasks.workunit.client.1.smithi070.stdout:0/496: dread d8/fc [0,4194304] 0 2024-09-10T04:05:17.518 INFO:tasks.workunit.client.0.smithi046.stdout:1/341: rmdir d3/d27/d5d/d60 39 2024-09-10T04:05:17.519 INFO:tasks.workunit.client.0.smithi046.stdout:1/342: fsync d3/d27/d32/d1f/f61 0 2024-09-10T04:05:17.522 INFO:tasks.workunit.client.0.smithi046.stdout:0/367: creat d1/dc/d49/f74 x:0 0 0 2024-09-10T04:05:17.522 INFO:tasks.workunit.client.1.smithi070.stdout:5/455: rmdir d17/d24/d77 39 2024-09-10T04:05:17.522 INFO:tasks.workunit.client.1.smithi070.stdout:0/497: dwrite d8/d13/d1a/f7e [0,4194304] 0 2024-09-10T04:05:17.523 INFO:tasks.workunit.client.0.smithi046.stdout:3/457: mknod d12/d5d/d71/c98 0 2024-09-10T04:05:17.523 INFO:tasks.workunit.client.1.smithi070.stdout:0/498: write d8/d13/d1a/f3c [4928838,30837] 0 2024-09-10T04:05:17.524 INFO:tasks.workunit.client.1.smithi070.stdout:5/456: truncate d17/d19/d35/f6c 182469 0 2024-09-10T04:05:17.524 INFO:tasks.workunit.client.0.smithi046.stdout:0/368: chown d1/dc/d25/d33/d4/d19/l5f 0 1 2024-09-10T04:05:17.525 INFO:tasks.workunit.client.1.smithi070.stdout:8/514: mknod d4/d16/d17/d19/d28/da8/cae 0 2024-09-10T04:05:17.526 INFO:tasks.workunit.client.0.smithi046.stdout:8/422: mknod d1/d29/d17/d1f/d46/d52/c91 0 2024-09-10T04:05:17.527 INFO:tasks.workunit.client.0.smithi046.stdout:1/343: dwrite d3/d27/d32/d28/f2c [0,4194304] 0 2024-09-10T04:05:17.528 INFO:tasks.workunit.client.0.smithi046.stdout:5/532: mknod d7/d11/d14/d8e/d96/cca 0 2024-09-10T04:05:17.530 INFO:tasks.workunit.client.1.smithi070.stdout:3/460: rename db/d83/f8d to db/d1c/d25/d28/d43/f97 0 2024-09-10T04:05:17.530 INFO:tasks.workunit.client.1.smithi070.stdout:0/499: dwrite d8/d13/d33/d81/d78/fa2 [0,4194304] 0 2024-09-10T04:05:17.531 INFO:tasks.workunit.client.1.smithi070.stdout:0/500: write d8/f92 [199713,27344] 0 2024-09-10T04:05:17.532 INFO:tasks.workunit.client.1.smithi070.stdout:4/400: chown de/d13/d18/d23/c4b 0 1 2024-09-10T04:05:17.532 INFO:tasks.workunit.client.1.smithi070.stdout:7/400: mknod d2/d7/db/d3a/c96 0 2024-09-10T04:05:17.534 INFO:tasks.workunit.client.0.smithi046.stdout:5/533: dread d7/d11/d14/d2f/f8c [0,4194304] 0 2024-09-10T04:05:17.536 INFO:tasks.workunit.client.0.smithi046.stdout:5/534: fsync d7/d11/d3f/f80 0 2024-09-10T04:05:17.537 INFO:tasks.workunit.client.1.smithi070.stdout:2/422: creat d15/d18/d56/d7c/d55/d6d/f96 x:0 0 0 2024-09-10T04:05:17.537 INFO:tasks.workunit.client.1.smithi070.stdout:7/401: write d2/d7/d9/d18/d3c/f58 [3046239,37490] 0 2024-09-10T04:05:17.537 INFO:tasks.workunit.client.1.smithi070.stdout:9/513: mknod d9/d59/c9c 0 2024-09-10T04:05:17.537 INFO:tasks.workunit.client.0.smithi046.stdout:8/423: symlink d1/d29/d19/d36/d6e/d75/l92 0 2024-09-10T04:05:17.537 INFO:tasks.workunit.client.0.smithi046.stdout:0/369: dread d1/dc/d25/d33/d3/d29/d2d/f32 [0,4194304] 0 2024-09-10T04:05:17.538 INFO:tasks.workunit.client.0.smithi046.stdout:8/424: chown d1/d29/d19/l3d 1930 1 2024-09-10T04:05:17.539 INFO:tasks.workunit.client.1.smithi070.stdout:9/514: write d9/d60/d8a/f81 [454338,3802] 0 2024-09-10T04:05:17.541 INFO:tasks.workunit.client.1.smithi070.stdout:6/531: link d0/d3/d1f/f37 d0/d3/d18/d1b/d17/fa5 0 2024-09-10T04:05:17.542 INFO:tasks.workunit.client.1.smithi070.stdout:9/515: readlink d9/d60/d8a/d1b/l27 0 2024-09-10T04:05:17.544 INFO:tasks.workunit.client.0.smithi046.stdout:1/344: dread d3/d27/d32/d1f/f35 [0,4194304] 0 2024-09-10T04:05:17.545 INFO:tasks.workunit.client.0.smithi046.stdout:1/345: chown d3/d27/d32/d28 221430 1 2024-09-10T04:05:17.545 INFO:tasks.workunit.client.1.smithi070.stdout:0/501: dwrite d8/d13/f19 [4194304,4194304] 0 2024-09-10T04:05:17.545 INFO:tasks.workunit.client.0.smithi046.stdout:0/370: mkdir d1/d13/d37/d75 0 2024-09-10T04:05:17.546 INFO:tasks.workunit.client.0.smithi046.stdout:1/346: stat d3/d27/d32/f2e 0 2024-09-10T04:05:17.547 INFO:tasks.workunit.client.1.smithi070.stdout:8/515: rmdir d4/d16/d17/d19/d28/d43 39 2024-09-10T04:05:17.547 INFO:tasks.workunit.client.0.smithi046.stdout:5/535: creat d7/d11/d3f/d53/d75/fcb x:0 0 0 2024-09-10T04:05:17.548 INFO:tasks.workunit.client.1.smithi070.stdout:5/457: read f8 [316920,107766] 0 2024-09-10T04:05:17.548 INFO:tasks.workunit.client.0.smithi046.stdout:1/347: write d3/d27/d5d/f6a [1100698,129944] 0 2024-09-10T04:05:17.549 INFO:tasks.workunit.client.0.smithi046.stdout:8/425: mknod d1/d29/d19/d73/c93 0 2024-09-10T04:05:17.549 INFO:tasks.workunit.client.1.smithi070.stdout:2/423: dwrite d15/d18/d56/d7c/d55/f65 [0,4194304] 0 2024-09-10T04:05:17.550 INFO:tasks.workunit.client.1.smithi070.stdout:5/458: readlink d17/d19/d6a/d54/l94 0 2024-09-10T04:05:17.550 INFO:tasks.workunit.client.1.smithi070.stdout:4/401: dread - de/d13/d18/d26/f2f zero size 2024-09-10T04:05:17.552 INFO:tasks.workunit.client.1.smithi070.stdout:6/532: dwrite d0/d3/d18/d1b/f3a [0,4194304] 0 2024-09-10T04:05:17.554 INFO:tasks.workunit.client.1.smithi070.stdout:1/383: mknod d3/d8/d13/d3f/d6d/c7c 0 2024-09-10T04:05:17.556 INFO:tasks.workunit.client.0.smithi046.stdout:1/348: dread d3/d27/d32/d28/f2c [0,4194304] 0 2024-09-10T04:05:17.557 INFO:tasks.workunit.client.1.smithi070.stdout:7/402: mknod d2/d3/d63/c97 0 2024-09-10T04:05:17.557 INFO:tasks.workunit.client.1.smithi070.stdout:7/403: readlink d2/d3/dc/l6a 0 2024-09-10T04:05:17.559 INFO:tasks.workunit.client.1.smithi070.stdout:7/404: write d2/d3/dc/d5d/d41/f53 [260933,83237] 0 2024-09-10T04:05:17.559 INFO:tasks.workunit.client.0.smithi046.stdout:5/536: rename d7/d11/d3f/d53/dac to d7/d11/d3f/d53/da9/d9c/da5/dcc 0 2024-09-10T04:05:17.559 INFO:tasks.workunit.client.1.smithi070.stdout:7/405: fdatasync d2/f55 0 2024-09-10T04:05:17.563 INFO:tasks.workunit.client.1.smithi070.stdout:0/502: rmdir d8/d24/d36 39 2024-09-10T04:05:17.565 INFO:tasks.workunit.client.0.smithi046.stdout:1/349: creat d3/dc/d34/f72 x:0 0 0 2024-09-10T04:05:17.565 INFO:tasks.workunit.client.1.smithi070.stdout:9/516: rename d9/d60/d8a/f19 to d9/d60/d8a/d82/d99/f9d 0 2024-09-10T04:05:17.570 INFO:tasks.workunit.client.0.smithi046.stdout:1/350: dread - d3/dc/d34/f72 zero size 2024-09-10T04:05:17.571 INFO:tasks.workunit.client.0.smithi046.stdout:5/537: dwrite d7/d11/d3f/d53/d75/f93 [0,4194304] 0 2024-09-10T04:05:17.576 INFO:tasks.workunit.client.1.smithi070.stdout:3/461: symlink db/d1c/l98 0 2024-09-10T04:05:17.577 INFO:tasks.workunit.client.0.smithi046.stdout:0/371: link d1/dc/d49/f4a d1/dc/f76 0 2024-09-10T04:05:17.578 INFO:tasks.workunit.client.1.smithi070.stdout:5/459: chown d17/d24/d77/c8b 1750849 1 2024-09-10T04:05:17.581 INFO:tasks.workunit.client.1.smithi070.stdout:2/424: creat d15/d18/d56/d7c/d55/d6d/f97 x:0 0 0 2024-09-10T04:05:17.581 INFO:tasks.workunit.client.1.smithi070.stdout:3/462: dwrite db/d1c/d25/d28/f73 [0,4194304] 0 2024-09-10T04:05:17.587 INFO:tasks.workunit.client.0.smithi046.stdout:1/351: creat d3/d62/f73 x:0 0 0 2024-09-10T04:05:17.588 INFO:tasks.workunit.client.1.smithi070.stdout:6/533: creat d0/d3/d18/d1b/d17/d53/fa6 x:0 0 0 2024-09-10T04:05:17.589 INFO:tasks.workunit.client.1.smithi070.stdout:6/534: chown d0/d3/d8c/d9a/d92/l9b 5303146 1 2024-09-10T04:05:17.589 INFO:tasks.workunit.client.0.smithi046.stdout:1/352: read d3/ff [3921682,23626] 0 2024-09-10T04:05:17.592 INFO:tasks.workunit.client.0.smithi046.stdout:8/426: getdents d1/d29/d19/d36/d4c 0 2024-09-10T04:05:17.597 INFO:tasks.workunit.client.0.smithi046.stdout:0/372: mkdir d1/dc/d25/d33/d3/dd/d55/d77 0 2024-09-10T04:05:17.598 INFO:tasks.workunit.client.0.smithi046.stdout:0/373: chown d1/dc/d25/d33/d3/f46 0 1 2024-09-10T04:05:17.603 INFO:tasks.workunit.client.0.smithi046.stdout:5/538: unlink d7/d11/d14/d2f/d2b/l8b 0 2024-09-10T04:05:17.606 INFO:tasks.workunit.client.0.smithi046.stdout:2/455: dwrite d2/d8/d37/f66 [0,4194304] 0 2024-09-10T04:05:17.607 INFO:tasks.workunit.client.0.smithi046.stdout:5/539: readlink d7/d11/d3f/d5a/d63/l68 0 2024-09-10T04:05:17.607 INFO:tasks.workunit.client.0.smithi046.stdout:1/353: symlink d3/dc/d34/d58/l74 0 2024-09-10T04:05:17.608 INFO:tasks.workunit.client.0.smithi046.stdout:2/456: fdatasync d2/d8/d37/d63/f75 0 2024-09-10T04:05:17.609 INFO:tasks.workunit.client.0.smithi046.stdout:5/540: fdatasync d7/d11/d3f/d53/d75/f93 0 2024-09-10T04:05:17.612 INFO:tasks.workunit.client.0.smithi046.stdout:8/427: dread d1/d29/d17/d1f/d46/d52/f65 [0,4194304] 0 2024-09-10T04:05:17.614 INFO:tasks.workunit.client.0.smithi046.stdout:0/374: symlink d1/dc/d25/d33/d3/d29/l78 0 2024-09-10T04:05:17.615 INFO:tasks.workunit.client.1.smithi070.stdout:8/516: rename d4/d16/d17/d19/c3a to d4/d16/d17/d19/d28/da8/caf 0 2024-09-10T04:05:17.620 INFO:tasks.workunit.client.0.smithi046.stdout:5/541: dwrite d7/d11/d14/d2f/d2b/d6a/fa1 [0,4194304] 0 2024-09-10T04:05:17.621 INFO:tasks.workunit.client.0.smithi046.stdout:7/464: write d5/d9/f5b [26154,103908] 0 2024-09-10T04:05:17.632 INFO:tasks.workunit.client.0.smithi046.stdout:9/454: dwrite d4/d7/d1f/d54/d56/f8b [0,4194304] 0 2024-09-10T04:05:17.633 INFO:tasks.workunit.client.0.smithi046.stdout:8/428: creat d1/d29/d17/d1f/d46/d6a/f94 x:0 0 0 2024-09-10T04:05:17.634 INFO:tasks.workunit.client.1.smithi070.stdout:1/384: mkdir d3/d8/d27/d7a/d7d 0 2024-09-10T04:05:17.635 INFO:tasks.workunit.client.0.smithi046.stdout:5/542: dwrite d7/d11/d14/f95 [0,4194304] 0 2024-09-10T04:05:17.645 INFO:tasks.workunit.client.1.smithi070.stdout:8/517: truncate d4/fe 296147 0 2024-09-10T04:05:17.648 INFO:tasks.workunit.client.0.smithi046.stdout:9/455: dwrite d4/d18/f33 [0,4194304] 0 2024-09-10T04:05:17.649 INFO:tasks.workunit.client.1.smithi070.stdout:9/517: link d9/f25 d9/d1c/d34/d8b/f9e 0 2024-09-10T04:05:17.651 INFO:tasks.workunit.client.1.smithi070.stdout:5/460: mkdir d17/d6f/d32/d90/d99 0 2024-09-10T04:05:17.652 INFO:tasks.workunit.client.1.smithi070.stdout:5/461: dread - d17/f8e zero size 2024-09-10T04:05:17.654 INFO:tasks.workunit.client.1.smithi070.stdout:3/463: mkdir db/d79/d94/d99 0 2024-09-10T04:05:17.655 INFO:tasks.workunit.client.0.smithi046.stdout:5/543: dwrite d7/d11/d3f/d5a/d63/f7f [0,4194304] 0 2024-09-10T04:05:17.656 INFO:tasks.workunit.client.1.smithi070.stdout:3/464: truncate db/d1c/d25/d28/f7d 1002959 0 2024-09-10T04:05:17.658 INFO:tasks.workunit.client.1.smithi070.stdout:5/462: dread f8 [0,4194304] 0 2024-09-10T04:05:17.660 INFO:tasks.workunit.client.0.smithi046.stdout:0/375: symlink d1/d13/d37/d75/l79 0 2024-09-10T04:05:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.658+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2b84000bf0 con 0x7f2b9006ded0 2024-09-10T04:05:17.660 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:17 smithi046 ceph-mon[28328]: pgmap v262: 65 pgs: 65 active+clean; 1.6 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 219 MiB/s wr, 337 op/s 2024-09-10T04:05:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.661+0000 7f2ba0d99640 1 -- 172.21.15.46:0/3127532265 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f2b84000bf0 con 0x7f2b9006ded0 2024-09-10T04:05:17.664 INFO:tasks.workunit.client.1.smithi070.stdout:7/406: link d2/d7/db/d3a/d7e/c85 d2/d7/d9/d18/d3c/c98 0 2024-09-10T04:05:17.665 INFO:tasks.workunit.client.0.smithi046.stdout:3/458: dwrite d12/d13/f18 [0,4194304] 0 2024-09-10T04:05:17.665 INFO:tasks.workunit.client.0.smithi046.stdout:8/429: creat d1/d29/d19/d36/d6e/f95 x:0 0 0 2024-09-10T04:05:17.666 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:17 smithi070 ceph-mon[35495]: pgmap v262: 65 pgs: 65 active+clean; 1.6 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail; 39 MiB/s rd, 219 MiB/s wr, 337 op/s 2024-09-10T04:05:17.668 INFO:tasks.workunit.client.1.smithi070.stdout:8/518: creat d4/d16/d73/d96/fb0 x:0 0 0 2024-09-10T04:05:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.666+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2b9006ded0 msgr2=0x7f2b90070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.666+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2b9006ded0 0x7f2b90070390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2ba41a7550 tx=0x7f2b8c005f90 comp rx=0 tx=0).stop 2024-09-10T04:05:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.666+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 msgr2=0x7f2ba41a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.666+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba41a60a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009230 tx=0x7f2b94004570 comp rx=0 tx=0).stop 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.667+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 shutdown_connections 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.667+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f2b9006ded0 0x7f2b90070390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.667+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ba40739b0 0x7f2ba41a65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.667+0000 7f2ba8a03640 1 --2- 172.21.15.46:0/3127532265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ba4072fe0 0x7f2ba41a60a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.667+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 >> 172.21.15.46:0/3127532265 conn(0x7f2ba406e8b0 msgr2=0x7f2ba410c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.667+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 shutdown_connections 2024-09-10T04:05:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.667+0000 7f2ba8a03640 1 -- 172.21.15.46:0/3127532265 wait complete. 2024-09-10T04:05:17.671 INFO:tasks.workunit.client.1.smithi070.stdout:9/518: mknod d9/d7a/d6f/d90/c9f 0 2024-09-10T04:05:17.671 INFO:tasks.workunit.client.1.smithi070.stdout:7/407: dwrite d2/d40/d6c/f7f [0,4194304] 0 2024-09-10T04:05:17.672 INFO:tasks.workunit.client.0.smithi046.stdout:9/456: dread d4/d18/f4f [0,4194304] 0 2024-09-10T04:05:17.676 INFO:tasks.workunit.client.0.smithi046.stdout:8/430: dwrite d1/d29/f4a [0,4194304] 0 2024-09-10T04:05:17.676 INFO:tasks.workunit.client.1.smithi070.stdout:7/408: dread d2/d3/dc/d5d/d41/f50 [0,4194304] 0 2024-09-10T04:05:17.677 INFO:tasks.workunit.client.1.smithi070.stdout:7/409: fdatasync d2/d7/db/f5c 0 2024-09-10T04:05:17.684 INFO:tasks.workunit.client.1.smithi070.stdout:9/519: dread d9/d7a/f51 [4194304,4194304] 0 2024-09-10T04:05:17.686 INFO:tasks.workunit.client.1.smithi070.stdout:6/535: creat d0/fa7 x:0 0 0 2024-09-10T04:05:17.687 INFO:tasks.workunit.client.1.smithi070.stdout:6/536: fsync d0/d3/d18/d1b/d33/f39 0 2024-09-10T04:05:17.690 INFO:tasks.workunit.client.0.smithi046.stdout:0/376: creat d1/dc/d25/d33/d1a/d41/f7a x:0 0 0 2024-09-10T04:05:17.695 INFO:tasks.workunit.client.0.smithi046.stdout:9/457: dread d4/d7/d1f/f15 [0,4194304] 0 2024-09-10T04:05:17.696 INFO:tasks.workunit.client.0.smithi046.stdout:1/354: getdents d3/d27/d32/d28 0 2024-09-10T04:05:17.696 INFO:tasks.workunit.client.1.smithi070.stdout:1/385: creat d3/d8/d12/d2f/f7e x:0 0 0 2024-09-10T04:05:17.701 INFO:tasks.workunit.client.1.smithi070.stdout:6/537: dread d0/d3/d1f/f22 [0,4194304] 0 2024-09-10T04:05:17.701 INFO:tasks.workunit.client.0.smithi046.stdout:9/458: dread d4/d29/f6d [0,4194304] 0 2024-09-10T04:05:17.702 INFO:tasks.workunit.client.0.smithi046.stdout:9/459: chown d4/d44/f69 30292810 1 2024-09-10T04:05:17.705 INFO:tasks.workunit.client.0.smithi046.stdout:8/431: rename d1/d29/d17/d1f/c72 to d1/d29/d17/d1f/d2a/c96 0 2024-09-10T04:05:17.706 INFO:tasks.workunit.client.0.smithi046.stdout:8/432: chown d1/d29/d17/d1f/d46/d67/d80 179761480 1 2024-09-10T04:05:17.707 INFO:tasks.workunit.client.1.smithi070.stdout:9/520: unlink d9/d59/d2a/d3f/f48 0 2024-09-10T04:05:17.707 INFO:tasks.workunit.client.1.smithi070.stdout:0/503: dwrite d8/d24/d2d/f51 [0,4194304] 0 2024-09-10T04:05:17.707 INFO:tasks.workunit.client.0.smithi046.stdout:8/433: write d1/d29/d17/d1f/f35 [1469333,94261] 0 2024-09-10T04:05:17.710 INFO:tasks.workunit.client.0.smithi046.stdout:5/544: mknod d7/d11/d3f/d53/da9/d9c/da5/dcc/dc0/ccd 0 2024-09-10T04:05:17.713 INFO:tasks.workunit.client.0.smithi046.stdout:5/545: truncate d7/d11/d14/d2f/d2b/f3a 1308464 0 2024-09-10T04:05:17.714 INFO:tasks.workunit.client.0.smithi046.stdout:3/459: unlink d12/d5d/d71/d49/l53 0 2024-09-10T04:05:17.715 INFO:tasks.workunit.client.0.smithi046.stdout:3/460: dread - d12/d5d/d71/d49/d5e/f97 zero size 2024-09-10T04:05:17.716 INFO:tasks.workunit.client.1.smithi070.stdout:8/519: rename d4/d16/d17/d19/d28/d43/d67/cac to d4/d39/cb1 0 2024-09-10T04:05:17.720 INFO:tasks.workunit.client.0.smithi046.stdout:4/467: sync 2024-09-10T04:05:17.723 INFO:tasks.workunit.client.0.smithi046.stdout:7/465: link d5/d7f/l26 d5/d9/d8b/l97 0 2024-09-10T04:05:17.723 INFO:tasks.workunit.client.0.smithi046.stdout:2/457: dwrite d2/d6/d23/d5d/fae [0,4194304] 0 2024-09-10T04:05:17.726 INFO:tasks.workunit.client.0.smithi046.stdout:2/458: dread d2/d49/f9b [0,4194304] 0 2024-09-10T04:05:17.727 INFO:tasks.workunit.client.0.smithi046.stdout:7/466: write d5/d6/db/dd/d4b/f52 [353449,6273] 0 2024-09-10T04:05:17.727 INFO:tasks.workunit.client.0.smithi046.stdout:2/459: read d2/d49/f9b [246320,101598] 0 2024-09-10T04:05:17.731 INFO:tasks.workunit.client.1.smithi070.stdout:3/465: link db/d1c/d53/f95 db/d1c/d25/f9a 0 2024-09-10T04:05:17.733 INFO:tasks.workunit.client.0.smithi046.stdout:9/460: write d4/d3c/f6b [4133104,31396] 0 2024-09-10T04:05:17.736 INFO:tasks.workunit.client.0.smithi046.stdout:9/461: read d4/d7/fe [314348,43499] 0 2024-09-10T04:05:17.738 INFO:tasks.workunit.client.0.smithi046.stdout:1/355: rename d3/dc/d34/d58/l74 to d3/d27/d32/d28/l75 0 2024-09-10T04:05:17.742 INFO:tasks.workunit.client.0.smithi046.stdout:8/434: creat d1/d29/d12/f97 x:0 0 0 2024-09-10T04:05:17.744 INFO:tasks.workunit.client.1.smithi070.stdout:0/504: mknod d8/d13/d33/d81/ca4 0 2024-09-10T04:05:17.744 INFO:tasks.workunit.client.1.smithi070.stdout:0/505: chown d8/d13/d33/d53/d59/c82 29029 1 2024-09-10T04:05:17.748 INFO:tasks.workunit.client.1.smithi070.stdout:9/521: rename d9/d60/d8a/d3c/c54 to d9/d1c/d34/d63/ca0 0 2024-09-10T04:05:17.751 INFO:tasks.workunit.client.1.smithi070.stdout:7/410: creat d2/d7/d9/d18/f99 x:0 0 0 2024-09-10T04:05:17.754 INFO:tasks.workunit.client.1.smithi070.stdout:6/538: symlink d0/d3/d1f/d81/la8 0 2024-09-10T04:05:17.755 INFO:tasks.workunit.client.1.smithi070.stdout:6/539: write d0/d3/d18/f4c [1567682,125528] 0 2024-09-10T04:05:17.756 INFO:tasks.workunit.client.1.smithi070.stdout:6/540: write d0/d3/d18/d1b/d17/d53/f61 [4068151,107514] 0 2024-09-10T04:05:17.758 INFO:tasks.workunit.client.1.smithi070.stdout:5/463: getdents d17/d6f/d32/d90 0 2024-09-10T04:05:17.764 INFO:tasks.workunit.client.0.smithi046.stdout:7/467: symlink d5/d6/db/d11/d1e/d20/l98 0 2024-09-10T04:05:17.764 INFO:tasks.workunit.client.1.smithi070.stdout:9/522: rename d9/d1c/d34/d63/c95 to d9/d1c/d34/d63/ca1 0 2024-09-10T04:05:17.765 INFO:tasks.workunit.client.1.smithi070.stdout:7/411: creat d2/d7/db/d3a/f9a x:0 0 0 2024-09-10T04:05:17.765 INFO:tasks.workunit.client.0.smithi046.stdout:4/468: rename d3/d12/d17/d29/d58/c77 to d3/d12/d17/d29/ca0 0 2024-09-10T04:05:17.769 INFO:tasks.workunit.client.1.smithi070.stdout:7/412: dwrite d2/d3/dc/d5d/d41/f53 [0,4194304] 0 2024-09-10T04:05:17.769 INFO:tasks.workunit.client.0.smithi046.stdout:1/356: creat d3/dc/d34/f76 x:0 0 0 2024-09-10T04:05:17.771 INFO:tasks.workunit.client.1.smithi070.stdout:3/466: mknod db/d1c/d25/d28/c9b 0 2024-09-10T04:05:17.773 INFO:tasks.workunit.client.1.smithi070.stdout:6/541: fdatasync d0/d3/d18/d1b/d17/d28/f5e 0 2024-09-10T04:05:17.774 INFO:tasks.workunit.client.1.smithi070.stdout:6/542: fdatasync d0/d3/d18/d62/d70/f7c 0 2024-09-10T04:05:17.778 INFO:tasks.workunit.client.0.smithi046.stdout:0/377: fdatasync d1/dc/d25/d33/d3/dd/d55/d4d/f71 0 2024-09-10T04:05:17.781 INFO:tasks.workunit.client.0.smithi046.stdout:2/460: unlink d2/f3 0 2024-09-10T04:05:17.782 INFO:tasks.workunit.client.1.smithi070.stdout:5/464: dwrite f16 [0,4194304] 0 2024-09-10T04:05:17.783 INFO:tasks.workunit.client.1.smithi070.stdout:9/523: creat d9/d60/d8a/d3c/fa2 x:0 0 0 2024-09-10T04:05:17.784 INFO:tasks.workunit.client.0.smithi046.stdout:7/468: stat d5/d9/f87 0 2024-09-10T04:05:17.786 INFO:tasks.workunit.client.0.smithi046.stdout:9/462: mknod d4/d3c/d4d/d80/c8c 0 2024-09-10T04:05:17.790 INFO:tasks.workunit.client.1.smithi070.stdout:7/413: unlink d2/d3/dc/d5d/d41/f53 0 2024-09-10T04:05:17.790 INFO:tasks.workunit.client.1.smithi070.stdout:7/414: fsync d2/d7/db/f5c 0 2024-09-10T04:05:17.793 INFO:tasks.workunit.client.0.smithi046.stdout:8/435: rename d1/d29/d17/l47 to d1/d29/d17/d1f/d46/d6a/d82/l98 0 2024-09-10T04:05:17.793 INFO:tasks.workunit.client.0.smithi046.stdout:5/546: write d7/d11/d14/d2f/d64/fc6 [294616,52853] 0 2024-09-10T04:05:17.794 INFO:tasks.workunit.client.1.smithi070.stdout:3/467: creat db/d1c/d25/d28/d43/f9c x:0 0 0 2024-09-10T04:05:17.794 INFO:tasks.workunit.client.0.smithi046.stdout:9/463: dwrite d4/d7/f1e [0,4194304] 0 2024-09-10T04:05:17.795 INFO:tasks.workunit.client.1.smithi070.stdout:3/468: stat db/d1c/l88 0 2024-09-10T04:05:17.796 INFO:tasks.workunit.client.1.smithi070.stdout:7/415: dwrite d2/d66/f78 [0,4194304] 0 2024-09-10T04:05:17.797 INFO:tasks.workunit.client.0.smithi046.stdout:9/464: readlink d4/d7/l60 0 2024-09-10T04:05:17.797 INFO:tasks.workunit.client.0.smithi046.stdout:4/469: creat d3/d12/d17/d22/d30/fa1 x:0 0 0 2024-09-10T04:05:17.798 INFO:tasks.workunit.client.0.smithi046.stdout:5/547: dread - d7/d11/d3f/d53/d75/fcb zero size 2024-09-10T04:05:17.800 INFO:tasks.workunit.client.1.smithi070.stdout:6/543: fdatasync d0/d3/d18/d1b/f11 0 2024-09-10T04:05:17.813 INFO:tasks.workunit.client.0.smithi046.stdout:3/461: write f3 [1167081,55613] 0 2024-09-10T04:05:17.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 -- 172.21.15.46:0/1661101333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528102b70 msgr2=0x7f2528102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 --2- 172.21.15.46:0/1661101333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528102b70 0x7f2528102ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f251c0097e0 tx=0x7f251c02a3f0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 -- 172.21.15.46:0/1661101333 shutdown_connections 2024-09-10T04:05:17.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 --2- 172.21.15.46:0/1661101333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528102b70 0x7f2528102ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 --2- 172.21.15.46:0/1661101333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528101970 0x7f2528101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 -- 172.21.15.46:0/1661101333 >> 172.21.15.46:0/1661101333 conn(0x7f25280fd140 msgr2=0x7f25280ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:17.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 -- 172.21.15.46:0/1661101333 shutdown_connections 2024-09-10T04:05:17.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.813+0000 7f252cbd0640 1 -- 172.21.15.46:0/1661101333 wait complete. 2024-09-10T04:05:17.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.814+0000 7f252cbd0640 1 Processor -- start 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.814+0000 7f252cbd0640 1 -- start start 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252cbd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528101970 0x7f25281974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252cbd0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528102b70 0x7f2528197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252cbd0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f252810a790 con 0x7f2528102b70 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252cbd0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f252810a610 con 0x7f2528101970 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528101970 0x7f25281974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528101970 0x7f25281974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:38572/0 (socket says 172.21.15.46:38572) 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252659c640 1 -- 172.21.15.46:0/3405875452 learned_addr learned my addr 172.21.15.46:0/3405875452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:17.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f2525d9b640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528102b70 0x7f2528197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252659c640 1 -- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528102b70 msgr2=0x7f2528197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252659c640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528102b70 0x7f2528197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.815+0000 7f252659c640 1 -- 172.21.15.46:0/3405875452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2528198090 con 0x7f2528101970 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.816+0000 7f252659c640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528101970 0x7f25281974d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f251800e970 tx=0x7f251800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.816+0000 7f25177fe640 1 -- 172.21.15.46:0/3405875452 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f251800cac0 con 0x7f2528101970 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.816+0000 7f25177fe640 1 -- 172.21.15.46:0/3405875452 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f251800cc60 con 0x7f2528101970 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.816+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252819cad0 con 0x7f2528101970 2024-09-10T04:05:17.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.816+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f252819d060 con 0x7f2528101970 2024-09-10T04:05:17.819 INFO:tasks.workunit.client.0.smithi046.stdout:6/420: sync 2024-09-10T04:05:17.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.816+0000 7f25177fe640 1 -- 172.21.15.46:0/3405875452 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2518003d10 con 0x7f2528101970 2024-09-10T04:05:17.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.818+0000 7f25177fe640 1 -- 172.21.15.46:0/3405875452 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f2518018450 con 0x7f2528101970 2024-09-10T04:05:17.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.818+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25080052c0 con 0x7f2528101970 2024-09-10T04:05:17.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.819+0000 7f25177fe640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f251006dee0 0x7f25100703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:17.821 INFO:tasks.workunit.client.0.smithi046.stdout:0/378: dwrite d1/dc/d25/d33/d3/dd/d55/d4d/f71 [0,4194304] 0 2024-09-10T04:05:17.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.819+0000 7f25177fe640 1 -- 172.21.15.46:0/3405875452 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f251808f110 con 0x7f2528101970 2024-09-10T04:05:17.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.819+0000 7f2525d9b640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f251006dee0 0x7f25100703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.820+0000 7f2525d9b640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f251006dee0 0x7f25100703a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f251c0095f0 tx=0x7f251c0308b0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:17.826 INFO:tasks.workunit.client.1.smithi070.stdout:7/416: creat d2/d4e/f9b x:0 0 0 2024-09-10T04:05:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:17.827+0000 7f25177fe640 1 -- 172.21.15.46:0/3405875452 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f251805cc90 con 0x7f2528101970 2024-09-10T04:05:17.830 INFO:tasks.workunit.client.0.smithi046.stdout:4/470: creat d3/d43/d5f/d85/fa2 x:0 0 0 2024-09-10T04:05:17.832 INFO:tasks.workunit.client.1.smithi070.stdout:7/417: dwrite d2/d7/d9/f82 [0,4194304] 0 2024-09-10T04:05:17.832 INFO:tasks.workunit.client.1.smithi070.stdout:8/520: rename d4/d16/f6b to d4/d16/d17/d5c/d87/d8e/fb2 0 2024-09-10T04:05:17.833 INFO:tasks.workunit.client.1.smithi070.stdout:8/521: dread - d4/d16/d17/d1a/fa9 zero size 2024-09-10T04:05:17.835 INFO:tasks.workunit.client.1.smithi070.stdout:5/465: link d17/d19/f50 d17/d19/d6a/d54/d57/d63/f9a 0 2024-09-10T04:05:17.837 INFO:tasks.workunit.client.0.smithi046.stdout:2/461: getdents d2/d8/d65/d90 0 2024-09-10T04:05:17.838 INFO:tasks.workunit.client.0.smithi046.stdout:8/436: write d1/d29/d17/d1f/d46/f4b [607783,62125] 0 2024-09-10T04:05:17.841 INFO:tasks.workunit.client.1.smithi070.stdout:4/402: sync 2024-09-10T04:05:17.841 INFO:tasks.workunit.client.0.smithi046.stdout:3/462: rmdir d12 39 2024-09-10T04:05:17.842 INFO:tasks.workunit.client.1.smithi070.stdout:4/403: dread - de/d13/d18/d26/f2f zero size 2024-09-10T04:05:17.843 INFO:tasks.workunit.client.1.smithi070.stdout:3/469: link db/d1c/d53/f57 db/d1c/d3e/f9d 0 2024-09-10T04:05:17.845 INFO:tasks.workunit.client.1.smithi070.stdout:3/470: read db/f22 [1822422,95237] 0 2024-09-10T04:05:17.845 INFO:tasks.workunit.client.1.smithi070.stdout:7/418: mknod d2/d3/dc/d5d/d42/c9c 0 2024-09-10T04:05:17.847 INFO:tasks.workunit.client.1.smithi070.stdout:7/419: write d2/d7/d9/d18/f99 [724306,20267] 0 2024-09-10T04:05:17.847 INFO:tasks.workunit.client.1.smithi070.stdout:9/524: link d9/d59/d5d/f66 d9/d7a/d6f/d90/d89/fa3 0 2024-09-10T04:05:17.848 INFO:tasks.workunit.client.1.smithi070.stdout:9/525: write d9/f73 [631235,124261] 0 2024-09-10T04:05:17.849 INFO:tasks.workunit.client.0.smithi046.stdout:4/471: dread d3/d12/d17/d22/d30/f82 [0,4194304] 0 2024-09-10T04:05:17.850 INFO:tasks.workunit.client.1.smithi070.stdout:8/522: read d4/d16/f22 [2760902,37271] 0 2024-09-10T04:05:17.852 INFO:tasks.workunit.client.1.smithi070.stdout:4/404: mkdir de/d13/d18/d26/d8c 0 2024-09-10T04:05:17.852 INFO:tasks.workunit.client.1.smithi070.stdout:4/405: fsync de/d13/f3e 0 2024-09-10T04:05:17.857 INFO:tasks.workunit.client.0.smithi046.stdout:1/357: creat d3/f77 x:0 0 0 2024-09-10T04:05:17.858 INFO:tasks.workunit.client.1.smithi070.stdout:4/406: dwrite de/d13/d43/f76 [0,4194304] 0 2024-09-10T04:05:17.859 INFO:tasks.workunit.client.1.smithi070.stdout:4/407: fsync fb 0 2024-09-10T04:05:17.859 INFO:tasks.workunit.client.0.smithi046.stdout:2/462: symlink d2/d6/d23/d5d/d5e/laf 0 2024-09-10T04:05:17.861 INFO:tasks.workunit.client.0.smithi046.stdout:8/437: mkdir d1/d29/d17/d1f/d46/d6a/d82/d99 0 2024-09-10T04:05:17.862 INFO:tasks.workunit.client.0.smithi046.stdout:7/469: write d5/d6/db/d11/d1e/f3e [779912,5430] 0 2024-09-10T04:05:17.862 INFO:tasks.workunit.client.1.smithi070.stdout:3/471: mknod db/d83/c9e 0 2024-09-10T04:05:17.862 INFO:tasks.workunit.client.1.smithi070.stdout:8/523: dread d4/d16/d17/d5c/f6d [0,4194304] 0 2024-09-10T04:05:17.864 INFO:tasks.workunit.client.1.smithi070.stdout:5/466: mkdir d17/d24/d84/d9b 0 2024-09-10T04:05:17.864 INFO:tasks.workunit.client.1.smithi070.stdout:8/524: write d4/d16/d17/d1a/d60/d7a/f9d [1019051,18109] 0 2024-09-10T04:05:17.872 INFO:tasks.workunit.client.0.smithi046.stdout:5/548: dwrite d7/d11/f23 [0,4194304] 0 2024-09-10T04:05:17.874 INFO:tasks.workunit.client.0.smithi046.stdout:5/549: chown d7/d11/d3f/d53/l6c 1 1 2024-09-10T04:05:17.875 INFO:tasks.workunit.client.0.smithi046.stdout:9/465: creat d4/d3c/d4d/d5c/f8d x:0 0 0 2024-09-10T04:05:17.879 INFO:tasks.workunit.client.0.smithi046.stdout:5/550: readlink d7/d11/d14/l3e 0 2024-09-10T04:05:17.879 INFO:tasks.workunit.client.0.smithi046.stdout:2/463: dread d2/d8/f29 [0,4194304] 0 2024-09-10T04:05:17.880 INFO:tasks.workunit.client.1.smithi070.stdout:6/544: link d0/d3/d18/d1b/d17/d28/f6b d0/d3/d18/d1b/d17/d3f/fa9 0 2024-09-10T04:05:17.880 INFO:tasks.workunit.client.0.smithi046.stdout:4/472: creat d3/d12/d17/fa3 x:0 0 0 2024-09-10T04:05:17.883 INFO:tasks.workunit.client.0.smithi046.stdout:5/551: read - d7/d11/d14/fc3 zero size 2024-09-10T04:05:17.885 INFO:tasks.workunit.client.1.smithi070.stdout:6/545: dwrite d0/d3/d18/f52 [0,4194304] 0 2024-09-10T04:05:17.888 INFO:tasks.workunit.client.1.smithi070.stdout:4/408: creat de/d13/d43/f8d x:0 0 0 2024-09-10T04:05:17.889 INFO:tasks.workunit.client.1.smithi070.stdout:6/546: dread d0/d3/d18/d1b/d17/d53/f61 [0,4194304] 0 2024-09-10T04:05:17.892 INFO:tasks.workunit.client.0.smithi046.stdout:6/421: write da/fd [2936001,117729] 0 2024-09-10T04:05:17.893 INFO:tasks.workunit.client.0.smithi046.stdout:5/552: chown d7/d11/d3f/d53/f9b 110074 1 2024-09-10T04:05:17.895 INFO:tasks.workunit.client.1.smithi070.stdout:0/506: rename d8/d24/c8b to d8/d13/d33/ca5 0 2024-09-10T04:05:17.895 INFO:tasks.workunit.client.0.smithi046.stdout:9/466: dwrite d4/d7/f7e [0,4194304] 0 2024-09-10T04:05:17.896 INFO:tasks.workunit.client.0.smithi046.stdout:6/422: fsync da/db/d10/f17 0 2024-09-10T04:05:17.896 INFO:tasks.workunit.client.1.smithi070.stdout:1/386: sync 2024-09-10T04:05:17.896 INFO:tasks.workunit.client.1.smithi070.stdout:2/425: sync 2024-09-10T04:05:17.896 INFO:tasks.workunit.client.1.smithi070.stdout:0/507: chown d8/d24/d2d/d34/f5b 54 1 2024-09-10T04:05:17.897 INFO:tasks.workunit.client.1.smithi070.stdout:0/508: stat f5 0 2024-09-10T04:05:17.897 INFO:tasks.workunit.client.0.smithi046.stdout:7/470: dread d5/d6/d39/d5a/f79 [0,4194304] 0 2024-09-10T04:05:17.898 INFO:tasks.workunit.client.0.smithi046.stdout:9/467: fdatasync d4/d29/d65/f75 0 2024-09-10T04:05:17.899 INFO:tasks.workunit.client.0.smithi046.stdout:0/379: creat d1/dc/f7b x:0 0 0 2024-09-10T04:05:17.903 INFO:tasks.workunit.client.1.smithi070.stdout:5/467: symlink d17/d24/d84/l9c 0 2024-09-10T04:05:17.904 INFO:tasks.workunit.client.0.smithi046.stdout:0/380: readlink d1/dc/d25/d33/d3/d29/l59 0 2024-09-10T04:05:17.906 INFO:tasks.workunit.client.1.smithi070.stdout:3/472: dwrite db/d1c/d25/d28/d43/f8b [0,4194304] 0 2024-09-10T04:05:17.906 INFO:tasks.workunit.client.0.smithi046.stdout:0/381: readlink d1/dc/d49/l67 0 2024-09-10T04:05:17.907 INFO:tasks.workunit.client.0.smithi046.stdout:4/473: dread d3/d12/d17/d22/d51/f8f [0,4194304] 0 2024-09-10T04:05:17.907 INFO:tasks.workunit.client.1.smithi070.stdout:7/420: unlink d2/c64 0 2024-09-10T04:05:17.910 INFO:tasks.workunit.client.0.smithi046.stdout:7/471: dwrite d5/d6/d32/f74 [4194304,4194304] 0 2024-09-10T04:05:17.916 INFO:tasks.workunit.client.1.smithi070.stdout:4/409: creat de/d13/d18/d23/d39/f8e x:0 0 0 2024-09-10T04:05:17.917 INFO:tasks.workunit.client.1.smithi070.stdout:4/410: dread - de/d13/d18/d26/d46/f60 zero size 2024-09-10T04:05:17.919 INFO:tasks.workunit.client.1.smithi070.stdout:1/387: mknod d3/d8/d27/c7f 0 2024-09-10T04:05:17.919 INFO:tasks.workunit.client.1.smithi070.stdout:1/388: fsync d3/d10/f73 0 2024-09-10T04:05:17.920 INFO:tasks.workunit.client.1.smithi070.stdout:0/509: mknod d8/d13/d47/d7b/ca6 0 2024-09-10T04:05:17.922 INFO:tasks.workunit.client.1.smithi070.stdout:5/468: creat d17/d24/d2b/f9d x:0 0 0 2024-09-10T04:05:17.926 INFO:tasks.workunit.client.1.smithi070.stdout:7/421: creat d2/d3/dc/d79/f9d x:0 0 0 2024-09-10T04:05:17.928 INFO:tasks.workunit.client.1.smithi070.stdout:9/526: creat d9/d59/fa4 x:0 0 0 2024-09-10T04:05:17.929 INFO:tasks.workunit.client.1.smithi070.stdout:9/527: read d9/d60/d8a/f81 [4029791,123292] 0 2024-09-10T04:05:17.930 INFO:tasks.workunit.client.1.smithi070.stdout:5/469: dread d17/d24/d45/f4f [0,4194304] 0 2024-09-10T04:05:17.933 INFO:tasks.workunit.client.1.smithi070.stdout:2/426: symlink d15/d18/d29/l98 0 2024-09-10T04:05:17.936 INFO:tasks.workunit.client.0.smithi046.stdout:1/358: link d3/d27/d32/d1f/f31 d3/dc/d34/d51/f78 0 2024-09-10T04:05:17.936 INFO:tasks.workunit.client.0.smithi046.stdout:1/359: stat d3/d27/d32 0 2024-09-10T04:05:17.938 INFO:tasks.workunit.client.0.smithi046.stdout:6/423: truncate da/db/d10/d27/f29 1559065 0 2024-09-10T04:05:17.940 INFO:tasks.workunit.client.0.smithi046.stdout:9/468: rename d4/d44/f69 to d4/d7/d61/f8e 0 2024-09-10T04:05:17.941 INFO:tasks.workunit.client.1.smithi070.stdout:8/525: link d4/d16/d17/d1a/d60/f9c d4/d16/d17/d19/d28/d43/d52/fb3 0 2024-09-10T04:05:17.943 INFO:tasks.workunit.client.0.smithi046.stdout:0/382: unlink d1/dc/d25/d33/d1a/d41/f7a 0 2024-09-10T04:05:17.947 INFO:tasks.workunit.client.1.smithi070.stdout:7/422: rmdir d2/d7/d9 39 2024-09-10T04:05:17.948 INFO:tasks.workunit.client.0.smithi046.stdout:9/469: dwrite d4/f79 [0,4194304] 0 2024-09-10T04:05:17.948 INFO:tasks.workunit.client.1.smithi070.stdout:3/473: write db/d1c/d53/f57 [563224,125942] 0 2024-09-10T04:05:17.948 INFO:tasks.workunit.client.1.smithi070.stdout:3/474: read - db/d1c/d25/f9a zero size 2024-09-10T04:05:17.954 INFO:tasks.workunit.client.1.smithi070.stdout:9/528: creat d9/d7a/d74/fa5 x:0 0 0 2024-09-10T04:05:17.956 INFO:tasks.workunit.client.0.smithi046.stdout:9/470: dwrite d4/d44/f49 [0,4194304] 0 2024-09-10T04:05:17.959 INFO:tasks.workunit.client.0.smithi046.stdout:6/424: mknod da/db/d11/d2b/d2f/c89 0 2024-09-10T04:05:17.959 INFO:tasks.workunit.client.1.smithi070.stdout:6/547: truncate d0/d3/d18/f42 3656993 0 2024-09-10T04:05:17.968 INFO:tasks.workunit.client.1.smithi070.stdout:1/389: mknod d3/d8/d13/d3f/c80 0 2024-09-10T04:05:17.969 INFO:tasks.workunit.client.1.smithi070.stdout:0/510: write d8/d13/d1a/f23 [3644567,41843] 0 2024-09-10T04:05:17.970 INFO:tasks.workunit.client.1.smithi070.stdout:0/511: write d8/f92 [634207,67902] 0 2024-09-10T04:05:17.971 INFO:tasks.workunit.client.0.smithi046.stdout:0/383: rmdir d1/d13/d37/d75 39 2024-09-10T04:05:17.971 INFO:tasks.workunit.client.0.smithi046.stdout:4/474: getdents d3/d43/d7a 0 2024-09-10T04:05:17.972 INFO:tasks.workunit.client.1.smithi070.stdout:3/475: dread fa [0,4194304] 0 2024-09-10T04:05:17.973 INFO:tasks.workunit.client.0.smithi046.stdout:2/464: truncate d2/d6/d23/d5d/fae 976326 0 2024-09-10T04:05:17.974 INFO:tasks.workunit.client.1.smithi070.stdout:7/423: mknod d2/d4e/c9e 0 2024-09-10T04:05:17.980 INFO:tasks.workunit.client.0.smithi046.stdout:4/475: dread d3/d12/d17/d22/d51/d5c/f76 [0,4194304] 0 2024-09-10T04:05:17.981 INFO:tasks.workunit.client.1.smithi070.stdout:2/427: write d15/d2b/f4c [2326913,65351] 0 2024-09-10T04:05:17.981 INFO:tasks.workunit.client.0.smithi046.stdout:0/384: dwrite d1/dc/d25/d33/d4/f6f [4194304,4194304] 0 2024-09-10T04:05:17.989 INFO:tasks.workunit.client.0.smithi046.stdout:7/472: truncate d5/d6/db/dd/d4b/f52 17761 0 2024-09-10T04:05:17.990 INFO:tasks.workunit.client.1.smithi070.stdout:8/526: dread d4/d16/d17/f40 [0,4194304] 0 2024-09-10T04:05:17.993 INFO:tasks.workunit.client.1.smithi070.stdout:4/411: rename de/d13/d18/d23/c4b to de/c8f 0 2024-09-10T04:05:17.995 INFO:tasks.workunit.client.0.smithi046.stdout:4/476: symlink d3/d12/d17/d29/la4 0 2024-09-10T04:05:17.997 INFO:tasks.workunit.client.1.smithi070.stdout:6/548: write d0/d3/d1f/f37 [234812,129137] 0 2024-09-10T04:05:17.997 INFO:tasks.workunit.client.1.smithi070.stdout:3/476: write db/d1c/d3e/f50 [12003087,43185] 0 2024-09-10T04:05:17.998 INFO:tasks.workunit.client.1.smithi070.stdout:6/549: fdatasync d0/d3/d18/d1b/fa0 0 2024-09-10T04:05:17.998 INFO:tasks.workunit.client.1.smithi070.stdout:3/477: fdatasync db/d1c/d25/f86 0 2024-09-10T04:05:18.001 INFO:tasks.workunit.client.1.smithi070.stdout:7/424: fsync d2/d7/d9/d18/d23/f47 0 2024-09-10T04:05:18.001 INFO:tasks.workunit.client.0.smithi046.stdout:9/471: truncate d4/d7/f4a 1869454 0 2024-09-10T04:05:18.003 INFO:tasks.workunit.client.0.smithi046.stdout:6/425: link da/db/d11/d2b/d2f/f66 da/db/d10/d27/f8a 0 2024-09-10T04:05:18.004 INFO:tasks.workunit.client.1.smithi070.stdout:5/470: creat d17/f9e x:0 0 0 2024-09-10T04:05:18.005 INFO:tasks.workunit.client.0.smithi046.stdout:7/473: creat d5/d6/d55/f99 x:0 0 0 2024-09-10T04:05:18.006 INFO:tasks.workunit.client.1.smithi070.stdout:5/471: write d17/d24/d2b/f81 [160463,81087] 0 2024-09-10T04:05:18.006 INFO:tasks.workunit.client.1.smithi070.stdout:7/425: dwrite d2/d7/d9/d18/f99 [0,4194304] 0 2024-09-10T04:05:18.010 INFO:tasks.workunit.client.0.smithi046.stdout:1/360: getdents d3/dc/d34/d58 0 2024-09-10T04:05:18.013 INFO:tasks.workunit.client.0.smithi046.stdout:9/472: symlink d4/d66/d7c/l8f 0 2024-09-10T04:05:18.019 INFO:tasks.workunit.client.1.smithi070.stdout:4/412: dread de/d13/f38 [0,4194304] 0 2024-09-10T04:05:18.019 INFO:tasks.workunit.client.1.smithi070.stdout:3/478: creat db/d1c/d89/f9f x:0 0 0 2024-09-10T04:05:18.020 INFO:tasks.workunit.client.1.smithi070.stdout:8/527: write d4/d16/d17/d1a/f2a [2533495,32470] 0 2024-09-10T04:05:18.022 INFO:tasks.workunit.client.1.smithi070.stdout:2/428: mkdir d15/d18/d99 0 2024-09-10T04:05:18.024 INFO:tasks.workunit.client.1.smithi070.stdout:5/472: unlink d17/d19/d35/c69 0 2024-09-10T04:05:18.025 INFO:tasks.workunit.client.0.smithi046.stdout:4/477: dwrite d3/d12/d14/f2b [0,4194304] 0 2024-09-10T04:05:18.026 INFO:tasks.workunit.client.1.smithi070.stdout:0/512: write d8/d24/d2d/f3a [2586452,81711] 0 2024-09-10T04:05:18.028 INFO:tasks.workunit.client.0.smithi046.stdout:4/478: chown d3/d12/d17/d22/d51/d5c/d6c 987322 1 2024-09-10T04:05:18.028 INFO:tasks.workunit.client.0.smithi046.stdout:7/474: write d5/d9/d43/f66 [4338583,100104] 0 2024-09-10T04:05:18.029 INFO:tasks.workunit.client.0.smithi046.stdout:6/426: mknod da/db/c8b 0 2024-09-10T04:05:18.030 INFO:tasks.workunit.client.0.smithi046.stdout:1/361: symlink d3/d27/d5d/l79 0 2024-09-10T04:05:18.038 INFO:tasks.workunit.client.0.smithi046.stdout:9/473: dwrite d4/d29/f3d [4194304,4194304] 0 2024-09-10T04:05:18.039 INFO:tasks.workunit.client.1.smithi070.stdout:5/473: dwrite f10 [0,4194304] 0 2024-09-10T04:05:18.039 INFO:tasks.workunit.client.1.smithi070.stdout:9/529: rename d9/d59/d2a/l71 to d9/d1c/la6 0 2024-09-10T04:05:18.039 INFO:tasks.workunit.client.0.smithi046.stdout:7/475: rmdir d5/d6/db/d11/d1f 39 2024-09-10T04:05:18.043 INFO:tasks.workunit.client.0.smithi046.stdout:7/476: chown d5/d6/db/dd/d24/d76/f93 3972582 1 2024-09-10T04:05:18.043 INFO:tasks.workunit.client.1.smithi070.stdout:4/413: creat de/d13/d18/d26/d8c/f90 x:0 0 0 2024-09-10T04:05:18.043 INFO:tasks.workunit.client.0.smithi046.stdout:9/474: read d4/d29/d32/f43 [3055764,35023] 0 2024-09-10T04:05:18.044 INFO:tasks.workunit.client.0.smithi046.stdout:1/362: symlink d3/l7a 0 2024-09-10T04:05:18.044 INFO:tasks.workunit.client.1.smithi070.stdout:4/414: truncate de/d13/d18/f4d 748677 0 2024-09-10T04:05:18.046 INFO:tasks.workunit.client.0.smithi046.stdout:4/479: dwrite d3/d12/d17/d22/f4e [0,4194304] 0 2024-09-10T04:05:18.047 INFO:tasks.workunit.client.0.smithi046.stdout:7/477: write d5/d6/d39/d5f/f96 [24633,113865] 0 2024-09-10T04:05:18.055 INFO:tasks.workunit.client.0.smithi046.stdout:1/363: chown d3/c1b 486112561 1 2024-09-10T04:05:18.056 INFO:tasks.workunit.client.1.smithi070.stdout:4/415: dread de/d13/d43/f6b [0,4194304] 0 2024-09-10T04:05:18.057 INFO:tasks.workunit.client.0.smithi046.stdout:7/478: dwrite d5/d6/d32/f74 [0,4194304] 0 2024-09-10T04:05:18.058 INFO:tasks.workunit.client.1.smithi070.stdout:4/416: write de/d13/d18/d23/d39/d48/f7e [1003677,7044] 0 2024-09-10T04:05:18.064 INFO:tasks.workunit.client.1.smithi070.stdout:1/390: rename d3/d8/d27/d7a/d7d to d3/d8/d13/d3f/d6d/d81 0 2024-09-10T04:05:18.066 INFO:tasks.workunit.client.1.smithi070.stdout:8/528: creat d4/d16/d7d/fb4 x:0 0 0 2024-09-10T04:05:18.068 INFO:tasks.workunit.client.0.smithi046.stdout:9/475: symlink d4/d18/d89/l90 0 2024-09-10T04:05:18.070 INFO:tasks.workunit.client.1.smithi070.stdout:2/429: dwrite d15/d18/f24 [0,4194304] 0 2024-09-10T04:05:18.070 INFO:tasks.workunit.client.0.smithi046.stdout:1/364: write d3/d27/d32/f69 [1105961,53459] 0 2024-09-10T04:05:18.072 INFO:tasks.workunit.client.1.smithi070.stdout:3/479: getdents db 0 2024-09-10T04:05:18.074 INFO:tasks.workunit.client.1.smithi070.stdout:0/513: write d8/f27 [4310193,62733] 0 2024-09-10T04:05:18.074 INFO:tasks.workunit.client.1.smithi070.stdout:4/417: rmdir de/d13/d5b 39 2024-09-10T04:05:18.074 INFO:tasks.workunit.client.0.smithi046.stdout:6/427: dwrite da/db/d11/d2b/f2c [0,4194304] 0 2024-09-10T04:05:18.075 INFO:tasks.workunit.client.1.smithi070.stdout:9/530: write d9/f25 [1110242,61111] 0 2024-09-10T04:05:18.077 INFO:tasks.workunit.client.0.smithi046.stdout:9/476: dread d4/f79 [0,4194304] 0 2024-09-10T04:05:18.078 INFO:tasks.workunit.client.1.smithi070.stdout:5/474: unlink d17/d19/d35/c85 0 2024-09-10T04:05:18.078 INFO:tasks.workunit.client.1.smithi070.stdout:9/531: dread d9/d1c/d34/f76 [4194304,4194304] 0 2024-09-10T04:05:18.079 INFO:tasks.workunit.client.0.smithi046.stdout:6/428: write da/db/d11/f32 [2389630,9090] 0 2024-09-10T04:05:18.083 INFO:tasks.workunit.client.0.smithi046.stdout:4/480: rename d3/d12/d17/l8a to d3/d12/la5 0 2024-09-10T04:05:18.085 INFO:tasks.workunit.client.0.smithi046.stdout:7/479: mknod d5/d7f/c9a 0 2024-09-10T04:05:18.087 INFO:tasks.workunit.client.0.smithi046.stdout:7/480: write d5/d6/db/f38 [3337571,74751] 0 2024-09-10T04:05:18.088 INFO:tasks.workunit.client.1.smithi070.stdout:8/529: dread - d4/d16/f34 zero size 2024-09-10T04:05:18.091 INFO:tasks.workunit.client.0.smithi046.stdout:7/481: dread d5/d6/d39/d5f/f96 [0,4194304] 0 2024-09-10T04:05:18.091 INFO:tasks.workunit.client.0.smithi046.stdout:6/429: creat da/db/d11/f8c x:0 0 0 2024-09-10T04:05:18.091 INFO:tasks.workunit.client.1.smithi070.stdout:0/514: fsync d8/d13/d33/d4f/f6e 0 2024-09-10T04:05:18.094 INFO:tasks.workunit.client.0.smithi046.stdout:9/477: unlink d4/f14 0 2024-09-10T04:05:18.095 INFO:tasks.workunit.client.1.smithi070.stdout:1/391: dread d3/d8/d12/d2f/d32/f41 [0,4194304] 0 2024-09-10T04:05:18.096 INFO:tasks.workunit.client.1.smithi070.stdout:4/418: symlink de/d13/d18/d5c/d7f/l91 0 2024-09-10T04:05:18.097 INFO:tasks.workunit.client.0.smithi046.stdout:4/481: creat d3/d12/d14/fa6 x:0 0 0 2024-09-10T04:05:18.103 INFO:tasks.workunit.client.0.smithi046.stdout:1/365: write d3/f26 [728345,466] 0 2024-09-10T04:05:18.103 INFO:tasks.workunit.client.1.smithi070.stdout:2/430: creat d15/d2b/d94/f9a x:0 0 0 2024-09-10T04:05:18.105 INFO:tasks.workunit.client.1.smithi070.stdout:2/431: truncate d15/d18/d29/d64/f81 275942 0 2024-09-10T04:05:18.107 INFO:tasks.workunit.client.1.smithi070.stdout:3/480: mknod db/d1c/d25/d28/ca0 0 2024-09-10T04:05:18.110 INFO:tasks.workunit.client.0.smithi046.stdout:7/482: rename d5/d6/db/dd/d24/d76/f7c to d5/d6/d55/d7d/f9b 0 2024-09-10T04:05:18.111 INFO:tasks.workunit.client.1.smithi070.stdout:3/481: dwrite db/f12 [0,4194304] 0 2024-09-10T04:05:18.112 INFO:tasks.workunit.client.1.smithi070.stdout:3/482: write db/f3c [736340,73599] 0 2024-09-10T04:05:18.117 INFO:tasks.workunit.client.0.smithi046.stdout:7/483: dwrite d5/d9/d43/f66 [0,4194304] 0 2024-09-10T04:05:18.118 INFO:tasks.workunit.client.1.smithi070.stdout:8/530: dwrite d4/d16/d17/d19/d28/d43/d67/d97/f98 [0,4194304] 0 2024-09-10T04:05:18.121 INFO:tasks.workunit.client.1.smithi070.stdout:9/532: dwrite d9/d60/d8a/d1b/f2d [0,4194304] 0 2024-09-10T04:05:18.124 INFO:tasks.workunit.client.1.smithi070.stdout:5/475: mknod d17/d19/d67/c9f 0 2024-09-10T04:05:18.127 INFO:tasks.workunit.client.0.smithi046.stdout:4/482: rename d3/d12/d17/d22/d51/l66 to d3/d43/d5f/d85/la7 0 2024-09-10T04:05:18.130 INFO:tasks.workunit.client.0.smithi046.stdout:6/430: dwrite da/db/d11/d13/f57 [4194304,4194304] 0 2024-09-10T04:05:18.134 INFO:tasks.workunit.client.0.smithi046.stdout:7/484: dwrite d5/d6/db/f38 [4194304,4194304] 0 2024-09-10T04:05:18.135 INFO:tasks.workunit.client.1.smithi070.stdout:2/432: rename d15/d18/d42 to d15/d18/d29/d2a/d2e/d32/d9b 0 2024-09-10T04:05:18.140 INFO:tasks.workunit.client.0.smithi046.stdout:4/483: unlink d3/d12/d17/f9c 0 2024-09-10T04:05:18.141 INFO:tasks.workunit.client.1.smithi070.stdout:2/433: dread d15/d18/d56/d7c/d55/f65 [0,4194304] 0 2024-09-10T04:05:18.142 INFO:tasks.workunit.client.0.smithi046.stdout:4/484: write d3/d12/d17/f5e [655340,77041] 0 2024-09-10T04:05:18.142 INFO:tasks.workunit.client.0.smithi046.stdout:1/366: getdents d3/d27/d5d 0 2024-09-10T04:05:18.144 INFO:tasks.workunit.client.0.smithi046.stdout:4/485: dread - d3/d12/d17/d29/d58/f90 zero size 2024-09-10T04:05:18.145 INFO:tasks.workunit.client.0.smithi046.stdout:1/367: write d3/d27/f42 [4653609,79710] 0 2024-09-10T04:05:18.146 INFO:tasks.workunit.client.1.smithi070.stdout:0/515: getdents d8/d24/d2d/d34/d9b 0 2024-09-10T04:05:18.146 INFO:tasks.workunit.client.0.smithi046.stdout:4/486: write d3/d12/f9a [250904,50894] 0 2024-09-10T04:05:18.147 INFO:tasks.workunit.client.0.smithi046.stdout:6/431: fdatasync da/db/d11/d2b/d2f/f37 0 2024-09-10T04:05:18.148 INFO:tasks.workunit.client.1.smithi070.stdout:6/550: sync 2024-09-10T04:05:18.151 INFO:tasks.workunit.client.1.smithi070.stdout:1/392: stat d3/d8/d12/d2f/d32/d4b/d54/c76 0 2024-09-10T04:05:18.151 INFO:tasks.workunit.client.1.smithi070.stdout:3/483: dread db/d1c/d25/d28/d43/f51 [0,4194304] 0 2024-09-10T04:05:18.151 INFO:tasks.workunit.client.1.smithi070.stdout:0/516: write d8/d24/d2d/d34/f5b [1564581,126541] 0 2024-09-10T04:05:18.152 INFO:tasks.workunit.client.1.smithi070.stdout:2/434: dwrite d15/d18/d29/d2a/d2e/d32/d36/f5b [0,4194304] 0 2024-09-10T04:05:18.156 INFO:tasks.workunit.client.1.smithi070.stdout:4/419: symlink de/d13/l92 0 2024-09-10T04:05:18.158 INFO:tasks.workunit.client.1.smithi070.stdout:3/484: dwrite db/d1c/d25/d28/f41 [0,4194304] 0 2024-09-10T04:05:18.160 INFO:tasks.workunit.client.1.smithi070.stdout:5/476: symlink d17/d24/d84/la0 0 2024-09-10T04:05:18.163 INFO:tasks.workunit.client.0.smithi046.stdout:4/487: mkdir d3/d43/d5f/d85/da8 0 2024-09-10T04:05:18.164 INFO:tasks.workunit.client.0.smithi046.stdout:4/488: write d3/fa [4273967,125304] 0 2024-09-10T04:05:18.165 INFO:tasks.workunit.client.0.smithi046.stdout:6/432: symlink da/db/d11/d13/d1e/d23/l8d 0 2024-09-10T04:05:18.174 INFO:tasks.workunit.client.0.smithi046.stdout:4/489: rmdir d3/d12/d14 39 2024-09-10T04:05:18.175 INFO:tasks.workunit.client.0.smithi046.stdout:4/490: truncate d3/d12/f9a 709794 0 2024-09-10T04:05:18.176 INFO:tasks.workunit.client.0.smithi046.stdout:6/433: mknod da/db/d11/d13/d1e/d23/d54/c8e 0 2024-09-10T04:05:18.183 INFO:tasks.workunit.client.0.smithi046.stdout:4/491: symlink d3/d12/la9 0 2024-09-10T04:05:18.183 INFO:tasks.workunit.client.0.smithi046.stdout:6/434: dwrite da/db/d11/d2b/f2c [0,4194304] 0 2024-09-10T04:05:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.183+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2508000bf0 con 0x7f251006dee0 2024-09-10T04:05:18.187 INFO:tasks.workunit.client.0.smithi046.stdout:4/492: creat d3/d12/d17/d22/d30/faa x:0 0 0 2024-09-10T04:05:18.190 INFO:tasks.workunit.client.0.smithi046.stdout:6/435: mkdir da/db/d10/d27/d8f 0 2024-09-10T04:05:18.192 INFO:tasks.workunit.client.0.smithi046.stdout:4/493: mkdir d3/dab 0 2024-09-10T04:05:18.194 INFO:tasks.workunit.client.0.smithi046.stdout:1/368: dwrite d3/d27/d32/d1f/f31 [0,4194304] 0 2024-09-10T04:05:18.196 INFO:tasks.workunit.client.0.smithi046.stdout:1/369: chown d3/dc/f6e 959771 1 2024-09-10T04:05:18.198 INFO:tasks.workunit.client.0.smithi046.stdout:7/485: truncate d5/d6/db/dd/f2f 3653081 0 2024-09-10T04:05:18.200 INFO:tasks.workunit.client.0.smithi046.stdout:6/436: mkdir da/db/d11/d13/d1e/d23/d54/d90 0 2024-09-10T04:05:18.202 INFO:tasks.workunit.client.0.smithi046.stdout:4/494: dwrite d3/fa [0,4194304] 0 2024-09-10T04:05:18.204 INFO:tasks.workunit.client.0.smithi046.stdout:1/370: mknod d3/d27/d32/d28/c7b 0 2024-09-10T04:05:18.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.203+0000 7f25177fe640 1 -- 172.21.15.46:0/3405875452 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3768 (secure 0 0 0) 0x7f2508000bf0 con 0x7f251006dee0 2024-09-10T04:05:18.205 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:05:18.205 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (6m) 2m ago 8m 26.6M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:05:18.205 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (8m) 2m ago 8m 8254k - 19.1.1-298-gb53b6d3e 54467abc55f8 3888d245a02a 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (7m) 2m ago 7m 5268k - 19.1.1-298-gb53b6d3e 54467abc55f8 54f37f2cb1f7 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (8m) 2m ago 8m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (7m) 2m ago 7m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (6m) 2m ago 8m 80.0M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (2m) 2m ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (2m) 2m ago 2m 17.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (2m) 2m ago 2m 14.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (2m) 2m ago 2m 13.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:9283,8765,8443 running (9m) 2m ago 9m 498M - 19.1.1-298-gb53b6d3e 54467abc55f8 a4fa65bf88c5 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (7m) 2m ago 7m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 06c733bd8286 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (9m) 2m ago 9m 47.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (7m) 2m ago 7m 34.7M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (8m) 2m ago 8m 15.3M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:05:18.206 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 2m ago 7m 17.0M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:05:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (6m) 2m ago 6m 57.0M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:05:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (5m) 2m ago 5m 54.7M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:05:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (5m) 2m ago 5m 46.4M 3948M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:05:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (4m) 2m ago 4m 48.0M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:05:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (4m) 2m ago 4m 42.1M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:05:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (3m) 2m ago 3m 49.3M 4970M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:05:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (6m) 2m ago 7m 61.3M - 2.43.0 a07b618ecd1d 15eb5a503403 2024-09-10T04:05:18.207 INFO:tasks.workunit.client.1.smithi070.stdout:1/393: unlink d3/f6a 0 2024-09-10T04:05:18.208 INFO:tasks.workunit.client.0.smithi046.stdout:1/371: read d3/dc/f23 [243854,105737] 0 2024-09-10T04:05:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f251006dee0 msgr2=0x7f25100703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f251006dee0 0x7f25100703a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f251c0095f0 tx=0x7f251c0308b0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528101970 msgr2=0x7f25281974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528101970 0x7f25281974d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f251800e970 tx=0x7f251800ee30 comp rx=0 tx=0).stop 2024-09-10T04:05:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 shutdown_connections 2024-09-10T04:05:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2528102b70 0x7f2528197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f251006dee0 0x7f25100703a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 --2- 172.21.15.46:0/3405875452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2528101970 0x7f25281974d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 >> 172.21.15.46:0/3405875452 conn(0x7f25280fd140 msgr2=0x7f2528106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 shutdown_connections 2024-09-10T04:05:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.207+0000 7f252cbd0640 1 -- 172.21.15.46:0/3405875452 wait complete. 2024-09-10T04:05:18.210 INFO:tasks.workunit.client.1.smithi070.stdout:9/533: symlink d9/la7 0 2024-09-10T04:05:18.211 INFO:tasks.workunit.client.1.smithi070.stdout:6/551: dread d0/d3/d18/d1b/d17/d28/f6b [0,4194304] 0 2024-09-10T04:05:18.212 INFO:tasks.workunit.client.0.smithi046.stdout:7/486: write d5/d6/db/dd/d24/d76/f93 [276449,86231] 0 2024-09-10T04:05:18.213 INFO:tasks.workunit.client.1.smithi070.stdout:6/552: write d0/d3/d18/d1b/d17/fa5 [4338793,98153] 0 2024-09-10T04:05:18.214 INFO:tasks.workunit.client.1.smithi070.stdout:2/435: creat d15/d18/d56/f9c x:0 0 0 2024-09-10T04:05:18.215 INFO:tasks.workunit.client.1.smithi070.stdout:2/436: stat d15/d18/d29/d64 0 2024-09-10T04:05:18.217 INFO:tasks.workunit.client.0.smithi046.stdout:6/437: truncate da/db/f16 341596 0 2024-09-10T04:05:18.222 INFO:tasks.workunit.client.1.smithi070.stdout:4/420: fsync de/d13/d18/d26/f2f 0 2024-09-10T04:05:18.223 INFO:tasks.workunit.client.0.smithi046.stdout:1/372: mknod d3/d62/c7c 0 2024-09-10T04:05:18.224 INFO:tasks.workunit.client.1.smithi070.stdout:3/485: symlink db/d83/la1 0 2024-09-10T04:05:18.227 INFO:tasks.workunit.client.0.smithi046.stdout:7/487: mkdir d5/d6/db/dd/d24/d76/d9c 0 2024-09-10T04:05:18.227 INFO:tasks.workunit.client.1.smithi070.stdout:5/477: mknod d17/d24/d45/ca1 0 2024-09-10T04:05:18.228 INFO:tasks.workunit.client.1.smithi070.stdout:5/478: readlink d17/d24/d84/la0 0 2024-09-10T04:05:18.228 INFO:tasks.workunit.client.1.smithi070.stdout:9/534: dread d9/d60/d8a/f20 [4194304,4194304] 0 2024-09-10T04:05:18.228 INFO:tasks.workunit.client.0.smithi046.stdout:6/438: symlink da/db/d11/d2b/d2f/l91 0 2024-09-10T04:05:18.229 INFO:tasks.workunit.client.1.smithi070.stdout:9/535: chown d9/d1c/d34/d8b/f9e 450782 1 2024-09-10T04:05:18.230 INFO:tasks.workunit.client.0.smithi046.stdout:4/495: mknod d3/d12/d17/d22/d51/d5c/d6c/cac 0 2024-09-10T04:05:18.235 INFO:tasks.workunit.client.1.smithi070.stdout:1/394: mkdir d3/d8/d12/d82 0 2024-09-10T04:05:18.237 INFO:tasks.workunit.client.0.smithi046.stdout:7/488: rename d5/d6/db/dd/d24/d76/f93 to d5/d6/d55/d89/f9d 0 2024-09-10T04:05:18.237 INFO:tasks.workunit.client.1.smithi070.stdout:8/531: dwrite d4/d16/d17/d1a/d60/f9c [0,4194304] 0 2024-09-10T04:05:18.239 INFO:tasks.workunit.client.1.smithi070.stdout:7/426: sync 2024-09-10T04:05:18.241 INFO:tasks.workunit.client.0.smithi046.stdout:6/439: creat da/db/d10/d27/f92 x:0 0 0 2024-09-10T04:05:18.242 INFO:tasks.workunit.client.0.smithi046.stdout:6/440: readlink da/db/d11/d2b/d55/l67 0 2024-09-10T04:05:18.248 INFO:tasks.workunit.client.1.smithi070.stdout:6/553: write d0/d3/f51 [2996951,99641] 0 2024-09-10T04:05:18.250 INFO:tasks.workunit.client.1.smithi070.stdout:0/517: creat d8/d13/d1a/d8f/d95/fa7 x:0 0 0 2024-09-10T04:05:18.254 INFO:tasks.workunit.client.1.smithi070.stdout:4/421: mkdir de/d13/d18/d26/d8c/d93 0 2024-09-10T04:05:18.258 INFO:tasks.workunit.client.1.smithi070.stdout:3/486: unlink db/d79/l7a 0 2024-09-10T04:05:18.260 INFO:tasks.workunit.client.1.smithi070.stdout:5/479: unlink d17/d19/d6a/f53 0 2024-09-10T04:05:18.265 INFO:tasks.workunit.client.0.smithi046.stdout:7/489: symlink d5/d6/db/d11/d1e/l9e 0 2024-09-10T04:05:18.266 INFO:tasks.workunit.client.0.smithi046.stdout:6/441: getdents da/db/d10/d27/d8f 0 2024-09-10T04:05:18.267 INFO:tasks.workunit.client.0.smithi046.stdout:6/442: stat da/db/d11/d2b/d2f 0 2024-09-10T04:05:18.268 INFO:tasks.workunit.client.0.smithi046.stdout:7/490: dread d5/d6/d55/d89/f9d [0,4194304] 0 2024-09-10T04:05:18.269 INFO:tasks.workunit.client.0.smithi046.stdout:1/373: link d3/d27/f20 d3/d27/f7d 0 2024-09-10T04:05:18.270 INFO:tasks.workunit.client.0.smithi046.stdout:1/374: chown d3/d62/c7c 9 1 2024-09-10T04:05:18.272 INFO:tasks.workunit.client.0.smithi046.stdout:4/496: truncate d3/d12/d17/d22/f4e 971495 0 2024-09-10T04:05:18.273 INFO:tasks.workunit.client.1.smithi070.stdout:7/427: unlink d2/d3/dc/d79/f80 0 2024-09-10T04:05:18.273 INFO:tasks.workunit.client.1.smithi070.stdout:7/428: stat d2 0 2024-09-10T04:05:18.274 INFO:tasks.workunit.client.0.smithi046.stdout:6/443: rmdir da/db/d18 39 2024-09-10T04:05:18.275 INFO:tasks.workunit.client.0.smithi046.stdout:6/444: dread - da/db/d10/d27/f88 zero size 2024-09-10T04:05:18.276 INFO:tasks.workunit.client.0.smithi046.stdout:6/445: write da/db/d11/f8c [426475,60155] 0 2024-09-10T04:05:18.278 INFO:tasks.workunit.client.0.smithi046.stdout:7/491: unlink d5/d6/db/dd/d4b/l54 0 2024-09-10T04:05:18.281 INFO:tasks.workunit.client.0.smithi046.stdout:1/375: creat d3/d27/d2a/f7e x:0 0 0 2024-09-10T04:05:18.282 INFO:tasks.workunit.client.1.smithi070.stdout:6/554: symlink d0/d3/d1f/d55/laa 0 2024-09-10T04:05:18.283 INFO:tasks.workunit.client.1.smithi070.stdout:6/555: readlink d0/d3/d1f/d81/l88 0 2024-09-10T04:05:18.285 INFO:tasks.workunit.client.0.smithi046.stdout:4/497: dread d3/d12/d17/f1b [0,4194304] 0 2024-09-10T04:05:18.286 INFO:tasks.workunit.client.1.smithi070.stdout:0/518: read d8/d13/d1a/f32 [1552762,100732] 0 2024-09-10T04:05:18.286 INFO:tasks.workunit.client.0.smithi046.stdout:4/498: write d3/d12/d17/f6e [795618,130042] 0 2024-09-10T04:05:18.287 INFO:tasks.workunit.client.1.smithi070.stdout:3/487: creat db/d1c/d53/fa2 x:0 0 0 2024-09-10T04:05:18.292 INFO:tasks.workunit.client.1.smithi070.stdout:9/536: mkdir d9/d60/d8a/d3c/d92/da8 0 2024-09-10T04:05:18.292 INFO:tasks.workunit.client.1.smithi070.stdout:0/519: dread d8/d13/d33/f65 [0,4194304] 0 2024-09-10T04:05:18.292 INFO:tasks.workunit.client.0.smithi046.stdout:1/376: dread d3/d27/f7d [0,4194304] 0 2024-09-10T04:05:18.293 INFO:tasks.workunit.client.1.smithi070.stdout:0/520: chown d8/d24/d2d/d5f/f60 996189 1 2024-09-10T04:05:18.293 INFO:tasks.workunit.client.0.smithi046.stdout:7/492: mkdir d5/d6/db/d11/d9f 0 2024-09-10T04:05:18.295 INFO:tasks.workunit.client.1.smithi070.stdout:1/395: mkdir d3/d8/d13/d67/d83 0 2024-09-10T04:05:18.301 INFO:tasks.workunit.client.0.smithi046.stdout:1/377: symlink d3/dc/l7f 0 2024-09-10T04:05:18.301 INFO:tasks.workunit.client.1.smithi070.stdout:7/429: mkdir d2/d7/db/d20/d9f 0 2024-09-10T04:05:18.305 INFO:tasks.workunit.client.1.smithi070.stdout:2/437: rename d15/d18/d29/d2a/d2e/l54 to d15/d18/d29/l9d 0 2024-09-10T04:05:18.305 INFO:tasks.workunit.client.0.smithi046.stdout:4/499: link d3/d12/la9 d3/lad 0 2024-09-10T04:05:18.306 INFO:tasks.workunit.client.1.smithi070.stdout:2/438: chown d15/d18/d29/d64/f81 1186753 1 2024-09-10T04:05:18.306 INFO:tasks.workunit.client.1.smithi070.stdout:2/439: chown d15 180246 1 2024-09-10T04:05:18.307 INFO:tasks.workunit.client.0.smithi046.stdout:6/446: dwrite da/db/d10/d27/f8a [0,4194304] 0 2024-09-10T04:05:18.308 INFO:tasks.workunit.client.1.smithi070.stdout:6/556: unlink d0/d3/d18/d1b/d17/d28/l58 0 2024-09-10T04:05:18.308 INFO:tasks.workunit.client.1.smithi070.stdout:6/557: fsync d0/d3/d18/f35 0 2024-09-10T04:05:18.311 INFO:tasks.workunit.client.0.smithi046.stdout:1/378: mknod d3/d27/d32/d28/c80 0 2024-09-10T04:05:18.314 INFO:tasks.workunit.client.0.smithi046.stdout:7/493: truncate d5/f60 911129 0 2024-09-10T04:05:18.317 INFO:tasks.workunit.client.0.smithi046.stdout:3/463: sync 2024-09-10T04:05:18.317 INFO:tasks.workunit.client.1.smithi070.stdout:5/480: mknod d17/d19/d6a/d54/ca2 0 2024-09-10T04:05:18.318 INFO:tasks.workunit.client.1.smithi070.stdout:5/481: dread - d17/d19/d6a/d54/d57/d63/f9a zero size 2024-09-10T04:05:18.319 INFO:tasks.workunit.client.0.smithi046.stdout:1/379: dwrite d3/d27/d5d/d60/f65 [0,4194304] 0 2024-09-10T04:05:18.323 INFO:tasks.workunit.client.1.smithi070.stdout:9/537: fsync d9/d59/f18 0 2024-09-10T04:05:18.325 INFO:tasks.workunit.client.0.smithi046.stdout:6/447: creat da/db/d11/f93 x:0 0 0 2024-09-10T04:05:18.326 INFO:tasks.workunit.client.0.smithi046.stdout:6/448: chown da/db/d11/d2b/d5a 5364104 1 2024-09-10T04:05:18.329 INFO:tasks.workunit.client.0.smithi046.stdout:4/500: dread d3/d12/d17/d22/d30/f3a [0,4194304] 0 2024-09-10T04:05:18.329 INFO:tasks.workunit.client.1.smithi070.stdout:3/488: dread f7 [0,4194304] 0 2024-09-10T04:05:18.330 INFO:tasks.workunit.client.1.smithi070.stdout:8/532: dwrite d4/d16/d17/d19/d62/f71 [0,4194304] 0 2024-09-10T04:05:18.330 INFO:tasks.workunit.client.1.smithi070.stdout:7/430: creat d2/d3/dc/d5d/d42/fa0 x:0 0 0 2024-09-10T04:05:18.330 INFO:tasks.workunit.client.0.smithi046.stdout:4/501: chown d3/f3c 37854 1 2024-09-10T04:05:18.334 INFO:tasks.workunit.client.1.smithi070.stdout:7/431: write d2/d3/f3b [4108196,124365] 0 2024-09-10T04:05:18.334 INFO:tasks.workunit.client.1.smithi070.stdout:2/440: stat d15/l17 0 2024-09-10T04:05:18.334 INFO:tasks.workunit.client.1.smithi070.stdout:6/558: fdatasync d0/d3/d18/d2a/f4f 0 2024-09-10T04:05:18.336 INFO:tasks.workunit.client.1.smithi070.stdout:9/538: dwrite d9/d1c/f87 [0,4194304] 0 2024-09-10T04:05:18.340 INFO:tasks.workunit.client.0.smithi046.stdout:6/449: mknod da/db/d10/d51/c94 0 2024-09-10T04:05:18.343 INFO:tasks.workunit.client.1.smithi070.stdout:9/539: dread d9/f22 [0,4194304] 0 2024-09-10T04:05:18.351 INFO:tasks.workunit.client.0.smithi046.stdout:4/502: dread d3/d12/d17/d22/f4c [0,4194304] 0 2024-09-10T04:05:18.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.349+0000 7fefe0d99640 1 -- 172.21.15.46:0/526101351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc101a60 msgr2=0x7fefdc101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.349+0000 7fefe0d99640 1 --2- 172.21.15.46:0/526101351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc101a60 0x7fefdc101ee0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009880 tx=0x7fefc402a4d0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.349+0000 7fefe0d99640 1 -- 172.21.15.46:0/526101351 shutdown_connections 2024-09-10T04:05:18.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.349+0000 7fefe0d99640 1 --2- 172.21.15.46:0/526101351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc101a60 0x7fefdc101ee0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.349+0000 7fefe0d99640 1 --2- 172.21.15.46:0/526101351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc100860 0x7fefdc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.349+0000 7fefe0d99640 1 -- 172.21.15.46:0/526101351 >> 172.21.15.46:0/526101351 conn(0x7fefdc0fc010 msgr2=0x7fefdc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:18.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.350+0000 7fefe0d99640 1 -- 172.21.15.46:0/526101351 shutdown_connections 2024-09-10T04:05:18.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.350+0000 7fefe0d99640 1 -- 172.21.15.46:0/526101351 wait complete. 2024-09-10T04:05:18.352 INFO:tasks.workunit.client.0.smithi046.stdout:4/503: chown d3/d12/d17/d22/f27 15 1 2024-09-10T04:05:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.351+0000 7fefe0d99640 1 Processor -- start 2024-09-10T04:05:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.351+0000 7fefe0d99640 1 -- start start 2024-09-10T04:05:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.351+0000 7fefe0d99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc100860 0x7fefdc196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.351+0000 7fefe0d99640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc101a60 0x7fefdc196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.351+0000 7fefe0d99640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc109720 con 0x7fefdc101a60 2024-09-10T04:05:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.351+0000 7fefe0d99640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1095a0 con 0x7fefdc100860 2024-09-10T04:05:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.353+0000 7fefd9d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc101a60 0x7fefdc196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.353+0000 7fefd9d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc101a60 0x7fefdc196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39604/0 (socket says 172.21.15.46:39604) 2024-09-10T04:05:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.353+0000 7fefd9d9b640 1 -- 172.21.15.46:0/683730066 learned_addr learned my addr 172.21.15.46:0/683730066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.353+0000 7fefda59c640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc100860 0x7fefdc196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:18.355 INFO:tasks.workunit.client.1.smithi070.stdout:0/521: symlink d8/d13/d1a/d9a/la8 0 2024-09-10T04:05:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.354+0000 7fefd9d9b640 1 -- 172.21.15.46:0/683730066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc100860 msgr2=0x7fefdc196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.354+0000 7fefd9d9b640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc100860 0x7fefdc196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.354+0000 7fefd9d9b640 1 -- 172.21.15.46:0/683730066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc197000 con 0x7fefdc101a60 2024-09-10T04:05:18.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.354+0000 7fefd9d9b640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc101a60 0x7fefdc196980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009230 tx=0x7fefc40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:18.356 INFO:tasks.workunit.client.0.smithi046.stdout:7/494: write d5/d9/f87 [2796187,80262] 0 2024-09-10T04:05:18.357 INFO:tasks.workunit.client.0.smithi046.stdout:1/380: write d3/d27/d5d/d60/f65 [5050696,114584] 0 2024-09-10T04:05:18.357 INFO:tasks.workunit.client.1.smithi070.stdout:8/533: dread d4/f29 [0,4194304] 0 2024-09-10T04:05:18.358 INFO:tasks.workunit.client.1.smithi070.stdout:3/489: fdatasync db/f14 0 2024-09-10T04:05:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.358+0000 7fefd37fe640 1 -- 172.21.15.46:0/683730066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fefc403f070 con 0x7fefdc101a60 2024-09-10T04:05:18.360 INFO:tasks.workunit.client.1.smithi070.stdout:1/396: dread d3/d8/d13/f25 [0,4194304] 0 2024-09-10T04:05:18.360 INFO:tasks.workunit.client.1.smithi070.stdout:8/534: read d4/d16/d17/d1a/d60/d7a/f9d [626823,62763] 0 2024-09-10T04:05:18.360 INFO:tasks.workunit.client.1.smithi070.stdout:3/490: write db/d1c/f6d [229427,52032] 0 2024-09-10T04:05:18.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.358+0000 7fefd37fe640 1 -- 172.21.15.46:0/683730066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fefc40044d0 con 0x7fefdc101a60 2024-09-10T04:05:18.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.358+0000 7fefd37fe640 1 -- 172.21.15.46:0/683730066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fefc4035030 con 0x7fefdc101a60 2024-09-10T04:05:18.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.358+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc069d00 con 0x7fefdc101a60 2024-09-10T04:05:18.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.358+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc06a310 con 0x7fefdc101a60 2024-09-10T04:05:18.361 INFO:tasks.workunit.client.0.smithi046.stdout:6/450: link da/db/d18/f49 da/db/d10/d27/d8f/f95 0 2024-09-10T04:05:18.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.360+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefbc0052c0 con 0x7fefdc101a60 2024-09-10T04:05:18.362 INFO:tasks.workunit.client.1.smithi070.stdout:8/535: truncate d4/d16/d17/d1a/f78 259909 0 2024-09-10T04:05:18.363 INFO:tasks.workunit.client.1.smithi070.stdout:8/536: write d4/d16/d73/d96/f59 [549152,28451] 0 2024-09-10T04:05:18.364 INFO:tasks.workunit.client.0.smithi046.stdout:6/451: truncate da/db/d10/d27/d8f/f95 4506106 0 2024-09-10T04:05:18.365 INFO:tasks.workunit.client.0.smithi046.stdout:6/452: write da/db/d11/d13/d1e/f3e [385401,27830] 0 2024-09-10T04:05:18.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.365+0000 7fefd37fe640 1 -- 172.21.15.46:0/683730066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7fefc4042070 con 0x7fefdc101a60 2024-09-10T04:05:18.367 INFO:tasks.workunit.client.0.smithi046.stdout:4/504: symlink d3/d12/d17/d29/d58/lae 0 2024-09-10T04:05:18.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.366+0000 7fefd37fe640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fefc806ded0 0x7fefc8070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:18.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.366+0000 7fefd37fe640 1 -- 172.21.15.46:0/683730066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7fefc40aeba0 con 0x7fefdc101a60 2024-09-10T04:05:18.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.367+0000 7fefda59c640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fefc806ded0 0x7fefc8070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:18.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.367+0000 7fefda59c640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fefc806ded0 0x7fefc8070390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fefdc1018c0 tx=0x7fefcc0094f0 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.369+0000 7fefd37fe640 1 -- 172.21.15.46:0/683730066 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefc407c7a0 con 0x7fefdc101a60 2024-09-10T04:05:18.371 INFO:tasks.workunit.client.1.smithi070.stdout:4/422: rename de/c17 to de/d13/d43/c94 0 2024-09-10T04:05:18.376 INFO:tasks.workunit.client.0.smithi046.stdout:1/381: getdents d3/dc/d34/d58 0 2024-09-10T04:05:18.378 INFO:tasks.workunit.client.1.smithi070.stdout:6/559: unlink d0/d3/d18/d1b/d17/d3f/c82 0 2024-09-10T04:05:18.380 INFO:tasks.workunit.client.0.smithi046.stdout:4/505: creat d3/d12/d17/d22/d51/d94/faf x:0 0 0 2024-09-10T04:05:18.381 INFO:tasks.workunit.client.1.smithi070.stdout:9/540: creat d9/d60/d8a/d3c/d92/fa9 x:0 0 0 2024-09-10T04:05:18.381 INFO:tasks.workunit.client.0.smithi046.stdout:1/382: dread - d3/d27/f41 zero size 2024-09-10T04:05:18.383 INFO:tasks.workunit.client.0.smithi046.stdout:4/506: mknod d3/d43/d5f/d85/cb0 0 2024-09-10T04:05:18.384 INFO:tasks.workunit.client.0.smithi046.stdout:4/507: dread - d3/d43/d5f/d85/fa2 zero size 2024-09-10T04:05:18.385 INFO:tasks.workunit.client.0.smithi046.stdout:6/453: getdents da/dc 0 2024-09-10T04:05:18.386 INFO:tasks.workunit.client.1.smithi070.stdout:1/397: mknod d3/d8/d12/d2f/d32/d4b/d54/c84 0 2024-09-10T04:05:18.386 INFO:tasks.workunit.client.1.smithi070.stdout:1/398: read - d3/d8/d12/d2f/d32/f58 zero size 2024-09-10T04:05:18.388 INFO:tasks.workunit.client.0.smithi046.stdout:4/508: mknod d3/d43/cb1 0 2024-09-10T04:05:18.388 INFO:tasks.workunit.client.1.smithi070.stdout:6/560: dread d0/d3/f98 [0,4194304] 0 2024-09-10T04:05:18.390 INFO:tasks.workunit.client.1.smithi070.stdout:5/482: rename d17/d24/d4e to d17/d19/d6a/d54/d57/da3 0 2024-09-10T04:05:18.390 INFO:tasks.workunit.client.0.smithi046.stdout:4/509: readlink d3/l7 0 2024-09-10T04:05:18.391 INFO:tasks.workunit.client.1.smithi070.stdout:5/483: write d17/d24/d2b/f81 [776465,193] 0 2024-09-10T04:05:18.391 INFO:tasks.workunit.client.0.smithi046.stdout:6/454: mkdir da/d96 0 2024-09-10T04:05:18.392 INFO:tasks.workunit.client.1.smithi070.stdout:5/484: chown d17/d6f/d32/d90 404633160 1 2024-09-10T04:05:18.392 INFO:tasks.workunit.client.1.smithi070.stdout:7/432: symlink d2/d7/la1 0 2024-09-10T04:05:18.394 INFO:tasks.workunit.client.0.smithi046.stdout:1/383: rename d3/d27/d5d/d60/l70 to d3/d27/d32/d1f/l81 0 2024-09-10T04:05:18.395 INFO:tasks.workunit.client.0.smithi046.stdout:6/455: rmdir da/db/d18 39 2024-09-10T04:05:18.396 INFO:tasks.workunit.client.1.smithi070.stdout:0/522: mkdir d8/d24/da9 0 2024-09-10T04:05:18.403 INFO:tasks.workunit.client.1.smithi070.stdout:3/491: link db/d1c/d25/d28/d43/f71 db/d79/fa3 0 2024-09-10T04:05:18.404 INFO:tasks.workunit.client.1.smithi070.stdout:8/537: mkdir d4/d16/d17/d6f/db5 0 2024-09-10T04:05:18.405 INFO:tasks.workunit.client.1.smithi070.stdout:6/561: mkdir d0/d3/d8c/dab 0 2024-09-10T04:05:18.408 INFO:tasks.workunit.client.1.smithi070.stdout:4/423: truncate fb 3851629 0 2024-09-10T04:05:18.409 INFO:tasks.workunit.client.1.smithi070.stdout:4/424: dread - de/d13/d18/d23/d39/f8e zero size 2024-09-10T04:05:18.410 INFO:tasks.workunit.client.0.smithi046.stdout:6/456: dread da/db/d10/d27/f77 [0,4194304] 0 2024-09-10T04:05:18.410 INFO:tasks.workunit.client.1.smithi070.stdout:5/485: fsync d17/d19/d6a/f5e 0 2024-09-10T04:05:18.411 INFO:tasks.workunit.client.0.smithi046.stdout:6/457: chown da/db/d11/d13/d1e/d23/d86 421 1 2024-09-10T04:05:18.412 INFO:tasks.workunit.client.0.smithi046.stdout:4/510: rename d3/d43 to d3/d12/d17/d29/d58/db2 0 2024-09-10T04:05:18.413 INFO:tasks.workunit.client.1.smithi070.stdout:7/433: write d2/d3/dc/d5d/d42/f7c [3847307,13963] 0 2024-09-10T04:05:18.413 INFO:tasks.workunit.client.1.smithi070.stdout:7/434: readlink d2/d7/d9/l16 0 2024-09-10T04:05:18.416 INFO:tasks.workunit.client.0.smithi046.stdout:6/458: rmdir da/d45 39 2024-09-10T04:05:18.417 INFO:tasks.workunit.client.1.smithi070.stdout:7/435: dwrite d2/d7/db/d3a/f68 [0,4194304] 0 2024-09-10T04:05:18.420 INFO:tasks.workunit.client.0.smithi046.stdout:1/384: rename d3/d27/d32/d28/c7b to d3/dc/d34/c82 0 2024-09-10T04:05:18.426 INFO:tasks.workunit.client.1.smithi070.stdout:2/441: truncate d15/d18/d29/f46 1949097 0 2024-09-10T04:05:18.426 INFO:tasks.workunit.client.0.smithi046.stdout:4/511: symlink d3/d12/d17/d22/d30/lb3 0 2024-09-10T04:05:18.429 INFO:tasks.workunit.client.0.smithi046.stdout:4/512: read - d3/d12/d17/d22/f9d zero size 2024-09-10T04:05:18.429 INFO:tasks.workunit.client.0.smithi046.stdout:1/385: dwrite d3/dc/f6e [0,4194304] 0 2024-09-10T04:05:18.430 INFO:tasks.workunit.client.1.smithi070.stdout:3/492: creat db/d1c/d25/d28/d43/fa4 x:0 0 0 2024-09-10T04:05:18.431 INFO:tasks.workunit.client.0.smithi046.stdout:1/386: read d3/d27/d32/d28/f2c [2207488,31939] 0 2024-09-10T04:05:18.432 INFO:tasks.workunit.client.0.smithi046.stdout:6/459: creat da/db/d11/d2b/d5a/f97 x:0 0 0 2024-09-10T04:05:18.432 INFO:tasks.workunit.client.0.smithi046.stdout:4/513: read d3/d12/f9a [519422,42750] 0 2024-09-10T04:05:18.435 INFO:tasks.workunit.client.1.smithi070.stdout:6/562: unlink d0/d3/d18/d1b/d17/d3f/l9d 0 2024-09-10T04:05:18.436 INFO:tasks.workunit.client.0.smithi046.stdout:6/460: truncate da/db/d10/d27/f36 4685295 0 2024-09-10T04:05:18.438 INFO:tasks.workunit.client.1.smithi070.stdout:3/493: dwrite db/f74 [0,4194304] 0 2024-09-10T04:05:18.440 INFO:tasks.workunit.client.1.smithi070.stdout:3/494: write db/d1c/f8e [468425,105867] 0 2024-09-10T04:05:18.446 INFO:tasks.workunit.client.0.smithi046.stdout:4/514: dwrite d3/fc [8388608,4194304] 0 2024-09-10T04:05:18.447 INFO:tasks.workunit.client.0.smithi046.stdout:6/461: symlink da/dc/d79/l98 0 2024-09-10T04:05:18.447 INFO:tasks.workunit.client.1.smithi070.stdout:9/541: link d9/d60/d8a/d3c/c57 d9/d60/d8a/caa 0 2024-09-10T04:05:18.450 INFO:tasks.workunit.client.1.smithi070.stdout:0/523: dwrite d8/d24/d2d/d5f/f60 [0,4194304] 0 2024-09-10T04:05:18.451 INFO:tasks.workunit.client.1.smithi070.stdout:8/538: link d4/d16/d73/d96/f59 d4/d16/d17/d5c/fb6 0 2024-09-10T04:05:18.452 INFO:tasks.workunit.client.1.smithi070.stdout:9/542: write d9/d60/d8a/d1b/f5c [2839606,34834] 0 2024-09-10T04:05:18.453 INFO:tasks.workunit.client.0.smithi046.stdout:6/462: stat da/db/f16 0 2024-09-10T04:05:18.454 INFO:tasks.workunit.client.1.smithi070.stdout:0/524: chown d8/d24/d2d/d8d 29 1 2024-09-10T04:05:18.457 INFO:tasks.workunit.client.1.smithi070.stdout:2/442: dwrite d15/d18/d29/d2a/f74 [0,4194304] 0 2024-09-10T04:05:18.461 INFO:tasks.workunit.client.1.smithi070.stdout:6/563: creat d0/d3/d8c/d9a/fac x:0 0 0 2024-09-10T04:05:18.464 INFO:tasks.workunit.client.1.smithi070.stdout:2/443: dwrite d15/d18/d29/f7f [0,4194304] 0 2024-09-10T04:05:18.465 INFO:tasks.workunit.client.0.smithi046.stdout:1/387: getdents d3 0 2024-09-10T04:05:18.470 INFO:tasks.workunit.client.1.smithi070.stdout:5/486: symlink d17/d19/d6a/d54/la4 0 2024-09-10T04:05:18.472 INFO:tasks.workunit.client.0.smithi046.stdout:1/388: rmdir d3/dc/d34/d51 39 2024-09-10T04:05:18.479 INFO:tasks.workunit.client.0.smithi046.stdout:1/389: dread d3/d27/d32/d1f/f31 [0,4194304] 0 2024-09-10T04:05:18.481 INFO:tasks.workunit.client.1.smithi070.stdout:1/399: getdents d3/d8/d13/d67 0 2024-09-10T04:05:18.481 INFO:tasks.workunit.client.0.smithi046.stdout:1/390: write d3/d27/d32/f43 [627133,13886] 0 2024-09-10T04:05:18.486 INFO:tasks.workunit.client.1.smithi070.stdout:0/525: creat d8/d13/d1a/d8f/d95/faa x:0 0 0 2024-09-10T04:05:18.491 INFO:tasks.workunit.client.1.smithi070.stdout:6/564: creat d0/d3/d8c/fad x:0 0 0 2024-09-10T04:05:18.491 INFO:tasks.workunit.client.1.smithi070.stdout:2/444: truncate d15/d18/d56/d7c/f76 1193266 0 2024-09-10T04:05:18.492 INFO:tasks.workunit.client.0.smithi046.stdout:1/391: dwrite d3/dc/d34/f76 [0,4194304] 0 2024-09-10T04:05:18.492 INFO:tasks.workunit.client.0.smithi046.stdout:4/515: dwrite d3/d12/d17/d29/d58/db2/f45 [0,4194304] 0 2024-09-10T04:05:18.493 INFO:tasks.workunit.client.1.smithi070.stdout:3/495: creat db/d79/d94/d99/fa5 x:0 0 0 2024-09-10T04:05:18.496 INFO:tasks.workunit.client.0.smithi046.stdout:4/516: fdatasync d3/d12/f5a 0 2024-09-10T04:05:18.505 INFO:tasks.workunit.client.1.smithi070.stdout:7/436: link d2/d7/c6d d2/d3/ca2 0 2024-09-10T04:05:18.507 INFO:tasks.workunit.client.0.smithi046.stdout:6/463: dwrite f6 [4194304,4194304] 0 2024-09-10T04:05:18.507 INFO:tasks.workunit.client.1.smithi070.stdout:8/539: dread d4/f77 [0,4194304] 0 2024-09-10T04:05:18.507 INFO:tasks.workunit.client.1.smithi070.stdout:8/540: readlink d4/l18 0 2024-09-10T04:05:18.508 INFO:tasks.workunit.client.0.smithi046.stdout:6/464: dread - da/d59/f85 zero size 2024-09-10T04:05:18.508 INFO:tasks.workunit.client.1.smithi070.stdout:8/541: write d4/fa [3921249,58667] 0 2024-09-10T04:05:18.510 INFO:tasks.workunit.client.1.smithi070.stdout:9/543: creat d9/fab x:0 0 0 2024-09-10T04:05:18.512 INFO:tasks.workunit.client.1.smithi070.stdout:0/526: creat d8/d24/da9/fab x:0 0 0 2024-09-10T04:05:18.514 INFO:tasks.workunit.client.1.smithi070.stdout:0/527: write d8/d13/d1a/d8f/d95/fa7 [617969,13971] 0 2024-09-10T04:05:18.514 INFO:tasks.workunit.client.0.smithi046.stdout:4/517: creat d3/d12/d17/d29/d58/db2/d5f/fb4 x:0 0 0 2024-09-10T04:05:18.515 INFO:tasks.workunit.client.0.smithi046.stdout:4/518: dread d3/d12/f73 [0,4194304] 0 2024-09-10T04:05:18.521 INFO:tasks.workunit.client.1.smithi070.stdout:3/496: rename db/d1c/c24 to db/d79/d94/ca6 0 2024-09-10T04:05:18.523 INFO:tasks.workunit.client.1.smithi070.stdout:3/497: dread - db/d1c/d25/d28/d43/f77 zero size 2024-09-10T04:05:18.523 INFO:tasks.workunit.client.1.smithi070.stdout:7/437: chown d2/d7/d9/l90 682455830 1 2024-09-10T04:05:18.524 INFO:tasks.workunit.client.0.smithi046.stdout:4/519: dwrite d3/d12/d17/d29/f92 [0,4194304] 0 2024-09-10T04:05:18.526 INFO:tasks.workunit.client.1.smithi070.stdout:1/400: truncate f0 174296 0 2024-09-10T04:05:18.527 INFO:tasks.workunit.client.1.smithi070.stdout:9/544: stat d9/d60/d8a/d3c/f40 0 2024-09-10T04:05:18.529 INFO:tasks.workunit.client.1.smithi070.stdout:9/545: dread - d9/d60/d8a/d3c/d92/fa9 zero size 2024-09-10T04:05:18.530 INFO:tasks.workunit.client.1.smithi070.stdout:3/498: dwrite db/f33 [8388608,4194304] 0 2024-09-10T04:05:18.530 INFO:tasks.workunit.client.1.smithi070.stdout:2/445: creat d15/d18/d56/d7c/f9e x:0 0 0 2024-09-10T04:05:18.531 INFO:tasks.workunit.client.0.smithi046.stdout:4/520: mknod d3/d12/d17/d29/d58/db2/d5f/d85/da8/cb5 0 2024-09-10T04:05:18.531 INFO:tasks.workunit.client.0.smithi046.stdout:4/521: fsync d3/d12/d17/d29/d58/f71 0 2024-09-10T04:05:18.532 INFO:tasks.workunit.client.1.smithi070.stdout:6/565: truncate d0/d3/d18/d1b/f7d 132646 0 2024-09-10T04:05:18.533 INFO:tasks.workunit.client.0.smithi046.stdout:4/522: fdatasync d3/d12/d17/d29/d58/f75 0 2024-09-10T04:05:18.533 INFO:tasks.workunit.client.1.smithi070.stdout:6/566: write d0/d3/f51 [3032317,21229] 0 2024-09-10T04:05:18.534 INFO:tasks.workunit.client.0.smithi046.stdout:4/523: chown d3/d12/d17/d22/d51/f64 233 1 2024-09-10T04:05:18.537 INFO:tasks.workunit.client.0.smithi046.stdout:4/524: chown d3/d12/d17/d22/d30/faa 8 1 2024-09-10T04:05:18.538 INFO:tasks.workunit.client.0.smithi046.stdout:6/465: dwrite da/db/d10/d27/f29 [0,4194304] 0 2024-09-10T04:05:18.540 INFO:tasks.workunit.client.1.smithi070.stdout:2/446: dwrite d15/d18/d29/d2a/d2e/d32/d36/f5b [0,4194304] 0 2024-09-10T04:05:18.551 INFO:tasks.workunit.client.1.smithi070.stdout:7/438: creat d2/d66/fa3 x:0 0 0 2024-09-10T04:05:18.556 INFO:tasks.workunit.client.1.smithi070.stdout:3/499: creat db/d1c/d89/fa7 x:0 0 0 2024-09-10T04:05:18.559 INFO:tasks.workunit.client.1.smithi070.stdout:0/528: rename d8/f11 to d8/d85/fac 0 2024-09-10T04:05:18.566 INFO:tasks.workunit.client.1.smithi070.stdout:6/567: chown d0/d3/d18/d1b/c44 421557240 1 2024-09-10T04:05:18.567 INFO:tasks.workunit.client.1.smithi070.stdout:4/425: sync 2024-09-10T04:05:18.567 INFO:tasks.workunit.client.0.smithi046.stdout:4/525: dwrite d3/d12/d17/f1b [0,4194304] 0 2024-09-10T04:05:18.568 INFO:tasks.workunit.client.1.smithi070.stdout:4/426: chown de/d13/d18/d23/d39/d48/c7c 1775 1 2024-09-10T04:05:18.569 INFO:tasks.workunit.client.1.smithi070.stdout:8/542: creat d4/fb7 x:0 0 0 2024-09-10T04:05:18.570 INFO:tasks.workunit.client.1.smithi070.stdout:4/427: truncate de/d13/d18/d23/d39/d48/f7e 1370655 0 2024-09-10T04:05:18.570 INFO:tasks.workunit.client.1.smithi070.stdout:8/543: fdatasync d4/d16/d17/d1a/f94 0 2024-09-10T04:05:18.571 INFO:tasks.workunit.client.1.smithi070.stdout:7/439: creat d2/d7/d9/d18/d3c/d52/fa4 x:0 0 0 2024-09-10T04:05:18.572 INFO:tasks.workunit.client.0.smithi046.stdout:8/438: sync 2024-09-10T04:05:18.573 INFO:tasks.workunit.client.1.smithi070.stdout:3/500: creat db/d79/fa8 x:0 0 0 2024-09-10T04:05:18.574 INFO:tasks.workunit.client.1.smithi070.stdout:7/440: truncate d2/d7/d9/d18/d3c/d52/fa4 373564 0 2024-09-10T04:05:18.576 INFO:tasks.workunit.client.0.smithi046.stdout:4/526: dwrite d3/d12/d17/f6e [0,4194304] 0 2024-09-10T04:05:18.577 INFO:tasks.workunit.client.0.smithi046.stdout:8/439: write d1/f86 [2259083,86576] 0 2024-09-10T04:05:18.580 INFO:tasks.workunit.client.1.smithi070.stdout:1/401: rename d3/l4 to d3/d8/d12/d2f/d32/d4b/l85 0 2024-09-10T04:05:18.580 INFO:tasks.workunit.client.0.smithi046.stdout:8/440: stat d1/d29/d17/d1f/f54 0 2024-09-10T04:05:18.582 INFO:tasks.workunit.client.1.smithi070.stdout:1/402: readlink d3/d20/l77 0 2024-09-10T04:05:18.582 INFO:tasks.workunit.client.0.smithi046.stdout:4/527: creat d3/d12/d17/d22/d51/d5c/fb6 x:0 0 0 2024-09-10T04:05:18.584 INFO:tasks.workunit.client.1.smithi070.stdout:7/441: dwrite d2/d7/d9/d18/f6b [4194304,4194304] 0 2024-09-10T04:05:18.586 INFO:tasks.workunit.client.0.smithi046.stdout:8/441: unlink d1/d29/d17/d1f/f35 0 2024-09-10T04:05:18.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:18 smithi046 ceph-mon[28328]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:18.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:18 smithi046 ceph-mon[28328]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:18.588 INFO:tasks.workunit.client.1.smithi070.stdout:8/544: mkdir d4/d16/d17/d1a/d60/d7a/db8 0 2024-09-10T04:05:18.590 INFO:tasks.workunit.client.0.smithi046.stdout:6/466: truncate da/db/d10/d27/f29 1449540 0 2024-09-10T04:05:18.592 INFO:tasks.workunit.client.1.smithi070.stdout:4/428: dwrite de/d13/d5b/f78 [0,4194304] 0 2024-09-10T04:05:18.592 INFO:tasks.workunit.client.0.smithi046.stdout:4/528: dwrite d3/d12/d17/d22/d30/fa1 [0,4194304] 0 2024-09-10T04:05:18.597 INFO:tasks.workunit.client.0.smithi046.stdout:8/442: unlink d1/d29/d17/d1f/d46/d52/f65 0 2024-09-10T04:05:18.601 INFO:tasks.workunit.client.1.smithi070.stdout:6/568: mknod d0/d3/d18/d62/cae 0 2024-09-10T04:05:18.604 INFO:tasks.workunit.client.0.smithi046.stdout:4/529: write d3/d12/d17/d22/d51/f8f [3017329,32690] 0 2024-09-10T04:05:18.605 INFO:tasks.workunit.client.1.smithi070.stdout:8/545: dread d4/d16/d73/f93 [0,4194304] 0 2024-09-10T04:05:18.606 INFO:tasks.workunit.client.0.smithi046.stdout:4/530: dread - d3/d12/d17/d22/d30/faa zero size 2024-09-10T04:05:18.606 INFO:tasks.workunit.client.0.smithi046.stdout:8/443: dwrite d1/d29/f4a [0,4194304] 0 2024-09-10T04:05:18.607 INFO:tasks.workunit.client.0.smithi046.stdout:6/467: mknod da/db/c99 0 2024-09-10T04:05:18.611 INFO:tasks.workunit.client.0.smithi046.stdout:4/531: truncate d3/d12/d14/f36 1157557 0 2024-09-10T04:05:18.611 INFO:tasks.workunit.client.1.smithi070.stdout:4/429: creat de/d13/d18/d26/d8c/f95 x:0 0 0 2024-09-10T04:05:18.611 INFO:tasks.workunit.client.0.smithi046.stdout:8/444: rmdir d1/d29/d17/d1f/d46 39 2024-09-10T04:05:18.612 INFO:tasks.workunit.client.1.smithi070.stdout:2/447: getdents d15 0 2024-09-10T04:05:18.613 INFO:tasks.workunit.client.0.smithi046.stdout:6/468: fsync da/db/d11/d2b/d5a/f5c 0 2024-09-10T04:05:18.615 INFO:tasks.workunit.client.0.smithi046.stdout:4/532: symlink d3/d12/d17/d22/d51/d5c/d6c/lb7 0 2024-09-10T04:05:18.618 INFO:tasks.workunit.client.1.smithi070.stdout:7/442: symlink d2/d7/d9/la5 0 2024-09-10T04:05:18.619 INFO:tasks.workunit.client.1.smithi070.stdout:7/443: chown d2/d7/d9/d18/d3c/d52/c61 16040404 1 2024-09-10T04:05:18.620 INFO:tasks.workunit.client.0.smithi046.stdout:6/469: unlink da/f61 0 2024-09-10T04:05:18.621 INFO:tasks.workunit.client.0.smithi046.stdout:4/533: creat d3/fb8 x:0 0 0 2024-09-10T04:05:18.623 INFO:tasks.workunit.client.0.smithi046.stdout:4/534: fdatasync d3/d12/d17/d29/d58/f75 0 2024-09-10T04:05:18.627 INFO:tasks.workunit.client.1.smithi070.stdout:1/403: link d3/d20/c63 d3/d8/d13/c86 0 2024-09-10T04:05:18.628 INFO:tasks.workunit.client.1.smithi070.stdout:7/444: symlink d2/d7/d9/d18/d3c/la6 0 2024-09-10T04:05:18.628 INFO:tasks.workunit.client.0.smithi046.stdout:8/445: link d1/d29/d17/c25 d1/d29/d17/d1f/d46/d76/c9a 0 2024-09-10T04:05:18.630 INFO:tasks.workunit.client.0.smithi046.stdout:8/446: chown d1/d29/d19/d36/d4c/l68 3438515 1 2024-09-10T04:05:18.631 INFO:tasks.workunit.client.1.smithi070.stdout:2/448: truncate d15/d18/f20 2060881 0 2024-09-10T04:05:18.632 INFO:tasks.workunit.client.0.smithi046.stdout:8/447: truncate d1/d29/d17/d1f/d46/d6a/f94 534562 0 2024-09-10T04:05:18.634 INFO:tasks.workunit.client.1.smithi070.stdout:4/430: rename de/d13/d18/d23/d47/d59/d79/d6a/f7a to de/d13/d18/d5c/f96 0 2024-09-10T04:05:18.634 INFO:tasks.workunit.client.0.smithi046.stdout:4/535: getdents d3/d12/d17/d22/d51 0 2024-09-10T04:05:18.635 INFO:tasks.workunit.client.0.smithi046.stdout:8/448: symlink d1/d29/d17/d1f/d2a/l9b 0 2024-09-10T04:05:18.636 INFO:tasks.workunit.client.0.smithi046.stdout:8/449: chown d1/d29/d19/d36/d8d/l90 2139 1 2024-09-10T04:05:18.637 INFO:tasks.workunit.client.1.smithi070.stdout:2/449: mknod d15/d18/d29/d2a/d2e/d32/c9f 0 2024-09-10T04:05:18.638 INFO:tasks.workunit.client.1.smithi070.stdout:2/450: chown d15/d18/f26 4836 1 2024-09-10T04:05:18.639 INFO:tasks.workunit.client.0.smithi046.stdout:4/536: mkdir d3/dab/db9 0 2024-09-10T04:05:18.640 INFO:tasks.workunit.client.1.smithi070.stdout:4/431: creat de/d13/d18/d23/d39/d77/f97 x:0 0 0 2024-09-10T04:05:18.640 INFO:tasks.workunit.client.1.smithi070.stdout:2/451: fsync d15/d18/d56/d7c/d55/d6d/f97 0 2024-09-10T04:05:18.641 INFO:tasks.workunit.client.1.smithi070.stdout:4/432: dread - de/d13/d5b/f80 zero size 2024-09-10T04:05:18.642 INFO:tasks.workunit.client.1.smithi070.stdout:7/445: link d2/d40/d6c/l76 d2/d7/db/la7 0 2024-09-10T04:05:18.645 INFO:tasks.workunit.client.1.smithi070.stdout:7/446: truncate d2/d7/d9/d18/f99 4823436 0 2024-09-10T04:05:18.645 INFO:tasks.workunit.client.0.smithi046.stdout:8/450: dread d1/d29/d19/f56 [0,4194304] 0 2024-09-10T04:05:18.646 INFO:tasks.workunit.client.1.smithi070.stdout:7/447: read d2/d7/db/d3a/f68 [3468084,121591] 0 2024-09-10T04:05:18.647 INFO:tasks.workunit.client.1.smithi070.stdout:7/448: fdatasync d2/d7/db/d3a/f9a 0 2024-09-10T04:05:18.647 INFO:tasks.workunit.client.0.smithi046.stdout:4/537: dwrite d3/d12/d17/d22/d30/f72 [4194304,4194304] 0 2024-09-10T04:05:18.648 INFO:tasks.workunit.client.0.smithi046.stdout:8/451: write d1/d29/f4a [3618339,124780] 0 2024-09-10T04:05:18.649 INFO:tasks.workunit.client.0.smithi046.stdout:4/538: write d3/d12/d17/d29/d58/f8c [633684,75853] 0 2024-09-10T04:05:18.651 INFO:tasks.workunit.client.1.smithi070.stdout:4/433: dread de/d13/d18/f3b [0,4194304] 0 2024-09-10T04:05:18.653 INFO:tasks.workunit.client.0.smithi046.stdout:8/452: fdatasync d1/fd 0 2024-09-10T04:05:18.655 INFO:tasks.workunit.client.1.smithi070.stdout:7/449: mkdir d2/d3/dc/d5d/d41/da8 0 2024-09-10T04:05:18.655 INFO:tasks.workunit.client.1.smithi070.stdout:1/404: dread d3/d10/f5e [0,4194304] 0 2024-09-10T04:05:18.659 INFO:tasks.workunit.client.0.smithi046.stdout:4/539: dread d3/d12/d17/f5e [0,4194304] 0 2024-09-10T04:05:18.662 INFO:tasks.workunit.client.1.smithi070.stdout:7/450: mkdir d2/d3/d63/da9 0 2024-09-10T04:05:18.663 INFO:tasks.workunit.client.1.smithi070.stdout:5/487: sync 2024-09-10T04:05:18.664 INFO:tasks.workunit.client.1.smithi070.stdout:1/405: write d3/f47 [4156970,70544] 0 2024-09-10T04:05:18.664 INFO:tasks.workunit.client.1.smithi070.stdout:5/488: chown d17/d24/d84/la0 74392941 1 2024-09-10T04:05:18.668 INFO:tasks.workunit.client.0.smithi046.stdout:4/540: mkdir d3/d12/d17/d29/dba 0 2024-09-10T04:05:18.669 INFO:tasks.workunit.client.1.smithi070.stdout:3/501: dwrite f7 [0,4194304] 0 2024-09-10T04:05:18.672 INFO:tasks.workunit.client.0.smithi046.stdout:4/541: creat d3/d12/d17/d29/fbb x:0 0 0 2024-09-10T04:05:18.673 INFO:tasks.workunit.client.1.smithi070.stdout:5/489: mkdir d17/d6f/d32/d90/da5 0 2024-09-10T04:05:18.673 INFO:tasks.workunit.client.0.smithi046.stdout:4/542: write d3/d12/d17/d29/d58/f6d [144362,86282] 0 2024-09-10T04:05:18.674 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:18 smithi070 ceph-mon[35495]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:18.674 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:18 smithi070 ceph-mon[35495]: from='client.14674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:18.675 INFO:tasks.workunit.client.1.smithi070.stdout:5/490: read - d17/d19/d35/f72 zero size 2024-09-10T04:05:18.676 INFO:tasks.workunit.client.1.smithi070.stdout:3/502: creat db/d79/fa9 x:0 0 0 2024-09-10T04:05:18.677 INFO:tasks.workunit.client.1.smithi070.stdout:3/503: chown db/d1c/d25/d28/f72 174470319 1 2024-09-10T04:05:18.680 INFO:tasks.workunit.client.0.smithi046.stdout:4/543: creat d3/d12/d17/d22/d51/d94/fbc x:0 0 0 2024-09-10T04:05:18.681 INFO:tasks.workunit.client.1.smithi070.stdout:5/491: dwrite d17/d19/d35/f72 [0,4194304] 0 2024-09-10T04:05:18.683 INFO:tasks.workunit.client.1.smithi070.stdout:3/504: creat db/d83/d92/faa x:0 0 0 2024-09-10T04:05:18.683 INFO:tasks.workunit.client.1.smithi070.stdout:1/406: link d3/la d3/d10/d1d/l87 0 2024-09-10T04:05:18.683 INFO:tasks.workunit.client.1.smithi070.stdout:1/407: chown d3/d8/d27 352606 1 2024-09-10T04:05:18.685 INFO:tasks.workunit.client.0.smithi046.stdout:4/544: rename d3/d12/d14/f36 to d3/dab/fbd 0 2024-09-10T04:05:18.685 INFO:tasks.workunit.client.1.smithi070.stdout:8/546: write d4/d16/d17/d1a/f32 [2736462,64911] 0 2024-09-10T04:05:18.689 INFO:tasks.workunit.client.1.smithi070.stdout:5/492: rename d17/d24/f2d to d17/d37/fa6 0 2024-09-10T04:05:18.689 INFO:tasks.workunit.client.1.smithi070.stdout:7/451: getdents d2/d3 0 2024-09-10T04:05:18.690 INFO:tasks.workunit.client.1.smithi070.stdout:6/569: truncate d0/d3/d18/d1b/f11 7664710 0 2024-09-10T04:05:18.690 INFO:tasks.workunit.client.0.smithi046.stdout:4/545: creat d3/d12/d17/d29/d58/db2/d5f/d85/da8/fbe x:0 0 0 2024-09-10T04:05:18.692 INFO:tasks.workunit.client.1.smithi070.stdout:1/408: creat d3/d10/d1d/f88 x:0 0 0 2024-09-10T04:05:18.693 INFO:tasks.workunit.client.0.smithi046.stdout:6/470: truncate da/db/d11/d13/f57 1239650 0 2024-09-10T04:05:18.695 INFO:tasks.workunit.client.1.smithi070.stdout:8/547: symlink d4/d16/d17/d1a/d60/lb9 0 2024-09-10T04:05:18.695 INFO:tasks.workunit.client.0.smithi046.stdout:4/546: dread d3/dab/fbd [0,4194304] 0 2024-09-10T04:05:18.700 INFO:tasks.workunit.client.0.smithi046.stdout:4/547: mkdir d3/d12/d17/d91/dbf 0 2024-09-10T04:05:18.700 INFO:tasks.workunit.client.0.smithi046.stdout:4/548: chown d3/d12/l3d 253 1 2024-09-10T04:05:18.701 INFO:tasks.workunit.client.1.smithi070.stdout:6/570: truncate d0/d3/d18/d1b/d17/d28/f76 167703 0 2024-09-10T04:05:18.702 INFO:tasks.workunit.client.1.smithi070.stdout:7/452: dread d2/d7/db/f37 [0,4194304] 0 2024-09-10T04:05:18.702 INFO:tasks.workunit.client.0.smithi046.stdout:6/471: unlink da/db/d11/d2b/d55/c7a 0 2024-09-10T04:05:18.704 INFO:tasks.workunit.client.0.smithi046.stdout:6/472: stat da/db/d11/d13/d1e/d23/d54/c8e 0 2024-09-10T04:05:18.705 INFO:tasks.workunit.client.0.smithi046.stdout:4/549: readlink d3/d12/la9 0 2024-09-10T04:05:18.705 INFO:tasks.workunit.client.0.smithi046.stdout:6/473: truncate da/db/d11/f8c 1142086 0 2024-09-10T04:05:18.706 INFO:tasks.workunit.client.1.smithi070.stdout:1/409: dread d3/d8/f2d [0,4194304] 0 2024-09-10T04:05:18.706 INFO:tasks.workunit.client.1.smithi070.stdout:1/410: readlink d3/d8/le 0 2024-09-10T04:05:18.709 INFO:tasks.workunit.client.1.smithi070.stdout:2/452: truncate d15/d18/f21 5876480 0 2024-09-10T04:05:18.714 INFO:tasks.workunit.client.1.smithi070.stdout:9/546: sync 2024-09-10T04:05:18.714 INFO:tasks.workunit.client.1.smithi070.stdout:0/529: sync 2024-09-10T04:05:18.715 INFO:tasks.workunit.client.1.smithi070.stdout:9/547: chown d9/d60/d8a/d1b/f5c 26565 1 2024-09-10T04:05:18.716 INFO:tasks.workunit.client.1.smithi070.stdout:6/571: creat d0/d3/d18/d1b/d17/d54/faf x:0 0 0 2024-09-10T04:05:18.717 INFO:tasks.workunit.client.1.smithi070.stdout:4/434: write de/d13/f38 [4795591,95402] 0 2024-09-10T04:05:18.718 INFO:tasks.workunit.client.0.smithi046.stdout:8/453: write d1/d29/d12/f57 [237394,58617] 0 2024-09-10T04:05:18.718 INFO:tasks.workunit.client.0.smithi046.stdout:6/474: dread da/db/d10/f5d [0,4194304] 0 2024-09-10T04:05:18.718 INFO:tasks.workunit.client.1.smithi070.stdout:7/453: fdatasync d2/d3/dc/d5d/f34 0 2024-09-10T04:05:18.721 INFO:tasks.workunit.client.1.smithi070.stdout:7/454: fsync d2/d7/d9/d18/f99 0 2024-09-10T04:05:18.722 INFO:tasks.workunit.client.1.smithi070.stdout:3/505: link db/d83/la1 db/d1c/d25/d28/d43/lab 0 2024-09-10T04:05:18.723 INFO:tasks.workunit.client.1.smithi070.stdout:6/572: dwrite d0/d3/d18/f27 [0,4194304] 0 2024-09-10T04:05:18.724 INFO:tasks.workunit.client.0.smithi046.stdout:6/475: creat da/db/d11/d2b/f9a x:0 0 0 2024-09-10T04:05:18.724 INFO:tasks.workunit.client.0.smithi046.stdout:6/476: stat da/dc/d22 0 2024-09-10T04:05:18.727 INFO:tasks.workunit.client.0.smithi046.stdout:8/454: creat d1/d29/d17/f9c x:0 0 0 2024-09-10T04:05:18.733 INFO:tasks.workunit.client.1.smithi070.stdout:2/453: symlink d15/d2b/la0 0 2024-09-10T04:05:18.734 INFO:tasks.workunit.client.1.smithi070.stdout:2/454: dread - d15/d18/d56/d7c/f9e zero size 2024-09-10T04:05:18.736 INFO:tasks.workunit.client.0.smithi046.stdout:8/455: dwrite d1/f39 [0,4194304] 0 2024-09-10T04:05:18.737 INFO:tasks.workunit.client.0.smithi046.stdout:4/550: getdents d3/d12/d14 0 2024-09-10T04:05:18.738 INFO:tasks.workunit.client.1.smithi070.stdout:0/530: truncate d8/d13/f28 3560409 0 2024-09-10T04:05:18.739 INFO:tasks.workunit.client.0.smithi046.stdout:4/551: dread - d3/d12/d17/d29/fbb zero size 2024-09-10T04:05:18.746 INFO:tasks.workunit.client.0.smithi046.stdout:8/456: mkdir d1/d29/d17/d1f/d46/d52/d9d 0 2024-09-10T04:05:18.748 INFO:tasks.workunit.client.0.smithi046.stdout:4/552: dwrite d3/d12/d17/d29/d58/f6d [0,4194304] 0 2024-09-10T04:05:18.749 INFO:tasks.workunit.client.0.smithi046.stdout:8/457: readlink d1/d29/d19/d36/d6e/l78 0 2024-09-10T04:05:18.749 INFO:tasks.workunit.client.0.smithi046.stdout:4/553: fdatasync d3/d12/d17/d29/f38 0 2024-09-10T04:05:18.753 INFO:tasks.workunit.client.0.smithi046.stdout:8/458: write d1/d29/d19/d36/d4c/d5e/f7f [983822,35961] 0 2024-09-10T04:05:18.753 INFO:tasks.workunit.client.1.smithi070.stdout:7/455: mknod d2/d3/caa 0 2024-09-10T04:05:18.760 INFO:tasks.workunit.client.0.smithi046.stdout:4/554: creat d3/d12/d17/d22/d51/d5c/d6c/fc0 x:0 0 0 2024-09-10T04:05:18.760 INFO:tasks.workunit.client.1.smithi070.stdout:1/411: symlink d3/d10/l89 0 2024-09-10T04:05:18.762 INFO:tasks.workunit.client.1.smithi070.stdout:8/548: write d4/d16/d17/f40 [8945727,34848] 0 2024-09-10T04:05:18.762 INFO:tasks.workunit.client.1.smithi070.stdout:1/412: readlink d3/d8/le 0 2024-09-10T04:05:18.763 INFO:tasks.workunit.client.1.smithi070.stdout:7/456: dwrite d2/d7/d9/d18/d23/f47 [0,4194304] 0 2024-09-10T04:05:18.765 INFO:tasks.workunit.client.0.smithi046.stdout:8/459: unlink d1/d29/d19/d73/f88 0 2024-09-10T04:05:18.765 INFO:tasks.workunit.client.1.smithi070.stdout:1/413: write d3/d8/d13/d3f/f52 [1944613,326] 0 2024-09-10T04:05:18.766 INFO:tasks.workunit.client.1.smithi070.stdout:2/455: dread d15/d18/d56/d7c/d55/f65 [0,4194304] 0 2024-09-10T04:05:18.766 INFO:tasks.workunit.client.0.smithi046.stdout:6/477: dread da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:18.769 INFO:tasks.workunit.client.1.smithi070.stdout:5/493: truncate d17/d6f/f52 2310914 0 2024-09-10T04:05:18.770 INFO:tasks.workunit.client.0.smithi046.stdout:4/555: dwrite d3/d12/d14/f2b [0,4194304] 0 2024-09-10T04:05:18.771 INFO:tasks.workunit.client.0.smithi046.stdout:4/556: read - d3/d12/d17/d22/f70 zero size 2024-09-10T04:05:18.772 INFO:tasks.workunit.client.1.smithi070.stdout:0/531: fdatasync d8/d13/d33/d53/d59/f64 0 2024-09-10T04:05:18.772 INFO:tasks.workunit.client.1.smithi070.stdout:2/456: dwrite f3 [4194304,4194304] 0 2024-09-10T04:05:18.777 INFO:tasks.workunit.client.0.smithi046.stdout:4/557: dread d3/d12/d14/f2b [0,4194304] 0 2024-09-10T04:05:18.780 INFO:tasks.workunit.client.0.smithi046.stdout:4/558: write d3/d12/d17/d22/d51/f8f [2247064,20466] 0 2024-09-10T04:05:18.791 INFO:tasks.workunit.client.1.smithi070.stdout:7/457: mknod d2/d3/d63/cab 0 2024-09-10T04:05:18.791 INFO:tasks.workunit.client.1.smithi070.stdout:7/458: dread - d2/d3/dc/d5d/d2d/f8e zero size 2024-09-10T04:05:18.792 INFO:tasks.workunit.client.0.smithi046.stdout:6/478: mkdir da/d9b 0 2024-09-10T04:05:18.794 INFO:tasks.workunit.client.0.smithi046.stdout:4/559: readlink d3/d12/la5 0 2024-09-10T04:05:18.797 INFO:tasks.workunit.client.1.smithi070.stdout:1/414: read d3/d8/d12/f4a [450906,56121] 0 2024-09-10T04:05:18.797 INFO:tasks.workunit.client.1.smithi070.stdout:6/573: dwrite d0/f8 [0,4194304] 0 2024-09-10T04:05:18.798 INFO:tasks.workunit.client.0.smithi046.stdout:6/479: write da/db/d11/d13/d1e/d69/f87 [2113923,52100] 0 2024-09-10T04:05:18.798 INFO:tasks.workunit.client.0.smithi046.stdout:6/480: write f6 [2305506,86173] 0 2024-09-10T04:05:18.801 INFO:tasks.workunit.client.0.smithi046.stdout:4/560: mknod d3/d12/d17/d29/d58/cc1 0 2024-09-10T04:05:18.808 INFO:tasks.workunit.client.1.smithi070.stdout:0/532: unlink d8/d24/d2d/d34/fa3 0 2024-09-10T04:05:18.809 INFO:tasks.workunit.client.1.smithi070.stdout:0/533: readlink d8/d13/l16 0 2024-09-10T04:05:18.809 INFO:tasks.workunit.client.1.smithi070.stdout:0/534: write d8/d24/d2d/f51 [4387450,121843] 0 2024-09-10T04:05:18.810 INFO:tasks.workunit.client.1.smithi070.stdout:3/506: truncate db/d1c/f6d 65812 0 2024-09-10T04:05:18.810 INFO:tasks.workunit.client.1.smithi070.stdout:0/535: stat d8/d13/d1a/f32 0 2024-09-10T04:05:18.813 INFO:tasks.workunit.client.0.smithi046.stdout:8/460: dwrite d1/fd [0,4194304] 0 2024-09-10T04:05:18.813 INFO:tasks.workunit.client.0.smithi046.stdout:8/461: readlink d1/d29/d19/d36/d8d/l90 0 2024-09-10T04:05:18.820 INFO:tasks.workunit.client.0.smithi046.stdout:5/553: sync 2024-09-10T04:05:18.820 INFO:tasks.workunit.client.0.smithi046.stdout:2/465: sync 2024-09-10T04:05:18.821 INFO:tasks.workunit.client.0.smithi046.stdout:4/561: mknod d3/d12/d14/cc2 0 2024-09-10T04:05:18.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.819+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fefbc005880 con 0x7fefdc101a60 2024-09-10T04:05:18.822 INFO:tasks.workunit.client.0.smithi046.stdout:5/554: chown d7/d11/d14/d8e/la2 12736 1 2024-09-10T04:05:18.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.822+0000 7fefd37fe640 1 -- 172.21.15.46:0/683730066 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+633 (secure 0 0 0) 0x7fefc4080450 con 0x7fefdc101a60 2024-09-10T04:05:18.823 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:05:18.823 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:05:18.823 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 14 2024-09-10T04:05:18.824 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:05:18.825 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:05:18.825 INFO:tasks.workunit.client.0.smithi046.stdout:8/462: mknod d1/d29/d19/d36/d6e/d75/c9e 0 2024-09-10T04:05:18.825 INFO:tasks.workunit.client.0.smithi046.stdout:5/555: chown d7/d11/d14/d2f/d2b/f3a 62076 1 2024-09-10T04:05:18.825 INFO:tasks.workunit.client.0.smithi046.stdout:4/562: read d3/d12/d17/d29/d58/db2/f48 [275917,117767] 0 2024-09-10T04:05:18.826 INFO:tasks.workunit.client.1.smithi070.stdout:7/459: creat d2/d7/db/d3a/fac x:0 0 0 2024-09-10T04:05:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.825+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fefc806ded0 msgr2=0x7fefc8070390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.825+0000 7fefe0d99640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fefc806ded0 0x7fefc8070390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fefdc1018c0 tx=0x7fefcc0094f0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.825+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc101a60 msgr2=0x7fefdc196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.825+0000 7fefe0d99640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc101a60 0x7fefdc196980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009230 tx=0x7fefc40048f0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.826+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 shutdown_connections 2024-09-10T04:05:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.826+0000 7fefe0d99640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefdc101a60 0x7fefdc196980 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.826+0000 7fefe0d99640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7fefc806ded0 0x7fefc8070390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.826+0000 7fefe0d99640 1 --2- 172.21.15.46:0/683730066 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefdc100860 0x7fefdc196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.826+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 >> 172.21.15.46:0/683730066 conn(0x7fefdc0fc010 msgr2=0x7fefdc0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.826+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 shutdown_connections 2024-09-10T04:05:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.826+0000 7fefe0d99640 1 -- 172.21.15.46:0/683730066 wait complete. 2024-09-10T04:05:18.829 INFO:tasks.workunit.client.0.smithi046.stdout:2/466: symlink d2/d6/d23/d5d/d77/d7b/d85/lb0 0 2024-09-10T04:05:18.830 INFO:tasks.workunit.client.1.smithi070.stdout:8/549: dwrite d4/d16/d17/f79 [0,4194304] 0 2024-09-10T04:05:18.831 INFO:tasks.workunit.client.0.smithi046.stdout:4/563: dread d3/d12/d17/d22/d51/f8f [4194304,4194304] 0 2024-09-10T04:05:18.831 INFO:tasks.workunit.client.1.smithi070.stdout:9/548: getdents d9/d1c/d34 0 2024-09-10T04:05:18.833 INFO:tasks.workunit.client.0.smithi046.stdout:4/564: write d3/d12/d17/d22/d30/fa1 [176171,24762] 0 2024-09-10T04:05:18.834 INFO:tasks.workunit.client.0.smithi046.stdout:4/565: write d3/d12/d14/fa6 [15226,119212] 0 2024-09-10T04:05:18.837 INFO:tasks.workunit.client.0.smithi046.stdout:5/556: read d7/f10 [2400589,73869] 0 2024-09-10T04:05:18.838 INFO:tasks.workunit.client.0.smithi046.stdout:4/566: read d3/d12/d17/d22/d51/f64 [3909772,88509] 0 2024-09-10T04:05:18.839 INFO:tasks.workunit.client.1.smithi070.stdout:5/494: write d17/d24/d45/f96 [946826,90744] 0 2024-09-10T04:05:18.840 INFO:tasks.workunit.client.1.smithi070.stdout:4/435: sync 2024-09-10T04:05:18.840 INFO:tasks.workunit.client.0.smithi046.stdout:4/567: dread d3/dab/fbd [0,4194304] 0 2024-09-10T04:05:18.842 INFO:tasks.workunit.client.1.smithi070.stdout:4/436: write de/d13/d18/d23/d47/d59/d79/f66 [635829,101446] 0 2024-09-10T04:05:18.842 INFO:tasks.workunit.client.1.smithi070.stdout:4/437: chown de/f31 1070048643 1 2024-09-10T04:05:18.843 INFO:tasks.workunit.client.0.smithi046.stdout:6/481: getdents da/db/d11/d13/d1e/d23/d54 0 2024-09-10T04:05:18.844 INFO:tasks.workunit.client.1.smithi070.stdout:6/574: rmdir d0/d3/d8c/d9a 39 2024-09-10T04:05:18.849 INFO:tasks.workunit.client.1.smithi070.stdout:6/575: dwrite d0/d3/d18/d1b/d17/d53/fa6 [0,4194304] 0 2024-09-10T04:05:18.851 INFO:tasks.workunit.client.1.smithi070.stdout:6/576: truncate d0/fa7 585300 0 2024-09-10T04:05:18.856 INFO:tasks.workunit.client.1.smithi070.stdout:0/536: rename d8/d13/d33/d81/d78 to d8/d24/d2d/d34/d9b/dad 0 2024-09-10T04:05:18.860 INFO:tasks.workunit.client.0.smithi046.stdout:4/568: symlink d3/d12/d17/d29/d58/db2/d5f/d85/lc3 0 2024-09-10T04:05:18.862 INFO:tasks.workunit.client.1.smithi070.stdout:7/460: mknod d2/d7/d9/d18/d23/cad 0 2024-09-10T04:05:18.867 INFO:tasks.workunit.client.0.smithi046.stdout:2/467: write d2/d49/f2e [379498,101995] 0 2024-09-10T04:05:18.868 INFO:tasks.workunit.client.0.smithi046.stdout:4/569: dwrite d3/d12/d17/d29/d58/db2/d5f/d85/fa2 [0,4194304] 0 2024-09-10T04:05:18.871 INFO:tasks.workunit.client.0.smithi046.stdout:4/570: truncate d3/d12/d17/d22/d51/d5c/d6c/fc0 170697 0 2024-09-10T04:05:18.873 INFO:tasks.workunit.client.0.smithi046.stdout:4/571: write d3/d12/d17/d29/fbb [578405,85684] 0 2024-09-10T04:05:18.874 INFO:tasks.workunit.client.0.smithi046.stdout:5/557: write d7/f10 [4594879,95904] 0 2024-09-10T04:05:18.876 INFO:tasks.workunit.client.0.smithi046.stdout:2/468: symlink d2/d86/lb1 0 2024-09-10T04:05:18.880 INFO:tasks.workunit.client.1.smithi070.stdout:4/438: creat de/d13/d18/d26/d8c/f98 x:0 0 0 2024-09-10T04:05:18.881 INFO:tasks.workunit.client.0.smithi046.stdout:8/463: getdents d1/d29/d17/d1f/d46/d67 0 2024-09-10T04:05:18.883 INFO:tasks.workunit.client.0.smithi046.stdout:8/464: fsync d1/d29/d19/d36/d6e/f95 0 2024-09-10T04:05:18.887 INFO:tasks.workunit.client.1.smithi070.stdout:5/495: write d17/d37/f3d [289828,60319] 0 2024-09-10T04:05:18.888 INFO:tasks.workunit.client.1.smithi070.stdout:2/457: getdents d15/d18/d56/d7c/d55 0 2024-09-10T04:05:18.888 INFO:tasks.workunit.client.1.smithi070.stdout:1/415: truncate d3/f60 6489713 0 2024-09-10T04:05:18.890 INFO:tasks.workunit.client.1.smithi070.stdout:3/507: link db/d1c/d25/d28/f73 db/d83/fac 0 2024-09-10T04:05:18.893 INFO:tasks.workunit.client.1.smithi070.stdout:6/577: creat d0/d3/d18/d2a/fb0 x:0 0 0 2024-09-10T04:05:18.896 INFO:tasks.workunit.client.0.smithi046.stdout:5/558: dread d7/f10 [0,4194304] 0 2024-09-10T04:05:18.897 INFO:tasks.workunit.client.0.smithi046.stdout:6/482: getdents da/db/d10/d51 0 2024-09-10T04:05:18.900 INFO:tasks.workunit.client.0.smithi046.stdout:4/572: dread d3/d12/d14/f18 [0,4194304] 0 2024-09-10T04:05:18.900 INFO:tasks.workunit.client.0.smithi046.stdout:8/465: unlink d1/d29/d17/d1f/d46/d6a/d82/c8a 0 2024-09-10T04:05:18.901 INFO:tasks.workunit.client.1.smithi070.stdout:7/461: unlink d2/d3/f70 0 2024-09-10T04:05:18.902 INFO:tasks.workunit.client.1.smithi070.stdout:7/462: dread - d2/d7/db/f5c zero size 2024-09-10T04:05:18.902 INFO:tasks.workunit.client.1.smithi070.stdout:8/550: truncate d4/d16/d73/d96/f59 90613 0 2024-09-10T04:05:18.903 INFO:tasks.workunit.client.1.smithi070.stdout:8/551: stat d4/d16/d17/d1a/f2a 0 2024-09-10T04:05:18.903 INFO:tasks.workunit.client.0.smithi046.stdout:6/483: dread f6 [4194304,4194304] 0 2024-09-10T04:05:18.908 INFO:tasks.workunit.client.1.smithi070.stdout:5/496: mknod d17/d37/ca7 0 2024-09-10T04:05:18.909 INFO:tasks.workunit.client.0.smithi046.stdout:2/469: dread d2/d6/d15/f3e [0,4194304] 0 2024-09-10T04:05:18.910 INFO:tasks.workunit.client.0.smithi046.stdout:2/470: chown d2/d6/d9d/la2 934903 1 2024-09-10T04:05:18.916 INFO:tasks.workunit.client.1.smithi070.stdout:6/578: unlink d0/d3/d1f/f37 0 2024-09-10T04:05:18.916 INFO:tasks.workunit.client.1.smithi070.stdout:2/458: dread d15/d18/d29/d2a/f58 [0,4194304] 0 2024-09-10T04:05:18.926 INFO:tasks.workunit.client.1.smithi070.stdout:7/463: creat d2/d7/d9/d18/d3c/fae x:0 0 0 2024-09-10T04:05:18.927 INFO:tasks.workunit.client.1.smithi070.stdout:7/464: write d2/d40/d6c/f7f [868909,79568] 0 2024-09-10T04:05:18.929 INFO:tasks.workunit.client.1.smithi070.stdout:8/552: creat d4/d16/d73/fba x:0 0 0 2024-09-10T04:05:18.930 INFO:tasks.workunit.client.1.smithi070.stdout:8/553: write d4/d16/f33 [4709316,108467] 0 2024-09-10T04:05:18.930 INFO:tasks.workunit.client.1.smithi070.stdout:4/439: symlink de/d13/d43/d88/l99 0 2024-09-10T04:05:18.933 INFO:tasks.workunit.client.1.smithi070.stdout:5/497: chown d17/d24/f2f 6403518 1 2024-09-10T04:05:18.935 INFO:tasks.workunit.client.1.smithi070.stdout:1/416: write d3/d8/d12/d2f/d32/f41 [3596141,38783] 0 2024-09-10T04:05:18.936 INFO:tasks.workunit.client.1.smithi070.stdout:3/508: mknod db/d1c/d25/cad 0 2024-09-10T04:05:18.937 INFO:tasks.workunit.client.1.smithi070.stdout:8/554: dwrite d4/d16/d17/d1a/f78 [0,4194304] 0 2024-09-10T04:05:18.941 INFO:tasks.workunit.client.1.smithi070.stdout:2/459: mkdir d15/d59/da1 0 2024-09-10T04:05:18.943 INFO:tasks.workunit.client.1.smithi070.stdout:0/537: creat d8/d24/d36/d87/fae x:0 0 0 2024-09-10T04:05:18.945 INFO:tasks.workunit.client.1.smithi070.stdout:9/549: rename d9/d60 to d9/d1c/dac 0 2024-09-10T04:05:18.946 INFO:tasks.workunit.client.1.smithi070.stdout:1/417: dwrite d3/d8/d12/d2f/f7e [0,4194304] 0 2024-09-10T04:05:18.946 INFO:tasks.workunit.client.0.smithi046.stdout:5/559: mkdir d7/d11/d14/d8e/d96/dce 0 2024-09-10T04:05:18.946 INFO:tasks.workunit.client.1.smithi070.stdout:9/550: readlink d9/la 0 2024-09-10T04:05:18.947 INFO:tasks.workunit.client.1.smithi070.stdout:3/509: dwrite db/d1c/d3e/f59 [0,4194304] 0 2024-09-10T04:05:18.950 INFO:tasks.workunit.client.1.smithi070.stdout:7/465: mknod d2/d3/dc/d79/caf 0 2024-09-10T04:05:18.959 INFO:tasks.workunit.client.0.smithi046.stdout:2/471: unlink d2/d6/d23/d5d/d77/d7b/d85/c64 0 2024-09-10T04:05:18.968 INFO:tasks.workunit.client.0.smithi046.stdout:6/484: symlink da/db/l9c 0 2024-09-10T04:05:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 -- 172.21.15.46:0/1796041372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 msgr2=0x7f56cc100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 --2- 172.21.15.46:0/1796041372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 0x7f56cc100c60 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009880 tx=0x7f56bc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 -- 172.21.15.46:0/1796041372 shutdown_connections 2024-09-10T04:05:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 --2- 172.21.15.46:0/1796041372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 0x7f56cc101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 --2- 172.21.15.46:0/1796041372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 0x7f56cc100c60 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 -- 172.21.15.46:0/1796041372 >> 172.21.15.46:0/1796041372 conn(0x7f56cc0fc010 msgr2=0x7f56cc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 -- 172.21.15.46:0/1796041372 shutdown_connections 2024-09-10T04:05:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.967+0000 7f56d22a1640 1 -- 172.21.15.46:0/1796041372 wait complete. 2024-09-10T04:05:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.968+0000 7f56d22a1640 1 Processor -- start 2024-09-10T04:05:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.968+0000 7f56d22a1640 1 -- start start 2024-09-10T04:05:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.968+0000 7f56d22a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 0x7f56cc197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.968+0000 7f56d22a1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 0x7f56cc197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56d22a1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc1096a0 con 0x7f56cc100860 2024-09-10T04:05:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56d22a1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f56cc109520 con 0x7f56cc101a60 2024-09-10T04:05:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 0x7f56cc197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:18.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 0x7f56cc197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:37184/0 (socket says 172.21.15.46:37184) 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cb7fe640 1 -- 172.21.15.46:0/545484450 learned_addr learned my addr 172.21.15.46:0/545484450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cbfff640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 0x7f56cc197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cb7fe640 1 -- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 msgr2=0x7f56cc197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cb7fe640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 0x7f56cc197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cb7fe640 1 -- 172.21.15.46:0/545484450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56cc1980e0 con 0x7f56cc101a60 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cbfff640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 0x7f56cc197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56cb7fe640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 0x7f56cc197a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e970 tx=0x7f56c000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.969+0000 7f56c97fa640 1 -- 172.21.15.46:0/545484450 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f56c000cac0 con 0x7f56cc101a60 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.970+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56cc19cb20 con 0x7f56cc101a60 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.970+0000 7f56c97fa640 1 -- 172.21.15.46:0/545484450 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56c00054e0 con 0x7f56cc101a60 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.970+0000 7f56c97fa640 1 -- 172.21.15.46:0/545484450 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f56c0010430 con 0x7f56cc101a60 2024-09-10T04:05:18.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.970+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56cc19d0b0 con 0x7f56cc101a60 2024-09-10T04:05:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.971+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56cc100c60 con 0x7f56cc101a60 2024-09-10T04:05:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.971+0000 7f56c97fa640 1 -- 172.21.15.46:0/545484450 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 19) ==== 92203+0+0 (secure 0 0 0) 0x7f56c0004f00 con 0x7f56cc101a60 2024-09-10T04:05:18.973 INFO:tasks.workunit.client.0.smithi046.stdout:5/560: rename d7/d11/l27 to d7/d11/d3f/d53/d81/lcf 0 2024-09-10T04:05:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.972+0000 7f56c97fa640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f56b406de90 0x7f56b4070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.972+0000 7f56c97fa640 1 -- 172.21.15.46:0/545484450 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5468+0+0 (secure 0 0 0) 0x7f56c008e2c0 con 0x7f56cc101a60 2024-09-10T04:05:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.973+0000 7f56cbfff640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f56b406de90 0x7f56b4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:18.975 INFO:tasks.workunit.client.0.smithi046.stdout:5/561: write d7/d11/d3f/d53/d75/f7c [186269,82163] 0 2024-09-10T04:05:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.974+0000 7f56cbfff640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f56b406de90 0x7f56b4070350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0096c0 tx=0x7f56bc009650 comp rx=0 tx=0).ready entity=mgr.14217 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:18.975 INFO:tasks.workunit.client.1.smithi070.stdout:2/460: symlink d15/la2 0 2024-09-10T04:05:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:18.978+0000 7f56c97fa640 1 -- 172.21.15.46:0/545484450 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56c005be40 con 0x7f56cc101a60 2024-09-10T04:05:18.981 INFO:tasks.workunit.client.1.smithi070.stdout:0/538: write d8/d13/d33/f3f [2222353,20638] 0 2024-09-10T04:05:18.982 INFO:tasks.workunit.client.1.smithi070.stdout:0/539: write d8/d13/d1a/f30 [2589200,72914] 0 2024-09-10T04:05:18.984 INFO:tasks.workunit.client.1.smithi070.stdout:6/579: rename d0/d3/d1f/f22 to d0/d3/d18/d62/d70/fb1 0 2024-09-10T04:05:18.984 INFO:tasks.workunit.client.0.smithi046.stdout:4/573: dwrite d3/d12/d17/d22/f4b [0,4194304] 0 2024-09-10T04:05:18.985 INFO:tasks.workunit.client.1.smithi070.stdout:0/540: dread - d8/d24/d36/d87/fae zero size 2024-09-10T04:05:18.986 INFO:tasks.workunit.client.1.smithi070.stdout:0/541: fsync d8/d24/d2d/f51 0 2024-09-10T04:05:18.987 INFO:tasks.workunit.client.0.smithi046.stdout:4/574: fdatasync d3/d12/d17/d29/d58/db2/d5f/d85/fa2 0 2024-09-10T04:05:18.987 INFO:tasks.workunit.client.1.smithi070.stdout:4/440: dwrite f8 [0,4194304] 0 2024-09-10T04:05:18.994 INFO:tasks.workunit.client.1.smithi070.stdout:5/498: dwrite fc [0,4194304] 0 2024-09-10T04:05:18.994 INFO:tasks.workunit.client.0.smithi046.stdout:8/466: creat d1/d29/d12/f9f x:0 0 0 2024-09-10T04:05:18.999 INFO:tasks.workunit.client.0.smithi046.stdout:5/562: rename d7/d11/d14/d2f/f20 to d7/d11/d3f/d53/d81/fd0 0 2024-09-10T04:05:19.000 INFO:tasks.workunit.client.1.smithi070.stdout:3/510: truncate db/d1c/d25/d28/d43/f58 359243 0 2024-09-10T04:05:19.002 INFO:tasks.workunit.client.0.smithi046.stdout:6/485: mkdir da/db/d11/d13/d1e/d9d 0 2024-09-10T04:05:19.004 INFO:tasks.workunit.client.1.smithi070.stdout:7/466: unlink d2/d7/d9/d18/d3c/f60 0 2024-09-10T04:05:19.006 INFO:tasks.workunit.client.0.smithi046.stdout:2/472: creat d2/d8/fb2 x:0 0 0 2024-09-10T04:05:19.013 INFO:tasks.workunit.client.0.smithi046.stdout:4/575: rename d3/d12/d17/fa3 to d3/d12/d17/d29/d58/db2/fc4 0 2024-09-10T04:05:19.013 INFO:tasks.workunit.client.0.smithi046.stdout:6/486: rename da/dc/d22 to da/dc/d22/d9e 22 2024-09-10T04:05:19.014 INFO:tasks.workunit.client.0.smithi046.stdout:5/563: mknod d7/d11/d14/d2f/d2b/d92/cd1 0 2024-09-10T04:05:19.017 INFO:tasks.workunit.client.0.smithi046.stdout:4/576: write d3/d12/d17/d22/f9d [1035802,111135] 0 2024-09-10T04:05:19.018 INFO:tasks.workunit.client.1.smithi070.stdout:1/418: dread d3/d8/d13/f17 [0,4194304] 0 2024-09-10T04:05:19.018 INFO:tasks.workunit.client.0.smithi046.stdout:8/467: dread d1/d29/fe [4194304,4194304] 0 2024-09-10T04:05:19.022 INFO:tasks.workunit.client.0.smithi046.stdout:0/385: sync 2024-09-10T04:05:19.025 INFO:tasks.workunit.client.0.smithi046.stdout:4/577: creat d3/d12/d17/fc5 x:0 0 0 2024-09-10T04:05:19.027 INFO:tasks.workunit.client.0.smithi046.stdout:6/487: dwrite da/db/d10/d27/f88 [0,4194304] 0 2024-09-10T04:05:19.027 INFO:tasks.workunit.client.0.smithi046.stdout:5/564: creat d7/d11/d3f/d53/da9/fd2 x:0 0 0 2024-09-10T04:05:19.028 INFO:tasks.workunit.client.1.smithi070.stdout:7/467: dread d2/d7/d9/f31 [0,4194304] 0 2024-09-10T04:05:19.028 INFO:tasks.workunit.client.0.smithi046.stdout:2/473: dread d2/f73 [0,4194304] 0 2024-09-10T04:05:19.029 INFO:tasks.workunit.client.0.smithi046.stdout:0/386: write d1/dc/d25/d33/d3/dd/d55/d4d/f71 [3923753,10520] 0 2024-09-10T04:05:19.031 INFO:tasks.workunit.client.0.smithi046.stdout:6/488: stat da/db/d10/d27/f88 0 2024-09-10T04:05:19.033 INFO:tasks.workunit.client.0.smithi046.stdout:4/578: write d3/d12/d17/d22/f59 [1970966,82247] 0 2024-09-10T04:05:19.033 INFO:tasks.workunit.client.1.smithi070.stdout:8/555: write d4/d16/d17/d5c/d87/d8e/fb2 [541181,15698] 0 2024-09-10T04:05:19.033 INFO:tasks.workunit.client.1.smithi070.stdout:5/499: rmdir d17/d6f/d32/d90 39 2024-09-10T04:05:19.039 INFO:tasks.workunit.client.1.smithi070.stdout:3/511: truncate db/d1c/d25/f9a 503769 0 2024-09-10T04:05:19.040 INFO:tasks.workunit.client.0.smithi046.stdout:8/468: dwrite d1/d29/fe [0,4194304] 0 2024-09-10T04:05:19.040 INFO:tasks.workunit.client.1.smithi070.stdout:5/500: dwrite d17/d37/f3d [0,4194304] 0 2024-09-10T04:05:19.044 INFO:tasks.workunit.client.0.smithi046.stdout:6/489: fdatasync da/db/d11/d2b/d5a/f5c 0 2024-09-10T04:05:19.046 INFO:tasks.workunit.client.0.smithi046.stdout:6/490: chown da/db/d11/d2b/f2c 4 1 2024-09-10T04:05:19.048 INFO:tasks.workunit.client.0.smithi046.stdout:6/491: stat da/db/d11/d2b/d2f/f37 0 2024-09-10T04:05:19.050 INFO:tasks.workunit.client.0.smithi046.stdout:2/474: mknod d2/d6/d23/d5d/d77/dac/cb3 0 2024-09-10T04:05:19.051 INFO:tasks.workunit.client.1.smithi070.stdout:5/501: dwrite d17/f75 [0,4194304] 0 2024-09-10T04:05:19.053 INFO:tasks.workunit.client.0.smithi046.stdout:2/475: truncate d2/d6/d23/d55/f7e 827896 0 2024-09-10T04:05:19.054 INFO:tasks.workunit.client.1.smithi070.stdout:1/419: creat d3/d8/d13/d67/f8a x:0 0 0 2024-09-10T04:05:19.057 INFO:tasks.workunit.client.0.smithi046.stdout:4/579: dread d3/d12/d17/d29/f3e [0,4194304] 0 2024-09-10T04:05:19.057 INFO:tasks.workunit.client.1.smithi070.stdout:2/461: dwrite d15/d2b/d94/f9a [0,4194304] 0 2024-09-10T04:05:19.058 INFO:tasks.workunit.client.1.smithi070.stdout:9/551: creat d9/fad x:0 0 0 2024-09-10T04:05:19.059 INFO:tasks.workunit.client.0.smithi046.stdout:5/565: mkdir d7/d11/d3f/dd3 0 2024-09-10T04:05:19.063 INFO:tasks.workunit.client.0.smithi046.stdout:0/387: creat d1/dc/d25/d33/d1a/d64/f7c x:0 0 0 2024-09-10T04:05:19.064 INFO:tasks.workunit.client.1.smithi070.stdout:1/420: dwrite d3/d8/d12/f66 [0,4194304] 0 2024-09-10T04:05:19.065 INFO:tasks.workunit.client.1.smithi070.stdout:7/468: truncate d2/d7/db/f37 2920612 0 2024-09-10T04:05:19.066 INFO:tasks.workunit.client.1.smithi070.stdout:7/469: stat d2/d3/dc/d5d/d2d/c4d 0 2024-09-10T04:05:19.068 INFO:tasks.workunit.client.0.smithi046.stdout:6/492: fdatasync f1 0 2024-09-10T04:05:19.070 INFO:tasks.workunit.client.1.smithi070.stdout:8/556: creat d4/d39/fbb x:0 0 0 2024-09-10T04:05:19.074 INFO:tasks.workunit.client.0.smithi046.stdout:4/580: mkdir d3/d12/d17/dc6 0 2024-09-10T04:05:19.076 INFO:tasks.workunit.client.1.smithi070.stdout:8/557: dwrite d4/d39/f57 [0,4194304] 0 2024-09-10T04:05:19.079 INFO:tasks.workunit.client.0.smithi046.stdout:0/388: truncate d1/dc/d25/d33/f1b 2129917 0 2024-09-10T04:05:19.082 INFO:tasks.workunit.client.1.smithi070.stdout:9/552: symlink d9/d7a/d6f/d90/lae 0 2024-09-10T04:05:19.085 INFO:tasks.workunit.client.0.smithi046.stdout:4/581: mkdir d3/d12/d17/d22/d30/dc7 0 2024-09-10T04:05:19.087 INFO:tasks.workunit.client.1.smithi070.stdout:0/542: creat d8/d13/d1a/faf x:0 0 0 2024-09-10T04:05:19.089 INFO:tasks.workunit.client.1.smithi070.stdout:0/543: write d8/d24/d2d/d8d/f79 [151913,82856] 0 2024-09-10T04:05:19.089 INFO:tasks.workunit.client.0.smithi046.stdout:4/582: truncate d3/fb8 830110 0 2024-09-10T04:05:19.089 INFO:tasks.workunit.client.0.smithi046.stdout:5/566: dread d7/d11/d14/d8e/d96/fbe [0,4194304] 0 2024-09-10T04:05:19.090 INFO:tasks.workunit.client.0.smithi046.stdout:6/493: dwrite da/dc/d22/f34 [0,4194304] 0 2024-09-10T04:05:19.093 INFO:tasks.workunit.client.0.smithi046.stdout:6/494: stat da/db/d11/d2b/d5a/f97 0 2024-09-10T04:05:19.093 INFO:tasks.workunit.client.0.smithi046.stdout:2/476: getdents d2/d8/d37/d63 0 2024-09-10T04:05:19.096 INFO:tasks.workunit.client.1.smithi070.stdout:7/470: unlink d2/d7/d9/d18/d23/f5a 0 2024-09-10T04:05:19.098 INFO:tasks.workunit.client.0.smithi046.stdout:0/389: symlink d1/dc/d25/d72/l7d 0 2024-09-10T04:05:19.100 INFO:tasks.workunit.client.1.smithi070.stdout:4/441: link de/d13/d18/d26/f2f de/d13/d18/d26/d46/d85/f9a 0 2024-09-10T04:05:19.102 INFO:tasks.workunit.client.0.smithi046.stdout:0/390: write d1/dc/d25/d33/d3/f14 [759791,86436] 0 2024-09-10T04:05:19.107 INFO:tasks.workunit.client.0.smithi046.stdout:4/583: rename d3/d12/d17/d22/f27 to d3/d12/d17/d22/d51/d5c/fc8 0 2024-09-10T04:05:19.109 INFO:tasks.workunit.client.0.smithi046.stdout:5/567: truncate d7/d11/d14/d45/d51/f67 108381 0 2024-09-10T04:05:19.110 INFO:tasks.workunit.client.1.smithi070.stdout:6/580: creat d0/d3/d18/d1b/d17/fb2 x:0 0 0 2024-09-10T04:05:19.111 INFO:tasks.workunit.client.0.smithi046.stdout:6/495: mkdir da/db/d11/d9f 0 2024-09-10T04:05:19.111 INFO:tasks.workunit.client.0.smithi046.stdout:0/391: dwrite d1/dc/d25/d33/d1a/d41/f6e [0,4194304] 0 2024-09-10T04:05:19.117 INFO:tasks.workunit.client.1.smithi070.stdout:8/558: creat d4/d16/d17/d19/d62/fbc x:0 0 0 2024-09-10T04:05:19.117 INFO:tasks.workunit.client.0.smithi046.stdout:4/584: dwrite d3/fb8 [0,4194304] 0 2024-09-10T04:05:19.119 INFO:tasks.workunit.client.1.smithi070.stdout:2/462: creat d15/d18/d56/fa3 x:0 0 0 2024-09-10T04:05:19.132 INFO:tasks.workunit.client.1.smithi070.stdout:1/421: dwrite d3/d10/f5e [0,4194304] 0 2024-09-10T04:05:19.134 INFO:tasks.workunit.client.0.smithi046.stdout:4/585: truncate d3/d12/d17/f20 7214195 0 2024-09-10T04:05:19.136 INFO:tasks.workunit.client.0.smithi046.stdout:4/586: write d3/d12/d17/d22/d51/d94/fbc [448342,41062] 0 2024-09-10T04:05:19.140 INFO:tasks.workunit.client.1.smithi070.stdout:6/581: chown d0/d3/d8c/d9a/d92/l9b 88569 1 2024-09-10T04:05:19.142 INFO:tasks.workunit.client.1.smithi070.stdout:5/502: getdents d17/d6f/d32/d90/d99 0 2024-09-10T04:05:19.144 INFO:tasks.workunit.client.0.smithi046.stdout:5/568: truncate d7/d11/d3f/d53/d75/f93 611137 0 2024-09-10T04:05:19.146 INFO:tasks.workunit.client.1.smithi070.stdout:6/582: dwrite d0/d3/d18/f27 [0,4194304] 0 2024-09-10T04:05:19.146 INFO:tasks.workunit.client.1.smithi070.stdout:8/559: creat d4/d16/d17/d5c/d74/da4/fbd x:0 0 0 2024-09-10T04:05:19.149 INFO:tasks.workunit.client.1.smithi070.stdout:2/463: creat d15/d18/d56/fa4 x:0 0 0 2024-09-10T04:05:19.151 INFO:tasks.workunit.client.0.smithi046.stdout:0/392: creat d1/dc/d25/d33/f7e x:0 0 0 2024-09-10T04:05:19.151 INFO:tasks.workunit.client.1.smithi070.stdout:9/553: write d9/d1c/d34/f76 [3913239,114263] 0 2024-09-10T04:05:19.152 INFO:tasks.workunit.client.1.smithi070.stdout:9/554: fdatasync d9/d59/f65 0 2024-09-10T04:05:19.153 INFO:tasks.workunit.client.0.smithi046.stdout:4/587: chown d3/d12/d17/d29/ca0 45612656 1 2024-09-10T04:05:19.159 INFO:tasks.workunit.client.1.smithi070.stdout:7/471: unlink d2/d7/d9/d18/d3c/c98 0 2024-09-10T04:05:19.162 INFO:tasks.workunit.client.0.smithi046.stdout:4/588: dwrite d3/fa [0,4194304] 0 2024-09-10T04:05:19.174 INFO:tasks.workunit.client.0.smithi046.stdout:4/589: rename d3/d12/d17/d29/d58/db2/f8d to d3/d12/d17/d22/d51/d5c/fc9 0 2024-09-10T04:05:19.174 INFO:tasks.workunit.client.1.smithi070.stdout:9/555: dread d9/d1c/f55 [0,4194304] 0 2024-09-10T04:05:19.175 INFO:tasks.workunit.client.0.smithi046.stdout:4/590: readlink d3/d12/d17/d22/d30/l5d 0 2024-09-10T04:05:19.177 INFO:tasks.workunit.client.1.smithi070.stdout:0/544: write d8/d85/fac [900124,98629] 0 2024-09-10T04:05:19.181 INFO:tasks.workunit.client.0.smithi046.stdout:5/569: truncate d7/d11/d14/d2f/d2b/f7a 987740 0 2024-09-10T04:05:19.184 INFO:tasks.workunit.client.0.smithi046.stdout:0/393: truncate d1/d13/f53 3757892 0 2024-09-10T04:05:19.187 INFO:tasks.workunit.client.0.smithi046.stdout:4/591: getdents d3/d12 0 2024-09-10T04:05:19.188 INFO:tasks.workunit.client.1.smithi070.stdout:8/560: truncate d4/d16/d17/d1a/f58 179888 0 2024-09-10T04:05:19.189 INFO:tasks.workunit.client.0.smithi046.stdout:5/570: creat d7/d11/d3f/fd4 x:0 0 0 2024-09-10T04:05:19.189 INFO:tasks.workunit.client.1.smithi070.stdout:8/561: write d4/d16/d17/d1a/f88 [4454572,101612] 0 2024-09-10T04:05:19.190 INFO:tasks.workunit.client.1.smithi070.stdout:2/464: rmdir d15 39 2024-09-10T04:05:19.192 INFO:tasks.workunit.client.0.smithi046.stdout:5/571: dread d7/d11/d14/d2f/da3/fbf [0,4194304] 0 2024-09-10T04:05:19.196 INFO:tasks.workunit.client.1.smithi070.stdout:7/472: chown d2/d7/db/d3a/d7e/c8a 3 1 2024-09-10T04:05:19.197 INFO:tasks.workunit.client.1.smithi070.stdout:1/422: mknod d3/d8/d13/d67/d83/c8b 0 2024-09-10T04:05:19.197 INFO:tasks.workunit.client.1.smithi070.stdout:2/465: dwrite f10 [0,4194304] 0 2024-09-10T04:05:19.197 INFO:tasks.workunit.client.1.smithi070.stdout:4/442: creat de/f9b x:0 0 0 2024-09-10T04:05:19.197 INFO:tasks.workunit.client.1.smithi070.stdout:9/556: mkdir d9/d59/d2a/d3f/daf 0 2024-09-10T04:05:19.198 INFO:tasks.workunit.client.1.smithi070.stdout:9/557: fsync d9/f68 0 2024-09-10T04:05:19.198 INFO:tasks.workunit.client.1.smithi070.stdout:0/545: mknod d8/d13/d1a/d8f/cb0 0 2024-09-10T04:05:19.199 INFO:tasks.workunit.client.0.smithi046.stdout:3/464: sync 2024-09-10T04:05:19.199 INFO:tasks.workunit.client.0.smithi046.stdout:9/478: sync 2024-09-10T04:05:19.199 INFO:tasks.workunit.client.0.smithi046.stdout:7/495: sync 2024-09-10T04:05:19.199 INFO:tasks.workunit.client.0.smithi046.stdout:1/392: sync 2024-09-10T04:05:19.199 INFO:tasks.workunit.client.1.smithi070.stdout:0/546: chown d8/d13/d1a/f23 7 1 2024-09-10T04:05:19.200 INFO:tasks.workunit.client.1.smithi070.stdout:3/512: getdents db 0 2024-09-10T04:05:19.200 INFO:tasks.workunit.client.0.smithi046.stdout:9/479: dread - d4/d66/d7c/f7d zero size 2024-09-10T04:05:19.201 INFO:tasks.workunit.client.1.smithi070.stdout:3/513: fsync db/d1c/d3e/f50 0 2024-09-10T04:05:19.201 INFO:tasks.workunit.client.1.smithi070.stdout:8/562: dread d4/f12 [0,4194304] 0 2024-09-10T04:05:19.205 INFO:tasks.workunit.client.1.smithi070.stdout:6/583: mknod d0/d3/d1f/d93/cb3 0 2024-09-10T04:05:19.206 INFO:tasks.workunit.client.1.smithi070.stdout:6/584: chown d0/d3/d1f/d81/la8 3654767 1 2024-09-10T04:05:19.213 INFO:tasks.workunit.client.0.smithi046.stdout:7/496: creat d5/d6/d39/fa0 x:0 0 0 2024-09-10T04:05:19.218 INFO:tasks.workunit.client.0.smithi046.stdout:3/465: truncate d12/d13/d37/f60 928106 0 2024-09-10T04:05:19.218 INFO:tasks.workunit.client.0.smithi046.stdout:4/592: dwrite d3/d12/d17/d29/d58/db2/f48 [0,4194304] 0 2024-09-10T04:05:19.220 INFO:tasks.workunit.client.1.smithi070.stdout:2/466: dread - d15/d18/d29/d64/f95 zero size 2024-09-10T04:05:19.223 INFO:tasks.workunit.client.1.smithi070.stdout:9/558: mknod d9/d59/d5d/cb0 0 2024-09-10T04:05:19.225 INFO:tasks.workunit.client.0.smithi046.stdout:1/393: creat d3/dc/d34/d51/f83 x:0 0 0 2024-09-10T04:05:19.225 INFO:tasks.workunit.client.0.smithi046.stdout:4/593: dread d3/d12/d17/d29/fbb [0,4194304] 0 2024-09-10T04:05:19.226 INFO:tasks.workunit.client.0.smithi046.stdout:1/394: stat d3/d27/d5d/d60 0 2024-09-10T04:05:19.228 INFO:tasks.workunit.client.1.smithi070.stdout:0/547: rename d8/d13/c61 to d8/d24/d2d/d34/d9b/cb1 0 2024-09-10T04:05:19.231 INFO:tasks.workunit.client.0.smithi046.stdout:5/572: write d7/d11/d3f/d53/da9/f83 [189182,58005] 0 2024-09-10T04:05:19.233 INFO:tasks.workunit.client.0.smithi046.stdout:5/573: truncate d7/d11/d3f/d53/f47 1233860 0 2024-09-10T04:05:19.236 INFO:tasks.workunit.client.1.smithi070.stdout:8/563: dwrite d4/f77 [4194304,4194304] 0 2024-09-10T04:05:19.239 INFO:tasks.workunit.client.0.smithi046.stdout:4/594: mkdir d3/dab/dca 0 2024-09-10T04:05:19.242 INFO:tasks.workunit.client.1.smithi070.stdout:1/423: dwrite d3/d10/d1d/f30 [0,4194304] 0 2024-09-10T04:05:19.243 INFO:tasks.workunit.client.1.smithi070.stdout:1/424: stat d3/d8/d13/f17 0 2024-09-10T04:05:19.246 INFO:tasks.workunit.client.1.smithi070.stdout:4/443: symlink de/d13/d18/d26/d8c/d93/l9c 0 2024-09-10T04:05:19.263 INFO:tasks.workunit.client.0.smithi046.stdout:9/480: rename d4/d29/d32/l37 to d4/d7/d1f/l91 0 2024-09-10T04:05:19.264 INFO:tasks.workunit.client.1.smithi070.stdout:2/467: mknod d15/d18/d56/ca5 0 2024-09-10T04:05:19.265 INFO:tasks.workunit.client.1.smithi070.stdout:2/468: chown d15/d18/d29/d2a/d2e/c5c 41980 1 2024-09-10T04:05:19.267 INFO:tasks.workunit.client.1.smithi070.stdout:9/559: rename d9/d59/f7e to d9/d1c/dac/d8a/d3c/d92/fb1 0 2024-09-10T04:05:19.269 INFO:tasks.workunit.client.1.smithi070.stdout:0/548: read - d8/d13/d1a/f45 zero size 2024-09-10T04:05:19.269 INFO:tasks.workunit.client.1.smithi070.stdout:7/473: truncate d2/f24 3281587 0 2024-09-10T04:05:19.270 INFO:tasks.workunit.client.1.smithi070.stdout:0/549: chown d8/d13/d1a 0 1 2024-09-10T04:05:19.271 INFO:tasks.workunit.client.1.smithi070.stdout:5/503: getdents d17/d19/d67 0 2024-09-10T04:05:19.272 INFO:tasks.workunit.client.0.smithi046.stdout:7/497: dwrite d5/f60 [0,4194304] 0 2024-09-10T04:05:19.273 INFO:tasks.workunit.client.1.smithi070.stdout:9/560: dwrite d9/d1c/f87 [0,4194304] 0 2024-09-10T04:05:19.274 INFO:tasks.workunit.client.1.smithi070.stdout:9/561: dread - d9/d1c/dac/d8a/d3c/d62/f8d zero size 2024-09-10T04:05:19.275 INFO:tasks.workunit.client.0.smithi046.stdout:4/595: creat d3/d12/d17/d29/fcb x:0 0 0 2024-09-10T04:05:19.276 INFO:tasks.workunit.client.0.smithi046.stdout:1/395: link d3/d27/d32/f69 d3/dc/f84 0 2024-09-10T04:05:19.277 INFO:tasks.workunit.client.1.smithi070.stdout:8/564: mkdir d4/d16/d17/d1a/d60/d7a/d84/dbe 0 2024-09-10T04:05:19.278 INFO:tasks.workunit.client.1.smithi070.stdout:8/565: chown d4/d16/d17/d1a/f2e 91001757 1 2024-09-10T04:05:19.287 INFO:tasks.workunit.client.1.smithi070.stdout:8/566: dread d4/d16/d17/d5c/f6d [0,4194304] 0 2024-09-10T04:05:19.288 INFO:tasks.workunit.client.1.smithi070.stdout:8/567: chown d4/d16/d17/l2d 336 1 2024-09-10T04:05:19.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.302+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 --> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f56cc06a420 con 0x7f56b406de90 2024-09-10T04:05:19.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.303+0000 7f56c97fa640 1 -- 172.21.15.46:0/545484450 <== mgr.14217 v2:172.21.15.46:6800/3538810660 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f56cc06a420 con 0x7f56b406de90 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [], 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "0/2 daemons upgraded", 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mgr daemons", 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:05:19.305 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:05:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f56b406de90 msgr2=0x7f56b4070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f56b406de90 0x7f56b4070350 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0096c0 tx=0x7f56bc009650 comp rx=0 tx=0).stop 2024-09-10T04:05:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 msgr2=0x7f56cc197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 0x7f56cc197a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f56c000e970 tx=0x7f56c000ed70 comp rx=0 tx=0).stop 2024-09-10T04:05:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 shutdown_connections 2024-09-10T04:05:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:6800/3538810660,v1:172.21.15.46:6801/3538810660] conn(0x7f56b406de90 0x7f56b4070350 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:19.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56cc101a60 0x7f56cc197a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:19.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 --2- 172.21.15.46:0/545484450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56cc100860 0x7f56cc197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:19.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 >> 172.21.15.46:0/545484450 conn(0x7f56cc0fc010 msgr2=0x7f56cc069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:19.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 shutdown_connections 2024-09-10T04:05:19.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:19.305+0000 7f56d22a1640 1 -- 172.21.15.46:0/545484450 wait complete. 2024-09-10T04:05:19.446 INFO:tasks.workunit.client.1.smithi070.stdout:4/444: symlink de/d13/d18/d23/d47/d59/l9d 0 2024-09-10T04:05:19.448 INFO:tasks.workunit.client.0.smithi046.stdout:5/574: mkdir d7/d11/d3f/d53/da9/d9c/dd5 0 2024-09-10T04:05:19.449 INFO:tasks.workunit.client.0.smithi046.stdout:5/575: chown d7/d11/d14/d2f/l77 2 1 2024-09-10T04:05:19.453 INFO:tasks.workunit.client.0.smithi046.stdout:4/596: creat d3/d12/d17/d22/d51/fcc x:0 0 0 2024-09-10T04:05:19.455 INFO:tasks.workunit.client.0.smithi046.stdout:1/396: truncate d3/dc/f10 3968312 0 2024-09-10T04:05:19.456 INFO:tasks.workunit.client.1.smithi070.stdout:7/474: creat d2/d7/d9/d18/d3c/d52/fb0 x:0 0 0 2024-09-10T04:05:19.459 INFO:tasks.workunit.client.1.smithi070.stdout:0/550: rename d8/d13/d33/d74/c8e to d8/d24/d2d/d34/d6b/cb2 0 2024-09-10T04:05:19.460 INFO:tasks.workunit.client.1.smithi070.stdout:0/551: write d8/d13/d1a/d8f/d95/faa [661604,111041] 0 2024-09-10T04:05:19.461 INFO:tasks.workunit.client.1.smithi070.stdout:6/585: creat d0/d3/d18/d1b/d17/d53/fb4 x:0 0 0 2024-09-10T04:05:19.461 INFO:tasks.workunit.client.0.smithi046.stdout:3/466: link d12/d13/d37/c90 d12/d5d/d71/d49/c99 0 2024-09-10T04:05:19.462 INFO:tasks.workunit.client.1.smithi070.stdout:6/586: chown d0/d3/d18/d62/d70/f7c 98635992 1 2024-09-10T04:05:19.463 INFO:tasks.workunit.client.1.smithi070.stdout:5/504: dread f14 [0,4194304] 0 2024-09-10T04:05:19.463 INFO:tasks.workunit.client.1.smithi070.stdout:9/562: symlink d9/d7a/d6f/d90/lb2 0 2024-09-10T04:05:19.463 INFO:tasks.workunit.client.1.smithi070.stdout:6/587: write d0/f8 [4687975,60587] 0 2024-09-10T04:05:19.465 INFO:tasks.workunit.client.1.smithi070.stdout:5/505: truncate d17/d37/f91 165101 0 2024-09-10T04:05:19.466 INFO:tasks.workunit.client.0.smithi046.stdout:1/397: rename d3/d27/d32/d28/l6c to d3/dc/d21/d3a/l85 0 2024-09-10T04:05:19.467 INFO:tasks.workunit.client.1.smithi070.stdout:4/445: mkdir de/d13/d5b/d9e 0 2024-09-10T04:05:19.468 INFO:tasks.workunit.client.0.smithi046.stdout:1/398: read d3/d27/d32/f43 [3871975,115878] 0 2024-09-10T04:05:19.469 INFO:tasks.workunit.client.0.smithi046.stdout:5/576: unlink d7/d11/d14/d2f/d2b/d6a/c79 0 2024-09-10T04:05:19.470 INFO:tasks.workunit.client.1.smithi070.stdout:2/469: mkdir d15/d18/d29/d2a/d2e/da6 0 2024-09-10T04:05:19.470 INFO:tasks.workunit.client.1.smithi070.stdout:9/563: dwrite d9/d1c/d34/f33 [4194304,4194304] 0 2024-09-10T04:05:19.470 INFO:tasks.workunit.client.0.smithi046.stdout:4/597: mkdir d3/dab/db9/dcd 0 2024-09-10T04:05:19.471 INFO:tasks.workunit.client.0.smithi046.stdout:4/598: chown d3/d12/d17/d22/d30/c89 692736096 1 2024-09-10T04:05:19.472 INFO:tasks.workunit.client.0.smithi046.stdout:3/467: dread d12/d13/d14/f36 [0,4194304] 0 2024-09-10T04:05:19.472 INFO:tasks.workunit.client.0.smithi046.stdout:8/469: sync 2024-09-10T04:05:19.472 INFO:tasks.workunit.client.0.smithi046.stdout:2/477: sync 2024-09-10T04:05:19.473 INFO:tasks.workunit.client.1.smithi070.stdout:3/514: sync 2024-09-10T04:05:19.473 INFO:tasks.workunit.client.0.smithi046.stdout:9/481: getdents d4/d7 0 2024-09-10T04:05:19.474 INFO:tasks.workunit.client.0.smithi046.stdout:1/399: readlink d3/d27/d32/d1f/l81 0 2024-09-10T04:05:19.475 INFO:tasks.workunit.client.0.smithi046.stdout:5/577: rmdir d7/d11/d3f/d5a 39 2024-09-10T04:05:19.478 INFO:tasks.workunit.client.0.smithi046.stdout:5/578: chown d7/d11/d3f/d53/f9e 96 1 2024-09-10T04:05:19.479 INFO:tasks.workunit.client.1.smithi070.stdout:5/506: rmdir d17/d24 39 2024-09-10T04:05:19.479 INFO:tasks.workunit.client.1.smithi070.stdout:4/446: dread de/d13/d18/d26/f42 [0,4194304] 0 2024-09-10T04:05:19.481 INFO:tasks.workunit.client.0.smithi046.stdout:2/478: mkdir d2/d6/d23/d42/db4 0 2024-09-10T04:05:19.486 INFO:tasks.workunit.client.1.smithi070.stdout:2/470: dread d15/d2b/f4c [0,4194304] 0 2024-09-10T04:05:19.486 INFO:tasks.workunit.client.0.smithi046.stdout:5/579: truncate d7/d11/d3f/d5a/d63/f65 4237078 0 2024-09-10T04:05:19.486 INFO:tasks.workunit.client.1.smithi070.stdout:0/552: symlink d8/d13/lb3 0 2024-09-10T04:05:19.487 INFO:tasks.workunit.client.0.smithi046.stdout:9/482: mkdir d4/d18/d92 0 2024-09-10T04:05:19.488 INFO:tasks.workunit.client.0.smithi046.stdout:5/580: chown d7/d11/d14/d2f/d2b/l5c 913 1 2024-09-10T04:05:19.488 INFO:tasks.workunit.client.0.smithi046.stdout:3/468: creat d12/d1b/d4b/d91/f9a x:0 0 0 2024-09-10T04:05:19.497 INFO:tasks.workunit.client.0.smithi046.stdout:5/581: creat d7/d11/d14/d2f/fd6 x:0 0 0 2024-09-10T04:05:19.498 INFO:tasks.workunit.client.0.smithi046.stdout:5/582: write d7/d11/d3f/d53/f9b [40170,38883] 0 2024-09-10T04:05:19.501 INFO:tasks.workunit.client.0.smithi046.stdout:3/469: symlink d12/d13/d14/d1e/d2d/d39/l9b 0 2024-09-10T04:05:19.504 INFO:tasks.workunit.client.0.smithi046.stdout:5/583: dwrite d7/d11/d14/f29 [0,4194304] 0 2024-09-10T04:05:19.506 INFO:tasks.workunit.client.0.smithi046.stdout:4/599: getdents d3/dab 0 2024-09-10T04:05:19.514 INFO:tasks.workunit.client.0.smithi046.stdout:9/483: creat d4/d3c/d4d/d5c/f93 x:0 0 0 2024-09-10T04:05:19.516 INFO:tasks.workunit.client.0.smithi046.stdout:4/600: truncate d3/d12/d17/d29/d58/db2/f98 469434 0 2024-09-10T04:05:19.517 INFO:tasks.workunit.client.0.smithi046.stdout:3/470: unlink d12/d13/d37/f54 0 2024-09-10T04:05:19.518 INFO:tasks.workunit.client.0.smithi046.stdout:9/484: rmdir d4/d44 39 2024-09-10T04:05:19.518 INFO:tasks.workunit.client.0.smithi046.stdout:3/471: read d12/d13/f17 [132511,15601] 0 2024-09-10T04:05:19.519 INFO:tasks.workunit.client.0.smithi046.stdout:4/601: creat d3/d12/d17/d29/d58/db2/d5f/d85/da8/fce x:0 0 0 2024-09-10T04:05:19.522 INFO:tasks.workunit.client.0.smithi046.stdout:3/472: rename d12/d1b to d12/d13/d37/d9c 0 2024-09-10T04:05:19.524 INFO:tasks.workunit.client.0.smithi046.stdout:4/602: fsync d3/d12/d17/d29/f4f 0 2024-09-10T04:05:19.525 INFO:tasks.workunit.client.0.smithi046.stdout:8/470: write d1/d29/d12/f1e [191637,92368] 0 2024-09-10T04:05:19.526 INFO:tasks.workunit.client.0.smithi046.stdout:9/485: creat d4/d44/f94 x:0 0 0 2024-09-10T04:05:19.529 INFO:tasks.workunit.client.0.smithi046.stdout:2/479: dwrite d2/d6/d23/d5d/d77/f69 [0,4194304] 0 2024-09-10T04:05:19.532 INFO:tasks.workunit.client.0.smithi046.stdout:4/603: mknod d3/d12/d17/d29/d58/db2/ccf 0 2024-09-10T04:05:19.538 INFO:tasks.workunit.client.0.smithi046.stdout:5/584: write d7/fc [2165498,110933] 0 2024-09-10T04:05:19.538 INFO:tasks.workunit.client.0.smithi046.stdout:9/486: dread d4/d29/d32/f63 [0,4194304] 0 2024-09-10T04:05:19.539 INFO:tasks.workunit.client.0.smithi046.stdout:9/487: chown d4/d66/d7c/f7d 23994428 1 2024-09-10T04:05:19.542 INFO:tasks.workunit.client.0.smithi046.stdout:2/480: creat d2/d57/fb5 x:0 0 0 2024-09-10T04:05:19.545 INFO:tasks.workunit.client.0.smithi046.stdout:5/585: creat d7/d11/d14/d2f/d64/fd7 x:0 0 0 2024-09-10T04:05:19.548 INFO:tasks.workunit.client.0.smithi046.stdout:9/488: mkdir d4/d29/d32/d95 0 2024-09-10T04:05:19.548 INFO:tasks.workunit.client.0.smithi046.stdout:2/481: dread - d2/d6/d23/d55/f67 zero size 2024-09-10T04:05:19.551 INFO:tasks.workunit.client.0.smithi046.stdout:8/471: write d1/fa [1889035,41686] 0 2024-09-10T04:05:19.552 INFO:tasks.workunit.client.1.smithi070.stdout:6/588: mkdir d0/d3/d18/d1b/d17/d54/db5 0 2024-09-10T04:05:19.552 INFO:tasks.workunit.client.1.smithi070.stdout:1/425: link d3/d8/d12/d2f/d32/d4b/l85 d3/d8/d12/d61/l8c 0 2024-09-10T04:05:19.553 INFO:tasks.workunit.client.0.smithi046.stdout:9/489: creat d4/d7/d1f/d54/d56/f96 x:0 0 0 2024-09-10T04:05:19.553 INFO:tasks.workunit.client.0.smithi046.stdout:3/473: link d12/d13/d37/d9c/d50/c83 d12/d13/d37/d9c/d50/c9d 0 2024-09-10T04:05:19.553 INFO:tasks.workunit.client.1.smithi070.stdout:8/568: rmdir d4/d16/d17/d1a/d60/d7a/d84/dbe 0 2024-09-10T04:05:19.554 INFO:tasks.workunit.client.1.smithi070.stdout:1/426: write d3/d8/fc [9214506,66445] 0 2024-09-10T04:05:19.554 INFO:tasks.workunit.client.1.smithi070.stdout:3/515: mknod db/d1c/cae 0 2024-09-10T04:05:19.555 INFO:tasks.workunit.client.0.smithi046.stdout:3/474: stat d12/d13/d14/d1e/l56 0 2024-09-10T04:05:19.556 INFO:tasks.workunit.client.0.smithi046.stdout:5/586: rename d7/d11/f60 to d7/d11/d14/d2f/d2b/d6a/fd8 0 2024-09-10T04:05:19.556 INFO:tasks.workunit.client.1.smithi070.stdout:8/569: write d4/d16/d17/d19/d62/fbc [357499,18221] 0 2024-09-10T04:05:19.558 INFO:tasks.workunit.client.0.smithi046.stdout:8/472: symlink d1/d29/la0 0 2024-09-10T04:05:19.559 INFO:tasks.workunit.client.1.smithi070.stdout:5/507: unlink d17/d24/d77/f8c 0 2024-09-10T04:05:19.560 INFO:tasks.workunit.client.0.smithi046.stdout:9/490: creat d4/d3c/f97 x:0 0 0 2024-09-10T04:05:19.561 INFO:tasks.workunit.client.1.smithi070.stdout:1/427: dwrite d3/d8/d13/d67/f8a [0,4194304] 0 2024-09-10T04:05:19.562 INFO:tasks.workunit.client.1.smithi070.stdout:1/428: truncate d3/f47 5131908 0 2024-09-10T04:05:19.564 INFO:tasks.workunit.client.1.smithi070.stdout:4/447: rename de/d13/d18/d23/l5e to de/l9f 0 2024-09-10T04:05:19.565 INFO:tasks.workunit.client.0.smithi046.stdout:9/491: write d4/d44/f94 [620109,55893] 0 2024-09-10T04:05:19.567 INFO:tasks.workunit.client.0.smithi046.stdout:3/475: fdatasync d12/f4d 0 2024-09-10T04:05:19.568 INFO:tasks.workunit.client.0.smithi046.stdout:2/482: dread d2/d8/d37/f41 [0,4194304] 0 2024-09-10T04:05:19.570 INFO:tasks.workunit.client.0.smithi046.stdout:5/587: dwrite d7/d11/d14/d45/fc2 [0,4194304] 0 2024-09-10T04:05:19.573 INFO:tasks.workunit.client.0.smithi046.stdout:4/604: dwrite d3/d12/d17/f20 [4194304,4194304] 0 2024-09-10T04:05:19.573 INFO:tasks.workunit.client.0.smithi046.stdout:8/473: creat d1/d29/d19/d36/d4c/d5e/fa1 x:0 0 0 2024-09-10T04:05:19.575 INFO:tasks.workunit.client.0.smithi046.stdout:5/588: stat d7/d11/d3f/d5a/d63/f7f 0 2024-09-10T04:05:19.576 INFO:tasks.workunit.client.1.smithi070.stdout:5/508: mkdir d17/d19/d6a/d54/d57/da8 0 2024-09-10T04:05:19.577 INFO:tasks.workunit.client.0.smithi046.stdout:4/605: chown d3/d12/d17/d22/d51/d5c/d6c/lb7 252300606 1 2024-09-10T04:05:19.579 INFO:tasks.workunit.client.0.smithi046.stdout:8/474: dread - d1/d29/d12/f9f zero size 2024-09-10T04:05:19.582 INFO:tasks.workunit.client.1.smithi070.stdout:1/429: dread d3/d8/d13/f18 [0,4194304] 0 2024-09-10T04:05:19.583 INFO:tasks.workunit.client.0.smithi046.stdout:2/483: unlink d2/d8/c99 0 2024-09-10T04:05:19.584 INFO:tasks.workunit.client.1.smithi070.stdout:6/589: write d0/d3/d18/d1b/d33/f65 [265153,113507] 0 2024-09-10T04:05:19.584 INFO:tasks.workunit.client.0.smithi046.stdout:2/484: fsync d2/d6/f61 0 2024-09-10T04:05:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:19 smithi070 ceph-mon[35495]: pgmap v263: 65 pgs: 65 active+clean; 1.8 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 52 MiB/s rd, 253 MiB/s wr, 410 op/s 2024-09-10T04:05:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:19 smithi070 ceph-mon[35495]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:19 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:19 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/683730066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:05:19.585 INFO:tasks.workunit.client.1.smithi070.stdout:2/471: creat d15/fa7 x:0 0 0 2024-09-10T04:05:19.585 INFO:tasks.workunit.client.1.smithi070.stdout:6/590: read - d0/d3/f99 zero size 2024-09-10T04:05:19.586 INFO:tasks.workunit.client.0.smithi046.stdout:5/589: dwrite d7/d11/d3f/d53/d75/fbd [0,4194304] 0 2024-09-10T04:05:19.586 INFO:tasks.workunit.client.1.smithi070.stdout:2/472: chown d15/d18/d56/d7c/d55 686863120 1 2024-09-10T04:05:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:19 smithi046 ceph-mon[28328]: pgmap v263: 65 pgs: 65 active+clean; 1.8 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 52 MiB/s rd, 253 MiB/s wr, 410 op/s 2024-09-10T04:05:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:19 smithi046 ceph-mon[28328]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:19 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:19 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/683730066' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:05:19.588 INFO:tasks.workunit.client.0.smithi046.stdout:3/476: chown d12/d13/d37/d9c/d50/d52/d86/c8c 29791726 1 2024-09-10T04:05:19.589 INFO:tasks.workunit.client.1.smithi070.stdout:9/564: getdents d9/d1c/dac 0 2024-09-10T04:05:19.590 INFO:tasks.workunit.client.1.smithi070.stdout:3/516: mknod db/d1c/caf 0 2024-09-10T04:05:19.593 INFO:tasks.workunit.client.1.smithi070.stdout:0/553: rename d8/d13/d33/d53/f57 to d8/d24/d97/fb4 0 2024-09-10T04:05:19.594 INFO:tasks.workunit.client.1.smithi070.stdout:5/509: symlink d17/d19/d35/la9 0 2024-09-10T04:05:19.595 INFO:tasks.workunit.client.1.smithi070.stdout:1/430: symlink d3/d8/d12/d2f/d32/d4b/d54/l8d 0 2024-09-10T04:05:19.597 INFO:tasks.workunit.client.0.smithi046.stdout:5/590: dread - d7/d11/d3f/d53/d81/fd0 zero size 2024-09-10T04:05:19.597 INFO:tasks.workunit.client.1.smithi070.stdout:0/554: write d8/d13/f19 [1664704,104411] 0 2024-09-10T04:05:19.598 INFO:tasks.workunit.client.1.smithi070.stdout:2/473: dwrite d15/d18/d29/d64/f95 [0,4194304] 0 2024-09-10T04:05:19.599 INFO:tasks.workunit.client.0.smithi046.stdout:5/591: fsync d7/d11/d3f/d53/da9/d9c/da5/fb6 0 2024-09-10T04:05:19.601 INFO:tasks.workunit.client.1.smithi070.stdout:2/474: fdatasync d15/d18/d29/d2a/d2e/f87 0 2024-09-10T04:05:19.602 INFO:tasks.workunit.client.1.smithi070.stdout:9/565: unlink d9/l37 0 2024-09-10T04:05:19.604 INFO:tasks.workunit.client.1.smithi070.stdout:3/517: dwrite db/d1c/d89/f8f [0,4194304] 0 2024-09-10T04:05:19.606 INFO:tasks.workunit.client.0.smithi046.stdout:8/475: mkdir d1/da2 0 2024-09-10T04:05:19.611 INFO:tasks.workunit.client.0.smithi046.stdout:4/606: creat d3/d12/d17/d29/d58/db2/d7a/fd0 x:0 0 0 2024-09-10T04:05:19.613 INFO:tasks.workunit.client.0.smithi046.stdout:3/477: mknod d12/c9e 0 2024-09-10T04:05:19.613 INFO:tasks.workunit.client.1.smithi070.stdout:6/591: dread d0/d3/f16 [0,4194304] 0 2024-09-10T04:05:19.613 INFO:tasks.workunit.client.1.smithi070.stdout:6/592: fsync d0/d3/d18/d1b/f3a 0 2024-09-10T04:05:19.618 INFO:tasks.workunit.client.0.smithi046.stdout:9/492: link d4/d7/d1f/c7b d4/d7/d1f/d54/d56/c98 0 2024-09-10T04:05:19.619 INFO:tasks.workunit.client.0.smithi046.stdout:8/476: dread - d1/d29/d12/f97 zero size 2024-09-10T04:05:19.620 INFO:tasks.workunit.client.1.smithi070.stdout:1/431: readlink d3/d8/l51 0 2024-09-10T04:05:19.623 INFO:tasks.workunit.client.1.smithi070.stdout:5/510: creat d17/d37/faa x:0 0 0 2024-09-10T04:05:19.623 INFO:tasks.workunit.client.1.smithi070.stdout:5/511: readlink d17/d19/d6a/d54/l94 0 2024-09-10T04:05:19.623 INFO:tasks.workunit.client.0.smithi046.stdout:3/478: mkdir d12/d5d/d71/d49/d5c/d9f 0 2024-09-10T04:05:19.624 INFO:tasks.workunit.client.1.smithi070.stdout:5/512: fsync d17/d24/d2b/f81 0 2024-09-10T04:05:19.629 INFO:tasks.workunit.client.0.smithi046.stdout:3/479: dread f4 [0,4194304] 0 2024-09-10T04:05:19.630 INFO:tasks.workunit.client.1.smithi070.stdout:5/513: dwrite d17/d19/d6a/d54/d57/f65 [0,4194304] 0 2024-09-10T04:05:19.630 INFO:tasks.workunit.client.0.smithi046.stdout:4/607: symlink d3/dab/db9/dcd/ld1 0 2024-09-10T04:05:19.632 INFO:tasks.workunit.client.1.smithi070.stdout:2/475: rename d15/d18/d56/d7c/d55/d6d/f96 to d15/d18/d56/d7c/d55/d6d/fa8 0 2024-09-10T04:05:19.633 INFO:tasks.workunit.client.0.smithi046.stdout:5/592: dwrite d7/d11/d3f/d5a/d63/fab [0,4194304] 0 2024-09-10T04:05:19.634 INFO:tasks.workunit.client.0.smithi046.stdout:9/493: write d4/d7/f8 [1589311,61398] 0 2024-09-10T04:05:19.638 INFO:tasks.workunit.client.0.smithi046.stdout:3/480: mkdir d12/d13/d14/d1e/d2d/d39/d67/da0 0 2024-09-10T04:05:19.640 INFO:tasks.workunit.client.0.smithi046.stdout:4/608: creat d3/d12/d14/fd2 x:0 0 0 2024-09-10T04:05:19.642 INFO:tasks.workunit.client.0.smithi046.stdout:4/609: truncate d3/d12/d17/d22/d30/faa 741854 0 2024-09-10T04:05:19.644 INFO:tasks.workunit.client.0.smithi046.stdout:4/610: chown d3/d12/d17/d22/d51/d5c/d6c/lb7 264568659 1 2024-09-10T04:05:19.646 INFO:tasks.workunit.client.0.smithi046.stdout:5/593: read d7/d11/d3f/d53/d75/f7c [329782,59753] 0 2024-09-10T04:05:19.646 INFO:tasks.workunit.client.0.smithi046.stdout:4/611: read d3/d12/f9a [660026,119311] 0 2024-09-10T04:05:19.648 INFO:tasks.workunit.client.0.smithi046.stdout:8/477: dread d1/d29/d17/d1f/f31 [0,4194304] 0 2024-09-10T04:05:19.648 INFO:tasks.workunit.client.0.smithi046.stdout:4/612: chown d3/d12/d17/d29/d58 11025314 1 2024-09-10T04:05:19.651 INFO:tasks.workunit.client.0.smithi046.stdout:4/613: dread d3/d12/d17/d22/f4c [0,4194304] 0 2024-09-10T04:05:19.679 INFO:tasks.workunit.client.1.smithi070.stdout:1/432: creat d3/d8/d13/d3f/d6d/f8e x:0 0 0 2024-09-10T04:05:19.681 INFO:tasks.workunit.client.0.smithi046.stdout:3/481: creat d12/d13/d14/d1e/d2d/d39/d67/fa1 x:0 0 0 2024-09-10T04:05:19.686 INFO:tasks.workunit.client.1.smithi070.stdout:0/555: unlink d8/d13/d33/d74/l80 0 2024-09-10T04:05:19.692 INFO:tasks.workunit.client.0.smithi046.stdout:5/594: mknod d7/d11/d14/d2f/d2b/cd9 0 2024-09-10T04:05:19.694 INFO:tasks.workunit.client.1.smithi070.stdout:5/514: mkdir d17/d24/d84/dab 0 2024-09-10T04:05:19.701 INFO:tasks.workunit.client.0.smithi046.stdout:4/614: mknod d3/dab/db9/dcd/cd3 0 2024-09-10T04:05:19.704 INFO:tasks.workunit.client.1.smithi070.stdout:6/593: rename d0/ca to d0/d3/d8c/d9a/cb6 0 2024-09-10T04:05:19.704 INFO:tasks.workunit.client.1.smithi070.stdout:6/594: chown d0/d3/d18/f4c 50705 1 2024-09-10T04:05:19.707 INFO:tasks.workunit.client.1.smithi070.stdout:0/556: creat d8/d13/d33/d74/fb5 x:0 0 0 2024-09-10T04:05:19.707 INFO:tasks.workunit.client.1.smithi070.stdout:0/557: chown d8/d24/f6d 7 1 2024-09-10T04:05:19.708 INFO:tasks.workunit.client.1.smithi070.stdout:0/558: write d8/d13/d33/f3f [4340771,36151] 0 2024-09-10T04:05:19.709 INFO:tasks.workunit.client.1.smithi070.stdout:0/559: fsync d8/d24/d2d/f3a 0 2024-09-10T04:05:19.710 INFO:tasks.workunit.client.1.smithi070.stdout:0/560: chown d8/d13/d33/d71 29486 1 2024-09-10T04:05:19.711 INFO:tasks.workunit.client.1.smithi070.stdout:0/561: write d8/d24/d2d/d34/f77 [953676,30993] 0 2024-09-10T04:05:19.711 INFO:tasks.workunit.client.1.smithi070.stdout:9/566: creat d9/fb3 x:0 0 0 2024-09-10T04:05:19.712 INFO:tasks.workunit.client.1.smithi070.stdout:0/562: write d8/d13/d47/f5c [107683,60862] 0 2024-09-10T04:05:19.715 INFO:tasks.workunit.client.0.smithi046.stdout:4/615: fsync d3/d12/d17/d22/d30/f3a 0 2024-09-10T04:05:19.716 INFO:tasks.workunit.client.0.smithi046.stdout:4/616: fsync d3/d12/d14/f7c 0 2024-09-10T04:05:19.717 INFO:tasks.workunit.client.1.smithi070.stdout:3/518: creat db/d1c/d25/fb0 x:0 0 0 2024-09-10T04:05:19.717 INFO:tasks.workunit.client.0.smithi046.stdout:3/482: getdents d12/d5d/d71/d49/d5c/d9f 0 2024-09-10T04:05:19.720 INFO:tasks.workunit.client.1.smithi070.stdout:6/595: truncate d0/d3/d18/d2a/f4f 4191589 0 2024-09-10T04:05:19.723 INFO:tasks.workunit.client.0.smithi046.stdout:8/478: creat d1/d29/d17/d1f/fa3 x:0 0 0 2024-09-10T04:05:19.724 INFO:tasks.workunit.client.0.smithi046.stdout:8/479: chown d1/d29/d19/d36/f3f 4595 1 2024-09-10T04:05:19.731 INFO:tasks.workunit.client.1.smithi070.stdout:2/476: mknod d15/d18/d29/d2a/d2e/d32/d9b/ca9 0 2024-09-10T04:05:19.731 INFO:tasks.workunit.client.1.smithi070.stdout:2/477: readlink d15/d2b/la0 0 2024-09-10T04:05:19.731 INFO:tasks.workunit.client.0.smithi046.stdout:4/617: mknod d3/d12/d17/dc6/cd4 0 2024-09-10T04:05:19.732 INFO:tasks.workunit.client.0.smithi046.stdout:0/394: sync 2024-09-10T04:05:19.732 INFO:tasks.workunit.client.0.smithi046.stdout:6/496: sync 2024-09-10T04:05:19.733 INFO:tasks.workunit.client.0.smithi046.stdout:8/480: mkdir d1/d29/d17/d1f/d46/d52/da4 0 2024-09-10T04:05:19.734 INFO:tasks.workunit.client.0.smithi046.stdout:0/395: chown d1/dc/f76 0 1 2024-09-10T04:05:19.736 INFO:tasks.workunit.client.0.smithi046.stdout:3/483: creat d12/d13/d37/d94/fa2 x:0 0 0 2024-09-10T04:05:19.736 INFO:tasks.workunit.client.0.smithi046.stdout:0/396: write d1/d13/d37/f61 [5219198,77902] 0 2024-09-10T04:05:19.739 INFO:tasks.workunit.client.0.smithi046.stdout:5/595: getdents d7/d11/d14/d2f/d64 0 2024-09-10T04:05:19.739 INFO:tasks.workunit.client.1.smithi070.stdout:6/596: fsync d0/d3/d18/d1b/d17/d3f/d40/f48 0 2024-09-10T04:05:19.740 INFO:tasks.workunit.client.1.smithi070.stdout:6/597: chown d0/d3/d18/d1b/d17/d3f/d40/d94 710 1 2024-09-10T04:05:19.741 INFO:tasks.workunit.client.0.smithi046.stdout:6/497: dread da/dc/f28 [0,4194304] 0 2024-09-10T04:05:19.741 INFO:tasks.workunit.client.1.smithi070.stdout:6/598: fdatasync d0/d3/d18/d62/d70/f86 0 2024-09-10T04:05:19.742 INFO:tasks.workunit.client.0.smithi046.stdout:5/596: chown d7/d11/d14/d45/d51/fbb 270 1 2024-09-10T04:05:19.742 INFO:tasks.workunit.client.1.smithi070.stdout:6/599: stat d0/d3/d18/d1b/d17/d53/fb4 0 2024-09-10T04:05:19.742 INFO:tasks.workunit.client.0.smithi046.stdout:6/498: chown da/f1f 1 1 2024-09-10T04:05:19.745 INFO:tasks.workunit.client.1.smithi070.stdout:3/519: truncate db/d1c/d25/f75 522246 0 2024-09-10T04:05:19.750 INFO:tasks.workunit.client.1.smithi070.stdout:2/478: mknod d15/caa 0 2024-09-10T04:05:19.750 INFO:tasks.workunit.client.1.smithi070.stdout:2/479: dread - d15/d18/d29/f3d zero size 2024-09-10T04:05:19.758 INFO:tasks.workunit.client.1.smithi070.stdout:2/480: fsync d15/d18/d56/d7c/d55/d6d/fa8 0 2024-09-10T04:05:19.759 INFO:tasks.workunit.client.1.smithi070.stdout:0/563: creat d8/d13/d47/fb6 x:0 0 0 2024-09-10T04:05:19.759 INFO:tasks.workunit.client.0.smithi046.stdout:5/597: rmdir d7/d11/d14/d45 39 2024-09-10T04:05:19.761 INFO:tasks.workunit.client.0.smithi046.stdout:6/499: creat da/db/d63/fa0 x:0 0 0 2024-09-10T04:05:19.762 INFO:tasks.workunit.client.0.smithi046.stdout:6/500: fsync da/db/d11/d2b/f9a 0 2024-09-10T04:05:19.764 INFO:tasks.workunit.client.0.smithi046.stdout:6/501: truncate da/db/d63/fa0 706491 0 2024-09-10T04:05:19.766 INFO:tasks.workunit.client.0.smithi046.stdout:4/618: truncate d3/d12/d17/d22/d30/f3a 89492 0 2024-09-10T04:05:19.767 INFO:tasks.workunit.client.1.smithi070.stdout:0/564: creat d8/d13/d33/d81/fb7 x:0 0 0 2024-09-10T04:05:19.768 INFO:tasks.workunit.client.0.smithi046.stdout:8/481: creat d1/d29/d17/d1f/d46/d6a/d82/d99/fa5 x:0 0 0 2024-09-10T04:05:19.769 INFO:tasks.workunit.client.1.smithi070.stdout:2/481: dwrite d15/d2b/f4c [0,4194304] 0 2024-09-10T04:05:19.770 INFO:tasks.workunit.client.0.smithi046.stdout:3/484: creat d12/d5d/d71/d49/d5c/d9f/fa3 x:0 0 0 2024-09-10T04:05:19.771 INFO:tasks.workunit.client.0.smithi046.stdout:3/485: chown d12/d13/d14/f3a 1 1 2024-09-10T04:05:19.772 INFO:tasks.workunit.client.0.smithi046.stdout:3/486: chown d12/d5d/d71/d49/d5e 2980 1 2024-09-10T04:05:19.775 INFO:tasks.workunit.client.0.smithi046.stdout:6/502: creat da/db/d63/fa1 x:0 0 0 2024-09-10T04:05:19.779 INFO:tasks.workunit.client.0.smithi046.stdout:8/482: symlink d1/d29/d19/d36/d8d/la6 0 2024-09-10T04:05:19.785 INFO:tasks.workunit.client.0.smithi046.stdout:6/503: creat da/db/d63/fa2 x:0 0 0 2024-09-10T04:05:19.786 INFO:tasks.workunit.client.1.smithi070.stdout:9/567: link d9/d7a/d6f/d90/lb2 d9/d59/d5d/lb4 0 2024-09-10T04:05:19.786 INFO:tasks.workunit.client.1.smithi070.stdout:9/568: readlink l4 0 2024-09-10T04:05:19.787 INFO:tasks.workunit.client.1.smithi070.stdout:8/570: sync 2024-09-10T04:05:19.787 INFO:tasks.workunit.client.1.smithi070.stdout:7/475: sync 2024-09-10T04:05:19.787 INFO:tasks.workunit.client.0.smithi046.stdout:4/619: creat d3/d12/d17/d29/dba/fd5 x:0 0 0 2024-09-10T04:05:19.787 INFO:tasks.workunit.client.1.smithi070.stdout:8/571: readlink d4/l18 0 2024-09-10T04:05:19.788 INFO:tasks.workunit.client.0.smithi046.stdout:4/620: read d3/d12/d17/d29/f3e [85816,75355] 0 2024-09-10T04:05:19.788 INFO:tasks.workunit.client.0.smithi046.stdout:8/483: chown d1/d29/d17/d1f/d46/d76/c9a 4 1 2024-09-10T04:05:19.789 INFO:tasks.workunit.client.0.smithi046.stdout:4/621: chown d3/d12/d17/d29/d58/db2/fc4 696973416 1 2024-09-10T04:05:19.790 INFO:tasks.workunit.client.0.smithi046.stdout:3/487: creat d12/d13/d37/d9c/d50/d7a/fa4 x:0 0 0 2024-09-10T04:05:19.791 INFO:tasks.workunit.client.0.smithi046.stdout:8/484: read d1/d29/d17/d1f/d2a/f85 [786203,126160] 0 2024-09-10T04:05:19.792 INFO:tasks.workunit.client.0.smithi046.stdout:8/485: fdatasync d1/d29/d17/d1f/d2a/f85 0 2024-09-10T04:05:19.793 INFO:tasks.workunit.client.0.smithi046.stdout:6/504: fsync da/db/d11/d13/f39 0 2024-09-10T04:05:19.794 INFO:tasks.workunit.client.0.smithi046.stdout:6/505: chown da/db/d10/d27/d8f 26345 1 2024-09-10T04:05:19.798 INFO:tasks.workunit.client.0.smithi046.stdout:4/622: creat d3/d12/d17/d22/d51/d94/fd6 x:0 0 0 2024-09-10T04:05:19.799 INFO:tasks.workunit.client.0.smithi046.stdout:4/623: write d3/d12/d17/d29/dba/fd5 [498153,35718] 0 2024-09-10T04:05:19.802 INFO:tasks.workunit.client.0.smithi046.stdout:8/486: write d1/d29/d12/f6b [560577,67551] 0 2024-09-10T04:05:19.806 INFO:tasks.workunit.client.1.smithi070.stdout:7/476: creat d2/d7/d9/fb1 x:0 0 0 2024-09-10T04:05:19.808 INFO:tasks.workunit.client.1.smithi070.stdout:8/572: rename d4/d16/d17/d5c to d4/dbf 0 2024-09-10T04:05:19.809 INFO:tasks.workunit.client.0.smithi046.stdout:3/488: mkdir d12/d5d/d71/d49/d5e/d96/da5 0 2024-09-10T04:05:19.811 INFO:tasks.workunit.client.1.smithi070.stdout:7/477: dwrite d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:19.815 INFO:tasks.workunit.client.1.smithi070.stdout:4/448: sync 2024-09-10T04:05:19.815 INFO:tasks.workunit.client.1.smithi070.stdout:8/573: creat d4/dbf/d87/fc0 x:0 0 0 2024-09-10T04:05:19.815 INFO:tasks.workunit.client.0.smithi046.stdout:4/624: creat d3/d12/d17/d29/d58/db2/d5f/d85/fd7 x:0 0 0 2024-09-10T04:05:19.816 INFO:tasks.workunit.client.0.smithi046.stdout:8/487: fsync d1/d29/d12/f57 0 2024-09-10T04:05:19.816 INFO:tasks.workunit.client.1.smithi070.stdout:7/478: mkdir d2/d40/d6c/db2 0 2024-09-10T04:05:19.816 INFO:tasks.workunit.client.0.smithi046.stdout:8/488: chown d1/d29/d17/d1f 4 1 2024-09-10T04:05:19.817 INFO:tasks.workunit.client.0.smithi046.stdout:8/489: chown d1/d29/d17/d1f/d46/c4d 6605885 1 2024-09-10T04:05:19.818 INFO:tasks.workunit.client.0.smithi046.stdout:6/506: mkdir da/db/d11/d13/d1e/d23/d54/d90/da3 0 2024-09-10T04:05:19.822 INFO:tasks.workunit.client.1.smithi070.stdout:7/479: symlink d2/d40/d6c/lb3 0 2024-09-10T04:05:19.825 INFO:tasks.workunit.client.1.smithi070.stdout:7/480: read d2/d3/d63/f7a [422525,1338] 0 2024-09-10T04:05:19.826 INFO:tasks.workunit.client.0.smithi046.stdout:3/489: dread d12/d13/d14/f25 [0,4194304] 0 2024-09-10T04:05:19.827 INFO:tasks.workunit.client.0.smithi046.stdout:8/490: dread - d1/d29/d19/d36/d4c/d5e/f5f zero size 2024-09-10T04:05:19.828 INFO:tasks.workunit.client.1.smithi070.stdout:5/515: dwrite d17/d19/d6a/d54/d57/d63/f9a [0,4194304] 0 2024-09-10T04:05:19.828 INFO:tasks.workunit.client.1.smithi070.stdout:7/481: read - d2/d3/dc/d5d/d42/fa0 zero size 2024-09-10T04:05:19.829 INFO:tasks.workunit.client.0.smithi046.stdout:3/490: dread d12/d13/d14/f36 [0,4194304] 0 2024-09-10T04:05:19.829 INFO:tasks.workunit.client.1.smithi070.stdout:4/449: dwrite de/d13/d18/d23/d39/f6d [0,4194304] 0 2024-09-10T04:05:19.830 INFO:tasks.workunit.client.0.smithi046.stdout:6/507: mknod da/dc/ca4 0 2024-09-10T04:05:19.831 INFO:tasks.workunit.client.1.smithi070.stdout:5/516: write d17/d19/d6a/d54/d57/da3/f97 [459174,14824] 0 2024-09-10T04:05:19.833 INFO:tasks.workunit.client.1.smithi070.stdout:4/450: truncate de/d13/d18/d26/f75 37908 0 2024-09-10T04:05:19.834 INFO:tasks.workunit.client.0.smithi046.stdout:4/625: link d3/d12/d17/d29/d58/db2/d7a/fd0 d3/d12/d14/fd8 0 2024-09-10T04:05:19.837 INFO:tasks.workunit.client.0.smithi046.stdout:8/491: rename d1/d29/d17/d1f/d2a/l43 to d1/d29/d17/d1f/d46/d52/la7 0 2024-09-10T04:05:19.839 INFO:tasks.workunit.client.0.smithi046.stdout:8/492: fsync d1/d29/d17/f9c 0 2024-09-10T04:05:19.846 INFO:tasks.workunit.client.1.smithi070.stdout:4/451: rename de/d13/d18/d5c/d7f to de/d13/d5b/da0 0 2024-09-10T04:05:19.847 INFO:tasks.workunit.client.0.smithi046.stdout:8/493: creat d1/d29/d17/d1f/d46/fa8 x:0 0 0 2024-09-10T04:05:19.848 INFO:tasks.workunit.client.0.smithi046.stdout:8/494: chown d1/d29/d19/d36/d6e/f95 7324724 1 2024-09-10T04:05:19.850 INFO:tasks.workunit.client.0.smithi046.stdout:3/491: link d12/d5d/d71/d49/d5e/f97 d12/d5d/d71/d49/d5e/d96/da5/fa6 0 2024-09-10T04:05:19.852 INFO:tasks.workunit.client.1.smithi070.stdout:5/517: dwrite d17/d24/d45/f98 [0,4194304] 0 2024-09-10T04:05:19.852 INFO:tasks.workunit.client.0.smithi046.stdout:4/626: unlink d3/d12/d17/d29/d58/db2/f7b 0 2024-09-10T04:05:19.857 INFO:tasks.workunit.client.1.smithi070.stdout:7/482: getdents d2/d3/dc/d5d/d2d 0 2024-09-10T04:05:19.857 INFO:tasks.workunit.client.0.smithi046.stdout:1/400: sync 2024-09-10T04:05:19.857 INFO:tasks.workunit.client.0.smithi046.stdout:7/498: sync 2024-09-10T04:05:19.857 INFO:tasks.workunit.client.0.smithi046.stdout:8/495: dwrite d1/d29/d17/d1f/d46/f4b [0,4194304] 0 2024-09-10T04:05:19.857 INFO:tasks.workunit.client.0.smithi046.stdout:4/627: write d3/d12/d17/d22/d30/faa [1250898,116644] 0 2024-09-10T04:05:19.857 INFO:tasks.workunit.client.0.smithi046.stdout:3/492: readlink d12/d13/d37/d9c/d50/l7f 0 2024-09-10T04:05:19.858 INFO:tasks.workunit.client.0.smithi046.stdout:1/401: write d3/d27/d32/f2e [4102995,57713] 0 2024-09-10T04:05:19.863 INFO:tasks.workunit.client.1.smithi070.stdout:5/518: fsync d17/d19/d35/f5f 0 2024-09-10T04:05:19.865 INFO:tasks.workunit.client.1.smithi070.stdout:3/520: dwrite db/f1f [0,4194304] 0 2024-09-10T04:05:19.871 INFO:tasks.workunit.client.1.smithi070.stdout:5/519: dread d17/f75 [0,4194304] 0 2024-09-10T04:05:19.872 INFO:tasks.workunit.client.1.smithi070.stdout:6/600: dwrite d0/d3/d18/d1b/d17/d28/f6b [0,4194304] 0 2024-09-10T04:05:19.872 INFO:tasks.workunit.client.0.smithi046.stdout:8/496: creat d1/d29/d19/d36/d4c/fa9 x:0 0 0 2024-09-10T04:05:19.874 INFO:tasks.workunit.client.0.smithi046.stdout:0/397: dwrite d1/dc/d25/d33/d4/d19/f58 [0,4194304] 0 2024-09-10T04:05:19.874 INFO:tasks.workunit.client.1.smithi070.stdout:5/520: write d17/d19/d6a/d54/d57/da3/f82 [380131,5306] 0 2024-09-10T04:05:19.874 INFO:tasks.workunit.client.1.smithi070.stdout:5/521: stat d17/d19/l28 0 2024-09-10T04:05:19.879 INFO:tasks.workunit.client.1.smithi070.stdout:7/483: mkdir d2/d7/d9/db4 0 2024-09-10T04:05:19.881 INFO:tasks.workunit.client.1.smithi070.stdout:0/565: write d8/d13/d33/d4f/f7f [698226,62039] 0 2024-09-10T04:05:19.881 INFO:tasks.workunit.client.0.smithi046.stdout:4/628: dwrite d3/fb8 [0,4194304] 0 2024-09-10T04:05:19.881 INFO:tasks.workunit.client.1.smithi070.stdout:6/601: dwrite d0/d3/d8c/d9a/fac [0,4194304] 0 2024-09-10T04:05:19.885 INFO:tasks.workunit.client.1.smithi070.stdout:5/522: dread f11 [4194304,4194304] 0 2024-09-10T04:05:19.887 INFO:tasks.workunit.client.1.smithi070.stdout:2/482: truncate d15/d18/d29/d2a/d2e/d32/d36/f5b 2504545 0 2024-09-10T04:05:19.888 INFO:tasks.workunit.client.0.smithi046.stdout:3/493: dread d12/d13/d14/f25 [0,4194304] 0 2024-09-10T04:05:19.889 INFO:tasks.workunit.client.0.smithi046.stdout:7/499: dwrite f4 [0,4194304] 0 2024-09-10T04:05:19.893 INFO:tasks.workunit.client.1.smithi070.stdout:0/566: unlink d8/d24/d2d/f88 0 2024-09-10T04:05:19.893 INFO:tasks.workunit.client.1.smithi070.stdout:0/567: readlink d8/l55 0 2024-09-10T04:05:19.895 INFO:tasks.workunit.client.0.smithi046.stdout:8/497: dwrite d1/d29/d17/d1f/fa3 [0,4194304] 0 2024-09-10T04:05:19.896 INFO:tasks.workunit.client.0.smithi046.stdout:7/500: dread - d5/d9/d12/d72/d77/f82 zero size 2024-09-10T04:05:19.904 INFO:tasks.workunit.client.1.smithi070.stdout:2/483: mknod d15/d18/d56/cab 0 2024-09-10T04:05:19.905 INFO:tasks.workunit.client.0.smithi046.stdout:1/402: dread d3/dc/f6e [0,4194304] 0 2024-09-10T04:05:19.905 INFO:tasks.workunit.client.0.smithi046.stdout:0/398: chown d1/dc/d25/d33/f1b 15412 1 2024-09-10T04:05:19.905 INFO:tasks.workunit.client.0.smithi046.stdout:5/598: unlink d7/d11/d3f/d53/f57 0 2024-09-10T04:05:19.905 INFO:tasks.workunit.client.1.smithi070.stdout:9/569: truncate d9/d1c/dac/d8a/d3c/f52 919471 0 2024-09-10T04:05:19.908 INFO:tasks.workunit.client.0.smithi046.stdout:4/629: creat d3/d12/d17/d29/dba/fd9 x:0 0 0 2024-09-10T04:05:19.909 INFO:tasks.workunit.client.1.smithi070.stdout:0/568: mknod d8/cb8 0 2024-09-10T04:05:19.909 INFO:tasks.workunit.client.0.smithi046.stdout:5/599: fsync d7/d11/d3f/d5a/fb2 0 2024-09-10T04:05:19.911 INFO:tasks.workunit.client.0.smithi046.stdout:7/501: chown d5/d6/db/dd/d24/c6a 138403546 1 2024-09-10T04:05:19.916 INFO:tasks.workunit.client.0.smithi046.stdout:4/630: chown d3/d12/d17/d22/d51/d94/fd6 122122 1 2024-09-10T04:05:19.917 INFO:tasks.workunit.client.0.smithi046.stdout:8/498: dwrite d1/d29/d12/f9f [0,4194304] 0 2024-09-10T04:05:19.920 INFO:tasks.workunit.client.0.smithi046.stdout:4/631: write d3/d12/d17/d29/d58/f71 [627443,59287] 0 2024-09-10T04:05:19.922 INFO:tasks.workunit.client.0.smithi046.stdout:8/499: write d1/f86 [2189966,48021] 0 2024-09-10T04:05:19.924 INFO:tasks.workunit.client.1.smithi070.stdout:6/602: link d0/d3/f99 d0/d3/d1f/d81/fb7 0 2024-09-10T04:05:19.925 INFO:tasks.workunit.client.1.smithi070.stdout:3/521: dread db/d1c/d3e/f54 [0,4194304] 0 2024-09-10T04:05:19.927 INFO:tasks.workunit.client.1.smithi070.stdout:3/522: write db/d1c/d25/fb0 [159313,97] 0 2024-09-10T04:05:19.928 INFO:tasks.workunit.client.1.smithi070.stdout:0/569: symlink d8/d24/d2d/lb9 0 2024-09-10T04:05:19.930 INFO:tasks.workunit.client.0.smithi046.stdout:3/494: dread d12/d13/d14/d1e/d2d/f2e [0,4194304] 0 2024-09-10T04:05:19.931 INFO:tasks.workunit.client.0.smithi046.stdout:3/495: stat d12/d13/l3e 0 2024-09-10T04:05:19.932 INFO:tasks.workunit.client.0.smithi046.stdout:3/496: write d12/d13/d37/d9c/d50/d52/f65 [3562178,71142] 0 2024-09-10T04:05:19.933 INFO:tasks.workunit.client.1.smithi070.stdout:2/484: rename d15/d18/c1f to d15/d18/d29/d2a/d2e/cac 0 2024-09-10T04:05:19.938 INFO:tasks.workunit.client.1.smithi070.stdout:3/523: symlink db/d79/d94/lb1 0 2024-09-10T04:05:19.940 INFO:tasks.workunit.client.1.smithi070.stdout:0/570: rmdir d8/d13/d1a 39 2024-09-10T04:05:19.941 INFO:tasks.workunit.client.0.smithi046.stdout:6/508: dwrite da/db/d11/d2b/d5a/f5c [0,4194304] 0 2024-09-10T04:05:19.944 INFO:tasks.workunit.client.1.smithi070.stdout:3/524: dwrite db/d83/d92/faa [0,4194304] 0 2024-09-10T04:05:19.946 INFO:tasks.workunit.client.1.smithi070.stdout:6/603: mkdir d0/d3/d18/d62/db8 0 2024-09-10T04:05:19.948 INFO:tasks.workunit.client.1.smithi070.stdout:2/485: fdatasync d15/d18/d29/f46 0 2024-09-10T04:05:19.954 INFO:tasks.workunit.client.0.smithi046.stdout:0/399: dread - d1/dc/d25/d33/d1a/f47 zero size 2024-09-10T04:05:19.955 INFO:tasks.workunit.client.1.smithi070.stdout:4/452: write de/d13/d18/d26/d46/d85/f9a [835506,65400] 0 2024-09-10T04:05:19.956 INFO:tasks.workunit.client.1.smithi070.stdout:0/571: dwrite d8/d13/d1a/f23 [4194304,4194304] 0 2024-09-10T04:05:19.957 INFO:tasks.workunit.client.1.smithi070.stdout:0/572: chown d8/d24/f4a 62062 1 2024-09-10T04:05:19.958 INFO:tasks.workunit.client.0.smithi046.stdout:0/400: chown d1/dc/d25/d33/d3/dd/d55/d77 249 1 2024-09-10T04:05:19.958 INFO:tasks.workunit.client.1.smithi070.stdout:0/573: readlink d8/d24/d2d/d34/d9b/dad/l9d 0 2024-09-10T04:05:19.965 INFO:tasks.workunit.client.0.smithi046.stdout:2/485: sync 2024-09-10T04:05:19.966 INFO:tasks.workunit.client.0.smithi046.stdout:5/600: rename d7/d11/d14/d2f/d2b/cd9 to d7/d11/d14/d2f/d2b/d92/cda 0 2024-09-10T04:05:19.968 INFO:tasks.workunit.client.0.smithi046.stdout:2/486: fsync d2/d8/d65/f7d 0 2024-09-10T04:05:19.969 INFO:tasks.workunit.client.0.smithi046.stdout:4/632: mknod d3/d12/d17/cda 0 2024-09-10T04:05:19.971 INFO:tasks.workunit.client.0.smithi046.stdout:7/502: dwrite d5/d6/db/d11/d1f/f48 [0,4194304] 0 2024-09-10T04:05:19.977 INFO:tasks.workunit.client.1.smithi070.stdout:7/484: write d2/d3/dc/d5d/d41/f50 [587933,110547] 0 2024-09-10T04:05:19.977 INFO:tasks.workunit.client.1.smithi070.stdout:0/574: mknod d8/d24/d2d/d34/cba 0 2024-09-10T04:05:19.979 INFO:tasks.workunit.client.0.smithi046.stdout:3/497: fsync f7 0 2024-09-10T04:05:19.979 INFO:tasks.workunit.client.0.smithi046.stdout:4/633: dread d3/d12/d17/d29/d58/f6d [0,4194304] 0 2024-09-10T04:05:19.979 INFO:tasks.workunit.client.1.smithi070.stdout:0/575: fdatasync d8/d13/d47/f5c 0 2024-09-10T04:05:19.980 INFO:tasks.workunit.client.0.smithi046.stdout:4/634: read d3/d12/d17/d29/f3e [1439171,113676] 0 2024-09-10T04:05:19.981 INFO:tasks.workunit.client.1.smithi070.stdout:0/576: truncate d8/d13/d33/d74/fb5 562510 0 2024-09-10T04:05:19.985 INFO:tasks.workunit.client.0.smithi046.stdout:6/509: mkdir da/dc/da5 0 2024-09-10T04:05:19.985 INFO:tasks.workunit.client.1.smithi070.stdout:1/433: sync 2024-09-10T04:05:19.986 INFO:tasks.workunit.client.1.smithi070.stdout:7/485: dread d2/d3/dc/d5d/d41/f50 [0,4194304] 0 2024-09-10T04:05:19.987 INFO:tasks.workunit.client.1.smithi070.stdout:7/486: readlink d2/d40/l4f 0 2024-09-10T04:05:19.990 INFO:tasks.workunit.client.1.smithi070.stdout:5/523: write d17/d24/d45/f96 [1145215,78309] 0 2024-09-10T04:05:19.993 INFO:tasks.workunit.client.1.smithi070.stdout:0/577: truncate d8/d13/d1a/d75/f84 908758 0 2024-09-10T04:05:19.994 INFO:tasks.workunit.client.0.smithi046.stdout:5/601: read d7/d11/d14/d2f/d2b/f7a [566364,69150] 0 2024-09-10T04:05:19.995 INFO:tasks.workunit.client.0.smithi046.stdout:2/487: rmdir d2/d6/d23 39 2024-09-10T04:05:19.999 INFO:tasks.workunit.client.1.smithi070.stdout:7/487: creat d2/d3/dc/d5d/d42/fb5 x:0 0 0 2024-09-10T04:05:19.999 INFO:tasks.workunit.client.0.smithi046.stdout:8/500: creat d1/da2/faa x:0 0 0 2024-09-10T04:05:20.004 INFO:tasks.workunit.client.1.smithi070.stdout:2/486: getdents d15/d2b/d6e 0 2024-09-10T04:05:20.004 INFO:tasks.workunit.client.0.smithi046.stdout:8/501: dread d1/d29/d17/f8e [0,4194304] 0 2024-09-10T04:05:20.005 INFO:tasks.workunit.client.0.smithi046.stdout:3/498: mkdir d12/d13/d37/d51/da7 0 2024-09-10T04:05:20.005 INFO:tasks.workunit.client.1.smithi070.stdout:2/487: write d15/d18/d56/d7c/d55/d6d/fa8 [432363,102980] 0 2024-09-10T04:05:20.007 INFO:tasks.workunit.client.1.smithi070.stdout:0/578: rename d8/d13/d1a/faf to d8/d13/d33/d4f/fbb 0 2024-09-10T04:05:20.008 INFO:tasks.workunit.client.0.smithi046.stdout:4/635: creat d3/dab/db9/dcd/fdb x:0 0 0 2024-09-10T04:05:20.012 INFO:tasks.workunit.client.0.smithi046.stdout:0/401: creat d1/d13/d37/d75/f7f x:0 0 0 2024-09-10T04:05:20.016 INFO:tasks.workunit.client.1.smithi070.stdout:5/524: mknod d17/d6f/d32/d90/d99/cac 0 2024-09-10T04:05:20.016 INFO:tasks.workunit.client.1.smithi070.stdout:2/488: fsync d15/d18/d29/d2a/f58 0 2024-09-10T04:05:20.017 INFO:tasks.workunit.client.0.smithi046.stdout:3/499: dread d12/d13/d37/d9c/d50/d52/f65 [0,4194304] 0 2024-09-10T04:05:20.018 INFO:tasks.workunit.client.1.smithi070.stdout:2/489: chown d15/d18/d29/d2a/f74 516198 1 2024-09-10T04:05:20.019 INFO:tasks.workunit.client.1.smithi070.stdout:3/525: write db/f22 [816213,51377] 0 2024-09-10T04:05:20.022 INFO:tasks.workunit.client.1.smithi070.stdout:7/488: mkdir d2/db6 0 2024-09-10T04:05:20.024 INFO:tasks.workunit.client.1.smithi070.stdout:5/525: symlink d17/d19/d6a/d54/d57/lad 0 2024-09-10T04:05:20.026 INFO:tasks.workunit.client.1.smithi070.stdout:2/490: fsync d15/d18/d56/d7c/d55/f65 0 2024-09-10T04:05:20.028 INFO:tasks.workunit.client.1.smithi070.stdout:0/579: creat d8/d13/d33/d7d/d8a/fbc x:0 0 0 2024-09-10T04:05:20.030 INFO:tasks.workunit.client.1.smithi070.stdout:3/526: mknod db/d1c/d25/d28/d43/cb2 0 2024-09-10T04:05:20.030 INFO:tasks.workunit.client.1.smithi070.stdout:5/526: dwrite d17/d24/d45/f98 [0,4194304] 0 2024-09-10T04:05:20.031 INFO:tasks.workunit.client.1.smithi070.stdout:3/527: stat db/d79/fa3 0 2024-09-10T04:05:20.032 INFO:tasks.workunit.client.1.smithi070.stdout:6/604: dwrite d0/d3/d18/f42 [4194304,4194304] 0 2024-09-10T04:05:20.032 INFO:tasks.workunit.client.1.smithi070.stdout:4/453: truncate f0 7305409 0 2024-09-10T04:05:20.034 INFO:tasks.workunit.client.1.smithi070.stdout:4/454: read - de/d13/d18/d26/d8c/f95 zero size 2024-09-10T04:05:20.036 INFO:tasks.workunit.client.1.smithi070.stdout:4/455: chown de/d13/d18/d23/d47/d59/d79/d6a/l73 13267 1 2024-09-10T04:05:20.039 INFO:tasks.workunit.client.1.smithi070.stdout:2/491: creat d15/d18/d56/d7c/d55/d6d/fad x:0 0 0 2024-09-10T04:05:20.040 INFO:tasks.workunit.client.1.smithi070.stdout:2/492: readlink d15/d18/d29/d2a/d2e/l5e 0 2024-09-10T04:05:20.041 INFO:tasks.workunit.client.1.smithi070.stdout:2/493: write d15/d18/f63 [3057911,118649] 0 2024-09-10T04:05:20.042 INFO:tasks.workunit.client.1.smithi070.stdout:5/527: symlink d17/d19/d6a/d54/d57/lae 0 2024-09-10T04:05:20.043 INFO:tasks.workunit.client.1.smithi070.stdout:2/494: write d15/fa7 [1028891,128817] 0 2024-09-10T04:05:20.044 INFO:tasks.workunit.client.1.smithi070.stdout:9/570: sync 2024-09-10T04:05:20.044 INFO:tasks.workunit.client.1.smithi070.stdout:7/489: mkdir d2/d40/d6c/db2/db7 0 2024-09-10T04:05:20.044 INFO:tasks.workunit.client.1.smithi070.stdout:8/574: sync 2024-09-10T04:05:20.044 INFO:tasks.workunit.client.1.smithi070.stdout:2/495: readlink d15/d18/d29/d2a/d2e/l5e 0 2024-09-10T04:05:20.045 INFO:tasks.workunit.client.1.smithi070.stdout:2/496: read d15/d18/d29/d64/f81 [250744,34242] 0 2024-09-10T04:05:20.047 INFO:tasks.workunit.client.1.smithi070.stdout:0/580: symlink d8/d24/lbd 0 2024-09-10T04:05:20.048 INFO:tasks.workunit.client.1.smithi070.stdout:1/434: write d3/d10/f2c [1084605,94265] 0 2024-09-10T04:05:20.049 INFO:tasks.workunit.client.1.smithi070.stdout:0/581: chown d8/d24/d2d/d5f/f60 20375 1 2024-09-10T04:05:20.052 INFO:tasks.workunit.client.1.smithi070.stdout:1/435: read d3/d8/d13/d3f/f52 [1987055,48173] 0 2024-09-10T04:05:20.053 INFO:tasks.workunit.client.1.smithi070.stdout:8/575: mknod d4/d16/d17/d19/cc1 0 2024-09-10T04:05:20.054 INFO:tasks.workunit.client.1.smithi070.stdout:7/490: symlink d2/d7/d9/d18/lb8 0 2024-09-10T04:05:20.054 INFO:tasks.workunit.client.1.smithi070.stdout:8/576: write d4/d16/f33 [4229478,103537] 0 2024-09-10T04:05:20.054 INFO:tasks.workunit.client.1.smithi070.stdout:7/491: readlink d2/l6 0 2024-09-10T04:05:20.055 INFO:tasks.workunit.client.1.smithi070.stdout:0/582: rename d8/d24/da9/fab to d8/d24/d97/fbe 0 2024-09-10T04:05:20.055 INFO:tasks.workunit.client.1.smithi070.stdout:8/577: stat d4/d16/d73/d96/l85 0 2024-09-10T04:05:20.056 INFO:tasks.workunit.client.1.smithi070.stdout:5/528: link d17/d19/d35/l7f d17/d24/d2b/laf 0 2024-09-10T04:05:20.057 INFO:tasks.workunit.client.1.smithi070.stdout:9/571: creat d9/d59/fb5 x:0 0 0 2024-09-10T04:05:20.058 INFO:tasks.workunit.client.1.smithi070.stdout:3/528: getdents db 0 2024-09-10T04:05:20.059 INFO:tasks.workunit.client.1.smithi070.stdout:0/583: mknod d8/d13/d1a/d8f/d95/cbf 0 2024-09-10T04:05:20.060 INFO:tasks.workunit.client.1.smithi070.stdout:2/497: link d15/d18/d29/d2a/d2e/d32/d9b/ca9 d15/d59/da1/cae 0 2024-09-10T04:05:20.060 INFO:tasks.workunit.client.1.smithi070.stdout:3/529: read db/d1c/f4b [1018598,109577] 0 2024-09-10T04:05:20.061 INFO:tasks.workunit.client.1.smithi070.stdout:9/572: creat d9/d1c/dac/d8a/d1b/fb6 x:0 0 0 2024-09-10T04:05:20.062 INFO:tasks.workunit.client.1.smithi070.stdout:0/584: rmdir d8/d24/d2d/d8d 39 2024-09-10T04:05:20.062 INFO:tasks.workunit.client.1.smithi070.stdout:3/530: stat db/d1c/d53/f57 0 2024-09-10T04:05:20.062 INFO:tasks.workunit.client.1.smithi070.stdout:8/578: dwrite d4/dbf/d87/f9f [0,4194304] 0 2024-09-10T04:05:20.062 INFO:tasks.workunit.client.1.smithi070.stdout:5/529: link d17/d19/f23 d17/d19/d6a/d54/d57/fb0 0 2024-09-10T04:05:20.064 INFO:tasks.workunit.client.1.smithi070.stdout:8/579: chown d4/d16/d17/f68 484 1 2024-09-10T04:05:20.064 INFO:tasks.workunit.client.1.smithi070.stdout:0/585: mknod d8/cc0 0 2024-09-10T04:05:20.065 INFO:tasks.workunit.client.1.smithi070.stdout:0/586: write f5 [4121382,29770] 0 2024-09-10T04:05:20.065 INFO:tasks.workunit.client.1.smithi070.stdout:3/531: creat db/d1c/d89/fb3 x:0 0 0 2024-09-10T04:05:20.066 INFO:tasks.workunit.client.0.smithi046.stdout:4/636: mknod d3/d12/d17/d22/d51/d94/cdc 0 2024-09-10T04:05:20.066 INFO:tasks.workunit.client.0.smithi046.stdout:6/510: symlink da/db/d10/d27/d8f/la6 0 2024-09-10T04:05:20.067 INFO:tasks.workunit.client.1.smithi070.stdout:2/498: rename d15/d18/d56/d7c/d55/l61 to d15/d18/d29/d2a/d2e/d32/d36/laf 0 2024-09-10T04:05:20.068 INFO:tasks.workunit.client.0.smithi046.stdout:0/402: unlink d1/dc/d49/f74 0 2024-09-10T04:05:20.068 INFO:tasks.workunit.client.0.smithi046.stdout:1/403: getdents d3/dc/d34/d51 0 2024-09-10T04:05:20.070 INFO:tasks.workunit.client.0.smithi046.stdout:2/488: unlink d2/d6/d23/d5d/d77/d7b/d85/ca1 0 2024-09-10T04:05:20.072 INFO:tasks.workunit.client.0.smithi046.stdout:0/403: chown d1/dc/d25/d33/d3/dd/d55/d4d 32 1 2024-09-10T04:05:20.073 INFO:tasks.workunit.client.0.smithi046.stdout:2/489: write d2/d6/d23/d42/f45 [1291481,26800] 0 2024-09-10T04:05:20.077 INFO:tasks.workunit.client.1.smithi070.stdout:0/587: creat d8/d24/d2d/d34/d6b/fc1 x:0 0 0 2024-09-10T04:05:20.078 INFO:tasks.workunit.client.0.smithi046.stdout:4/637: mkdir d3/d12/d17/d29/ddd 0 2024-09-10T04:05:20.078 INFO:tasks.workunit.client.1.smithi070.stdout:2/499: chown d15/c31 843028629 1 2024-09-10T04:05:20.079 INFO:tasks.workunit.client.0.smithi046.stdout:1/404: unlink d3/d27/f53 0 2024-09-10T04:05:20.081 INFO:tasks.workunit.client.0.smithi046.stdout:5/602: write d7/d11/d3f/d53/d99/fa0 [456947,100523] 0 2024-09-10T04:05:20.082 INFO:tasks.workunit.client.0.smithi046.stdout:7/503: dwrite d5/f56 [0,4194304] 0 2024-09-10T04:05:20.084 INFO:tasks.workunit.client.0.smithi046.stdout:5/603: fdatasync d7/d11/d3f/d53/da9/f83 0 2024-09-10T04:05:20.085 INFO:tasks.workunit.client.1.smithi070.stdout:3/532: truncate db/d1c/f4f 1094640 0 2024-09-10T04:05:20.087 INFO:tasks.workunit.client.0.smithi046.stdout:2/490: mkdir d2/d6/d9d/db6 0 2024-09-10T04:05:20.088 INFO:tasks.workunit.client.1.smithi070.stdout:0/588: stat d8/d24/d2d/d8d/l67 0 2024-09-10T04:05:20.089 INFO:tasks.workunit.client.0.smithi046.stdout:0/404: mkdir d1/dc/d25/d33/d1a/d80 0 2024-09-10T04:05:20.090 INFO:tasks.workunit.client.0.smithi046.stdout:0/405: write d1/dc/d25/d33/d3/f46 [2077919,18776] 0 2024-09-10T04:05:20.092 INFO:tasks.workunit.client.0.smithi046.stdout:6/511: mknod da/db/d11/d9f/ca7 0 2024-09-10T04:05:20.094 INFO:tasks.workunit.client.0.smithi046.stdout:0/406: truncate d1/dc/d25/d33/d3/d29/d2d/f32 4721441 0 2024-09-10T04:05:20.096 INFO:tasks.workunit.client.0.smithi046.stdout:3/500: write d12/d13/d14/f47 [4452179,47620] 0 2024-09-10T04:05:20.097 INFO:tasks.workunit.client.0.smithi046.stdout:4/638: creat d3/d12/d17/d22/d51/d5c/fde x:0 0 0 2024-09-10T04:05:20.097 INFO:tasks.workunit.client.0.smithi046.stdout:3/501: chown d12/d13/d14/f25 0 1 2024-09-10T04:05:20.099 INFO:tasks.workunit.client.1.smithi070.stdout:2/500: mknod d15/d18/d29/cb0 0 2024-09-10T04:05:20.100 INFO:tasks.workunit.client.0.smithi046.stdout:6/512: dread da/db/d10/d27/f29 [0,4194304] 0 2024-09-10T04:05:20.102 INFO:tasks.workunit.client.1.smithi070.stdout:3/533: fdatasync db/d1c/d25/f75 0 2024-09-10T04:05:20.105 INFO:tasks.workunit.client.1.smithi070.stdout:2/501: dwrite d15/d18/d56/d7c/d55/d6d/f7d [0,4194304] 0 2024-09-10T04:05:20.107 INFO:tasks.workunit.client.1.smithi070.stdout:2/502: chown d15/d18/d56/d7c/d55/d6d/fad 32126647 1 2024-09-10T04:05:20.108 INFO:tasks.workunit.client.1.smithi070.stdout:3/534: readlink db/d1c/d3e/l52 0 2024-09-10T04:05:20.110 INFO:tasks.workunit.client.1.smithi070.stdout:3/535: read - db/d1c/d53/fa2 zero size 2024-09-10T04:05:20.111 INFO:tasks.workunit.client.1.smithi070.stdout:4/456: truncate de/d13/d18/f21 1660516 0 2024-09-10T04:05:20.112 INFO:tasks.workunit.client.1.smithi070.stdout:6/605: dwrite d0/d3/f16 [0,4194304] 0 2024-09-10T04:05:20.113 INFO:tasks.workunit.client.0.smithi046.stdout:0/407: rename d1/dc/d25/d33/d3/f63 to d1/dc/d25/d33/d3/dd/d55/f81 0 2024-09-10T04:05:20.113 INFO:tasks.workunit.client.1.smithi070.stdout:4/457: dread - de/d13/d18/d26/d46/f71 zero size 2024-09-10T04:05:20.113 INFO:tasks.workunit.client.1.smithi070.stdout:4/458: stat de/d13/d18/d23/d47/c6f 0 2024-09-10T04:05:20.117 INFO:tasks.workunit.client.0.smithi046.stdout:4/639: unlink d3/d12/d17/l40 0 2024-09-10T04:05:20.118 INFO:tasks.workunit.client.1.smithi070.stdout:2/503: readlink d15/d18/d29/d2a/l53 0 2024-09-10T04:05:20.118 INFO:tasks.workunit.client.1.smithi070.stdout:2/504: dread - d15/d18/d56/f86 zero size 2024-09-10T04:05:20.123 INFO:tasks.workunit.client.1.smithi070.stdout:3/536: creat db/d1c/d89/fb4 x:0 0 0 2024-09-10T04:05:20.125 INFO:tasks.workunit.client.1.smithi070.stdout:6/606: mknod d0/d3/d8c/d9a/d92/cb9 0 2024-09-10T04:05:20.127 INFO:tasks.workunit.client.1.smithi070.stdout:6/607: write d0/f8 [1590463,102411] 0 2024-09-10T04:05:20.128 INFO:tasks.workunit.client.1.smithi070.stdout:1/436: dwrite d3/d8/d12/f24 [0,4194304] 0 2024-09-10T04:05:20.129 INFO:tasks.workunit.client.0.smithi046.stdout:2/491: creat d2/d6/d9d/db6/fb7 x:0 0 0 2024-09-10T04:05:20.129 INFO:tasks.workunit.client.1.smithi070.stdout:1/437: chown d3/d10/f2c 127 1 2024-09-10T04:05:20.130 INFO:tasks.workunit.client.0.smithi046.stdout:2/492: dread - d2/d6/d9d/db6/fb7 zero size 2024-09-10T04:05:20.132 INFO:tasks.workunit.client.1.smithi070.stdout:0/589: getdents d8 0 2024-09-10T04:05:20.133 INFO:tasks.workunit.client.1.smithi070.stdout:1/438: dread d3/d8/d12/d2f/d32/f41 [4194304,4194304] 0 2024-09-10T04:05:20.135 INFO:tasks.workunit.client.1.smithi070.stdout:3/537: mkdir db/d79/db5 0 2024-09-10T04:05:20.136 INFO:tasks.workunit.client.1.smithi070.stdout:3/538: write db/d1c/d25/d28/d43/f9c [235557,71528] 0 2024-09-10T04:05:20.139 INFO:tasks.workunit.client.0.smithi046.stdout:3/502: rename d12/d13/d14/d1e/d2d/c6d to d12/d5d/d71/d49/d5c/ca8 0 2024-09-10T04:05:20.140 INFO:tasks.workunit.client.0.smithi046.stdout:3/503: chown d12/d13/d14/d1e/d2d/d39/d67/l73 120065 1 2024-09-10T04:05:20.141 INFO:tasks.workunit.client.0.smithi046.stdout:4/640: mkdir d3/dab/db9/dcd/ddf 0 2024-09-10T04:05:20.142 INFO:tasks.workunit.client.1.smithi070.stdout:5/530: write d17/d37/fa6 [2940420,80608] 0 2024-09-10T04:05:20.144 INFO:tasks.workunit.client.0.smithi046.stdout:6/513: symlink da/db/d18/la8 0 2024-09-10T04:05:20.145 INFO:tasks.workunit.client.1.smithi070.stdout:5/531: dread - d17/d6f/d32/f68 zero size 2024-09-10T04:05:20.146 INFO:tasks.workunit.client.0.smithi046.stdout:6/514: write da/db/d10/d51/f80 [3108505,46647] 0 2024-09-10T04:05:20.147 INFO:tasks.workunit.client.0.smithi046.stdout:2/493: mkdir d2/d6/d23/d42/db8 0 2024-09-10T04:05:20.148 INFO:tasks.workunit.client.1.smithi070.stdout:3/539: dwrite db/f1f [4194304,4194304] 0 2024-09-10T04:05:20.149 INFO:tasks.workunit.client.1.smithi070.stdout:8/580: dwrite d4/d16/d17/f2c [0,4194304] 0 2024-09-10T04:05:20.150 INFO:tasks.workunit.client.0.smithi046.stdout:4/641: creat d3/d12/d17/fe0 x:0 0 0 2024-09-10T04:05:20.150 INFO:tasks.workunit.client.1.smithi070.stdout:6/608: readlink d0/d3/d18/d2a/l32 0 2024-09-10T04:05:20.153 INFO:tasks.workunit.client.0.smithi046.stdout:3/504: symlink d12/d13/d14/d1e/d2d/d6f/la9 0 2024-09-10T04:05:20.156 INFO:tasks.workunit.client.1.smithi070.stdout:0/590: mknod d8/cc2 0 2024-09-10T04:05:20.159 INFO:tasks.workunit.client.1.smithi070.stdout:1/439: truncate d3/d8/d12/d2f/f34 3290475 0 2024-09-10T04:05:20.165 INFO:tasks.workunit.client.0.smithi046.stdout:3/505: rmdir d12/d5d/d71/d49/d5e/d96/da5 39 2024-09-10T04:05:20.167 INFO:tasks.workunit.client.0.smithi046.stdout:7/504: write d5/d6/d32/f5e [272951,127260] 0 2024-09-10T04:05:20.171 INFO:tasks.workunit.client.0.smithi046.stdout:7/505: write d5/d6/d32/f5e [4393269,80109] 0 2024-09-10T04:05:20.171 INFO:tasks.workunit.client.0.smithi046.stdout:5/604: write d7/d11/d3f/d53/d81/fd0 [214126,28027] 0 2024-09-10T04:05:20.172 INFO:tasks.workunit.client.0.smithi046.stdout:1/405: dwrite d3/d27/f7d [0,4194304] 0 2024-09-10T04:05:20.174 INFO:tasks.workunit.client.0.smithi046.stdout:7/506: chown d5/d6/db/d11/d1f 594 1 2024-09-10T04:05:20.175 INFO:tasks.workunit.client.0.smithi046.stdout:7/507: stat d5/d6/db/f29 0 2024-09-10T04:05:20.184 INFO:tasks.workunit.client.0.smithi046.stdout:5/605: mknod d7/d11/d14/d2f/d64/cdb 0 2024-09-10T04:05:20.185 INFO:tasks.workunit.client.1.smithi070.stdout:7/492: sync 2024-09-10T04:05:20.186 INFO:tasks.workunit.client.0.smithi046.stdout:1/406: rename d3/d27/d32/d28/c3b to d3/d27/d32/d28/c86 0 2024-09-10T04:05:20.188 INFO:tasks.workunit.client.0.smithi046.stdout:4/642: rename d3/dab/db9/dcd to d3/d12/d17/de1 0 2024-09-10T04:05:20.190 INFO:tasks.workunit.client.1.smithi070.stdout:4/459: write fb [575776,304] 0 2024-09-10T04:05:20.193 INFO:tasks.workunit.client.0.smithi046.stdout:7/508: dread d5/d9/d12/f5d [0,4194304] 0 2024-09-10T04:05:20.194 INFO:tasks.workunit.client.0.smithi046.stdout:5/606: creat d7/d11/d3f/d53/da9/d9c/fdc x:0 0 0 2024-09-10T04:05:20.194 INFO:tasks.workunit.client.0.smithi046.stdout:7/509: chown d5/d6/db/d11/d1f/f48 0 1 2024-09-10T04:05:20.195 INFO:tasks.workunit.client.0.smithi046.stdout:1/407: dread - d3/d27/d32/d1f/f55 zero size 2024-09-10T04:05:20.197 INFO:tasks.workunit.client.0.smithi046.stdout:1/408: stat d3/dc/d21/d3a/l4c 0 2024-09-10T04:05:20.203 INFO:tasks.workunit.client.0.smithi046.stdout:3/506: getdents d12/d13/d14/d1e 0 2024-09-10T04:05:20.205 INFO:tasks.workunit.client.1.smithi070.stdout:3/540: mkdir db/d79/db6 0 2024-09-10T04:05:20.207 INFO:tasks.workunit.client.0.smithi046.stdout:5/607: dread d7/d11/d3f/f80 [0,4194304] 0 2024-09-10T04:05:20.207 INFO:tasks.workunit.client.0.smithi046.stdout:6/515: write da/db/d11/d2b/d2f/f37 [118658,19084] 0 2024-09-10T04:05:20.208 INFO:tasks.workunit.client.0.smithi046.stdout:7/510: truncate d5/fc 3550944 0 2024-09-10T04:05:20.209 INFO:tasks.workunit.client.0.smithi046.stdout:7/511: write d5/d6/db/f29 [2421405,55012] 0 2024-09-10T04:05:20.210 INFO:tasks.workunit.client.1.smithi070.stdout:3/541: dwrite db/f22 [0,4194304] 0 2024-09-10T04:05:20.212 INFO:tasks.workunit.client.1.smithi070.stdout:3/542: dread - db/d79/fa8 zero size 2024-09-10T04:05:20.212 INFO:tasks.workunit.client.0.smithi046.stdout:3/507: rename d12/f63 to d12/d13/d37/faa 0 2024-09-10T04:05:20.212 INFO:tasks.workunit.client.1.smithi070.stdout:6/609: creat d0/d3/d18/d62/d70/fba x:0 0 0 2024-09-10T04:05:20.213 INFO:tasks.workunit.client.0.smithi046.stdout:2/494: write d2/d6/d23/f87 [354629,100664] 0 2024-09-10T04:05:20.213 INFO:tasks.workunit.client.0.smithi046.stdout:1/409: fsync d3/dc/f10 0 2024-09-10T04:05:20.217 INFO:tasks.workunit.client.0.smithi046.stdout:5/608: stat d7/d11/d14/d45/d51/f86 0 2024-09-10T04:05:20.220 INFO:tasks.workunit.client.0.smithi046.stdout:6/516: chown da/db/d11/d2b/d2f/c43 4 1 2024-09-10T04:05:20.222 INFO:tasks.workunit.client.1.smithi070.stdout:7/493: creat d2/d3/dc/d79/fb9 x:0 0 0 2024-09-10T04:05:20.229 INFO:tasks.workunit.client.1.smithi070.stdout:0/591: dread d8/fc [0,4194304] 0 2024-09-10T04:05:20.231 INFO:tasks.workunit.client.1.smithi070.stdout:7/494: dread d2/d7/d9/d18/d23/f47 [0,4194304] 0 2024-09-10T04:05:20.234 INFO:tasks.workunit.client.1.smithi070.stdout:9/573: sync 2024-09-10T04:05:20.235 INFO:tasks.workunit.client.1.smithi070.stdout:2/505: getdents d15/d18/d29/d2a/d2e/d32 0 2024-09-10T04:05:20.235 INFO:tasks.workunit.client.1.smithi070.stdout:2/506: chown d15/d2b/d6e/f89 1211220779 1 2024-09-10T04:05:20.236 INFO:tasks.workunit.client.1.smithi070.stdout:9/574: truncate d9/d7a/d74/fa5 558315 0 2024-09-10T04:05:20.237 INFO:tasks.workunit.client.1.smithi070.stdout:9/575: write d9/d7a/d74/fa5 [69851,13221] 0 2024-09-10T04:05:20.238 INFO:tasks.workunit.client.1.smithi070.stdout:4/460: unlink de/c8f 0 2024-09-10T04:05:20.238 INFO:tasks.workunit.client.0.smithi046.stdout:9/494: sync 2024-09-10T04:05:20.239 INFO:tasks.workunit.client.1.smithi070.stdout:4/461: write de/d13/d5b/f78 [3764433,113826] 0 2024-09-10T04:05:20.239 INFO:tasks.workunit.client.1.smithi070.stdout:5/532: creat d17/d19/d6a/d54/fb1 x:0 0 0 2024-09-10T04:05:20.240 INFO:tasks.workunit.client.1.smithi070.stdout:9/576: truncate d9/d1c/dac/d8a/d3c/d62/f8d 712505 0 2024-09-10T04:05:20.240 INFO:tasks.workunit.client.1.smithi070.stdout:4/462: stat de/d13/c22 0 2024-09-10T04:05:20.241 INFO:tasks.workunit.client.1.smithi070.stdout:0/592: creat d8/d24/d2d/d34/d9b/dad/fc3 x:0 0 0 2024-09-10T04:05:20.241 INFO:tasks.workunit.client.1.smithi070.stdout:9/577: readlink d9/d1c/dac/d8a/d1b/l27 0 2024-09-10T04:05:20.242 INFO:tasks.workunit.client.1.smithi070.stdout:9/578: dread - d9/f9a zero size 2024-09-10T04:05:20.242 INFO:tasks.workunit.client.1.smithi070.stdout:8/581: creat d4/d16/d17/d19/d28/d43/d67/fc2 x:0 0 0 2024-09-10T04:05:20.242 INFO:tasks.workunit.client.1.smithi070.stdout:9/579: fdatasync d9/d1c/d34/d63/f69 0 2024-09-10T04:05:20.243 INFO:tasks.workunit.client.1.smithi070.stdout:3/543: symlink db/d1c/d25/d28/lb7 0 2024-09-10T04:05:20.246 INFO:tasks.workunit.client.1.smithi070.stdout:3/544: write db/d1c/d89/f8f [3652983,88668] 0 2024-09-10T04:05:20.246 INFO:tasks.workunit.client.1.smithi070.stdout:6/610: creat d0/d3/d18/d1b/d17/d54/db5/fbb x:0 0 0 2024-09-10T04:05:20.248 INFO:tasks.workunit.client.1.smithi070.stdout:3/545: chown db/d1c/d89/f9f 92 1 2024-09-10T04:05:20.248 INFO:tasks.workunit.client.1.smithi070.stdout:6/611: write d0/d3/d18/f4c [681458,87003] 0 2024-09-10T04:05:20.249 INFO:tasks.workunit.client.1.smithi070.stdout:0/593: dwrite d8/f49 [4194304,4194304] 0 2024-09-10T04:05:20.250 INFO:tasks.workunit.client.1.smithi070.stdout:5/533: mknod d17/d24/d2b/cb2 0 2024-09-10T04:05:20.254 INFO:tasks.workunit.client.1.smithi070.stdout:3/546: dread db/d1c/d89/f8f [0,4194304] 0 2024-09-10T04:05:20.262 INFO:tasks.workunit.client.1.smithi070.stdout:5/534: symlink d17/d24/d84/lb3 0 2024-09-10T04:05:20.264 INFO:tasks.workunit.client.1.smithi070.stdout:2/507: truncate d15/d18/d29/d2a/d2e/d32/d36/f5b 154634 0 2024-09-10T04:05:20.264 INFO:tasks.workunit.client.1.smithi070.stdout:1/440: truncate d3/d8/f2d 1907233 0 2024-09-10T04:05:20.266 INFO:tasks.workunit.client.1.smithi070.stdout:1/441: chown d3/d8/d13/d67/d83/c8b 4040415 1 2024-09-10T04:05:20.266 INFO:tasks.workunit.client.1.smithi070.stdout:3/547: fdatasync db/d79/fa3 0 2024-09-10T04:05:20.268 INFO:tasks.workunit.client.1.smithi070.stdout:6/612: mknod d0/d3/d18/d1b/d17/cbc 0 2024-09-10T04:05:20.269 INFO:tasks.workunit.client.1.smithi070.stdout:5/535: mkdir d17/d24/d77/db4 0 2024-09-10T04:05:20.270 INFO:tasks.workunit.client.0.smithi046.stdout:7/512: mknod d5/d9/d12/ca1 0 2024-09-10T04:05:20.271 INFO:tasks.workunit.client.0.smithi046.stdout:3/508: rename d12/d13/d37/d9c/d50/d52/d86 to d12/d13/d37/dab 0 2024-09-10T04:05:20.271 INFO:tasks.workunit.client.0.smithi046.stdout:5/609: dwrite d7/d11/d3f/d53/d99/fae [0,4194304] 0 2024-09-10T04:05:20.271 INFO:tasks.workunit.client.0.smithi046.stdout:2/495: rmdir d2/d6/d23/d5d/d77 39 2024-09-10T04:05:20.273 INFO:tasks.workunit.client.1.smithi070.stdout:9/580: creat d9/fb7 x:0 0 0 2024-09-10T04:05:20.273 INFO:tasks.workunit.client.0.smithi046.stdout:6/517: truncate f9 991639 0 2024-09-10T04:05:20.274 INFO:tasks.workunit.client.1.smithi070.stdout:7/495: dwrite d2/d7/db/d3a/f68 [4194304,4194304] 0 2024-09-10T04:05:20.276 INFO:tasks.workunit.client.0.smithi046.stdout:6/518: fsync da/db/d11/d13/f7c 0 2024-09-10T04:05:20.280 INFO:tasks.workunit.client.0.smithi046.stdout:1/410: unlink d3/d27/d32/d28/f5b 0 2024-09-10T04:05:20.281 INFO:tasks.workunit.client.1.smithi070.stdout:8/582: creat d4/d16/d17/d19/d28/d43/d67/fc3 x:0 0 0 2024-09-10T04:05:20.281 INFO:tasks.workunit.client.1.smithi070.stdout:9/581: write d9/d1c/dac/d8a/d3c/d92/fa9 [190821,104441] 0 2024-09-10T04:05:20.281 INFO:tasks.workunit.client.1.smithi070.stdout:7/496: dread - d2/d7/db/f92 zero size 2024-09-10T04:05:20.281 INFO:tasks.workunit.client.1.smithi070.stdout:7/497: read d2/d40/d6c/f7f [1891220,63068] 0 2024-09-10T04:05:20.281 INFO:tasks.workunit.client.1.smithi070.stdout:7/498: fsync d2/f55 0 2024-09-10T04:05:20.281 INFO:tasks.workunit.client.1.smithi070.stdout:7/499: dread - d2/d7/d9/d18/d3c/fae zero size 2024-09-10T04:05:20.282 INFO:tasks.workunit.client.0.smithi046.stdout:1/411: chown d3/dc/d34/d51/f78 0 1 2024-09-10T04:05:20.282 INFO:tasks.workunit.client.0.smithi046.stdout:3/509: mkdir d12/d13/d37/dab/dac 0 2024-09-10T04:05:20.284 INFO:tasks.workunit.client.1.smithi070.stdout:3/548: dread db/d1c/d89/f8f [0,4194304] 0 2024-09-10T04:05:20.285 INFO:tasks.workunit.client.0.smithi046.stdout:4/643: chown d3/d12/d17/d29/d58/db2/d5f/d85/fd7 426 1 2024-09-10T04:05:20.285 INFO:tasks.workunit.client.1.smithi070.stdout:3/549: dread - db/d1c/d89/fb4 zero size 2024-09-10T04:05:20.286 INFO:tasks.workunit.client.1.smithi070.stdout:3/550: read db/d1c/d25/f75 [97902,119935] 0 2024-09-10T04:05:20.289 INFO:tasks.workunit.client.0.smithi046.stdout:7/513: rename d5/c13 to d5/d9/d12/d72/d77/ca2 0 2024-09-10T04:05:20.301 INFO:tasks.workunit.client.1.smithi070.stdout:0/594: rename d8/d13/d1a/d8f/d95/faa to d8/d24/d2d/fc4 0 2024-09-10T04:05:20.301 INFO:tasks.workunit.client.1.smithi070.stdout:2/508: fsync d15/d18/f21 0 2024-09-10T04:05:20.303 INFO:tasks.workunit.client.1.smithi070.stdout:1/442: symlink d3/d10/l8f 0 2024-09-10T04:05:20.303 INFO:tasks.workunit.client.1.smithi070.stdout:8/583: creat d4/d16/d17/d19/d28/d43/d52/fc4 x:0 0 0 2024-09-10T04:05:20.303 INFO:tasks.workunit.client.1.smithi070.stdout:7/500: creat d2/d3/dc/d5d/d2d/fba x:0 0 0 2024-09-10T04:05:20.304 INFO:tasks.workunit.client.1.smithi070.stdout:2/509: dread - d15/d18/d56/fa4 zero size 2024-09-10T04:05:20.304 INFO:tasks.workunit.client.1.smithi070.stdout:1/443: fsync d3/d8/d13/d67/f8a 0 2024-09-10T04:05:20.305 INFO:tasks.workunit.client.1.smithi070.stdout:8/584: write d4/d16/d17/d1a/fa9 [518857,33271] 0 2024-09-10T04:05:20.306 INFO:tasks.workunit.client.1.smithi070.stdout:1/444: read d3/d8/d13/d3f/f42 [3865901,73674] 0 2024-09-10T04:05:20.310 INFO:tasks.workunit.client.0.smithi046.stdout:4/644: dwrite d3/d12/d17/d91/f97 [0,4194304] 0 2024-09-10T04:05:20.312 INFO:tasks.workunit.client.0.smithi046.stdout:2/496: rename d2/d8/l30 to d2/d6/d23/d5d/d77/dac/lb9 0 2024-09-10T04:05:20.315 INFO:tasks.workunit.client.0.smithi046.stdout:9/495: creat d4/f99 x:0 0 0 2024-09-10T04:05:20.317 INFO:tasks.workunit.client.0.smithi046.stdout:5/610: write d7/d11/d14/d2f/d64/f7d [964785,97144] 0 2024-09-10T04:05:20.317 INFO:tasks.workunit.client.0.smithi046.stdout:1/412: truncate d3/d27/d32/f69 1003032 0 2024-09-10T04:05:20.318 INFO:tasks.workunit.client.0.smithi046.stdout:5/611: chown d7/d11/d14/d2f/f5e 127730721 1 2024-09-10T04:05:20.321 INFO:tasks.workunit.client.0.smithi046.stdout:7/514: creat d5/d6/db/dd/d24/d76/d9c/fa3 x:0 0 0 2024-09-10T04:05:20.322 INFO:tasks.workunit.client.1.smithi070.stdout:0/595: mkdir d8/d24/d97/dc5 0 2024-09-10T04:05:20.323 INFO:tasks.workunit.client.1.smithi070.stdout:6/613: creat d0/fbd x:0 0 0 2024-09-10T04:05:20.324 INFO:tasks.workunit.client.1.smithi070.stdout:7/501: symlink d2/d40/d6c/lbb 0 2024-09-10T04:05:20.327 INFO:tasks.workunit.client.1.smithi070.stdout:6/614: dread d0/d3/d18/d1b/d33/f65 [0,4194304] 0 2024-09-10T04:05:20.328 INFO:tasks.workunit.client.0.smithi046.stdout:9/496: rename d4/d7/c34 to d4/d66/c9a 0 2024-09-10T04:05:20.329 INFO:tasks.workunit.client.1.smithi070.stdout:8/585: creat d4/d16/d17/d6f/fc5 x:0 0 0 2024-09-10T04:05:20.330 INFO:tasks.workunit.client.0.smithi046.stdout:9/497: write d4/d7/f77 [113221,67772] 0 2024-09-10T04:05:20.331 INFO:tasks.workunit.client.0.smithi046.stdout:3/510: link d12/d13/d14/f33 d12/d13/d14/fad 0 2024-09-10T04:05:20.333 INFO:tasks.workunit.client.0.smithi046.stdout:5/612: dread d7/d11/d14/d2f/d2b/d6a/f82 [0,4194304] 0 2024-09-10T04:05:20.334 INFO:tasks.workunit.client.1.smithi070.stdout:7/502: dwrite d2/d7/d9/d18/f6b [4194304,4194304] 0 2024-09-10T04:05:20.335 INFO:tasks.workunit.client.1.smithi070.stdout:3/551: symlink db/d79/db5/lb8 0 2024-09-10T04:05:20.335 INFO:tasks.workunit.client.1.smithi070.stdout:4/463: dread de/d13/d43/f65 [0,4194304] 0 2024-09-10T04:05:20.336 INFO:tasks.workunit.client.0.smithi046.stdout:2/497: creat d2/d8/d65/d90/fba x:0 0 0 2024-09-10T04:05:20.338 INFO:tasks.workunit.client.1.smithi070.stdout:7/503: chown d2/d3/d63/c97 62 1 2024-09-10T04:05:20.340 INFO:tasks.workunit.client.0.smithi046.stdout:9/498: mknod d4/d7/d61/c9b 0 2024-09-10T04:05:20.340 INFO:tasks.workunit.client.1.smithi070.stdout:7/504: dread - d2/d7/db/d3a/fac zero size 2024-09-10T04:05:20.341 INFO:tasks.workunit.client.0.smithi046.stdout:9/499: readlink d4/d7/l3b 0 2024-09-10T04:05:20.342 INFO:tasks.workunit.client.1.smithi070.stdout:8/586: dwrite d4/d39/fbb [0,4194304] 0 2024-09-10T04:05:20.342 INFO:tasks.workunit.client.0.smithi046.stdout:7/515: link d5/d6/d32/f95 d5/d9/d12/fa4 0 2024-09-10T04:05:20.343 INFO:tasks.workunit.client.1.smithi070.stdout:7/505: write d2/d3/dc/d5d/d2d/fba [583955,58585] 0 2024-09-10T04:05:20.344 INFO:tasks.workunit.client.1.smithi070.stdout:0/596: mkdir d8/d13/d33/d53/dc6 0 2024-09-10T04:05:20.345 INFO:tasks.workunit.client.0.smithi046.stdout:3/511: stat d12/d13/d14/d1e/d2d/d39/d67/d92/c95 0 2024-09-10T04:05:20.346 INFO:tasks.workunit.client.0.smithi046.stdout:4/645: write d3/d12/d17/d22/d51/f8f [8378136,70240] 0 2024-09-10T04:05:20.350 INFO:tasks.workunit.client.0.smithi046.stdout:4/646: chown d3/d12/d17/d29/c3b 13 1 2024-09-10T04:05:20.352 INFO:tasks.workunit.client.1.smithi070.stdout:7/506: dwrite d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:20.353 INFO:tasks.workunit.client.0.smithi046.stdout:4/647: fsync d3/d12/d17/d29/fcb 0 2024-09-10T04:05:20.353 INFO:tasks.workunit.client.1.smithi070.stdout:1/445: dread d3/f60 [4194304,4194304] 0 2024-09-10T04:05:20.354 INFO:tasks.workunit.client.1.smithi070.stdout:7/507: truncate d2/d66/fa3 1009368 0 2024-09-10T04:05:20.356 INFO:tasks.workunit.client.1.smithi070.stdout:7/508: fdatasync d2/d7/db/d3a/d7e/f93 0 2024-09-10T04:05:20.356 INFO:tasks.workunit.client.0.smithi046.stdout:3/512: dwrite d12/d13/d14/d1e/d2d/d39/d67/fa1 [0,4194304] 0 2024-09-10T04:05:20.358 INFO:tasks.workunit.client.0.smithi046.stdout:7/516: creat d5/d6/d39/d5a/fa5 x:0 0 0 2024-09-10T04:05:20.359 INFO:tasks.workunit.client.1.smithi070.stdout:1/446: dwrite d3/d8/d12/f24 [0,4194304] 0 2024-09-10T04:05:20.361 INFO:tasks.workunit.client.1.smithi070.stdout:5/536: getdents d17/d19/d6a/d54/d57/da3 0 2024-09-10T04:05:20.368 INFO:tasks.workunit.client.1.smithi070.stdout:4/464: write de/d13/d43/f6b [3309802,107816] 0 2024-09-10T04:05:20.370 INFO:tasks.workunit.client.1.smithi070.stdout:8/587: mkdir d4/d39/dc6 0 2024-09-10T04:05:20.374 INFO:tasks.workunit.client.0.smithi046.stdout:4/648: dread d3/d12/d17/d29/d58/f8c [0,4194304] 0 2024-09-10T04:05:20.375 INFO:tasks.workunit.client.1.smithi070.stdout:0/597: mknod d8/d24/d2d/d8d/cc7 0 2024-09-10T04:05:20.376 INFO:tasks.workunit.client.1.smithi070.stdout:0/598: read - d8/d13/d1a/d8f/f9c zero size 2024-09-10T04:05:20.377 INFO:tasks.workunit.client.1.smithi070.stdout:0/599: chown d8/d13/d33/d53/d59/c82 259959 1 2024-09-10T04:05:20.377 INFO:tasks.workunit.client.0.smithi046.stdout:4/649: dread - d3/d12/d17/d29/d58/db2/d5f/d85/fd7 zero size 2024-09-10T04:05:20.378 INFO:tasks.workunit.client.0.smithi046.stdout:2/498: creat d2/d6/d15/fbb x:0 0 0 2024-09-10T04:05:20.380 INFO:tasks.workunit.client.0.smithi046.stdout:5/613: dwrite d7/d11/d14/d45/d51/f86 [0,4194304] 0 2024-09-10T04:05:20.380 INFO:tasks.workunit.client.1.smithi070.stdout:2/510: creat d15/d18/d56/d7c/fb1 x:0 0 0 2024-09-10T04:05:20.385 INFO:tasks.workunit.client.0.smithi046.stdout:3/513: write d12/d13/d14/f36 [2693551,85312] 0 2024-09-10T04:05:20.388 INFO:tasks.workunit.client.0.smithi046.stdout:7/517: link d5/d9/d43/f5c d5/d7f/d78/fa6 0 2024-09-10T04:05:20.391 INFO:tasks.workunit.client.0.smithi046.stdout:2/499: dread d2/d8/d37/d63/f75 [0,4194304] 0 2024-09-10T04:05:20.395 INFO:tasks.workunit.client.0.smithi046.stdout:5/614: rename d7/d11/c76 to d7/d11/d14/d2f/d2b/cdd 0 2024-09-10T04:05:20.396 INFO:tasks.workunit.client.1.smithi070.stdout:8/588: rmdir d4/d16/d17/d1a/d60/d7a/d84 39 2024-09-10T04:05:20.397 INFO:tasks.workunit.client.1.smithi070.stdout:6/615: truncate d0/d3/f51 3754907 0 2024-09-10T04:05:20.399 INFO:tasks.workunit.client.1.smithi070.stdout:6/616: write d0/d3/d18/d1b/d17/d53/fa3 [162404,124724] 0 2024-09-10T04:05:20.401 INFO:tasks.workunit.client.1.smithi070.stdout:6/617: stat d0/d3/d18/d1b/d17/d3f 0 2024-09-10T04:05:20.401 INFO:tasks.workunit.client.0.smithi046.stdout:4/650: fdatasync d3/d12/d17/d22/f70 0 2024-09-10T04:05:20.403 INFO:tasks.workunit.client.0.smithi046.stdout:4/651: write d3/d12/d17/f6e [2429898,16104] 0 2024-09-10T04:05:20.403 INFO:tasks.workunit.client.1.smithi070.stdout:4/465: dread de/d13/d18/d26/f2c [4194304,4194304] 0 2024-09-10T04:05:20.408 INFO:tasks.workunit.client.0.smithi046.stdout:9/500: link d4/c42 d4/d29/c9c 0 2024-09-10T04:05:20.408 INFO:tasks.workunit.client.0.smithi046.stdout:7/518: write d5/d9/d12/f5d [1681724,86227] 0 2024-09-10T04:05:20.409 INFO:tasks.workunit.client.0.smithi046.stdout:5/615: symlink d7/d11/d14/d2f/da3/lde 0 2024-09-10T04:05:20.410 INFO:tasks.workunit.client.0.smithi046.stdout:7/519: readlink d5/d6/db/dd/d24/d76/l8f 0 2024-09-10T04:05:20.411 INFO:tasks.workunit.client.1.smithi070.stdout:8/589: rename d4/d16/d17/f79 to d4/d16/d17/d1a/d60/fc7 0 2024-09-10T04:05:20.411 INFO:tasks.workunit.client.1.smithi070.stdout:0/600: dwrite d8/d13/d47/d7b/f8c [0,4194304] 0 2024-09-10T04:05:20.412 INFO:tasks.workunit.client.1.smithi070.stdout:5/537: write d17/d19/d6a/d54/d57/fb0 [1192441,4963] 0 2024-09-10T04:05:20.413 INFO:tasks.workunit.client.1.smithi070.stdout:0/601: chown d8/d13/f5a 148 1 2024-09-10T04:05:20.413 INFO:tasks.workunit.client.1.smithi070.stdout:2/511: fsync d15/d18/f20 0 2024-09-10T04:05:20.413 INFO:tasks.workunit.client.1.smithi070.stdout:5/538: chown d17/d24/d77/c8b 12871 1 2024-09-10T04:05:20.415 INFO:tasks.workunit.client.1.smithi070.stdout:7/509: link d2/d4e/c9e d2/d40/d6c/db2/db7/cbc 0 2024-09-10T04:05:20.415 INFO:tasks.workunit.client.0.smithi046.stdout:2/500: mkdir d2/d86/d9c/dbc 0 2024-09-10T04:05:20.416 INFO:tasks.workunit.client.1.smithi070.stdout:5/539: read d17/d24/d2b/f81 [324850,109663] 0 2024-09-10T04:05:20.416 INFO:tasks.workunit.client.1.smithi070.stdout:4/466: creat de/d13/d18/d26/d46/fa1 x:0 0 0 2024-09-10T04:05:20.418 INFO:tasks.workunit.client.0.smithi046.stdout:4/652: truncate d3/d12/d17/d29/f4f 287889 0 2024-09-10T04:05:20.420 INFO:tasks.workunit.client.0.smithi046.stdout:9/501: mknod d4/d44/c9d 0 2024-09-10T04:05:20.424 INFO:tasks.workunit.client.0.smithi046.stdout:7/520: write d5/d6/d55/d89/f9d [1361392,98142] 0 2024-09-10T04:05:20.425 INFO:tasks.workunit.client.0.smithi046.stdout:3/514: creat d12/d5d/fae x:0 0 0 2024-09-10T04:05:20.425 INFO:tasks.workunit.client.1.smithi070.stdout:8/590: creat d4/d39/fc8 x:0 0 0 2024-09-10T04:05:20.429 INFO:tasks.workunit.client.0.smithi046.stdout:4/653: creat d3/d12/fe2 x:0 0 0 2024-09-10T04:05:20.430 INFO:tasks.workunit.client.0.smithi046.stdout:9/502: rmdir d4/d3c/d4d 39 2024-09-10T04:05:20.432 INFO:tasks.workunit.client.0.smithi046.stdout:5/616: link d7/d11/d14/d2f/d64/fc6 d7/d11/d3f/d5a/fdf 0 2024-09-10T04:05:20.433 INFO:tasks.workunit.client.0.smithi046.stdout:5/617: stat d7/d11/d14/d2f/d64/fc6 0 2024-09-10T04:05:20.434 INFO:tasks.workunit.client.1.smithi070.stdout:6/618: mknod d0/d3/d18/cbe 0 2024-09-10T04:05:20.436 INFO:tasks.workunit.client.1.smithi070.stdout:7/510: mkdir d2/d3/dc/d13/dbd 0 2024-09-10T04:05:20.438 INFO:tasks.workunit.client.1.smithi070.stdout:2/512: write d15/d18/f27 [4927358,27055] 0 2024-09-10T04:05:20.439 INFO:tasks.workunit.client.1.smithi070.stdout:2/513: write d15/d18/d56/d7c/d55/d6d/f7d [3649653,79924] 0 2024-09-10T04:05:20.440 INFO:tasks.workunit.client.1.smithi070.stdout:4/467: unlink de/d13/d18/d26/d46/c58 0 2024-09-10T04:05:20.441 INFO:tasks.workunit.client.1.smithi070.stdout:3/552: getdents db 0 2024-09-10T04:05:20.441 INFO:tasks.workunit.client.0.smithi046.stdout:7/521: write d5/d6/d32/f95 [695176,69611] 0 2024-09-10T04:05:20.443 INFO:tasks.workunit.client.0.smithi046.stdout:4/654: creat d3/fe3 x:0 0 0 2024-09-10T04:05:20.443 INFO:tasks.workunit.client.0.smithi046.stdout:3/515: dwrite d12/d5d/d71/f5b [0,4194304] 0 2024-09-10T04:05:20.445 INFO:tasks.workunit.client.1.smithi070.stdout:8/591: creat d4/d16/d17/d19/d28/d43/d52/fc9 x:0 0 0 2024-09-10T04:05:20.445 INFO:tasks.workunit.client.1.smithi070.stdout:8/592: dread - d4/d16/f34 zero size 2024-09-10T04:05:20.446 INFO:tasks.workunit.client.0.smithi046.stdout:9/503: truncate d4/fb 2206843 0 2024-09-10T04:05:20.447 INFO:tasks.workunit.client.1.smithi070.stdout:8/593: write d4/d16/d17/d19/d62/fbc [449055,120430] 0 2024-09-10T04:05:20.450 INFO:tasks.workunit.client.0.smithi046.stdout:2/501: rename d2/d6/d23/d42/l50 to d2/d6/lbd 0 2024-09-10T04:05:20.451 INFO:tasks.workunit.client.1.smithi070.stdout:6/619: dread d0/d3/d18/d1b/d33/f65 [0,4194304] 0 2024-09-10T04:05:20.452 INFO:tasks.workunit.client.0.smithi046.stdout:3/516: dwrite d12/d5d/d71/d49/d5c/d9f/fa3 [0,4194304] 0 2024-09-10T04:05:20.454 INFO:tasks.workunit.client.1.smithi070.stdout:5/540: link d17/d19/f23 d17/d24/d84/d9b/fb5 0 2024-09-10T04:05:20.455 INFO:tasks.workunit.client.0.smithi046.stdout:9/504: rmdir d4/d7 39 2024-09-10T04:05:20.455 INFO:tasks.workunit.client.0.smithi046.stdout:3/517: fdatasync d12/d13/d14/d1e/d2d/f88 0 2024-09-10T04:05:20.459 INFO:tasks.workunit.client.1.smithi070.stdout:4/468: symlink de/d13/d5b/d7b/la2 0 2024-09-10T04:05:20.461 INFO:tasks.workunit.client.0.smithi046.stdout:7/522: rename d5/d6/d39/d6c to d5/d9/d12/d72/d77/da7 0 2024-09-10T04:05:20.462 INFO:tasks.workunit.client.1.smithi070.stdout:3/553: symlink db/d83/lb9 0 2024-09-10T04:05:20.463 INFO:tasks.workunit.client.0.smithi046.stdout:2/502: write d2/d6/d23/d55/f7e [59490,24919] 0 2024-09-10T04:05:20.466 INFO:tasks.workunit.client.1.smithi070.stdout:4/469: dwrite de/d13/d43/f8d [0,4194304] 0 2024-09-10T04:05:20.471 INFO:tasks.workunit.client.0.smithi046.stdout:3/518: mkdir d12/d13/d14/d1e/d2d/d39/d67/daf 0 2024-09-10T04:05:20.475 INFO:tasks.workunit.client.1.smithi070.stdout:6/620: unlink d0/d3/d18/d1b/l97 0 2024-09-10T04:05:20.475 INFO:tasks.workunit.client.0.smithi046.stdout:5/618: write d7/d11/d3f/d53/da9/f62 [1320013,124021] 0 2024-09-10T04:05:20.476 INFO:tasks.workunit.client.0.smithi046.stdout:9/505: dread d4/d7/d1f/d54/f6e [0,4194304] 0 2024-09-10T04:05:20.477 INFO:tasks.workunit.client.0.smithi046.stdout:7/523: creat d5/d6/d55/d89/fa8 x:0 0 0 2024-09-10T04:05:20.479 INFO:tasks.workunit.client.0.smithi046.stdout:9/506: write d4/d44/f86 [576594,101791] 0 2024-09-10T04:05:20.479 INFO:tasks.workunit.client.1.smithi070.stdout:5/541: rename d17/f64 to d17/d6f/d32/d90/fb6 0 2024-09-10T04:05:20.482 INFO:tasks.workunit.client.0.smithi046.stdout:2/503: dread d2/d6/d15/fa4 [0,4194304] 0 2024-09-10T04:05:20.482 INFO:tasks.workunit.client.1.smithi070.stdout:2/514: mkdir d15/d18/db2 0 2024-09-10T04:05:20.483 INFO:tasks.workunit.client.1.smithi070.stdout:2/515: write d15/d18/d29/f46 [4884658,101574] 0 2024-09-10T04:05:20.487 INFO:tasks.workunit.client.0.smithi046.stdout:7/524: unlink d5/d6/d32/d59/l90 0 2024-09-10T04:05:20.488 INFO:tasks.workunit.client.1.smithi070.stdout:0/602: getdents d8/d13/d1a 0 2024-09-10T04:05:20.490 INFO:tasks.workunit.client.0.smithi046.stdout:5/619: creat d7/d11/d3f/d53/da9/d9c/da5/dcc/dc0/fe0 x:0 0 0 2024-09-10T04:05:20.490 INFO:tasks.workunit.client.1.smithi070.stdout:0/603: dread d8/d24/d2d/d8d/f79 [0,4194304] 0 2024-09-10T04:05:20.491 INFO:tasks.workunit.client.1.smithi070.stdout:4/470: rmdir de/d13/d5b/d7b 39 2024-09-10T04:05:20.491 INFO:tasks.workunit.client.1.smithi070.stdout:0/604: chown d8/d13/d33/d4f 14517416 1 2024-09-10T04:05:20.493 INFO:tasks.workunit.client.0.smithi046.stdout:9/507: rename d4/d7/d1f/d54/d56 to d4/d18/d89/d9e 0 2024-09-10T04:05:20.495 INFO:tasks.workunit.client.1.smithi070.stdout:7/511: creat d2/fbe x:0 0 0 2024-09-10T04:05:20.495 INFO:tasks.workunit.client.1.smithi070.stdout:7/512: chown d2/d40/l59 1 1 2024-09-10T04:05:20.496 INFO:tasks.workunit.client.1.smithi070.stdout:6/621: mknod d0/d3/d18/d1b/d17/d3f/d40/cbf 0 2024-09-10T04:05:20.496 INFO:tasks.workunit.client.0.smithi046.stdout:4/655: getdents d3/d12/d17/d29/ddd 0 2024-09-10T04:05:20.497 INFO:tasks.workunit.client.1.smithi070.stdout:7/513: truncate d2/d3/dc/d5d/d42/fa0 57947 0 2024-09-10T04:05:20.497 INFO:tasks.workunit.client.1.smithi070.stdout:7/514: write d2/d3/dc/d5d/f46 [2226293,107731] 0 2024-09-10T04:05:20.500 INFO:tasks.workunit.client.1.smithi070.stdout:2/516: mknod d15/d18/d29/d2a/d2e/d32/d36/cb3 0 2024-09-10T04:05:20.502 INFO:tasks.workunit.client.1.smithi070.stdout:2/517: truncate d15/d18/d29/d2a/d2e/d32/f49 5047977 0 2024-09-10T04:05:20.502 INFO:tasks.workunit.client.1.smithi070.stdout:2/518: stat d15/d18/f26 0 2024-09-10T04:05:20.503 INFO:tasks.workunit.client.1.smithi070.stdout:2/519: chown d15/d18/d29/d2a/d2e/c5c 13 1 2024-09-10T04:05:20.509 INFO:tasks.workunit.client.1.smithi070.stdout:4/471: read de/d13/d18/d23/f61 [395201,112099] 0 2024-09-10T04:05:20.509 INFO:tasks.workunit.client.0.smithi046.stdout:2/504: dwrite d2/d8/d37/f41 [4194304,4194304] 0 2024-09-10T04:05:20.510 INFO:tasks.workunit.client.1.smithi070.stdout:4/472: read de/d13/d5b/f78 [2904024,98832] 0 2024-09-10T04:05:20.510 INFO:tasks.workunit.client.1.smithi070.stdout:8/594: creat d4/d16/d17/d1a/d60/d7a/fca x:0 0 0 2024-09-10T04:05:20.511 INFO:tasks.workunit.client.1.smithi070.stdout:4/473: readlink de/d13/d18/d23/l27 0 2024-09-10T04:05:20.512 INFO:tasks.workunit.client.1.smithi070.stdout:8/595: readlink d4/dbf/d87/d8e/l75 0 2024-09-10T04:05:20.512 INFO:tasks.workunit.client.0.smithi046.stdout:5/620: mknod d7/d11/ce1 0 2024-09-10T04:05:20.513 INFO:tasks.workunit.client.1.smithi070.stdout:8/596: chown d4/d39/l3b 698750 1 2024-09-10T04:05:20.513 INFO:tasks.workunit.client.1.smithi070.stdout:5/542: symlink d17/d37/d93/lb7 0 2024-09-10T04:05:20.514 INFO:tasks.workunit.client.1.smithi070.stdout:5/543: chown d17/d37/fa6 24270 1 2024-09-10T04:05:20.515 INFO:tasks.workunit.client.1.smithi070.stdout:0/605: dwrite d8/fc [4194304,4194304] 0 2024-09-10T04:05:20.516 INFO:tasks.workunit.client.0.smithi046.stdout:7/525: dwrite d5/d7f/d78/fa6 [0,4194304] 0 2024-09-10T04:05:20.516 INFO:tasks.workunit.client.0.smithi046.stdout:9/508: dread d4/d3c/d4d/d5c/f4e [0,4194304] 0 2024-09-10T04:05:20.517 INFO:tasks.workunit.client.1.smithi070.stdout:0/606: write d8/d13/d33/d81/fb7 [976507,10849] 0 2024-09-10T04:05:20.518 INFO:tasks.workunit.client.0.smithi046.stdout:4/656: mkdir d3/dab/dca/de4 0 2024-09-10T04:05:20.519 INFO:tasks.workunit.client.0.smithi046.stdout:4/657: dread - d3/d12/d17/d29/d58/db2/d5f/d85/da8/fbe zero size 2024-09-10T04:05:20.520 INFO:tasks.workunit.client.1.smithi070.stdout:7/515: symlink d2/d7/d9/d18/d3c/lbf 0 2024-09-10T04:05:20.520 INFO:tasks.workunit.client.1.smithi070.stdout:3/554: link db/d1c/d25/d28/f41 db/d1c/d25/d28/fba 0 2024-09-10T04:05:20.522 INFO:tasks.workunit.client.1.smithi070.stdout:2/520: symlink d15/d18/d56/d7c/d55/d6d/lb4 0 2024-09-10T04:05:20.526 INFO:tasks.workunit.client.0.smithi046.stdout:7/526: mknod d5/d6/d32/ca9 0 2024-09-10T04:05:20.529 INFO:tasks.workunit.client.0.smithi046.stdout:7/527: write d5/f60 [5117167,129143] 0 2024-09-10T04:05:20.530 INFO:tasks.workunit.client.1.smithi070.stdout:9/582: sync 2024-09-10T04:05:20.530 INFO:tasks.workunit.client.1.smithi070.stdout:6/622: mkdir d0/d3/da4/dc0 0 2024-09-10T04:05:20.531 INFO:tasks.workunit.client.1.smithi070.stdout:4/474: rename de/d13/d18/d23/d39/d77/f97 to de/d13/d43/d88/fa3 0 2024-09-10T04:05:20.532 INFO:tasks.workunit.client.1.smithi070.stdout:9/583: read d9/d1c/d34/f76 [3009392,109538] 0 2024-09-10T04:05:20.533 INFO:tasks.workunit.client.0.smithi046.stdout:4/658: rename d3/d12/d17/f5e to d3/dab/fe5 0 2024-09-10T04:05:20.536 INFO:tasks.workunit.client.0.smithi046.stdout:4/659: write d3/d12/d17/d29/d58/db2/d5f/d85/fa2 [4251459,17078] 0 2024-09-10T04:05:20.539 INFO:tasks.workunit.client.1.smithi070.stdout:9/584: dread d9/d7a/d74/f47 [4194304,4194304] 0 2024-09-10T04:05:20.542 INFO:tasks.workunit.client.1.smithi070.stdout:3/555: creat db/d79/fbb x:0 0 0 2024-09-10T04:05:20.542 INFO:tasks.workunit.client.0.smithi046.stdout:7/528: mknod d5/d6/db/dd/d24/caa 0 2024-09-10T04:05:20.545 INFO:tasks.workunit.client.1.smithi070.stdout:2/521: unlink d15/la2 0 2024-09-10T04:05:20.546 INFO:tasks.workunit.client.0.smithi046.stdout:9/509: link d4/d3c/c74 d4/d3c/d4d/d5c/d87/c9f 0 2024-09-10T04:05:20.549 INFO:tasks.workunit.client.0.smithi046.stdout:2/505: truncate d2/d6/d23/d42/f53 2766586 0 2024-09-10T04:05:20.550 INFO:tasks.workunit.client.0.smithi046.stdout:7/529: dwrite d5/d9/d12/fa4 [0,4194304] 0 2024-09-10T04:05:20.552 INFO:tasks.workunit.client.0.smithi046.stdout:7/530: readlink d5/d33/l91 0 2024-09-10T04:05:20.553 INFO:tasks.workunit.client.0.smithi046.stdout:9/510: symlink d4/d3c/d4d/d80/la0 0 2024-09-10T04:05:20.553 INFO:tasks.workunit.client.0.smithi046.stdout:5/621: dwrite d7/d11/d14/f71 [0,4194304] 0 2024-09-10T04:05:20.554 INFO:tasks.workunit.client.0.smithi046.stdout:7/531: write d5/d6/d39/fa0 [742572,77377] 0 2024-09-10T04:05:20.556 INFO:tasks.workunit.client.0.smithi046.stdout:9/511: fdatasync d4/d7/d61/f6f 0 2024-09-10T04:05:20.559 INFO:tasks.workunit.client.0.smithi046.stdout:5/622: truncate d7/d11/d3f/d5a/fb2 513184 0 2024-09-10T04:05:20.559 INFO:tasks.workunit.client.0.smithi046.stdout:2/506: fdatasync d2/d49/f74 0 2024-09-10T04:05:20.560 INFO:tasks.workunit.client.1.smithi070.stdout:0/607: rename d8/d13/d1a/d8f/d95/cbf to d8/d13/d1a/da0/cc8 0 2024-09-10T04:05:20.562 INFO:tasks.workunit.client.0.smithi046.stdout:7/532: mkdir d5/d6/d32/dab 0 2024-09-10T04:05:20.563 INFO:tasks.workunit.client.1.smithi070.stdout:5/544: creat d17/d24/d84/dab/fb8 x:0 0 0 2024-09-10T04:05:20.563 INFO:tasks.workunit.client.0.smithi046.stdout:9/512: fsync d4/d18/d89/d9e/f8b 0 2024-09-10T04:05:20.564 INFO:tasks.workunit.client.0.smithi046.stdout:7/533: write d5/d6/d55/d89/f9d [1440109,35935] 0 2024-09-10T04:05:20.566 INFO:tasks.workunit.client.1.smithi070.stdout:7/516: creat d2/d40/d6c/db2/db7/fc0 x:0 0 0 2024-09-10T04:05:20.567 INFO:tasks.workunit.client.1.smithi070.stdout:7/517: read d2/d7/d9/d18/f3f [432969,69000] 0 2024-09-10T04:05:20.568 INFO:tasks.workunit.client.0.smithi046.stdout:5/623: unlink d7/d11/d14/d2f/da3/cb0 0 2024-09-10T04:05:20.569 INFO:tasks.workunit.client.1.smithi070.stdout:7/518: write d2/d7/d9/d18/d3c/f73 [232539,74470] 0 2024-09-10T04:05:20.570 INFO:tasks.workunit.client.1.smithi070.stdout:2/522: creat d15/d18/d29/d2a/d2e/d32/fb5 x:0 0 0 2024-09-10T04:05:20.570 INFO:tasks.workunit.client.0.smithi046.stdout:5/624: chown d7/f94 436 1 2024-09-10T04:05:20.570 INFO:tasks.workunit.client.0.smithi046.stdout:2/507: dwrite d2/d6/d23/d5d/d77/f54 [0,4194304] 0 2024-09-10T04:05:20.570 INFO:tasks.workunit.client.1.smithi070.stdout:2/523: readlink d15/d18/d29/d2a/d2e/d32/d9b/l8d 0 2024-09-10T04:05:20.575 INFO:tasks.workunit.client.1.smithi070.stdout:1/447: sync 2024-09-10T04:05:20.579 INFO:tasks.workunit.client.1.smithi070.stdout:8/597: creat d4/d16/d17/d1a/d60/d7a/fcb x:0 0 0 2024-09-10T04:05:20.581 INFO:tasks.workunit.client.0.smithi046.stdout:5/625: mkdir d7/d11/d14/d8e/de2 0 2024-09-10T04:05:20.582 INFO:tasks.workunit.client.0.smithi046.stdout:4/660: truncate d3/dab/fbd 973815 0 2024-09-10T04:05:20.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:20 smithi070 ceph-mon[35495]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:20.585 INFO:tasks.workunit.client.1.smithi070.stdout:0/608: symlink d8/d13/d1a/d8f/lc9 0 2024-09-10T04:05:20.587 INFO:tasks.workunit.client.1.smithi070.stdout:5/545: mkdir d17/d19/d6a/d54/d57/d63/db9 0 2024-09-10T04:05:20.589 INFO:tasks.workunit.client.1.smithi070.stdout:9/585: creat d9/d7a/d74/d8c/fb8 x:0 0 0 2024-09-10T04:05:20.593 INFO:tasks.workunit.client.0.smithi046.stdout:5/626: dread d7/d11/d3f/d5a/d63/f65 [0,4194304] 0 2024-09-10T04:05:20.593 INFO:tasks.workunit.client.0.smithi046.stdout:9/513: link d4/d44/f94 d4/fa1 0 2024-09-10T04:05:20.594 INFO:tasks.workunit.client.0.smithi046.stdout:9/514: stat d4/d7/d1f/f26 0 2024-09-10T04:05:20.598 INFO:tasks.workunit.client.1.smithi070.stdout:6/623: rename d0/d3/d18/d1b/d17/c26 to d0/d3/d18/d1b/d17/d3f/d40/d94/cc1 0 2024-09-10T04:05:20.600 INFO:tasks.workunit.client.1.smithi070.stdout:6/624: read d0/d3/d18/d62/d70/fb1 [244351,80644] 0 2024-09-10T04:05:20.600 INFO:tasks.workunit.client.1.smithi070.stdout:0/609: mkdir d8/d85/dca 0 2024-09-10T04:05:20.600 INFO:tasks.workunit.client.0.smithi046.stdout:2/508: dwrite d2/d6/d23/d55/f67 [0,4194304] 0 2024-09-10T04:05:20.605 INFO:tasks.workunit.client.0.smithi046.stdout:4/661: link d3/d12/d14/f18 d3/dab/dca/fe6 0 2024-09-10T04:05:20.607 INFO:tasks.workunit.client.0.smithi046.stdout:5/627: dwrite d7/d11/d3f/d53/fc7 [0,4194304] 0 2024-09-10T04:05:20.612 INFO:tasks.workunit.client.0.smithi046.stdout:2/509: mkdir d2/d6/d9d/db6/dbe 0 2024-09-10T04:05:20.612 INFO:tasks.workunit.client.1.smithi070.stdout:8/598: mkdir d4/d16/dcc 0 2024-09-10T04:05:20.613 INFO:tasks.workunit.client.1.smithi070.stdout:8/599: readlink d4/d16/d17/d19/d62/la6 0 2024-09-10T04:05:20.614 INFO:tasks.workunit.client.1.smithi070.stdout:1/448: write d3/d8/d13/d3f/f42 [4813736,89914] 0 2024-09-10T04:05:20.615 INFO:tasks.workunit.client.0.smithi046.stdout:2/510: dread d2/d6/d15/fa4 [0,4194304] 0 2024-09-10T04:05:20.616 INFO:tasks.workunit.client.0.smithi046.stdout:2/511: readlink d2/d6/d23/l94 0 2024-09-10T04:05:20.617 INFO:tasks.workunit.client.0.smithi046.stdout:2/512: dread - d2/d8/d65/fa9 zero size 2024-09-10T04:05:20.620 INFO:tasks.workunit.client.0.smithi046.stdout:4/662: getdents d3/d12/d17/d91/dbf 0 2024-09-10T04:05:20.623 INFO:tasks.workunit.client.0.smithi046.stdout:5/628: creat d7/d11/d3f/dd3/fe3 x:0 0 0 2024-09-10T04:05:20.624 INFO:tasks.workunit.client.0.smithi046.stdout:2/513: dwrite d2/d6/d23/f4b [0,4194304] 0 2024-09-10T04:05:20.628 INFO:tasks.workunit.client.0.smithi046.stdout:4/663: read d3/d12/d17/d29/d58/db2/f45 [3559583,47929] 0 2024-09-10T04:05:20.630 INFO:tasks.workunit.client.1.smithi070.stdout:3/556: rename db/d1c/l85 to db/d1c/d3e/lbc 0 2024-09-10T04:05:20.631 INFO:tasks.workunit.client.1.smithi070.stdout:1/449: dread d3/d10/f2c [0,4194304] 0 2024-09-10T04:05:20.634 INFO:tasks.workunit.client.0.smithi046.stdout:4/664: mknod d3/d12/d17/d29/dba/ce7 0 2024-09-10T04:05:20.635 INFO:tasks.workunit.client.0.smithi046.stdout:5/629: dwrite d7/d11/d3f/d5a/f69 [0,4194304] 0 2024-09-10T04:05:20.638 INFO:tasks.workunit.client.0.smithi046.stdout:5/630: symlink d7/d11/d14/d2f/d64/le4 0 2024-09-10T04:05:20.640 INFO:tasks.workunit.client.0.smithi046.stdout:4/665: dread d3/d12/d17/d29/f4f [0,4194304] 0 2024-09-10T04:05:20.642 INFO:tasks.workunit.client.0.smithi046.stdout:5/631: mkdir d7/d11/d3f/d53/da9/d9c/da5/dcc/de5 0 2024-09-10T04:05:20.643 INFO:tasks.workunit.client.0.smithi046.stdout:4/666: dread d3/d12/d17/d22/f4c [0,4194304] 0 2024-09-10T04:05:20.645 INFO:tasks.workunit.client.0.smithi046.stdout:5/632: chown d7/d11/d14/d2f/d2b/lb9 470 1 2024-09-10T04:05:20.646 INFO:tasks.workunit.client.0.smithi046.stdout:5/633: write d7/d11/d3f/d53/d75/fbd [3816226,55108] 0 2024-09-10T04:05:20.647 INFO:tasks.workunit.client.0.smithi046.stdout:4/667: mkdir d3/dab/dca/de8 0 2024-09-10T04:05:20.651 INFO:tasks.workunit.client.0.smithi046.stdout:5/634: symlink d7/d11/d14/d8e/de2/le6 0 2024-09-10T04:05:20.652 INFO:tasks.workunit.client.0.smithi046.stdout:4/668: creat d3/dab/dca/de4/fe9 x:0 0 0 2024-09-10T04:05:20.653 INFO:tasks.workunit.client.0.smithi046.stdout:5/635: dread - d7/d11/d14/d2f/fd6 zero size 2024-09-10T04:05:20.655 INFO:tasks.workunit.client.0.smithi046.stdout:2/514: dwrite d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:20.655 INFO:tasks.workunit.client.0.smithi046.stdout:4/669: write d3/d12/d14/f7c [619633,18842] 0 2024-09-10T04:05:20.657 INFO:tasks.workunit.client.0.smithi046.stdout:5/636: symlink d7/d11/d14/d8e/d96/le7 0 2024-09-10T04:05:20.659 INFO:tasks.workunit.client.0.smithi046.stdout:5/637: write d7/d11/d14/d45/fc2 [1795956,117880] 0 2024-09-10T04:05:20.661 INFO:tasks.workunit.client.1.smithi070.stdout:5/546: truncate d17/d19/d6a/d54/d57/d63/f9a 2630966 0 2024-09-10T04:05:20.661 INFO:tasks.workunit.client.1.smithi070.stdout:0/610: mknod d8/d24/d97/dc5/ccb 0 2024-09-10T04:05:20.661 INFO:tasks.workunit.client.1.smithi070.stdout:3/557: chown db/l48 12 1 2024-09-10T04:05:20.662 INFO:tasks.workunit.client.0.smithi046.stdout:4/670: creat d3/d12/d17/d29/d58/db2/d5f/d85/da8/fea x:0 0 0 2024-09-10T04:05:20.665 INFO:tasks.workunit.client.0.smithi046.stdout:4/671: write d3/d12/d17/d29/d58/db2/d5f/d85/fd7 [809026,64966] 0 2024-09-10T04:05:20.665 INFO:tasks.workunit.client.1.smithi070.stdout:7/519: rename d2/d7/db/d20 to d2/d3/d63/da9/dc1 0 2024-09-10T04:05:20.666 INFO:tasks.workunit.client.0.smithi046.stdout:5/638: creat d7/d11/d3f/d53/d81/fe8 x:0 0 0 2024-09-10T04:05:20.667 INFO:tasks.workunit.client.1.smithi070.stdout:7/520: stat d2/d3/caa 0 2024-09-10T04:05:20.668 INFO:tasks.workunit.client.0.smithi046.stdout:2/515: write d2/d6/d23/d42/f53 [2582990,129506] 0 2024-09-10T04:05:20.670 INFO:tasks.workunit.client.1.smithi070.stdout:8/600: creat d4/fcd x:0 0 0 2024-09-10T04:05:20.670 INFO:tasks.workunit.client.0.smithi046.stdout:2/516: chown d2/d8/d37/c96 1 1 2024-09-10T04:05:20.671 INFO:tasks.workunit.client.1.smithi070.stdout:3/558: dwrite db/d79/fa8 [0,4194304] 0 2024-09-10T04:05:20.671 INFO:tasks.workunit.client.1.smithi070.stdout:7/521: truncate d2/d7/db/d3a/f9a 982606 0 2024-09-10T04:05:20.672 INFO:tasks.workunit.client.1.smithi070.stdout:3/559: dread - db/d79/fa9 zero size 2024-09-10T04:05:20.672 INFO:tasks.workunit.client.1.smithi070.stdout:7/522: chown d2/d7/d9/d18/d23 527791 1 2024-09-10T04:05:20.673 INFO:tasks.workunit.client.0.smithi046.stdout:5/639: fdatasync d7/d11/d14/d2f/f22 0 2024-09-10T04:05:20.676 INFO:tasks.workunit.client.0.smithi046.stdout:4/672: dwrite d3/d12/d17/d29/d58/f71 [0,4194304] 0 2024-09-10T04:05:20.676 INFO:tasks.workunit.client.1.smithi070.stdout:2/524: rename d15/d2b/f4c to d15/d59/da1/fb6 0 2024-09-10T04:05:20.676 INFO:tasks.workunit.client.1.smithi070.stdout:5/547: rename d17/d19/d6a to d17/d19/d6a/d54/d57/d63/db9/dba 22 2024-09-10T04:05:20.681 INFO:tasks.workunit.client.1.smithi070.stdout:5/548: dwrite fc [0,4194304] 0 2024-09-10T04:05:20.682 INFO:tasks.workunit.client.0.smithi046.stdout:2/517: dwrite d2/d6/d23/f4b [0,4194304] 0 2024-09-10T04:05:20.685 INFO:tasks.workunit.client.1.smithi070.stdout:0/611: dread d8/f1b [0,4194304] 0 2024-09-10T04:05:20.686 INFO:tasks.workunit.client.1.smithi070.stdout:9/586: link d9/d1c/d34/d63/ca1 d9/d1c/cb9 0 2024-09-10T04:05:20.686 INFO:tasks.workunit.client.1.smithi070.stdout:0/612: chown d8/d13/d33/d4f/fbb 0 1 2024-09-10T04:05:20.688 INFO:tasks.workunit.client.1.smithi070.stdout:0/613: truncate d8/d13/d33/d4f/fbb 427550 0 2024-09-10T04:05:20.689 INFO:tasks.workunit.client.1.smithi070.stdout:0/614: chown d8/cb8 2606 1 2024-09-10T04:05:20.690 INFO:tasks.workunit.client.0.smithi046.stdout:4/673: symlink d3/d12/d17/de1/ddf/leb 0 2024-09-10T04:05:20.691 INFO:tasks.workunit.client.0.smithi046.stdout:2/518: mknod d2/d6/d23/d5d/cbf 0 2024-09-10T04:05:20.692 INFO:tasks.workunit.client.1.smithi070.stdout:7/523: fsync d2/d7/d9/d18/d23/f47 0 2024-09-10T04:05:20.694 INFO:tasks.workunit.client.0.smithi046.stdout:5/640: creat d7/d11/fe9 x:0 0 0 2024-09-10T04:05:20.697 INFO:tasks.workunit.client.1.smithi070.stdout:8/601: dwrite d4/d16/f22 [0,4194304] 0 2024-09-10T04:05:20.698 INFO:tasks.workunit.client.0.smithi046.stdout:2/519: symlink d2/d6/d23/d5d/d5e/lc0 0 2024-09-10T04:05:20.701 INFO:tasks.workunit.client.0.smithi046.stdout:5/641: symlink d7/d11/d14/d2f/d2b/d92/lea 0 2024-09-10T04:05:20.702 INFO:tasks.workunit.client.0.smithi046.stdout:4/674: mkdir d3/d12/dec 0 2024-09-10T04:05:20.705 INFO:tasks.workunit.client.0.smithi046.stdout:2/520: rename d2/d6/d23/f51 to d2/d8/d37/fc1 0 2024-09-10T04:05:20.708 INFO:tasks.workunit.client.1.smithi070.stdout:9/587: creat d9/d1c/dac/d8a/d1b/fba x:0 0 0 2024-09-10T04:05:20.708 INFO:tasks.workunit.client.0.smithi046.stdout:4/675: mknod d3/d12/d17/d22/d51/d5c/d6c/ced 0 2024-09-10T04:05:20.711 INFO:tasks.workunit.client.1.smithi070.stdout:0/615: creat d8/d13/d1a/d8f/d95/fcc x:0 0 0 2024-09-10T04:05:20.712 INFO:tasks.workunit.client.0.smithi046.stdout:4/676: dread d3/d12/f73 [0,4194304] 0 2024-09-10T04:05:20.714 INFO:tasks.workunit.client.1.smithi070.stdout:7/524: unlink d2/d3/caa 0 2024-09-10T04:05:20.714 INFO:tasks.workunit.client.0.smithi046.stdout:4/677: read d3/d12/d17/f6e [3885842,68295] 0 2024-09-10T04:05:20.716 INFO:tasks.workunit.client.1.smithi070.stdout:5/549: mknod d17/d24/d84/d9b/cbb 0 2024-09-10T04:05:20.718 INFO:tasks.workunit.client.0.smithi046.stdout:2/521: read - d2/d8/d37/fc1 zero size 2024-09-10T04:05:20.719 INFO:tasks.workunit.client.1.smithi070.stdout:3/560: creat db/d1c/d25/fbd x:0 0 0 2024-09-10T04:05:20.719 INFO:tasks.workunit.client.0.smithi046.stdout:2/522: chown d2/d8/d65/fa9 0 1 2024-09-10T04:05:20.720 INFO:tasks.workunit.client.1.smithi070.stdout:2/525: dread d15/d18/f21 [0,4194304] 0 2024-09-10T04:05:20.720 INFO:tasks.workunit.client.1.smithi070.stdout:0/616: dwrite d8/d13/d33/d6f/f90 [0,4194304] 0 2024-09-10T04:05:20.722 INFO:tasks.workunit.client.0.smithi046.stdout:4/678: dwrite d3/d12/d17/d22/d51/fcc [0,4194304] 0 2024-09-10T04:05:20.722 INFO:tasks.workunit.client.1.smithi070.stdout:3/561: readlink db/d1c/d25/d28/d43/l6c 0 2024-09-10T04:05:20.724 INFO:tasks.workunit.client.1.smithi070.stdout:1/450: getdents d3/d8/d13 0 2024-09-10T04:05:20.725 INFO:tasks.workunit.client.1.smithi070.stdout:0/617: truncate d8/d13/d1a/d75/f84 1792303 0 2024-09-10T04:05:20.728 INFO:tasks.workunit.client.0.smithi046.stdout:4/679: fsync d3/d12/d14/f62 0 2024-09-10T04:05:20.728 INFO:tasks.workunit.client.1.smithi070.stdout:0/618: write d8/d13/d1a/f3c [515757,60118] 0 2024-09-10T04:05:20.728 INFO:tasks.workunit.client.1.smithi070.stdout:7/525: creat d2/d40/d6c/db2/fc2 x:0 0 0 2024-09-10T04:05:20.731 INFO:tasks.workunit.client.1.smithi070.stdout:7/526: stat d2/d7/d9/d18/l28 0 2024-09-10T04:05:20.731 INFO:tasks.workunit.client.1.smithi070.stdout:9/588: dwrite d9/d1c/dac/d8a/d82/d99/f9d [4194304,4194304] 0 2024-09-10T04:05:20.732 INFO:tasks.workunit.client.0.smithi046.stdout:2/523: fsync d2/f35 0 2024-09-10T04:05:20.733 INFO:tasks.workunit.client.0.smithi046.stdout:2/524: fsync d2/d6/f18 0 2024-09-10T04:05:20.734 INFO:tasks.workunit.client.1.smithi070.stdout:7/527: write d2/d7/db/d3a/fac [380506,89337] 0 2024-09-10T04:05:20.734 INFO:tasks.workunit.client.1.smithi070.stdout:9/589: write d9/fb7 [741540,42049] 0 2024-09-10T04:05:20.736 INFO:tasks.workunit.client.1.smithi070.stdout:8/602: dread d4/d16/d17/f68 [0,4194304] 0 2024-09-10T04:05:20.736 INFO:tasks.workunit.client.1.smithi070.stdout:5/550: rename d17/d19/d6a/d54/la4 to d17/d37/d93/lbc 0 2024-09-10T04:05:20.736 INFO:tasks.workunit.client.0.smithi046.stdout:8/502: sync 2024-09-10T04:05:20.736 INFO:tasks.workunit.client.1.smithi070.stdout:9/590: stat d9/l16 0 2024-09-10T04:05:20.742 INFO:tasks.workunit.client.0.smithi046.stdout:2/525: mkdir d2/d8/d37/dc2 0 2024-09-10T04:05:20.744 INFO:tasks.workunit.client.1.smithi070.stdout:3/562: rename db/f22 to db/d79/db6/fbe 0 2024-09-10T04:05:20.744 INFO:tasks.workunit.client.0.smithi046.stdout:4/680: mknod d3/d12/d17/d29/ddd/cee 0 2024-09-10T04:05:20.745 INFO:tasks.workunit.client.1.smithi070.stdout:7/528: dread d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:20.745 INFO:tasks.workunit.client.1.smithi070.stdout:2/526: mknod d15/d18/cb7 0 2024-09-10T04:05:20.746 INFO:tasks.workunit.client.1.smithi070.stdout:2/527: stat f11 0 2024-09-10T04:05:20.746 INFO:tasks.workunit.client.0.smithi046.stdout:2/526: creat d2/d6/d23/d5d/d77/fc3 x:0 0 0 2024-09-10T04:05:20.748 INFO:tasks.workunit.client.1.smithi070.stdout:5/551: dwrite d17/d19/d6a/d54/fb1 [0,4194304] 0 2024-09-10T04:05:20.748 INFO:tasks.workunit.client.1.smithi070.stdout:7/529: chown d2/d3/dc/d5d/f46 8123185 1 2024-09-10T04:05:20.749 INFO:tasks.workunit.client.1.smithi070.stdout:2/528: fsync d15/d18/d29/d2a/d2e/d32/f49 0 2024-09-10T04:05:20.750 INFO:tasks.workunit.client.1.smithi070.stdout:0/619: mknod d8/d13/d33/d81/ccd 0 2024-09-10T04:05:20.750 INFO:tasks.workunit.client.1.smithi070.stdout:3/563: truncate db/d1c/d25/d28/d43/f97 61291 0 2024-09-10T04:05:20.752 INFO:tasks.workunit.client.1.smithi070.stdout:8/603: rmdir d4/dbf/d74/da4 39 2024-09-10T04:05:20.752 INFO:tasks.workunit.client.1.smithi070.stdout:0/620: write f5 [3576169,123301] 0 2024-09-10T04:05:20.755 INFO:tasks.workunit.client.1.smithi070.stdout:0/621: write d8/d24/d36/d87/fae [770258,54820] 0 2024-09-10T04:05:20.756 INFO:tasks.workunit.client.1.smithi070.stdout:1/451: dread d3/d8/d12/d2f/d32/f4c [0,4194304] 0 2024-09-10T04:05:20.756 INFO:tasks.workunit.client.1.smithi070.stdout:4/475: sync 2024-09-10T04:05:20.757 INFO:tasks.workunit.client.1.smithi070.stdout:0/622: dread - d8/d13/d47/fb6 zero size 2024-09-10T04:05:20.758 INFO:tasks.workunit.client.0.smithi046.stdout:4/681: write d3/d12/d17/d29/d58/f6d [4392218,16747] 0 2024-09-10T04:05:20.758 INFO:tasks.workunit.client.1.smithi070.stdout:0/623: read d8/fa [3796273,75907] 0 2024-09-10T04:05:20.759 INFO:tasks.workunit.client.0.smithi046.stdout:4/682: fsync d3/d12/d17/d22/f4b 0 2024-09-10T04:05:20.759 INFO:tasks.workunit.client.0.smithi046.stdout:2/527: getdents d2/d6/d23/d55 0 2024-09-10T04:05:20.760 INFO:tasks.workunit.client.1.smithi070.stdout:0/624: chown d8/d24/d36/f48 14770 1 2024-09-10T04:05:20.760 INFO:tasks.workunit.client.1.smithi070.stdout:0/625: fdatasync d8/d13/d33/d81/fb7 0 2024-09-10T04:05:20.763 INFO:tasks.workunit.client.0.smithi046.stdout:4/683: dread - d3/d12/f5a zero size 2024-09-10T04:05:20.764 INFO:tasks.workunit.client.1.smithi070.stdout:2/529: mkdir d15/d2b/d6e/db8 0 2024-09-10T04:05:20.764 INFO:tasks.workunit.client.0.smithi046.stdout:2/528: mkdir d2/d6/d23/d5d/d77/d7b/dc4 0 2024-09-10T04:05:20.766 INFO:tasks.workunit.client.1.smithi070.stdout:9/591: mkdir d9/d7a/d6f/d90/d89/dbb 0 2024-09-10T04:05:20.767 INFO:tasks.workunit.client.1.smithi070.stdout:7/530: mknod d2/d7/d9/d18/d49/cc3 0 2024-09-10T04:05:20.769 INFO:tasks.workunit.client.1.smithi070.stdout:1/452: mkdir d3/d90 0 2024-09-10T04:05:20.769 INFO:tasks.workunit.client.1.smithi070.stdout:3/564: dwrite db/d79/db6/fbe [0,4194304] 0 2024-09-10T04:05:20.770 INFO:tasks.workunit.client.0.smithi046.stdout:2/529: dwrite d2/d6/d23/d5d/d77/f54 [0,4194304] 0 2024-09-10T04:05:20.772 INFO:tasks.workunit.client.1.smithi070.stdout:4/476: fdatasync de/d13/d5b/f78 0 2024-09-10T04:05:20.773 INFO:tasks.workunit.client.0.smithi046.stdout:4/684: link d3/d12/d17/d29/d58/db2/l52 d3/d12/d17/de1/ddf/lef 0 2024-09-10T04:05:20.774 INFO:tasks.workunit.client.1.smithi070.stdout:4/477: stat de/d13/f45 0 2024-09-10T04:05:20.776 INFO:tasks.workunit.client.1.smithi070.stdout:5/552: symlink d17/d19/d6a/d54/d57/da8/lbd 0 2024-09-10T04:05:20.778 INFO:tasks.workunit.client.1.smithi070.stdout:9/592: creat d9/d1c/d34/fbc x:0 0 0 2024-09-10T04:05:20.779 INFO:tasks.workunit.client.1.smithi070.stdout:8/604: creat d4/d16/d17/dab/fce x:0 0 0 2024-09-10T04:05:20.780 INFO:tasks.workunit.client.0.smithi046.stdout:2/530: dwrite d2/d57/fb5 [0,4194304] 0 2024-09-10T04:05:20.781 INFO:tasks.workunit.client.1.smithi070.stdout:1/453: dwrite d3/d8/d13/f43 [0,4194304] 0 2024-09-10T04:05:20.781 INFO:tasks.workunit.client.0.smithi046.stdout:4/685: rename d3/d12/d17/dc6 to d3/d12/dec/df0 0 2024-09-10T04:05:20.781 INFO:tasks.workunit.client.1.smithi070.stdout:3/565: dwrite db/d1c/d53/f57 [0,4194304] 0 2024-09-10T04:05:20.783 INFO:tasks.workunit.client.0.smithi046.stdout:2/531: truncate d2/d6/d23/d42/f45 2329310 0 2024-09-10T04:05:20.783 INFO:tasks.workunit.client.1.smithi070.stdout:8/605: write d4/d16/d17/d1a/f78 [5211312,84090] 0 2024-09-10T04:05:20.787 INFO:tasks.workunit.client.0.smithi046.stdout:4/686: creat d3/d12/d17/d22/d51/d5c/ff1 x:0 0 0 2024-09-10T04:05:20.790 INFO:tasks.workunit.client.0.smithi046.stdout:2/532: chown d2/d6/d23/d5d/d77/d7b/d85/l3d 2009 1 2024-09-10T04:05:20.790 INFO:tasks.workunit.client.1.smithi070.stdout:8/606: dread d4/dbf/d87/f9f [0,4194304] 0 2024-09-10T04:05:20.794 INFO:tasks.workunit.client.0.smithi046.stdout:4/687: dread d3/d12/d17/d22/d51/fcc [0,4194304] 0 2024-09-10T04:05:20.798 INFO:tasks.workunit.client.1.smithi070.stdout:2/530: link d15/d18/f7e d15/d18/d90/fb9 0 2024-09-10T04:05:20.800 INFO:tasks.workunit.client.1.smithi070.stdout:4/478: creat de/d13/d18/d26/fa4 x:0 0 0 2024-09-10T04:05:20.801 INFO:tasks.workunit.client.1.smithi070.stdout:4/479: chown de/d13/d18/d23/d47/d59/d79 33415928 1 2024-09-10T04:05:20.802 INFO:tasks.workunit.client.0.smithi046.stdout:4/688: unlink d3/d12/d17/d22/d51/d94/cdc 0 2024-09-10T04:05:20.804 INFO:tasks.workunit.client.1.smithi070.stdout:9/593: creat d9/d7a/d6f/d90/d89/fbd x:0 0 0 2024-09-10T04:05:20.805 INFO:tasks.workunit.client.1.smithi070.stdout:7/531: fsync d2/d3/dc/d5d/f34 0 2024-09-10T04:05:20.806 INFO:tasks.workunit.client.1.smithi070.stdout:7/532: fdatasync d2/d7/d9/d18/d23/f91 0 2024-09-10T04:05:20.807 INFO:tasks.workunit.client.1.smithi070.stdout:1/454: symlink d3/d8/d27/l91 0 2024-09-10T04:05:20.810 INFO:tasks.workunit.client.0.smithi046.stdout:2/533: getdents d2/d6/d23/d5d/d77/d7b 0 2024-09-10T04:05:20.813 INFO:tasks.workunit.client.1.smithi070.stdout:0/626: rename d8/d24/d97/dc5 to d8/d24/dce 0 2024-09-10T04:05:20.813 INFO:tasks.workunit.client.1.smithi070.stdout:4/480: stat de/d13/d5b/d7b/f81 0 2024-09-10T04:05:20.815 INFO:tasks.workunit.client.1.smithi070.stdout:4/481: write de/d13/d18/d26/d46/d85/f9a [264181,67242] 0 2024-09-10T04:05:20.815 INFO:tasks.workunit.client.0.smithi046.stdout:2/534: rename d2/d6/d23/f33 to d2/d6/d23/d5d/d77/d7b/d85/d8d/fc5 0 2024-09-10T04:05:20.817 INFO:tasks.workunit.client.1.smithi070.stdout:5/553: fsync d17/d19/f50 0 2024-09-10T04:05:20.820 INFO:tasks.workunit.client.0.smithi046.stdout:2/535: write d2/f73 [4871338,71661] 0 2024-09-10T04:05:20.822 INFO:tasks.workunit.client.0.smithi046.stdout:4/689: truncate d3/fa 1734756 0 2024-09-10T04:05:20.823 INFO:tasks.workunit.client.1.smithi070.stdout:1/455: fdatasync d3/d10/f3b 0 2024-09-10T04:05:20.826 INFO:tasks.workunit.client.1.smithi070.stdout:0/627: dread d8/d13/d1a/f35 [0,4194304] 0 2024-09-10T04:05:20.827 INFO:tasks.workunit.client.0.smithi046.stdout:2/536: dwrite d2/d6/d23/d5d/d77/d7b/f92 [0,4194304] 0 2024-09-10T04:05:20.828 INFO:tasks.workunit.client.0.smithi046.stdout:2/537: fdatasync d2/d6/f18 0 2024-09-10T04:05:20.828 INFO:tasks.workunit.client.1.smithi070.stdout:3/566: truncate db/f33 5620151 0 2024-09-10T04:05:20.832 INFO:tasks.workunit.client.1.smithi070.stdout:8/607: getdents d4/d16/d17/d1a/d60/d7a/db8 0 2024-09-10T04:05:20.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:20 smithi046 ceph-mon[28328]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:20.837 INFO:tasks.workunit.client.1.smithi070.stdout:4/482: creat de/d13/d18/d23/d47/d59/d79/fa5 x:0 0 0 2024-09-10T04:05:20.839 INFO:tasks.workunit.client.1.smithi070.stdout:7/533: fdatasync d2/f24 0 2024-09-10T04:05:20.839 INFO:tasks.workunit.client.1.smithi070.stdout:8/608: dwrite d4/d16/d17/d19/f72 [0,4194304] 0 2024-09-10T04:05:20.839 INFO:tasks.workunit.client.1.smithi070.stdout:1/456: truncate d3/d10/d1d/f46 4643223 0 2024-09-10T04:05:20.840 INFO:tasks.workunit.client.1.smithi070.stdout:1/457: fdatasync d3/d10/f73 0 2024-09-10T04:05:20.843 INFO:tasks.workunit.client.1.smithi070.stdout:9/594: rename d9/d1c/dac/d8a/d3c/c57 to d9/d7a/d6f/d90/d89/dbb/cbe 0 2024-09-10T04:05:20.847 INFO:tasks.workunit.client.1.smithi070.stdout:1/458: stat d3/d8/d12/d2f/f34 0 2024-09-10T04:05:20.847 INFO:tasks.workunit.client.0.smithi046.stdout:2/538: creat d2/d6/d23/d5d/d77/d7b/dc4/fc6 x:0 0 0 2024-09-10T04:05:20.848 INFO:tasks.workunit.client.1.smithi070.stdout:4/483: mkdir de/d13/da6 0 2024-09-10T04:05:20.850 INFO:tasks.workunit.client.0.smithi046.stdout:0/408: sync 2024-09-10T04:05:20.850 INFO:tasks.workunit.client.1.smithi070.stdout:7/534: dread d2/d7/d9/d18/f6b [4194304,4194304] 0 2024-09-10T04:05:20.850 INFO:tasks.workunit.client.1.smithi070.stdout:0/628: dwrite d8/f1b [8388608,4194304] 0 2024-09-10T04:05:20.851 INFO:tasks.workunit.client.0.smithi046.stdout:4/690: getdents d3/d12/d17/d22/d51/d5c/d6c 0 2024-09-10T04:05:20.852 INFO:tasks.workunit.client.1.smithi070.stdout:4/484: write de/d13/d18/d26/d46/f71 [317611,51130] 0 2024-09-10T04:05:20.852 INFO:tasks.workunit.client.1.smithi070.stdout:7/535: truncate d2/f55 739569 0 2024-09-10T04:05:20.854 INFO:tasks.workunit.client.1.smithi070.stdout:5/554: getdents d17/d19/d6a/d54/d57 0 2024-09-10T04:05:20.856 INFO:tasks.workunit.client.1.smithi070.stdout:5/555: stat d17/d19/f44 0 2024-09-10T04:05:20.857 INFO:tasks.workunit.client.1.smithi070.stdout:0/629: symlink d8/d13/d47/d7b/lcf 0 2024-09-10T04:05:20.859 INFO:tasks.workunit.client.1.smithi070.stdout:8/609: dread d4/d16/d17/d1a/d60/d7a/f7b [0,4194304] 0 2024-09-10T04:05:20.860 INFO:tasks.workunit.client.1.smithi070.stdout:8/610: write d4/d39/fbb [4657780,39688] 0 2024-09-10T04:05:20.862 INFO:tasks.workunit.client.1.smithi070.stdout:1/459: mkdir d3/d8/d92 0 2024-09-10T04:05:20.863 INFO:tasks.workunit.client.1.smithi070.stdout:7/536: creat d2/d3/dc/d5d/d42/fc4 x:0 0 0 2024-09-10T04:05:20.864 INFO:tasks.workunit.client.1.smithi070.stdout:9/595: dread d9/d1c/f2c [0,4194304] 0 2024-09-10T04:05:20.865 INFO:tasks.workunit.client.1.smithi070.stdout:3/567: write db/d1c/d25/d28/f73 [3436058,488] 0 2024-09-10T04:05:20.868 INFO:tasks.workunit.client.1.smithi070.stdout:5/556: rename d17/d24/f2c to d17/d19/d6a/d54/d57/d63/db9/fbe 0 2024-09-10T04:05:20.869 INFO:tasks.workunit.client.1.smithi070.stdout:1/460: creat d3/d8/d27/f93 x:0 0 0 2024-09-10T04:05:20.870 INFO:tasks.workunit.client.1.smithi070.stdout:5/557: write d17/d37/faa [479015,104223] 0 2024-09-10T04:05:20.871 INFO:tasks.workunit.client.1.smithi070.stdout:1/461: dread - d3/d8/d12/d2f/d32/f6b zero size 2024-09-10T04:05:20.872 INFO:tasks.workunit.client.1.smithi070.stdout:1/462: stat d3/d8/d13/d67 0 2024-09-10T04:05:20.873 INFO:tasks.workunit.client.1.smithi070.stdout:9/596: unlink d9/d1c/dac/d8a/d1b/l29 0 2024-09-10T04:05:20.873 INFO:tasks.workunit.client.1.smithi070.stdout:9/597: fdatasync d9/f73 0 2024-09-10T04:05:20.873 INFO:tasks.workunit.client.1.smithi070.stdout:4/485: creat de/d13/d18/d23/d47/fa7 x:0 0 0 2024-09-10T04:05:20.877 INFO:tasks.workunit.client.1.smithi070.stdout:3/568: dread db/d1c/f4b [0,4194304] 0 2024-09-10T04:05:20.879 INFO:tasks.workunit.client.1.smithi070.stdout:9/598: dwrite d9/d1c/dac/d8a/d3c/fa2 [0,4194304] 0 2024-09-10T04:05:20.879 INFO:tasks.workunit.client.1.smithi070.stdout:1/463: fsync d3/d10/f2c 0 2024-09-10T04:05:20.882 INFO:tasks.workunit.client.0.smithi046.stdout:0/409: dwrite d1/d13/f1d [0,4194304] 0 2024-09-10T04:05:20.884 INFO:tasks.workunit.client.0.smithi046.stdout:4/691: dwrite d3/d12/d17/d22/d51/d5c/f76 [0,4194304] 0 2024-09-10T04:05:20.890 INFO:tasks.workunit.client.0.smithi046.stdout:0/410: symlink d1/dc/d25/d33/d4/l82 0 2024-09-10T04:05:20.892 INFO:tasks.workunit.client.1.smithi070.stdout:7/537: dwrite d2/d40/d6c/f7f [4194304,4194304] 0 2024-09-10T04:05:20.892 INFO:tasks.workunit.client.1.smithi070.stdout:5/558: mkdir d17/d24/dbf 0 2024-09-10T04:05:20.893 INFO:tasks.workunit.client.0.smithi046.stdout:4/692: rmdir d3/d12/d17/d91 39 2024-09-10T04:05:20.894 INFO:tasks.workunit.client.1.smithi070.stdout:7/538: stat d2/d3/dc/d5d/d42/c9c 0 2024-09-10T04:05:20.895 INFO:tasks.workunit.client.1.smithi070.stdout:7/539: chown d2/d7/d9/d18/d23 1 1 2024-09-10T04:05:20.896 INFO:tasks.workunit.client.1.smithi070.stdout:0/630: getdents d8/d13/d47 0 2024-09-10T04:05:20.900 INFO:tasks.workunit.client.1.smithi070.stdout:3/569: mkdir db/d83/dbf 0 2024-09-10T04:05:20.901 INFO:tasks.workunit.client.1.smithi070.stdout:1/464: mkdir d3/d8/d27/d7a/d94 0 2024-09-10T04:05:20.901 INFO:tasks.workunit.client.1.smithi070.stdout:9/599: creat d9/d1c/dac/d8a/fbf x:0 0 0 2024-09-10T04:05:20.902 INFO:tasks.workunit.client.1.smithi070.stdout:8/611: rename d4/dbf/d87/d8e to d4/dcf 0 2024-09-10T04:05:20.904 INFO:tasks.workunit.client.1.smithi070.stdout:1/465: fdatasync d3/d10/d1d/f6c 0 2024-09-10T04:05:20.906 INFO:tasks.workunit.client.1.smithi070.stdout:7/540: chown d2/d7/d9/d18/d23/l88 0 1 2024-09-10T04:05:20.906 INFO:tasks.workunit.client.1.smithi070.stdout:5/559: fdatasync f7 0 2024-09-10T04:05:20.907 INFO:tasks.workunit.client.1.smithi070.stdout:4/486: dwrite de/d13/d18/d23/f3c [0,4194304] 0 2024-09-10T04:05:20.908 INFO:tasks.workunit.client.1.smithi070.stdout:6/625: sync 2024-09-10T04:05:20.909 INFO:tasks.workunit.client.1.smithi070.stdout:4/487: dread - de/d13/d18/d23/d47/d59/f87 zero size 2024-09-10T04:05:20.911 INFO:tasks.workunit.client.1.smithi070.stdout:3/570: unlink db/d1c/d89/fb3 0 2024-09-10T04:05:20.911 INFO:tasks.workunit.client.0.smithi046.stdout:4/693: dread d3/d12/d17/d22/d30/faa [0,4194304] 0 2024-09-10T04:05:20.913 INFO:tasks.workunit.client.1.smithi070.stdout:3/571: fsync db/d83/fac 0 2024-09-10T04:05:20.913 INFO:tasks.workunit.client.1.smithi070.stdout:1/466: dread d3/d8/d13/d3f/f52 [0,4194304] 0 2024-09-10T04:05:20.914 INFO:tasks.workunit.client.1.smithi070.stdout:5/560: dread d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:20.915 INFO:tasks.workunit.client.1.smithi070.stdout:3/572: chown db/d1c/d25/d28/l68 0 1 2024-09-10T04:05:20.915 INFO:tasks.workunit.client.1.smithi070.stdout:3/573: readlink db/d1c/l46 0 2024-09-10T04:05:20.916 INFO:tasks.workunit.client.1.smithi070.stdout:8/612: creat d4/d16/d73/fd0 x:0 0 0 2024-09-10T04:05:20.917 INFO:tasks.workunit.client.1.smithi070.stdout:8/613: chown d4/d16/d17/d19/d28/f30 32687748 1 2024-09-10T04:05:20.929 INFO:tasks.workunit.client.0.smithi046.stdout:4/694: dread d3/d12/d17/d22/d51/d5c/fc8 [0,4194304] 0 2024-09-10T04:05:20.931 INFO:tasks.workunit.client.1.smithi070.stdout:4/488: symlink de/d13/d18/d23/d39/d48/la8 0 2024-09-10T04:05:20.933 INFO:tasks.workunit.client.1.smithi070.stdout:3/574: rmdir db/d1c 39 2024-09-10T04:05:20.934 INFO:tasks.workunit.client.1.smithi070.stdout:5/561: rmdir d17/d19/d6a/d54/d57/d63/d7b 39 2024-09-10T04:05:20.934 INFO:tasks.workunit.client.1.smithi070.stdout:5/562: chown d17/d19/d6a/d54/ca2 9879 1 2024-09-10T04:05:20.935 INFO:tasks.workunit.client.1.smithi070.stdout:8/614: mkdir d4/d16/d17/d19/d62/dd1 0 2024-09-10T04:05:20.938 INFO:tasks.workunit.client.1.smithi070.stdout:9/600: creat d9/d59/d2a/d3f/daf/fc0 x:0 0 0 2024-09-10T04:05:20.939 INFO:tasks.workunit.client.1.smithi070.stdout:7/541: mkdir d2/dc5 0 2024-09-10T04:05:20.940 INFO:tasks.workunit.client.1.smithi070.stdout:5/563: dwrite d17/d19/d35/f6c [0,4194304] 0 2024-09-10T04:05:20.940 INFO:tasks.workunit.client.1.smithi070.stdout:7/542: readlink d2/d7/d9/l16 0 2024-09-10T04:05:20.940 INFO:tasks.workunit.client.1.smithi070.stdout:5/564: chown d17 0 1 2024-09-10T04:05:20.941 INFO:tasks.workunit.client.1.smithi070.stdout:5/565: readlink d17/d24/d77/l89 0 2024-09-10T04:05:20.943 INFO:tasks.workunit.client.1.smithi070.stdout:0/631: rename d8/d24/d2d/d34/d9b/dad/fc3 to d8/d13/d33/fd0 0 2024-09-10T04:05:20.944 INFO:tasks.workunit.client.0.smithi046.stdout:4/695: link d3/d12/d14/f2b d3/d12/d17/d29/ddd/ff2 0 2024-09-10T04:05:20.945 INFO:tasks.workunit.client.1.smithi070.stdout:5/566: read d17/d19/d6a/d54/d57/f80 [3966424,80711] 0 2024-09-10T04:05:20.946 INFO:tasks.workunit.client.0.smithi046.stdout:4/696: chown d3/d12/d17/d22/d51/d5c/fb6 34022849 1 2024-09-10T04:05:20.946 INFO:tasks.workunit.client.1.smithi070.stdout:4/489: write de/d13/d5b/f80 [781140,115503] 0 2024-09-10T04:05:20.947 INFO:tasks.workunit.client.1.smithi070.stdout:9/601: mknod d9/d1c/dac/d8a/d3c/d92/cc1 0 2024-09-10T04:05:20.947 INFO:tasks.workunit.client.1.smithi070.stdout:6/626: link d0/l8b d0/d3/d18/d62/db8/lc2 0 2024-09-10T04:05:20.948 INFO:tasks.workunit.client.1.smithi070.stdout:3/575: chown db/d1c/d25/f55 0 1 2024-09-10T04:05:20.949 INFO:tasks.workunit.client.0.smithi046.stdout:4/697: readlink d3/d12/d17/d29/l74 0 2024-09-10T04:05:20.950 INFO:tasks.workunit.client.1.smithi070.stdout:0/632: rmdir d8/d24 39 2024-09-10T04:05:20.951 INFO:tasks.workunit.client.1.smithi070.stdout:8/615: mknod d4/d16/d17/d19/d28/d43/d67/cd2 0 2024-09-10T04:05:20.952 INFO:tasks.workunit.client.0.smithi046.stdout:4/698: mkdir d3/d12/d17/d22/d30/dc7/df3 0 2024-09-10T04:05:20.953 INFO:tasks.workunit.client.1.smithi070.stdout:1/467: dwrite d3/d8/f49 [0,4194304] 0 2024-09-10T04:05:20.953 INFO:tasks.workunit.client.0.smithi046.stdout:4/699: write d3/f9 [332452,114032] 0 2024-09-10T04:05:20.954 INFO:tasks.workunit.client.1.smithi070.stdout:7/543: dwrite d2/d7/d9/d18/d3c/fae [0,4194304] 0 2024-09-10T04:05:20.957 INFO:tasks.workunit.client.1.smithi070.stdout:2/531: sync 2024-09-10T04:05:20.957 INFO:tasks.workunit.client.1.smithi070.stdout:7/544: read - d2/d7/db/f5c zero size 2024-09-10T04:05:20.958 INFO:tasks.workunit.client.1.smithi070.stdout:3/576: symlink db/d1c/d89/lc0 0 2024-09-10T04:05:20.959 INFO:tasks.workunit.client.1.smithi070.stdout:9/602: symlink d9/d1c/dac/d8a/d1b/lc2 0 2024-09-10T04:05:20.959 INFO:tasks.workunit.client.1.smithi070.stdout:4/490: truncate de/d13/d43/f65 4044396 0 2024-09-10T04:05:20.962 INFO:tasks.workunit.client.1.smithi070.stdout:1/468: rename d3/d10/f3b to d3/d8/d27/f95 0 2024-09-10T04:05:20.963 INFO:tasks.workunit.client.1.smithi070.stdout:6/627: symlink d0/d3/d18/lc3 0 2024-09-10T04:05:20.964 INFO:tasks.workunit.client.1.smithi070.stdout:1/469: chown d3/d8/d13/d3f/l75 13954 1 2024-09-10T04:05:20.968 INFO:tasks.workunit.client.1.smithi070.stdout:2/532: creat d15/d18/d56/d7c/d55/d6d/fba x:0 0 0 2024-09-10T04:05:20.972 INFO:tasks.workunit.client.1.smithi070.stdout:0/633: dread d8/d13/d1a/f32 [0,4194304] 0 2024-09-10T04:05:20.972 INFO:tasks.workunit.client.1.smithi070.stdout:6/628: mkdir d0/d3/d18/d1b/d17/d54/dc4 0 2024-09-10T04:05:20.973 INFO:tasks.workunit.client.1.smithi070.stdout:6/629: chown d0/d3/d1f/d93/cb3 338646551 1 2024-09-10T04:05:20.973 INFO:tasks.workunit.client.1.smithi070.stdout:4/491: symlink de/d13/d5b/la9 0 2024-09-10T04:05:20.975 INFO:tasks.workunit.client.1.smithi070.stdout:4/492: chown de/d13/d18/d23/d47/d59/d79/c50 4 1 2024-09-10T04:05:20.975 INFO:tasks.workunit.client.1.smithi070.stdout:2/533: mknod d15/d18/d56/d7c/d55/d6d/cbb 0 2024-09-10T04:05:20.976 INFO:tasks.workunit.client.1.smithi070.stdout:4/493: read de/d13/d18/d26/f2f [184323,81321] 0 2024-09-10T04:05:20.979 INFO:tasks.workunit.client.1.smithi070.stdout:5/567: write d17/d19/d6a/f7c [5546256,14932] 0 2024-09-10T04:05:20.981 INFO:tasks.workunit.client.1.smithi070.stdout:2/534: unlink d15/d18/d29/f46 0 2024-09-10T04:05:20.981 INFO:tasks.workunit.client.1.smithi070.stdout:7/545: link d2/d7/d9/d18/d49/c7b d2/d7/d9/d18/d23/cc6 0 2024-09-10T04:05:20.982 INFO:tasks.workunit.client.1.smithi070.stdout:5/568: dread d17/d19/d6a/d54/d57/fb0 [0,4194304] 0 2024-09-10T04:05:20.982 INFO:tasks.workunit.client.1.smithi070.stdout:2/535: read f3 [3072870,70160] 0 2024-09-10T04:05:20.983 INFO:tasks.workunit.client.1.smithi070.stdout:4/494: creat de/d13/d18/d23/d39/d77/faa x:0 0 0 2024-09-10T04:05:20.984 INFO:tasks.workunit.client.1.smithi070.stdout:6/630: creat d0/d3/d18/d62/db8/fc5 x:0 0 0 2024-09-10T04:05:20.985 INFO:tasks.workunit.client.1.smithi070.stdout:0/634: fsync d8/d24/d2d/d34/f69 0 2024-09-10T04:05:20.988 INFO:tasks.workunit.client.1.smithi070.stdout:8/616: rename d4/d16/d73/d96/f59 to d4/d16/d17/fd3 0 2024-09-10T04:05:20.989 INFO:tasks.workunit.client.1.smithi070.stdout:9/603: write d9/d1c/dac/d8a/d3c/f40 [309424,76187] 0 2024-09-10T04:05:20.993 INFO:tasks.workunit.client.1.smithi070.stdout:1/470: write f0 [1009378,34869] 0 2024-09-10T04:05:20.993 INFO:tasks.workunit.client.1.smithi070.stdout:4/495: stat de/lf 0 2024-09-10T04:05:20.994 INFO:tasks.workunit.client.1.smithi070.stdout:2/536: rmdir d15/d18/d29/d2a/d2e/d32/d36 39 2024-09-10T04:05:20.997 INFO:tasks.workunit.client.1.smithi070.stdout:7/546: dread d2/d7/fe [0,4194304] 0 2024-09-10T04:05:20.997 INFO:tasks.workunit.client.1.smithi070.stdout:8/617: write d4/f99 [71288,13363] 0 2024-09-10T04:05:20.998 INFO:tasks.workunit.client.1.smithi070.stdout:3/577: rename fa to db/d79/db6/fc1 0 2024-09-10T04:05:21.000 INFO:tasks.workunit.client.1.smithi070.stdout:7/547: dread d2/d3/dc/d5d/d42/fa0 [0,4194304] 0 2024-09-10T04:05:21.001 INFO:tasks.workunit.client.1.smithi070.stdout:1/471: dwrite d3/d8/d13/d3f/f42 [0,4194304] 0 2024-09-10T04:05:21.006 INFO:tasks.workunit.client.1.smithi070.stdout:2/537: read f11 [345337,56086] 0 2024-09-10T04:05:21.007 INFO:tasks.workunit.client.1.smithi070.stdout:2/538: write d15/d2b/d94/f9a [2555046,50216] 0 2024-09-10T04:05:21.009 INFO:tasks.workunit.client.1.smithi070.stdout:7/548: read d2/d7/d9/d18/f6b [1893510,14625] 0 2024-09-10T04:05:21.010 INFO:tasks.workunit.client.1.smithi070.stdout:7/549: chown d2/d7/d9/l16 42 1 2024-09-10T04:05:21.012 INFO:tasks.workunit.client.1.smithi070.stdout:7/550: fdatasync d2/d40/d6c/db2/fc2 0 2024-09-10T04:05:21.014 INFO:tasks.workunit.client.1.smithi070.stdout:2/539: dwrite d15/d18/d29/d2a/d2e/d32/fb5 [0,4194304] 0 2024-09-10T04:05:21.016 INFO:tasks.workunit.client.1.smithi070.stdout:5/569: truncate d17/d37/f3d 677315 0 2024-09-10T04:05:21.017 INFO:tasks.workunit.client.1.smithi070.stdout:4/496: write de/d13/d18/d26/f2b [972497,15296] 0 2024-09-10T04:05:21.019 INFO:tasks.workunit.client.1.smithi070.stdout:8/618: write d4/d16/d17/d1a/f2e [5172749,28831] 0 2024-09-10T04:05:21.020 INFO:tasks.workunit.client.1.smithi070.stdout:3/578: write db/d1c/d89/f8f [3096299,43333] 0 2024-09-10T04:05:21.022 INFO:tasks.workunit.client.1.smithi070.stdout:9/604: dwrite d9/d1c/dac/d8a/d1b/f45 [0,4194304] 0 2024-09-10T04:05:21.022 INFO:tasks.workunit.client.1.smithi070.stdout:7/551: rmdir d2/d3/d63/da9/dc1 39 2024-09-10T04:05:21.022 INFO:tasks.workunit.client.1.smithi070.stdout:3/579: stat db/c60 0 2024-09-10T04:05:21.027 INFO:tasks.workunit.client.1.smithi070.stdout:1/472: symlink d3/d8/d12/d2f/d32/d7b/l96 0 2024-09-10T04:05:21.028 INFO:tasks.workunit.client.1.smithi070.stdout:6/631: dwrite d0/d3/d18/d1b/f7d [0,4194304] 0 2024-09-10T04:05:21.028 INFO:tasks.workunit.client.1.smithi070.stdout:9/605: chown d9/d1c/dac/d8a/f20 2 1 2024-09-10T04:05:21.030 INFO:tasks.workunit.client.1.smithi070.stdout:7/552: rmdir d2/d3/d63 39 2024-09-10T04:05:21.030 INFO:tasks.workunit.client.1.smithi070.stdout:5/570: write d17/d24/d84/d9b/fb5 [1040828,65970] 0 2024-09-10T04:05:21.032 INFO:tasks.workunit.client.1.smithi070.stdout:2/540: mknod d15/d18/d29/cbc 0 2024-09-10T04:05:21.033 INFO:tasks.workunit.client.0.smithi046.stdout:6/519: sync 2024-09-10T04:05:21.038 INFO:tasks.workunit.client.1.smithi070.stdout:2/541: fdatasync d15/d18/d56/fa3 0 2024-09-10T04:05:21.039 INFO:tasks.workunit.client.0.smithi046.stdout:6/520: rename da/db/d11/d2b/d2f/l91 to da/d71/la9 0 2024-09-10T04:05:21.040 INFO:tasks.workunit.client.0.smithi046.stdout:6/521: chown da/db/d63/fa1 1570 1 2024-09-10T04:05:21.041 INFO:tasks.workunit.client.1.smithi070.stdout:8/619: dwrite d4/d16/d73/fd0 [0,4194304] 0 2024-09-10T04:05:21.044 INFO:tasks.workunit.client.1.smithi070.stdout:7/553: dread d2/f55 [0,4194304] 0 2024-09-10T04:05:21.044 INFO:tasks.workunit.client.0.smithi046.stdout:6/522: rmdir da/db/d11/d2b/d2f 39 2024-09-10T04:05:21.046 INFO:tasks.workunit.client.1.smithi070.stdout:7/554: stat d2/fbe 0 2024-09-10T04:05:21.047 INFO:tasks.workunit.client.1.smithi070.stdout:1/473: write d3/d8/d13/d3f/f53 [857468,22702] 0 2024-09-10T04:05:21.048 INFO:tasks.workunit.client.1.smithi070.stdout:9/606: rename d9/d1c/dac/d8a/d3c/d62/f7d to d9/d59/d5d/fc3 0 2024-09-10T04:05:21.050 INFO:tasks.workunit.client.1.smithi070.stdout:4/497: truncate de/d13/d18/d23/d47/f51 129452 0 2024-09-10T04:05:21.051 INFO:tasks.workunit.client.0.smithi046.stdout:6/523: creat da/db/d11/d13/d1e/d23/d54/d90/faa x:0 0 0 2024-09-10T04:05:21.052 INFO:tasks.workunit.client.1.smithi070.stdout:9/607: write d9/d1c/dac/d8a/d1b/f5c [3369489,92227] 0 2024-09-10T04:05:21.054 INFO:tasks.workunit.client.1.smithi070.stdout:2/542: dwrite d15/d18/d56/d7c/d55/d6d/fba [0,4194304] 0 2024-09-10T04:05:21.055 INFO:tasks.workunit.client.1.smithi070.stdout:3/580: link db/d1c/d53/l56 db/d1c/d3e/lc2 0 2024-09-10T04:05:21.055 INFO:tasks.workunit.client.1.smithi070.stdout:4/498: truncate de/d13/d18/d26/d46/fa1 450114 0 2024-09-10T04:05:21.056 INFO:tasks.workunit.client.1.smithi070.stdout:5/571: dread d17/d19/d6a/d54/d57/f65 [0,4194304] 0 2024-09-10T04:05:21.057 INFO:tasks.workunit.client.1.smithi070.stdout:6/632: dread d0/d3/d18/d62/f68 [0,4194304] 0 2024-09-10T04:05:21.058 INFO:tasks.workunit.client.0.smithi046.stdout:6/524: mknod da/dc/da5/cab 0 2024-09-10T04:05:21.060 INFO:tasks.workunit.client.1.smithi070.stdout:6/633: dread d0/f57 [0,4194304] 0 2024-09-10T04:05:21.061 INFO:tasks.workunit.client.1.smithi070.stdout:5/572: dread d17/d19/d6a/d54/d57/da3/f97 [0,4194304] 0 2024-09-10T04:05:21.063 INFO:tasks.workunit.client.1.smithi070.stdout:7/555: creat d2/d3/dc/d13/fc7 x:0 0 0 2024-09-10T04:05:21.066 INFO:tasks.workunit.client.0.smithi046.stdout:6/525: chown da/db/d11/d2b/d2f/f74 4 1 2024-09-10T04:05:21.067 INFO:tasks.workunit.client.1.smithi070.stdout:1/474: rmdir d3/d8/d12/d61 39 2024-09-10T04:05:21.068 INFO:tasks.workunit.client.1.smithi070.stdout:1/475: fdatasync d3/d8/d13/d3f/f78 0 2024-09-10T04:05:21.069 INFO:tasks.workunit.client.1.smithi070.stdout:1/476: dread - d3/d8/d12/d2f/d32/f58 zero size 2024-09-10T04:05:21.072 INFO:tasks.workunit.client.1.smithi070.stdout:9/608: mkdir d9/d1c/dac/d8a/dc4 0 2024-09-10T04:05:21.073 INFO:tasks.workunit.client.1.smithi070.stdout:8/620: link d4/f12 d4/d16/d17/d19/d62/fd4 0 2024-09-10T04:05:21.074 INFO:tasks.workunit.client.1.smithi070.stdout:3/581: mkdir db/d79/db6/dc3 0 2024-09-10T04:05:21.078 INFO:tasks.workunit.client.1.smithi070.stdout:1/477: mknod d3/d8/d13/d67/c97 0 2024-09-10T04:05:21.078 INFO:tasks.workunit.client.1.smithi070.stdout:2/543: fsync d15/d18/f1c 0 2024-09-10T04:05:21.080 INFO:tasks.workunit.client.1.smithi070.stdout:8/621: dwrite d4/f77 [0,4194304] 0 2024-09-10T04:05:21.080 INFO:tasks.workunit.client.1.smithi070.stdout:9/609: creat d9/d59/d2a/d3f/daf/fc5 x:0 0 0 2024-09-10T04:05:21.081 INFO:tasks.workunit.client.0.smithi046.stdout:6/526: dread da/f1f [0,4194304] 0 2024-09-10T04:05:21.082 INFO:tasks.workunit.client.1.smithi070.stdout:9/610: dread - d9/d59/fa4 zero size 2024-09-10T04:05:21.084 INFO:tasks.workunit.client.1.smithi070.stdout:3/582: dwrite db/d1c/d25/d28/d43/f77 [0,4194304] 0 2024-09-10T04:05:21.086 INFO:tasks.workunit.client.1.smithi070.stdout:6/634: link d0/d3/d18/d1b/d33/f39 d0/d3/d18/d1b/d17/d54/fc6 0 2024-09-10T04:05:21.087 INFO:tasks.workunit.client.1.smithi070.stdout:2/544: dwrite d15/d18/d29/d2a/d2e/d32/fb5 [0,4194304] 0 2024-09-10T04:05:21.088 INFO:tasks.workunit.client.1.smithi070.stdout:6/635: write d0/d3/d18/d62/db8/fc5 [892573,14225] 0 2024-09-10T04:05:21.088 INFO:tasks.workunit.client.1.smithi070.stdout:2/545: dread - d15/d18/d56/f86 zero size 2024-09-10T04:05:21.096 INFO:tasks.workunit.client.1.smithi070.stdout:7/556: mkdir d2/dc8 0 2024-09-10T04:05:21.098 INFO:tasks.workunit.client.1.smithi070.stdout:5/573: write d17/d19/d6a/d54/d57/da3/f55 [3674583,50001] 0 2024-09-10T04:05:21.099 INFO:tasks.workunit.client.1.smithi070.stdout:5/574: chown d17/d24/d84/d9b/fb5 3 1 2024-09-10T04:05:21.102 INFO:tasks.workunit.client.1.smithi070.stdout:5/575: dread d17/d24/d84/d9b/fb5 [0,4194304] 0 2024-09-10T04:05:21.105 INFO:tasks.workunit.client.0.smithi046.stdout:6/527: dread da/db/d11/d2b/f2c [0,4194304] 0 2024-09-10T04:05:21.106 INFO:tasks.workunit.client.0.smithi046.stdout:6/528: write da/db/d11/d2b/d2f/f66 [1860483,101419] 0 2024-09-10T04:05:21.135 INFO:tasks.workunit.client.1.smithi070.stdout:0/635: sync 2024-09-10T04:05:21.137 INFO:tasks.workunit.client.0.smithi046.stdout:6/529: fdatasync da/db/d11/d13/d1e/d23/f2e 0 2024-09-10T04:05:21.139 INFO:tasks.workunit.client.1.smithi070.stdout:3/583: fsync db/d79/fa3 0 2024-09-10T04:05:21.141 INFO:tasks.workunit.client.0.smithi046.stdout:6/530: unlink da/db/d10/d51/l6b 0 2024-09-10T04:05:21.144 INFO:tasks.workunit.client.0.smithi046.stdout:6/531: dread da/f7b [0,4194304] 0 2024-09-10T04:05:21.145 INFO:tasks.workunit.client.0.smithi046.stdout:6/532: readlink da/db/d11/d13/d1e/d23/l8d 0 2024-09-10T04:05:21.146 INFO:tasks.workunit.client.1.smithi070.stdout:6/636: unlink d0/d3/d18/d1b/d17/d3f/d40/l80 0 2024-09-10T04:05:21.146 INFO:tasks.workunit.client.1.smithi070.stdout:7/557: rmdir d2/d7/db/d3a/d7e 39 2024-09-10T04:05:21.147 INFO:tasks.workunit.client.1.smithi070.stdout:2/546: chown d15/d18/d29/d2a/d2e/cac 17303875 1 2024-09-10T04:05:21.147 INFO:tasks.workunit.client.1.smithi070.stdout:9/611: dwrite d9/d1c/dac/d8a/f3d [4194304,4194304] 0 2024-09-10T04:05:21.148 INFO:tasks.workunit.client.1.smithi070.stdout:9/612: chown d9/d7a/d6f/d90/lae 11444 1 2024-09-10T04:05:21.149 INFO:tasks.workunit.client.0.smithi046.stdout:6/533: creat da/db/d10/fac x:0 0 0 2024-09-10T04:05:21.153 INFO:tasks.workunit.client.1.smithi070.stdout:0/636: write d8/d13/d33/d4f/fbb [1417523,100767] 0 2024-09-10T04:05:21.156 INFO:tasks.workunit.client.1.smithi070.stdout:4/499: truncate de/d13/d18/d23/d47/f51 446262 0 2024-09-10T04:05:21.158 INFO:tasks.workunit.client.1.smithi070.stdout:3/584: creat db/d79/fc4 x:0 0 0 2024-09-10T04:05:21.159 INFO:tasks.workunit.client.0.smithi046.stdout:6/534: dwrite da/d45/f56 [0,4194304] 0 2024-09-10T04:05:21.179 INFO:tasks.workunit.client.0.smithi046.stdout:6/535: dread da/db/d18/f35 [4194304,4194304] 0 2024-09-10T04:05:21.180 INFO:tasks.workunit.client.0.smithi046.stdout:6/536: fsync da/db/d10/d27/f88 0 2024-09-10T04:05:21.201 INFO:tasks.workunit.client.1.smithi070.stdout:3/585: readlink db/d1c/d3e/l84 0 2024-09-10T04:05:21.202 INFO:tasks.workunit.client.1.smithi070.stdout:2/547: unlink d15/d18/d29/cb0 0 2024-09-10T04:05:21.210 INFO:tasks.workunit.client.1.smithi070.stdout:7/558: write d2/d3/d63/f87 [3541814,119220] 0 2024-09-10T04:05:21.210 INFO:tasks.workunit.client.1.smithi070.stdout:3/586: mknod db/d1c/cc5 0 2024-09-10T04:05:21.213 INFO:tasks.workunit.client.1.smithi070.stdout:1/478: dwrite d3/d8/d27/f95 [0,4194304] 0 2024-09-10T04:05:21.213 INFO:tasks.workunit.client.1.smithi070.stdout:8/622: dwrite d4/d16/d17/d19/f25 [0,4194304] 0 2024-09-10T04:05:21.214 INFO:tasks.workunit.client.1.smithi070.stdout:5/576: rename d17/d19/d6a/d54/d57/d63/d7b to d17/d24/d45/dc0 0 2024-09-10T04:05:21.215 INFO:tasks.workunit.client.1.smithi070.stdout:0/637: dread d8/d24/d2d/f3a [0,4194304] 0 2024-09-10T04:05:21.217 INFO:tasks.workunit.client.1.smithi070.stdout:2/548: symlink d15/d18/d29/d2a/d2e/d32/d9b/lbd 0 2024-09-10T04:05:21.218 INFO:tasks.workunit.client.1.smithi070.stdout:5/577: readlink d17/d19/d35/la9 0 2024-09-10T04:05:21.222 INFO:tasks.workunit.client.1.smithi070.stdout:3/587: unlink db/d1c/d25/d28/d43/c80 0 2024-09-10T04:05:21.222 INFO:tasks.workunit.client.1.smithi070.stdout:4/500: rename fb to de/d13/d18/d23/d39/d48/fab 0 2024-09-10T04:05:21.223 INFO:tasks.workunit.client.1.smithi070.stdout:2/549: creat d15/d18/d29/d2a/fbe x:0 0 0 2024-09-10T04:05:21.224 INFO:tasks.workunit.client.1.smithi070.stdout:1/479: symlink d3/d10/d1d/l98 0 2024-09-10T04:05:21.225 INFO:tasks.workunit.client.1.smithi070.stdout:9/613: dwrite d9/d7a/d6f/d90/d89/fa3 [0,4194304] 0 2024-09-10T04:05:21.227 INFO:tasks.workunit.client.1.smithi070.stdout:3/588: write db/f3f [9587029,103754] 0 2024-09-10T04:05:21.227 INFO:tasks.workunit.client.1.smithi070.stdout:4/501: fdatasync de/d13/d18/d23/d39/f8e 0 2024-09-10T04:05:21.227 INFO:tasks.workunit.client.1.smithi070.stdout:8/623: unlink d4/d16/d17/d19/d28/d43/d67/d97/f98 0 2024-09-10T04:05:21.229 INFO:tasks.workunit.client.1.smithi070.stdout:1/480: write d3/d10/f73 [342424,79621] 0 2024-09-10T04:05:21.235 INFO:tasks.workunit.client.1.smithi070.stdout:0/638: creat d8/d24/d2d/fd1 x:0 0 0 2024-09-10T04:05:21.236 INFO:tasks.workunit.client.1.smithi070.stdout:9/614: creat d9/d7a/d74/fc6 x:0 0 0 2024-09-10T04:05:21.236 INFO:tasks.workunit.client.1.smithi070.stdout:4/502: creat de/d13/d5b/d7b/fac x:0 0 0 2024-09-10T04:05:21.238 INFO:tasks.workunit.client.1.smithi070.stdout:9/615: write d9/d59/fa4 [40381,36057] 0 2024-09-10T04:05:21.239 INFO:tasks.workunit.client.1.smithi070.stdout:0/639: write d8/d13/f1f [4175919,46209] 0 2024-09-10T04:05:21.239 INFO:tasks.workunit.client.1.smithi070.stdout:7/559: dwrite d2/d7/d9/f82 [4194304,4194304] 0 2024-09-10T04:05:21.240 INFO:tasks.workunit.client.1.smithi070.stdout:1/481: symlink d3/d90/l99 0 2024-09-10T04:05:21.244 INFO:tasks.workunit.client.1.smithi070.stdout:7/560: fdatasync d2/d7/d9/d18/d3c/f58 0 2024-09-10T04:05:21.244 INFO:tasks.workunit.client.1.smithi070.stdout:3/589: rmdir db/d83/dbf 0 2024-09-10T04:05:21.245 INFO:tasks.workunit.client.1.smithi070.stdout:7/561: chown d2/d3/d63/c8d 346139503 1 2024-09-10T04:05:21.247 INFO:tasks.workunit.client.1.smithi070.stdout:9/616: rmdir d9/d7a/d74 39 2024-09-10T04:05:21.249 INFO:tasks.workunit.client.1.smithi070.stdout:3/590: creat db/d79/db6/fc6 x:0 0 0 2024-09-10T04:05:21.250 INFO:tasks.workunit.client.1.smithi070.stdout:9/617: write d9/d59/d2a/d3f/daf/fc5 [30929,76451] 0 2024-09-10T04:05:21.252 INFO:tasks.workunit.client.1.smithi070.stdout:0/640: rename d8/d24/l76 to d8/d13/d1a/d9a/ld2 0 2024-09-10T04:05:21.252 INFO:tasks.workunit.client.1.smithi070.stdout:2/550: write d15/d18/d29/d2a/f58 [5237886,122197] 0 2024-09-10T04:05:21.252 INFO:tasks.workunit.client.1.smithi070.stdout:7/562: dwrite d2/d7/d9/f31 [0,4194304] 0 2024-09-10T04:05:21.253 INFO:tasks.workunit.client.1.smithi070.stdout:8/624: truncate d4/f77 4594997 0 2024-09-10T04:05:21.254 INFO:tasks.workunit.client.1.smithi070.stdout:7/563: chown d2/d40/d6c 127013083 1 2024-09-10T04:05:21.258 INFO:tasks.workunit.client.1.smithi070.stdout:1/482: write d3/d8/d13/f18 [1341745,72143] 0 2024-09-10T04:05:21.259 INFO:tasks.workunit.client.1.smithi070.stdout:1/483: chown d3/d8/d13/d3f 4030994 1 2024-09-10T04:05:21.260 INFO:tasks.workunit.client.1.smithi070.stdout:0/641: creat d8/d24/d2d/d5f/fd3 x:0 0 0 2024-09-10T04:05:21.266 INFO:tasks.workunit.client.1.smithi070.stdout:1/484: rmdir d3/d8/d13/d3f 39 2024-09-10T04:05:21.266 INFO:tasks.workunit.client.1.smithi070.stdout:0/642: symlink d8/d24/d2d/d5f/ld4 0 2024-09-10T04:05:21.268 INFO:tasks.workunit.client.1.smithi070.stdout:4/503: link de/d13/d18/d23/d39/d48/c4e de/cad 0 2024-09-10T04:05:21.268 INFO:tasks.workunit.client.1.smithi070.stdout:2/551: link d15/d18/l68 d15/d59/da1/lbf 0 2024-09-10T04:05:21.269 INFO:tasks.workunit.client.1.smithi070.stdout:7/564: rmdir d2/db6 0 2024-09-10T04:05:21.270 INFO:tasks.workunit.client.1.smithi070.stdout:4/504: read - de/d13/d18/d23/d39/f5a zero size 2024-09-10T04:05:21.270 INFO:tasks.workunit.client.1.smithi070.stdout:3/591: getdents db/d1c/d25/d28/d43 0 2024-09-10T04:05:21.271 INFO:tasks.workunit.client.1.smithi070.stdout:9/618: link d9/d1c/dac/d8a/c6c d9/cc7 0 2024-09-10T04:05:21.272 INFO:tasks.workunit.client.1.smithi070.stdout:7/565: chown d2/d7/la1 476473 1 2024-09-10T04:05:21.273 INFO:tasks.workunit.client.1.smithi070.stdout:8/625: rename d4/d16/l69 to d4/d16/d17/d19/d28/d43/d67/ld5 0 2024-09-10T04:05:21.275 INFO:tasks.workunit.client.1.smithi070.stdout:3/592: fdatasync db/d1c/d53/f6b 0 2024-09-10T04:05:21.277 INFO:tasks.workunit.client.1.smithi070.stdout:8/626: chown d4/d16/d17/d1a/d60 20542429 1 2024-09-10T04:05:21.277 INFO:tasks.workunit.client.1.smithi070.stdout:7/566: fdatasync d2/f24 0 2024-09-10T04:05:21.279 INFO:tasks.workunit.client.1.smithi070.stdout:1/485: rename d3/d8/d12/d2f/d32/d7b to d3/d8/d27/d9a 0 2024-09-10T04:05:21.281 INFO:tasks.workunit.client.1.smithi070.stdout:0/643: dwrite d8/d13/d1a/d8f/d95/fcc [0,4194304] 0 2024-09-10T04:05:21.282 INFO:tasks.workunit.client.1.smithi070.stdout:2/552: dwrite d15/d18/d29/d2a/d2e/d32/f33 [0,4194304] 0 2024-09-10T04:05:21.284 INFO:tasks.workunit.client.1.smithi070.stdout:9/619: truncate d9/d1c/dac/d8a/f81 5203234 0 2024-09-10T04:05:21.286 INFO:tasks.workunit.client.1.smithi070.stdout:3/593: mkdir db/d83/d92/dc7 0 2024-09-10T04:05:21.286 INFO:tasks.workunit.client.1.smithi070.stdout:8/627: truncate d4/d16/d73/d96/fb0 327998 0 2024-09-10T04:05:21.287 INFO:tasks.workunit.client.1.smithi070.stdout:0/644: fsync d8/d13/d33/d4f/f7f 0 2024-09-10T04:05:21.289 INFO:tasks.workunit.client.1.smithi070.stdout:7/567: creat d2/d7/d9/d18/d3c/d52/fc9 x:0 0 0 2024-09-10T04:05:21.290 INFO:tasks.workunit.client.1.smithi070.stdout:9/620: write d9/fb7 [298835,52990] 0 2024-09-10T04:05:21.292 INFO:tasks.workunit.client.1.smithi070.stdout:7/568: fdatasync d2/d7/d9/d18/d3c/d52/fc9 0 2024-09-10T04:05:21.294 INFO:tasks.workunit.client.1.smithi070.stdout:0/645: dread d8/f92 [0,4194304] 0 2024-09-10T04:05:21.295 INFO:tasks.workunit.client.1.smithi070.stdout:1/486: symlink d3/d8/d13/d3f/d6d/d81/l9b 0 2024-09-10T04:05:21.296 INFO:tasks.workunit.client.1.smithi070.stdout:3/594: write db/d1c/d25/d28/d43/f51 [4222928,87246] 0 2024-09-10T04:05:21.300 INFO:tasks.workunit.client.1.smithi070.stdout:8/628: rename d4/d16/d73/l8c to d4/d16/d17/d1a/d60/d7a/db8/ld6 0 2024-09-10T04:05:21.301 INFO:tasks.workunit.client.1.smithi070.stdout:0/646: rmdir d8/d13/d33/d81 39 2024-09-10T04:05:21.302 INFO:tasks.workunit.client.1.smithi070.stdout:1/487: mknod d3/d8/d27/c9c 0 2024-09-10T04:05:21.304 INFO:tasks.workunit.client.1.smithi070.stdout:2/553: mkdir d15/d18/d29/d2a/d2e/da6/dc0 0 2024-09-10T04:05:21.304 INFO:tasks.workunit.client.1.smithi070.stdout:3/595: dread - db/d1c/d25/d28/f7c zero size 2024-09-10T04:05:21.305 INFO:tasks.workunit.client.1.smithi070.stdout:9/621: symlink d9/d7a/lc8 0 2024-09-10T04:05:21.309 INFO:tasks.workunit.client.1.smithi070.stdout:9/622: readlink d9/d1c/dac/d8a/d3c/d62/l70 0 2024-09-10T04:05:21.310 INFO:tasks.workunit.client.1.smithi070.stdout:0/647: dwrite d8/d13/d33/d4f/f7f [0,4194304] 0 2024-09-10T04:05:21.311 INFO:tasks.workunit.client.1.smithi070.stdout:1/488: mknod d3/d10/c9d 0 2024-09-10T04:05:21.312 INFO:tasks.workunit.client.1.smithi070.stdout:7/569: getdents d2/d3 0 2024-09-10T04:05:21.313 INFO:tasks.workunit.client.1.smithi070.stdout:7/570: fsync d2/d7/db/d3a/fac 0 2024-09-10T04:05:21.314 INFO:tasks.workunit.client.1.smithi070.stdout:3/596: rename db/d83/la1 to db/d1c/d89/lc8 0 2024-09-10T04:05:21.315 INFO:tasks.workunit.client.1.smithi070.stdout:0/648: mknod d8/d24/d2d/d34/d6b/cd5 0 2024-09-10T04:05:21.316 INFO:tasks.workunit.client.1.smithi070.stdout:1/489: chown d3/d8/d12/d2f/d32/d4b/l71 92 1 2024-09-10T04:05:21.319 INFO:tasks.workunit.client.1.smithi070.stdout:7/571: rmdir d2/d66 39 2024-09-10T04:05:21.321 INFO:tasks.workunit.client.1.smithi070.stdout:3/597: write db/d1c/d25/d28/d43/f97 [965998,101602] 0 2024-09-10T04:05:21.323 INFO:tasks.workunit.client.1.smithi070.stdout:8/629: dwrite d4/f12 [0,4194304] 0 2024-09-10T04:05:21.325 INFO:tasks.workunit.client.1.smithi070.stdout:1/490: link d3/d10/f73 d3/d8/d27/d7a/d94/f9e 0 2024-09-10T04:05:21.325 INFO:tasks.workunit.client.1.smithi070.stdout:7/572: chown d2/d7/db/d3a/f68 0 1 2024-09-10T04:05:21.326 INFO:tasks.workunit.client.1.smithi070.stdout:9/623: link d9/d1c/dac/d8a/c5b d9/d1c/dac/d8a/d1b/cc9 0 2024-09-10T04:05:21.328 INFO:tasks.workunit.client.1.smithi070.stdout:2/554: dwrite d15/d18/d29/f3d [0,4194304] 0 2024-09-10T04:05:21.329 INFO:tasks.workunit.client.1.smithi070.stdout:3/598: creat db/d79/d94/d99/fc9 x:0 0 0 2024-09-10T04:05:21.329 INFO:tasks.workunit.client.1.smithi070.stdout:7/573: fsync d2/d7/d9/f82 0 2024-09-10T04:05:21.330 INFO:tasks.workunit.client.1.smithi070.stdout:8/630: unlink d4/f15 0 2024-09-10T04:05:21.330 INFO:tasks.workunit.client.1.smithi070.stdout:1/491: dread - d3/d8/d12/d2f/d32/f58 zero size 2024-09-10T04:05:21.331 INFO:tasks.workunit.client.1.smithi070.stdout:9/624: write d9/f68 [1371322,70346] 0 2024-09-10T04:05:21.334 INFO:tasks.workunit.client.1.smithi070.stdout:3/599: chown db/l61 36915407 1 2024-09-10T04:05:21.335 INFO:tasks.workunit.client.1.smithi070.stdout:8/631: write d4/d16/d17/dab/fce [764995,92792] 0 2024-09-10T04:05:21.337 INFO:tasks.workunit.client.1.smithi070.stdout:3/600: chown db/d1c/f8c 50735 1 2024-09-10T04:05:21.338 INFO:tasks.workunit.client.1.smithi070.stdout:9/625: stat d9/d1c/dac/d8a/d3c/d62/f8d 0 2024-09-10T04:05:21.341 INFO:tasks.workunit.client.1.smithi070.stdout:2/555: getdents d15/d18/d90 0 2024-09-10T04:05:21.341 INFO:tasks.workunit.client.0.smithi046.stdout:1/413: sync 2024-09-10T04:05:21.342 INFO:tasks.workunit.client.0.smithi046.stdout:1/414: chown d3/dc/l7f 291 1 2024-09-10T04:05:21.342 INFO:tasks.workunit.client.1.smithi070.stdout:8/632: write d4/d16/d17/d19/d28/f31 [1110078,66150] 0 2024-09-10T04:05:21.343 INFO:tasks.workunit.client.1.smithi070.stdout:1/492: truncate d3/d10/f2c 2628154 0 2024-09-10T04:05:21.343 INFO:tasks.workunit.client.0.smithi046.stdout:1/415: creat d3/f87 x:0 0 0 2024-09-10T04:05:21.346 INFO:tasks.workunit.client.1.smithi070.stdout:0/649: dread d8/d13/f5a [0,4194304] 0 2024-09-10T04:05:21.348 INFO:tasks.workunit.client.1.smithi070.stdout:2/556: write d15/d18/d29/d2a/d2e/d32/f33 [5086132,24773] 0 2024-09-10T04:05:21.349 INFO:tasks.workunit.client.1.smithi070.stdout:1/493: mknod d3/d8/d13/d67/c9f 0 2024-09-10T04:05:21.352 INFO:tasks.workunit.client.1.smithi070.stdout:8/633: rename d4/d16/d17/d1a/d60/fc7 to d4/d16/d7d/fd7 0 2024-09-10T04:05:21.353 INFO:tasks.workunit.client.1.smithi070.stdout:3/601: dwrite db/f74 [0,4194304] 0 2024-09-10T04:05:21.354 INFO:tasks.workunit.client.1.smithi070.stdout:7/574: dwrite d2/d7/d9/d18/f6b [0,4194304] 0 2024-09-10T04:05:21.354 INFO:tasks.workunit.client.1.smithi070.stdout:9/626: link d9/d1c/dac/d8a/l2f d9/d1c/d34/d8b/lca 0 2024-09-10T04:05:21.356 INFO:tasks.workunit.client.1.smithi070.stdout:3/602: stat db/d1c/d25/d28/d43/f9c 0 2024-09-10T04:05:21.356 INFO:tasks.workunit.client.1.smithi070.stdout:1/494: dread d3/d10/f73 [0,4194304] 0 2024-09-10T04:05:21.356 INFO:tasks.workunit.client.1.smithi070.stdout:0/650: unlink d8/d24/d2d/d34/c38 0 2024-09-10T04:05:21.358 INFO:tasks.workunit.client.1.smithi070.stdout:2/557: symlink d15/d18/d29/d2a/d2e/da6/lc1 0 2024-09-10T04:05:21.360 INFO:tasks.workunit.client.0.smithi046.stdout:1/416: rename d3/dc/f6e to d3/dc/f88 0 2024-09-10T04:05:21.360 INFO:tasks.workunit.client.1.smithi070.stdout:2/558: readlink d15/d18/d56/d7c/l83 0 2024-09-10T04:05:21.364 INFO:tasks.workunit.client.1.smithi070.stdout:3/603: symlink db/d1c/d3e/lca 0 2024-09-10T04:05:21.365 INFO:tasks.workunit.client.1.smithi070.stdout:1/495: fsync d3/d8/d12/f45 0 2024-09-10T04:05:21.365 INFO:tasks.workunit.client.1.smithi070.stdout:8/634: symlink d4/dbf/d74/ld8 0 2024-09-10T04:05:21.365 INFO:tasks.workunit.client.1.smithi070.stdout:2/559: mknod d15/d2b/cc2 0 2024-09-10T04:05:21.369 INFO:tasks.workunit.client.1.smithi070.stdout:9/627: dwrite d9/fb3 [0,4194304] 0 2024-09-10T04:05:21.369 INFO:tasks.workunit.client.0.smithi046.stdout:1/417: dwrite d3/d27/d5d/f6a [0,4194304] 0 2024-09-10T04:05:21.371 INFO:tasks.workunit.client.1.smithi070.stdout:8/635: rmdir d4/d16/d73 39 2024-09-10T04:05:21.372 INFO:tasks.workunit.client.1.smithi070.stdout:0/651: link d8/d13/d33/d53/d59/f64 d8/d24/d2d/d34/d9b/dad/fd6 0 2024-09-10T04:05:21.372 INFO:tasks.workunit.client.1.smithi070.stdout:1/496: chown d3/d8/d12/d2f/d32/d4b/l85 151 1 2024-09-10T04:05:21.373 INFO:tasks.workunit.client.0.smithi046.stdout:1/418: read d3/d27/f7d [191516,99049] 0 2024-09-10T04:05:21.373 INFO:tasks.workunit.client.1.smithi070.stdout:2/560: mkdir d15/d18/db2/dc3 0 2024-09-10T04:05:21.374 INFO:tasks.workunit.client.1.smithi070.stdout:3/604: truncate db/d1c/d53/f6b 6387939 0 2024-09-10T04:05:21.375 INFO:tasks.workunit.client.1.smithi070.stdout:8/636: dread d4/d16/f33 [4194304,4194304] 0 2024-09-10T04:05:21.376 INFO:tasks.workunit.client.1.smithi070.stdout:9/628: mknod d9/d1c/dac/d8a/d82/ccb 0 2024-09-10T04:05:21.376 INFO:tasks.workunit.client.1.smithi070.stdout:0/652: symlink d8/d24/d36/d87/ld7 0 2024-09-10T04:05:21.378 INFO:tasks.workunit.client.1.smithi070.stdout:0/653: stat d8/d24/d2d/d34/l89 0 2024-09-10T04:05:21.379 INFO:tasks.workunit.client.0.smithi046.stdout:1/419: mknod d3/dc/d34/d51/c89 0 2024-09-10T04:05:21.379 INFO:tasks.workunit.client.1.smithi070.stdout:0/654: chown d8/cc2 6881 1 2024-09-10T04:05:21.380 INFO:tasks.workunit.client.1.smithi070.stdout:0/655: fdatasync d8/d13/d1a/f3c 0 2024-09-10T04:05:21.384 INFO:tasks.workunit.client.0.smithi046.stdout:1/420: unlink d3/d27/f7d 0 2024-09-10T04:05:21.387 INFO:tasks.workunit.client.1.smithi070.stdout:9/629: rename d9/d1c/dac/d8a/d82/d99 to d9/d1c/d34/d8b/dcc 0 2024-09-10T04:05:21.388 INFO:tasks.workunit.client.0.smithi046.stdout:1/421: creat d3/d27/d5d/d60/f8a x:0 0 0 2024-09-10T04:05:21.389 INFO:tasks.workunit.client.1.smithi070.stdout:8/637: dread d4/d16/f36 [0,4194304] 0 2024-09-10T04:05:21.390 INFO:tasks.workunit.client.1.smithi070.stdout:1/497: creat d3/d8/d27/d7a/d94/fa0 x:0 0 0 2024-09-10T04:05:21.391 INFO:tasks.workunit.client.1.smithi070.stdout:2/561: dread d15/d18/d29/d2a/f74 [4194304,4194304] 0 2024-09-10T04:05:21.392 INFO:tasks.workunit.client.1.smithi070.stdout:0/656: write d8/d13/d33/d81/fb7 [325449,110039] 0 2024-09-10T04:05:21.392 INFO:tasks.workunit.client.0.smithi046.stdout:1/422: write d3/f26 [933344,9708] 0 2024-09-10T04:05:21.392 INFO:tasks.workunit.client.1.smithi070.stdout:1/498: truncate d3/d10/f72 4392841 0 2024-09-10T04:05:21.394 INFO:tasks.workunit.client.0.smithi046.stdout:1/423: truncate d3/d27/d32/f2e 4967073 0 2024-09-10T04:05:21.395 INFO:tasks.workunit.client.0.smithi046.stdout:1/424: dread - d3/f77 zero size 2024-09-10T04:05:21.396 INFO:tasks.workunit.client.0.smithi046.stdout:1/425: read - d3/f77 zero size 2024-09-10T04:05:21.397 INFO:tasks.workunit.client.1.smithi070.stdout:2/562: creat d15/d18/d29/d2a/fc4 x:0 0 0 2024-09-10T04:05:21.397 INFO:tasks.workunit.client.0.smithi046.stdout:1/426: read d3/d27/d32/d1f/f35 [2288961,8251] 0 2024-09-10T04:05:21.398 INFO:tasks.workunit.client.1.smithi070.stdout:0/657: creat d8/d13/d47/d7b/fd8 x:0 0 0 2024-09-10T04:05:21.401 INFO:tasks.workunit.client.1.smithi070.stdout:9/630: symlink d9/d1c/lcd 0 2024-09-10T04:05:21.403 INFO:tasks.workunit.client.1.smithi070.stdout:3/605: dwrite db/d1c/d25/f75 [0,4194304] 0 2024-09-10T04:05:21.408 INFO:tasks.workunit.client.1.smithi070.stdout:8/638: dwrite d4/d16/f3f [0,4194304] 0 2024-09-10T04:05:21.411 INFO:tasks.workunit.client.1.smithi070.stdout:9/631: dread d9/d7a/d72/f7b [0,4194304] 0 2024-09-10T04:05:21.412 INFO:tasks.workunit.client.1.smithi070.stdout:3/606: truncate db/d79/fa3 545500 0 2024-09-10T04:05:21.412 INFO:tasks.workunit.client.1.smithi070.stdout:1/499: write d3/d8/d12/f4a [431371,32851] 0 2024-09-10T04:05:21.414 INFO:tasks.workunit.client.1.smithi070.stdout:9/632: rmdir d9/d7a/d6f/d90 39 2024-09-10T04:05:21.414 INFO:tasks.workunit.client.0.smithi046.stdout:1/427: dwrite d3/d27/f20 [0,4194304] 0 2024-09-10T04:05:21.415 INFO:tasks.workunit.client.1.smithi070.stdout:2/563: dwrite d15/d2b/d94/f9a [0,4194304] 0 2024-09-10T04:05:21.415 INFO:tasks.workunit.client.0.smithi046.stdout:1/428: fdatasync d3/d27/d32/d1f/f55 0 2024-09-10T04:05:21.417 INFO:tasks.workunit.client.0.smithi046.stdout:1/429: write d3/d27/f42 [4103436,31865] 0 2024-09-10T04:05:21.417 INFO:tasks.workunit.client.1.smithi070.stdout:3/607: dread db/d1c/d25/f9a [0,4194304] 0 2024-09-10T04:05:21.418 INFO:tasks.workunit.client.1.smithi070.stdout:9/633: stat d9/d1c/d34/d8b/dcc 0 2024-09-10T04:05:21.420 INFO:tasks.workunit.client.1.smithi070.stdout:8/639: creat d4/d16/d17/d19/d28/fd9 x:0 0 0 2024-09-10T04:05:21.421 INFO:tasks.workunit.client.0.smithi046.stdout:1/430: rename d3/d27/d32/d28/f2c to d3/d27/d2a/f8b 0 2024-09-10T04:05:21.422 INFO:tasks.workunit.client.1.smithi070.stdout:1/500: symlink d3/d8/d13/d3f/d5d/la1 0 2024-09-10T04:05:21.423 INFO:tasks.workunit.client.1.smithi070.stdout:0/658: dread d8/fc [4194304,4194304] 0 2024-09-10T04:05:21.423 INFO:tasks.workunit.client.1.smithi070.stdout:1/501: readlink d3/d20/l77 0 2024-09-10T04:05:21.424 INFO:tasks.workunit.client.0.smithi046.stdout:1/431: dread - d3/d27/d32/d1f/f3c zero size 2024-09-10T04:05:21.425 INFO:tasks.workunit.client.1.smithi070.stdout:3/608: mknod db/d1c/ccb 0 2024-09-10T04:05:21.428 INFO:tasks.workunit.client.1.smithi070.stdout:2/564: dread d15/d18/f26 [0,4194304] 0 2024-09-10T04:05:21.429 INFO:tasks.workunit.client.1.smithi070.stdout:0/659: chown d8/d24/d2d/d34/d6b/cb2 6 1 2024-09-10T04:05:21.431 INFO:tasks.workunit.client.1.smithi070.stdout:3/609: read db/f33 [4663013,118780] 0 2024-09-10T04:05:21.431 INFO:tasks.workunit.client.1.smithi070.stdout:0/660: write d8/d24/d2d/d5f/fd3 [732718,62630] 0 2024-09-10T04:05:21.432 INFO:tasks.workunit.client.1.smithi070.stdout:9/634: write d9/d1c/f55 [3361460,50556] 0 2024-09-10T04:05:21.432 INFO:tasks.workunit.client.1.smithi070.stdout:3/610: write db/d1c/d53/f6e [392293,10677] 0 2024-09-10T04:05:21.433 INFO:tasks.workunit.client.1.smithi070.stdout:2/565: mknod d15/d18/d29/d2a/d2e/da6/cc5 0 2024-09-10T04:05:21.435 INFO:tasks.workunit.client.1.smithi070.stdout:9/635: write d9/d1c/f87 [2352270,110780] 0 2024-09-10T04:05:21.435 INFO:tasks.workunit.client.1.smithi070.stdout:1/502: getdents d3/d10/d1d 0 2024-09-10T04:05:21.437 INFO:tasks.workunit.client.1.smithi070.stdout:2/566: symlink d15/d18/d29/d2a/lc6 0 2024-09-10T04:05:21.437 INFO:tasks.workunit.client.1.smithi070.stdout:9/636: chown d9/d1c/f87 288 1 2024-09-10T04:05:21.437 INFO:tasks.workunit.client.1.smithi070.stdout:3/611: mkdir db/dcc 0 2024-09-10T04:05:21.438 INFO:tasks.workunit.client.1.smithi070.stdout:9/637: fsync d9/d59/f18 0 2024-09-10T04:05:21.439 INFO:tasks.workunit.client.1.smithi070.stdout:1/503: mkdir d3/d8/d12/d82/da2 0 2024-09-10T04:05:21.440 INFO:tasks.workunit.client.1.smithi070.stdout:9/638: chown d9/d59/d2a/d3f/daf 25 1 2024-09-10T04:05:21.440 INFO:tasks.workunit.client.1.smithi070.stdout:1/504: stat d3/d8/d27/d7a/d94 0 2024-09-10T04:05:21.440 INFO:tasks.workunit.client.1.smithi070.stdout:3/612: fdatasync db/d1c/d3e/f50 0 2024-09-10T04:05:21.441 INFO:tasks.workunit.client.1.smithi070.stdout:2/567: mknod d15/d2b/d6e/db8/cc7 0 2024-09-10T04:05:21.442 INFO:tasks.workunit.client.1.smithi070.stdout:0/661: dread d8/d13/d1a/f35 [0,4194304] 0 2024-09-10T04:05:21.442 INFO:tasks.workunit.client.1.smithi070.stdout:2/568: dread - d15/d18/d56/f86 zero size 2024-09-10T04:05:21.443 INFO:tasks.workunit.client.1.smithi070.stdout:3/613: write db/d79/d94/d99/fa5 [290586,118335] 0 2024-09-10T04:05:21.443 INFO:tasks.workunit.client.1.smithi070.stdout:9/639: creat d9/d1c/d34/fce x:0 0 0 2024-09-10T04:05:21.443 INFO:tasks.workunit.client.1.smithi070.stdout:0/662: stat d8/d24/d2d/d34/d6b 0 2024-09-10T04:05:21.447 INFO:tasks.workunit.client.1.smithi070.stdout:9/640: chown d9/d1c/dac/d8a/d1b/fba 13249 1 2024-09-10T04:05:21.448 INFO:tasks.workunit.client.1.smithi070.stdout:0/663: creat d8/d13/d33/d53/dc6/fd9 x:0 0 0 2024-09-10T04:05:21.449 INFO:tasks.workunit.client.1.smithi070.stdout:3/614: creat db/d1c/d53/fcd x:0 0 0 2024-09-10T04:05:21.449 INFO:tasks.workunit.client.1.smithi070.stdout:2/569: write d15/d18/d29/d2a/d2e/d32/fb5 [3151487,51332] 0 2024-09-10T04:05:21.451 INFO:tasks.workunit.client.1.smithi070.stdout:9/641: write d9/d7a/f64 [4277659,115543] 0 2024-09-10T04:05:21.452 INFO:tasks.workunit.client.1.smithi070.stdout:3/615: creat db/d83/fce x:0 0 0 2024-09-10T04:05:21.453 INFO:tasks.workunit.client.1.smithi070.stdout:0/664: creat d8/d13/d33/d81/fda x:0 0 0 2024-09-10T04:05:21.453 INFO:tasks.workunit.client.1.smithi070.stdout:9/642: dread - d9/fad zero size 2024-09-10T04:05:21.454 INFO:tasks.workunit.client.1.smithi070.stdout:1/505: dwrite d3/d10/f73 [0,4194304] 0 2024-09-10T04:05:21.455 INFO:tasks.workunit.client.1.smithi070.stdout:0/665: readlink d8/d24/d36/d87/ld7 0 2024-09-10T04:05:21.458 INFO:tasks.workunit.client.1.smithi070.stdout:9/643: truncate d9/d1c/dac/d8a/d1b/fba 913381 0 2024-09-10T04:05:21.460 INFO:tasks.workunit.client.1.smithi070.stdout:9/644: fdatasync d9/d59/f65 0 2024-09-10T04:05:21.463 INFO:tasks.workunit.client.1.smithi070.stdout:0/666: write d8/fa [2629320,97957] 0 2024-09-10T04:05:21.464 INFO:tasks.workunit.client.1.smithi070.stdout:3/616: truncate db/d1c/d3e/f59 1432601 0 2024-09-10T04:05:21.464 INFO:tasks.workunit.client.1.smithi070.stdout:1/506: dwrite f0 [0,4194304] 0 2024-09-10T04:05:21.466 INFO:tasks.workunit.client.1.smithi070.stdout:9/645: truncate d9/d59/d5d/f67 293034 0 2024-09-10T04:05:21.469 INFO:tasks.workunit.client.1.smithi070.stdout:3/617: write db/d1c/f8c [437633,30321] 0 2024-09-10T04:05:21.469 INFO:tasks.workunit.client.1.smithi070.stdout:0/667: mknod d8/d24/d2d/cdb 0 2024-09-10T04:05:21.470 INFO:tasks.workunit.client.1.smithi070.stdout:9/646: dread - d9/d7a/d6f/d90/d89/fbd zero size 2024-09-10T04:05:21.472 INFO:tasks.workunit.client.1.smithi070.stdout:3/618: fsync db/d1c/d25/d28/f73 0 2024-09-10T04:05:21.473 INFO:tasks.workunit.client.1.smithi070.stdout:9/647: creat d9/d59/d2a/fcf x:0 0 0 2024-09-10T04:05:21.476 INFO:tasks.workunit.client.1.smithi070.stdout:1/507: rename d3/d10/c74 to d3/d8/d12/d2f/d32/ca3 0 2024-09-10T04:05:21.477 INFO:tasks.workunit.client.1.smithi070.stdout:3/619: creat db/d1c/fcf x:0 0 0 2024-09-10T04:05:21.479 INFO:tasks.workunit.client.1.smithi070.stdout:9/648: truncate d9/d7a/d72/f7b 910346 0 2024-09-10T04:05:21.479 INFO:tasks.workunit.client.1.smithi070.stdout:0/668: dwrite d8/d24/d2d/d5f/f60 [0,4194304] 0 2024-09-10T04:05:21.480 INFO:tasks.workunit.client.1.smithi070.stdout:9/649: readlink d9/d7a/lc8 0 2024-09-10T04:05:21.480 INFO:tasks.workunit.client.1.smithi070.stdout:0/669: chown d8/d13/d1a/d75 812 1 2024-09-10T04:05:21.481 INFO:tasks.workunit.client.1.smithi070.stdout:3/620: creat db/d1c/d53/fd0 x:0 0 0 2024-09-10T04:05:21.481 INFO:tasks.workunit.client.1.smithi070.stdout:0/670: chown d8/f1b 59700597 1 2024-09-10T04:05:21.482 INFO:tasks.workunit.client.1.smithi070.stdout:9/650: creat d9/d1c/d34/d8b/dcc/fd0 x:0 0 0 2024-09-10T04:05:21.486 INFO:tasks.workunit.client.1.smithi070.stdout:0/671: creat d8/d13/d33/d4f/fdc x:0 0 0 2024-09-10T04:05:21.487 INFO:tasks.workunit.client.1.smithi070.stdout:3/621: creat db/d1c/d25/fd1 x:0 0 0 2024-09-10T04:05:21.489 INFO:tasks.workunit.client.1.smithi070.stdout:0/672: mknod d8/d24/dce/cdd 0 2024-09-10T04:05:21.490 INFO:tasks.workunit.client.1.smithi070.stdout:9/651: dwrite d9/d1c/dac/d8a/f20 [0,4194304] 0 2024-09-10T04:05:21.492 INFO:tasks.workunit.client.1.smithi070.stdout:3/622: creat db/fd2 x:0 0 0 2024-09-10T04:05:21.493 INFO:tasks.workunit.client.1.smithi070.stdout:0/673: symlink d8/d24/d97/lde 0 2024-09-10T04:05:21.496 INFO:tasks.workunit.client.1.smithi070.stdout:1/508: dwrite d3/d8/d12/d2f/d32/f4c [4194304,4194304] 0 2024-09-10T04:05:21.496 INFO:tasks.workunit.client.1.smithi070.stdout:0/674: read d8/d13/d1a/d75/f84 [759938,124943] 0 2024-09-10T04:05:21.499 INFO:tasks.workunit.client.1.smithi070.stdout:3/623: mkdir db/d1c/d25/d28/dd3 0 2024-09-10T04:05:21.499 INFO:tasks.workunit.client.1.smithi070.stdout:0/675: dread - d8/d13/d1a/f39 zero size 2024-09-10T04:05:21.501 INFO:tasks.workunit.client.1.smithi070.stdout:3/624: mkdir db/d79/dd4 0 2024-09-10T04:05:21.503 INFO:tasks.workunit.client.1.smithi070.stdout:3/625: write db/d1c/d25/fb0 [24793,109963] 0 2024-09-10T04:05:21.504 INFO:tasks.workunit.client.1.smithi070.stdout:0/676: unlink d8/d13/f19 0 2024-09-10T04:05:21.505 INFO:tasks.workunit.client.1.smithi070.stdout:0/677: chown d8/d13/d33/d7d/l83 75796 1 2024-09-10T04:05:21.507 INFO:tasks.workunit.client.1.smithi070.stdout:1/509: link d3/d10/c9d d3/ca4 0 2024-09-10T04:05:21.511 INFO:tasks.workunit.client.1.smithi070.stdout:1/510: mkdir d3/d8/d27/d7a/d94/da5 0 2024-09-10T04:05:21.514 INFO:tasks.workunit.client.1.smithi070.stdout:1/511: unlink d3/d8/d27/l91 0 2024-09-10T04:05:21.516 INFO:tasks.workunit.client.1.smithi070.stdout:0/678: getdents d8/d24/d2d/d34/d9b 0 2024-09-10T04:05:21.516 INFO:tasks.workunit.client.1.smithi070.stdout:9/652: dwrite d9/d7a/f51 [0,4194304] 0 2024-09-10T04:05:21.517 INFO:tasks.workunit.client.1.smithi070.stdout:9/653: chown d9/d1c/d34/fbc 363240 1 2024-09-10T04:05:21.519 INFO:tasks.workunit.client.1.smithi070.stdout:1/512: creat d3/d8/d12/d82/da2/fa6 x:0 0 0 2024-09-10T04:05:21.520 INFO:tasks.workunit.client.1.smithi070.stdout:9/654: symlink d9/d1c/d34/d8b/ld1 0 2024-09-10T04:05:21.521 INFO:tasks.workunit.client.1.smithi070.stdout:1/513: write d3/d8/d27/f95 [3923222,33785] 0 2024-09-10T04:05:21.523 INFO:tasks.workunit.client.1.smithi070.stdout:9/655: mknod d9/d7a/d6f/d90/d89/cd2 0 2024-09-10T04:05:21.528 INFO:tasks.workunit.client.1.smithi070.stdout:9/656: dread d9/d1c/d34/d63/f69 [0,4194304] 0 2024-09-10T04:05:21.529 INFO:tasks.workunit.client.1.smithi070.stdout:3/626: dwrite db/d1c/f6d [0,4194304] 0 2024-09-10T04:05:21.532 INFO:tasks.workunit.client.1.smithi070.stdout:0/679: dread d8/f27 [0,4194304] 0 2024-09-10T04:05:21.533 INFO:tasks.workunit.client.1.smithi070.stdout:3/627: creat db/dcc/fd5 x:0 0 0 2024-09-10T04:05:21.535 INFO:tasks.workunit.client.1.smithi070.stdout:3/628: symlink db/dcc/ld6 0 2024-09-10T04:05:21.536 INFO:tasks.workunit.client.1.smithi070.stdout:0/680: symlink d8/d13/d1a/d8f/d95/ldf 0 2024-09-10T04:05:21.536 INFO:tasks.workunit.client.1.smithi070.stdout:9/657: link d9/d1c/lcd d9/d1c/d34/ld3 0 2024-09-10T04:05:21.538 INFO:tasks.workunit.client.1.smithi070.stdout:0/681: symlink d8/d24/dce/le0 0 2024-09-10T04:05:21.541 INFO:tasks.workunit.client.1.smithi070.stdout:1/514: dread d3/d8/d12/f24 [0,4194304] 0 2024-09-10T04:05:21.545 INFO:tasks.workunit.client.1.smithi070.stdout:3/629: dwrite db/d79/db6/fc1 [0,4194304] 0 2024-09-10T04:05:21.547 INFO:tasks.workunit.client.1.smithi070.stdout:0/682: dwrite f5 [4194304,4194304] 0 2024-09-10T04:05:21.548 INFO:tasks.workunit.client.1.smithi070.stdout:0/683: fdatasync d8/d13/d33/f3f 0 2024-09-10T04:05:21.551 INFO:tasks.workunit.client.1.smithi070.stdout:0/684: read d8/d13/d33/f3f [472584,109499] 0 2024-09-10T04:05:21.552 INFO:tasks.workunit.client.1.smithi070.stdout:3/630: creat db/d79/dd4/fd7 x:0 0 0 2024-09-10T04:05:21.553 INFO:tasks.workunit.client.1.smithi070.stdout:0/685: dread - d8/f99 zero size 2024-09-10T04:05:21.556 INFO:tasks.workunit.client.1.smithi070.stdout:0/686: dread - d8/d13/d33/d81/fda zero size 2024-09-10T04:05:21.557 INFO:tasks.workunit.client.1.smithi070.stdout:3/631: read db/d1c/d25/d28/d43/f97 [607720,27364] 0 2024-09-10T04:05:21.558 INFO:tasks.workunit.client.1.smithi070.stdout:0/687: dread - d8/d13/d1a/d8f/f9c zero size 2024-09-10T04:05:21.558 INFO:tasks.workunit.client.1.smithi070.stdout:1/515: dwrite d3/d8/d13/f14 [0,4194304] 0 2024-09-10T04:05:21.559 INFO:tasks.workunit.client.1.smithi070.stdout:0/688: stat d8/d13/d33/d53/dc6/fd9 0 2024-09-10T04:05:21.560 INFO:tasks.workunit.client.1.smithi070.stdout:1/516: chown d3/d8/d13/d3f/f53 446995530 1 2024-09-10T04:05:21.563 INFO:tasks.workunit.client.1.smithi070.stdout:0/689: symlink d8/d24/d36/d87/le1 0 2024-09-10T04:05:21.563 INFO:tasks.workunit.client.1.smithi070.stdout:3/632: mkdir db/d1c/d25/dd8 0 2024-09-10T04:05:21.563 INFO:tasks.workunit.client.1.smithi070.stdout:1/517: write d3/d8/d12/f24 [3829220,66044] 0 2024-09-10T04:05:21.567 INFO:tasks.workunit.client.1.smithi070.stdout:6/637: sync 2024-09-10T04:05:21.567 INFO:tasks.workunit.client.1.smithi070.stdout:4/505: sync 2024-09-10T04:05:21.567 INFO:tasks.workunit.client.1.smithi070.stdout:5/578: sync 2024-09-10T04:05:21.569 INFO:tasks.workunit.client.1.smithi070.stdout:1/518: creat d3/d8/d27/d7a/fa7 x:0 0 0 2024-09-10T04:05:21.570 INFO:tasks.workunit.client.1.smithi070.stdout:3/633: symlink db/d1c/d25/d28/ld9 0 2024-09-10T04:05:21.571 INFO:tasks.workunit.client.1.smithi070.stdout:0/690: rename d8/d13/d33 to d8/d13/de2 0 2024-09-10T04:05:21.572 INFO:tasks.workunit.client.1.smithi070.stdout:5/579: mknod d17/d24/d2b/cc1 0 2024-09-10T04:05:21.572 INFO:tasks.workunit.client.1.smithi070.stdout:0/691: chown d8/d24/d36/c3b 1707011872 1 2024-09-10T04:05:21.575 INFO:tasks.workunit.client.1.smithi070.stdout:6/638: mknod d0/d3/d18/d1b/cc7 0 2024-09-10T04:05:21.576 INFO:tasks.workunit.client.1.smithi070.stdout:5/580: mkdir d17/d19/d6a/d54/d57/d63/db9/dc2 0 2024-09-10T04:05:21.577 INFO:tasks.workunit.client.1.smithi070.stdout:0/692: symlink d8/d85/dca/le3 0 2024-09-10T04:05:21.620 INFO:tasks.workunit.client.0.smithi046.stdout:3/519: sync 2024-09-10T04:05:21.620 INFO:tasks.workunit.client.0.smithi046.stdout:3/520: readlink d12/d13/d37/d9c/l28 0 2024-09-10T04:05:21.623 INFO:tasks.workunit.client.0.smithi046.stdout:3/521: write f4 [298583,32337] 0 2024-09-10T04:05:21.627 INFO:tasks.workunit.client.0.smithi046.stdout:3/522: rename d12/d13/d37/dab/c8c to d12/d5d/d71/d49/d5c/cb0 0 2024-09-10T04:05:21.632 INFO:tasks.workunit.client.0.smithi046.stdout:3/523: dwrite d12/d13/d37/d9c/f4c [0,4194304] 0 2024-09-10T04:05:21.632 INFO:tasks.workunit.client.0.smithi046.stdout:3/524: chown d12/d5d/d71/d49 26 1 2024-09-10T04:05:21.635 INFO:tasks.workunit.client.0.smithi046.stdout:3/525: mkdir d12/d5d/d71/db1 0 2024-09-10T04:05:21.637 INFO:tasks.workunit.client.0.smithi046.stdout:3/526: creat d12/d13/d37/dab/fb2 x:0 0 0 2024-09-10T04:05:21.642 INFO:tasks.workunit.client.0.smithi046.stdout:3/527: dread d12/d13/d14/d1e/d2d/f80 [0,4194304] 0 2024-09-10T04:05:21.643 INFO:tasks.workunit.client.0.smithi046.stdout:3/528: creat d12/d13/d37/d94/fb3 x:0 0 0 2024-09-10T04:05:21.645 INFO:tasks.workunit.client.0.smithi046.stdout:3/529: fsync d12/f5f 0 2024-09-10T04:05:21.646 INFO:tasks.workunit.client.0.smithi046.stdout:3/530: chown d12/d13/d14/d1e/d2d/d39 453582 1 2024-09-10T04:05:21.652 INFO:tasks.workunit.client.0.smithi046.stdout:3/531: dwrite d12/d13/d37/d9c/f4c [0,4194304] 0 2024-09-10T04:05:21.654 INFO:tasks.workunit.client.0.smithi046.stdout:3/532: truncate d12/d13/d14/f33 1898815 0 2024-09-10T04:05:21.655 INFO:tasks.workunit.client.0.smithi046.stdout:3/533: getdents d12/d5d/d71/db1 0 2024-09-10T04:05:21.657 INFO:tasks.workunit.client.0.smithi046.stdout:3/534: truncate d12/d5d/d71/d49/d5e/f93 139553 0 2024-09-10T04:05:21.657 INFO:tasks.workunit.client.0.smithi046.stdout:3/535: chown d12/d5d/d71/d49 1159 1 2024-09-10T04:05:21.660 INFO:tasks.workunit.client.1.smithi070.stdout:3/634: write db/f33 [6273008,113397] 0 2024-09-10T04:05:21.661 INFO:tasks.workunit.client.1.smithi070.stdout:0/693: mknod d8/d85/ce4 0 2024-09-10T04:05:21.664 INFO:tasks.workunit.client.1.smithi070.stdout:6/639: link d0/f10 d0/d3/d1f/fc8 0 2024-09-10T04:05:21.668 INFO:tasks.workunit.client.1.smithi070.stdout:6/640: creat d0/d7f/fc9 x:0 0 0 2024-09-10T04:05:21.670 INFO:tasks.workunit.client.1.smithi070.stdout:4/506: dwrite de/d13/d18/d26/d46/d85/f9a [0,4194304] 0 2024-09-10T04:05:21.672 INFO:tasks.workunit.client.1.smithi070.stdout:1/519: dwrite d3/d10/d1d/f46 [0,4194304] 0 2024-09-10T04:05:21.673 INFO:tasks.workunit.client.1.smithi070.stdout:3/635: write db/d1c/d25/d28/f72 [801842,69701] 0 2024-09-10T04:05:21.674 INFO:tasks.workunit.client.1.smithi070.stdout:1/520: chown d3/d8/d13/f25 4097 1 2024-09-10T04:05:21.676 INFO:tasks.workunit.client.1.smithi070.stdout:6/641: mknod d0/d3/d18/d62/d70/cca 0 2024-09-10T04:05:21.677 INFO:tasks.workunit.client.0.smithi046.stdout:3/536: dwrite d12/d13/d14/d1e/d2d/f2e [0,4194304] 0 2024-09-10T04:05:21.681 INFO:tasks.workunit.client.1.smithi070.stdout:5/581: dwrite d17/d19/d6a/d54/f62 [0,4194304] 0 2024-09-10T04:05:21.682 INFO:tasks.workunit.client.1.smithi070.stdout:1/521: mkdir d3/d8/d12/d2f/d32/da8 0 2024-09-10T04:05:21.684 INFO:tasks.workunit.client.0.smithi046.stdout:3/537: creat d12/d5d/d71/d49/d5e/d96/da5/fb4 x:0 0 0 2024-09-10T04:05:21.685 INFO:tasks.workunit.client.1.smithi070.stdout:5/582: write d17/d24/d2b/f9d [1011778,89629] 0 2024-09-10T04:05:21.685 INFO:tasks.workunit.client.0.smithi046.stdout:7/534: sync 2024-09-10T04:05:21.688 INFO:tasks.workunit.client.1.smithi070.stdout:6/642: unlink d0/d3/d1f/d55/c8d 0 2024-09-10T04:05:21.689 INFO:tasks.workunit.client.1.smithi070.stdout:3/636: rename db/d1c/d3e/lc2 to db/d1c/d25/d28/dd3/lda 0 2024-09-10T04:05:21.690 INFO:tasks.workunit.client.1.smithi070.stdout:6/643: dread d0/fa7 [0,4194304] 0 2024-09-10T04:05:21.690 INFO:tasks.workunit.client.1.smithi070.stdout:3/637: dread - db/d1c/d89/f9f zero size 2024-09-10T04:05:21.693 INFO:tasks.workunit.client.0.smithi046.stdout:3/538: rename d12/d13/d37/d9c to d12/d13/d37/d51/db5 0 2024-09-10T04:05:21.696 INFO:tasks.workunit.client.0.smithi046.stdout:3/539: creat d12/d13/d37/d51/db5/d4b/fb6 x:0 0 0 2024-09-10T04:05:21.698 INFO:tasks.workunit.client.1.smithi070.stdout:4/507: dread de/d13/f45 [0,4194304] 0 2024-09-10T04:05:21.700 INFO:tasks.workunit.client.1.smithi070.stdout:5/583: link d17/d19/d6a/d54/f62 d17/d19/fc3 0 2024-09-10T04:05:21.705 INFO:tasks.workunit.client.1.smithi070.stdout:5/584: creat d17/d19/d6a/d54/d57/d63/db9/fc4 x:0 0 0 2024-09-10T04:05:21.705 INFO:tasks.workunit.client.1.smithi070.stdout:4/508: dwrite de/d13/d18/d23/f3c [0,4194304] 0 2024-09-10T04:05:21.706 INFO:tasks.workunit.client.0.smithi046.stdout:7/535: write d5/d9/d12/fa4 [4738061,28927] 0 2024-09-10T04:05:21.708 INFO:tasks.workunit.client.1.smithi070.stdout:6/644: dread d0/f8 [0,4194304] 0 2024-09-10T04:05:21.708 INFO:tasks.workunit.client.1.smithi070.stdout:1/522: truncate d3/d10/f5e 2035784 0 2024-09-10T04:05:21.710 INFO:tasks.workunit.client.0.smithi046.stdout:7/536: creat d5/d6/db/dd/d24/d76/fac x:0 0 0 2024-09-10T04:05:21.712 INFO:tasks.workunit.client.1.smithi070.stdout:4/509: symlink de/d13/d18/d23/d47/lae 0 2024-09-10T04:05:21.713 INFO:tasks.workunit.client.0.smithi046.stdout:3/540: truncate d12/d13/d14/d1e/d2d/f80 732170 0 2024-09-10T04:05:21.713 INFO:tasks.workunit.client.1.smithi070.stdout:4/510: fsync de/d13/d18/d26/d8c/f95 0 2024-09-10T04:05:21.714 INFO:tasks.workunit.client.0.smithi046.stdout:7/537: symlink d5/d9/lad 0 2024-09-10T04:05:21.714 INFO:tasks.workunit.client.0.smithi046.stdout:3/541: chown d12/d13/d37/d51/db5/f4c 340426463 1 2024-09-10T04:05:21.715 INFO:tasks.workunit.client.1.smithi070.stdout:7/575: sync 2024-09-10T04:05:21.716 INFO:tasks.workunit.client.0.smithi046.stdout:7/538: symlink d5/d6/d39/d5f/lae 0 2024-09-10T04:05:21.718 INFO:tasks.workunit.client.1.smithi070.stdout:1/523: link d3/d10/f73 d3/d8/d12/d2f/fa9 0 2024-09-10T04:05:21.718 INFO:tasks.workunit.client.0.smithi046.stdout:3/542: mkdir d12/d13/d37/d51/db5/d4b/db7 0 2024-09-10T04:05:21.719 INFO:tasks.workunit.client.1.smithi070.stdout:7/576: symlink d2/d3/lca 0 2024-09-10T04:05:21.720 INFO:tasks.workunit.client.0.smithi046.stdout:3/543: fsync f3 0 2024-09-10T04:05:21.722 INFO:tasks.workunit.client.1.smithi070.stdout:5/585: dwrite d17/f75 [0,4194304] 0 2024-09-10T04:05:21.723 INFO:tasks.workunit.client.1.smithi070.stdout:4/511: dwrite de/d13/d5b/d7b/fac [0,4194304] 0 2024-09-10T04:05:21.724 INFO:tasks.workunit.client.1.smithi070.stdout:1/524: mknod d3/d8/d12/caa 0 2024-09-10T04:05:21.724 INFO:tasks.workunit.client.0.smithi046.stdout:3/544: rename d12/d5d/d71/d49/d5c/c76 to d12/d13/d14/d1e/d2d/d39/d67/d92/cb8 0 2024-09-10T04:05:21.725 INFO:tasks.workunit.client.1.smithi070.stdout:4/512: write de/d13/d18/d23/d47/d59/f87 [900728,106580] 0 2024-09-10T04:05:21.726 INFO:tasks.workunit.client.0.smithi046.stdout:3/545: stat d12/d13/d37/d51/db5/d4b/d91 0 2024-09-10T04:05:21.726 INFO:tasks.workunit.client.1.smithi070.stdout:1/525: write d3/d8/d13/f18 [4631293,39554] 0 2024-09-10T04:05:21.727 INFO:tasks.workunit.client.1.smithi070.stdout:4/513: stat de/f34 0 2024-09-10T04:05:21.729 INFO:tasks.workunit.client.1.smithi070.stdout:5/586: symlink d17/d24/dbf/lc5 0 2024-09-10T04:05:21.733 INFO:tasks.workunit.client.1.smithi070.stdout:4/514: mkdir de/d13/d18/d26/d46/daf 0 2024-09-10T04:05:21.735 INFO:tasks.workunit.client.0.smithi046.stdout:3/546: dwrite d12/d13/d14/d1e/d2d/f88 [0,4194304] 0 2024-09-10T04:05:21.735 INFO:tasks.workunit.client.1.smithi070.stdout:1/526: dwrite d3/d8/f49 [0,4194304] 0 2024-09-10T04:05:21.736 INFO:tasks.workunit.client.1.smithi070.stdout:1/527: write d3/d8/d12/d2f/d32/f6b [928875,85163] 0 2024-09-10T04:05:21.738 INFO:tasks.workunit.client.0.smithi046.stdout:7/539: write d5/d6/d39/d5f/f96 [763332,106513] 0 2024-09-10T04:05:21.740 INFO:tasks.workunit.client.1.smithi070.stdout:4/515: chown de/d13/d43/c94 0 1 2024-09-10T04:05:21.741 INFO:tasks.workunit.client.0.smithi046.stdout:3/547: symlink d12/d13/d37/d51/db5/d4b/db7/lb9 0 2024-09-10T04:05:21.742 INFO:tasks.workunit.client.1.smithi070.stdout:4/516: fsync de/d13/d18/d26/f2b 0 2024-09-10T04:05:21.744 INFO:tasks.workunit.client.1.smithi070.stdout:7/577: chown d2/d3/dc/d5d/f46 677883 1 2024-09-10T04:05:21.745 INFO:tasks.workunit.client.0.smithi046.stdout:3/548: mknod d12/d13/d37/d51/db5/d4b/cba 0 2024-09-10T04:05:21.748 INFO:tasks.workunit.client.1.smithi070.stdout:1/528: mknod d3/d8/d27/d7a/d94/cab 0 2024-09-10T04:05:21.749 INFO:tasks.workunit.client.1.smithi070.stdout:1/529: write d3/d8/d13/f14 [3822658,54717] 0 2024-09-10T04:05:21.750 INFO:tasks.workunit.client.1.smithi070.stdout:5/587: creat d17/d19/d6a/d54/d57/d63/fc6 x:0 0 0 2024-09-10T04:05:21.751 INFO:tasks.workunit.client.0.smithi046.stdout:3/549: fdatasync d12/d13/d37/f45 0 2024-09-10T04:05:21.755 INFO:tasks.workunit.client.1.smithi070.stdout:7/578: creat d2/d4e/fcb x:0 0 0 2024-09-10T04:05:21.756 INFO:tasks.workunit.client.1.smithi070.stdout:7/579: stat d2/d7/d9/l95 0 2024-09-10T04:05:21.757 INFO:tasks.workunit.client.1.smithi070.stdout:7/580: fdatasync d2/d3/dc/d5d/f1d 0 2024-09-10T04:05:21.761 INFO:tasks.workunit.client.1.smithi070.stdout:1/530: rename d3/d10/d1d/f6c to d3/d8/d27/fac 0 2024-09-10T04:05:21.762 INFO:tasks.workunit.client.1.smithi070.stdout:1/531: chown d3/d8/d13/d3f/d5d/la1 3028 1 2024-09-10T04:05:21.762 INFO:tasks.workunit.client.1.smithi070.stdout:5/588: truncate d17/d24/f25 166889 0 2024-09-10T04:05:21.762 INFO:tasks.workunit.client.1.smithi070.stdout:5/589: fsync d17/d19/d35/f5f 0 2024-09-10T04:05:21.763 INFO:tasks.workunit.client.1.smithi070.stdout:1/532: dread - d3/d8/d27/d7a/d94/fa0 zero size 2024-09-10T04:05:21.764 INFO:tasks.workunit.client.1.smithi070.stdout:4/517: mkdir de/db0 0 2024-09-10T04:05:21.764 INFO:tasks.workunit.client.0.smithi046.stdout:7/540: dwrite d5/d6/db/dd/f2f [0,4194304] 0 2024-09-10T04:05:21.766 INFO:tasks.workunit.client.1.smithi070.stdout:7/581: unlink d2/d7/d9/d18/d3c/d52/fa4 0 2024-09-10T04:05:21.766 INFO:tasks.workunit.client.1.smithi070.stdout:5/590: read - d17/d37/f73 zero size 2024-09-10T04:05:21.767 INFO:tasks.workunit.client.0.smithi046.stdout:7/541: rmdir d5/d9/d12/d72/d77/da7 39 2024-09-10T04:05:21.769 INFO:tasks.workunit.client.1.smithi070.stdout:7/582: write d2/d3/dc/d13/fc7 [799646,33058] 0 2024-09-10T04:05:21.770 INFO:tasks.workunit.client.0.smithi046.stdout:7/542: symlink d5/d7f/d78/laf 0 2024-09-10T04:05:21.771 INFO:tasks.workunit.client.1.smithi070.stdout:1/533: dwrite d3/d8/d27/f93 [0,4194304] 0 2024-09-10T04:05:21.775 INFO:tasks.workunit.client.0.smithi046.stdout:7/543: rename d5/d6/d32/c40 to d5/d6/d55/cb0 0 2024-09-10T04:05:21.776 INFO:tasks.workunit.client.1.smithi070.stdout:7/583: readlink d2/d3/d63/da9/dc1/l4b 0 2024-09-10T04:05:21.780 INFO:tasks.workunit.client.0.smithi046.stdout:7/544: rename d5/d6/d39/f69 to d5/d6/d32/dab/fb1 0 2024-09-10T04:05:21.781 INFO:tasks.workunit.client.1.smithi070.stdout:5/591: mknod d17/d19/d6a/d54/d57/d63/db9/dc2/cc7 0 2024-09-10T04:05:21.784 INFO:tasks.workunit.client.1.smithi070.stdout:5/592: dread d17/f75 [0,4194304] 0 2024-09-10T04:05:21.788 INFO:tasks.workunit.client.0.smithi046.stdout:7/545: dwrite d5/d6/f1c [0,4194304] 0 2024-09-10T04:05:21.790 INFO:tasks.workunit.client.0.smithi046.stdout:7/546: write d5/f35 [5587159,43780] 0 2024-09-10T04:05:21.791 INFO:tasks.workunit.client.0.smithi046.stdout:7/547: write d5/d6/d55/d89/fa8 [253207,35615] 0 2024-09-10T04:05:21.793 INFO:tasks.workunit.client.1.smithi070.stdout:5/593: rename f14 to d17/d6f/fc8 0 2024-09-10T04:05:21.794 INFO:tasks.workunit.client.1.smithi070.stdout:5/594: stat d17/d19/f1d 0 2024-09-10T04:05:21.796 INFO:tasks.workunit.client.1.smithi070.stdout:1/534: creat d3/d8/d12/d2f/d32/fad x:0 0 0 2024-09-10T04:05:21.797 INFO:tasks.workunit.client.0.smithi046.stdout:7/548: creat d5/d6/db/dd/d24/d76/d9c/fb2 x:0 0 0 2024-09-10T04:05:21.798 INFO:tasks.workunit.client.1.smithi070.stdout:4/518: dwrite de/d13/d18/d23/d47/f51 [0,4194304] 0 2024-09-10T04:05:21.800 INFO:tasks.workunit.client.1.smithi070.stdout:4/519: dread - de/d13/d18/d23/d39/f8e zero size 2024-09-10T04:05:21.801 INFO:tasks.workunit.client.1.smithi070.stdout:7/584: creat d2/d7/d9/d18/fcc x:0 0 0 2024-09-10T04:05:21.808 INFO:tasks.workunit.client.1.smithi070.stdout:5/595: link d17/d19/f44 d17/d6f/d32/d90/d99/fc9 0 2024-09-10T04:05:21.810 INFO:tasks.workunit.client.1.smithi070.stdout:4/520: dwrite de/d13/d18/d26/d46/f71 [0,4194304] 0 2024-09-10T04:05:21.812 INFO:tasks.workunit.client.1.smithi070.stdout:4/521: write de/d13/f3e [418225,32452] 0 2024-09-10T04:05:21.813 INFO:tasks.workunit.client.1.smithi070.stdout:1/535: link d3/d8/d12/d2f/d32/d4b/d54/c76 d3/d8/d13/d67/d83/cae 0 2024-09-10T04:05:21.815 INFO:tasks.workunit.client.1.smithi070.stdout:5/596: mknod d17/d6f/d32/d90/da5/cca 0 2024-09-10T04:05:21.817 INFO:tasks.workunit.client.1.smithi070.stdout:7/585: link d2/d40/d6c/l76 d2/d7/d9/d18/d3c/d52/lcd 0 2024-09-10T04:05:21.819 INFO:tasks.workunit.client.0.smithi046.stdout:7/549: dread d5/d6/db/f29 [0,4194304] 0 2024-09-10T04:05:21.819 INFO:tasks.workunit.client.1.smithi070.stdout:1/536: creat d3/d8/d13/faf x:0 0 0 2024-09-10T04:05:21.820 INFO:tasks.workunit.client.0.smithi046.stdout:7/550: fdatasync d5/d6/d55/d89/f9d 0 2024-09-10T04:05:21.821 INFO:tasks.workunit.client.1.smithi070.stdout:4/522: dwrite de/d13/d18/d23/d47/f51 [0,4194304] 0 2024-09-10T04:05:21.821 INFO:tasks.workunit.client.1.smithi070.stdout:7/586: unlink d2/d7/db/c71 0 2024-09-10T04:05:21.822 INFO:tasks.workunit.client.1.smithi070.stdout:1/537: truncate d3/d8/d27/d7a/fa7 905223 0 2024-09-10T04:05:21.824 INFO:tasks.workunit.client.0.smithi046.stdout:7/551: write d5/d6/db/f29 [2855885,43359] 0 2024-09-10T04:05:21.825 INFO:tasks.workunit.client.1.smithi070.stdout:1/538: chown d3/d8/d12/d2f/d32/f4c 237389 1 2024-09-10T04:05:21.826 INFO:tasks.workunit.client.1.smithi070.stdout:5/597: link d17/d37/f3d d17/d24/d84/d9b/fcb 0 2024-09-10T04:05:21.826 INFO:tasks.workunit.client.1.smithi070.stdout:4/523: dread de/d13/f3e [0,4194304] 0 2024-09-10T04:05:21.828 INFO:tasks.workunit.client.0.smithi046.stdout:7/552: creat d5/d6/d32/dab/fb3 x:0 0 0 2024-09-10T04:05:21.829 INFO:tasks.workunit.client.1.smithi070.stdout:1/539: mknod d3/d8/d12/d82/da2/cb0 0 2024-09-10T04:05:21.831 INFO:tasks.workunit.client.0.smithi046.stdout:7/553: mkdir d5/d9/d43/db4 0 2024-09-10T04:05:21.831 INFO:tasks.workunit.client.1.smithi070.stdout:5/598: mkdir d17/d24/dcc 0 2024-09-10T04:05:21.832 INFO:tasks.workunit.client.1.smithi070.stdout:7/587: dwrite d2/d4e/f9b [0,4194304] 0 2024-09-10T04:05:21.833 INFO:tasks.workunit.client.1.smithi070.stdout:5/599: write d17/d37/f8d [183300,68993] 0 2024-09-10T04:05:21.834 INFO:tasks.workunit.client.1.smithi070.stdout:4/524: rmdir de/d13/d18/d23/d39/d77 39 2024-09-10T04:05:21.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:21 smithi070 ceph-mon[35495]: pgmap v264: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 61 MiB/s rd, 256 MiB/s wr, 419 op/s 2024-09-10T04:05:21.835 INFO:tasks.workunit.client.1.smithi070.stdout:7/588: creat d2/d40/d6c/db2/fce x:0 0 0 2024-09-10T04:05:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:21 smithi046 ceph-mon[28328]: pgmap v264: 65 pgs: 65 active+clean; 1.9 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 61 MiB/s rd, 256 MiB/s wr, 419 op/s 2024-09-10T04:05:21.837 INFO:tasks.workunit.client.1.smithi070.stdout:5/600: unlink d17/d19/d6a/f5e 0 2024-09-10T04:05:21.839 INFO:tasks.workunit.client.1.smithi070.stdout:1/540: link d3/d8/d13/d3f/d6d/d81/l9b d3/d8/d27/d9a/lb1 0 2024-09-10T04:05:21.841 INFO:tasks.workunit.client.1.smithi070.stdout:4/525: rename de/d13/d18/d23/d47/d59/d79/fa5 to de/d13/d18/d23/d39/d77/fb1 0 2024-09-10T04:05:21.841 INFO:tasks.workunit.client.0.smithi046.stdout:9/515: sync 2024-09-10T04:05:21.842 INFO:tasks.workunit.client.0.smithi046.stdout:9/516: readlink d4/d29/d32/l3e 0 2024-09-10T04:05:21.843 INFO:tasks.workunit.client.1.smithi070.stdout:1/541: mkdir d3/d10/db2 0 2024-09-10T04:05:21.844 INFO:tasks.workunit.client.1.smithi070.stdout:7/589: dwrite d2/d3/dc/d79/fb9 [0,4194304] 0 2024-09-10T04:05:21.845 INFO:tasks.workunit.client.1.smithi070.stdout:4/526: symlink de/d13/d18/d26/d8c/d93/lb2 0 2024-09-10T04:05:21.845 INFO:tasks.workunit.client.1.smithi070.stdout:1/542: chown d3/d8/d27/d7a/d94/fa0 943801456 1 2024-09-10T04:05:21.847 INFO:tasks.workunit.client.1.smithi070.stdout:5/601: write d17/d19/d6a/d54/d57/da3/f97 [969119,65829] 0 2024-09-10T04:05:21.850 INFO:tasks.workunit.client.1.smithi070.stdout:7/590: fdatasync d2/d3/dc/d5d/d41/f50 0 2024-09-10T04:05:21.850 INFO:tasks.workunit.client.1.smithi070.stdout:1/543: truncate d3/d8/d13/f17 2492997 0 2024-09-10T04:05:21.853 INFO:tasks.workunit.client.1.smithi070.stdout:4/527: dwrite de/d13/f45 [0,4194304] 0 2024-09-10T04:05:21.863 INFO:tasks.workunit.client.1.smithi070.stdout:4/528: creat de/d13/d18/d23/d47/d59/d79/fb3 x:0 0 0 2024-09-10T04:05:21.863 INFO:tasks.workunit.client.1.smithi070.stdout:1/544: dwrite d3/d8/fc [8388608,4194304] 0 2024-09-10T04:05:21.863 INFO:tasks.workunit.client.1.smithi070.stdout:7/591: dwrite d2/d3/dc/d5d/d42/fb5 [0,4194304] 0 2024-09-10T04:05:21.864 INFO:tasks.workunit.client.1.smithi070.stdout:4/529: fsync de/d13/d18/f4d 0 2024-09-10T04:05:21.867 INFO:tasks.workunit.client.1.smithi070.stdout:4/530: chown de/d13/d18/d23/d47/d59/d79/f66 85841 1 2024-09-10T04:05:21.870 INFO:tasks.workunit.client.1.smithi070.stdout:1/545: dread - d3/d8/d13/d3f/f69 zero size 2024-09-10T04:05:21.872 INFO:tasks.workunit.client.1.smithi070.stdout:4/531: write de/d13/d5b/d7b/f81 [936688,53415] 0 2024-09-10T04:05:21.873 INFO:tasks.workunit.client.1.smithi070.stdout:7/592: creat d2/d3/dc/d5d/d41/da8/fcf x:0 0 0 2024-09-10T04:05:21.875 INFO:tasks.workunit.client.1.smithi070.stdout:4/532: write de/d13/f45 [3295953,121723] 0 2024-09-10T04:05:21.876 INFO:tasks.workunit.client.1.smithi070.stdout:7/593: write d2/d3/dc/d5d/d2d/f8e [289940,73662] 0 2024-09-10T04:05:21.877 INFO:tasks.workunit.client.1.smithi070.stdout:7/594: readlink d2/d7/d9/l81 0 2024-09-10T04:05:21.878 INFO:tasks.workunit.client.1.smithi070.stdout:1/546: dwrite f1 [4194304,4194304] 0 2024-09-10T04:05:21.879 INFO:tasks.workunit.client.1.smithi070.stdout:7/595: chown d2/d7/d9/d18/c1f 0 1 2024-09-10T04:05:21.879 INFO:tasks.workunit.client.1.smithi070.stdout:2/570: sync 2024-09-10T04:05:21.879 INFO:tasks.workunit.client.1.smithi070.stdout:8/640: sync 2024-09-10T04:05:21.884 INFO:tasks.workunit.client.1.smithi070.stdout:1/547: truncate d3/d8/d27/d7a/fa7 1419541 0 2024-09-10T04:05:21.885 INFO:tasks.workunit.client.1.smithi070.stdout:4/533: dread de/d13/d18/d26/f2f [0,4194304] 0 2024-09-10T04:05:21.886 INFO:tasks.workunit.client.1.smithi070.stdout:7/596: rename d2/d3/lca to d2/dc5/ld0 0 2024-09-10T04:05:21.887 INFO:tasks.workunit.client.1.smithi070.stdout:1/548: write d3/d8/d12/d2f/d32/f6b [158824,110642] 0 2024-09-10T04:05:21.889 INFO:tasks.workunit.client.1.smithi070.stdout:8/641: mknod d4/d16/d17/d1a/d60/d7a/d84/cda 0 2024-09-10T04:05:21.889 INFO:tasks.workunit.client.1.smithi070.stdout:8/642: chown d4/d16/d17/dab 1886233 1 2024-09-10T04:05:21.891 INFO:tasks.workunit.client.1.smithi070.stdout:7/597: symlink d2/d7/db/ld1 0 2024-09-10T04:05:21.892 INFO:tasks.workunit.client.1.smithi070.stdout:8/643: truncate d4/dcf/f9a 345090 0 2024-09-10T04:05:21.897 INFO:tasks.workunit.client.1.smithi070.stdout:2/571: dread d15/d18/f27 [0,4194304] 0 2024-09-10T04:05:21.897 INFO:tasks.workunit.client.1.smithi070.stdout:4/534: write f0 [7542165,87914] 0 2024-09-10T04:05:21.898 INFO:tasks.workunit.client.1.smithi070.stdout:8/644: rename d4/d39/cb1 to d4/dcf/cdb 0 2024-09-10T04:05:21.899 INFO:tasks.workunit.client.1.smithi070.stdout:8/645: fdatasync d4/dbf/f6d 0 2024-09-10T04:05:21.899 INFO:tasks.workunit.client.1.smithi070.stdout:7/598: creat d2/d3/dc/d5d/fd2 x:0 0 0 2024-09-10T04:05:21.901 INFO:tasks.workunit.client.1.smithi070.stdout:2/572: symlink d15/d18/db2/dc3/lc8 0 2024-09-10T04:05:21.902 INFO:tasks.workunit.client.1.smithi070.stdout:7/599: fdatasync d2/d3/d63/f87 0 2024-09-10T04:05:21.904 INFO:tasks.workunit.client.1.smithi070.stdout:8/646: rmdir d4/d16/d73/d96 39 2024-09-10T04:05:21.904 INFO:tasks.workunit.client.1.smithi070.stdout:1/549: dwrite d3/f60 [0,4194304] 0 2024-09-10T04:05:21.905 INFO:tasks.workunit.client.1.smithi070.stdout:7/600: fdatasync d2/d7/d9/d18/d3c/d52/fb0 0 2024-09-10T04:05:21.906 INFO:tasks.workunit.client.1.smithi070.stdout:2/573: creat d15/d18/d29/d2a/d2e/d32/fc9 x:0 0 0 2024-09-10T04:05:21.907 INFO:tasks.workunit.client.1.smithi070.stdout:4/535: truncate de/d13/d43/f8d 926073 0 2024-09-10T04:05:21.909 INFO:tasks.workunit.client.1.smithi070.stdout:8/647: creat d4/d16/d17/dab/fdc x:0 0 0 2024-09-10T04:05:21.912 INFO:tasks.workunit.client.1.smithi070.stdout:7/601: write d2/d7/d9/d18/f3f [344727,103514] 0 2024-09-10T04:05:21.914 INFO:tasks.workunit.client.1.smithi070.stdout:4/536: creat de/d13/d5b/da0/fb4 x:0 0 0 2024-09-10T04:05:21.917 INFO:tasks.workunit.client.1.smithi070.stdout:8/648: mknod d4/d16/d7d/cdd 0 2024-09-10T04:05:21.920 INFO:tasks.workunit.client.1.smithi070.stdout:7/602: chown d2/d7/db/d3a/d7e/c83 238935 1 2024-09-10T04:05:21.921 INFO:tasks.workunit.client.1.smithi070.stdout:4/537: mkdir de/d13/d43/d88/db5 0 2024-09-10T04:05:21.924 INFO:tasks.workunit.client.1.smithi070.stdout:2/574: mkdir d15/d18/d29/d2a/d2e/da6/dc0/dca 0 2024-09-10T04:05:21.928 INFO:tasks.workunit.client.1.smithi070.stdout:8/649: dread d4/d16/d17/d19/d28/f46 [0,4194304] 0 2024-09-10T04:05:21.933 INFO:tasks.workunit.client.1.smithi070.stdout:7/603: rename d2/d3/dc/d5d/d42/l75 to d2/d3/dc/d5d/d41/da8/ld3 0 2024-09-10T04:05:21.935 INFO:tasks.workunit.client.1.smithi070.stdout:1/550: write d3/d10/f2c [2950064,49848] 0 2024-09-10T04:05:21.936 INFO:tasks.workunit.client.1.smithi070.stdout:8/650: mknod d4/d16/d17/d6f/cde 0 2024-09-10T04:05:21.940 INFO:tasks.workunit.client.1.smithi070.stdout:2/575: rename d15/d18/d90 to d15/d18/d29/d2a/d2e/da6/dc0/dcb 0 2024-09-10T04:05:21.944 INFO:tasks.workunit.client.1.smithi070.stdout:8/651: write d4/dbf/d74/da4/fbd [908570,126049] 0 2024-09-10T04:05:21.949 INFO:tasks.workunit.client.1.smithi070.stdout:4/538: getdents de/d13/d18 0 2024-09-10T04:05:21.949 INFO:tasks.workunit.client.1.smithi070.stdout:8/652: truncate d4/d16/d17/d19/d28/d43/d67/d97/f82 924161 0 2024-09-10T04:05:21.951 INFO:tasks.workunit.client.1.smithi070.stdout:1/551: dread d3/d8/d13/f25 [0,4194304] 0 2024-09-10T04:05:21.951 INFO:tasks.workunit.client.1.smithi070.stdout:8/653: fdatasync d4/d16/d17/d19/d28/d43/d52/fc9 0 2024-09-10T04:05:21.952 INFO:tasks.workunit.client.1.smithi070.stdout:1/552: write d3/d8/f49 [4881136,123626] 0 2024-09-10T04:05:21.953 INFO:tasks.workunit.client.1.smithi070.stdout:8/654: dread - d4/fb7 zero size 2024-09-10T04:05:21.955 INFO:tasks.workunit.client.1.smithi070.stdout:7/604: truncate d2/d3/dc/d5d/f46 3494597 0 2024-09-10T04:05:21.958 INFO:tasks.workunit.client.1.smithi070.stdout:2/576: dread d15/d18/d29/d2a/d2e/d32/f33 [0,4194304] 0 2024-09-10T04:05:21.958 INFO:tasks.workunit.client.1.smithi070.stdout:7/605: fsync d2/d7/d9/d18/fcc 0 2024-09-10T04:05:21.959 INFO:tasks.workunit.client.1.smithi070.stdout:4/539: dwrite de/d13/d5b/da0/fb4 [0,4194304] 0 2024-09-10T04:05:21.962 INFO:tasks.workunit.client.1.smithi070.stdout:8/655: rename d4/d16/d7d/fd7 to d4/d16/d17/d1a/fdf 0 2024-09-10T04:05:21.965 INFO:tasks.workunit.client.1.smithi070.stdout:4/540: fsync de/d13/d18/d26/f2f 0 2024-09-10T04:05:21.966 INFO:tasks.workunit.client.1.smithi070.stdout:8/656: symlink d4/d16/d17/d6f/db5/le0 0 2024-09-10T04:05:21.967 INFO:tasks.workunit.client.1.smithi070.stdout:4/541: read - de/f9b zero size 2024-09-10T04:05:21.967 INFO:tasks.workunit.client.1.smithi070.stdout:1/553: rename d3/d8/le to d3/d8/d12/d2f/d32/d4b/lb3 0 2024-09-10T04:05:21.968 INFO:tasks.workunit.client.1.smithi070.stdout:4/542: chown de/d13/d18 12 1 2024-09-10T04:05:21.968 INFO:tasks.workunit.client.1.smithi070.stdout:8/657: dread - d4/d16/f34 zero size 2024-09-10T04:05:21.969 INFO:tasks.workunit.client.1.smithi070.stdout:2/577: dwrite d15/d18/d29/d64/f95 [0,4194304] 0 2024-09-10T04:05:21.971 INFO:tasks.workunit.client.1.smithi070.stdout:1/554: rmdir d3/d8/d13 39 2024-09-10T04:05:21.973 INFO:tasks.workunit.client.1.smithi070.stdout:8/658: rmdir d4/d16/d17/d6f 39 2024-09-10T04:05:21.973 INFO:tasks.workunit.client.1.smithi070.stdout:7/606: dwrite d2/d7/db/d3a/d7e/f93 [0,4194304] 0 2024-09-10T04:05:21.976 INFO:tasks.workunit.client.1.smithi070.stdout:1/555: mkdir d3/d8/d27/d7a/db4 0 2024-09-10T04:05:21.977 INFO:tasks.workunit.client.1.smithi070.stdout:9/658: sync 2024-09-10T04:05:21.977 INFO:tasks.workunit.client.1.smithi070.stdout:0/694: sync 2024-09-10T04:05:21.978 INFO:tasks.workunit.client.0.smithi046.stdout:1/432: sync 2024-09-10T04:05:21.978 INFO:tasks.workunit.client.0.smithi046.stdout:6/537: sync 2024-09-10T04:05:21.978 INFO:tasks.workunit.client.0.smithi046.stdout:0/411: sync 2024-09-10T04:05:21.978 INFO:tasks.workunit.client.0.smithi046.stdout:4/700: sync 2024-09-10T04:05:21.978 INFO:tasks.workunit.client.0.smithi046.stdout:8/503: sync 2024-09-10T04:05:21.978 INFO:tasks.workunit.client.0.smithi046.stdout:5/642: sync 2024-09-10T04:05:21.978 INFO:tasks.workunit.client.0.smithi046.stdout:2/539: sync 2024-09-10T04:05:21.979 INFO:tasks.workunit.client.0.smithi046.stdout:3/550: sync 2024-09-10T04:05:21.979 INFO:tasks.workunit.client.1.smithi070.stdout:0/695: stat d8/d24/d36/d87/ld7 0 2024-09-10T04:05:21.983 INFO:tasks.workunit.client.1.smithi070.stdout:2/578: mknod d15/d18/d29/d2a/d2e/d32/d9b/ccc 0 2024-09-10T04:05:21.985 INFO:tasks.workunit.client.1.smithi070.stdout:4/543: write de/d13/d18/f37 [2322164,10737] 0 2024-09-10T04:05:21.989 INFO:tasks.workunit.client.0.smithi046.stdout:2/540: dwrite d2/d6/f61 [0,4194304] 0 2024-09-10T04:05:21.990 INFO:tasks.workunit.client.0.smithi046.stdout:0/412: creat d1/d13/d37/d75/f83 x:0 0 0 2024-09-10T04:05:21.990 INFO:tasks.workunit.client.1.smithi070.stdout:0/696: rmdir d8/d13/de2/d53 39 2024-09-10T04:05:21.991 INFO:tasks.workunit.client.0.smithi046.stdout:0/413: readlink d1/dc/d25/d33/d4/d19/l5f 0 2024-09-10T04:05:21.992 INFO:tasks.workunit.client.1.smithi070.stdout:7/607: dread d2/d3/dc/d5d/f1d [0,4194304] 0 2024-09-10T04:05:21.994 INFO:tasks.workunit.client.0.smithi046.stdout:8/504: creat d1/d29/d17/d1f/d46/d76/fab x:0 0 0 2024-09-10T04:05:21.996 INFO:tasks.workunit.client.1.smithi070.stdout:8/659: dread d4/d16/d17/d1a/f88 [0,4194304] 0 2024-09-10T04:05:21.996 INFO:tasks.workunit.client.0.smithi046.stdout:6/538: mkdir da/db/d11/d13/d1e/d23/d54/dad 0 2024-09-10T04:05:21.997 INFO:tasks.workunit.client.1.smithi070.stdout:2/579: rmdir d15/d18/d29/d2a/d2e 39 2024-09-10T04:05:22.001 INFO:tasks.workunit.client.0.smithi046.stdout:5/643: rename d7/d11/d3f/d53/da9/d9c to d7/d11/d14/d2f/deb 0 2024-09-10T04:05:22.002 INFO:tasks.workunit.client.1.smithi070.stdout:8/660: dwrite d4/d16/d17/d1a/d60/f9c [0,4194304] 0 2024-09-10T04:05:22.002 INFO:tasks.workunit.client.0.smithi046.stdout:3/551: rmdir d12/d13/d37/d51/db5/d4b/db7 39 2024-09-10T04:05:22.003 INFO:tasks.workunit.client.1.smithi070.stdout:9/659: creat d9/d7a/d74/d8c/fd4 x:0 0 0 2024-09-10T04:05:22.004 INFO:tasks.workunit.client.0.smithi046.stdout:1/433: dwrite d3/d27/f41 [0,4194304] 0 2024-09-10T04:05:22.005 INFO:tasks.workunit.client.0.smithi046.stdout:9/517: sync 2024-09-10T04:05:22.005 INFO:tasks.workunit.client.0.smithi046.stdout:7/554: sync 2024-09-10T04:05:22.006 INFO:tasks.workunit.client.1.smithi070.stdout:0/697: rmdir d8/d24/d2d/d5f 39 2024-09-10T04:05:22.006 INFO:tasks.workunit.client.1.smithi070.stdout:2/580: read f10 [1015113,3868] 0 2024-09-10T04:05:22.008 INFO:tasks.workunit.client.0.smithi046.stdout:4/701: symlink d3/d12/d17/d29/d58/db2/d5f/d85/da8/lf4 0 2024-09-10T04:05:22.010 INFO:tasks.workunit.client.0.smithi046.stdout:2/541: creat d2/d8/d65/fc7 x:0 0 0 2024-09-10T04:05:22.012 INFO:tasks.workunit.client.0.smithi046.stdout:0/414: write d1/dc/d25/d33/d3/d29/d2d/f32 [4121177,96919] 0 2024-09-10T04:05:22.014 INFO:tasks.workunit.client.0.smithi046.stdout:0/415: truncate d1/d13/d37/d75/f7f 973385 0 2024-09-10T04:05:22.016 INFO:tasks.workunit.client.1.smithi070.stdout:4/544: mkdir de/d13/d18/d5c/db6 0 2024-09-10T04:05:22.016 INFO:tasks.workunit.client.0.smithi046.stdout:5/644: creat d7/d11/d14/d2f/deb/da5/dcc/fec x:0 0 0 2024-09-10T04:05:22.018 INFO:tasks.workunit.client.0.smithi046.stdout:3/552: creat d12/d5d/d71/d49/d5e/fbb x:0 0 0 2024-09-10T04:05:22.019 INFO:tasks.workunit.client.0.smithi046.stdout:3/553: chown d12/d13/d14/d1e/d2d/d39/d67 70158309 1 2024-09-10T04:05:22.019 INFO:tasks.workunit.client.0.smithi046.stdout:7/555: creat d5/d6/fb5 x:0 0 0 2024-09-10T04:05:22.020 INFO:tasks.workunit.client.0.smithi046.stdout:3/554: chown d12/d5d/d71/c98 14839530 1 2024-09-10T04:05:22.022 INFO:tasks.workunit.client.0.smithi046.stdout:7/556: truncate d5/d6/d55/d89/fa8 587874 0 2024-09-10T04:05:22.022 INFO:tasks.workunit.client.1.smithi070.stdout:7/608: unlink d2/d3/dc/d5d/d41/da8/ld3 0 2024-09-10T04:05:22.022 INFO:tasks.workunit.client.0.smithi046.stdout:3/555: write d12/d13/d37/d51/db5/d4b/fb6 [574252,124315] 0 2024-09-10T04:05:22.024 INFO:tasks.workunit.client.1.smithi070.stdout:4/545: dwrite de/d13/d18/d26/d8c/f98 [0,4194304] 0 2024-09-10T04:05:22.024 INFO:tasks.workunit.client.0.smithi046.stdout:1/434: mknod d3/dc/d21/d3a/c8c 0 2024-09-10T04:05:22.028 INFO:tasks.workunit.client.1.smithi070.stdout:7/609: dread d2/d7/d9/d18/d23/f91 [0,4194304] 0 2024-09-10T04:05:22.030 INFO:tasks.workunit.client.0.smithi046.stdout:4/702: rmdir d3/d12/d17 39 2024-09-10T04:05:22.033 INFO:tasks.workunit.client.0.smithi046.stdout:2/542: mkdir d2/d6/d23/d42/dc8 0 2024-09-10T04:05:22.035 INFO:tasks.workunit.client.0.smithi046.stdout:2/543: dread d2/d49/f9b [0,4194304] 0 2024-09-10T04:05:22.035 INFO:tasks.workunit.client.1.smithi070.stdout:7/610: dwrite d2/d7/d9/d18/d3c/f73 [8388608,4194304] 0 2024-09-10T04:05:22.035 INFO:tasks.workunit.client.0.smithi046.stdout:0/416: dread - d1/dc/d25/d33/f52 zero size 2024-09-10T04:05:22.036 INFO:tasks.workunit.client.1.smithi070.stdout:7/611: chown d2/d3/dc/d5d/d41 1237978380 1 2024-09-10T04:05:22.036 INFO:tasks.workunit.client.0.smithi046.stdout:2/544: chown d2/d6/d23/d5d/d77 457466 1 2024-09-10T04:05:22.037 INFO:tasks.workunit.client.1.smithi070.stdout:9/660: mkdir d9/d7a/d6f/dd5 0 2024-09-10T04:05:22.038 INFO:tasks.workunit.client.1.smithi070.stdout:7/612: write d2/d7/d9/f82 [4940729,14236] 0 2024-09-10T04:05:22.038 INFO:tasks.workunit.client.0.smithi046.stdout:0/417: truncate d1/dc/d25/d33/d1a/d64/f7c 168373 0 2024-09-10T04:05:22.043 INFO:tasks.workunit.client.0.smithi046.stdout:9/518: write d4/d3c/d4d/d5c/f6a [937695,15596] 0 2024-09-10T04:05:22.043 INFO:tasks.workunit.client.0.smithi046.stdout:3/556: rename d12/d13/d37/dab/fb2 to d12/d5d/d71/d49/d5e/fbc 0 2024-09-10T04:05:22.046 INFO:tasks.workunit.client.1.smithi070.stdout:0/698: dwrite d8/d24/d36/f48 [0,4194304] 0 2024-09-10T04:05:22.048 INFO:tasks.workunit.client.1.smithi070.stdout:1/556: getdents d3/d8/d13 0 2024-09-10T04:05:22.049 INFO:tasks.workunit.client.0.smithi046.stdout:0/418: creat d1/dc/d25/d33/d1a/d41/f84 x:0 0 0 2024-09-10T04:05:22.050 INFO:tasks.workunit.client.0.smithi046.stdout:0/419: truncate d1/dc/f7b 482801 0 2024-09-10T04:05:22.053 INFO:tasks.workunit.client.1.smithi070.stdout:1/557: chown d3/d10/d1d/l87 51466670 1 2024-09-10T04:05:22.054 INFO:tasks.workunit.client.1.smithi070.stdout:9/661: creat d9/d1c/dac/d8a/d3c/fd6 x:0 0 0 2024-09-10T04:05:22.054 INFO:tasks.workunit.client.1.smithi070.stdout:4/546: mkdir de/d13/d18/d26/d46/daf/db7 0 2024-09-10T04:05:22.055 INFO:tasks.workunit.client.1.smithi070.stdout:2/581: creat d15/d18/d29/fcd x:0 0 0 2024-09-10T04:05:22.056 INFO:tasks.workunit.client.1.smithi070.stdout:4/547: chown de/d13/d18/d23/d47/d59/d79 42 1 2024-09-10T04:05:22.056 INFO:tasks.workunit.client.0.smithi046.stdout:3/557: fdatasync d12/d13/d37/d51/db5/d50/d52/f65 0 2024-09-10T04:05:22.060 INFO:tasks.workunit.client.1.smithi070.stdout:4/548: fsync de/d13/d5b/d7b/f8a 0 2024-09-10T04:05:22.060 INFO:tasks.workunit.client.1.smithi070.stdout:0/699: fdatasync d8/d13/f5a 0 2024-09-10T04:05:22.061 INFO:tasks.workunit.client.0.smithi046.stdout:3/558: stat d12/d13/d14/d1e/d2d/d39/d67/l73 0 2024-09-10T04:05:22.061 INFO:tasks.workunit.client.0.smithi046.stdout:8/505: getdents d1/d29/d19/d36/d8d 0 2024-09-10T04:05:22.061 INFO:tasks.workunit.client.1.smithi070.stdout:8/661: link d4/d16/d17/d19/d28/d43/d67/ld5 d4/dcf/le1 0 2024-09-10T04:05:22.065 INFO:tasks.workunit.client.0.smithi046.stdout:1/435: rename d3/f6 to d3/d27/d32/d28/f8d 0 2024-09-10T04:05:22.066 INFO:tasks.workunit.client.1.smithi070.stdout:2/582: dwrite d15/d18/d56/d7c/f9e [0,4194304] 0 2024-09-10T04:05:22.067 INFO:tasks.workunit.client.1.smithi070.stdout:1/558: mknod d3/d10/db2/cb5 0 2024-09-10T04:05:22.067 INFO:tasks.workunit.client.0.smithi046.stdout:4/703: dread d3/d12/d17/f20 [0,4194304] 0 2024-09-10T04:05:22.067 INFO:tasks.workunit.client.0.smithi046.stdout:7/557: truncate d5/d6/d55/d7d/f9b 270744 0 2024-09-10T04:05:22.069 INFO:tasks.workunit.client.0.smithi046.stdout:4/704: read d3/d12/d17/d22/f4c [284062,110617] 0 2024-09-10T04:05:22.072 INFO:tasks.workunit.client.0.smithi046.stdout:9/519: truncate d4/d29/f6d 3884411 0 2024-09-10T04:05:22.074 INFO:tasks.workunit.client.0.smithi046.stdout:9/520: dread - d4/d3c/d4d/d5c/f8d zero size 2024-09-10T04:05:22.077 INFO:tasks.workunit.client.0.smithi046.stdout:7/558: dwrite f4 [4194304,4194304] 0 2024-09-10T04:05:22.081 INFO:tasks.workunit.client.1.smithi070.stdout:8/662: dwrite d4/d16/f27 [0,4194304] 0 2024-09-10T04:05:22.082 INFO:tasks.workunit.client.0.smithi046.stdout:5/645: getdents d7/d11/d3f 0 2024-09-10T04:05:22.088 INFO:tasks.workunit.client.0.smithi046.stdout:0/420: dwrite d1/dc/f27 [0,4194304] 0 2024-09-10T04:05:22.088 INFO:tasks.workunit.client.1.smithi070.stdout:4/549: creat de/d13/d18/d26/d46/d85/fb8 x:0 0 0 2024-09-10T04:05:22.092 INFO:tasks.workunit.client.0.smithi046.stdout:0/421: write d1/dc/f7b [988119,67846] 0 2024-09-10T04:05:22.092 INFO:tasks.workunit.client.1.smithi070.stdout:9/662: rename d9/d7a/d74/l6e to d9/d7a/d6f/ld7 0 2024-09-10T04:05:22.093 INFO:tasks.workunit.client.0.smithi046.stdout:0/422: write d1/dc/f17 [2762525,23304] 0 2024-09-10T04:05:22.094 INFO:tasks.workunit.client.1.smithi070.stdout:9/663: dread - d9/d59/d2a/d3f/daf/fc0 zero size 2024-09-10T04:05:22.095 INFO:tasks.workunit.client.1.smithi070.stdout:9/664: chown d9/d7a/c4d 1159789 1 2024-09-10T04:05:22.096 INFO:tasks.workunit.client.0.smithi046.stdout:2/545: rename d2/d6/d23/l2b to d2/d6/d9d/db6/dbe/lc9 0 2024-09-10T04:05:22.098 INFO:tasks.workunit.client.0.smithi046.stdout:2/546: chown d2/d8/d65/f7d 599947 1 2024-09-10T04:05:22.099 INFO:tasks.workunit.client.0.smithi046.stdout:2/547: chown d2/d6/d23/d5d/d77/l6e 62384580 1 2024-09-10T04:05:22.102 INFO:tasks.workunit.client.1.smithi070.stdout:4/550: creat de/d13/d5b/d7b/fb9 x:0 0 0 2024-09-10T04:05:22.103 INFO:tasks.workunit.client.1.smithi070.stdout:0/700: rename d8/d13/d1a/f35 to d8/d24/d2d/d8d/fe5 0 2024-09-10T04:05:22.104 INFO:tasks.workunit.client.0.smithi046.stdout:9/521: creat d4/d18/d89/fa2 x:0 0 0 2024-09-10T04:05:22.105 INFO:tasks.workunit.client.1.smithi070.stdout:1/559: dwrite d3/d10/f73 [0,4194304] 0 2024-09-10T04:05:22.105 INFO:tasks.workunit.client.0.smithi046.stdout:9/522: write d4/d18/d89/fa2 [776003,49423] 0 2024-09-10T04:05:22.106 INFO:tasks.workunit.client.1.smithi070.stdout:9/665: unlink d9/d1c/d34/d63/ca1 0 2024-09-10T04:05:22.110 INFO:tasks.workunit.client.1.smithi070.stdout:8/663: link d4/d16/d17/d19/d28/d43/d67/fc3 d4/d16/d17/d1a/d60/d7a/db8/fe2 0 2024-09-10T04:05:22.110 INFO:tasks.workunit.client.1.smithi070.stdout:8/664: truncate d4/fcd 815137 0 2024-09-10T04:05:22.116 INFO:tasks.workunit.client.1.smithi070.stdout:4/551: creat de/d13/d18/d26/d8c/fba x:0 0 0 2024-09-10T04:05:22.122 INFO:tasks.workunit.client.0.smithi046.stdout:1/436: chown d3/d27/d32/d1f/f55 31601 1 2024-09-10T04:05:22.122 INFO:tasks.workunit.client.0.smithi046.stdout:3/559: truncate d12/d13/d37/d51/f62 1057056 0 2024-09-10T04:05:22.124 INFO:tasks.workunit.client.1.smithi070.stdout:1/560: rename d3/d20/l4f to d3/d8/d12/d2f/d32/da8/lb6 0 2024-09-10T04:05:22.125 INFO:tasks.workunit.client.1.smithi070.stdout:0/701: dwrite d8/d24/d2d/fc4 [0,4194304] 0 2024-09-10T04:05:22.125 INFO:tasks.workunit.client.1.smithi070.stdout:2/583: link d15/d18/d29/d64/f81 d15/d18/d29/d2a/d2e/da6/dc0/fce 0 2024-09-10T04:05:22.126 INFO:tasks.workunit.client.1.smithi070.stdout:1/561: fdatasync d3/d10/f72 0 2024-09-10T04:05:22.133 INFO:tasks.workunit.client.0.smithi046.stdout:7/559: rename d5/d6/d39/d5a to d5/d6/d32/dab/db6 0 2024-09-10T04:05:22.134 INFO:tasks.workunit.client.0.smithi046.stdout:7/560: chown d5/d6/db/dd/d24 495154736 1 2024-09-10T04:05:22.134 INFO:tasks.workunit.client.1.smithi070.stdout:2/584: dwrite d15/d18/d29/d2a/d2e/d32/f82 [0,4194304] 0 2024-09-10T04:05:22.135 INFO:tasks.workunit.client.0.smithi046.stdout:7/561: chown d5/d6/d32/dab/fb1 1853018076 1 2024-09-10T04:05:22.135 INFO:tasks.workunit.client.1.smithi070.stdout:2/585: readlink d15/d18/l2c 0 2024-09-10T04:05:22.145 INFO:tasks.workunit.client.0.smithi046.stdout:6/539: sync 2024-09-10T04:05:22.147 INFO:tasks.workunit.client.1.smithi070.stdout:5/602: sync 2024-09-10T04:05:22.148 INFO:tasks.workunit.client.1.smithi070.stdout:6/645: sync 2024-09-10T04:05:22.148 INFO:tasks.workunit.client.1.smithi070.stdout:3/638: sync 2024-09-10T04:05:22.151 INFO:tasks.workunit.client.1.smithi070.stdout:4/552: creat de/d13/d5b/d9e/fbb x:0 0 0 2024-09-10T04:05:22.153 INFO:tasks.workunit.client.1.smithi070.stdout:1/562: fsync d3/d8/d12/d2f/f5b 0 2024-09-10T04:05:22.153 INFO:tasks.workunit.client.0.smithi046.stdout:8/506: symlink d1/d29/d19/d36/d4c/lac 0 2024-09-10T04:05:22.155 INFO:tasks.workunit.client.1.smithi070.stdout:1/563: chown d3/d8/d27/d7a/fa7 1067520 1 2024-09-10T04:05:22.155 INFO:tasks.workunit.client.1.smithi070.stdout:4/553: dread de/d13/d18/d26/d46/fa1 [0,4194304] 0 2024-09-10T04:05:22.158 INFO:tasks.workunit.client.1.smithi070.stdout:2/586: dread d15/d2b/d94/f9a [0,4194304] 0 2024-09-10T04:05:22.159 INFO:tasks.workunit.client.1.smithi070.stdout:2/587: dread - d15/d18/d56/d7c/f92 zero size 2024-09-10T04:05:22.163 INFO:tasks.workunit.client.1.smithi070.stdout:9/666: rename d9/d1c/f87 to d9/d59/d5d/fd8 0 2024-09-10T04:05:22.168 INFO:tasks.workunit.client.0.smithi046.stdout:7/562: mkdir d5/d6/db/dd/d24/db7 0 2024-09-10T04:05:22.170 INFO:tasks.workunit.client.0.smithi046.stdout:2/548: mkdir d2/d6/d23/d42/db4/dca 0 2024-09-10T04:05:22.172 INFO:tasks.workunit.client.0.smithi046.stdout:5/646: write d7/d11/d14/d45/d51/f67 [779165,52607] 0 2024-09-10T04:05:22.174 INFO:tasks.workunit.client.0.smithi046.stdout:6/540: mknod da/db/d63/cae 0 2024-09-10T04:05:22.175 INFO:tasks.workunit.client.0.smithi046.stdout:6/541: stat da/db/d10/d27/f88 0 2024-09-10T04:05:22.178 INFO:tasks.workunit.client.0.smithi046.stdout:1/437: symlink d3/d27/d32/d1f/l8e 0 2024-09-10T04:05:22.184 INFO:tasks.workunit.client.0.smithi046.stdout:2/549: symlink d2/d6/d9d/db6/lcb 0 2024-09-10T04:05:22.185 INFO:tasks.workunit.client.0.smithi046.stdout:4/705: getdents d3/d12/dec/df0 0 2024-09-10T04:05:22.189 INFO:tasks.workunit.client.0.smithi046.stdout:5/647: unlink d7/d11/fe9 0 2024-09-10T04:05:22.192 INFO:tasks.workunit.client.0.smithi046.stdout:8/507: creat d1/d29/d17/d1f/d46/d52/d9d/fad x:0 0 0 2024-09-10T04:05:22.194 INFO:tasks.workunit.client.0.smithi046.stdout:4/706: dwrite d3/d12/d17/d22/d51/d94/fd6 [0,4194304] 0 2024-09-10T04:05:22.197 INFO:tasks.workunit.client.0.smithi046.stdout:1/438: unlink d3/d27/l5f 0 2024-09-10T04:05:22.197 INFO:tasks.workunit.client.0.smithi046.stdout:1/439: fdatasync d3/d27/d5d/f6a 0 2024-09-10T04:05:22.198 INFO:tasks.workunit.client.0.smithi046.stdout:0/423: rename d1/d13/d37/d75/l79 to d1/dc/d25/d33/d3/dd/d55/d4d/l85 0 2024-09-10T04:05:22.203 INFO:tasks.workunit.client.0.smithi046.stdout:3/560: truncate d12/d13/d37/d51/f72 2202205 0 2024-09-10T04:05:22.204 INFO:tasks.workunit.client.0.smithi046.stdout:9/523: getdents d4/d18/d89 0 2024-09-10T04:05:22.206 INFO:tasks.workunit.client.0.smithi046.stdout:0/424: dwrite d1/dc/f17 [0,4194304] 0 2024-09-10T04:05:22.208 INFO:tasks.workunit.client.0.smithi046.stdout:5/648: mkdir d7/d11/d3f/d5a/d63/ded 0 2024-09-10T04:05:22.209 INFO:tasks.workunit.client.0.smithi046.stdout:6/542: symlink da/db/d11/d13/d1e/laf 0 2024-09-10T04:05:22.211 INFO:tasks.workunit.client.0.smithi046.stdout:5/649: read d7/d11/d14/f71 [2310860,55170] 0 2024-09-10T04:05:22.216 INFO:tasks.workunit.client.0.smithi046.stdout:7/563: rename d5/d7f/d78 to d5/d6/d32/dab/db8 0 2024-09-10T04:05:22.218 INFO:tasks.workunit.client.1.smithi070.stdout:7/613: sync 2024-09-10T04:05:22.218 INFO:tasks.workunit.client.1.smithi070.stdout:3/639: truncate db/d1c/d25/f87 906807 0 2024-09-10T04:05:22.219 INFO:tasks.workunit.client.1.smithi070.stdout:3/640: chown db/d1c/d53/f6e 0 1 2024-09-10T04:05:22.220 INFO:tasks.workunit.client.1.smithi070.stdout:4/554: write de/d13/d18/d23/d39/d48/fab [1814375,53209] 0 2024-09-10T04:05:22.221 INFO:tasks.workunit.client.0.smithi046.stdout:3/561: mkdir d12/d13/d37/d51/dbd 0 2024-09-10T04:05:22.221 INFO:tasks.workunit.client.1.smithi070.stdout:3/641: write db/f12 [530390,37643] 0 2024-09-10T04:05:22.223 INFO:tasks.workunit.client.0.smithi046.stdout:9/524: truncate d4/d7/d1f/f21 4605383 0 2024-09-10T04:05:22.224 INFO:tasks.workunit.client.0.smithi046.stdout:9/525: fsync d4/d3c/d4d/d5c/f93 0 2024-09-10T04:05:22.227 INFO:tasks.workunit.client.1.smithi070.stdout:2/588: chown d15/d18/d29/d2a/d2e/d32/d36/c80 214 1 2024-09-10T04:05:22.228 INFO:tasks.workunit.client.0.smithi046.stdout:8/508: truncate d1/d29/d19/d36/d4c/d5e/fa1 358429 0 2024-09-10T04:05:22.228 INFO:tasks.workunit.client.1.smithi070.stdout:3/642: dwrite db/d1c/d89/fb4 [0,4194304] 0 2024-09-10T04:05:22.228 INFO:tasks.workunit.client.1.smithi070.stdout:3/643: rename db to db/d79/d94/ddb 22 2024-09-10T04:05:22.229 INFO:tasks.workunit.client.0.smithi046.stdout:2/550: write d2/d6/d23/d5d/d5e/f84 [1980061,27696] 0 2024-09-10T04:05:22.230 INFO:tasks.workunit.client.1.smithi070.stdout:3/644: write db/d1c/d89/fb4 [1210555,129944] 0 2024-09-10T04:05:22.231 INFO:tasks.workunit.client.0.smithi046.stdout:5/650: symlink d7/d11/d3f/d5a/d63/lee 0 2024-09-10T04:05:22.233 INFO:tasks.workunit.client.0.smithi046.stdout:1/440: truncate d3/f4a 455748 0 2024-09-10T04:05:22.236 INFO:tasks.workunit.client.1.smithi070.stdout:6/646: mknod d0/d3/d18/d1b/d17/d3f/ccb 0 2024-09-10T04:05:22.237 INFO:tasks.workunit.client.0.smithi046.stdout:7/564: mkdir d5/d6/db/dd/d24/db9 0 2024-09-10T04:05:22.240 INFO:tasks.workunit.client.0.smithi046.stdout:3/562: chown d12/d13/d37/d51/db5/d50/f55 19 1 2024-09-10T04:05:22.242 INFO:tasks.workunit.client.0.smithi046.stdout:9/526: read d4/d18/f27 [211519,121790] 0 2024-09-10T04:05:22.243 INFO:tasks.workunit.client.1.smithi070.stdout:0/702: truncate d8/d24/d2d/d34/f5b 699794 0 2024-09-10T04:05:22.249 INFO:tasks.workunit.client.0.smithi046.stdout:6/543: mknod da/db/d11/d13/d1e/d23/d86/cb0 0 2024-09-10T04:05:22.250 INFO:tasks.workunit.client.0.smithi046.stdout:2/551: fsync d2/d6/d9d/fa5 0 2024-09-10T04:05:22.253 INFO:tasks.workunit.client.0.smithi046.stdout:5/651: mknod d7/d11/d14/d2f/deb/da5/dcc/cef 0 2024-09-10T04:05:22.254 INFO:tasks.workunit.client.1.smithi070.stdout:5/603: write d17/d24/d45/f76 [2092261,41527] 0 2024-09-10T04:05:22.254 INFO:tasks.workunit.client.0.smithi046.stdout:1/441: mknod d3/d27/d2a/c8f 0 2024-09-10T04:05:22.255 INFO:tasks.workunit.client.0.smithi046.stdout:1/442: fsync d3/d27/f20 0 2024-09-10T04:05:22.260 INFO:tasks.workunit.client.0.smithi046.stdout:5/652: dwrite d7/d11/d14/fc3 [0,4194304] 0 2024-09-10T04:05:22.264 INFO:tasks.workunit.client.0.smithi046.stdout:5/653: fdatasync d7/f43 0 2024-09-10T04:05:22.264 INFO:tasks.workunit.client.0.smithi046.stdout:3/563: mknod d12/d13/d37/d51/cbe 0 2024-09-10T04:05:22.265 INFO:tasks.workunit.client.1.smithi070.stdout:9/667: dread d9/d1c/dac/d8a/f3d [0,4194304] 0 2024-09-10T04:05:22.265 INFO:tasks.workunit.client.1.smithi070.stdout:3/645: rename db/d1c/d25/d28/d43/f58 to db/d1c/d3e/fdc 0 2024-09-10T04:05:22.268 INFO:tasks.workunit.client.1.smithi070.stdout:6/647: symlink d0/d3/d18/d62/d70/lcc 0 2024-09-10T04:05:22.269 INFO:tasks.workunit.client.1.smithi070.stdout:6/648: fsync d0/f57 0 2024-09-10T04:05:22.270 INFO:tasks.workunit.client.1.smithi070.stdout:1/564: truncate d3/d8/d13/d3f/f52 3414992 0 2024-09-10T04:05:22.272 INFO:tasks.workunit.client.0.smithi046.stdout:8/509: symlink d1/d29/d19/lae 0 2024-09-10T04:05:22.274 INFO:tasks.workunit.client.1.smithi070.stdout:0/703: truncate d8/d24/f6d 1034277 0 2024-09-10T04:05:22.277 INFO:tasks.workunit.client.0.smithi046.stdout:2/552: mknod d2/d6/d23/d42/db4/ccc 0 2024-09-10T04:05:22.277 INFO:tasks.workunit.client.0.smithi046.stdout:2/553: chown d2/d57 1135 1 2024-09-10T04:05:22.282 INFO:tasks.workunit.client.1.smithi070.stdout:1/565: dread f0 [0,4194304] 0 2024-09-10T04:05:22.283 INFO:tasks.workunit.client.1.smithi070.stdout:5/604: unlink d17/d24/d45/dc0/f95 0 2024-09-10T04:05:22.283 INFO:tasks.workunit.client.1.smithi070.stdout:1/566: write d3/d8/d12/f24 [2494614,38823] 0 2024-09-10T04:05:22.285 INFO:tasks.workunit.client.1.smithi070.stdout:8/665: sync 2024-09-10T04:05:22.285 INFO:tasks.workunit.client.0.smithi046.stdout:4/707: rename d3/d12/d17/d22/d51/d94/fbc to d3/d12/d17/ff5 0 2024-09-10T04:05:22.286 INFO:tasks.workunit.client.1.smithi070.stdout:7/614: truncate d2/d7/d9/d18/d3c/f73 12249294 0 2024-09-10T04:05:22.287 INFO:tasks.workunit.client.0.smithi046.stdout:4/708: chown d3/d12/d17/d29/d58/db2/d5f/d85/cb0 29927 1 2024-09-10T04:05:22.288 INFO:tasks.workunit.client.0.smithi046.stdout:1/443: rmdir d3/d27/d5d 39 2024-09-10T04:05:22.288 INFO:tasks.workunit.client.0.smithi046.stdout:4/709: chown d3/fc 7911 1 2024-09-10T04:05:22.292 INFO:tasks.workunit.client.0.smithi046.stdout:7/565: mknod d5/d6/db/d11/d1e/cba 0 2024-09-10T04:05:22.294 INFO:tasks.workunit.client.0.smithi046.stdout:7/566: write d5/d9/f87 [3291984,80820] 0 2024-09-10T04:05:22.295 INFO:tasks.workunit.client.0.smithi046.stdout:4/710: dwrite d3/d12/d17/d29/d58/f71 [0,4194304] 0 2024-09-10T04:05:22.297 INFO:tasks.workunit.client.1.smithi070.stdout:6/649: rename d0/d3/d18/d1b/d17/d28/l41 to d0/d3/d18/d1b/d17/d28/lcd 0 2024-09-10T04:05:22.297 INFO:tasks.workunit.client.1.smithi070.stdout:1/567: dread d3/d8/d13/f43 [0,4194304] 0 2024-09-10T04:05:22.303 INFO:tasks.workunit.client.0.smithi046.stdout:3/564: mkdir d12/d13/d37/d94/dbf 0 2024-09-10T04:05:22.316 INFO:tasks.workunit.client.0.smithi046.stdout:7/567: dread f4 [0,4194304] 0 2024-09-10T04:05:22.317 INFO:tasks.workunit.client.0.smithi046.stdout:5/654: dread d7/d11/d14/f1d [0,4194304] 0 2024-09-10T04:05:22.319 INFO:tasks.workunit.client.0.smithi046.stdout:0/425: rename d1/dc/d25/d33/d3/dd/d55/c57 to d1/dc/d25/d33/d1a/d64/c86 0 2024-09-10T04:05:22.325 INFO:tasks.workunit.client.0.smithi046.stdout:2/554: write d2/d8/d37/f6c [499841,67107] 0 2024-09-10T04:05:22.325 INFO:tasks.workunit.client.0.smithi046.stdout:8/510: dwrite d1/d29/d17/f23 [0,4194304] 0 2024-09-10T04:05:22.327 INFO:tasks.workunit.client.0.smithi046.stdout:2/555: stat d2/d6/d23/d55/fa3 0 2024-09-10T04:05:22.331 INFO:tasks.workunit.client.1.smithi070.stdout:7/615: write d2/f55 [536708,129394] 0 2024-09-10T04:05:22.332 INFO:tasks.workunit.client.0.smithi046.stdout:4/711: symlink d3/d12/d17/d22/d51/d5c/lf6 0 2024-09-10T04:05:22.333 INFO:tasks.workunit.client.1.smithi070.stdout:7/616: write d2/d7/db/d3a/d7e/f93 [703691,128706] 0 2024-09-10T04:05:22.336 INFO:tasks.workunit.client.1.smithi070.stdout:6/650: creat d0/d3/d8c/d9a/d92/fce x:0 0 0 2024-09-10T04:05:22.338 INFO:tasks.workunit.client.1.smithi070.stdout:1/568: unlink d3/d8/d12/f66 0 2024-09-10T04:05:22.340 INFO:tasks.workunit.client.1.smithi070.stdout:0/704: fdatasync d8/d24/d2d/d8d/fe5 0 2024-09-10T04:05:22.341 INFO:tasks.workunit.client.1.smithi070.stdout:4/555: getdents de/d13/d18/d26/d46/daf 0 2024-09-10T04:05:22.341 INFO:tasks.workunit.client.1.smithi070.stdout:0/705: chown d8/d24/l37 58046 1 2024-09-10T04:05:22.343 INFO:tasks.workunit.client.1.smithi070.stdout:2/589: getdents d15/d59 0 2024-09-10T04:05:22.344 INFO:tasks.workunit.client.0.smithi046.stdout:5/655: mkdir d7/d11/d14/d2f/df0 0 2024-09-10T04:05:22.345 INFO:tasks.workunit.client.0.smithi046.stdout:5/656: read d7/f10 [2632070,70271] 0 2024-09-10T04:05:22.345 INFO:tasks.workunit.client.0.smithi046.stdout:3/565: dwrite d12/d13/d14/f15 [0,4194304] 0 2024-09-10T04:05:22.346 INFO:tasks.workunit.client.0.smithi046.stdout:3/566: fsync d12/d13/d14/f33 0 2024-09-10T04:05:22.349 INFO:tasks.workunit.client.1.smithi070.stdout:5/605: mkdir d17/d19/d6a/d54/d57/d63/dcd 0 2024-09-10T04:05:22.350 INFO:tasks.workunit.client.1.smithi070.stdout:5/606: chown d17/d24/d84/la0 335 1 2024-09-10T04:05:22.352 INFO:tasks.workunit.client.0.smithi046.stdout:1/444: mkdir d3/dc/d21/d3a/d90 0 2024-09-10T04:05:22.354 INFO:tasks.workunit.client.0.smithi046.stdout:2/556: symlink d2/d6/d23/d5d/d77/d7b/lcd 0 2024-09-10T04:05:22.354 INFO:tasks.workunit.client.1.smithi070.stdout:3/646: link db/d83/lb9 db/d79/dd4/ldd 0 2024-09-10T04:05:22.358 INFO:tasks.workunit.client.1.smithi070.stdout:3/647: read db/d1c/d3e/f54 [483164,9898] 0 2024-09-10T04:05:22.358 INFO:tasks.workunit.client.1.smithi070.stdout:7/617: symlink d2/d7/d9/d18/d23/ld4 0 2024-09-10T04:05:22.359 INFO:tasks.workunit.client.0.smithi046.stdout:9/527: getdents d4/d3c/d4d/d80 0 2024-09-10T04:05:22.360 INFO:tasks.workunit.client.1.smithi070.stdout:3/648: chown db/d1c/d25/l81 23 1 2024-09-10T04:05:22.360 INFO:tasks.workunit.client.0.smithi046.stdout:8/511: dwrite d1/d29/d17/f8e [0,4194304] 0 2024-09-10T04:05:22.361 INFO:tasks.workunit.client.1.smithi070.stdout:3/649: fsync db/d79/fa9 0 2024-09-10T04:05:22.361 INFO:tasks.workunit.client.0.smithi046.stdout:6/544: link da/db/d10/c15 da/cb1 0 2024-09-10T04:05:22.362 INFO:tasks.workunit.client.0.smithi046.stdout:8/512: write d1/d29/d17/d1f/d46/f8b [614266,63084] 0 2024-09-10T04:05:22.363 INFO:tasks.workunit.client.1.smithi070.stdout:7/618: dread d2/d3/dc/d5d/f1d [0,4194304] 0 2024-09-10T04:05:22.371 INFO:tasks.workunit.client.1.smithi070.stdout:0/706: write d8/f49 [861049,87031] 0 2024-09-10T04:05:22.372 INFO:tasks.workunit.client.1.smithi070.stdout:4/556: mkdir de/d13/d18/d23/d47/d59/d79/dbc 0 2024-09-10T04:05:22.375 INFO:tasks.workunit.client.0.smithi046.stdout:5/657: creat d7/d11/d14/d2f/ff1 x:0 0 0 2024-09-10T04:05:22.377 INFO:tasks.workunit.client.0.smithi046.stdout:4/712: dwrite d3/d12/d17/d29/f99 [0,4194304] 0 2024-09-10T04:05:22.380 INFO:tasks.workunit.client.1.smithi070.stdout:2/590: mkdir d15/d2b/d6e/db8/dcf 0 2024-09-10T04:05:22.380 INFO:tasks.workunit.client.0.smithi046.stdout:7/568: write d5/d9/f42 [1224422,81171] 0 2024-09-10T04:05:22.380 INFO:tasks.workunit.client.1.smithi070.stdout:5/607: mkdir d17/d37/dce 0 2024-09-10T04:05:22.381 INFO:tasks.workunit.client.1.smithi070.stdout:5/608: chown d17/d19/d35/l46 303 1 2024-09-10T04:05:22.382 INFO:tasks.workunit.client.0.smithi046.stdout:9/528: dread d4/d3c/f6b [0,4194304] 0 2024-09-10T04:05:22.383 INFO:tasks.workunit.client.0.smithi046.stdout:4/713: truncate d3/d12/d14/fa6 1169818 0 2024-09-10T04:05:22.383 INFO:tasks.workunit.client.0.smithi046.stdout:3/567: fsync d12/d5d/d71/d49/d5e/f97 0 2024-09-10T04:05:22.383 INFO:tasks.workunit.client.1.smithi070.stdout:5/609: chown d17/d6f/l3e 7625 1 2024-09-10T04:05:22.384 INFO:tasks.workunit.client.1.smithi070.stdout:5/610: dread - d17/d37/f73 zero size 2024-09-10T04:05:22.385 INFO:tasks.workunit.client.1.smithi070.stdout:9/668: link d9/d1c/dac/d8a/d3c/d62/f97 d9/d1c/d34/fd9 0 2024-09-10T04:05:22.386 INFO:tasks.workunit.client.1.smithi070.stdout:9/669: fsync d9/fb7 0 2024-09-10T04:05:22.387 INFO:tasks.workunit.client.0.smithi046.stdout:8/513: dread d1/d29/fe [4194304,4194304] 0 2024-09-10T04:05:22.387 INFO:tasks.workunit.client.0.smithi046.stdout:0/426: truncate d1/d13/d37/f61 4508974 0 2024-09-10T04:05:22.388 INFO:tasks.workunit.client.1.smithi070.stdout:2/591: dwrite d15/d18/d29/d2a/fc4 [0,4194304] 0 2024-09-10T04:05:22.388 INFO:tasks.workunit.client.0.smithi046.stdout:1/445: creat d3/dc/d21/d3a/f91 x:0 0 0 2024-09-10T04:05:22.390 INFO:tasks.workunit.client.1.smithi070.stdout:6/651: mknod d0/d3/d1f/d81/ccf 0 2024-09-10T04:05:22.391 INFO:tasks.workunit.client.0.smithi046.stdout:1/446: read d3/d27/d2a/f8b [1283487,19133] 0 2024-09-10T04:05:22.392 INFO:tasks.workunit.client.1.smithi070.stdout:6/652: fsync d0/d3/d18/d1b/d17/d28/f95 0 2024-09-10T04:05:22.392 INFO:tasks.workunit.client.1.smithi070.stdout:6/653: fdatasync d0/d3/d18/f4c 0 2024-09-10T04:05:22.395 INFO:tasks.workunit.client.1.smithi070.stdout:1/569: getdents d3/d10/d1d/d79 0 2024-09-10T04:05:22.397 INFO:tasks.workunit.client.1.smithi070.stdout:0/707: creat d8/d24/d2d/d8d/fe6 x:0 0 0 2024-09-10T04:05:22.398 INFO:tasks.workunit.client.1.smithi070.stdout:4/557: mknod de/d13/d18/d23/d47/d59/d79/cbd 0 2024-09-10T04:05:22.400 INFO:tasks.workunit.client.1.smithi070.stdout:0/708: fdatasync d8/d13/de2/d7d/d8a/fbc 0 2024-09-10T04:05:22.402 INFO:tasks.workunit.client.0.smithi046.stdout:5/658: fsync d7/d11/d14/d2f/d2b/f7a 0 2024-09-10T04:05:22.402 INFO:tasks.workunit.client.1.smithi070.stdout:5/611: creat d17/d37/fcf x:0 0 0 2024-09-10T04:05:22.403 INFO:tasks.workunit.client.1.smithi070.stdout:9/670: chown d9/d1c/dac/d8a/caa 186 1 2024-09-10T04:05:22.406 INFO:tasks.workunit.client.1.smithi070.stdout:2/592: creat d15/d2b/d6e/fd0 x:0 0 0 2024-09-10T04:05:22.409 INFO:tasks.workunit.client.0.smithi046.stdout:9/529: creat d4/d7/d61/fa3 x:0 0 0 2024-09-10T04:05:22.412 INFO:tasks.workunit.client.1.smithi070.stdout:7/619: mkdir d2/d7/d9/dd5 0 2024-09-10T04:05:22.413 INFO:tasks.workunit.client.0.smithi046.stdout:4/714: mknod d3/d12/d17/d22/d51/d5c/d6c/cf7 0 2024-09-10T04:05:22.415 INFO:tasks.workunit.client.1.smithi070.stdout:6/654: creat d0/d3/d18/d62/db8/fd0 x:0 0 0 2024-09-10T04:05:22.415 INFO:tasks.workunit.client.0.smithi046.stdout:8/514: mknod d1/d29/d17/d1f/d46/d67/caf 0 2024-09-10T04:05:22.417 INFO:tasks.workunit.client.1.smithi070.stdout:7/620: dread d2/d3/dc/d5d/d2d/fba [0,4194304] 0 2024-09-10T04:05:22.418 INFO:tasks.workunit.client.1.smithi070.stdout:6/655: write d0/d3/d18/d1b/d17/d54/db5/fbb [192191,76864] 0 2024-09-10T04:05:22.419 INFO:tasks.workunit.client.1.smithi070.stdout:9/671: dread d9/f68 [0,4194304] 0 2024-09-10T04:05:22.420 INFO:tasks.workunit.client.1.smithi070.stdout:6/656: chown d0/d3/d18/d1b/d17/d28/f3b 0 1 2024-09-10T04:05:22.421 INFO:tasks.workunit.client.0.smithi046.stdout:7/569: dread d5/d6/d32/dab/db6/f64 [0,4194304] 0 2024-09-10T04:05:22.422 INFO:tasks.workunit.client.0.smithi046.stdout:7/570: write d5/d6/d55/d89/f9d [1686525,108555] 0 2024-09-10T04:05:22.423 INFO:tasks.workunit.client.0.smithi046.stdout:7/571: chown d5/d6/d32/dab/db8/laf 97400 1 2024-09-10T04:05:22.425 INFO:tasks.workunit.client.1.smithi070.stdout:0/709: creat d8/d24/d2d/d5f/fe7 x:0 0 0 2024-09-10T04:05:22.428 INFO:tasks.workunit.client.1.smithi070.stdout:4/558: symlink de/d13/d18/d26/d46/daf/db7/lbe 0 2024-09-10T04:05:22.429 INFO:tasks.workunit.client.0.smithi046.stdout:9/530: creat d4/d3c/d4d/d80/fa4 x:0 0 0 2024-09-10T04:05:22.429 INFO:tasks.workunit.client.1.smithi070.stdout:8/666: sync 2024-09-10T04:05:22.431 INFO:tasks.workunit.client.1.smithi070.stdout:8/667: chown d4/d16/d17/d1a/c37 29602 1 2024-09-10T04:05:22.431 INFO:tasks.workunit.client.1.smithi070.stdout:5/612: symlink d17/d19/ld0 0 2024-09-10T04:05:22.433 INFO:tasks.workunit.client.1.smithi070.stdout:8/668: write d4/d16/f3f [4760843,22663] 0 2024-09-10T04:05:22.436 INFO:tasks.workunit.client.1.smithi070.stdout:9/672: rename d9/d1c/dac/d8a/d82 to d9/d1c/dac/d8a/dc4/dda 0 2024-09-10T04:05:22.436 INFO:tasks.workunit.client.0.smithi046.stdout:8/515: creat d1/d29/d17/fb0 x:0 0 0 2024-09-10T04:05:22.439 INFO:tasks.workunit.client.0.smithi046.stdout:0/427: fsync d1/dc/d25/d33/d3/dd/f23 0 2024-09-10T04:05:22.440 INFO:tasks.workunit.client.0.smithi046.stdout:2/557: link d2/d6/d9d/cab d2/d8/d37/cce 0 2024-09-10T04:05:22.440 INFO:tasks.workunit.client.1.smithi070.stdout:6/657: dwrite d0/f8 [0,4194304] 0 2024-09-10T04:05:22.440 INFO:tasks.workunit.client.1.smithi070.stdout:2/593: write d15/d18/f1c [1127843,44678] 0 2024-09-10T04:05:22.440 INFO:tasks.workunit.client.0.smithi046.stdout:2/558: write d2/d57/fb5 [3010104,43357] 0 2024-09-10T04:05:22.441 INFO:tasks.workunit.client.1.smithi070.stdout:6/658: stat d0/l9 0 2024-09-10T04:05:22.442 INFO:tasks.workunit.client.1.smithi070.stdout:7/621: dread d2/d3/f3b [0,4194304] 0 2024-09-10T04:05:22.446 INFO:tasks.workunit.client.1.smithi070.stdout:5/613: dwrite f10 [0,4194304] 0 2024-09-10T04:05:22.450 INFO:tasks.workunit.client.0.smithi046.stdout:7/572: creat d5/d6/db/d11/d1e/d20/fbb x:0 0 0 2024-09-10T04:05:22.451 INFO:tasks.workunit.client.0.smithi046.stdout:4/715: write d3/dab/dca/fe6 [5743884,97884] 0 2024-09-10T04:05:22.452 INFO:tasks.workunit.client.0.smithi046.stdout:4/716: readlink d3/d12/d17/de1/ddf/leb 0 2024-09-10T04:05:22.456 INFO:tasks.workunit.client.0.smithi046.stdout:5/659: unlink d7/d11/d3f/d53/d75/f93 0 2024-09-10T04:05:22.457 INFO:tasks.workunit.client.1.smithi070.stdout:9/673: fsync d9/f35 0 2024-09-10T04:05:22.458 INFO:tasks.workunit.client.0.smithi046.stdout:5/660: fsync d7/d11/d14/d45/fc2 0 2024-09-10T04:05:22.458 INFO:tasks.workunit.client.0.smithi046.stdout:9/531: chown d4/d29/d65/l7a 51760654 1 2024-09-10T04:05:22.458 INFO:tasks.workunit.client.1.smithi070.stdout:9/674: chown d9/d1c/d34/d8b/f9e 1048091776 1 2024-09-10T04:05:22.459 INFO:tasks.workunit.client.0.smithi046.stdout:9/532: read - d4/d3c/f97 zero size 2024-09-10T04:05:22.461 INFO:tasks.workunit.client.0.smithi046.stdout:5/661: truncate d7/d11/d3f/d53/d75/fcb 3872 0 2024-09-10T04:05:22.462 INFO:tasks.workunit.client.0.smithi046.stdout:2/559: creat d2/d57/fcf x:0 0 0 2024-09-10T04:05:22.463 INFO:tasks.workunit.client.0.smithi046.stdout:2/560: stat d2/d8/f27 0 2024-09-10T04:05:22.464 INFO:tasks.workunit.client.0.smithi046.stdout:1/447: rename d3/f87 to d3/dc/d34/f92 0 2024-09-10T04:05:22.466 INFO:tasks.workunit.client.1.smithi070.stdout:8/669: creat d4/d39/dc6/fe3 x:0 0 0 2024-09-10T04:05:22.467 INFO:tasks.workunit.client.1.smithi070.stdout:8/670: stat d4/d16/d17/d1a/fdf 0 2024-09-10T04:05:22.471 INFO:tasks.workunit.client.0.smithi046.stdout:3/568: getdents d12/d5d/d71/d49/d5e 0 2024-09-10T04:05:22.474 INFO:tasks.workunit.client.1.smithi070.stdout:2/594: mknod d15/d18/cd1 0 2024-09-10T04:05:22.475 INFO:tasks.workunit.client.0.smithi046.stdout:9/533: fdatasync d4/fa1 0 2024-09-10T04:05:22.476 INFO:tasks.workunit.client.0.smithi046.stdout:5/662: creat d7/d11/d14/d45/ff2 x:0 0 0 2024-09-10T04:05:22.477 INFO:tasks.workunit.client.1.smithi070.stdout:9/675: dread d9/d7a/f64 [0,4194304] 0 2024-09-10T04:05:22.478 INFO:tasks.workunit.client.0.smithi046.stdout:2/561: dread - d2/f8b zero size 2024-09-10T04:05:22.482 INFO:tasks.workunit.client.1.smithi070.stdout:0/710: rename d8/d13/d47/fb6 to d8/d13/fe8 0 2024-09-10T04:05:22.485 INFO:tasks.workunit.client.0.smithi046.stdout:4/717: creat d3/dab/dca/de8/ff8 x:0 0 0 2024-09-10T04:05:22.488 INFO:tasks.workunit.client.1.smithi070.stdout:5/614: creat d17/d19/fd1 x:0 0 0 2024-09-10T04:05:22.489 INFO:tasks.workunit.client.1.smithi070.stdout:8/671: write d4/d16/d17/d1a/d60/d7a/db8/fe2 [303403,92475] 0 2024-09-10T04:05:22.490 INFO:tasks.workunit.client.0.smithi046.stdout:9/534: chown d4/d18/d89/d9e/f8b 821551 1 2024-09-10T04:05:22.491 INFO:tasks.workunit.client.0.smithi046.stdout:2/562: creat d2/d86/d9c/fd0 x:0 0 0 2024-09-10T04:05:22.493 INFO:tasks.workunit.client.1.smithi070.stdout:4/559: rename de/d13/d18/d23/c40 to de/d13/d18/d26/d46/daf/db7/cbf 0 2024-09-10T04:05:22.494 INFO:tasks.workunit.client.1.smithi070.stdout:0/711: creat d8/d85/dca/fe9 x:0 0 0 2024-09-10T04:05:22.494 INFO:tasks.workunit.client.1.smithi070.stdout:7/622: dwrite d2/d7/db/d3a/f68 [4194304,4194304] 0 2024-09-10T04:05:22.495 INFO:tasks.workunit.client.0.smithi046.stdout:9/535: dread d4/d3c/d4d/d5c/f6a [0,4194304] 0 2024-09-10T04:05:22.495 INFO:tasks.workunit.client.0.smithi046.stdout:4/718: chown d3/d12/c81 664635724 1 2024-09-10T04:05:22.497 INFO:tasks.workunit.client.0.smithi046.stdout:3/569: mkdir d12/d13/d37/d51/db5/d4b/db7/dc0 0 2024-09-10T04:05:22.498 INFO:tasks.workunit.client.1.smithi070.stdout:9/676: dwrite d9/d7a/f64 [0,4194304] 0 2024-09-10T04:05:22.498 INFO:tasks.workunit.client.0.smithi046.stdout:3/570: write d12/d13/d37/d94/fb3 [949985,130758] 0 2024-09-10T04:05:22.501 INFO:tasks.workunit.client.1.smithi070.stdout:6/659: link d0/d3/d5c/c7a d0/d3/d18/d1b/d17/d3f/cd1 0 2024-09-10T04:05:22.502 INFO:tasks.workunit.client.1.smithi070.stdout:3/650: sync 2024-09-10T04:05:22.505 INFO:tasks.workunit.client.1.smithi070.stdout:4/560: write de/d13/d18/d5c/f96 [749602,111607] 0 2024-09-10T04:05:22.505 INFO:tasks.workunit.client.0.smithi046.stdout:8/516: rename d1/d29/d17/d1f/d2a/f2c to d1/d29/d19/fb1 0 2024-09-10T04:05:22.507 INFO:tasks.workunit.client.0.smithi046.stdout:7/573: getdents d5/d6/db/d11 0 2024-09-10T04:05:22.508 INFO:tasks.workunit.client.1.smithi070.stdout:9/677: mknod d9/d1c/d34/d8b/dcc/cdb 0 2024-09-10T04:05:22.516 INFO:tasks.workunit.client.0.smithi046.stdout:8/517: dwrite d1/d29/d17/fb0 [0,4194304] 0 2024-09-10T04:05:22.524 INFO:tasks.workunit.client.0.smithi046.stdout:7/574: mknod d5/d6/db/d11/d1f/d7a/cbc 0 2024-09-10T04:05:22.528 INFO:tasks.workunit.client.0.smithi046.stdout:5/663: getdents d7/d11/d3f/d53/d99 0 2024-09-10T04:05:22.528 INFO:tasks.workunit.client.1.smithi070.stdout:9/678: mkdir d9/d7a/d6f/d90/d89/ddc 0 2024-09-10T04:05:22.530 INFO:tasks.workunit.client.0.smithi046.stdout:8/518: stat d1/d29/d19/d36/d4c/f5b 0 2024-09-10T04:05:22.531 INFO:tasks.workunit.client.0.smithi046.stdout:8/519: write d1/d29/d19/d36/d4c/fa9 [292929,128994] 0 2024-09-10T04:05:22.533 INFO:tasks.workunit.client.1.smithi070.stdout:3/651: symlink db/d1c/d25/lde 0 2024-09-10T04:05:22.533 INFO:tasks.workunit.client.0.smithi046.stdout:4/719: link d3/d12/d17/d29/d58/lae d3/d12/d17/d29/d58/db2/d5f/d85/da8/lf9 0 2024-09-10T04:05:22.534 INFO:tasks.workunit.client.1.smithi070.stdout:2/595: dwrite d15/d18/d29/d2a/d2e/d32/d36/f5b [0,4194304] 0 2024-09-10T04:05:22.535 INFO:tasks.workunit.client.0.smithi046.stdout:4/720: write d3/d12/d17/d29/f38 [96760,80633] 0 2024-09-10T04:05:22.535 INFO:tasks.workunit.client.1.smithi070.stdout:0/712: link d8/d24/d2d/d34/d9b/dad/l9d d8/d24/d2d/d8d/lea 0 2024-09-10T04:05:22.535 INFO:tasks.workunit.client.0.smithi046.stdout:5/664: rmdir d7/d11/d14/d45 39 2024-09-10T04:05:22.536 INFO:tasks.workunit.client.0.smithi046.stdout:4/721: fsync d3/d12/d14/f18 0 2024-09-10T04:05:22.538 INFO:tasks.workunit.client.0.smithi046.stdout:4/722: fdatasync d3/d12/d17/d22/d51/d94/fd6 0 2024-09-10T04:05:22.538 INFO:tasks.workunit.client.1.smithi070.stdout:5/615: dwrite f12 [4194304,4194304] 0 2024-09-10T04:05:22.539 INFO:tasks.workunit.client.1.smithi070.stdout:8/672: dwrite d4/d16/d17/d19/d28/d43/f5d [0,4194304] 0 2024-09-10T04:05:22.539 INFO:tasks.workunit.client.0.smithi046.stdout:2/563: getdents d2/d6/d15 0 2024-09-10T04:05:22.541 INFO:tasks.workunit.client.1.smithi070.stdout:9/679: truncate d9/d1c/dac/d8a/d3c/d92/fa9 825630 0 2024-09-10T04:05:22.541 INFO:tasks.workunit.client.1.smithi070.stdout:7/623: getdents d2/d7/d9/d18/d3c 0 2024-09-10T04:05:22.542 INFO:tasks.workunit.client.0.smithi046.stdout:0/428: rename d1/dc/d25/d33/f52 to d1/d13/d37/f87 0 2024-09-10T04:05:22.545 INFO:tasks.workunit.client.1.smithi070.stdout:9/680: read - d9/d1c/d34/d8b/dcc/fd0 zero size 2024-09-10T04:05:22.552 INFO:tasks.workunit.client.1.smithi070.stdout:5/616: dwrite d17/d19/d35/f72 [0,4194304] 0 2024-09-10T04:05:22.552 INFO:tasks.workunit.client.1.smithi070.stdout:2/596: dread d15/d18/d56/d7c/f9e [0,4194304] 0 2024-09-10T04:05:22.554 INFO:tasks.workunit.client.1.smithi070.stdout:2/597: dread - d15/d18/d56/fa3 zero size 2024-09-10T04:05:22.557 INFO:tasks.workunit.client.1.smithi070.stdout:5/617: read d17/d19/d6a/d54/f62 [3410918,98041] 0 2024-09-10T04:05:22.582 INFO:tasks.workunit.client.0.smithi046.stdout:7/575: creat d5/d9/d43/db4/fbd x:0 0 0 2024-09-10T04:05:22.583 INFO:tasks.workunit.client.0.smithi046.stdout:7/576: chown d5/d6/db/f2a 1594 1 2024-09-10T04:05:22.584 INFO:tasks.workunit.client.1.smithi070.stdout:6/660: getdents d0/d3/d1f 0 2024-09-10T04:05:22.585 INFO:tasks.workunit.client.1.smithi070.stdout:3/652: mknod db/d79/db6/cdf 0 2024-09-10T04:05:22.588 INFO:tasks.workunit.client.1.smithi070.stdout:0/713: write d8/d13/de2/d53/dc6/fd9 [845937,94641] 0 2024-09-10T04:05:22.591 INFO:tasks.workunit.client.0.smithi046.stdout:5/665: mkdir d7/d11/d14/d2f/deb/da5/dcc/dc0/df3 0 2024-09-10T04:05:22.595 INFO:tasks.workunit.client.1.smithi070.stdout:0/714: dwrite d8/d13/d47/f5c [0,4194304] 0 2024-09-10T04:05:22.596 INFO:tasks.workunit.client.1.smithi070.stdout:0/715: stat d8/d24/d2d/d8d/cc7 0 2024-09-10T04:05:22.596 INFO:tasks.workunit.client.0.smithi046.stdout:1/448: rename d3/d27/d2a/c48 to d3/d27/d5d/d60/c93 0 2024-09-10T04:05:22.597 INFO:tasks.workunit.client.0.smithi046.stdout:7/577: dread d5/d9/d43/f66 [0,4194304] 0 2024-09-10T04:05:22.597 INFO:tasks.workunit.client.1.smithi070.stdout:0/716: chown d8/d13/f3d 26 1 2024-09-10T04:05:22.599 INFO:tasks.workunit.client.1.smithi070.stdout:9/681: rename d9/d7a/f64 to d9/d7a/d6f/d90/d89/fdd 0 2024-09-10T04:05:22.599 INFO:tasks.workunit.client.1.smithi070.stdout:0/717: write d8/d13/de2/d4f/f7f [4891663,82600] 0 2024-09-10T04:05:22.600 INFO:tasks.workunit.client.0.smithi046.stdout:0/429: fdatasync d1/dc/d25/d33/f3f 0 2024-09-10T04:05:22.602 INFO:tasks.workunit.client.1.smithi070.stdout:2/598: unlink d15/d18/d29/d2a/d2e/l30 0 2024-09-10T04:05:22.603 INFO:tasks.workunit.client.0.smithi046.stdout:8/520: symlink d1/d29/d19/d36/lb2 0 2024-09-10T04:05:22.603 INFO:tasks.workunit.client.1.smithi070.stdout:2/599: write d15/d18/d29/d2a/f58 [3870240,16895] 0 2024-09-10T04:05:22.604 INFO:tasks.workunit.client.0.smithi046.stdout:8/521: read - d1/d29/d17/f9c zero size 2024-09-10T04:05:22.606 INFO:tasks.workunit.client.0.smithi046.stdout:2/564: mknod d2/d6/d23/d42/dc8/cd1 0 2024-09-10T04:05:22.608 INFO:tasks.workunit.client.1.smithi070.stdout:7/624: dwrite d2/d3/dc/d5d/d2d/fba [0,4194304] 0 2024-09-10T04:05:22.608 INFO:tasks.workunit.client.1.smithi070.stdout:3/653: rmdir db/d79 39 2024-09-10T04:05:22.610 INFO:tasks.workunit.client.1.smithi070.stdout:1/570: sync 2024-09-10T04:05:22.610 INFO:tasks.workunit.client.0.smithi046.stdout:4/723: symlink d3/d12/d17/d22/d30/dc7/df3/lfa 0 2024-09-10T04:05:22.612 INFO:tasks.workunit.client.1.smithi070.stdout:8/673: symlink d4/d16/d17/d19/d62/d8d/le4 0 2024-09-10T04:05:22.614 INFO:tasks.workunit.client.1.smithi070.stdout:8/674: fdatasync d4/d16/f27 0 2024-09-10T04:05:22.614 INFO:tasks.workunit.client.0.smithi046.stdout:9/536: rename d4/d18/d89/d9e/c64 to d4/d18/d89/ca5 0 2024-09-10T04:05:22.614 INFO:tasks.workunit.client.0.smithi046.stdout:2/565: dread d2/d49/f2e [0,4194304] 0 2024-09-10T04:05:22.619 INFO:tasks.workunit.client.1.smithi070.stdout:9/682: mknod d9/d7a/d6f/d90/d89/cde 0 2024-09-10T04:05:22.626 INFO:tasks.workunit.client.0.smithi046.stdout:5/666: mknod d7/d11/d14/d2f/deb/da5/dcc/dc0/df3/cf4 0 2024-09-10T04:05:22.627 INFO:tasks.workunit.client.0.smithi046.stdout:5/667: chown d7/d11/d3f/f80 95 1 2024-09-10T04:05:22.628 INFO:tasks.workunit.client.1.smithi070.stdout:2/600: mknod d15/d2b/d94/cd2 0 2024-09-10T04:05:22.629 INFO:tasks.workunit.client.1.smithi070.stdout:2/601: dread - d15/d18/d56/d7c/d55/d6d/f97 zero size 2024-09-10T04:05:22.629 INFO:tasks.workunit.client.0.smithi046.stdout:4/724: mknod d3/d12/d17/d29/ddd/cfb 0 2024-09-10T04:05:22.630 INFO:tasks.workunit.client.1.smithi070.stdout:4/561: sync 2024-09-10T04:05:22.632 INFO:tasks.workunit.client.0.smithi046.stdout:1/449: mknod d3/d27/c94 0 2024-09-10T04:05:22.635 INFO:tasks.workunit.client.0.smithi046.stdout:8/522: write d1/d29/d17/d1f/d2a/f33 [2329299,50774] 0 2024-09-10T04:05:22.637 INFO:tasks.workunit.client.1.smithi070.stdout:7/625: mkdir d2/d3/dc/d5d/d41/dd6 0 2024-09-10T04:05:22.637 INFO:tasks.workunit.client.1.smithi070.stdout:4/562: dwrite de/d13/d5b/d7b/fac [0,4194304] 0 2024-09-10T04:05:22.639 INFO:tasks.workunit.client.0.smithi046.stdout:8/523: write d1/d29/d17/d1f/d46/d6a/d82/d99/fa5 [892750,7772] 0 2024-09-10T04:05:22.639 INFO:tasks.workunit.client.1.smithi070.stdout:3/654: truncate db/f82 869336 0 2024-09-10T04:05:22.640 INFO:tasks.workunit.client.0.smithi046.stdout:8/524: fdatasync d1/d29/d12/f97 0 2024-09-10T04:05:22.641 INFO:tasks.workunit.client.0.smithi046.stdout:0/430: mknod d1/dc/d25/d33/d3/dd/d55/d4d/c88 0 2024-09-10T04:05:22.642 INFO:tasks.workunit.client.0.smithi046.stdout:8/525: write d1/d29/f4a [3283990,65674] 0 2024-09-10T04:05:22.644 INFO:tasks.workunit.client.0.smithi046.stdout:8/526: write d1/f39 [4053599,105981] 0 2024-09-10T04:05:22.645 INFO:tasks.workunit.client.0.smithi046.stdout:8/527: chown d1/d29/d19/l21 6952509 1 2024-09-10T04:05:22.647 INFO:tasks.workunit.client.1.smithi070.stdout:6/661: truncate d0/d3/d18/d62/f85 153678 0 2024-09-10T04:05:22.647 INFO:tasks.workunit.client.0.smithi046.stdout:8/528: dread - d1/d29/d17/d1f/f54 zero size 2024-09-10T04:05:22.647 INFO:tasks.workunit.client.1.smithi070.stdout:6/662: fsync d0/d3/d8c/fad 0 2024-09-10T04:05:22.654 INFO:tasks.workunit.client.0.smithi046.stdout:4/725: mkdir d3/d12/d17/de1/ddf/dfc 0 2024-09-10T04:05:22.655 INFO:tasks.workunit.client.1.smithi070.stdout:0/718: truncate d8/d24/d2d/d34/f5b 1050890 0 2024-09-10T04:05:22.657 INFO:tasks.workunit.client.1.smithi070.stdout:1/571: write d3/d8/d13/d3f/f69 [527880,61935] 0 2024-09-10T04:05:22.657 INFO:tasks.workunit.client.0.smithi046.stdout:1/450: symlink d3/dc/l95 0 2024-09-10T04:05:22.657 INFO:tasks.workunit.client.1.smithi070.stdout:3/655: dread db/f3c [0,4194304] 0 2024-09-10T04:05:22.658 INFO:tasks.workunit.client.0.smithi046.stdout:2/566: dread d2/d8/f29 [0,4194304] 0 2024-09-10T04:05:22.659 INFO:tasks.workunit.client.1.smithi070.stdout:3/656: chown f7 872005 1 2024-09-10T04:05:22.659 INFO:tasks.workunit.client.0.smithi046.stdout:8/529: dwrite d1/d29/d19/d36/f3f [0,4194304] 0 2024-09-10T04:05:22.660 INFO:tasks.workunit.client.1.smithi070.stdout:3/657: write db/f3c [897075,110079] 0 2024-09-10T04:05:22.663 INFO:tasks.workunit.client.0.smithi046.stdout:7/578: dwrite d5/fc [4194304,4194304] 0 2024-09-10T04:05:22.667 INFO:tasks.workunit.client.0.smithi046.stdout:7/579: write d5/d6/d39/d5f/f96 [1348630,97369] 0 2024-09-10T04:05:22.668 INFO:tasks.workunit.client.1.smithi070.stdout:7/626: dwrite d2/d3/f3b [0,4194304] 0 2024-09-10T04:05:22.675 INFO:tasks.workunit.client.0.smithi046.stdout:1/451: fsync d3/f77 0 2024-09-10T04:05:22.677 INFO:tasks.workunit.client.1.smithi070.stdout:8/675: mkdir d4/d16/d17/d6f/de5 0 2024-09-10T04:05:22.678 INFO:tasks.workunit.client.1.smithi070.stdout:8/676: fdatasync d4/d16/d17/d1a/d60/d7a/fca 0 2024-09-10T04:05:22.681 INFO:tasks.workunit.client.0.smithi046.stdout:4/726: write d3/d12/d17/d22/d30/f82 [4436817,107808] 0 2024-09-10T04:05:22.681 INFO:tasks.workunit.client.0.smithi046.stdout:7/580: dwrite d5/d6/fb5 [0,4194304] 0 2024-09-10T04:05:22.682 INFO:tasks.workunit.client.0.smithi046.stdout:7/581: truncate d5/f56 4605350 0 2024-09-10T04:05:22.684 INFO:tasks.workunit.client.0.smithi046.stdout:0/431: creat d1/dc/d25/d33/d3/d29/d2d/d5e/f89 x:0 0 0 2024-09-10T04:05:22.685 INFO:tasks.workunit.client.1.smithi070.stdout:5/618: getdents d17/d37/d93 0 2024-09-10T04:05:22.686 INFO:tasks.workunit.client.0.smithi046.stdout:0/432: write d1/f35 [3567232,63977] 0 2024-09-10T04:05:22.689 INFO:tasks.workunit.client.0.smithi046.stdout:8/530: creat d1/d29/d17/d1f/d46/d6a/d82/d99/fb3 x:0 0 0 2024-09-10T04:05:22.690 INFO:tasks.workunit.client.1.smithi070.stdout:6/663: write d0/d3/d1f/d55/f8e [324078,19448] 0 2024-09-10T04:05:22.692 INFO:tasks.workunit.client.0.smithi046.stdout:0/433: truncate d1/dc/f27 4786675 0 2024-09-10T04:05:22.692 INFO:tasks.workunit.client.1.smithi070.stdout:1/572: creat d3/d10/db2/fb7 x:0 0 0 2024-09-10T04:05:22.694 INFO:tasks.workunit.client.0.smithi046.stdout:7/582: dread d5/d6/f1c [0,4194304] 0 2024-09-10T04:05:22.694 INFO:tasks.workunit.client.0.smithi046.stdout:5/668: dwrite d7/d11/f37 [0,4194304] 0 2024-09-10T04:05:22.694 INFO:tasks.workunit.client.0.smithi046.stdout:0/434: write d1/dc/f7b [394124,37964] 0 2024-09-10T04:05:22.696 INFO:tasks.workunit.client.0.smithi046.stdout:7/583: chown d5/d6/d32/dab/db8 14 1 2024-09-10T04:05:22.698 INFO:tasks.workunit.client.0.smithi046.stdout:7/584: write d5/d6/fb5 [2121717,86582] 0 2024-09-10T04:05:22.700 INFO:tasks.workunit.client.1.smithi070.stdout:9/683: link d9/d1c/dac/d8a/d1b/c50 d9/d7a/d74/cdf 0 2024-09-10T04:05:22.701 INFO:tasks.workunit.client.0.smithi046.stdout:8/531: unlink d1/d29/d17/d1f/d2a/l2b 0 2024-09-10T04:05:22.703 INFO:tasks.workunit.client.1.smithi070.stdout:9/684: chown d9/d7a/d74/d8c/fd4 1780277 1 2024-09-10T04:05:22.706 INFO:tasks.workunit.client.1.smithi070.stdout:8/677: truncate d4/f29 2892992 0 2024-09-10T04:05:22.707 INFO:tasks.workunit.client.1.smithi070.stdout:0/719: mkdir d8/deb 0 2024-09-10T04:05:22.708 INFO:tasks.workunit.client.1.smithi070.stdout:6/664: dread d0/d3/d18/d1b/f3a [0,4194304] 0 2024-09-10T04:05:22.709 INFO:tasks.workunit.client.0.smithi046.stdout:1/452: dwrite d3/d62/f73 [0,4194304] 0 2024-09-10T04:05:22.712 INFO:tasks.workunit.client.0.smithi046.stdout:4/727: mknod d3/d12/d17/cfd 0 2024-09-10T04:05:22.712 INFO:tasks.workunit.client.0.smithi046.stdout:0/435: dwrite d1/dc/d25/d33/d4/f60 [0,4194304] 0 2024-09-10T04:05:22.712 INFO:tasks.workunit.client.1.smithi070.stdout:5/619: rmdir d17/d19/d6a/d54/d57/da8 39 2024-09-10T04:05:22.715 INFO:tasks.workunit.client.0.smithi046.stdout:0/436: dread - d1/f6c zero size 2024-09-10T04:05:22.718 INFO:tasks.workunit.client.1.smithi070.stdout:1/573: mknod d3/d8/d13/d67/cb8 0 2024-09-10T04:05:22.719 INFO:tasks.workunit.client.0.smithi046.stdout:8/532: dwrite d1/da2/faa [0,4194304] 0 2024-09-10T04:05:22.722 INFO:tasks.workunit.client.1.smithi070.stdout:8/678: dwrite d4/d16/d17/f2c [0,4194304] 0 2024-09-10T04:05:22.722 INFO:tasks.workunit.client.1.smithi070.stdout:3/658: dread db/f11 [0,4194304] 0 2024-09-10T04:05:22.723 INFO:tasks.workunit.client.0.smithi046.stdout:7/585: mkdir d5/d6/dbe 0 2024-09-10T04:05:22.725 INFO:tasks.workunit.client.1.smithi070.stdout:4/563: dwrite de/d13/d43/f65 [0,4194304] 0 2024-09-10T04:05:22.726 INFO:tasks.workunit.client.0.smithi046.stdout:2/567: getdents d2/d6/d23/d5d 0 2024-09-10T04:05:22.727 INFO:tasks.workunit.client.1.smithi070.stdout:7/627: dread d2/d7/f43 [0,4194304] 0 2024-09-10T04:05:22.728 INFO:tasks.workunit.client.0.smithi046.stdout:1/453: symlink d3/dc/l96 0 2024-09-10T04:05:22.729 INFO:tasks.workunit.client.1.smithi070.stdout:4/564: write de/d13/d18/f21 [2771629,37022] 0 2024-09-10T04:05:22.730 INFO:tasks.workunit.client.0.smithi046.stdout:8/533: dwrite d1/d29/d19/d36/d4c/d5e/f7f [0,4194304] 0 2024-09-10T04:05:22.731 INFO:tasks.workunit.client.1.smithi070.stdout:5/620: creat d17/d19/d6a/fd2 x:0 0 0 2024-09-10T04:05:22.731 INFO:tasks.workunit.client.1.smithi070.stdout:2/602: link d15/d18/d29/d2a/d2e/d32/d36/laf d15/d2b/d6e/ld3 0 2024-09-10T04:05:22.736 INFO:tasks.workunit.client.1.smithi070.stdout:8/679: dwrite d4/d16/d17/d19/d62/f71 [0,4194304] 0 2024-09-10T04:05:22.737 INFO:tasks.workunit.client.1.smithi070.stdout:4/565: write de/d13/d18/d26/d46/d85/fb8 [937746,81654] 0 2024-09-10T04:05:22.739 INFO:tasks.workunit.client.0.smithi046.stdout:5/669: unlink d7/d11/d14/d45/l87 0 2024-09-10T04:05:22.741 INFO:tasks.workunit.client.0.smithi046.stdout:5/670: fdatasync d7/d11/d3f/d53/da9/f62 0 2024-09-10T04:05:22.745 INFO:tasks.workunit.client.1.smithi070.stdout:3/659: symlink db/dcc/le0 0 2024-09-10T04:05:22.745 INFO:tasks.workunit.client.1.smithi070.stdout:7/628: dwrite d2/d7/d9/d18/d3c/fae [0,4194304] 0 2024-09-10T04:05:22.749 INFO:tasks.workunit.client.0.smithi046.stdout:7/586: creat d5/fbf x:0 0 0 2024-09-10T04:05:22.754 INFO:tasks.workunit.client.1.smithi070.stdout:6/665: mkdir d0/d3/d18/d1b/d17/d54/dc4/dd2 0 2024-09-10T04:05:22.755 INFO:tasks.workunit.client.1.smithi070.stdout:9/685: write d9/f68 [2885545,61910] 0 2024-09-10T04:05:22.756 INFO:tasks.workunit.client.1.smithi070.stdout:0/720: write d8/d13/de2/f65 [1871314,76477] 0 2024-09-10T04:05:22.759 INFO:tasks.workunit.client.1.smithi070.stdout:5/621: creat d17/d24/dbf/fd3 x:0 0 0 2024-09-10T04:05:22.760 INFO:tasks.workunit.client.1.smithi070.stdout:9/686: dread d9/d1c/dac/d8a/d1b/fba [0,4194304] 0 2024-09-10T04:05:22.763 INFO:tasks.workunit.client.1.smithi070.stdout:9/687: fdatasync d9/fb7 0 2024-09-10T04:05:22.766 INFO:tasks.workunit.client.0.smithi046.stdout:4/728: dwrite d3/d12/f73 [0,4194304] 0 2024-09-10T04:05:22.766 INFO:tasks.workunit.client.1.smithi070.stdout:2/603: dwrite d15/d18/f27 [0,4194304] 0 2024-09-10T04:05:22.767 INFO:tasks.workunit.client.1.smithi070.stdout:8/680: dread d4/d16/d17/f40 [0,4194304] 0 2024-09-10T04:05:22.767 INFO:tasks.workunit.client.0.smithi046.stdout:2/568: mknod d2/d6/d23/d55/d9f/cd2 0 2024-09-10T04:05:22.769 INFO:tasks.workunit.client.0.smithi046.stdout:4/729: chown d3/d12/d17/ff5 6483620 1 2024-09-10T04:05:22.770 INFO:tasks.workunit.client.0.smithi046.stdout:9/537: sync 2024-09-10T04:05:22.770 INFO:tasks.workunit.client.0.smithi046.stdout:6/545: sync 2024-09-10T04:05:22.770 INFO:tasks.workunit.client.0.smithi046.stdout:0/437: dwrite d1/dc/d25/d33/f11 [0,4194304] 0 2024-09-10T04:05:22.770 INFO:tasks.workunit.client.0.smithi046.stdout:3/571: sync 2024-09-10T04:05:22.771 INFO:tasks.workunit.client.1.smithi070.stdout:0/721: dwrite d8/d13/d47/d7b/f8c [0,4194304] 0 2024-09-10T04:05:22.774 INFO:tasks.workunit.client.1.smithi070.stdout:0/722: chown d8/d24/d2d/d34/d6b/cd5 47 1 2024-09-10T04:05:22.775 INFO:tasks.workunit.client.0.smithi046.stdout:5/671: mkdir d7/d11/d3f/d5a/d63/ded/df5 0 2024-09-10T04:05:22.777 INFO:tasks.workunit.client.0.smithi046.stdout:0/438: write d1/dc/d25/d33/d3/d29/d2d/f4f [1490465,110480] 0 2024-09-10T04:05:22.782 INFO:tasks.workunit.client.0.smithi046.stdout:5/672: truncate d7/d11/d14/d2f/d2b/d6a/fd8 645989 0 2024-09-10T04:05:22.784 INFO:tasks.workunit.client.1.smithi070.stdout:1/574: write d3/d8/f2d [299007,128621] 0 2024-09-10T04:05:22.784 INFO:tasks.workunit.client.0.smithi046.stdout:7/587: symlink d5/d6/db/dd/d24/db9/lc0 0 2024-09-10T04:05:22.787 INFO:tasks.workunit.client.0.smithi046.stdout:1/454: write d3/dc/f84 [1799255,52182] 0 2024-09-10T04:05:22.789 INFO:tasks.workunit.client.1.smithi070.stdout:7/629: write d2/d7/db/f37 [2505191,130861] 0 2024-09-10T04:05:22.789 INFO:tasks.workunit.client.0.smithi046.stdout:2/569: unlink d2/d6/d23/d42/c89 0 2024-09-10T04:05:22.790 INFO:tasks.workunit.client.1.smithi070.stdout:7/630: stat d2/d3/dc/d5d/d42/f7c 0 2024-09-10T04:05:22.791 INFO:tasks.workunit.client.0.smithi046.stdout:2/570: chown d2/d6/d23/d5d/d77/dac 1963 1 2024-09-10T04:05:22.792 INFO:tasks.workunit.client.0.smithi046.stdout:2/571: write d2/d8/fb2 [515464,118065] 0 2024-09-10T04:05:22.792 INFO:tasks.workunit.client.0.smithi046.stdout:9/538: unlink d4/d7/d1f/ld 0 2024-09-10T04:05:22.794 INFO:tasks.workunit.client.1.smithi070.stdout:4/566: symlink de/d13/d18/d5c/db6/lc0 0 2024-09-10T04:05:22.796 INFO:tasks.workunit.client.0.smithi046.stdout:6/546: rename da/dc/f28 to da/db/d11/d2b/d55/fb2 0 2024-09-10T04:05:22.797 INFO:tasks.workunit.client.0.smithi046.stdout:0/439: write d1/dc/d25/d33/d3/dd/d55/f81 [1784961,56661] 0 2024-09-10T04:05:22.798 INFO:tasks.workunit.client.0.smithi046.stdout:6/547: write da/db/d11/d13/d1e/f7e [4660723,11661] 0 2024-09-10T04:05:22.798 INFO:tasks.workunit.client.0.smithi046.stdout:5/673: mknod d7/d11/d14/d45/d51/cf6 0 2024-09-10T04:05:22.798 INFO:tasks.workunit.client.1.smithi070.stdout:8/681: chown d4/dcf/f9a 31941518 1 2024-09-10T04:05:22.800 INFO:tasks.workunit.client.1.smithi070.stdout:2/604: creat d15/d2b/d6e/fd4 x:0 0 0 2024-09-10T04:05:22.800 INFO:tasks.workunit.client.0.smithi046.stdout:7/588: mkdir d5/d6/d62/dc1 0 2024-09-10T04:05:22.801 INFO:tasks.workunit.client.0.smithi046.stdout:8/534: getdents d1/d29/d17/d1f/d2a 0 2024-09-10T04:05:22.802 INFO:tasks.workunit.client.0.smithi046.stdout:8/535: fdatasync d1/d29/d12/f6b 0 2024-09-10T04:05:22.803 INFO:tasks.workunit.client.1.smithi070.stdout:5/622: dwrite d17/d19/f1d [4194304,4194304] 0 2024-09-10T04:05:22.803 INFO:tasks.workunit.client.0.smithi046.stdout:2/572: chown d2/d6/d23/d5d/d77/d7b/d85/d8d/l91 15 1 2024-09-10T04:05:22.804 INFO:tasks.workunit.client.1.smithi070.stdout:4/567: dwrite de/d13/d5b/d7b/fac [0,4194304] 0 2024-09-10T04:05:22.805 INFO:tasks.workunit.client.1.smithi070.stdout:0/723: creat d8/d13/de2/d6f/fec x:0 0 0 2024-09-10T04:05:22.807 INFO:tasks.workunit.client.1.smithi070.stdout:1/575: creat d3/d10/d1d/fb9 x:0 0 0 2024-09-10T04:05:22.809 INFO:tasks.workunit.client.0.smithi046.stdout:1/455: dwrite d3/dc/d34/f92 [0,4194304] 0 2024-09-10T04:05:22.810 INFO:tasks.workunit.client.1.smithi070.stdout:8/682: dwrite d4/d16/d17/d19/d28/d43/f5d [0,4194304] 0 2024-09-10T04:05:22.814 INFO:tasks.workunit.client.1.smithi070.stdout:3/660: mkdir db/d79/db6/dc3/de1 0 2024-09-10T04:05:22.815 INFO:tasks.workunit.client.1.smithi070.stdout:3/661: stat db/d1c/d25/dd8 0 2024-09-10T04:05:22.817 INFO:tasks.workunit.client.1.smithi070.stdout:1/576: dwrite f1 [8388608,4194304] 0 2024-09-10T04:05:22.819 INFO:tasks.workunit.client.0.smithi046.stdout:4/730: rename d3/d12/d17/d29/d58/db2/f48 to d3/d12/d17/d29/d58/ffe 0 2024-09-10T04:05:22.823 INFO:tasks.workunit.client.0.smithi046.stdout:7/589: dwrite d5/f56 [0,4194304] 0 2024-09-10T04:05:22.830 INFO:tasks.workunit.client.0.smithi046.stdout:0/440: creat d1/dc/d25/d33/d1a/d80/f8a x:0 0 0 2024-09-10T04:05:22.833 INFO:tasks.workunit.client.1.smithi070.stdout:5/623: symlink d17/d24/d45/dc0/ld4 0 2024-09-10T04:05:22.833 INFO:tasks.workunit.client.1.smithi070.stdout:5/624: chown d17/d37/f8d 10 1 2024-09-10T04:05:22.834 INFO:tasks.workunit.client.1.smithi070.stdout:5/625: fdatasync d17/d19/d35/f72 0 2024-09-10T04:05:22.834 INFO:tasks.workunit.client.1.smithi070.stdout:4/568: creat de/d13/d18/d23/d47/d59/fc1 x:0 0 0 2024-09-10T04:05:22.835 INFO:tasks.workunit.client.0.smithi046.stdout:7/590: dwrite d5/d9/d43/db4/fbd [0,4194304] 0 2024-09-10T04:05:22.835 INFO:tasks.workunit.client.1.smithi070.stdout:0/724: write d8/d24/f43 [2897082,25533] 0 2024-09-10T04:05:22.839 INFO:tasks.workunit.client.0.smithi046.stdout:0/441: dread d1/d13/d37/f6a [0,4194304] 0 2024-09-10T04:05:22.840 INFO:tasks.workunit.client.1.smithi070.stdout:7/631: symlink d2/d7/db/ld7 0 2024-09-10T04:05:22.841 INFO:tasks.workunit.client.0.smithi046.stdout:7/591: readlink d5/d9/d12/d72/l86 0 2024-09-10T04:05:22.842 INFO:tasks.workunit.client.0.smithi046.stdout:6/548: dwrite da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:22.846 INFO:tasks.workunit.client.0.smithi046.stdout:3/572: rename d12/d5d/d71/d49/d5e to d12/d13/d37/d51/db5/d50/d7a/dc1 0 2024-09-10T04:05:22.849 INFO:tasks.workunit.client.0.smithi046.stdout:4/731: dread d3/d12/d17/d29/d58/f75 [0,4194304] 0 2024-09-10T04:05:22.849 INFO:tasks.workunit.client.1.smithi070.stdout:6/666: getdents d0/d3/d18/d1b/d17/d54 0 2024-09-10T04:05:22.850 INFO:tasks.workunit.client.0.smithi046.stdout:6/549: write da/db/d11/d13/f7c [4573673,28913] 0 2024-09-10T04:05:22.850 INFO:tasks.workunit.client.0.smithi046.stdout:8/536: dread d1/d29/d19/d36/f4f [0,4194304] 0 2024-09-10T04:05:22.851 INFO:tasks.workunit.client.0.smithi046.stdout:4/732: readlink d3/d12/d17/d22/d30/lb3 0 2024-09-10T04:05:22.855 INFO:tasks.workunit.client.1.smithi070.stdout:2/605: mknod d15/d18/d29/d2a/d2e/cd5 0 2024-09-10T04:05:22.859 INFO:tasks.workunit.client.1.smithi070.stdout:4/569: creat de/d13/d18/d23/d39/d77/fc2 x:0 0 0 2024-09-10T04:05:22.860 INFO:tasks.workunit.client.0.smithi046.stdout:2/573: truncate d2/d6/d23/d55/f7e 108594 0 2024-09-10T04:05:22.860 INFO:tasks.workunit.client.1.smithi070.stdout:4/570: chown de/d13/c4a 3051008 1 2024-09-10T04:05:22.860 INFO:tasks.workunit.client.0.smithi046.stdout:0/442: read d1/dc/d25/d33/f3f [19148,1496] 0 2024-09-10T04:05:22.861 INFO:tasks.workunit.client.1.smithi070.stdout:4/571: write de/d13/d18/d23/d47/d59/f7d [2870810,110329] 0 2024-09-10T04:05:22.863 INFO:tasks.workunit.client.0.smithi046.stdout:9/539: rename d4/d18/d89/d9e to d4/d66/da6 0 2024-09-10T04:05:22.865 INFO:tasks.workunit.client.1.smithi070.stdout:9/688: getdents d9/d7a/d74/d8c 0 2024-09-10T04:05:22.866 INFO:tasks.workunit.client.1.smithi070.stdout:6/667: readlink d0/d3/d18/d2a/l32 0 2024-09-10T04:05:22.867 INFO:tasks.workunit.client.1.smithi070.stdout:6/668: truncate d0/d3/d18/d1b/d17/fb2 933042 0 2024-09-10T04:05:22.867 INFO:tasks.workunit.client.0.smithi046.stdout:3/573: rmdir d12/d13/d37/d51/db5/d50/d7a/dc1/d96 39 2024-09-10T04:05:22.869 INFO:tasks.workunit.client.0.smithi046.stdout:8/537: fsync d1/d29/fe 0 2024-09-10T04:05:22.869 INFO:tasks.workunit.client.0.smithi046.stdout:1/456: dwrite d3/f4a [0,4194304] 0 2024-09-10T04:05:22.869 INFO:tasks.workunit.client.1.smithi070.stdout:2/606: creat d15/d18/db2/fd6 x:0 0 0 2024-09-10T04:05:22.870 INFO:tasks.workunit.client.0.smithi046.stdout:8/538: chown d1/d29/d17/d1f 448327 1 2024-09-10T04:05:22.870 INFO:tasks.workunit.client.0.smithi046.stdout:3/574: dread - d12/d13/d37/d51/db5/d50/d7a/dc1/fbb zero size 2024-09-10T04:05:22.871 INFO:tasks.workunit.client.0.smithi046.stdout:6/550: symlink da/db/d11/d2b/d5a/lb3 0 2024-09-10T04:05:22.871 INFO:tasks.workunit.client.0.smithi046.stdout:8/539: chown d1/d29/d19/d36/d4c 2079 1 2024-09-10T04:05:22.874 INFO:tasks.workunit.client.0.smithi046.stdout:4/733: creat d3/d12/d17/d22/d30/fff x:0 0 0 2024-09-10T04:05:22.877 INFO:tasks.workunit.client.0.smithi046.stdout:5/674: getdents d7/d11/d14/d2f/deb/da5/dcc 0 2024-09-10T04:05:22.880 INFO:tasks.workunit.client.0.smithi046.stdout:2/574: symlink d2/d6/d23/d42/db4/ld3 0 2024-09-10T04:05:22.882 INFO:tasks.workunit.client.0.smithi046.stdout:0/443: creat d1/d13/d37/d75/f8b x:0 0 0 2024-09-10T04:05:22.884 INFO:tasks.workunit.client.1.smithi070.stdout:3/662: rename db/d79/db6/fc1 to db/d1c/fe2 0 2024-09-10T04:05:22.885 INFO:tasks.workunit.client.0.smithi046.stdout:9/540: truncate d4/f6 1042356 0 2024-09-10T04:05:22.885 INFO:tasks.workunit.client.1.smithi070.stdout:0/725: write d8/d13/de2/d53/d59/f64 [809894,105288] 0 2024-09-10T04:05:22.886 INFO:tasks.workunit.client.0.smithi046.stdout:7/592: mknod d5/d6/db/dd/d24/db7/cc2 0 2024-09-10T04:05:22.889 INFO:tasks.workunit.client.0.smithi046.stdout:1/457: rmdir d3 39 2024-09-10T04:05:22.891 INFO:tasks.workunit.client.0.smithi046.stdout:6/551: mkdir da/db/d11/db4 0 2024-09-10T04:05:22.892 INFO:tasks.workunit.client.1.smithi070.stdout:7/632: link d2/d7/d9/d18/d3c/la6 d2/d40/d6c/ld8 0 2024-09-10T04:05:22.892 INFO:tasks.workunit.client.1.smithi070.stdout:7/633: readlink d2/l11 0 2024-09-10T04:05:22.893 INFO:tasks.workunit.client.1.smithi070.stdout:7/634: dread - d2/d40/d6c/db2/db7/fc0 zero size 2024-09-10T04:05:22.893 INFO:tasks.workunit.client.0.smithi046.stdout:3/575: chown d12/d13/d37/d51/db5/d50/c78 2802 1 2024-09-10T04:05:22.895 INFO:tasks.workunit.client.0.smithi046.stdout:8/540: mknod d1/d29/d17/d1f/d46/d67/cb4 0 2024-09-10T04:05:22.897 INFO:tasks.workunit.client.0.smithi046.stdout:4/734: mkdir d3/d12/d17/d22/d51/d5c/d100 0 2024-09-10T04:05:22.899 INFO:tasks.workunit.client.0.smithi046.stdout:5/675: fsync d7/f49 0 2024-09-10T04:05:22.901 INFO:tasks.workunit.client.0.smithi046.stdout:2/575: stat d2/d6/d23/d5d/d77/dac/lb9 0 2024-09-10T04:05:22.901 INFO:tasks.workunit.client.1.smithi070.stdout:5/626: link d17/d24/d45/ca1 d17/d24/d45/dc0/cd5 0 2024-09-10T04:05:22.902 INFO:tasks.workunit.client.0.smithi046.stdout:0/444: fsync d1/dc/d25/d33/d1a/d41/f5a 0 2024-09-10T04:05:22.904 INFO:tasks.workunit.client.0.smithi046.stdout:7/593: creat d5/d6/db/dd/d24/d76/fc3 x:0 0 0 2024-09-10T04:05:22.905 INFO:tasks.workunit.client.0.smithi046.stdout:7/594: stat d5/d6/d62/l7e 0 2024-09-10T04:05:22.905 INFO:tasks.workunit.client.1.smithi070.stdout:7/635: symlink d2/d7/d9/d18/d49/ld9 0 2024-09-10T04:05:22.907 INFO:tasks.workunit.client.1.smithi070.stdout:1/577: rename d3/d10/c9d to d3/d8/d12/d61/cba 0 2024-09-10T04:05:22.908 INFO:tasks.workunit.client.1.smithi070.stdout:3/663: mknod db/d1c/d25/ce3 0 2024-09-10T04:05:22.908 INFO:tasks.workunit.client.1.smithi070.stdout:1/578: write d3/d8/d12/f45 [4544442,111347] 0 2024-09-10T04:05:22.909 INFO:tasks.workunit.client.1.smithi070.stdout:3/664: write db/d1c/d53/f6b [2905315,124979] 0 2024-09-10T04:05:22.909 INFO:tasks.workunit.client.1.smithi070.stdout:6/669: creat d0/d3/d18/d1b/fd3 x:0 0 0 2024-09-10T04:05:22.910 INFO:tasks.workunit.client.1.smithi070.stdout:3/665: chown db/d1c 1655 1 2024-09-10T04:05:22.911 INFO:tasks.workunit.client.1.smithi070.stdout:2/607: creat d15/d18/d29/d2a/d2e/d32/fd7 x:0 0 0 2024-09-10T04:05:22.912 INFO:tasks.workunit.client.1.smithi070.stdout:2/608: read d15/d18/f27 [582896,90671] 0 2024-09-10T04:05:22.913 INFO:tasks.workunit.client.1.smithi070.stdout:4/572: link de/d13/d18/l74 de/d13/d18/d23/d39/lc3 0 2024-09-10T04:05:22.913 INFO:tasks.workunit.client.0.smithi046.stdout:3/576: mknod d12/d13/d37/d94/cc2 0 2024-09-10T04:05:22.915 INFO:tasks.workunit.client.1.smithi070.stdout:9/689: rename d9/d7a/d6f to d9/d1c/dac/d8a/dc4/dda/de0 0 2024-09-10T04:05:22.916 INFO:tasks.workunit.client.1.smithi070.stdout:3/666: dwrite db/d79/db6/fbe [0,4194304] 0 2024-09-10T04:05:22.917 INFO:tasks.workunit.client.0.smithi046.stdout:2/576: creat d2/d6/d23/d5d/d77/d7b/d85/d8d/fd4 x:0 0 0 2024-09-10T04:05:22.919 INFO:tasks.workunit.client.0.smithi046.stdout:7/595: truncate d5/f6b 838557 0 2024-09-10T04:05:22.921 INFO:tasks.workunit.client.1.smithi070.stdout:4/573: dread de/d13/d18/d23/d39/d48/f7e [0,4194304] 0 2024-09-10T04:05:22.922 INFO:tasks.workunit.client.1.smithi070.stdout:5/627: mknod d17/d19/d6a/d54/d57/da8/cd6 0 2024-09-10T04:05:22.923 INFO:tasks.workunit.client.0.smithi046.stdout:6/552: symlink da/db/d11/d13/d1e/lb5 0 2024-09-10T04:05:22.923 INFO:tasks.workunit.client.1.smithi070.stdout:7/636: mknod d2/d3/cda 0 2024-09-10T04:05:22.925 INFO:tasks.workunit.client.1.smithi070.stdout:5/628: write d17/d19/d6a/d54/d57/da3/f55 [548920,71811] 0 2024-09-10T04:05:22.925 INFO:tasks.workunit.client.0.smithi046.stdout:2/577: dwrite d2/d86/d9c/fd0 [0,4194304] 0 2024-09-10T04:05:22.927 INFO:tasks.workunit.client.0.smithi046.stdout:2/578: write d2/d8/d37/f6c [1307162,12098] 0 2024-09-10T04:05:22.929 INFO:tasks.workunit.client.0.smithi046.stdout:2/579: truncate d2/d8/d65/fa9 845693 0 2024-09-10T04:05:22.930 INFO:tasks.workunit.client.1.smithi070.stdout:0/726: rename d8/d13/de2/d81/fb7 to d8/d13/de2/d71/fed 0 2024-09-10T04:05:22.930 INFO:tasks.workunit.client.0.smithi046.stdout:2/580: chown d2/d6/c7a 2954 1 2024-09-10T04:05:22.931 INFO:tasks.workunit.client.0.smithi046.stdout:6/553: dread da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:22.932 INFO:tasks.workunit.client.1.smithi070.stdout:9/690: dread d9/d7a/f51 [0,4194304] 0 2024-09-10T04:05:22.936 INFO:tasks.workunit.client.1.smithi070.stdout:1/579: symlink d3/d8/d27/d7a/d94/da5/lbb 0 2024-09-10T04:05:22.937 INFO:tasks.workunit.client.0.smithi046.stdout:6/554: dwrite da/db/d11/d2b/d2f/f66 [0,4194304] 0 2024-09-10T04:05:22.938 INFO:tasks.workunit.client.1.smithi070.stdout:3/667: mknod db/d1c/d89/ce4 0 2024-09-10T04:05:22.940 INFO:tasks.workunit.client.1.smithi070.stdout:8/683: sync 2024-09-10T04:05:22.941 INFO:tasks.workunit.client.1.smithi070.stdout:8/684: dread - d4/d16/d17/d6f/fc5 zero size 2024-09-10T04:05:22.942 INFO:tasks.workunit.client.1.smithi070.stdout:1/580: dwrite d3/d10/d1d/f46 [0,4194304] 0 2024-09-10T04:05:22.944 INFO:tasks.workunit.client.0.smithi046.stdout:6/555: dread da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:22.947 INFO:tasks.workunit.client.1.smithi070.stdout:4/574: creat de/d13/d18/d26/d46/daf/db7/fc4 x:0 0 0 2024-09-10T04:05:22.947 INFO:tasks.workunit.client.0.smithi046.stdout:1/458: write d3/d27/d32/d1f/f29 [221797,83570] 0 2024-09-10T04:05:22.951 INFO:tasks.workunit.client.1.smithi070.stdout:7/637: write d2/d3/dc/d5d/d42/fc4 [980665,70906] 0 2024-09-10T04:05:22.952 INFO:tasks.workunit.client.1.smithi070.stdout:8/685: dwrite d4/d16/d17/d19/d28/d43/d67/fc2 [0,4194304] 0 2024-09-10T04:05:22.953 INFO:tasks.workunit.client.1.smithi070.stdout:7/638: stat d2/d7/db/d3a 0 2024-09-10T04:05:22.955 INFO:tasks.workunit.client.1.smithi070.stdout:9/691: write d9/d59/d5d/fd8 [3270050,92237] 0 2024-09-10T04:05:22.959 INFO:tasks.workunit.client.1.smithi070.stdout:6/670: creat d0/d3/d18/fd4 x:0 0 0 2024-09-10T04:05:22.962 INFO:tasks.workunit.client.1.smithi070.stdout:3/668: dread db/d1c/f6d [0,4194304] 0 2024-09-10T04:05:22.963 INFO:tasks.workunit.client.1.smithi070.stdout:3/669: stat db/d1c/f4b 0 2024-09-10T04:05:22.967 INFO:tasks.workunit.client.1.smithi070.stdout:7/639: mknod d2/d3/dc/d5d/d2d/cdb 0 2024-09-10T04:05:22.971 INFO:tasks.workunit.client.1.smithi070.stdout:5/629: creat d17/fd7 x:0 0 0 2024-09-10T04:05:22.973 INFO:tasks.workunit.client.1.smithi070.stdout:2/609: rename d15/d2b/l4b to d15/d18/d29/d2a/ld8 0 2024-09-10T04:05:22.973 INFO:tasks.workunit.client.1.smithi070.stdout:7/640: dwrite d2/d7/db/d3a/d7e/f93 [4194304,4194304] 0 2024-09-10T04:05:22.973 INFO:tasks.workunit.client.1.smithi070.stdout:5/630: write d17/d19/d6a/d54/d57/da3/f82 [579063,107026] 0 2024-09-10T04:05:22.974 INFO:tasks.workunit.client.1.smithi070.stdout:8/686: mknod d4/d16/dcc/ce6 0 2024-09-10T04:05:22.974 INFO:tasks.workunit.client.1.smithi070.stdout:2/610: readlink d15/d2b/la0 0 2024-09-10T04:05:22.976 INFO:tasks.workunit.client.1.smithi070.stdout:3/670: mkdir db/d1c/d25/d28/dd3/de5 0 2024-09-10T04:05:22.980 INFO:tasks.workunit.client.1.smithi070.stdout:1/581: creat d3/d8/fbc x:0 0 0 2024-09-10T04:05:22.984 INFO:tasks.workunit.client.1.smithi070.stdout:4/575: rename de/d13/d18/d26/d46/d85/fb8 to de/d13/d43/d88/db5/fc5 0 2024-09-10T04:05:22.986 INFO:tasks.workunit.client.1.smithi070.stdout:2/611: mknod d15/d2b/cd9 0 2024-09-10T04:05:22.987 INFO:tasks.workunit.client.1.smithi070.stdout:2/612: dread - d15/d2b/d6e/fd0 zero size 2024-09-10T04:05:22.988 INFO:tasks.workunit.client.0.smithi046.stdout:8/541: link d1/da2/faa d1/d29/d17/d1f/d46/d6a/d82/d99/fb5 0 2024-09-10T04:05:22.988 INFO:tasks.workunit.client.1.smithi070.stdout:7/641: truncate d2/d3/dc/d5d/d42/f7c 3110425 0 2024-09-10T04:05:22.988 INFO:tasks.workunit.client.0.smithi046.stdout:8/542: dread - d1/d29/d12/f97 zero size 2024-09-10T04:05:22.992 INFO:tasks.workunit.client.1.smithi070.stdout:7/642: dwrite d2/d7/d9/d18/d3c/f73 [8388608,4194304] 0 2024-09-10T04:05:22.993 INFO:tasks.workunit.client.0.smithi046.stdout:3/577: fsync d12/d13/d37/d51/db5/d50/d7a/dc1/fbc 0 2024-09-10T04:05:22.997 INFO:tasks.workunit.client.1.smithi070.stdout:9/692: rename d9/d59/d5d to d9/d1c/dac/d8a/dc4/de1 0 2024-09-10T04:05:22.999 INFO:tasks.workunit.client.1.smithi070.stdout:4/576: chown de/d13/d18/l19 0 1 2024-09-10T04:05:23.001 INFO:tasks.workunit.client.1.smithi070.stdout:0/727: truncate d8/d24/d2d/f3a 593731 0 2024-09-10T04:05:23.003 INFO:tasks.workunit.client.1.smithi070.stdout:2/613: mknod d15/d18/d29/d2a/d2e/da6/dc0/dcb/cda 0 2024-09-10T04:05:23.003 INFO:tasks.workunit.client.1.smithi070.stdout:2/614: fsync d15/fa7 0 2024-09-10T04:05:23.004 INFO:tasks.workunit.client.1.smithi070.stdout:2/615: chown f3 153 1 2024-09-10T04:05:23.005 INFO:tasks.workunit.client.1.smithi070.stdout:7/643: chown d2/d3/ca2 1173539 1 2024-09-10T04:05:23.006 INFO:tasks.workunit.client.0.smithi046.stdout:2/581: fsync d2/d8/d37/f66 0 2024-09-10T04:05:23.009 INFO:tasks.workunit.client.1.smithi070.stdout:5/631: creat d17/d24/fd8 x:0 0 0 2024-09-10T04:05:23.010 INFO:tasks.workunit.client.1.smithi070.stdout:5/632: chown d17/d19/d6a/d54/l71 3904 1 2024-09-10T04:05:23.010 INFO:tasks.workunit.client.0.smithi046.stdout:6/556: symlink da/db/d11/d2b/d2f/lb6 0 2024-09-10T04:05:23.012 INFO:tasks.workunit.client.0.smithi046.stdout:1/459: unlink d3/d27/d32/c2d 0 2024-09-10T04:05:23.014 INFO:tasks.workunit.client.1.smithi070.stdout:3/671: creat db/d1c/d25/fe6 x:0 0 0 2024-09-10T04:05:23.015 INFO:tasks.workunit.client.0.smithi046.stdout:4/735: rmdir d3/d12/d17/d29/d58/db2/d7a 39 2024-09-10T04:05:23.019 INFO:tasks.workunit.client.0.smithi046.stdout:5/676: link d7/d11/d14/d2f/d2b/d6a/caa d7/d11/d14/d2f/df0/cf7 0 2024-09-10T04:05:23.021 INFO:tasks.workunit.client.1.smithi070.stdout:0/728: chown d8/d13/d1a/da0 13 1 2024-09-10T04:05:23.022 INFO:tasks.workunit.client.0.smithi046.stdout:7/596: creat d5/d6/d62/dc1/fc4 x:0 0 0 2024-09-10T04:05:23.022 INFO:tasks.workunit.client.1.smithi070.stdout:0/729: truncate d8/d13/de2/d53/dc6/fd9 982298 0 2024-09-10T04:05:23.023 INFO:tasks.workunit.client.1.smithi070.stdout:2/616: mkdir d15/d18/d29/ddb 0 2024-09-10T04:05:23.024 INFO:tasks.workunit.client.0.smithi046.stdout:7/597: read d5/d9/d12/f5d [414125,44849] 0 2024-09-10T04:05:23.024 INFO:tasks.workunit.client.1.smithi070.stdout:0/730: write d8/d24/d2d/d8d/fe5 [4133556,60200] 0 2024-09-10T04:05:23.025 INFO:tasks.workunit.client.0.smithi046.stdout:7/598: write d5/d6/d32/dab/fb1 [3151258,104908] 0 2024-09-10T04:05:23.026 INFO:tasks.workunit.client.1.smithi070.stdout:2/617: read d15/d18/d56/d7c/d55/d6d/fa8 [283439,91920] 0 2024-09-10T04:05:23.026 INFO:tasks.workunit.client.1.smithi070.stdout:2/618: read - d15/d18/d56/d7c/f92 zero size 2024-09-10T04:05:23.029 INFO:tasks.workunit.client.0.smithi046.stdout:3/578: dread d12/d13/d14/d1e/f3f [0,4194304] 0 2024-09-10T04:05:23.030 INFO:tasks.workunit.client.0.smithi046.stdout:2/582: mkdir d2/d6/d23/dd5 0 2024-09-10T04:05:23.030 INFO:tasks.workunit.client.0.smithi046.stdout:3/579: stat d12/d5d/d71/f5b 0 2024-09-10T04:05:23.031 INFO:tasks.workunit.client.0.smithi046.stdout:7/599: dread d5/d6/fb5 [0,4194304] 0 2024-09-10T04:05:23.031 INFO:tasks.workunit.client.1.smithi070.stdout:7/644: symlink d2/d3/d63/ldc 0 2024-09-10T04:05:23.033 INFO:tasks.workunit.client.0.smithi046.stdout:6/557: fdatasync da/db/f16 0 2024-09-10T04:05:23.038 INFO:tasks.workunit.client.0.smithi046.stdout:1/460: dread d3/dc/f23 [0,4194304] 0 2024-09-10T04:05:23.038 INFO:tasks.workunit.client.0.smithi046.stdout:0/445: write d1/dc/d25/d33/d1a/f31 [269647,25877] 0 2024-09-10T04:05:23.039 INFO:tasks.workunit.client.0.smithi046.stdout:1/461: stat d3/d62/f73 0 2024-09-10T04:05:23.042 INFO:tasks.workunit.client.0.smithi046.stdout:5/677: creat d7/d11/d14/d2f/deb/ff8 x:0 0 0 2024-09-10T04:05:23.042 INFO:tasks.workunit.client.0.smithi046.stdout:5/678: chown d7/d11/d14/d2f/da3 1037 1 2024-09-10T04:05:23.043 INFO:tasks.workunit.client.0.smithi046.stdout:5/679: chown d7/f43 7113 1 2024-09-10T04:05:23.044 INFO:tasks.workunit.client.1.smithi070.stdout:2/619: truncate d15/d18/f21 2998572 0 2024-09-10T04:05:23.047 INFO:tasks.workunit.client.1.smithi070.stdout:4/577: rename de/d13/d43/f8d to de/d13/d18/d23/fc6 0 2024-09-10T04:05:23.049 INFO:tasks.workunit.client.1.smithi070.stdout:0/731: creat d8/d13/d1a/d8f/d95/fee x:0 0 0 2024-09-10T04:05:23.050 INFO:tasks.workunit.client.0.smithi046.stdout:7/600: unlink d5/d6/l3a 0 2024-09-10T04:05:23.052 INFO:tasks.workunit.client.1.smithi070.stdout:9/693: write d9/d1c/f2c [2845116,85967] 0 2024-09-10T04:05:23.052 INFO:tasks.workunit.client.1.smithi070.stdout:3/672: link db/l48 db/d1c/d25/dd8/le7 0 2024-09-10T04:05:23.053 INFO:tasks.workunit.client.0.smithi046.stdout:6/558: symlink da/db/d11/d13/d1e/d23/d54/lb7 0 2024-09-10T04:05:23.055 INFO:tasks.workunit.client.1.smithi070.stdout:2/620: unlink d15/d2b/la0 0 2024-09-10T04:05:23.055 INFO:tasks.workunit.client.0.smithi046.stdout:8/543: creat d1/d29/d19/d36/fb6 x:0 0 0 2024-09-10T04:05:23.056 INFO:tasks.workunit.client.0.smithi046.stdout:8/544: dread - d1/d29/d17/f9c zero size 2024-09-10T04:05:23.057 INFO:tasks.workunit.client.1.smithi070.stdout:7/645: mknod d2/d7/d9/d18/cdd 0 2024-09-10T04:05:23.058 INFO:tasks.workunit.client.1.smithi070.stdout:3/673: dwrite db/dcc/fd5 [0,4194304] 0 2024-09-10T04:05:23.059 INFO:tasks.workunit.client.0.smithi046.stdout:8/545: write d1/d29/d17/d1f/d46/d6a/d82/d99/fb3 [35949,5252] 0 2024-09-10T04:05:23.060 INFO:tasks.workunit.client.1.smithi070.stdout:3/674: dread - db/d1c/d25/d28/d43/fa4 zero size 2024-09-10T04:05:23.060 INFO:tasks.workunit.client.0.smithi046.stdout:8/546: read d1/d29/d17/d1f/d46/d6a/d82/d99/fb5 [3009027,106209] 0 2024-09-10T04:05:23.061 INFO:tasks.workunit.client.1.smithi070.stdout:0/732: creat d8/d24/d36/d87/fef x:0 0 0 2024-09-10T04:05:23.061 INFO:tasks.workunit.client.0.smithi046.stdout:4/736: write d3/d12/d17/d22/f4a [3767737,93550] 0 2024-09-10T04:05:23.063 INFO:tasks.workunit.client.0.smithi046.stdout:1/462: mknod d3/dc/c97 0 2024-09-10T04:05:23.067 INFO:tasks.workunit.client.0.smithi046.stdout:8/547: dread d1/d29/d17/fb0 [0,4194304] 0 2024-09-10T04:05:23.068 INFO:tasks.workunit.client.1.smithi070.stdout:4/578: truncate de/d13/d18/d26/f2b 201165 0 2024-09-10T04:05:23.068 INFO:tasks.workunit.client.0.smithi046.stdout:8/548: stat d1/d29/f4a 0 2024-09-10T04:05:23.071 INFO:tasks.workunit.client.0.smithi046.stdout:3/580: creat d12/d13/d37/d94/dbf/fc3 x:0 0 0 2024-09-10T04:05:23.073 INFO:tasks.workunit.client.1.smithi070.stdout:4/579: dwrite de/d13/d18/d23/d39/f8e [0,4194304] 0 2024-09-10T04:05:23.077 INFO:tasks.workunit.client.1.smithi070.stdout:7/646: rename d2/d7/d9/l81 to d2/d3/dc/d5d/d2d/lde 0 2024-09-10T04:05:23.078 INFO:tasks.workunit.client.0.smithi046.stdout:4/737: dread d3/fc [4194304,4194304] 0 2024-09-10T04:05:23.078 INFO:tasks.workunit.client.0.smithi046.stdout:6/559: mknod da/db/d11/d2b/d2f/d3b/cb8 0 2024-09-10T04:05:23.079 INFO:tasks.workunit.client.1.smithi070.stdout:3/675: write db/f3c [113781,95838] 0 2024-09-10T04:05:23.079 INFO:tasks.workunit.client.0.smithi046.stdout:4/738: chown d3/d12/d17/d29/ddd/cee 30134309 1 2024-09-10T04:05:23.086 INFO:tasks.workunit.client.1.smithi070.stdout:0/733: mknod d8/d13/d1a/d8f/cf0 0 2024-09-10T04:05:23.089 INFO:tasks.workunit.client.0.smithi046.stdout:6/560: dwrite da/db/d63/fa0 [0,4194304] 0 2024-09-10T04:05:23.090 INFO:tasks.workunit.client.0.smithi046.stdout:4/739: dwrite d3/d12/d17/d29/fcb [0,4194304] 0 2024-09-10T04:05:23.095 INFO:tasks.workunit.client.1.smithi070.stdout:9/694: write d9/d1c/dac/d8a/f3d [5468260,11492] 0 2024-09-10T04:05:23.102 INFO:tasks.workunit.client.0.smithi046.stdout:4/740: dwrite d3/d12/d14/fa6 [0,4194304] 0 2024-09-10T04:05:23.102 INFO:tasks.workunit.client.1.smithi070.stdout:4/580: creat de/d13/d18/d26/d46/fc7 x:0 0 0 2024-09-10T04:05:23.103 INFO:tasks.workunit.client.1.smithi070.stdout:4/581: dread - de/d13/d18/d26/d8c/f90 zero size 2024-09-10T04:05:23.104 INFO:tasks.workunit.client.0.smithi046.stdout:7/601: truncate d5/f6b 1025781 0 2024-09-10T04:05:23.106 INFO:tasks.workunit.client.0.smithi046.stdout:7/602: chown d5/d9/d43/f66 3020357 1 2024-09-10T04:05:23.108 INFO:tasks.workunit.client.1.smithi070.stdout:9/695: stat d9/d1c/dac/d8a/c6c 0 2024-09-10T04:05:23.111 INFO:tasks.workunit.client.1.smithi070.stdout:9/696: fdatasync d9/d59/d2a/d3f/daf/fc5 0 2024-09-10T04:05:23.114 INFO:tasks.workunit.client.1.smithi070.stdout:9/697: chown d9/d1c/dac/d8a/dc4/dda/de0/d90/d89 974 1 2024-09-10T04:05:23.114 INFO:tasks.workunit.client.0.smithi046.stdout:6/561: creat da/dc/d79/fb9 x:0 0 0 2024-09-10T04:05:23.116 INFO:tasks.workunit.client.1.smithi070.stdout:2/621: truncate d15/d18/d56/d7c/d55/d6d/fba 2462097 0 2024-09-10T04:05:23.116 INFO:tasks.workunit.client.1.smithi070.stdout:0/734: dwrite d8/d13/fe8 [0,4194304] 0 2024-09-10T04:05:23.118 INFO:tasks.workunit.client.1.smithi070.stdout:4/582: getdents de/db0 0 2024-09-10T04:05:23.119 INFO:tasks.workunit.client.1.smithi070.stdout:2/622: chown d15/d18/d29/d2a/d2e/d32/f57 15110 1 2024-09-10T04:05:23.120 INFO:tasks.workunit.client.0.smithi046.stdout:1/463: truncate d3/d27/d5d/f6a 2045421 0 2024-09-10T04:05:23.121 INFO:tasks.workunit.client.1.smithi070.stdout:4/583: chown de/d13/d18/d23/d47/d59/d79/f66 0 1 2024-09-10T04:05:23.122 INFO:tasks.workunit.client.0.smithi046.stdout:8/549: mkdir d1/d29/d19/db7 0 2024-09-10T04:05:23.123 INFO:tasks.workunit.client.0.smithi046.stdout:3/581: mkdir d12/d13/d37/d51/db5/d4b/dc4 0 2024-09-10T04:05:23.123 INFO:tasks.workunit.client.0.smithi046.stdout:8/550: read - d1/d29/d17/d1f/d46/d76/fab zero size 2024-09-10T04:05:23.124 INFO:tasks.workunit.client.1.smithi070.stdout:9/698: chown d9/d7a/d72/f7b 288635 1 2024-09-10T04:05:23.124 INFO:tasks.workunit.client.1.smithi070.stdout:9/699: stat d9/d1c/d34 0 2024-09-10T04:05:23.126 INFO:tasks.workunit.client.0.smithi046.stdout:4/741: rename d3/d12/d17/d22/f4c to d3/d12/d17/d22/d51/d5c/d100/f101 0 2024-09-10T04:05:23.129 INFO:tasks.workunit.client.1.smithi070.stdout:4/584: creat de/d13/d5b/da0/fc8 x:0 0 0 2024-09-10T04:05:23.131 INFO:tasks.workunit.client.0.smithi046.stdout:0/446: getdents d1/d13/d37 0 2024-09-10T04:05:23.132 INFO:tasks.workunit.client.0.smithi046.stdout:0/447: chown d1/dc/d25/d33/d3/f46 1 1 2024-09-10T04:05:23.133 INFO:tasks.workunit.client.0.smithi046.stdout:6/562: symlink da/db/d18/lba 0 2024-09-10T04:05:23.135 INFO:tasks.workunit.client.0.smithi046.stdout:0/448: write d1/d13/d37/d75/f83 [150956,126519] 0 2024-09-10T04:05:23.135 INFO:tasks.workunit.client.1.smithi070.stdout:7/647: dwrite d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:23.135 INFO:tasks.workunit.client.0.smithi046.stdout:6/563: read - da/db/d11/d13/d1e/d23/d54/d90/faa zero size 2024-09-10T04:05:23.140 INFO:tasks.workunit.client.0.smithi046.stdout:5/680: getdents d7/d11 0 2024-09-10T04:05:23.141 INFO:tasks.workunit.client.1.smithi070.stdout:4/585: truncate de/d13/d18/d26/d46/fa1 716441 0 2024-09-10T04:05:23.144 INFO:tasks.workunit.client.0.smithi046.stdout:1/464: unlink d3/d27/d32/d1f/f31 0 2024-09-10T04:05:23.147 INFO:tasks.workunit.client.1.smithi070.stdout:0/735: getdents d8/d13/d1a/d8f 0 2024-09-10T04:05:23.147 INFO:tasks.workunit.client.1.smithi070.stdout:0/736: dread - d8/f99 zero size 2024-09-10T04:05:23.150 INFO:tasks.workunit.client.0.smithi046.stdout:4/742: dread d3/d12/d17/d22/d30/fa1 [0,4194304] 0 2024-09-10T04:05:23.151 INFO:tasks.workunit.client.1.smithi070.stdout:4/586: creat de/d13/d5b/da0/fc9 x:0 0 0 2024-09-10T04:05:23.155 INFO:tasks.workunit.client.1.smithi070.stdout:2/623: truncate d15/d18/d29/d2a/d2e/d32/f33 2061134 0 2024-09-10T04:05:23.156 INFO:tasks.workunit.client.1.smithi070.stdout:0/737: chown d8/d13/l7c 13876395 1 2024-09-10T04:05:23.156 INFO:tasks.workunit.client.1.smithi070.stdout:7/648: symlink d2/d7/d9/d18/ldf 0 2024-09-10T04:05:23.156 INFO:tasks.workunit.client.1.smithi070.stdout:4/587: chown de/d13/d18/d23/d47/d59/d79 1391650392 1 2024-09-10T04:05:23.158 INFO:tasks.workunit.client.1.smithi070.stdout:2/624: dread - d15/d18/d29/d2a/d2e/da6/dc0/dcb/fb9 zero size 2024-09-10T04:05:23.163 INFO:tasks.workunit.client.0.smithi046.stdout:0/449: rmdir d1/dc 39 2024-09-10T04:05:23.164 INFO:tasks.workunit.client.1.smithi070.stdout:4/588: dwrite de/d13/d18/d23/d47/d59/f87 [0,4194304] 0 2024-09-10T04:05:23.167 INFO:tasks.workunit.client.0.smithi046.stdout:3/582: dread d12/d13/d14/d1e/d2d/f4f [0,4194304] 0 2024-09-10T04:05:23.169 INFO:tasks.workunit.client.0.smithi046.stdout:9/541: sync 2024-09-10T04:05:23.174 INFO:tasks.workunit.client.0.smithi046.stdout:8/551: dwrite d1/d29/d19/d36/f4f [0,4194304] 0 2024-09-10T04:05:23.174 INFO:tasks.workunit.client.0.smithi046.stdout:4/743: mknod d3/dab/dca/c102 0 2024-09-10T04:05:23.180 INFO:tasks.workunit.client.1.smithi070.stdout:4/589: dread de/d13/d18/d23/d47/f51 [0,4194304] 0 2024-09-10T04:05:23.181 INFO:tasks.workunit.client.0.smithi046.stdout:8/552: write d1/d29/d17/d1f/d46/f4b [2449811,64610] 0 2024-09-10T04:05:23.183 INFO:tasks.workunit.client.0.smithi046.stdout:3/583: dwrite d12/d13/d37/d94/dbf/fc3 [0,4194304] 0 2024-09-10T04:05:23.191 INFO:tasks.workunit.client.0.smithi046.stdout:0/450: stat d1/dc/d25/d33/d3/d29/l5d 0 2024-09-10T04:05:23.198 INFO:tasks.workunit.client.0.smithi046.stdout:0/451: fdatasync d1/dc/d25/d33/d4/d19/f58 0 2024-09-10T04:05:23.199 INFO:tasks.workunit.client.0.smithi046.stdout:6/564: dwrite da/db/d11/d13/d1e/f4d [0,4194304] 0 2024-09-10T04:05:23.199 INFO:tasks.workunit.client.0.smithi046.stdout:8/553: write d1/d29/d17/d1f/d46/d6a/d82/d99/fa5 [528011,90587] 0 2024-09-10T04:05:23.199 INFO:tasks.workunit.client.0.smithi046.stdout:4/744: symlink d3/d12/d17/d22/d51/d5c/l103 0 2024-09-10T04:05:23.201 INFO:tasks.workunit.client.0.smithi046.stdout:7/603: dwrite d5/d6/db/dd/d4b/f52 [0,4194304] 0 2024-09-10T04:05:23.202 INFO:tasks.workunit.client.0.smithi046.stdout:5/681: dwrite d7/d11/d14/d2f/d2b/f3a [0,4194304] 0 2024-09-10T04:05:23.206 INFO:tasks.workunit.client.0.smithi046.stdout:1/465: getdents d3/dc/d34/d51 0 2024-09-10T04:05:23.209 INFO:tasks.workunit.client.0.smithi046.stdout:3/584: unlink d12/d13/d14/f47 0 2024-09-10T04:05:23.211 INFO:tasks.workunit.client.0.smithi046.stdout:9/542: dread d4/d29/f3d [0,4194304] 0 2024-09-10T04:05:23.213 INFO:tasks.workunit.client.0.smithi046.stdout:1/466: write d3/d27/d32/d1f/f55 [631946,120946] 0 2024-09-10T04:05:23.214 INFO:tasks.workunit.client.0.smithi046.stdout:9/543: chown d4/d3c/d4d/d80 252284 1 2024-09-10T04:05:23.219 INFO:tasks.workunit.client.0.smithi046.stdout:6/565: dread da/db/d11/d13/d1e/f4d [0,4194304] 0 2024-09-10T04:05:23.220 INFO:tasks.workunit.client.0.smithi046.stdout:6/566: chown da/db/d11/d2b/d5a/lb3 0 1 2024-09-10T04:05:23.222 INFO:tasks.workunit.client.0.smithi046.stdout:2/583: sync 2024-09-10T04:05:23.223 INFO:tasks.workunit.client.0.smithi046.stdout:7/604: mkdir d5/d9/d12/dc5 0 2024-09-10T04:05:23.223 INFO:tasks.workunit.client.0.smithi046.stdout:7/605: chown d5/d9/d43/f66 11913781 1 2024-09-10T04:05:23.224 INFO:tasks.workunit.client.1.smithi070.stdout:7/649: creat d2/d3/d63/fe0 x:0 0 0 2024-09-10T04:05:23.225 INFO:tasks.workunit.client.0.smithi046.stdout:7/606: write d5/d9/d43/f5c [2954506,17761] 0 2024-09-10T04:05:23.227 INFO:tasks.workunit.client.0.smithi046.stdout:5/682: read d7/d11/f1a [1061846,62057] 0 2024-09-10T04:05:23.228 INFO:tasks.workunit.client.1.smithi070.stdout:0/738: mknod d8/d13/de2/d6f/cf1 0 2024-09-10T04:05:23.229 INFO:tasks.workunit.client.1.smithi070.stdout:2/625: readlink d15/d2b/d6e/ld3 0 2024-09-10T04:05:23.231 INFO:tasks.workunit.client.1.smithi070.stdout:7/650: dwrite d2/d3/dc/d5d/d2d/f8e [0,4194304] 0 2024-09-10T04:05:23.231 INFO:tasks.workunit.client.0.smithi046.stdout:4/745: rename d3/d12/d17/d22/d30/dc7/df3/lfa to d3/d12/d17/de1/ddf/l104 0 2024-09-10T04:05:23.234 INFO:tasks.workunit.client.0.smithi046.stdout:1/467: mknod d3/dc/d34/d51/c98 0 2024-09-10T04:05:23.239 INFO:tasks.workunit.client.0.smithi046.stdout:9/544: creat d4/d18/d89/fa7 x:0 0 0 2024-09-10T04:05:23.241 INFO:tasks.workunit.client.0.smithi046.stdout:9/545: chown d4/d7/ca 1 1 2024-09-10T04:05:23.241 INFO:tasks.workunit.client.0.smithi046.stdout:6/567: truncate da/fe 756375 0 2024-09-10T04:05:23.242 INFO:tasks.workunit.client.1.smithi070.stdout:7/651: creat d2/dc5/fe1 x:0 0 0 2024-09-10T04:05:23.244 INFO:tasks.workunit.client.0.smithi046.stdout:3/585: dread d12/d13/d14/f24 [0,4194304] 0 2024-09-10T04:05:23.245 INFO:tasks.workunit.client.1.smithi070.stdout:0/739: dread d8/d24/f6d [0,4194304] 0 2024-09-10T04:05:23.247 INFO:tasks.workunit.client.0.smithi046.stdout:7/607: unlink d5/d7f/c9a 0 2024-09-10T04:05:23.248 INFO:tasks.workunit.client.1.smithi070.stdout:7/652: unlink d2/d40/d6c/lb3 0 2024-09-10T04:05:23.250 INFO:tasks.workunit.client.1.smithi070.stdout:2/626: link d15/d18/f1a d15/d18/d29/d2a/d2e/d32/d36/fdc 0 2024-09-10T04:05:23.250 INFO:tasks.workunit.client.0.smithi046.stdout:5/683: stat d7/d11/d3f/d53/da9/c6b 0 2024-09-10T04:05:23.251 INFO:tasks.workunit.client.1.smithi070.stdout:0/740: rmdir d8/d24/d36 39 2024-09-10T04:05:23.251 INFO:tasks.workunit.client.1.smithi070.stdout:0/741: stat d8/d13/d47/d7b/f8c 0 2024-09-10T04:05:23.253 INFO:tasks.workunit.client.1.smithi070.stdout:7/653: fsync d2/d3/dc/d5d/d41/f50 0 2024-09-10T04:05:23.255 INFO:tasks.workunit.client.0.smithi046.stdout:8/554: creat d1/d29/d19/d36/fb8 x:0 0 0 2024-09-10T04:05:23.255 INFO:tasks.workunit.client.0.smithi046.stdout:3/586: dwrite d12/d13/d37/d51/db5/d4b/d91/f9a [0,4194304] 0 2024-09-10T04:05:23.256 INFO:tasks.workunit.client.0.smithi046.stdout:2/584: mkdir d2/d6/d23/d42/db4/dca/dd6 0 2024-09-10T04:05:23.257 INFO:tasks.workunit.client.0.smithi046.stdout:8/555: fsync d1/d29/d17/d1f/d46/fa8 0 2024-09-10T04:05:23.259 INFO:tasks.workunit.client.1.smithi070.stdout:0/742: read d8/d13/d1a/f7e [3654220,8709] 0 2024-09-10T04:05:23.263 INFO:tasks.workunit.client.0.smithi046.stdout:7/608: creat d5/d6/db/d11/fc6 x:0 0 0 2024-09-10T04:05:23.263 INFO:tasks.workunit.client.1.smithi070.stdout:2/627: dread d15/d18/f1c [0,4194304] 0 2024-09-10T04:05:23.265 INFO:tasks.workunit.client.1.smithi070.stdout:2/628: truncate d15/d18/d29/d2a/d2e/d32/d36/f5b 4762609 0 2024-09-10T04:05:23.267 INFO:tasks.workunit.client.0.smithi046.stdout:4/746: mkdir d3/d12/d17/d105 0 2024-09-10T04:05:23.268 INFO:tasks.workunit.client.0.smithi046.stdout:0/452: getdents d1/dc/d25/d33/d3/dd 0 2024-09-10T04:05:23.269 INFO:tasks.workunit.client.1.smithi070.stdout:0/743: mknod d8/d24/d36/d87/cf2 0 2024-09-10T04:05:23.273 INFO:tasks.workunit.client.0.smithi046.stdout:2/585: dread - d2/d6/f4f zero size 2024-09-10T04:05:23.274 INFO:tasks.workunit.client.1.smithi070.stdout:2/629: dwrite d15/d18/d29/d2a/fbe [0,4194304] 0 2024-09-10T04:05:23.278 INFO:tasks.workunit.client.1.smithi070.stdout:0/744: dwrite d8/fa [0,4194304] 0 2024-09-10T04:05:23.278 INFO:tasks.workunit.client.1.smithi070.stdout:7/654: getdents d2/d3/dc/d5d/d41 0 2024-09-10T04:05:23.282 INFO:tasks.workunit.client.0.smithi046.stdout:8/556: rename d1/d29/d17/d1f/d46/d67/c7e to d1/d29/d19/d73/cb9 0 2024-09-10T04:05:23.285 INFO:tasks.workunit.client.0.smithi046.stdout:9/546: dwrite d4/f50 [0,4194304] 0 2024-09-10T04:05:23.290 INFO:tasks.workunit.client.0.smithi046.stdout:7/609: creat d5/d6/db/dd/d4b/fc7 x:0 0 0 2024-09-10T04:05:23.291 INFO:tasks.workunit.client.1.smithi070.stdout:0/745: creat d8/d24/d2d/d34/d6b/ff3 x:0 0 0 2024-09-10T04:05:23.292 INFO:tasks.workunit.client.1.smithi070.stdout:2/630: symlink d15/d18/d29/d2a/d2e/ldd 0 2024-09-10T04:05:23.293 INFO:tasks.workunit.client.1.smithi070.stdout:0/746: read d8/d13/de2/d53/dc6/fd9 [127766,21180] 0 2024-09-10T04:05:23.294 INFO:tasks.workunit.client.1.smithi070.stdout:7/655: creat d2/d7/d9/dd5/fe2 x:0 0 0 2024-09-10T04:05:23.295 INFO:tasks.workunit.client.1.smithi070.stdout:7/656: stat d2/d3/dc/d5d/l77 0 2024-09-10T04:05:23.296 INFO:tasks.workunit.client.0.smithi046.stdout:3/587: creat d12/d13/d37/d51/da7/fc5 x:0 0 0 2024-09-10T04:05:23.296 INFO:tasks.workunit.client.1.smithi070.stdout:7/657: readlink d2/d3/d63/da9/dc1/l8f 0 2024-09-10T04:05:23.298 INFO:tasks.workunit.client.1.smithi070.stdout:7/658: write d2/d3/dc/d79/fb9 [4472087,37961] 0 2024-09-10T04:05:23.299 INFO:tasks.workunit.client.1.smithi070.stdout:7/659: chown d2/d3/dc/d5d/d2d 480620 1 2024-09-10T04:05:23.299 INFO:tasks.workunit.client.1.smithi070.stdout:2/631: creat d15/d59/da1/fde x:0 0 0 2024-09-10T04:05:23.300 INFO:tasks.workunit.client.1.smithi070.stdout:3/676: sync 2024-09-10T04:05:23.300 INFO:tasks.workunit.client.1.smithi070.stdout:8/687: sync 2024-09-10T04:05:23.300 INFO:tasks.workunit.client.1.smithi070.stdout:1/582: sync 2024-09-10T04:05:23.300 INFO:tasks.workunit.client.1.smithi070.stdout:5/633: sync 2024-09-10T04:05:23.301 INFO:tasks.workunit.client.1.smithi070.stdout:1/583: readlink d3/d8/l51 0 2024-09-10T04:05:23.301 INFO:tasks.workunit.client.0.smithi046.stdout:2/586: rename d2/d6/c8a to d2/d6/d23/d5d/d5e/cd7 0 2024-09-10T04:05:23.301 INFO:tasks.workunit.client.1.smithi070.stdout:6/671: sync 2024-09-10T04:05:23.302 INFO:tasks.workunit.client.1.smithi070.stdout:9/700: sync 2024-09-10T04:05:23.302 INFO:tasks.workunit.client.1.smithi070.stdout:3/677: fdatasync db/d79/fa9 0 2024-09-10T04:05:23.302 INFO:tasks.workunit.client.0.smithi046.stdout:8/557: unlink d1/d29/d17/c84 0 2024-09-10T04:05:23.302 INFO:tasks.workunit.client.1.smithi070.stdout:0/747: write d8/f27 [5105363,21639] 0 2024-09-10T04:05:23.303 INFO:tasks.workunit.client.0.smithi046.stdout:5/684: creat d7/ff9 x:0 0 0 2024-09-10T04:05:23.303 INFO:tasks.workunit.client.1.smithi070.stdout:2/632: write d15/d18/d29/d2a/fc4 [2217283,97663] 0 2024-09-10T04:05:23.306 INFO:tasks.workunit.client.1.smithi070.stdout:7/660: rename d2/d4e/fcb to d2/d40/d6c/db2/fe3 0 2024-09-10T04:05:23.307 INFO:tasks.workunit.client.1.smithi070.stdout:7/661: chown d2/d40/d6c/db2/db7 43022996 1 2024-09-10T04:05:23.309 INFO:tasks.workunit.client.0.smithi046.stdout:3/588: mknod d12/d13/d14/d1e/d2d/d39/d67/d92/cc6 0 2024-09-10T04:05:23.310 INFO:tasks.workunit.client.0.smithi046.stdout:5/685: dwrite d7/d11/d14/d45/f4b [0,4194304] 0 2024-09-10T04:05:23.313 INFO:tasks.workunit.client.1.smithi070.stdout:9/701: creat d9/d1c/dac/fe2 x:0 0 0 2024-09-10T04:05:23.314 INFO:tasks.workunit.client.0.smithi046.stdout:6/568: link da/d71/la9 da/db/d18/lbb 0 2024-09-10T04:05:23.316 INFO:tasks.workunit.client.1.smithi070.stdout:0/748: dread d8/d24/d36/f48 [0,4194304] 0 2024-09-10T04:05:23.317 INFO:tasks.workunit.client.1.smithi070.stdout:0/749: readlink d8/d24/lbd 0 2024-09-10T04:05:23.318 INFO:tasks.workunit.client.0.smithi046.stdout:2/587: write d2/d49/f9b [34268,112964] 0 2024-09-10T04:05:23.318 INFO:tasks.workunit.client.1.smithi070.stdout:0/750: chown d8/d24/d2d/d34/d9b/cb1 2863 1 2024-09-10T04:05:23.319 INFO:tasks.workunit.client.1.smithi070.stdout:0/751: dread - d8/d13/d47/d7b/fd8 zero size 2024-09-10T04:05:23.321 INFO:tasks.workunit.client.0.smithi046.stdout:9/547: mkdir d4/d18/da8 0 2024-09-10T04:05:23.323 INFO:tasks.workunit.client.1.smithi070.stdout:1/584: rename d3/d8/d12/d82 to d3/d10/d1d/dbd 0 2024-09-10T04:05:23.326 INFO:tasks.workunit.client.1.smithi070.stdout:5/634: mknod d17/d19/d6a/d54/cd9 0 2024-09-10T04:05:23.329 INFO:tasks.workunit.client.1.smithi070.stdout:7/662: unlink d2/d7/d9/d18/f6b 0 2024-09-10T04:05:23.330 INFO:tasks.workunit.client.1.smithi070.stdout:7/663: chown d2/d3/d63/l72 498 1 2024-09-10T04:05:23.330 INFO:tasks.workunit.client.0.smithi046.stdout:5/686: symlink d7/d11/d14/d8e/d96/lfa 0 2024-09-10T04:05:23.331 INFO:tasks.workunit.client.1.smithi070.stdout:7/664: dread - d2/d7/d9/d18/d3c/d52/fc9 zero size 2024-09-10T04:05:23.333 INFO:tasks.workunit.client.0.smithi046.stdout:2/588: mkdir d2/d6/d23/d5d/d77/d7b/dd8 0 2024-09-10T04:05:23.334 INFO:tasks.workunit.client.1.smithi070.stdout:7/665: truncate d2/d40/d6c/db2/db7/fc0 695567 0 2024-09-10T04:05:23.335 INFO:tasks.workunit.client.1.smithi070.stdout:9/702: symlink d9/d59/d2a/d3f/le3 0 2024-09-10T04:05:23.337 INFO:tasks.workunit.client.0.smithi046.stdout:0/453: truncate d1/d13/d37/d75/f7f 887427 0 2024-09-10T04:05:23.338 INFO:tasks.workunit.client.0.smithi046.stdout:9/548: creat d4/d18/d89/fa9 x:0 0 0 2024-09-10T04:05:23.339 INFO:tasks.workunit.client.1.smithi070.stdout:5/635: dread fc [0,4194304] 0 2024-09-10T04:05:23.340 INFO:tasks.workunit.client.0.smithi046.stdout:3/589: unlink d12/d13/d14/d1e/d2d/f80 0 2024-09-10T04:05:23.341 INFO:tasks.workunit.client.0.smithi046.stdout:4/747: dwrite d3/d12/f9a [0,4194304] 0 2024-09-10T04:05:23.342 INFO:tasks.workunit.client.1.smithi070.stdout:3/678: write db/d1c/d3e/f54 [551777,80976] 0 2024-09-10T04:05:23.343 INFO:tasks.workunit.client.1.smithi070.stdout:0/752: mkdir d8/d24/d2d/d34/d9b/dad/df4 0 2024-09-10T04:05:23.344 INFO:tasks.workunit.client.0.smithi046.stdout:8/558: write d1/d29/f27 [1964907,112278] 0 2024-09-10T04:05:23.345 INFO:tasks.workunit.client.0.smithi046.stdout:2/589: symlink d2/d8/d37/d63/ld9 0 2024-09-10T04:05:23.349 INFO:tasks.workunit.client.0.smithi046.stdout:5/687: dwrite d7/d11/f13 [0,4194304] 0 2024-09-10T04:05:23.350 INFO:tasks.workunit.client.0.smithi046.stdout:9/549: rmdir d4/d44 39 2024-09-10T04:05:23.351 INFO:tasks.workunit.client.0.smithi046.stdout:5/688: dread - d7/d11/d3f/d53/d81/fe8 zero size 2024-09-10T04:05:23.354 INFO:tasks.workunit.client.0.smithi046.stdout:5/689: write d7/d11/d3f/d53/f47 [1049528,73889] 0 2024-09-10T04:05:23.355 INFO:tasks.workunit.client.0.smithi046.stdout:0/454: rename d1/dc/d25/d33/d4/f60 to d1/d13/f8c 0 2024-09-10T04:05:23.357 INFO:tasks.workunit.client.0.smithi046.stdout:3/590: creat d12/d13/fc7 x:0 0 0 2024-09-10T04:05:23.359 INFO:tasks.workunit.client.0.smithi046.stdout:3/591: chown d12/d13/d37/f45 328279 1 2024-09-10T04:05:23.360 INFO:tasks.workunit.client.1.smithi070.stdout:6/672: dwrite d0/d3/d18/d62/f85 [0,4194304] 0 2024-09-10T04:05:23.361 INFO:tasks.workunit.client.0.smithi046.stdout:3/592: chown d12/d13/d37/l34 49 1 2024-09-10T04:05:23.364 INFO:tasks.workunit.client.1.smithi070.stdout:7/666: creat d2/d3/d63/da9/dc1/fe4 x:0 0 0 2024-09-10T04:05:23.364 INFO:tasks.workunit.client.0.smithi046.stdout:0/455: dwrite d1/f35 [0,4194304] 0 2024-09-10T04:05:23.364 INFO:tasks.workunit.client.0.smithi046.stdout:4/748: dread - d3/d12/d17/d29/d58/db2/d5f/fb4 zero size 2024-09-10T04:05:23.366 INFO:tasks.workunit.client.0.smithi046.stdout:0/456: write d1/dc/f27 [1703330,117038] 0 2024-09-10T04:05:23.367 INFO:tasks.workunit.client.1.smithi070.stdout:9/703: rename d9/d1c/dac/d8a/d1b/f5c to d9/d59/d2a/d3f/fe4 0 2024-09-10T04:05:23.368 INFO:tasks.workunit.client.0.smithi046.stdout:4/749: chown d3/d12/d17/d29/d58 583315 1 2024-09-10T04:05:23.369 INFO:tasks.workunit.client.1.smithi070.stdout:9/704: chown d9/d1c/dac/d8a/d1b/lc2 902580149 1 2024-09-10T04:05:23.369 INFO:tasks.workunit.client.0.smithi046.stdout:4/750: chown d3/dab/dca/de8 15 1 2024-09-10T04:05:23.369 INFO:tasks.workunit.client.0.smithi046.stdout:8/559: fsync d1/f2e 0 2024-09-10T04:05:23.372 INFO:tasks.workunit.client.0.smithi046.stdout:2/590: mknod d2/d6/d23/d5d/d77/d7b/d85/cda 0 2024-09-10T04:05:23.382 INFO:tasks.workunit.client.0.smithi046.stdout:9/550: rename d4/d3c/d4d/d5c/f6a to d4/d3c/d4d/d5c/d87/faa 0 2024-09-10T04:05:23.382 INFO:tasks.workunit.client.0.smithi046.stdout:0/457: dread d1/dc/d25/d33/d1a/f31 [0,4194304] 0 2024-09-10T04:05:23.389 INFO:tasks.workunit.client.1.smithi070.stdout:0/753: rename d8/d24/d36/d87 to d8/d85/df5 0 2024-09-10T04:05:23.390 INFO:tasks.workunit.client.1.smithi070.stdout:1/585: truncate d3/d8/d12/f45 1172661 0 2024-09-10T04:05:23.390 INFO:tasks.workunit.client.0.smithi046.stdout:5/690: write d7/d11/d3f/d5a/d63/f7f [3566123,29962] 0 2024-09-10T04:05:23.391 INFO:tasks.workunit.client.1.smithi070.stdout:9/705: rmdir d9/d1c/d34 39 2024-09-10T04:05:23.391 INFO:tasks.workunit.client.1.smithi070.stdout:0/754: truncate d8/d13/de2/d81/fda 86299 0 2024-09-10T04:05:23.391 INFO:tasks.workunit.client.0.smithi046.stdout:5/691: chown d7/d11/d3f/d5a/fb2 181787 1 2024-09-10T04:05:23.394 INFO:tasks.workunit.client.1.smithi070.stdout:2/633: getdents d15/d59/da1 0 2024-09-10T04:05:23.394 INFO:tasks.workunit.client.0.smithi046.stdout:2/591: symlink d2/d8/d65/ldb 0 2024-09-10T04:05:23.396 INFO:tasks.workunit.client.1.smithi070.stdout:3/679: creat db/d1c/d25/dd8/fe8 x:0 0 0 2024-09-10T04:05:23.397 INFO:tasks.workunit.client.1.smithi070.stdout:0/755: dwrite d8/d13/fe8 [0,4194304] 0 2024-09-10T04:05:23.398 INFO:tasks.workunit.client.1.smithi070.stdout:7/667: rename d2/d3/dc/d5d/fd2 to d2/d7/d9/d18/d23/fe5 0 2024-09-10T04:05:23.398 INFO:tasks.workunit.client.1.smithi070.stdout:3/680: dread - db/d1c/d89/f9f zero size 2024-09-10T04:05:23.399 INFO:tasks.workunit.client.1.smithi070.stdout:1/586: mkdir d3/d8/d13/d3f/d6d/dbe 0 2024-09-10T04:05:23.399 INFO:tasks.workunit.client.1.smithi070.stdout:6/673: dwrite d0/d3/d18/d1b/d17/d28/f3b [0,4194304] 0 2024-09-10T04:05:23.399 INFO:tasks.workunit.client.1.smithi070.stdout:0/756: dread - d8/d13/de2/d6f/fec zero size 2024-09-10T04:05:23.401 INFO:tasks.workunit.client.1.smithi070.stdout:3/681: chown db/d1c/d25/c38 27653189 1 2024-09-10T04:05:23.402 INFO:tasks.workunit.client.0.smithi046.stdout:2/592: dwrite d2/d6/d23/d5d/d5e/f84 [0,4194304] 0 2024-09-10T04:05:23.404 INFO:tasks.workunit.client.1.smithi070.stdout:0/757: readlink d8/d13/de2/l40 0 2024-09-10T04:05:23.406 INFO:tasks.workunit.client.1.smithi070.stdout:2/634: mknod d15/d2b/cdf 0 2024-09-10T04:05:23.407 INFO:tasks.workunit.client.1.smithi070.stdout:2/635: chown ce 95984 1 2024-09-10T04:05:23.407 INFO:tasks.workunit.client.0.smithi046.stdout:4/751: truncate d3/fa 113507 0 2024-09-10T04:05:23.408 INFO:tasks.workunit.client.0.smithi046.stdout:4/752: readlink d3/d12/d17/d22/d51/d5c/l78 0 2024-09-10T04:05:23.413 INFO:tasks.workunit.client.1.smithi070.stdout:3/682: dwrite db/d1c/d89/f9f [0,4194304] 0 2024-09-10T04:05:23.415 INFO:tasks.workunit.client.1.smithi070.stdout:4/590: sync 2024-09-10T04:05:23.418 INFO:tasks.workunit.client.1.smithi070.stdout:2/636: dread d15/d18/d29/d2a/d2e/d32/d36/f5b [0,4194304] 0 2024-09-10T04:05:23.418 INFO:tasks.workunit.client.0.smithi046.stdout:8/560: link d1/d29/d17/d1f/d46/d6a/d82/d99/fb3 d1/d29/d19/d73/fba 0 2024-09-10T04:05:23.420 INFO:tasks.workunit.client.0.smithi046.stdout:8/561: read d1/d29/d19/d36/d4c/d5e/fa1 [79124,25083] 0 2024-09-10T04:05:23.423 INFO:tasks.workunit.client.1.smithi070.stdout:5/636: getdents d17/d24/d84/d9b 0 2024-09-10T04:05:23.425 INFO:tasks.workunit.client.1.smithi070.stdout:9/706: rmdir d9/d1c/d34/d8b 39 2024-09-10T04:05:23.426 INFO:tasks.workunit.client.0.smithi046.stdout:0/458: fsync d1/dc/d25/d33/d3/dd/d55/d4d/f4e 0 2024-09-10T04:05:23.426 INFO:tasks.workunit.client.1.smithi070.stdout:4/591: dwrite f8 [0,4194304] 0 2024-09-10T04:05:23.428 INFO:tasks.workunit.client.1.smithi070.stdout:0/758: chown d8/d13/d1a/f3c 575115471 1 2024-09-10T04:05:23.429 INFO:tasks.workunit.client.1.smithi070.stdout:1/587: truncate d3/d8/d12/d2f/d32/f41 2860725 0 2024-09-10T04:05:23.429 INFO:tasks.workunit.client.1.smithi070.stdout:2/637: dwrite d15/d18/d29/f7f [0,4194304] 0 2024-09-10T04:05:23.430 INFO:tasks.workunit.client.0.smithi046.stdout:2/593: mknod d2/d6/d23/d5d/d77/d7b/d85/d8d/cdc 0 2024-09-10T04:05:23.432 INFO:tasks.workunit.client.0.smithi046.stdout:4/753: mkdir d3/d12/dec/d106 0 2024-09-10T04:05:23.434 INFO:tasks.workunit.client.0.smithi046.stdout:4/754: read d3/fc [9512952,11836] 0 2024-09-10T04:05:23.437 INFO:tasks.workunit.client.1.smithi070.stdout:1/588: dwrite d3/d8/f2d [0,4194304] 0 2024-09-10T04:05:23.438 INFO:tasks.workunit.client.0.smithi046.stdout:5/692: dwrite d7/d11/d14/d2f/d2b/d92/f9a [0,4194304] 0 2024-09-10T04:05:23.440 INFO:tasks.workunit.client.1.smithi070.stdout:3/683: mknod db/d79/db6/dc3/ce9 0 2024-09-10T04:05:23.441 INFO:tasks.workunit.client.0.smithi046.stdout:8/562: rmdir d1/d29/d19/d36/d6e 39 2024-09-10T04:05:23.443 INFO:tasks.workunit.client.1.smithi070.stdout:6/674: dwrite d0/d3/d18/d1b/d17/d53/f61 [0,4194304] 0 2024-09-10T04:05:23.447 INFO:tasks.workunit.client.1.smithi070.stdout:9/707: dread d9/d7a/d74/fa5 [0,4194304] 0 2024-09-10T04:05:23.449 INFO:tasks.workunit.client.0.smithi046.stdout:4/755: dwrite d3/d12/d17/d29/d58/db2/d5f/d85/da8/fce [0,4194304] 0 2024-09-10T04:05:23.453 INFO:tasks.workunit.client.0.smithi046.stdout:3/593: getdents d12/d13/d37/d51/db5/d4b/db7 0 2024-09-10T04:05:23.454 INFO:tasks.workunit.client.0.smithi046.stdout:2/594: dread - d2/d6/d23/d55/fa3 zero size 2024-09-10T04:05:23.457 INFO:tasks.workunit.client.1.smithi070.stdout:5/637: unlink d17/d19/d6a/d54/d57/da3/l78 0 2024-09-10T04:05:23.461 INFO:tasks.workunit.client.0.smithi046.stdout:5/693: creat d7/d11/d14/d45/ffb x:0 0 0 2024-09-10T04:05:23.464 INFO:tasks.workunit.client.1.smithi070.stdout:7/668: truncate d2/d4e/f9b 987018 0 2024-09-10T04:05:23.465 INFO:tasks.workunit.client.1.smithi070.stdout:5/638: dwrite d17/d19/d6a/f7c [4194304,4194304] 0 2024-09-10T04:05:23.465 INFO:tasks.workunit.client.1.smithi070.stdout:3/684: dread db/d79/fa8 [0,4194304] 0 2024-09-10T04:05:23.466 INFO:tasks.workunit.client.1.smithi070.stdout:5/639: readlink d17/d19/ld0 0 2024-09-10T04:05:23.467 INFO:tasks.workunit.client.1.smithi070.stdout:3/685: chown db/d1c/d25/d28/d43/l6c 12315813 1 2024-09-10T04:05:23.468 INFO:tasks.workunit.client.1.smithi070.stdout:0/759: rmdir d8/d13/d1a/da0 39 2024-09-10T04:05:23.468 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:23 smithi046 ceph-mon[28328]: pgmap v265: 65 pgs: 65 active+clean; 2.3 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 76 MiB/s rd, 283 MiB/s wr, 489 op/s 2024-09-10T04:05:23.468 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:23 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:23.468 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:23 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:23.468 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:23 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:05:23.470 INFO:tasks.workunit.client.0.smithi046.stdout:9/551: getdents d4/d29/d32 0 2024-09-10T04:05:23.472 INFO:tasks.workunit.client.1.smithi070.stdout:2/638: write d15/d18/d29/d2a/d2e/d32/d36/f5b [50115,65861] 0 2024-09-10T04:05:23.473 INFO:tasks.workunit.client.1.smithi070.stdout:1/589: mknod d3/d8/d27/d9a/cbf 0 2024-09-10T04:05:23.474 INFO:tasks.workunit.client.1.smithi070.stdout:5/640: dwrite d17/d19/fd1 [0,4194304] 0 2024-09-10T04:05:23.476 INFO:tasks.workunit.client.0.smithi046.stdout:3/594: unlink d12/d13/d14/d1e/d2d/d6f/c82 0 2024-09-10T04:05:23.478 INFO:tasks.workunit.client.1.smithi070.stdout:6/675: mknod d0/d3/d5c/cd5 0 2024-09-10T04:05:23.482 INFO:tasks.workunit.client.0.smithi046.stdout:5/694: creat d7/d11/ffc x:0 0 0 2024-09-10T04:05:23.483 INFO:tasks.workunit.client.0.smithi046.stdout:8/563: truncate d1/d29/d12/f45 3110208 0 2024-09-10T04:05:23.486 INFO:tasks.workunit.client.1.smithi070.stdout:4/592: mknod de/d13/d18/d5c/cca 0 2024-09-10T04:05:23.492 INFO:tasks.workunit.client.0.smithi046.stdout:3/595: symlink d12/d13/d37/d51/db5/d4b/d91/lc8 0 2024-09-10T04:05:23.492 INFO:tasks.workunit.client.0.smithi046.stdout:8/564: dwrite d1/d29/d19/d36/d4c/fa9 [0,4194304] 0 2024-09-10T04:05:23.493 INFO:tasks.workunit.client.1.smithi070.stdout:7/669: fdatasync d2/d3/dc/d79/f9d 0 2024-09-10T04:05:23.495 INFO:tasks.workunit.client.0.smithi046.stdout:5/695: dread - d7/f94 zero size 2024-09-10T04:05:23.496 INFO:tasks.workunit.client.0.smithi046.stdout:5/696: fdatasync d7/f43 0 2024-09-10T04:05:23.499 INFO:tasks.workunit.client.1.smithi070.stdout:0/760: rmdir d8/d85/df5 39 2024-09-10T04:05:23.500 INFO:tasks.workunit.client.1.smithi070.stdout:9/708: write d9/d7a/d72/f7b [1740938,25068] 0 2024-09-10T04:05:23.501 INFO:tasks.workunit.client.0.smithi046.stdout:9/552: creat d4/d29/d32/d95/fab x:0 0 0 2024-09-10T04:05:23.504 INFO:tasks.workunit.client.1.smithi070.stdout:2/639: rmdir d15/d2b/d6e/db8 39 2024-09-10T04:05:23.506 INFO:tasks.workunit.client.1.smithi070.stdout:0/761: dwrite d8/d13/de2/f65 [0,4194304] 0 2024-09-10T04:05:23.507 INFO:tasks.workunit.client.0.smithi046.stdout:3/596: fdatasync d12/d13/d14/d1e/f3f 0 2024-09-10T04:05:23.508 INFO:tasks.workunit.client.1.smithi070.stdout:0/762: fsync d8/d24/d2d/fd1 0 2024-09-10T04:05:23.509 INFO:tasks.workunit.client.1.smithi070.stdout:5/641: symlink d17/d19/d6a/d54/d57/d63/db9/lda 0 2024-09-10T04:05:23.510 INFO:tasks.workunit.client.0.smithi046.stdout:2/595: link d2/d6/d23/d55/d9f/cd2 d2/d8/d37/dc2/cdd 0 2024-09-10T04:05:23.515 INFO:tasks.workunit.client.1.smithi070.stdout:1/590: dread d3/d8/d13/f43 [0,4194304] 0 2024-09-10T04:05:23.519 INFO:tasks.workunit.client.0.smithi046.stdout:4/756: getdents d3/d12/d17/d29/d58/db2/d5f/d85 0 2024-09-10T04:05:23.521 INFO:tasks.workunit.client.1.smithi070.stdout:4/593: creat de/d13/d18/d23/d47/d59/fcb x:0 0 0 2024-09-10T04:05:23.521 INFO:tasks.workunit.client.1.smithi070.stdout:1/591: dwrite d3/d8/d27/f68 [0,4194304] 0 2024-09-10T04:05:23.524 INFO:tasks.workunit.client.0.smithi046.stdout:3/597: dread d12/d13/d14/f15 [0,4194304] 0 2024-09-10T04:05:23.524 INFO:tasks.workunit.client.1.smithi070.stdout:8/688: sync 2024-09-10T04:05:23.525 INFO:tasks.workunit.client.0.smithi046.stdout:9/553: write d4/d18/f25 [1930204,124157] 0 2024-09-10T04:05:23.526 INFO:tasks.workunit.client.0.smithi046.stdout:3/598: write d12/d13/d37/d94/fa2 [568818,68810] 0 2024-09-10T04:05:23.526 INFO:tasks.workunit.client.1.smithi070.stdout:4/594: truncate de/d13/d18/d23/d39/d77/fb1 218859 0 2024-09-10T04:05:23.529 INFO:tasks.workunit.client.0.smithi046.stdout:2/596: fsync d2/d6/f43 0 2024-09-10T04:05:23.533 INFO:tasks.workunit.client.1.smithi070.stdout:2/640: truncate d15/d2b/f7a 30408 0 2024-09-10T04:05:23.534 INFO:tasks.workunit.client.1.smithi070.stdout:0/763: dread d8/d24/f43 [0,4194304] 0 2024-09-10T04:05:23.534 INFO:tasks.workunit.client.1.smithi070.stdout:2/641: dread - d15/d18/d56/f86 zero size 2024-09-10T04:05:23.537 INFO:tasks.workunit.client.1.smithi070.stdout:6/676: symlink d0/d3/d8c/dab/ld6 0 2024-09-10T04:05:23.542 INFO:tasks.workunit.client.1.smithi070.stdout:1/592: creat d3/d8/d12/d61/fc0 x:0 0 0 2024-09-10T04:05:23.542 INFO:tasks.workunit.client.1.smithi070.stdout:6/677: dwrite d0/d3/d18/d62/db8/fd0 [0,4194304] 0 2024-09-10T04:05:23.543 INFO:tasks.workunit.client.0.smithi046.stdout:4/757: write d3/f3c [774296,16349] 0 2024-09-10T04:05:23.543 INFO:tasks.workunit.client.0.smithi046.stdout:3/599: write d12/d13/d37/faa [1111775,39917] 0 2024-09-10T04:05:23.546 INFO:tasks.workunit.client.0.smithi046.stdout:5/697: link d7/f43 d7/ffd 0 2024-09-10T04:05:23.548 INFO:tasks.workunit.client.1.smithi070.stdout:7/670: mkdir d2/d3/dc/d5d/d41/dd6/de6 0 2024-09-10T04:05:23.550 INFO:tasks.workunit.client.1.smithi070.stdout:3/686: creat db/fea x:0 0 0 2024-09-10T04:05:23.551 INFO:tasks.workunit.client.0.smithi046.stdout:2/597: dread d2/d6/d23/d5d/d77/f54 [0,4194304] 0 2024-09-10T04:05:23.553 INFO:tasks.workunit.client.0.smithi046.stdout:3/600: dwrite d12/d13/d37/d51/db5/d50/d52/f65 [0,4194304] 0 2024-09-10T04:05:23.554 INFO:tasks.workunit.client.1.smithi070.stdout:5/642: dwrite d17/d19/d35/f3a [0,4194304] 0 2024-09-10T04:05:23.555 INFO:tasks.workunit.client.1.smithi070.stdout:5/643: stat d17/d24/d84/lb3 0 2024-09-10T04:05:23.556 INFO:tasks.workunit.client.0.smithi046.stdout:9/554: write d4/d7/d1f/f15 [1893382,60414] 0 2024-09-10T04:05:23.558 INFO:tasks.workunit.client.0.smithi046.stdout:4/758: mkdir d3/d12/d17/d22/d51/d94/d107 0 2024-09-10T04:05:23.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/709: symlink d9/d1c/d34/d8b/dcc/le5 0 2024-09-10T04:05:23.559 INFO:tasks.workunit.client.1.smithi070.stdout:5/644: write d17/d24/d84/dab/fb8 [232180,103635] 0 2024-09-10T04:05:23.562 INFO:tasks.workunit.client.1.smithi070.stdout:3/687: dwrite db/d1c/d53/fa2 [0,4194304] 0 2024-09-10T04:05:23.563 INFO:tasks.workunit.client.1.smithi070.stdout:0/764: mkdir d8/d24/d2d/d5f/df6 0 2024-09-10T04:05:23.563 INFO:tasks.workunit.client.0.smithi046.stdout:8/565: truncate d1/d29/d12/f45 484713 0 2024-09-10T04:05:23.569 INFO:tasks.workunit.client.0.smithi046.stdout:2/598: creat d2/d6/d9d/db6/dbe/fde x:0 0 0 2024-09-10T04:05:23.569 INFO:tasks.workunit.client.0.smithi046.stdout:2/599: readlink d2/d6/l14 0 2024-09-10T04:05:23.571 INFO:tasks.workunit.client.0.smithi046.stdout:5/698: chown d7/d11/d14/d2f/deb/da5/dcc 789211 1 2024-09-10T04:05:23.571 INFO:tasks.workunit.client.0.smithi046.stdout:2/600: write d2/d6/d15/fbb [247928,30626] 0 2024-09-10T04:05:23.573 INFO:tasks.workunit.client.0.smithi046.stdout:8/566: mkdir d1/d29/d17/d1f/d46/d6a/dbb 0 2024-09-10T04:05:23.576 INFO:tasks.workunit.client.1.smithi070.stdout:6/678: creat d0/d3/d18/d1b/d17/d3f/d40/fd7 x:0 0 0 2024-09-10T04:05:23.576 INFO:tasks.workunit.client.1.smithi070.stdout:8/689: rename d4/d16/d7d/fb4 to d4/d16/d73/d96/fe7 0 2024-09-10T04:05:23.577 INFO:tasks.workunit.client.1.smithi070.stdout:6/679: chown d0/d3/d8c/d9a/fac 37645 1 2024-09-10T04:05:23.577 INFO:tasks.workunit.client.0.smithi046.stdout:5/699: readlink d7/d11/d3f/d53/l6c 0 2024-09-10T04:05:23.579 INFO:tasks.workunit.client.0.smithi046.stdout:2/601: mknod d2/d8/d65/d90/cdf 0 2024-09-10T04:05:23.582 INFO:tasks.workunit.client.1.smithi070.stdout:9/710: mknod d9/d7a/d74/d8c/ce6 0 2024-09-10T04:05:23.582 INFO:tasks.workunit.client.1.smithi070.stdout:5/645: fdatasync d17/d19/fc3 0 2024-09-10T04:05:23.582 INFO:tasks.workunit.client.0.smithi046.stdout:4/759: rename d3/d12/d17/d22/d51/d5c/fc8 to d3/d12/d17/d29/d58/db2/f108 0 2024-09-10T04:05:23.583 INFO:tasks.workunit.client.1.smithi070.stdout:9/711: fsync d9/f68 0 2024-09-10T04:05:23.584 INFO:tasks.workunit.client.0.smithi046.stdout:5/700: unlink d7/f4f 0 2024-09-10T04:05:23.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:23 smithi070 ceph-mon[35495]: pgmap v265: 65 pgs: 65 active+clean; 2.3 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 76 MiB/s rd, 283 MiB/s wr, 489 op/s 2024-09-10T04:05:23.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:23 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:23.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:23 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:23.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:23 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:05:23.585 INFO:tasks.workunit.client.0.smithi046.stdout:8/567: dwrite d1/d29/d19/d36/d4c/d5e/f7f [0,4194304] 0 2024-09-10T04:05:23.586 INFO:tasks.workunit.client.0.smithi046.stdout:8/568: stat d1/f86 0 2024-09-10T04:05:23.586 INFO:tasks.workunit.client.1.smithi070.stdout:3/688: mknod db/d83/ceb 0 2024-09-10T04:05:23.587 INFO:tasks.workunit.client.0.smithi046.stdout:9/555: dread d4/d66/da6/f8b [0,4194304] 0 2024-09-10T04:05:23.592 INFO:tasks.workunit.client.0.smithi046.stdout:4/760: write d3/d12/d17/d29/d58/ffe [942662,21839] 0 2024-09-10T04:05:23.595 INFO:tasks.workunit.client.1.smithi070.stdout:7/671: symlink d2/d7/db/d3a/le7 0 2024-09-10T04:05:23.598 INFO:tasks.workunit.client.1.smithi070.stdout:8/690: readlink d4/d16/d17/d1a/l35 0 2024-09-10T04:05:23.600 INFO:tasks.workunit.client.0.smithi046.stdout:8/569: chown d1/d29/d19/d36/d6e/d75/c9e 0 1 2024-09-10T04:05:23.602 INFO:tasks.workunit.client.1.smithi070.stdout:8/691: dread d4/dbf/d74/da4/fbd [0,4194304] 0 2024-09-10T04:05:23.606 INFO:tasks.workunit.client.1.smithi070.stdout:8/692: dread d4/d16/d17/d1a/fa9 [0,4194304] 0 2024-09-10T04:05:23.607 INFO:tasks.workunit.client.1.smithi070.stdout:4/595: truncate de/d13/d18/f21 391048 0 2024-09-10T04:05:23.608 INFO:tasks.workunit.client.1.smithi070.stdout:9/712: mkdir d9/d1c/d34/d8b/dcc/de7 0 2024-09-10T04:05:23.609 INFO:tasks.workunit.client.0.smithi046.stdout:4/761: dread d3/d12/d17/d22/d51/d94/fd6 [0,4194304] 0 2024-09-10T04:05:23.614 INFO:tasks.workunit.client.1.smithi070.stdout:3/689: rename db/d1c/d25/cad to db/d79/db6/dc3/cec 0 2024-09-10T04:05:23.619 INFO:tasks.workunit.client.0.smithi046.stdout:4/762: readlink d3/d12/d17/de1/ddf/l104 0 2024-09-10T04:05:23.619 INFO:tasks.workunit.client.0.smithi046.stdout:2/602: truncate d2/d8/fb2 417217 0 2024-09-10T04:05:23.622 INFO:tasks.workunit.client.0.smithi046.stdout:8/570: mknod d1/d29/d12/cbc 0 2024-09-10T04:05:23.625 INFO:tasks.workunit.client.1.smithi070.stdout:6/680: fsync d0/d3/d18/d1b/d17/d28/f76 0 2024-09-10T04:05:23.626 INFO:tasks.workunit.client.1.smithi070.stdout:0/765: truncate d8/d13/de2/d71/fed 965317 0 2024-09-10T04:05:23.627 INFO:tasks.workunit.client.0.smithi046.stdout:5/701: getdents d7 0 2024-09-10T04:05:23.630 INFO:tasks.workunit.client.1.smithi070.stdout:5/646: symlink d17/d24/dcc/ldb 0 2024-09-10T04:05:23.631 INFO:tasks.workunit.client.1.smithi070.stdout:8/693: creat d4/d16/dcc/fe8 x:0 0 0 2024-09-10T04:05:23.632 INFO:tasks.workunit.client.1.smithi070.stdout:8/694: dread - d4/d16/f34 zero size 2024-09-10T04:05:23.634 INFO:tasks.workunit.client.1.smithi070.stdout:4/596: truncate de/d13/f1a 1459138 0 2024-09-10T04:05:23.635 INFO:tasks.workunit.client.1.smithi070.stdout:4/597: stat de/d13/d18/d23/d47/fa7 0 2024-09-10T04:05:23.636 INFO:tasks.workunit.client.0.smithi046.stdout:9/556: link d4/d7/d1f/c7b d4/d29/d32/cac 0 2024-09-10T04:05:23.637 INFO:tasks.workunit.client.1.smithi070.stdout:4/598: chown de/d13/d43/f65 3785097 1 2024-09-10T04:05:23.638 INFO:tasks.workunit.client.1.smithi070.stdout:2/642: getdents d15/d18/d29/d2a/d2e 0 2024-09-10T04:05:23.639 INFO:tasks.workunit.client.0.smithi046.stdout:5/702: unlink d7/d11/d14/d45/ff2 0 2024-09-10T04:05:23.640 INFO:tasks.workunit.client.1.smithi070.stdout:8/695: dwrite d4/d16/f3f [4194304,4194304] 0 2024-09-10T04:05:23.640 INFO:tasks.workunit.client.0.smithi046.stdout:8/571: read d1/d29/d12/f45 [380177,61143] 0 2024-09-10T04:05:23.642 INFO:tasks.workunit.client.1.smithi070.stdout:1/593: getdents d3/d8/d13/d3f/d5d 0 2024-09-10T04:05:23.643 INFO:tasks.workunit.client.0.smithi046.stdout:2/603: write d2/f35 [470816,8698] 0 2024-09-10T04:05:23.644 INFO:tasks.workunit.client.1.smithi070.stdout:7/672: creat d2/dc8/fe8 x:0 0 0 2024-09-10T04:05:23.648 INFO:tasks.workunit.client.0.smithi046.stdout:9/557: write d4/d18/f38 [1913702,15276] 0 2024-09-10T04:05:23.650 INFO:tasks.workunit.client.1.smithi070.stdout:9/713: write d9/d7a/d74/f5f [989754,109763] 0 2024-09-10T04:05:23.652 INFO:tasks.workunit.client.1.smithi070.stdout:3/690: write db/f11 [930839,51632] 0 2024-09-10T04:05:23.652 INFO:tasks.workunit.client.0.smithi046.stdout:2/604: dwrite d2/d6/d23/d5d/d5e/f84 [0,4194304] 0 2024-09-10T04:05:23.654 INFO:tasks.workunit.client.0.smithi046.stdout:5/703: creat d7/d11/d3f/ffe x:0 0 0 2024-09-10T04:05:23.656 INFO:tasks.workunit.client.0.smithi046.stdout:4/763: mknod d3/d12/d17/d91/dbf/c109 0 2024-09-10T04:05:23.659 INFO:tasks.workunit.client.0.smithi046.stdout:2/605: fsync d2/d6/d23/d5d/d77/f54 0 2024-09-10T04:05:23.660 INFO:tasks.workunit.client.1.smithi070.stdout:5/647: creat d17/d19/d6a/d54/d57/da8/fdc x:0 0 0 2024-09-10T04:05:23.661 INFO:tasks.workunit.client.0.smithi046.stdout:5/704: fsync d7/d11/d3f/d53/fc7 0 2024-09-10T04:05:23.661 INFO:tasks.workunit.client.0.smithi046.stdout:5/705: dread - d7/ff9 zero size 2024-09-10T04:05:23.665 INFO:tasks.workunit.client.1.smithi070.stdout:9/714: dread d9/d1c/dac/d8a/d3c/d92/fb1 [0,4194304] 0 2024-09-10T04:05:23.665 INFO:tasks.workunit.client.1.smithi070.stdout:9/715: chown d9/d59/d2a 6904743 1 2024-09-10T04:05:23.667 INFO:tasks.workunit.client.1.smithi070.stdout:9/716: truncate d9/d1c/dac/d8a/d3c/d62/f8d 1551950 0 2024-09-10T04:05:23.668 INFO:tasks.workunit.client.1.smithi070.stdout:1/594: rename d3/d8/d12/d61/l8c to d3/d8/d27/d7a/d94/da5/lc1 0 2024-09-10T04:05:23.672 INFO:tasks.workunit.client.1.smithi070.stdout:3/691: creat db/d79/fed x:0 0 0 2024-09-10T04:05:23.673 INFO:tasks.workunit.client.1.smithi070.stdout:5/648: creat d17/d24/d77/fdd x:0 0 0 2024-09-10T04:05:23.676 INFO:tasks.workunit.client.1.smithi070.stdout:0/766: creat d8/d24/ff7 x:0 0 0 2024-09-10T04:05:23.677 INFO:tasks.workunit.client.1.smithi070.stdout:3/692: creat db/d83/d92/fee x:0 0 0 2024-09-10T04:05:23.680 INFO:tasks.workunit.client.1.smithi070.stdout:6/681: dwrite d0/d3/d18/d1b/d17/fa5 [4194304,4194304] 0 2024-09-10T04:05:23.681 INFO:tasks.workunit.client.1.smithi070.stdout:9/717: dwrite d9/d59/d2a/d3f/fe4 [0,4194304] 0 2024-09-10T04:05:23.687 INFO:tasks.workunit.client.1.smithi070.stdout:2/643: creat d15/d2b/d6e/fe0 x:0 0 0 2024-09-10T04:05:23.689 INFO:tasks.workunit.client.1.smithi070.stdout:8/696: creat d4/fe9 x:0 0 0 2024-09-10T04:05:23.689 INFO:tasks.workunit.client.1.smithi070.stdout:7/673: link d2/d7/d9/d18/d23/ld4 d2/d3/dc/d13/dbd/le9 0 2024-09-10T04:05:23.691 INFO:tasks.workunit.client.1.smithi070.stdout:0/767: symlink d8/d24/da9/lf8 0 2024-09-10T04:05:23.695 INFO:tasks.workunit.client.1.smithi070.stdout:3/693: creat db/d1c/d53/fef x:0 0 0 2024-09-10T04:05:23.696 INFO:tasks.workunit.client.0.smithi046.stdout:4/764: dwrite d3/d12/d14/f62 [0,4194304] 0 2024-09-10T04:05:23.696 INFO:tasks.workunit.client.0.smithi046.stdout:2/606: mknod d2/d8/d37/dc2/ce0 0 2024-09-10T04:05:23.698 INFO:tasks.workunit.client.0.smithi046.stdout:9/558: getdents d4/d3c/d4d/d5c/d59 0 2024-09-10T04:05:23.699 INFO:tasks.workunit.client.0.smithi046.stdout:9/559: chown d4/d18/d89/fa2 5 1 2024-09-10T04:05:23.701 INFO:tasks.workunit.client.1.smithi070.stdout:4/599: getdents de/d13/d5b/d9e 0 2024-09-10T04:05:23.704 INFO:tasks.workunit.client.1.smithi070.stdout:8/697: chown d4/d16/d17/d19/d28/d43/d52/ca1 124436396 1 2024-09-10T04:05:23.704 INFO:tasks.workunit.client.0.smithi046.stdout:2/607: dwrite d2/d8/d65/fa9 [0,4194304] 0 2024-09-10T04:05:23.714 INFO:tasks.workunit.client.1.smithi070.stdout:3/694: creat db/d79/db6/dc3/ff0 x:0 0 0 2024-09-10T04:05:23.716 INFO:tasks.workunit.client.1.smithi070.stdout:3/695: write db/d1c/d25/d28/d43/f77 [3360095,117127] 0 2024-09-10T04:05:23.717 INFO:tasks.workunit.client.0.smithi046.stdout:5/706: dwrite d7/d11/d3f/d53/fc7 [0,4194304] 0 2024-09-10T04:05:23.718 INFO:tasks.workunit.client.0.smithi046.stdout:9/560: write d4/d29/f6d [1815406,77226] 0 2024-09-10T04:05:23.719 INFO:tasks.workunit.client.0.smithi046.stdout:9/561: write d4/f99 [244251,92998] 0 2024-09-10T04:05:23.720 INFO:tasks.workunit.client.1.smithi070.stdout:4/600: creat de/d13/d18/d23/d47/d59/d79/d6a/fcc x:0 0 0 2024-09-10T04:05:23.720 INFO:tasks.workunit.client.0.smithi046.stdout:9/562: stat d4/d66/da6 0 2024-09-10T04:05:23.720 INFO:tasks.workunit.client.0.smithi046.stdout:4/765: write d3/d12/d17/d22/f4e [1851675,125676] 0 2024-09-10T04:05:23.721 INFO:tasks.workunit.client.1.smithi070.stdout:8/698: rmdir d4/d16/d17/d1a/d60/d7a/db8 39 2024-09-10T04:05:23.721 INFO:tasks.workunit.client.1.smithi070.stdout:6/682: dwrite d0/d3/d18/d1b/d17/d28/f5e [0,4194304] 0 2024-09-10T04:05:23.721 INFO:tasks.workunit.client.0.smithi046.stdout:9/563: read d4/d7/f5e [699163,22784] 0 2024-09-10T04:05:23.724 INFO:tasks.workunit.client.0.smithi046.stdout:5/707: chown d7/d11/f28 281 1 2024-09-10T04:05:23.724 INFO:tasks.workunit.client.1.smithi070.stdout:4/601: chown de/d13/d18/d26/d8c/d93/l9c 100375 1 2024-09-10T04:05:23.724 INFO:tasks.workunit.client.1.smithi070.stdout:7/674: mknod d2/d7/d9/d18/cea 0 2024-09-10T04:05:23.724 INFO:tasks.workunit.client.1.smithi070.stdout:5/649: getdents d17/d6f 0 2024-09-10T04:05:23.726 INFO:tasks.workunit.client.1.smithi070.stdout:4/602: chown de/d13/d5b/d9e/fbb 2113597 1 2024-09-10T04:05:23.731 INFO:tasks.workunit.client.0.smithi046.stdout:9/564: dwrite d4/d7/f77 [0,4194304] 0 2024-09-10T04:05:23.732 INFO:tasks.workunit.client.1.smithi070.stdout:6/683: chown d0/d3/d18/d1b/d17/c9c 12576 1 2024-09-10T04:05:23.736 INFO:tasks.workunit.client.0.smithi046.stdout:9/565: rmdir d4/d66 39 2024-09-10T04:05:23.739 INFO:tasks.workunit.client.1.smithi070.stdout:0/768: link d8/d24/d2d/d34/d9b/dad/fa2 d8/d13/de2/d4f/ff9 0 2024-09-10T04:05:23.740 INFO:tasks.workunit.client.1.smithi070.stdout:5/650: dread fe [0,4194304] 0 2024-09-10T04:05:23.740 INFO:tasks.workunit.client.1.smithi070.stdout:9/718: getdents d9/d1c/dac/d8a/dc4/dda/de0/d90/d89/dbb 0 2024-09-10T04:05:23.741 INFO:tasks.workunit.client.0.smithi046.stdout:9/566: dread d4/f99 [0,4194304] 0 2024-09-10T04:05:23.742 INFO:tasks.workunit.client.0.smithi046.stdout:5/708: write d7/d11/d14/f71 [4022289,14886] 0 2024-09-10T04:05:23.742 INFO:tasks.workunit.client.0.smithi046.stdout:5/709: write d7/d11/d14/f25 [2681290,38505] 0 2024-09-10T04:05:23.745 INFO:tasks.workunit.client.1.smithi070.stdout:2/644: link d15/d2b/f7a d15/d18/d99/fe1 0 2024-09-10T04:05:23.749 INFO:tasks.workunit.client.0.smithi046.stdout:9/567: mknod d4/d66/cad 0 2024-09-10T04:05:23.753 INFO:tasks.workunit.client.1.smithi070.stdout:9/719: unlink d9/d1c/d34/d8b/dcc/fd0 0 2024-09-10T04:05:23.754 INFO:tasks.workunit.client.0.smithi046.stdout:9/568: truncate d4/f6 1736683 0 2024-09-10T04:05:23.755 INFO:tasks.workunit.client.1.smithi070.stdout:6/684: write d0/d3/d1f/d81/f96 [4071,80781] 0 2024-09-10T04:05:23.755 INFO:tasks.workunit.client.1.smithi070.stdout:7/675: write d2/d4e/f9b [1813558,77162] 0 2024-09-10T04:05:23.756 INFO:tasks.workunit.client.1.smithi070.stdout:3/696: dwrite db/d79/fa3 [0,4194304] 0 2024-09-10T04:05:23.756 INFO:tasks.workunit.client.1.smithi070.stdout:4/603: creat de/d13/d18/d23/d47/fcd x:0 0 0 2024-09-10T04:05:23.758 INFO:tasks.workunit.client.1.smithi070.stdout:0/769: symlink d8/d13/d1a/d9a/lfa 0 2024-09-10T04:05:23.759 INFO:tasks.workunit.client.1.smithi070.stdout:3/697: write db/f74 [630176,21671] 0 2024-09-10T04:05:23.760 INFO:tasks.workunit.client.1.smithi070.stdout:8/699: getdents d4/dbf/d87 0 2024-09-10T04:05:23.760 INFO:tasks.workunit.client.1.smithi070.stdout:9/720: mknod d9/d59/d2a/d3f/daf/ce8 0 2024-09-10T04:05:23.761 INFO:tasks.workunit.client.1.smithi070.stdout:4/604: unlink de/d13/c30 0 2024-09-10T04:05:23.763 INFO:tasks.workunit.client.0.smithi046.stdout:5/710: dread d7/d11/d3f/d5a/d63/f7f [0,4194304] 0 2024-09-10T04:05:23.763 INFO:tasks.workunit.client.0.smithi046.stdout:9/569: getdents d4/d7/d1f 0 2024-09-10T04:05:23.764 INFO:tasks.workunit.client.1.smithi070.stdout:4/605: readlink de/d13/d18/l19 0 2024-09-10T04:05:23.766 INFO:tasks.workunit.client.1.smithi070.stdout:8/700: rmdir d4/d16/d17/d6f/db5 39 2024-09-10T04:05:23.768 INFO:tasks.workunit.client.0.smithi046.stdout:5/711: creat d7/d11/d3f/dd3/fff x:0 0 0 2024-09-10T04:05:23.769 INFO:tasks.workunit.client.0.smithi046.stdout:9/570: truncate d4/d3c/d4d/d5c/d59/f5d 695034 0 2024-09-10T04:05:23.769 INFO:tasks.workunit.client.1.smithi070.stdout:6/685: dread d0/d3/d18/d62/d70/f7e [0,4194304] 0 2024-09-10T04:05:23.772 INFO:tasks.workunit.client.1.smithi070.stdout:2/645: dread d15/d18/d29/d2a/d2e/d32/f33 [0,4194304] 0 2024-09-10T04:05:23.773 INFO:tasks.workunit.client.1.smithi070.stdout:3/698: dwrite db/d1c/d25/f75 [4194304,4194304] 0 2024-09-10T04:05:23.773 INFO:tasks.workunit.client.1.smithi070.stdout:8/701: write d4/d16/d17/d19/d28/d43/d52/fb3 [317877,19390] 0 2024-09-10T04:05:23.773 INFO:tasks.workunit.client.1.smithi070.stdout:4/606: mkdir de/d13/d18/d23/d47/d59/d79/d6a/dce 0 2024-09-10T04:05:23.773 INFO:tasks.workunit.client.0.smithi046.stdout:9/571: creat d4/d66/fae x:0 0 0 2024-09-10T04:05:23.775 INFO:tasks.workunit.client.1.smithi070.stdout:2/646: stat d15/d18/d29/d2a/d2e/d32/fd7 0 2024-09-10T04:05:23.777 INFO:tasks.workunit.client.0.smithi046.stdout:5/712: mknod d7/d11/d14/c100 0 2024-09-10T04:05:23.779 INFO:tasks.workunit.client.0.smithi046.stdout:9/572: mknod d4/d7/d1f/d54/caf 0 2024-09-10T04:05:23.782 INFO:tasks.workunit.client.1.smithi070.stdout:6/686: fsync d0/d3/d18/d1b/d17/d28/f76 0 2024-09-10T04:05:23.783 INFO:tasks.workunit.client.1.smithi070.stdout:3/699: rmdir db/d1c/d25/d28/d43 39 2024-09-10T04:05:23.784 INFO:tasks.workunit.client.1.smithi070.stdout:6/687: chown d0/d3/d18/d1b/d17/fb2 47760 1 2024-09-10T04:05:23.784 INFO:tasks.workunit.client.1.smithi070.stdout:7/676: getdents d2/d3/dc/d5d 0 2024-09-10T04:05:23.785 INFO:tasks.workunit.client.1.smithi070.stdout:6/688: fsync d0/f57 0 2024-09-10T04:05:23.786 INFO:tasks.workunit.client.1.smithi070.stdout:7/677: write d2/d3/dc/d13/fc7 [110429,96032] 0 2024-09-10T04:05:23.788 INFO:tasks.workunit.client.1.smithi070.stdout:4/607: creat de/d13/d43/d88/fcf x:0 0 0 2024-09-10T04:05:23.790 INFO:tasks.workunit.client.1.smithi070.stdout:0/770: link d8/d24/f4a d8/d13/d1a/d75/ffb 0 2024-09-10T04:05:23.791 INFO:tasks.workunit.client.1.smithi070.stdout:9/721: getdents d9/d59/d2a/d3f 0 2024-09-10T04:05:23.791 INFO:tasks.workunit.client.0.smithi046.stdout:9/573: dread d4/d3c/f6b [0,4194304] 0 2024-09-10T04:05:23.793 INFO:tasks.workunit.client.1.smithi070.stdout:3/700: creat db/d1c/d3e/ff1 x:0 0 0 2024-09-10T04:05:23.793 INFO:tasks.workunit.client.0.smithi046.stdout:5/713: write d7/d11/d14/d2f/f22 [4379662,75797] 0 2024-09-10T04:05:23.794 INFO:tasks.workunit.client.1.smithi070.stdout:3/701: write db/d1c/d25/fe6 [83511,69668] 0 2024-09-10T04:05:23.794 INFO:tasks.workunit.client.0.smithi046.stdout:9/574: write d4/d3c/d4d/d5c/f4e [36476,43984] 0 2024-09-10T04:05:23.795 INFO:tasks.workunit.client.1.smithi070.stdout:3/702: truncate db/d79/db6/fc6 665441 0 2024-09-10T04:05:23.795 INFO:tasks.workunit.client.0.smithi046.stdout:5/714: chown d7/d11/d3f/d53/da9/c6b 1036887 1 2024-09-10T04:05:23.796 INFO:tasks.workunit.client.1.smithi070.stdout:8/702: creat d4/d16/d17/d6f/db5/fea x:0 0 0 2024-09-10T04:05:23.797 INFO:tasks.workunit.client.1.smithi070.stdout:8/703: write d4/d16/d17/d1a/f32 [2230045,10406] 0 2024-09-10T04:05:23.798 INFO:tasks.workunit.client.1.smithi070.stdout:7/678: truncate d2/f24 751716 0 2024-09-10T04:05:23.799 INFO:tasks.workunit.client.0.smithi046.stdout:9/575: creat d4/d7/d1f/d54/fb0 x:0 0 0 2024-09-10T04:05:23.799 INFO:tasks.workunit.client.1.smithi070.stdout:4/608: mknod de/d13/d18/d5c/db6/cd0 0 2024-09-10T04:05:23.801 INFO:tasks.workunit.client.1.smithi070.stdout:2/647: rename d15/d18/d29/d2a/d2e/d32/d9b/ca9 to d15/d18/d56/d7c/d55/d6d/ce2 0 2024-09-10T04:05:23.802 INFO:tasks.workunit.client.1.smithi070.stdout:2/648: fsync d15/d18/db2/fd6 0 2024-09-10T04:05:23.802 INFO:tasks.workunit.client.1.smithi070.stdout:0/771: rmdir d8/d24/d2d 39 2024-09-10T04:05:23.803 INFO:tasks.workunit.client.0.smithi046.stdout:9/576: stat d4/d44 0 2024-09-10T04:05:23.803 INFO:tasks.workunit.client.1.smithi070.stdout:9/722: rmdir d9/d1c/dac/d8a/dc4/dda/de0/d90 39 2024-09-10T04:05:23.803 INFO:tasks.workunit.client.0.smithi046.stdout:9/577: fsync d4/d29/f3d 0 2024-09-10T04:05:23.804 INFO:tasks.workunit.client.0.smithi046.stdout:5/715: getdents d7/d11/d14/d8e/d96/dce 0 2024-09-10T04:05:23.805 INFO:tasks.workunit.client.0.smithi046.stdout:9/578: fsync d4/d29/f6d 0 2024-09-10T04:05:23.805 INFO:tasks.workunit.client.0.smithi046.stdout:5/716: dread - d7/d11/d14/d2f/fd6 zero size 2024-09-10T04:05:23.807 INFO:tasks.workunit.client.0.smithi046.stdout:5/717: creat d7/d11/d14/d2f/d2b/f101 x:0 0 0 2024-09-10T04:05:23.808 INFO:tasks.workunit.client.1.smithi070.stdout:7/679: write d2/d40/d6c/db2/fe3 [235231,20802] 0 2024-09-10T04:05:23.808 INFO:tasks.workunit.client.0.smithi046.stdout:5/718: rename d7 to d7/d11/d14/d2f/df0/d102 22 2024-09-10T04:05:23.810 INFO:tasks.workunit.client.1.smithi070.stdout:2/649: rmdir d15/d59 39 2024-09-10T04:05:23.810 INFO:tasks.workunit.client.0.smithi046.stdout:5/719: truncate d7/d11/d14/d2f/d64/fd7 880501 0 2024-09-10T04:05:23.810 INFO:tasks.workunit.client.1.smithi070.stdout:0/772: symlink d8/d13/de2/d4f/lfc 0 2024-09-10T04:05:23.811 INFO:tasks.workunit.client.1.smithi070.stdout:7/680: write d2/d7/db/f92 [1033073,24007] 0 2024-09-10T04:05:23.813 INFO:tasks.workunit.client.1.smithi070.stdout:3/703: dwrite db/d1c/d25/d28/d43/f9c [0,4194304] 0 2024-09-10T04:05:23.813 INFO:tasks.workunit.client.1.smithi070.stdout:2/650: chown d15/d18/d56/cab 538438 1 2024-09-10T04:05:23.815 INFO:tasks.workunit.client.1.smithi070.stdout:9/723: creat d9/d1c/dac/d8a/d3c/d92/da8/fe9 x:0 0 0 2024-09-10T04:05:23.817 INFO:tasks.workunit.client.1.smithi070.stdout:8/704: rename d4/d16/d17/d1a/d60/d7a/fca to d4/dcf/feb 0 2024-09-10T04:05:23.817 INFO:tasks.workunit.client.0.smithi046.stdout:5/720: getdents d7/d11/d14/d8e/d96/dce 0 2024-09-10T04:05:23.819 INFO:tasks.workunit.client.1.smithi070.stdout:0/773: dwrite d8/f1b [8388608,4194304] 0 2024-09-10T04:05:23.821 INFO:tasks.workunit.client.1.smithi070.stdout:2/651: mknod d15/d18/db2/ce3 0 2024-09-10T04:05:23.826 INFO:tasks.workunit.client.1.smithi070.stdout:6/689: write d0/d3/d18/d1b/d17/f47 [496165,117877] 0 2024-09-10T04:05:23.828 INFO:tasks.workunit.client.1.smithi070.stdout:4/609: write de/d13/d18/d23/f61 [1952102,129656] 0 2024-09-10T04:05:23.831 INFO:tasks.workunit.client.1.smithi070.stdout:0/774: read d8/d13/d1a/d75/f84 [461352,4840] 0 2024-09-10T04:05:23.831 INFO:tasks.workunit.client.1.smithi070.stdout:3/704: creat db/d79/db6/dc3/de1/ff2 x:0 0 0 2024-09-10T04:05:23.833 INFO:tasks.workunit.client.1.smithi070.stdout:3/705: dread - db/d1c/d53/fef zero size 2024-09-10T04:05:23.835 INFO:tasks.workunit.client.0.smithi046.stdout:5/721: dread d7/d11/d14/d8e/d96/fbe [0,4194304] 0 2024-09-10T04:05:23.836 INFO:tasks.workunit.client.0.smithi046.stdout:5/722: write d7/d11/ffc [894322,59468] 0 2024-09-10T04:05:23.836 INFO:tasks.workunit.client.1.smithi070.stdout:4/610: truncate de/f34 885093 0 2024-09-10T04:05:23.839 INFO:tasks.workunit.client.1.smithi070.stdout:7/681: rename d2/dc8/fe8 to d2/d7/db/feb 0 2024-09-10T04:05:23.840 INFO:tasks.workunit.client.1.smithi070.stdout:8/705: write d4/d16/f36 [2141419,82256] 0 2024-09-10T04:05:23.842 INFO:tasks.workunit.client.0.smithi046.stdout:5/723: dwrite d7/d11/d14/d2f/fd6 [0,4194304] 0 2024-09-10T04:05:23.843 INFO:tasks.workunit.client.1.smithi070.stdout:6/690: rename d0/d3/d18/d62/d70/lcc to d0/d3/d1f/d93/ld8 0 2024-09-10T04:05:23.845 INFO:tasks.workunit.client.1.smithi070.stdout:9/724: dwrite d9/d1c/dac/d8a/d1b/fba [0,4194304] 0 2024-09-10T04:05:23.847 INFO:tasks.workunit.client.1.smithi070.stdout:6/691: fdatasync d0/d3/d18/fd4 0 2024-09-10T04:05:23.847 INFO:tasks.workunit.client.1.smithi070.stdout:0/775: creat d8/d24/ffd x:0 0 0 2024-09-10T04:05:23.848 INFO:tasks.workunit.client.0.smithi046.stdout:5/724: symlink d7/d11/d3f/d53/d81/l103 0 2024-09-10T04:05:23.851 INFO:tasks.workunit.client.1.smithi070.stdout:2/652: rename d15/d18/d29/d2a/d2e/d32/c9f to d15/d59/da1/ce4 0 2024-09-10T04:05:23.853 INFO:tasks.workunit.client.1.smithi070.stdout:4/611: truncate de/d13/d18/d26/f2d 737857 0 2024-09-10T04:05:23.854 INFO:tasks.workunit.client.0.smithi046.stdout:5/725: rename d7/d11/d14/d2f/deb/da5/dcc/dc0/ccd to d7/d11/d14/d2f/deb/da5/dcc/c104 0 2024-09-10T04:05:23.855 INFO:tasks.workunit.client.1.smithi070.stdout:3/706: dread db/d1c/d3e/f59 [0,4194304] 0 2024-09-10T04:05:23.855 INFO:tasks.workunit.client.1.smithi070.stdout:6/692: write d0/d3/d18/d1b/d17/d28/f3b [461196,36635] 0 2024-09-10T04:05:23.855 INFO:tasks.workunit.client.1.smithi070.stdout:0/776: mknod d8/d24/d36/cfe 0 2024-09-10T04:05:23.860 INFO:tasks.workunit.client.0.smithi046.stdout:5/726: unlink d7/d11/d3f/d5a/fdf 0 2024-09-10T04:05:23.862 INFO:tasks.workunit.client.0.smithi046.stdout:5/727: write d7/d11/d14/d45/d51/f67 [615421,55556] 0 2024-09-10T04:05:23.862 INFO:tasks.workunit.client.1.smithi070.stdout:8/706: dwrite d4/d39/fc8 [0,4194304] 0 2024-09-10T04:05:23.863 INFO:tasks.workunit.client.0.smithi046.stdout:5/728: dread - d7/d11/d3f/ffe zero size 2024-09-10T04:05:23.864 INFO:tasks.workunit.client.0.smithi046.stdout:5/729: stat d7/d11/d14/d45/f4b 0 2024-09-10T04:05:23.865 INFO:tasks.workunit.client.1.smithi070.stdout:6/693: rmdir d0/d3/d1f/d81 39 2024-09-10T04:05:23.865 INFO:tasks.workunit.client.1.smithi070.stdout:3/707: chown db/d1c/d25/d28/d43/c65 0 1 2024-09-10T04:05:23.866 INFO:tasks.workunit.client.1.smithi070.stdout:9/725: dwrite d9/d59/f18 [0,4194304] 0 2024-09-10T04:05:23.866 INFO:tasks.workunit.client.1.smithi070.stdout:8/707: stat d4/f12 0 2024-09-10T04:05:23.867 INFO:tasks.workunit.client.1.smithi070.stdout:7/682: dread d2/d3/dc/d5d/d42/fb5 [0,4194304] 0 2024-09-10T04:05:23.867 INFO:tasks.workunit.client.0.smithi046.stdout:5/730: rename d7/d11/d14/fc3 to d7/f105 0 2024-09-10T04:05:23.871 INFO:tasks.workunit.client.1.smithi070.stdout:9/726: fsync d9/d1c/d34/d8b/dcc/f9d 0 2024-09-10T04:05:23.872 INFO:tasks.workunit.client.1.smithi070.stdout:3/708: creat db/dcc/ff3 x:0 0 0 2024-09-10T04:05:23.876 INFO:tasks.workunit.client.1.smithi070.stdout:3/709: stat db/d1c/d25/d28/ca0 0 2024-09-10T04:05:23.879 INFO:tasks.workunit.client.1.smithi070.stdout:2/653: getdents d15/d2b/d6e/db8 0 2024-09-10T04:05:23.881 INFO:tasks.workunit.client.1.smithi070.stdout:4/612: write de/d13/d18/d23/d39/f57 [1373250,57503] 0 2024-09-10T04:05:23.881 INFO:tasks.workunit.client.1.smithi070.stdout:8/708: mkdir d4/d16/d17/d19/d28/d43/d67/dec 0 2024-09-10T04:05:23.883 INFO:tasks.workunit.client.1.smithi070.stdout:7/683: dwrite d2/d7/d9/d18/f99 [0,4194304] 0 2024-09-10T04:05:23.885 INFO:tasks.workunit.client.1.smithi070.stdout:0/777: truncate d8/d85/fac 477451 0 2024-09-10T04:05:23.886 INFO:tasks.workunit.client.1.smithi070.stdout:9/727: dwrite d9/d7a/d74/f47 [4194304,4194304] 0 2024-09-10T04:05:23.887 INFO:tasks.workunit.client.1.smithi070.stdout:6/694: rmdir d0/d3/d18/d1b/d17/d8a 0 2024-09-10T04:05:23.887 INFO:tasks.workunit.client.1.smithi070.stdout:7/684: chown d2/d3/dc/d5d/d42/fa0 830 1 2024-09-10T04:05:23.888 INFO:tasks.workunit.client.0.smithi046.stdout:5/731: rename d7/l7b to d7/d11/d3f/l106 0 2024-09-10T04:05:23.890 INFO:tasks.workunit.client.1.smithi070.stdout:6/695: readlink d0/d3/d18/d1b/d17/d54/l71 0 2024-09-10T04:05:23.891 INFO:tasks.workunit.client.1.smithi070.stdout:3/710: rename db/d79/d94/lb1 to db/d1c/d25/d28/dd3/de5/lf4 0 2024-09-10T04:05:23.893 INFO:tasks.workunit.client.1.smithi070.stdout:2/654: truncate d15/d18/f26 598037 0 2024-09-10T04:05:23.895 INFO:tasks.workunit.client.1.smithi070.stdout:3/711: dread - db/d1c/fcf zero size 2024-09-10T04:05:23.896 INFO:tasks.workunit.client.1.smithi070.stdout:7/685: rename d2/d7/d9/d18/c1f to d2/d7/d9/db4/cec 0 2024-09-10T04:05:23.898 INFO:tasks.workunit.client.1.smithi070.stdout:4/613: dwrite de/d13/d18/d23/d47/d59/f7d [0,4194304] 0 2024-09-10T04:05:23.898 INFO:tasks.workunit.client.1.smithi070.stdout:2/655: chown d15/d18/d56/ca5 3450 1 2024-09-10T04:05:23.901 INFO:tasks.workunit.client.1.smithi070.stdout:8/709: dread d4/d16/d17/d1a/f58 [0,4194304] 0 2024-09-10T04:05:23.905 INFO:tasks.workunit.client.1.smithi070.stdout:3/712: mknod db/d83/d92/dc7/cf5 0 2024-09-10T04:05:23.905 INFO:tasks.workunit.client.1.smithi070.stdout:7/686: truncate d2/d3/dc/d79/fb9 5054950 0 2024-09-10T04:05:23.906 INFO:tasks.workunit.client.0.smithi046.stdout:1/468: sync 2024-09-10T04:05:23.906 INFO:tasks.workunit.client.0.smithi046.stdout:7/610: sync 2024-09-10T04:05:23.907 INFO:tasks.workunit.client.0.smithi046.stdout:7/611: readlink d5/d6/d32/dab/db8/laf 0 2024-09-10T04:05:23.908 INFO:tasks.workunit.client.1.smithi070.stdout:9/728: dwrite d9/fab [0,4194304] 0 2024-09-10T04:05:23.912 INFO:tasks.workunit.client.1.smithi070.stdout:4/614: dwrite de/d13/f45 [0,4194304] 0 2024-09-10T04:05:23.917 INFO:tasks.workunit.client.1.smithi070.stdout:0/778: truncate d8/d13/fe8 1269481 0 2024-09-10T04:05:23.919 INFO:tasks.workunit.client.1.smithi070.stdout:7/687: rename d2/d3/dc/d5d/f34 to d2/d3/dc/d79/fed 0 2024-09-10T04:05:23.920 INFO:tasks.workunit.client.0.smithi046.stdout:7/612: creat d5/d9/d43/db4/fc8 x:0 0 0 2024-09-10T04:05:23.921 INFO:tasks.workunit.client.1.smithi070.stdout:7/688: write d2/d40/d6c/db2/fce [53412,48411] 0 2024-09-10T04:05:23.922 INFO:tasks.workunit.client.0.smithi046.stdout:5/732: rename d7/d11 to d7/d11/d3f/d53/d81/d107 22 2024-09-10T04:05:23.923 INFO:tasks.workunit.client.1.smithi070.stdout:4/615: chown de/d13/d18/d23/fc6 804 1 2024-09-10T04:05:23.924 INFO:tasks.workunit.client.1.smithi070.stdout:8/710: write d4/d16/d17/d1a/fa9 [1127781,26077] 0 2024-09-10T04:05:23.925 INFO:tasks.workunit.client.1.smithi070.stdout:2/656: creat d15/d18/d29/d2a/d2e/d32/fe5 x:0 0 0 2024-09-10T04:05:23.926 INFO:tasks.workunit.client.1.smithi070.stdout:4/616: write de/d13/d18/f21 [828252,73531] 0 2024-09-10T04:05:23.927 INFO:tasks.workunit.client.1.smithi070.stdout:7/689: dread d2/f55 [0,4194304] 0 2024-09-10T04:05:23.927 INFO:tasks.workunit.client.1.smithi070.stdout:2/657: chown d15/d18/d29/d2a/l3e 48 1 2024-09-10T04:05:23.932 INFO:tasks.workunit.client.1.smithi070.stdout:0/779: symlink d8/lff 0 2024-09-10T04:05:23.932 INFO:tasks.workunit.client.1.smithi070.stdout:0/780: dread - d8/d13/de2/fd0 zero size 2024-09-10T04:05:23.933 INFO:tasks.workunit.client.1.smithi070.stdout:9/729: creat d9/d1c/dac/d8a/dc4/dda/de0/d90/d89/ddc/fea x:0 0 0 2024-09-10T04:05:23.934 INFO:tasks.workunit.client.1.smithi070.stdout:2/658: mknod d15/d59/da1/ce6 0 2024-09-10T04:05:23.938 INFO:tasks.workunit.client.1.smithi070.stdout:8/711: symlink d4/d16/d17/d19/d28/led 0 2024-09-10T04:05:23.941 INFO:tasks.workunit.client.1.smithi070.stdout:4/617: link de/d13/d43/f6b de/d13/d18/d23/d47/d59/d79/dbc/fd1 0 2024-09-10T04:05:23.941 INFO:tasks.workunit.client.1.smithi070.stdout:0/781: dwrite d8/d13/d1a/f23 [4194304,4194304] 0 2024-09-10T04:05:23.942 INFO:tasks.workunit.client.1.smithi070.stdout:4/618: chown de/d13/d18/d23/d39/d48/l4c 13828173 1 2024-09-10T04:05:23.943 INFO:tasks.workunit.client.1.smithi070.stdout:7/690: link d2/d3/dc/d5d/d41/c5f d2/dc5/cee 0 2024-09-10T04:05:23.945 INFO:tasks.workunit.client.1.smithi070.stdout:0/782: write d8/d24/d2d/d5f/fd3 [122065,130420] 0 2024-09-10T04:05:23.946 INFO:tasks.workunit.client.1.smithi070.stdout:0/783: dread - d8/d13/de2/d4f/fdc zero size 2024-09-10T04:05:23.950 INFO:tasks.workunit.client.0.smithi046.stdout:7/613: mknod d5/d6/d39/cc9 0 2024-09-10T04:05:23.952 INFO:tasks.workunit.client.0.smithi046.stdout:1/469: creat d3/dc/d34/d51/f99 x:0 0 0 2024-09-10T04:05:23.953 INFO:tasks.workunit.client.0.smithi046.stdout:1/470: chown d3/dc/d21 769086 1 2024-09-10T04:05:23.954 INFO:tasks.workunit.client.1.smithi070.stdout:0/784: creat d8/d24/d2d/d5f/f100 x:0 0 0 2024-09-10T04:05:23.956 INFO:tasks.workunit.client.0.smithi046.stdout:7/614: dread d5/d6/d55/d7d/f9b [0,4194304] 0 2024-09-10T04:05:23.959 INFO:tasks.workunit.client.1.smithi070.stdout:8/712: link d4/f77 d4/dbf/d74/fee 0 2024-09-10T04:05:23.960 INFO:tasks.workunit.client.0.smithi046.stdout:7/615: creat d5/d9/d43/db4/fca x:0 0 0 2024-09-10T04:05:23.961 INFO:tasks.workunit.client.1.smithi070.stdout:0/785: symlink d8/d24/d36/l101 0 2024-09-10T04:05:23.963 INFO:tasks.workunit.client.0.smithi046.stdout:1/471: rename d3/d27/d5d/f6a to d3/d27/d5d/f9a 0 2024-09-10T04:05:23.963 INFO:tasks.workunit.client.1.smithi070.stdout:7/691: getdents d2/d7/d9/d18/d23 0 2024-09-10T04:05:23.964 INFO:tasks.workunit.client.1.smithi070.stdout:7/692: readlink d2/d7/d9/l95 0 2024-09-10T04:05:23.965 INFO:tasks.workunit.client.0.smithi046.stdout:1/472: read d3/d27/d32/d1f/f55 [13090,128154] 0 2024-09-10T04:05:23.966 INFO:tasks.workunit.client.1.smithi070.stdout:8/713: symlink d4/d39/lef 0 2024-09-10T04:05:23.966 INFO:tasks.workunit.client.1.smithi070.stdout:8/714: chown d4/dcf/l66 113 1 2024-09-10T04:05:23.967 INFO:tasks.workunit.client.0.smithi046.stdout:1/473: chown d3/d27/c94 592 1 2024-09-10T04:05:23.967 INFO:tasks.workunit.client.0.smithi046.stdout:7/616: dwrite d5/d9/d43/f5c [0,4194304] 0 2024-09-10T04:05:23.971 INFO:tasks.workunit.client.1.smithi070.stdout:8/715: mkdir d4/d16/d17/d19/d62/df0 0 2024-09-10T04:05:23.972 INFO:tasks.workunit.client.1.smithi070.stdout:8/716: dread - d4/d16/f34 zero size 2024-09-10T04:05:23.973 INFO:tasks.workunit.client.0.smithi046.stdout:7/617: mknod d5/d6/db/dd/d24/d76/d9c/ccb 0 2024-09-10T04:05:23.976 INFO:tasks.workunit.client.1.smithi070.stdout:7/693: link d2/d3/dc/d5d/d41/f50 d2/d3/d63/fef 0 2024-09-10T04:05:23.978 INFO:tasks.workunit.client.1.smithi070.stdout:8/717: dwrite d4/fcd [0,4194304] 0 2024-09-10T04:05:23.980 INFO:tasks.workunit.client.1.smithi070.stdout:7/694: creat d2/d3/dc/d5d/ff0 x:0 0 0 2024-09-10T04:05:23.981 INFO:tasks.workunit.client.1.smithi070.stdout:8/718: rmdir d4/dbf/d74 39 2024-09-10T04:05:23.982 INFO:tasks.workunit.client.0.smithi046.stdout:7/618: creat d5/d6/db/d11/d1f/fcc x:0 0 0 2024-09-10T04:05:23.982 INFO:tasks.workunit.client.0.smithi046.stdout:7/619: stat f3 0 2024-09-10T04:05:23.984 INFO:tasks.workunit.client.1.smithi070.stdout:8/719: creat d4/d16/d17/d6f/db5/ff1 x:0 0 0 2024-09-10T04:05:23.987 INFO:tasks.workunit.client.1.smithi070.stdout:8/720: creat d4/dbf/d87/ff2 x:0 0 0 2024-09-10T04:05:23.987 INFO:tasks.workunit.client.1.smithi070.stdout:8/721: fsync d4/f12 0 2024-09-10T04:05:23.988 INFO:tasks.workunit.client.1.smithi070.stdout:8/722: chown d4/d16/d17/d6f/de5 4149735 1 2024-09-10T04:05:23.989 INFO:tasks.workunit.client.1.smithi070.stdout:8/723: fdatasync d4/fe 0 2024-09-10T04:05:23.989 INFO:tasks.workunit.client.1.smithi070.stdout:7/695: getdents d2/d7/d9/d18/d3c/d52 0 2024-09-10T04:05:23.992 INFO:tasks.workunit.client.1.smithi070.stdout:8/724: stat d4/dcf/le1 0 2024-09-10T04:05:23.993 INFO:tasks.workunit.client.1.smithi070.stdout:7/696: readlink d2/d3/dc/d5d/d2d/lde 0 2024-09-10T04:05:23.998 INFO:tasks.workunit.client.1.smithi070.stdout:4/619: write de/d13/d18/d26/f2b [1134624,33101] 0 2024-09-10T04:05:23.999 INFO:tasks.workunit.client.0.smithi046.stdout:5/733: write d7/d11/d3f/fc8 [226327,8290] 0 2024-09-10T04:05:24.000 INFO:tasks.workunit.client.0.smithi046.stdout:5/734: fsync d7/d11/d14/f89 0 2024-09-10T04:05:24.002 INFO:tasks.workunit.client.0.smithi046.stdout:5/735: readlink d7/d11/d14/d2f/d64/lc1 0 2024-09-10T04:05:24.004 INFO:tasks.workunit.client.1.smithi070.stdout:4/620: dwrite de/d13/d18/d23/f61 [0,4194304] 0 2024-09-10T04:05:24.004 INFO:tasks.workunit.client.0.smithi046.stdout:5/736: rmdir d7/d11/d14/d2f/deb/da5/dcc/dc0/df3 39 2024-09-10T04:05:24.006 INFO:tasks.workunit.client.0.smithi046.stdout:5/737: rmdir d7/d11/d14 39 2024-09-10T04:05:24.007 INFO:tasks.workunit.client.0.smithi046.stdout:5/738: stat d7/d11/d3f/d5a/d63/ded 0 2024-09-10T04:05:24.008 INFO:tasks.workunit.client.1.smithi070.stdout:4/621: fsync de/d13/d18/d23/d39/d77/faa 0 2024-09-10T04:05:24.008 INFO:tasks.workunit.client.1.smithi070.stdout:0/786: write d8/d24/f43 [4940418,43078] 0 2024-09-10T04:05:24.009 INFO:tasks.workunit.client.1.smithi070.stdout:4/622: mknod de/d13/d43/cd2 0 2024-09-10T04:05:24.014 INFO:tasks.workunit.client.0.smithi046.stdout:5/739: rename d7/d11/d3f/d5a/d63/ded to d7/d11/d14/d8e/de2/d108 0 2024-09-10T04:05:24.015 INFO:tasks.workunit.client.1.smithi070.stdout:4/623: mknod de/d13/d18/d5c/cd3 0 2024-09-10T04:05:24.016 INFO:tasks.workunit.client.0.smithi046.stdout:7/620: getdents d5/d6/db/d11/d1f 0 2024-09-10T04:05:24.017 INFO:tasks.workunit.client.0.smithi046.stdout:5/740: mknod d7/d11/d3f/dd3/c109 0 2024-09-10T04:05:24.019 INFO:tasks.workunit.client.0.smithi046.stdout:7/621: mknod d5/d6/d32/ccd 0 2024-09-10T04:05:24.021 INFO:tasks.workunit.client.0.smithi046.stdout:5/741: write d7/f10 [356459,94457] 0 2024-09-10T04:05:24.022 INFO:tasks.workunit.client.0.smithi046.stdout:7/622: dread - d5/d6/db/d11/d1e/d20/fbb zero size 2024-09-10T04:05:24.025 INFO:tasks.workunit.client.1.smithi070.stdout:8/725: dwrite d4/dbf/d74/fee [0,4194304] 0 2024-09-10T04:05:24.026 INFO:tasks.workunit.client.0.smithi046.stdout:7/623: unlink d5/d9/d12/d72/d77/da7/l80 0 2024-09-10T04:05:24.029 INFO:tasks.workunit.client.1.smithi070.stdout:8/726: creat d4/d16/d17/d1a/d60/d7a/d84/ff3 x:0 0 0 2024-09-10T04:05:24.030 INFO:tasks.workunit.client.1.smithi070.stdout:4/624: write de/d13/d18/d23/f52 [477204,773] 0 2024-09-10T04:05:24.030 INFO:tasks.workunit.client.0.smithi046.stdout:7/624: creat d5/d6/db/d11/fce x:0 0 0 2024-09-10T04:05:24.032 INFO:tasks.workunit.client.1.smithi070.stdout:0/787: dread d8/d13/d1a/f32 [0,4194304] 0 2024-09-10T04:05:24.032 INFO:tasks.workunit.client.0.smithi046.stdout:7/625: write d5/f60 [5680623,92163] 0 2024-09-10T04:05:24.035 INFO:tasks.workunit.client.0.smithi046.stdout:7/626: write d5/d6/db/dd/d4b/f52 [1815796,110264] 0 2024-09-10T04:05:24.036 INFO:tasks.workunit.client.1.smithi070.stdout:4/625: rename de/d13/d5b/d9e/fbb to de/d13/d18/d5c/db6/fd4 0 2024-09-10T04:05:24.037 INFO:tasks.workunit.client.1.smithi070.stdout:0/788: chown d8/d13/de2/d71/fed 12888 1 2024-09-10T04:05:24.038 INFO:tasks.workunit.client.0.smithi046.stdout:7/627: chown d5/d6/db/dd/d24/d76/d9c/ccb 471116 1 2024-09-10T04:05:24.039 INFO:tasks.workunit.client.0.smithi046.stdout:7/628: write d5/d6/db/dd/d24/d76/fc3 [448964,77908] 0 2024-09-10T04:05:24.041 INFO:tasks.workunit.client.1.smithi070.stdout:8/727: link d4/fa d4/d16/d17/ff4 0 2024-09-10T04:05:24.043 INFO:tasks.workunit.client.1.smithi070.stdout:4/626: truncate de/f34 1775914 0 2024-09-10T04:05:24.044 INFO:tasks.workunit.client.1.smithi070.stdout:4/627: dread - de/d13/d18/d26/d8c/fba zero size 2024-09-10T04:05:24.045 INFO:tasks.workunit.client.1.smithi070.stdout:8/728: symlink d4/d16/d17/d1a/lf5 0 2024-09-10T04:05:24.046 INFO:tasks.workunit.client.1.smithi070.stdout:8/729: write d4/d16/d17/d1a/d60/f9c [3693810,96778] 0 2024-09-10T04:05:24.047 INFO:tasks.workunit.client.1.smithi070.stdout:8/730: chown d4/d16/d17/d1a/d60/lb9 3594 1 2024-09-10T04:05:24.050 INFO:tasks.workunit.client.0.smithi046.stdout:7/629: getdents d5/d6/db 0 2024-09-10T04:05:24.054 INFO:tasks.workunit.client.0.smithi046.stdout:7/630: creat d5/d6/db/dd/d24/db7/fcf x:0 0 0 2024-09-10T04:05:24.054 INFO:tasks.workunit.client.1.smithi070.stdout:0/789: write d8/d24/d2d/f3a [1440361,104821] 0 2024-09-10T04:05:24.055 INFO:tasks.workunit.client.1.smithi070.stdout:0/790: chown d8/d13/lb3 5 1 2024-09-10T04:05:24.056 INFO:tasks.workunit.client.1.smithi070.stdout:8/731: truncate d4/d16/d17/d1a/fa9 268052 0 2024-09-10T04:05:24.057 INFO:tasks.workunit.client.0.smithi046.stdout:7/631: symlink d5/d7f/ld0 0 2024-09-10T04:05:24.057 INFO:tasks.workunit.client.0.smithi046.stdout:7/632: truncate d5/fbf 209422 0 2024-09-10T04:05:24.058 INFO:tasks.workunit.client.1.smithi070.stdout:8/732: stat d4/d16/d17/d19/d28/d43/d67/d97/la3 0 2024-09-10T04:05:24.059 INFO:tasks.workunit.client.1.smithi070.stdout:0/791: creat d8/d13/de2/d53/f102 x:0 0 0 2024-09-10T04:05:24.060 INFO:tasks.workunit.client.0.smithi046.stdout:7/633: symlink d5/d6/db/d11/d1e/d20/ld1 0 2024-09-10T04:05:24.060 INFO:tasks.workunit.client.1.smithi070.stdout:4/628: dwrite de/d13/d18/d23/d39/d48/f7e [0,4194304] 0 2024-09-10T04:05:24.063 INFO:tasks.workunit.client.1.smithi070.stdout:8/733: fdatasync d4/dbf/fb6 0 2024-09-10T04:05:24.067 INFO:tasks.workunit.client.1.smithi070.stdout:0/792: rename d8/d24/d2d/d8d/lea to d8/d13/de2/d81/l103 0 2024-09-10T04:05:24.067 INFO:tasks.workunit.client.0.smithi046.stdout:7/634: dwrite d5/d6/db/dd/d24/d76/fac [0,4194304] 0 2024-09-10T04:05:24.071 INFO:tasks.workunit.client.0.smithi046.stdout:6/569: sync 2024-09-10T04:05:24.072 INFO:tasks.workunit.client.0.smithi046.stdout:0/459: sync 2024-09-10T04:05:24.074 INFO:tasks.workunit.client.1.smithi070.stdout:8/734: symlink d4/d16/d17/d19/d28/da8/lf6 0 2024-09-10T04:05:24.075 INFO:tasks.workunit.client.0.smithi046.stdout:7/635: creat d5/d6/fd2 x:0 0 0 2024-09-10T04:05:24.076 INFO:tasks.workunit.client.0.smithi046.stdout:7/636: dread - d5/d6/db/d11/fce zero size 2024-09-10T04:05:24.077 INFO:tasks.workunit.client.1.smithi070.stdout:4/629: rename de/d13/d18/d26/d46 to de/d13/d5b/d7b/dd5 0 2024-09-10T04:05:24.077 INFO:tasks.workunit.client.1.smithi070.stdout:4/630: fsync de/d13/d5b/d7b/fac 0 2024-09-10T04:05:24.078 INFO:tasks.workunit.client.0.smithi046.stdout:6/570: mknod da/db/d11/d13/d1e/d23/d54/cbc 0 2024-09-10T04:05:24.078 INFO:tasks.workunit.client.1.smithi070.stdout:4/631: readlink de/d13/d18/d26/l86 0 2024-09-10T04:05:24.079 INFO:tasks.workunit.client.1.smithi070.stdout:0/793: creat d8/d24/d2d/d34/f104 x:0 0 0 2024-09-10T04:05:24.080 INFO:tasks.workunit.client.0.smithi046.stdout:0/460: dwrite d1/dc/f17 [0,4194304] 0 2024-09-10T04:05:24.080 INFO:tasks.workunit.client.1.smithi070.stdout:4/632: truncate de/d13/f45 5092350 0 2024-09-10T04:05:24.081 INFO:tasks.workunit.client.1.smithi070.stdout:8/735: fsync d4/d16/d17/d1a/f88 0 2024-09-10T04:05:24.083 INFO:tasks.workunit.client.0.smithi046.stdout:7/637: mknod d5/d6/db/dd/d24/d76/d9c/cd3 0 2024-09-10T04:05:24.084 INFO:tasks.workunit.client.0.smithi046.stdout:6/571: truncate da/db/d11/d13/d1e/d69/f87 3377595 0 2024-09-10T04:05:24.086 INFO:tasks.workunit.client.0.smithi046.stdout:6/572: readlink da/db/d11/d13/d1e/d23/l58 0 2024-09-10T04:05:24.086 INFO:tasks.workunit.client.0.smithi046.stdout:0/461: truncate d1/d13/d37/f87 660111 0 2024-09-10T04:05:24.088 INFO:tasks.workunit.client.1.smithi070.stdout:4/633: dread de/d13/d18/d23/d39/f57 [0,4194304] 0 2024-09-10T04:05:24.088 INFO:tasks.workunit.client.0.smithi046.stdout:7/638: rename d5/fc to d5/d6/d55/d7d/fd4 0 2024-09-10T04:05:24.090 INFO:tasks.workunit.client.0.smithi046.stdout:0/462: write d1/d13/f8c [4792497,101813] 0 2024-09-10T04:05:24.090 INFO:tasks.workunit.client.1.smithi070.stdout:1/595: sync 2024-09-10T04:05:24.090 INFO:tasks.workunit.client.1.smithi070.stdout:5/651: sync 2024-09-10T04:05:24.094 INFO:tasks.workunit.client.0.smithi046.stdout:0/463: mknod d1/dc/c8d 0 2024-09-10T04:05:24.095 INFO:tasks.workunit.client.1.smithi070.stdout:4/634: getdents de/db0 0 2024-09-10T04:05:24.096 INFO:tasks.workunit.client.0.smithi046.stdout:7/639: rename d5/d6/d39/fa0 to d5/d6/db/d11/fd5 0 2024-09-10T04:05:24.097 INFO:tasks.workunit.client.1.smithi070.stdout:4/635: fdatasync de/d13/d18/d23/d47/d59/f7d 0 2024-09-10T04:05:24.097 INFO:tasks.workunit.client.1.smithi070.stdout:0/794: dread d8/d24/d2d/fc4 [0,4194304] 0 2024-09-10T04:05:24.097 INFO:tasks.workunit.client.1.smithi070.stdout:4/636: fsync de/d13/d18/d23/d39/f5a 0 2024-09-10T04:05:24.098 INFO:tasks.workunit.client.1.smithi070.stdout:5/652: mknod d17/d24/d2b/cde 0 2024-09-10T04:05:24.099 INFO:tasks.workunit.client.0.smithi046.stdout:7/640: truncate d5/d6/db/dd/d24/d76/d9c/fa3 337319 0 2024-09-10T04:05:24.100 INFO:tasks.workunit.client.1.smithi070.stdout:1/596: mknod d3/d8/d13/d3f/d5d/cc2 0 2024-09-10T04:05:24.103 INFO:tasks.workunit.client.1.smithi070.stdout:8/736: rename d4/d16/d7d to d4/d16/d17/d1a/df7 0 2024-09-10T04:05:24.104 INFO:tasks.workunit.client.0.smithi046.stdout:7/641: write d5/d6/d55/d7d/fd4 [656314,109258] 0 2024-09-10T04:05:24.105 INFO:tasks.workunit.client.0.smithi046.stdout:7/642: stat d5/d6/d32/ca9 0 2024-09-10T04:05:24.107 INFO:tasks.workunit.client.1.smithi070.stdout:0/795: creat d8/d85/dca/f105 x:0 0 0 2024-09-10T04:05:24.108 INFO:tasks.workunit.client.0.smithi046.stdout:0/464: dread d1/dc/d49/f4a [0,4194304] 0 2024-09-10T04:05:24.109 INFO:tasks.workunit.client.1.smithi070.stdout:5/653: truncate d17/f20 1013238 0 2024-09-10T04:05:24.109 INFO:tasks.workunit.client.1.smithi070.stdout:0/796: write d8/d24/d36/f46 [7825990,188] 0 2024-09-10T04:05:24.110 INFO:tasks.workunit.client.0.smithi046.stdout:7/643: creat d5/d6/db/d11/d1f/fd6 x:0 0 0 2024-09-10T04:05:24.111 INFO:tasks.workunit.client.0.smithi046.stdout:7/644: write d5/d9/d12/fa4 [5183620,33725] 0 2024-09-10T04:05:24.113 INFO:tasks.workunit.client.1.smithi070.stdout:1/597: creat d3/d8/d13/fc3 x:0 0 0 2024-09-10T04:05:24.118 INFO:tasks.workunit.client.1.smithi070.stdout:5/654: creat d17/d37/d93/fdf x:0 0 0 2024-09-10T04:05:24.119 INFO:tasks.workunit.client.1.smithi070.stdout:5/655: truncate d17/d19/d6a/d54/f62 4606416 0 2024-09-10T04:05:24.119 INFO:tasks.workunit.client.0.smithi046.stdout:7/645: rename d5/d6/db/d11/d1e/d20 to d5/d6/dbe/dd7 0 2024-09-10T04:05:24.127 INFO:tasks.workunit.client.0.smithi046.stdout:7/646: mknod d5/d6/db/d11/d1e/cd8 0 2024-09-10T04:05:24.128 INFO:tasks.workunit.client.0.smithi046.stdout:7/647: chown d5/d6/db/f29 213193591 1 2024-09-10T04:05:24.128 INFO:tasks.workunit.client.1.smithi070.stdout:5/656: truncate d17/d24/d84/d9b/fb5 323259 0 2024-09-10T04:05:24.128 INFO:tasks.workunit.client.0.smithi046.stdout:0/465: write d1/dc/d25/d33/d3/d29/f3c [587334,56538] 0 2024-09-10T04:05:24.129 INFO:tasks.workunit.client.1.smithi070.stdout:4/637: link de/d13/d43/d88/l99 de/d13/d5b/da0/ld6 0 2024-09-10T04:05:24.129 INFO:tasks.workunit.client.1.smithi070.stdout:4/638: chown de/db0 163828282 1 2024-09-10T04:05:24.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/466: readlink d1/dc/d25/d33/l22 0 2024-09-10T04:05:24.129 INFO:tasks.workunit.client.0.smithi046.stdout:7/648: chown d5/d6/db/dd/d24/db9/lc0 50 1 2024-09-10T04:05:24.131 INFO:tasks.workunit.client.1.smithi070.stdout:8/737: creat d4/dcf/ff8 x:0 0 0 2024-09-10T04:05:24.133 INFO:tasks.workunit.client.0.smithi046.stdout:7/649: symlink d5/d6/db/dd/ld9 0 2024-09-10T04:05:24.135 INFO:tasks.workunit.client.1.smithi070.stdout:0/797: write d8/d13/d1a/d75/ffb [3501993,36052] 0 2024-09-10T04:05:24.136 INFO:tasks.workunit.client.1.smithi070.stdout:5/657: read d17/d24/d45/f79 [303548,53222] 0 2024-09-10T04:05:24.136 INFO:tasks.workunit.client.1.smithi070.stdout:0/798: write d8/d13/f1f [559583,64053] 0 2024-09-10T04:05:24.137 INFO:tasks.workunit.client.1.smithi070.stdout:5/658: readlink d17/d37/d93/lb7 0 2024-09-10T04:05:24.137 INFO:tasks.workunit.client.1.smithi070.stdout:8/738: rename d4/d16/d73/d96/c51 to d4/d16/d17/d1a/df7/cf9 0 2024-09-10T04:05:24.139 INFO:tasks.workunit.client.1.smithi070.stdout:5/659: write d17/d37/d93/fdf [318357,92568] 0 2024-09-10T04:05:24.141 INFO:tasks.workunit.client.1.smithi070.stdout:8/739: chown d4/d16/d17/d1a/d60/d7a/f7b 1352096 1 2024-09-10T04:05:24.141 INFO:tasks.workunit.client.0.smithi046.stdout:0/467: getdents d1/dc/d25/d33/d1a/d80 0 2024-09-10T04:05:24.142 INFO:tasks.workunit.client.1.smithi070.stdout:6/696: sync 2024-09-10T04:05:24.143 INFO:tasks.workunit.client.0.smithi046.stdout:0/468: write d1/dc/d25/d33/d3/dd/d55/d4d/f71 [909936,113009] 0 2024-09-10T04:05:24.144 INFO:tasks.workunit.client.0.smithi046.stdout:0/469: readlink d1/dc/d25/d33/d3/d29/l5d 0 2024-09-10T04:05:24.148 INFO:tasks.workunit.client.1.smithi070.stdout:1/598: dread d3/d8/d13/d3f/f42 [0,4194304] 0 2024-09-10T04:05:24.149 INFO:tasks.workunit.client.1.smithi070.stdout:0/799: rename d8/d24/d2d/d34/d9b/dad to d8/d13/d1a/d8f/d95/d106 0 2024-09-10T04:05:24.151 INFO:tasks.workunit.client.1.smithi070.stdout:5/660: dread f11 [4194304,4194304] 0 2024-09-10T04:05:24.152 INFO:tasks.workunit.client.1.smithi070.stdout:4/639: creat de/d13/d18/d5c/fd7 x:0 0 0 2024-09-10T04:05:24.154 INFO:tasks.workunit.client.0.smithi046.stdout:7/650: dwrite d5/d6/db/d11/f14 [0,4194304] 0 2024-09-10T04:05:24.158 INFO:tasks.workunit.client.0.smithi046.stdout:7/651: unlink d5/d6/db/dd/d24/db7/fcf 0 2024-09-10T04:05:24.162 INFO:tasks.workunit.client.1.smithi070.stdout:5/661: creat d17/d24/d84/d9b/fe0 x:0 0 0 2024-09-10T04:05:24.163 INFO:tasks.workunit.client.1.smithi070.stdout:0/800: read d8/d24/d2d/d34/f5b [641306,30476] 0 2024-09-10T04:05:24.166 INFO:tasks.workunit.client.0.smithi046.stdout:0/470: dwrite d1/dc/f76 [4194304,4194304] 0 2024-09-10T04:05:24.168 INFO:tasks.workunit.client.0.smithi046.stdout:0/471: write d1/dc/d25/d33/d1a/d41/f84 [466996,78292] 0 2024-09-10T04:05:24.208 INFO:tasks.workunit.client.1.smithi070.stdout:2/659: sync 2024-09-10T04:05:24.208 INFO:tasks.workunit.client.1.smithi070.stdout:7/697: sync 2024-09-10T04:05:24.209 INFO:tasks.workunit.client.1.smithi070.stdout:9/730: sync 2024-09-10T04:05:24.209 INFO:tasks.workunit.client.1.smithi070.stdout:3/713: sync 2024-09-10T04:05:24.209 INFO:tasks.workunit.client.1.smithi070.stdout:2/660: chown d15/d18/d99 28920 1 2024-09-10T04:05:24.209 INFO:tasks.workunit.client.1.smithi070.stdout:7/698: chown d2/d3/dc/d5d/d2d 41599 1 2024-09-10T04:05:24.210 INFO:tasks.workunit.client.1.smithi070.stdout:3/714: write db/d1c/d53/fa2 [1681184,43123] 0 2024-09-10T04:05:24.212 INFO:tasks.workunit.client.1.smithi070.stdout:7/699: write d2/d40/d6c/db2/fe3 [681409,116959] 0 2024-09-10T04:05:24.216 INFO:tasks.workunit.client.1.smithi070.stdout:9/731: dwrite d9/d1c/dac/d8a/dc4/dda/de0/d90/d89/fa3 [0,4194304] 0 2024-09-10T04:05:24.284 INFO:tasks.workunit.client.0.smithi046.stdout:7/652: mknod d5/d6/d39/d5f/cda 0 2024-09-10T04:05:24.287 INFO:tasks.workunit.client.0.smithi046.stdout:0/472: dread - d1/dc/d25/d33/d4/f62 zero size 2024-09-10T04:05:24.291 INFO:tasks.workunit.client.1.smithi070.stdout:6/697: dwrite d0/d3/d18/fa2 [0,4194304] 0 2024-09-10T04:05:24.291 INFO:tasks.workunit.client.0.smithi046.stdout:7/653: mknod d5/d6/db/dd/d24/db7/cdb 0 2024-09-10T04:05:24.296 INFO:tasks.workunit.client.0.smithi046.stdout:0/473: dwrite d1/dc/d25/d33/d3/dd/d55/d4d/f71 [0,4194304] 0 2024-09-10T04:05:24.298 INFO:tasks.workunit.client.0.smithi046.stdout:0/474: stat d1/dc/d25/d33/d4/f6f 0 2024-09-10T04:05:24.301 INFO:tasks.workunit.client.0.smithi046.stdout:7/654: symlink d5/d6/db/dd/d4b/ldc 0 2024-09-10T04:05:24.305 INFO:tasks.workunit.client.0.smithi046.stdout:7/655: mknod d5/d6/db/dd/cdd 0 2024-09-10T04:05:24.307 INFO:tasks.workunit.client.1.smithi070.stdout:1/599: write d3/d8/d13/f43 [3457451,78708] 0 2024-09-10T04:05:24.310 INFO:tasks.workunit.client.0.smithi046.stdout:7/656: rename d5/d9/f87 to d5/d6/db/dd/d24/db9/fde 0 2024-09-10T04:05:24.313 INFO:tasks.workunit.client.1.smithi070.stdout:1/600: dwrite d3/d8/d12/d2f/fa9 [0,4194304] 0 2024-09-10T04:05:24.313 INFO:tasks.workunit.client.0.smithi046.stdout:7/657: mknod d5/d9/d12/cdf 0 2024-09-10T04:05:24.314 INFO:tasks.workunit.client.1.smithi070.stdout:1/601: fdatasync d3/d10/db2/fb7 0 2024-09-10T04:05:24.315 INFO:tasks.workunit.client.0.smithi046.stdout:3/601: sync 2024-09-10T04:05:24.316 INFO:tasks.workunit.client.0.smithi046.stdout:3/602: stat d12/d13/d37/d51/f57 0 2024-09-10T04:05:24.319 INFO:tasks.workunit.client.0.smithi046.stdout:7/658: dread d5/d6/d55/d89/fa8 [0,4194304] 0 2024-09-10T04:05:24.322 INFO:tasks.workunit.client.1.smithi070.stdout:1/602: dread d3/d8/d12/f24 [0,4194304] 0 2024-09-10T04:05:24.323 INFO:tasks.workunit.client.0.smithi046.stdout:3/603: unlink d12/d13/d37/d51/db5/d4b/cba 0 2024-09-10T04:05:24.323 INFO:tasks.workunit.client.1.smithi070.stdout:1/603: stat d3/d8/f49 0 2024-09-10T04:05:24.328 INFO:tasks.workunit.client.0.smithi046.stdout:0/475: write d1/dc/d25/d33/d3/dd/d55/f43 [132023,62875] 0 2024-09-10T04:05:24.330 INFO:tasks.workunit.client.0.smithi046.stdout:0/476: write d1/dc/f7b [76228,57410] 0 2024-09-10T04:05:24.331 INFO:tasks.workunit.client.1.smithi070.stdout:0/801: mknod d8/d24/d2d/d5f/c107 0 2024-09-10T04:05:24.332 INFO:tasks.workunit.client.1.smithi070.stdout:0/802: write d8/d13/de2/d81/fda [650080,62797] 0 2024-09-10T04:05:24.335 INFO:tasks.workunit.client.0.smithi046.stdout:0/477: fsync d1/dc/d25/d33/d3/d29/f38 0 2024-09-10T04:05:24.341 INFO:tasks.workunit.client.1.smithi070.stdout:3/715: unlink db/d1c/d25/d28/d43/f9c 0 2024-09-10T04:05:24.343 INFO:tasks.workunit.client.0.smithi046.stdout:0/478: creat d1/dc/d25/d33/d1a/f8e x:0 0 0 2024-09-10T04:05:24.344 INFO:tasks.workunit.client.1.smithi070.stdout:7/700: creat d2/d3/dc/d5d/ff1 x:0 0 0 2024-09-10T04:05:24.345 INFO:tasks.workunit.client.0.smithi046.stdout:0/479: stat d1/d13/d37/f87 0 2024-09-10T04:05:24.346 INFO:tasks.workunit.client.1.smithi070.stdout:4/640: fsync de/d13/d18/d26/f2d 0 2024-09-10T04:05:24.347 INFO:tasks.workunit.client.1.smithi070.stdout:4/641: write f0 [4902297,129024] 0 2024-09-10T04:05:24.351 INFO:tasks.workunit.client.1.smithi070.stdout:8/740: rename d4/d16/d17/d1a/f58 to d4/d16/d17/ffa 0 2024-09-10T04:05:24.351 INFO:tasks.workunit.client.1.smithi070.stdout:7/701: dwrite d2/d4e/f9b [0,4194304] 0 2024-09-10T04:05:24.351 INFO:tasks.workunit.client.0.smithi046.stdout:7/659: write d5/d6/fb5 [911889,32142] 0 2024-09-10T04:05:24.354 INFO:tasks.workunit.client.1.smithi070.stdout:8/741: write d4/d16/d17/d6f/db5/fea [845262,78236] 0 2024-09-10T04:05:24.355 INFO:tasks.workunit.client.1.smithi070.stdout:7/702: chown d2/d7/d9/d18/d23/fe5 67447437 1 2024-09-10T04:05:24.355 INFO:tasks.workunit.client.1.smithi070.stdout:5/662: mkdir d17/d19/d6a/d54/de1 0 2024-09-10T04:05:24.359 INFO:tasks.workunit.client.1.smithi070.stdout:8/742: write d4/fb7 [725978,115733] 0 2024-09-10T04:05:24.359 INFO:tasks.workunit.client.0.smithi046.stdout:3/604: dwrite f3 [0,4194304] 0 2024-09-10T04:05:24.360 INFO:tasks.workunit.client.1.smithi070.stdout:7/703: write d2/d7/fe [1586311,85458] 0 2024-09-10T04:05:24.360 INFO:tasks.workunit.client.1.smithi070.stdout:2/661: write d15/d18/d29/d2a/f67 [3677084,2748] 0 2024-09-10T04:05:24.361 INFO:tasks.workunit.client.1.smithi070.stdout:5/663: chown d17/d19/d6a/d54/d57/d63/fc6 1542267175 1 2024-09-10T04:05:24.361 INFO:tasks.workunit.client.1.smithi070.stdout:4/642: dwrite de/d13/d18/d23/d47/d59/f7d [0,4194304] 0 2024-09-10T04:05:24.363 INFO:tasks.workunit.client.1.smithi070.stdout:8/743: fdatasync d4/d16/d17/d6f/db5/fea 0 2024-09-10T04:05:24.364 INFO:tasks.workunit.client.1.smithi070.stdout:4/643: dread - de/d13/d18/d26/d8c/fba zero size 2024-09-10T04:05:24.366 INFO:tasks.workunit.client.1.smithi070.stdout:4/644: dread - de/d13/d5b/d7b/fb9 zero size 2024-09-10T04:05:24.371 INFO:tasks.workunit.client.0.smithi046.stdout:7/660: rename d5/d6/db/d11 to d5/d9/d12/d72/d77/de0 0 2024-09-10T04:05:24.376 INFO:tasks.workunit.client.1.smithi070.stdout:9/732: unlink d9/d1c/d34/ld3 0 2024-09-10T04:05:24.377 INFO:tasks.workunit.client.0.smithi046.stdout:7/661: creat d5/d9/d12/d72/d77/de0/d1f/fe1 x:0 0 0 2024-09-10T04:05:24.379 INFO:tasks.workunit.client.0.smithi046.stdout:3/605: fsync d12/d13/d37/d51/f62 0 2024-09-10T04:05:24.382 INFO:tasks.workunit.client.0.smithi046.stdout:7/662: mknod d5/d6/ce2 0 2024-09-10T04:05:24.382 INFO:tasks.workunit.client.1.smithi070.stdout:9/733: dwrite d9/d59/d2a/d3f/daf/fc5 [0,4194304] 0 2024-09-10T04:05:24.386 INFO:tasks.workunit.client.0.smithi046.stdout:3/606: truncate d12/d13/d37/f45 1303817 0 2024-09-10T04:05:24.387 INFO:tasks.workunit.client.1.smithi070.stdout:1/604: dwrite d3/d10/f5e [0,4194304] 0 2024-09-10T04:05:24.392 INFO:tasks.workunit.client.0.smithi046.stdout:3/607: mknod d12/d13/d14/d1e/d2d/d6f/cc9 0 2024-09-10T04:05:24.393 INFO:tasks.workunit.client.0.smithi046.stdout:3/608: chown d12/l16 3785 1 2024-09-10T04:05:24.394 INFO:tasks.workunit.client.1.smithi070.stdout:0/803: symlink d8/deb/l108 0 2024-09-10T04:05:24.397 INFO:tasks.workunit.client.0.smithi046.stdout:3/609: rmdir d12 39 2024-09-10T04:05:24.397 INFO:tasks.workunit.client.0.smithi046.stdout:7/663: dwrite d5/d9/d12/d72/d77/de0/fd5 [0,4194304] 0 2024-09-10T04:05:24.401 INFO:tasks.workunit.client.1.smithi070.stdout:5/664: creat d17/d19/d6a/d54/d57/da8/fe2 x:0 0 0 2024-09-10T04:05:24.402 INFO:tasks.workunit.client.1.smithi070.stdout:8/744: fdatasync d4/d16/f33 0 2024-09-10T04:05:24.404 INFO:tasks.workunit.client.1.smithi070.stdout:8/745: chown d4/d16/f33 751181 1 2024-09-10T04:05:24.405 INFO:tasks.workunit.client.1.smithi070.stdout:6/698: dwrite d0/d3/d18/d1b/d17/f90 [0,4194304] 0 2024-09-10T04:05:24.407 INFO:tasks.workunit.client.1.smithi070.stdout:6/699: write d0/d3/d18/d1b/f7d [5137282,49942] 0 2024-09-10T04:05:24.410 INFO:tasks.workunit.client.0.smithi046.stdout:7/664: rename d5/d6/d55/d7d/fd4 to d5/d9/d12/d72/d77/de0/fe3 0 2024-09-10T04:05:24.418 INFO:tasks.workunit.client.1.smithi070.stdout:7/704: creat d2/d3/d63/da9/dc1/d9f/ff2 x:0 0 0 2024-09-10T04:05:24.422 INFO:tasks.workunit.client.0.smithi046.stdout:3/610: getdents d12/d13/d37/d51/db5/d50/d7a/dc1/d96 0 2024-09-10T04:05:24.425 INFO:tasks.workunit.client.1.smithi070.stdout:0/804: write d8/d13/d1a/d8f/d95/fa7 [1662329,49460] 0 2024-09-10T04:05:24.427 INFO:tasks.workunit.client.0.smithi046.stdout:3/611: write d12/d13/d14/f25 [679648,125679] 0 2024-09-10T04:05:24.427 INFO:tasks.workunit.client.0.smithi046.stdout:7/665: dwrite d5/d6/db/f29 [0,4194304] 0 2024-09-10T04:05:24.428 INFO:tasks.workunit.client.1.smithi070.stdout:3/716: creat db/d1c/ff6 x:0 0 0 2024-09-10T04:05:24.428 INFO:tasks.workunit.client.0.smithi046.stdout:7/666: chown d5/d33 1269 1 2024-09-10T04:05:24.430 INFO:tasks.workunit.client.1.smithi070.stdout:5/665: mknod d17/d6f/d32/d90/da5/ce3 0 2024-09-10T04:05:24.430 INFO:tasks.workunit.client.0.smithi046.stdout:3/612: rmdir d12/d13/d37 39 2024-09-10T04:05:24.432 INFO:tasks.workunit.client.1.smithi070.stdout:5/666: fsync d17/d19/d35/f5f 0 2024-09-10T04:05:24.433 INFO:tasks.workunit.client.1.smithi070.stdout:6/700: creat d0/d3/d8c/d9a/fd9 x:0 0 0 2024-09-10T04:05:24.434 INFO:tasks.workunit.client.1.smithi070.stdout:5/667: fsync d17/d24/fd8 0 2024-09-10T04:05:24.435 INFO:tasks.workunit.client.1.smithi070.stdout:2/662: creat d15/d18/d56/d7c/fe7 x:0 0 0 2024-09-10T04:05:24.436 INFO:tasks.workunit.client.0.smithi046.stdout:7/667: rename d5/d9/d12/d72/d77/c84 to d5/d9/d12/d72/d77/ce4 0 2024-09-10T04:05:24.437 INFO:tasks.workunit.client.1.smithi070.stdout:0/805: mknod d8/d13/d47/c109 0 2024-09-10T04:05:24.438 INFO:tasks.workunit.client.1.smithi070.stdout:3/717: dwrite db/d1c/d3e/f54 [4194304,4194304] 0 2024-09-10T04:05:24.439 INFO:tasks.workunit.client.1.smithi070.stdout:0/806: chown d8/d24/d2d/l41 8818 1 2024-09-10T04:05:24.439 INFO:tasks.workunit.client.1.smithi070.stdout:3/718: write db/d83/d92/faa [666841,89328] 0 2024-09-10T04:05:24.442 INFO:tasks.workunit.client.0.smithi046.stdout:7/668: creat d5/d6/d55/d89/fe5 x:0 0 0 2024-09-10T04:05:24.442 INFO:tasks.workunit.client.1.smithi070.stdout:9/734: link d9/d59/c9c d9/d1c/dac/d8a/dc4/ceb 0 2024-09-10T04:05:24.444 INFO:tasks.workunit.client.1.smithi070.stdout:3/719: dread db/d1c/d53/fa2 [0,4194304] 0 2024-09-10T04:05:24.444 INFO:tasks.workunit.client.1.smithi070.stdout:1/605: link d3/d8/d12/d2f/d32/f6b d3/d8/d13/fc4 0 2024-09-10T04:05:24.447 INFO:tasks.workunit.client.1.smithi070.stdout:4/645: getdents de/d13/d18/d23/d47/d59/d79/dbc 0 2024-09-10T04:05:24.448 INFO:tasks.workunit.client.0.smithi046.stdout:7/669: creat d5/d6/db/dd/d24/d76/fe6 x:0 0 0 2024-09-10T04:05:24.451 INFO:tasks.workunit.client.1.smithi070.stdout:9/735: readlink d9/d7a/l53 0 2024-09-10T04:05:24.451 INFO:tasks.workunit.client.1.smithi070.stdout:5/668: mknod d17/d24/ce4 0 2024-09-10T04:05:24.451 INFO:tasks.workunit.client.1.smithi070.stdout:5/669: stat fd 0 2024-09-10T04:05:24.454 INFO:tasks.workunit.client.1.smithi070.stdout:7/705: creat d2/d3/d63/da9/ff3 x:0 0 0 2024-09-10T04:05:24.456 INFO:tasks.workunit.client.0.smithi046.stdout:7/670: dwrite d5/d9/d12/d72/d77/f82 [0,4194304] 0 2024-09-10T04:05:24.456 INFO:tasks.workunit.client.1.smithi070.stdout:6/701: symlink d0/d3/d18/d1b/lda 0 2024-09-10T04:05:24.456 INFO:tasks.workunit.client.1.smithi070.stdout:7/706: fdatasync d2/d3/dc/d5d/f1d 0 2024-09-10T04:05:24.457 INFO:tasks.workunit.client.1.smithi070.stdout:7/707: dread - d2/d3/d63/fe0 zero size 2024-09-10T04:05:24.462 INFO:tasks.workunit.client.1.smithi070.stdout:8/746: getdents d4/dbf/d74 0 2024-09-10T04:05:24.465 INFO:tasks.workunit.client.1.smithi070.stdout:9/736: rmdir d9/d1c/d34/d63 39 2024-09-10T04:05:24.466 INFO:tasks.workunit.client.1.smithi070.stdout:0/807: dread d8/d13/f28 [0,4194304] 0 2024-09-10T04:05:24.466 INFO:tasks.workunit.client.1.smithi070.stdout:3/720: fsync db/d1c/f5f 0 2024-09-10T04:05:24.467 INFO:tasks.workunit.client.0.smithi046.stdout:7/671: link d5/d6/db/dd/d24/caa d5/d9/d43/db4/ce7 0 2024-09-10T04:05:24.467 INFO:tasks.workunit.client.1.smithi070.stdout:0/808: read d8/d13/f28 [1208911,84938] 0 2024-09-10T04:05:24.468 INFO:tasks.workunit.client.0.smithi046.stdout:7/672: fsync d5/d6/d39/d5f/f96 0 2024-09-10T04:05:24.468 INFO:tasks.workunit.client.1.smithi070.stdout:5/670: fdatasync f11 0 2024-09-10T04:05:24.469 INFO:tasks.workunit.client.1.smithi070.stdout:0/809: dread d8/d13/d1a/f32 [4194304,4194304] 0 2024-09-10T04:05:24.470 INFO:tasks.workunit.client.1.smithi070.stdout:5/671: chown d17/d19/d35/l70 0 1 2024-09-10T04:05:24.471 INFO:tasks.workunit.client.1.smithi070.stdout:1/606: rename d3/d8/fc to d3/d8/d13/d67/d83/fc5 0 2024-09-10T04:05:24.472 INFO:tasks.workunit.client.1.smithi070.stdout:1/607: fdatasync d3/d10/d1d/dbd/da2/fa6 0 2024-09-10T04:05:24.473 INFO:tasks.workunit.client.0.smithi046.stdout:7/673: mknod d5/d6/d55/d89/ce8 0 2024-09-10T04:05:24.474 INFO:tasks.workunit.client.1.smithi070.stdout:4/646: symlink de/db0/ld8 0 2024-09-10T04:05:24.474 INFO:tasks.workunit.client.0.smithi046.stdout:7/674: write d5/d6/d55/d89/f9d [955104,118673] 0 2024-09-10T04:05:24.478 INFO:tasks.workunit.client.0.smithi046.stdout:7/675: creat d5/d6/db/dd/d24/db7/fe9 x:0 0 0 2024-09-10T04:05:24.479 INFO:tasks.workunit.client.1.smithi070.stdout:9/737: dread d9/d7a/d74/fa5 [0,4194304] 0 2024-09-10T04:05:24.479 INFO:tasks.workunit.client.1.smithi070.stdout:2/663: dwrite d15/d18/f1a [0,4194304] 0 2024-09-10T04:05:24.479 INFO:tasks.workunit.client.0.smithi046.stdout:7/676: write d5/d9/d12/d72/d77/de0/d1f/fe1 [37004,56087] 0 2024-09-10T04:05:24.481 INFO:tasks.workunit.client.1.smithi070.stdout:6/702: rename d0/d3/d1f/d93/cb3 to d0/d3/d18/d1b/d17/d54/cdb 0 2024-09-10T04:05:24.482 INFO:tasks.workunit.client.1.smithi070.stdout:1/608: mknod d3/d90/cc6 0 2024-09-10T04:05:24.483 INFO:tasks.workunit.client.0.smithi046.stdout:7/677: mknod d5/d9/d12/d72/d77/cea 0 2024-09-10T04:05:24.485 INFO:tasks.workunit.client.1.smithi070.stdout:8/747: link d4/f99 d4/d16/d17/d19/d28/da8/ffb 0 2024-09-10T04:05:24.486 INFO:tasks.workunit.client.0.smithi046.stdout:7/678: write d5/d9/d12/d72/d77/de0/fce [343018,129206] 0 2024-09-10T04:05:24.487 INFO:tasks.workunit.client.0.smithi046.stdout:7/679: chown d5/d6/d39/cc9 183 1 2024-09-10T04:05:24.487 INFO:tasks.workunit.client.1.smithi070.stdout:8/748: stat d4/dbf/d74/da4/fbd 0 2024-09-10T04:05:24.489 INFO:tasks.workunit.client.1.smithi070.stdout:7/708: creat d2/d7/ff4 x:0 0 0 2024-09-10T04:05:24.491 INFO:tasks.workunit.client.1.smithi070.stdout:2/664: readlink d15/d59/da1/lbf 0 2024-09-10T04:05:24.493 INFO:tasks.workunit.client.1.smithi070.stdout:9/738: creat d9/d1c/dac/d8a/d3c/d92/fec x:0 0 0 2024-09-10T04:05:24.493 INFO:tasks.workunit.client.0.smithi046.stdout:7/680: dwrite d5/f35 [0,4194304] 0 2024-09-10T04:05:24.494 INFO:tasks.workunit.client.1.smithi070.stdout:7/709: stat d2/d3/dc/d5d/d2d/cdb 0 2024-09-10T04:05:24.494 INFO:tasks.workunit.client.1.smithi070.stdout:6/703: dwrite d0/d3/d18/d1b/d17/d54/db5/fbb [0,4194304] 0 2024-09-10T04:05:24.497 INFO:tasks.workunit.client.0.smithi046.stdout:7/681: creat d5/d6/d32/dab/feb x:0 0 0 2024-09-10T04:05:24.498 INFO:tasks.workunit.client.0.smithi046.stdout:8/572: sync 2024-09-10T04:05:24.499 INFO:tasks.workunit.client.0.smithi046.stdout:2/608: sync 2024-09-10T04:05:24.501 INFO:tasks.workunit.client.1.smithi070.stdout:7/710: truncate d2/d7/db/feb 76877 0 2024-09-10T04:05:24.502 INFO:tasks.workunit.client.1.smithi070.stdout:9/739: unlink d9/fad 0 2024-09-10T04:05:24.503 INFO:tasks.workunit.client.0.smithi046.stdout:7/682: mknod d5/d9/d43/cec 0 2024-09-10T04:05:24.505 INFO:tasks.workunit.client.0.smithi046.stdout:2/609: creat d2/d86/d9c/fe1 x:0 0 0 2024-09-10T04:05:24.506 INFO:tasks.workunit.client.1.smithi070.stdout:8/749: dread d4/f12 [0,4194304] 0 2024-09-10T04:05:24.507 INFO:tasks.workunit.client.1.smithi070.stdout:2/665: rename d15/d18/d56/d7c/d55/d6d/f97 to d15/d18/db2/fe8 0 2024-09-10T04:05:24.509 INFO:tasks.workunit.client.0.smithi046.stdout:8/573: symlink d1/d29/d17/d1f/d46/d6a/lbd 0 2024-09-10T04:05:24.510 INFO:tasks.workunit.client.0.smithi046.stdout:8/574: dread - d1/d29/d19/d36/fb8 zero size 2024-09-10T04:05:24.510 INFO:tasks.workunit.client.1.smithi070.stdout:0/810: truncate d8/f27 4613257 0 2024-09-10T04:05:24.511 INFO:tasks.workunit.client.1.smithi070.stdout:3/721: getdents db/d79/db6 0 2024-09-10T04:05:24.512 INFO:tasks.workunit.client.1.smithi070.stdout:5/672: write d17/d37/f3d [400629,100881] 0 2024-09-10T04:05:24.512 INFO:tasks.workunit.client.1.smithi070.stdout:0/811: write d8/d13/d1a/d75/ffb [1900517,57935] 0 2024-09-10T04:05:24.513 INFO:tasks.workunit.client.0.smithi046.stdout:7/683: creat d5/d9/d12/d72/fed x:0 0 0 2024-09-10T04:05:24.513 INFO:tasks.workunit.client.1.smithi070.stdout:3/722: fdatasync db/d1c/d25/d28/d43/fa4 0 2024-09-10T04:05:24.519 INFO:tasks.workunit.client.0.smithi046.stdout:8/575: symlink d1/d29/d17/d1f/d46/d67/lbe 0 2024-09-10T04:05:24.519 INFO:tasks.workunit.client.1.smithi070.stdout:4/647: truncate de/d13/d18/d5c/f96 828362 0 2024-09-10T04:05:24.520 INFO:tasks.workunit.client.1.smithi070.stdout:4/648: dread - de/d13/d18/d23/d47/fcd zero size 2024-09-10T04:05:24.520 INFO:tasks.workunit.client.1.smithi070.stdout:4/649: stat de/d13/d43/c94 0 2024-09-10T04:05:24.523 INFO:tasks.workunit.client.1.smithi070.stdout:1/609: truncate d3/d10/d1d/f30 3862579 0 2024-09-10T04:05:24.526 INFO:tasks.workunit.client.1.smithi070.stdout:8/750: rename d4/d16/d17/d19/d28 to d4/d39/dfc 0 2024-09-10T04:05:24.528 INFO:tasks.workunit.client.0.smithi046.stdout:7/684: dread d5/d6/d32/dab/fb1 [0,4194304] 0 2024-09-10T04:05:24.529 INFO:tasks.workunit.client.0.smithi046.stdout:7/685: dread - d5/d9/d43/db4/fc8 zero size 2024-09-10T04:05:24.529 INFO:tasks.workunit.client.1.smithi070.stdout:2/666: mkdir d15/d18/db2/de9 0 2024-09-10T04:05:24.531 INFO:tasks.workunit.client.1.smithi070.stdout:6/704: write d0/d3/d18/d1b/d33/f65 [1347621,48339] 0 2024-09-10T04:05:24.531 INFO:tasks.workunit.client.0.smithi046.stdout:2/610: write d2/d8/d37/f66 [2127949,82430] 0 2024-09-10T04:05:24.539 INFO:tasks.workunit.client.0.smithi046.stdout:2/611: dwrite d2/d6/d23/d5d/d77/f69 [0,4194304] 0 2024-09-10T04:05:24.541 INFO:tasks.workunit.client.1.smithi070.stdout:3/723: creat db/d79/db6/dc3/ff7 x:0 0 0 2024-09-10T04:05:24.542 INFO:tasks.workunit.client.0.smithi046.stdout:8/576: link d1/d29/d19/d36/d4c/c77 d1/d29/d19/d36/d4c/cbf 0 2024-09-10T04:05:24.547 INFO:tasks.workunit.client.0.smithi046.stdout:8/577: mkdir d1/d29/d19/d36/d4c/d5e/dc0 0 2024-09-10T04:05:24.548 INFO:tasks.workunit.client.1.smithi070.stdout:4/650: symlink de/d13/d18/ld9 0 2024-09-10T04:05:24.550 INFO:tasks.workunit.client.1.smithi070.stdout:9/740: fsync d9/d1c/dac/d8a/dc4/de1/f67 0 2024-09-10T04:05:24.552 INFO:tasks.workunit.client.1.smithi070.stdout:8/751: mknod d4/d16/d17/d1a/df7/cfd 0 2024-09-10T04:05:24.554 INFO:tasks.workunit.client.1.smithi070.stdout:5/673: dread d17/f75 [0,4194304] 0 2024-09-10T04:05:24.554 INFO:tasks.workunit.client.0.smithi046.stdout:7/686: dwrite d5/d6/d32/f74 [0,4194304] 0 2024-09-10T04:05:24.554 INFO:tasks.workunit.client.0.smithi046.stdout:8/578: rename d1/d29/d17/d1f/d46/d52 to d1/d29/d17/d1f/d46/d67/dc1 0 2024-09-10T04:05:24.555 INFO:tasks.workunit.client.1.smithi070.stdout:8/752: write d4/d16/d17/d19/d62/fd4 [3779943,28892] 0 2024-09-10T04:05:24.555 INFO:tasks.workunit.client.0.smithi046.stdout:7/687: readlink d5/d7f/ld0 0 2024-09-10T04:05:24.556 INFO:tasks.workunit.client.1.smithi070.stdout:6/705: mknod d0/d3/d5c/cdc 0 2024-09-10T04:05:24.556 INFO:tasks.workunit.client.1.smithi070.stdout:5/674: write d17/d19/d35/f5f [606694,104089] 0 2024-09-10T04:05:24.559 INFO:tasks.workunit.client.1.smithi070.stdout:4/651: dwrite de/d13/d43/d88/fcf [0,4194304] 0 2024-09-10T04:05:24.564 INFO:tasks.workunit.client.1.smithi070.stdout:8/753: dread d4/d39/dfc/d43/d67/fc3 [0,4194304] 0 2024-09-10T04:05:24.565 INFO:tasks.workunit.client.0.smithi046.stdout:8/579: dwrite d1/d29/d17/d1f/d46/fa8 [0,4194304] 0 2024-09-10T04:05:24.566 INFO:tasks.workunit.client.1.smithi070.stdout:8/754: chown d4/d39/dfc/d43/d67/dec 559 1 2024-09-10T04:05:24.567 INFO:tasks.workunit.client.0.smithi046.stdout:7/688: dwrite d5/d9/d12/d72/d77/f82 [0,4194304] 0 2024-09-10T04:05:24.570 INFO:tasks.workunit.client.1.smithi070.stdout:2/667: symlink d15/d18/d29/ddb/lea 0 2024-09-10T04:05:24.571 INFO:tasks.workunit.client.0.smithi046.stdout:8/580: mkdir d1/d29/d17/d1f/d46/d6a/d82/dc2 0 2024-09-10T04:05:24.573 INFO:tasks.workunit.client.1.smithi070.stdout:9/741: creat d9/d1c/d34/d8b/fed x:0 0 0 2024-09-10T04:05:24.574 INFO:tasks.workunit.client.1.smithi070.stdout:1/610: dread d3/d8/d12/d2f/d32/f41 [0,4194304] 0 2024-09-10T04:05:24.575 INFO:tasks.workunit.client.0.smithi046.stdout:1/474: sync 2024-09-10T04:05:24.575 INFO:tasks.workunit.client.0.smithi046.stdout:9/579: sync 2024-09-10T04:05:24.575 INFO:tasks.workunit.client.0.smithi046.stdout:4/766: sync 2024-09-10T04:05:24.575 INFO:tasks.workunit.client.1.smithi070.stdout:8/755: dwrite d4/d16/dcc/fe8 [0,4194304] 0 2024-09-10T04:05:24.577 INFO:tasks.workunit.client.0.smithi046.stdout:9/580: readlink d4/d7/l2e 0 2024-09-10T04:05:24.577 INFO:tasks.workunit.client.1.smithi070.stdout:6/706: dread d0/d3/d18/f4c [0,4194304] 0 2024-09-10T04:05:24.577 INFO:tasks.workunit.client.0.smithi046.stdout:4/767: chown d3/d12/d17/d22/d51/c65 172916 1 2024-09-10T04:05:24.580 INFO:tasks.workunit.client.1.smithi070.stdout:6/707: chown d0/d3/d18/d1b/d17/d54/faf 2234906 1 2024-09-10T04:05:24.580 INFO:tasks.workunit.client.1.smithi070.stdout:8/756: write d4/d39/dfc/d43/f5d [1753168,7099] 0 2024-09-10T04:05:24.582 INFO:tasks.workunit.client.1.smithi070.stdout:5/675: dread d17/d6f/d32/f68 [0,4194304] 0 2024-09-10T04:05:24.584 INFO:tasks.workunit.client.1.smithi070.stdout:0/812: link d8/d13/de2/f3f d8/d24/f10a 0 2024-09-10T04:05:24.585 INFO:tasks.workunit.client.0.smithi046.stdout:1/475: dwrite d3/d27/d32/d1f/f29 [0,4194304] 0 2024-09-10T04:05:24.586 INFO:tasks.workunit.client.1.smithi070.stdout:0/813: dread d8/d13/d1a/d75/f84 [0,4194304] 0 2024-09-10T04:05:24.588 INFO:tasks.workunit.client.1.smithi070.stdout:0/814: write d8/d13/d1a/f3c [365326,99774] 0 2024-09-10T04:05:24.593 INFO:tasks.workunit.client.1.smithi070.stdout:4/652: rename de/d13/d18/d23/d47/d59/d79/c68 to de/d13/d43/d88/cda 0 2024-09-10T04:05:24.595 INFO:tasks.workunit.client.0.smithi046.stdout:8/581: write d1/f79 [224853,94632] 0 2024-09-10T04:05:24.595 INFO:tasks.workunit.client.1.smithi070.stdout:7/711: link d2/d7/d9/d18/d3c/d52/l8c d2/lf5 0 2024-09-10T04:05:24.597 INFO:tasks.workunit.client.0.smithi046.stdout:9/581: mknod d4/d7/d81/cb1 0 2024-09-10T04:05:24.599 INFO:tasks.workunit.client.0.smithi046.stdout:7/689: getdents d5/d6/d55/d89 0 2024-09-10T04:05:24.600 INFO:tasks.workunit.client.1.smithi070.stdout:2/668: creat d15/d59/feb x:0 0 0 2024-09-10T04:05:24.603 INFO:tasks.workunit.client.1.smithi070.stdout:9/742: read d9/d1c/dac/d8a/d3c/f52 [735699,106934] 0 2024-09-10T04:05:24.610 INFO:tasks.workunit.client.0.smithi046.stdout:4/768: link d3/f3c d3/d12/d17/d29/d58/db2/d5f/f10a 0 2024-09-10T04:05:24.611 INFO:tasks.workunit.client.0.smithi046.stdout:4/769: chown d3/d12/c81 0 1 2024-09-10T04:05:24.612 INFO:tasks.workunit.client.1.smithi070.stdout:6/708: chown d0/d3/d1f/d93/ld8 641 1 2024-09-10T04:05:24.612 INFO:tasks.workunit.client.1.smithi070.stdout:6/709: fsync d0/d3/f16 0 2024-09-10T04:05:24.613 INFO:tasks.workunit.client.0.smithi046.stdout:4/770: write d3/d12/d17/d29/f38 [3243655,118829] 0 2024-09-10T04:05:24.613 INFO:tasks.workunit.client.1.smithi070.stdout:6/710: chown d0/d3/d18 567 1 2024-09-10T04:05:24.614 INFO:tasks.workunit.client.0.smithi046.stdout:1/476: rename d3/d27/d32/d28 to d3/d9b 0 2024-09-10T04:05:24.615 INFO:tasks.workunit.client.1.smithi070.stdout:5/676: mkdir d17/d24/d45/dc0/de5 0 2024-09-10T04:05:24.616 INFO:tasks.workunit.client.0.smithi046.stdout:8/582: dread d1/d29/d19/d36/d4c/f5b [0,4194304] 0 2024-09-10T04:05:24.617 INFO:tasks.workunit.client.0.smithi046.stdout:7/690: symlink d5/d9/d12/d72/d77/de0/d1e/lee 0 2024-09-10T04:05:24.617 INFO:tasks.workunit.client.0.smithi046.stdout:8/583: chown d1/d29/d19/d36/d4c 3972972 1 2024-09-10T04:05:24.622 INFO:tasks.workunit.client.0.smithi046.stdout:7/691: dread d5/d9/d12/d72/d77/de0/fd5 [0,4194304] 0 2024-09-10T04:05:24.623 INFO:tasks.workunit.client.0.smithi046.stdout:8/584: dwrite d1/d29/d12/f57 [0,4194304] 0 2024-09-10T04:05:24.697 INFO:tasks.workunit.client.1.smithi070.stdout:3/724: creat db/ff8 x:0 0 0 2024-09-10T04:05:24.701 INFO:tasks.workunit.client.1.smithi070.stdout:8/757: dwrite d4/dbf/d74/da4/fbd [0,4194304] 0 2024-09-10T04:05:24.739 INFO:tasks.workunit.client.0.smithi046.stdout:6/573: sync 2024-09-10T04:05:24.762 INFO:tasks.workunit.client.0.smithi046.stdout:9/582: rename d4/d7/d1f/l91 to d4/d18/d92/lb2 0 2024-09-10T04:05:24.766 INFO:tasks.workunit.client.1.smithi070.stdout:1/611: rename d3/d8/d12/d2f/d32/d4b to d3/d10/db2/dc7 0 2024-09-10T04:05:24.770 INFO:tasks.workunit.client.0.smithi046.stdout:1/477: dread - d3/d27/f66 zero size 2024-09-10T04:05:24.775 INFO:tasks.workunit.client.1.smithi070.stdout:2/669: fsync f10 0 2024-09-10T04:05:24.778 INFO:tasks.workunit.client.1.smithi070.stdout:9/743: symlink d9/d1c/d34/d8b/lee 0 2024-09-10T04:05:24.780 INFO:tasks.workunit.client.0.smithi046.stdout:7/692: read d5/d9/d12/f5d [1186960,24144] 0 2024-09-10T04:05:24.785 INFO:tasks.workunit.client.0.smithi046.stdout:6/574: mknod da/db/d11/d13/d1e/d69/cbd 0 2024-09-10T04:05:24.791 INFO:tasks.workunit.client.0.smithi046.stdout:4/771: rename d3/d12/d17/d29/ca0 to d3/d12/d17/d29/d58/db2/d5f/d85/da8/c10b 0 2024-09-10T04:05:24.793 INFO:tasks.workunit.client.0.smithi046.stdout:4/772: write d3/d12/d17/d29/d58/db2/d5f/d85/fd7 [1761986,33086] 0 2024-09-10T04:05:24.795 INFO:tasks.workunit.client.0.smithi046.stdout:4/773: chown d3/d12/d17/d22/d51/d5c/fde 0 1 2024-09-10T04:05:24.797 INFO:tasks.workunit.client.1.smithi070.stdout:6/711: dread d0/d3/f98 [0,4194304] 0 2024-09-10T04:05:24.801 INFO:tasks.workunit.client.0.smithi046.stdout:7/693: mkdir d5/d9/d43/db4/def 0 2024-09-10T04:05:24.803 INFO:tasks.workunit.client.0.smithi046.stdout:7/694: write d5/d6/db/dd/d24/d76/d9c/fb2 [862070,33660] 0 2024-09-10T04:05:24.803 INFO:tasks.workunit.client.0.smithi046.stdout:8/585: getdents d1/d29/d17/d1f/d46/d67/d80 0 2024-09-10T04:05:24.808 INFO:tasks.workunit.client.1.smithi070.stdout:7/712: creat d2/d3/dc/d13/dbd/ff6 x:0 0 0 2024-09-10T04:05:24.809 INFO:tasks.workunit.client.1.smithi070.stdout:7/713: write d2/d7/d9/f82 [5694465,58968] 0 2024-09-10T04:05:24.811 INFO:tasks.workunit.client.0.smithi046.stdout:4/774: fsync d3/d12/d17/d22/d30/faa 0 2024-09-10T04:05:24.814 INFO:tasks.workunit.client.0.smithi046.stdout:7/695: mknod d5/d6/d32/dab/db6/cf0 0 2024-09-10T04:05:24.815 INFO:tasks.workunit.client.1.smithi070.stdout:9/744: fdatasync d9/d59/f1a 0 2024-09-10T04:05:24.818 INFO:tasks.workunit.client.0.smithi046.stdout:8/586: symlink d1/d29/d17/d1f/d46/d67/dc1/lc3 0 2024-09-10T04:05:24.819 INFO:tasks.workunit.client.1.smithi070.stdout:5/677: symlink d17/d19/d6a/d54/de1/le6 0 2024-09-10T04:05:24.823 INFO:tasks.workunit.client.0.smithi046.stdout:4/775: symlink d3/d12/d17/d22/d30/dc7/l10c 0 2024-09-10T04:05:24.826 INFO:tasks.workunit.client.1.smithi070.stdout:6/712: creat d0/d3/d18/d2a/fdd x:0 0 0 2024-09-10T04:05:24.828 INFO:tasks.workunit.client.0.smithi046.stdout:7/696: dread d5/d6/d32/f5e [0,4194304] 0 2024-09-10T04:05:24.829 INFO:tasks.workunit.client.0.smithi046.stdout:1/478: creat d3/dc/d34/d51/f9c x:0 0 0 2024-09-10T04:05:24.830 INFO:tasks.workunit.client.0.smithi046.stdout:7/697: chown d5/d9/d12/d72/d77/de0/d1e/cba 9914 1 2024-09-10T04:05:24.831 INFO:tasks.workunit.client.1.smithi070.stdout:6/713: dwrite d0/d3/d18/d1b/f3a [0,4194304] 0 2024-09-10T04:05:24.832 INFO:tasks.workunit.client.0.smithi046.stdout:8/587: truncate d1/d29/d19/d36/d4c/f7d 710382 0 2024-09-10T04:05:24.833 INFO:tasks.workunit.client.0.smithi046.stdout:8/588: write d1/d29/d19/d36/d4c/d5e/f7f [4384079,61263] 0 2024-09-10T04:05:24.837 INFO:tasks.workunit.client.1.smithi070.stdout:7/714: mknod d2/d3/dc/d5d/d41/da8/cf7 0 2024-09-10T04:05:24.839 INFO:tasks.workunit.client.0.smithi046.stdout:1/479: symlink d3/d9b/l9d 0 2024-09-10T04:05:24.840 INFO:tasks.workunit.client.0.smithi046.stdout:1/480: fdatasync d3/dc/f84 0 2024-09-10T04:05:24.842 INFO:tasks.workunit.client.0.smithi046.stdout:8/589: dread d1/d29/d17/d1f/d46/f8b [0,4194304] 0 2024-09-10T04:05:24.846 INFO:tasks.workunit.client.1.smithi070.stdout:9/745: unlink d9/d1c/d34/d8b/l8e 0 2024-09-10T04:05:24.849 INFO:tasks.workunit.client.1.smithi070.stdout:5/678: creat d17/d6f/d32/d90/da5/fe7 x:0 0 0 2024-09-10T04:05:24.850 INFO:tasks.workunit.client.1.smithi070.stdout:5/679: dread - d17/d19/d6a/d54/d57/da8/fdc zero size 2024-09-10T04:05:24.850 INFO:tasks.workunit.client.1.smithi070.stdout:5/680: read - d17/d24/dbf/fd3 zero size 2024-09-10T04:05:24.856 INFO:tasks.workunit.client.0.smithi046.stdout:1/481: dread d3/d27/d32/f2b [0,4194304] 0 2024-09-10T04:05:24.857 INFO:tasks.workunit.client.1.smithi070.stdout:9/746: creat d9/d1c/d34/d8b/fef x:0 0 0 2024-09-10T04:05:24.857 INFO:tasks.workunit.client.0.smithi046.stdout:1/482: write d3/f77 [201637,13784] 0 2024-09-10T04:05:24.861 INFO:tasks.workunit.client.1.smithi070.stdout:7/715: creat d2/d3/dc/d5d/d41/dd6/de6/ff8 x:0 0 0 2024-09-10T04:05:24.863 INFO:tasks.workunit.client.1.smithi070.stdout:9/747: dread d9/d1c/dac/d8a/d3c/d92/fb1 [0,4194304] 0 2024-09-10T04:05:24.865 INFO:tasks.workunit.client.0.smithi046.stdout:1/483: read d3/d27/d32/d1f/f61 [54804,5470] 0 2024-09-10T04:05:24.865 INFO:tasks.workunit.client.0.smithi046.stdout:7/698: getdents d5/d6/db/dd/d4b 0 2024-09-10T04:05:24.869 INFO:tasks.workunit.client.0.smithi046.stdout:1/484: fdatasync d3/d27/d32/d1f/f55 0 2024-09-10T04:05:24.871 INFO:tasks.workunit.client.0.smithi046.stdout:1/485: read d3/d27/f41 [1256391,59831] 0 2024-09-10T04:05:24.874 INFO:tasks.workunit.client.0.smithi046.stdout:7/699: creat d5/d6/d55/d7d/ff1 x:0 0 0 2024-09-10T04:05:24.874 INFO:tasks.workunit.client.0.smithi046.stdout:5/742: sync 2024-09-10T04:05:24.876 INFO:tasks.workunit.client.0.smithi046.stdout:7/700: dread - d5/d6/dbe/dd7/fbb zero size 2024-09-10T04:05:24.878 INFO:tasks.workunit.client.0.smithi046.stdout:1/486: dwrite d3/d62/f73 [0,4194304] 0 2024-09-10T04:05:24.882 INFO:tasks.workunit.client.0.smithi046.stdout:5/743: mknod d7/d11/d14/d2f/d2b/c10a 0 2024-09-10T04:05:24.887 INFO:tasks.workunit.client.0.smithi046.stdout:5/744: creat d7/d11/d14/d2f/deb/da5/dcc/de5/f10b x:0 0 0 2024-09-10T04:05:24.889 INFO:tasks.workunit.client.0.smithi046.stdout:5/745: fsync d7/d11/d3f/d53/da9/f83 0 2024-09-10T04:05:24.896 INFO:tasks.workunit.client.1.smithi070.stdout:4/653: rename de/d13/d5b/d9e to de/d13/da6/ddb 0 2024-09-10T04:05:24.898 INFO:tasks.workunit.client.1.smithi070.stdout:3/725: rename db/d1c/d25/d28/d43/f97 to db/d1c/d25/d28/dd3/ff9 0 2024-09-10T04:05:24.899 INFO:tasks.workunit.client.1.smithi070.stdout:3/726: write db/d83/fce [983182,71567] 0 2024-09-10T04:05:24.900 INFO:tasks.workunit.client.1.smithi070.stdout:4/654: read de/d13/d18/d23/d47/d59/d79/f66 [536194,32704] 0 2024-09-10T04:05:24.900 INFO:tasks.workunit.client.1.smithi070.stdout:6/714: rename d0/d3/d8c/d9a/d92 to d0/d3/d18/d1b/d17/d54/dc4/dd2/dde 0 2024-09-10T04:05:24.902 INFO:tasks.workunit.client.1.smithi070.stdout:5/681: symlink d17/d6f/d32/le8 0 2024-09-10T04:05:24.905 INFO:tasks.workunit.client.1.smithi070.stdout:6/715: mknod d0/d3/cdf 0 2024-09-10T04:05:24.905 INFO:tasks.workunit.client.1.smithi070.stdout:4/655: dread de/d13/d18/d23/d47/d59/d79/f66 [0,4194304] 0 2024-09-10T04:05:24.905 INFO:tasks.workunit.client.1.smithi070.stdout:5/682: mknod d17/d37/ce9 0 2024-09-10T04:05:24.906 INFO:tasks.workunit.client.1.smithi070.stdout:9/748: mkdir d9/d1c/dac/d8a/dc4/dda/de0/d90/df0 0 2024-09-10T04:05:24.909 INFO:tasks.workunit.client.1.smithi070.stdout:3/727: truncate db/d83/fac 186064 0 2024-09-10T04:05:24.912 INFO:tasks.workunit.client.1.smithi070.stdout:5/683: symlink d17/d19/d6a/d54/d57/da3/lea 0 2024-09-10T04:05:24.913 INFO:tasks.workunit.client.1.smithi070.stdout:9/749: mknod d9/d1c/dac/d8a/d3c/d92/da8/cf1 0 2024-09-10T04:05:24.916 INFO:tasks.workunit.client.1.smithi070.stdout:4/656: mknod de/d13/d18/d23/d47/cdc 0 2024-09-10T04:05:24.917 INFO:tasks.workunit.client.1.smithi070.stdout:5/684: stat d17/d24/d2b/laf 0 2024-09-10T04:05:24.919 INFO:tasks.workunit.client.1.smithi070.stdout:6/716: dread d0/d3/f51 [0,4194304] 0 2024-09-10T04:05:24.930 INFO:tasks.workunit.client.1.smithi070.stdout:5/685: rmdir d17/d24/d77 39 2024-09-10T04:05:24.934 INFO:tasks.workunit.client.1.smithi070.stdout:5/686: unlink d17/d6f/d32/c38 0 2024-09-10T04:05:24.934 INFO:tasks.workunit.client.1.smithi070.stdout:5/687: stat d17/d24/d2b/cc1 0 2024-09-10T04:05:24.940 INFO:tasks.workunit.client.1.smithi070.stdout:5/688: write d17/d24/d45/f4f [541123,121577] 0 2024-09-10T04:05:24.941 INFO:tasks.workunit.client.1.smithi070.stdout:5/689: truncate d17/d37/d93/fdf 1105347 0 2024-09-10T04:05:24.947 INFO:tasks.workunit.client.1.smithi070.stdout:5/690: symlink d17/d19/d6a/d54/d57/d63/dcd/leb 0 2024-09-10T04:05:24.948 INFO:tasks.workunit.client.1.smithi070.stdout:5/691: readlink d17/d24/d84/l9c 0 2024-09-10T04:05:24.951 INFO:tasks.workunit.client.1.smithi070.stdout:5/692: symlink d17/d19/d6a/d54/d57/da8/lec 0 2024-09-10T04:05:24.956 INFO:tasks.workunit.client.1.smithi070.stdout:5/693: creat d17/d24/d77/db4/fed x:0 0 0 2024-09-10T04:05:24.960 INFO:tasks.workunit.client.1.smithi070.stdout:8/758: dwrite d4/d39/dfc/f30 [0,4194304] 0 2024-09-10T04:05:24.960 INFO:tasks.workunit.client.1.smithi070.stdout:5/694: write d17/d6f/d32/d90/da5/fe7 [822553,62968] 0 2024-09-10T04:05:24.962 INFO:tasks.workunit.client.1.smithi070.stdout:8/759: chown d4/d39/l3b 2654 1 2024-09-10T04:05:24.964 INFO:tasks.workunit.client.1.smithi070.stdout:0/815: dwrite d8/d13/d1a/f31 [0,4194304] 0 2024-09-10T04:05:24.965 INFO:tasks.workunit.client.1.smithi070.stdout:5/695: creat d17/d24/d84/fee x:0 0 0 2024-09-10T04:05:24.970 INFO:tasks.workunit.client.1.smithi070.stdout:2/670: dwrite f3 [0,4194304] 0 2024-09-10T04:05:24.973 INFO:tasks.workunit.client.1.smithi070.stdout:8/760: creat d4/d16/d17/d1a/d60/d7a/d84/ffe x:0 0 0 2024-09-10T04:05:24.977 INFO:tasks.workunit.client.1.smithi070.stdout:2/671: mknod d15/d18/d29/d2a/d2e/d32/d36/cec 0 2024-09-10T04:05:24.979 INFO:tasks.workunit.client.1.smithi070.stdout:8/761: mknod d4/d16/d73/d96/cff 0 2024-09-10T04:05:24.980 INFO:tasks.workunit.client.1.smithi070.stdout:5/696: creat d17/d19/d6a/d54/fef x:0 0 0 2024-09-10T04:05:24.981 INFO:tasks.workunit.client.1.smithi070.stdout:2/672: mkdir d15/d18/db2/ded 0 2024-09-10T04:05:24.983 INFO:tasks.workunit.client.1.smithi070.stdout:8/762: symlink d4/d16/d17/d6f/db5/l100 0 2024-09-10T04:05:24.984 INFO:tasks.workunit.client.1.smithi070.stdout:2/673: read d15/d18/d56/d7c/d55/d6d/f7d [5204572,116264] 0 2024-09-10T04:05:24.988 INFO:tasks.workunit.client.1.smithi070.stdout:8/763: creat d4/dcf/f101 x:0 0 0 2024-09-10T04:05:24.990 INFO:tasks.workunit.client.1.smithi070.stdout:8/764: read d4/d16/f36 [4028815,50462] 0 2024-09-10T04:05:24.990 INFO:tasks.workunit.client.1.smithi070.stdout:2/674: mknod d15/d18/d29/ddb/cee 0 2024-09-10T04:05:24.992 INFO:tasks.workunit.client.1.smithi070.stdout:5/697: dread d17/d6f/f52 [0,4194304] 0 2024-09-10T04:05:24.995 INFO:tasks.workunit.client.1.smithi070.stdout:5/698: creat d17/d24/d84/dab/ff0 x:0 0 0 2024-09-10T04:05:24.997 INFO:tasks.workunit.client.1.smithi070.stdout:2/675: link d15/d2b/d6e/f89 d15/d18/d56/d7c/fef 0 2024-09-10T04:05:24.998 INFO:tasks.workunit.client.1.smithi070.stdout:1/612: creat d3/d8/fc8 x:0 0 0 2024-09-10T04:05:24.999 INFO:tasks.workunit.client.1.smithi070.stdout:2/676: truncate d15/d18/d56/d7c/f92 846464 0 2024-09-10T04:05:24.999 INFO:tasks.workunit.client.1.smithi070.stdout:5/699: dread d17/d6f/d32/d90/d99/fc9 [0,4194304] 0 2024-09-10T04:05:25.001 INFO:tasks.workunit.client.1.smithi070.stdout:5/700: fdatasync d17/d19/d6a/d54/d57/da8/fdc 0 2024-09-10T04:05:25.001 INFO:tasks.workunit.client.1.smithi070.stdout:1/613: symlink d3/d8/d13/d67/d83/lc9 0 2024-09-10T04:05:25.002 INFO:tasks.workunit.client.1.smithi070.stdout:1/614: chown d3/d8/d12/d2f/d32 1109617 1 2024-09-10T04:05:25.003 INFO:tasks.workunit.client.1.smithi070.stdout:5/701: symlink d17/d6f/d32/d90/da5/lf1 0 2024-09-10T04:05:25.005 INFO:tasks.workunit.client.1.smithi070.stdout:1/615: chown d3/d8/d12/d2f/d32/da8/lb6 13146 1 2024-09-10T04:05:25.007 INFO:tasks.workunit.client.1.smithi070.stdout:2/677: rmdir d15/d2b/d6e/db8/dcf 0 2024-09-10T04:05:25.008 INFO:tasks.workunit.client.1.smithi070.stdout:0/816: mknod d8/d13/c10b 0 2024-09-10T04:05:25.010 INFO:tasks.workunit.client.1.smithi070.stdout:2/678: creat d15/d18/d29/d2a/d2e/da6/ff0 x:0 0 0 2024-09-10T04:05:25.011 INFO:tasks.workunit.client.0.smithi046.stdout:2/612: sync 2024-09-10T04:05:25.011 INFO:tasks.workunit.client.0.smithi046.stdout:0/480: sync 2024-09-10T04:05:25.012 INFO:tasks.workunit.client.0.smithi046.stdout:3/613: sync 2024-09-10T04:05:25.014 INFO:tasks.workunit.client.1.smithi070.stdout:7/716: truncate d2/d3/dc/d5d/d42/fc4 892629 0 2024-09-10T04:05:25.015 INFO:tasks.workunit.client.1.smithi070.stdout:7/717: fdatasync d2/d7/db/f5c 0 2024-09-10T04:05:25.018 INFO:tasks.workunit.client.1.smithi070.stdout:1/616: dread d3/d8/d13/f17 [0,4194304] 0 2024-09-10T04:05:25.018 INFO:tasks.workunit.client.0.smithi046.stdout:0/481: truncate d1/d13/d37/f61 2624808 0 2024-09-10T04:05:25.019 INFO:tasks.workunit.client.0.smithi046.stdout:2/613: symlink d2/d86/d9c/dbc/le2 0 2024-09-10T04:05:25.020 INFO:tasks.workunit.client.1.smithi070.stdout:2/679: link d15/d18/cd1 d15/d2b/d6e/db8/cf1 0 2024-09-10T04:05:25.021 INFO:tasks.workunit.client.0.smithi046.stdout:3/614: chown d12/d13/d37/d51/db5/d50/c7e 9 1 2024-09-10T04:05:25.022 INFO:tasks.workunit.client.1.smithi070.stdout:1/617: symlink d3/d8/d27/d9a/lca 0 2024-09-10T04:05:25.024 INFO:tasks.workunit.client.0.smithi046.stdout:2/614: read d2/f25 [336486,70564] 0 2024-09-10T04:05:25.026 INFO:tasks.workunit.client.0.smithi046.stdout:2/615: write d2/d6/d23/f4b [2277972,4370] 0 2024-09-10T04:05:25.026 INFO:tasks.workunit.client.1.smithi070.stdout:2/680: creat d15/d18/d29/d2a/d2e/da6/dc0/dcb/ff2 x:0 0 0 2024-09-10T04:05:25.026 INFO:tasks.workunit.client.0.smithi046.stdout:1/487: write d3/dc/d34/f72 [31792,111209] 0 2024-09-10T04:05:25.027 INFO:tasks.workunit.client.0.smithi046.stdout:3/615: dread - d12/d13/d37/d51/db5/d50/d7a/fa4 zero size 2024-09-10T04:05:25.028 INFO:tasks.workunit.client.0.smithi046.stdout:3/616: read d12/d13/d14/f15 [5021159,4780] 0 2024-09-10T04:05:25.030 INFO:tasks.workunit.client.0.smithi046.stdout:3/617: write d12/d13/d37/d94/dbf/fc3 [382268,123880] 0 2024-09-10T04:05:25.032 INFO:tasks.workunit.client.0.smithi046.stdout:2/616: dread d2/d6/d23/d5d/d77/f69 [0,4194304] 0 2024-09-10T04:05:25.032 INFO:tasks.workunit.client.1.smithi070.stdout:2/681: symlink d15/d2b/lf3 0 2024-09-10T04:05:25.033 INFO:tasks.workunit.client.1.smithi070.stdout:2/682: write d15/d18/d29/f3d [3608074,104596] 0 2024-09-10T04:05:25.034 INFO:tasks.workunit.client.1.smithi070.stdout:4/657: rename de/d13/d18/d23/d47/d59/l9d to de/d13/d5b/da0/ldd 0 2024-09-10T04:05:25.036 INFO:tasks.workunit.client.0.smithi046.stdout:5/746: write d7/d11/d14/d2f/f48 [1318396,45396] 0 2024-09-10T04:05:25.036 INFO:tasks.workunit.client.1.smithi070.stdout:7/718: getdents d2/d40 0 2024-09-10T04:05:25.037 INFO:tasks.workunit.client.0.smithi046.stdout:5/747: fsync d7/d11/d14/d2f/fd6 0 2024-09-10T04:05:25.041 INFO:tasks.workunit.client.0.smithi046.stdout:0/482: creat d1/f8f x:0 0 0 2024-09-10T04:05:25.042 INFO:tasks.workunit.client.1.smithi070.stdout:4/658: dwrite de/d13/d5b/d7b/dd5/fc7 [0,4194304] 0 2024-09-10T04:05:25.048 INFO:tasks.workunit.client.0.smithi046.stdout:3/618: dread d12/d13/d37/d51/f8e [0,4194304] 0 2024-09-10T04:05:25.054 INFO:tasks.workunit.client.1.smithi070.stdout:4/659: dread de/d13/d5b/d7b/dd5/f71 [0,4194304] 0 2024-09-10T04:05:25.057 INFO:tasks.workunit.client.0.smithi046.stdout:3/619: dwrite d12/d13/d37/d51/db5/d50/d7a/dc1/fbb [0,4194304] 0 2024-09-10T04:05:25.061 INFO:tasks.workunit.client.0.smithi046.stdout:5/748: unlink d7/d11/d3f/d53/l6c 0 2024-09-10T04:05:25.064 INFO:tasks.workunit.client.0.smithi046.stdout:0/483: creat d1/dc/d25/d33/d1a/d64/f90 x:0 0 0 2024-09-10T04:05:25.064 INFO:tasks.workunit.client.1.smithi070.stdout:4/660: rmdir de/d13/d5b/da0 39 2024-09-10T04:05:25.065 INFO:tasks.workunit.client.1.smithi070.stdout:3/728: write db/d1c/d25/d28/f41 [1572959,43935] 0 2024-09-10T04:05:25.065 INFO:tasks.workunit.client.0.smithi046.stdout:3/620: dwrite f4 [0,4194304] 0 2024-09-10T04:05:25.067 INFO:tasks.workunit.client.0.smithi046.stdout:2/617: creat d2/d6/d23/d42/db4/dca/dd6/fe3 x:0 0 0 2024-09-10T04:05:25.071 INFO:tasks.workunit.client.0.smithi046.stdout:4/776: symlink d3/l10d 0 2024-09-10T04:05:25.073 INFO:tasks.workunit.client.1.smithi070.stdout:4/661: mknod de/d13/d18/cde 0 2024-09-10T04:05:25.074 INFO:tasks.workunit.client.1.smithi070.stdout:3/729: chown db/d1c/d3e/fdc 247072615 1 2024-09-10T04:05:25.076 INFO:tasks.workunit.client.1.smithi070.stdout:4/662: fsync de/d13/f3e 0 2024-09-10T04:05:25.081 INFO:tasks.workunit.client.0.smithi046.stdout:5/749: mknod d7/c10c 0 2024-09-10T04:05:25.085 INFO:tasks.workunit.client.0.smithi046.stdout:3/621: link d12/d13/d37/faa d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fca 0 2024-09-10T04:05:25.087 INFO:tasks.workunit.client.1.smithi070.stdout:4/663: rename de/d13/d18/d23/l27 to de/d13/d5b/da0/ldf 0 2024-09-10T04:05:25.089 INFO:tasks.workunit.client.0.smithi046.stdout:5/750: read d7/d11/d14/f25 [669364,67694] 0 2024-09-10T04:05:25.091 INFO:tasks.workunit.client.0.smithi046.stdout:0/484: creat d1/f91 x:0 0 0 2024-09-10T04:05:25.092 INFO:tasks.workunit.client.0.smithi046.stdout:0/485: dread - d1/dc/d25/d33/d1a/f47 zero size 2024-09-10T04:05:25.093 INFO:tasks.workunit.client.1.smithi070.stdout:4/664: rename de/d13/d18/d23/d47/cdc to de/d13/d5b/d7b/dd5/ce0 0 2024-09-10T04:05:25.095 INFO:tasks.workunit.client.0.smithi046.stdout:3/622: rename d12/d13/d37/d51/db5/d4b/db7/lb9 to d12/d13/d37/d51/dbd/lcb 0 2024-09-10T04:05:25.097 INFO:tasks.workunit.client.1.smithi070.stdout:4/665: creat de/d13/d5b/d7b/dd5/daf/fe1 x:0 0 0 2024-09-10T04:05:25.099 INFO:tasks.workunit.client.0.smithi046.stdout:4/777: getdents d3/d12/d17/d29/ddd 0 2024-09-10T04:05:25.100 INFO:tasks.workunit.client.1.smithi070.stdout:4/666: creat de/d13/d5b/d7b/fe2 x:0 0 0 2024-09-10T04:05:25.101 INFO:tasks.workunit.client.1.smithi070.stdout:4/667: chown de/d13/d18/d26/f75 31 1 2024-09-10T04:05:25.101 INFO:tasks.workunit.client.0.smithi046.stdout:5/751: dread d7/d11/d14/d2f/d2b/d6a/f82 [0,4194304] 0 2024-09-10T04:05:25.102 INFO:tasks.workunit.client.0.smithi046.stdout:0/486: mknod d1/dc/d25/d33/d3/dd/d55/d77/c92 0 2024-09-10T04:05:25.104 INFO:tasks.workunit.client.0.smithi046.stdout:0/487: write d1/dc/d25/d33/d3/d29/d2d/d5e/f89 [569553,25298] 0 2024-09-10T04:05:25.109 INFO:tasks.workunit.client.0.smithi046.stdout:0/488: fsync d1/d13/d37/f87 0 2024-09-10T04:05:25.112 INFO:tasks.workunit.client.0.smithi046.stdout:0/489: fdatasync d1/dc/d25/d33/d1a/d41/f5a 0 2024-09-10T04:05:25.118 INFO:tasks.workunit.client.0.smithi046.stdout:0/490: dwrite d1/dc/d25/d33/d1a/f8e [0,4194304] 0 2024-09-10T04:05:25.119 INFO:tasks.workunit.client.0.smithi046.stdout:8/590: write d1/d29/d19/d36/d4c/f7d [295033,89231] 0 2024-09-10T04:05:25.124 INFO:tasks.workunit.client.0.smithi046.stdout:8/591: unlink d1/d29/d17/d1f/d2a/c96 0 2024-09-10T04:05:25.125 INFO:tasks.workunit.client.0.smithi046.stdout:0/491: mknod d1/dc/d25/d33/d3/dd/d55/d4d/c93 0 2024-09-10T04:05:25.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/492: truncate d1/dc/d25/d33/d1a/d41/f6e 755461 0 2024-09-10T04:05:25.132 INFO:tasks.workunit.client.0.smithi046.stdout:0/493: dread - d1/d13/d37/d75/f8b zero size 2024-09-10T04:05:25.133 INFO:tasks.workunit.client.0.smithi046.stdout:8/592: rmdir d1/d29/d17/d1f/d46/d67/d80 0 2024-09-10T04:05:25.133 INFO:tasks.workunit.client.0.smithi046.stdout:0/494: mkdir d1/d13/d37/d75/d94 0 2024-09-10T04:05:25.133 INFO:tasks.workunit.client.0.smithi046.stdout:8/593: creat d1/d29/d17/d1f/d46/d6a/d82/fc4 x:0 0 0 2024-09-10T04:05:25.137 INFO:tasks.workunit.client.0.smithi046.stdout:0/495: creat d1/dc/f95 x:0 0 0 2024-09-10T04:05:25.138 INFO:tasks.workunit.client.1.smithi070.stdout:0/817: write d8/d24/d2d/f51 [429664,66965] 0 2024-09-10T04:05:25.138 INFO:tasks.workunit.client.0.smithi046.stdout:0/496: chown d1/dc/d25/d33/d3/d29/l78 1 1 2024-09-10T04:05:25.140 INFO:tasks.workunit.client.0.smithi046.stdout:0/497: write d1/dc/f27 [3345640,57346] 0 2024-09-10T04:05:25.142 INFO:tasks.workunit.client.0.smithi046.stdout:0/498: fsync d1/dc/d25/d33/d3/dd/d55/f43 0 2024-09-10T04:05:25.145 INFO:tasks.workunit.client.0.smithi046.stdout:8/594: dread d1/f39 [0,4194304] 0 2024-09-10T04:05:25.146 INFO:tasks.workunit.client.1.smithi070.stdout:0/818: creat d8/d13/de2/d7d/f10c x:0 0 0 2024-09-10T04:05:25.147 INFO:tasks.workunit.client.1.smithi070.stdout:0/819: write d8/d24/d2d/d5f/f100 [868662,121498] 0 2024-09-10T04:05:25.150 INFO:tasks.workunit.client.0.smithi046.stdout:0/499: creat d1/dc/d25/d33/d3/dd/f96 x:0 0 0 2024-09-10T04:05:25.151 INFO:tasks.workunit.client.0.smithi046.stdout:0/500: write d1/f6c [804953,110999] 0 2024-09-10T04:05:25.151 INFO:tasks.workunit.client.0.smithi046.stdout:8/595: fsync d1/f2e 0 2024-09-10T04:05:25.153 INFO:tasks.workunit.client.1.smithi070.stdout:5/702: mkdir d17/d19/d6a/d54/d57/df2 0 2024-09-10T04:05:25.156 INFO:tasks.workunit.client.1.smithi070.stdout:0/820: rename d8/d13/d1a/c44 to d8/d13/d47/c10d 0 2024-09-10T04:05:25.158 INFO:tasks.workunit.client.1.smithi070.stdout:5/703: creat d17/d24/d77/db4/ff3 x:0 0 0 2024-09-10T04:05:25.160 INFO:tasks.workunit.client.1.smithi070.stdout:1/618: dwrite d3/d8/d13/fc4 [0,4194304] 0 2024-09-10T04:05:25.160 INFO:tasks.workunit.client.0.smithi046.stdout:1/488: write d3/dc/f23 [1590459,31800] 0 2024-09-10T04:05:25.161 INFO:tasks.workunit.client.0.smithi046.stdout:8/596: mknod d1/d29/d19/d36/cc5 0 2024-09-10T04:05:25.162 INFO:tasks.workunit.client.0.smithi046.stdout:8/597: readlink d1/d29/d17/l24 0 2024-09-10T04:05:25.167 INFO:tasks.workunit.client.0.smithi046.stdout:1/489: rmdir d3 39 2024-09-10T04:05:25.168 INFO:tasks.workunit.client.1.smithi070.stdout:0/821: creat d8/d85/df5/f10e x:0 0 0 2024-09-10T04:05:25.171 INFO:tasks.workunit.client.0.smithi046.stdout:8/598: rename d1/d29/d17/f2f to d1/d29/d17/d1f/d46/d67/dc1/fc6 0 2024-09-10T04:05:25.173 INFO:tasks.workunit.client.1.smithi070.stdout:1/619: mkdir d3/d90/dcb 0 2024-09-10T04:05:25.173 INFO:tasks.workunit.client.0.smithi046.stdout:0/501: link d1/d13/d37/d75/f7f d1/dc/d25/d33/d1a/f97 0 2024-09-10T04:05:25.174 INFO:tasks.workunit.client.1.smithi070.stdout:2/683: truncate d15/d18/d29/d2a/fbe 2469975 0 2024-09-10T04:05:25.175 INFO:tasks.workunit.client.1.smithi070.stdout:2/684: chown d15/d18/d56/ca5 436705188 1 2024-09-10T04:05:25.177 INFO:tasks.workunit.client.0.smithi046.stdout:8/599: mkdir d1/d29/d19/d36/d4c/d5e/dc7 0 2024-09-10T04:05:25.179 INFO:tasks.workunit.client.0.smithi046.stdout:8/600: write d1/f86 [809583,33628] 0 2024-09-10T04:05:25.180 INFO:tasks.workunit.client.1.smithi070.stdout:7/719: truncate d2/d3/dc/d5d/d42/f7c 2016159 0 2024-09-10T04:05:25.181 INFO:tasks.workunit.client.0.smithi046.stdout:0/502: dwrite d1/dc/d25/d33/d3/dd/d55/d4d/f71 [4194304,4194304] 0 2024-09-10T04:05:25.183 INFO:tasks.workunit.client.0.smithi046.stdout:1/490: dread - d3/dc/d34/d51/f9c zero size 2024-09-10T04:05:25.189 INFO:tasks.workunit.client.0.smithi046.stdout:0/503: rename d1/dc/l2e to d1/dc/d25/d72/l98 0 2024-09-10T04:05:25.191 INFO:tasks.workunit.client.0.smithi046.stdout:1/491: creat d3/d9b/f9e x:0 0 0 2024-09-10T04:05:25.192 INFO:tasks.workunit.client.1.smithi070.stdout:3/730: truncate db/d83/d92/faa 807863 0 2024-09-10T04:05:25.195 INFO:tasks.workunit.client.0.smithi046.stdout:0/504: symlink d1/d13/l99 0 2024-09-10T04:05:25.197 INFO:tasks.workunit.client.0.smithi046.stdout:8/601: link d1/d29/d19/d36/f3f d1/d29/d17/d1f/fc8 0 2024-09-10T04:05:25.200 INFO:tasks.workunit.client.0.smithi046.stdout:8/602: read d1/d29/d17/d1f/d46/d6a/d82/d99/fb3 [3539,87668] 0 2024-09-10T04:05:25.201 INFO:tasks.workunit.client.0.smithi046.stdout:3/623: write d12/d13/d37/f81 [1858096,117255] 0 2024-09-10T04:05:25.204 INFO:tasks.workunit.client.0.smithi046.stdout:5/752: write d7/d11/d14/f59 [395018,54244] 0 2024-09-10T04:05:25.205 INFO:tasks.workunit.client.0.smithi046.stdout:5/753: fdatasync d7/d11/d3f/d53/f47 0 2024-09-10T04:05:25.206 INFO:tasks.workunit.client.0.smithi046.stdout:4/778: truncate d3/f9 1458317 0 2024-09-10T04:05:25.206 INFO:tasks.workunit.client.0.smithi046.stdout:6/575: sync 2024-09-10T04:05:25.206 INFO:tasks.workunit.client.0.smithi046.stdout:9/583: sync 2024-09-10T04:05:25.208 INFO:tasks.workunit.client.0.smithi046.stdout:8/603: getdents d1/d29/d19/d36/d4c/d5e/dc7 0 2024-09-10T04:05:25.208 INFO:tasks.workunit.client.0.smithi046.stdout:5/754: write d7/d11/d14/d45/d51/f86 [4252663,9336] 0 2024-09-10T04:05:25.208 INFO:tasks.workunit.client.1.smithi070.stdout:4/668: dwrite de/f34 [0,4194304] 0 2024-09-10T04:05:25.209 INFO:tasks.workunit.client.1.smithi070.stdout:4/669: dread - de/d13/d18/d5c/db6/fd4 zero size 2024-09-10T04:05:25.211 INFO:tasks.workunit.client.0.smithi046.stdout:3/624: unlink d12/d13/d37/d51/f72 0 2024-09-10T04:05:25.221 INFO:tasks.workunit.client.0.smithi046.stdout:8/604: chown d1/d29/d19/f34 3904 1 2024-09-10T04:05:25.224 INFO:tasks.workunit.client.1.smithi070.stdout:4/670: dread de/d13/d18/d23/f3c [0,4194304] 0 2024-09-10T04:05:25.225 INFO:tasks.workunit.client.0.smithi046.stdout:6/576: dwrite da/db/d10/d27/f29 [0,4194304] 0 2024-09-10T04:05:25.330 INFO:tasks.workunit.client.0.smithi046.stdout:5/755: mkdir d7/d11/d3f/d53/d99/d10d 0 2024-09-10T04:05:25.331 INFO:tasks.workunit.client.0.smithi046.stdout:5/756: read d7/d11/d3f/d53/da9/f62 [1378888,65888] 0 2024-09-10T04:05:25.333 INFO:tasks.workunit.client.0.smithi046.stdout:5/757: write d7/d11/d14/d2f/d2b/f7a [606869,106635] 0 2024-09-10T04:05:25.333 INFO:tasks.workunit.client.0.smithi046.stdout:9/584: symlink d4/d3c/lb3 0 2024-09-10T04:05:25.374 INFO:tasks.workunit.client.0.smithi046.stdout:7/701: sync 2024-09-10T04:05:25.374 INFO:tasks.workunit.client.0.smithi046.stdout:7/702: write d5/d6/db/dd/d4b/fc7 [871798,39363] 0 2024-09-10T04:05:25.401 INFO:tasks.workunit.client.1.smithi070.stdout:0/822: symlink d8/d13/de2/d81/l10f 0 2024-09-10T04:05:25.402 INFO:tasks.workunit.client.1.smithi070.stdout:0/823: chown d8/d24/d2d/d8d/l67 12939857 1 2024-09-10T04:05:25.404 INFO:tasks.workunit.client.0.smithi046.stdout:8/605: mkdir d1/d29/d17/d1f/d46/dc9 0 2024-09-10T04:05:25.414 INFO:tasks.workunit.client.0.smithi046.stdout:9/585: mknod d4/d66/cb4 0 2024-09-10T04:05:25.418 INFO:tasks.workunit.client.0.smithi046.stdout:7/703: creat d5/d6/d32/ff2 x:0 0 0 2024-09-10T04:05:25.419 INFO:tasks.workunit.client.0.smithi046.stdout:5/758: dwrite d7/d11/d3f/d5a/d63/f7f [0,4194304] 0 2024-09-10T04:05:25.422 INFO:tasks.workunit.client.0.smithi046.stdout:4/779: rename d3/d12/d17/d22/d51/d5c/d100/f101 to d3/d12/d17/d29/d58/db2/d7a/f10e 0 2024-09-10T04:05:25.430 INFO:tasks.workunit.client.0.smithi046.stdout:5/759: creat d7/d11/d14/d2f/deb/f10e x:0 0 0 2024-09-10T04:05:25.435 INFO:tasks.workunit.client.0.smithi046.stdout:4/780: read d3/fb8 [3749491,3940] 0 2024-09-10T04:05:25.438 INFO:tasks.workunit.client.1.smithi070.stdout:3/731: rename db/d1c/f5f to db/d1c/d25/ffa 0 2024-09-10T04:05:25.439 INFO:tasks.workunit.client.0.smithi046.stdout:5/760: dwrite d7/d11/d3f/d53/d81/fe8 [0,4194304] 0 2024-09-10T04:05:25.441 INFO:tasks.workunit.client.0.smithi046.stdout:6/577: creat da/db/d11/d2b/fbe x:0 0 0 2024-09-10T04:05:25.442 INFO:tasks.workunit.client.0.smithi046.stdout:5/761: dread d7/d11/d3f/d53/d81/fd0 [0,4194304] 0 2024-09-10T04:05:25.450 INFO:tasks.workunit.client.0.smithi046.stdout:4/781: symlink d3/d12/d17/d91/dbf/l10f 0 2024-09-10T04:05:25.452 INFO:tasks.workunit.client.0.smithi046.stdout:4/782: truncate d3/dab/dca/fe6 5946047 0 2024-09-10T04:05:25.453 INFO:tasks.workunit.client.1.smithi070.stdout:3/732: rename db/d1c/f4f to db/d1c/d25/dd8/ffb 0 2024-09-10T04:05:25.454 INFO:tasks.workunit.client.1.smithi070.stdout:3/733: chown db/f74 986 1 2024-09-10T04:05:25.458 INFO:tasks.workunit.client.0.smithi046.stdout:6/578: rename da/db/d10/d27/d8f to da/db/d11/d13/d1e/d23/d86/dbf 0 2024-09-10T04:05:25.461 INFO:tasks.workunit.client.1.smithi070.stdout:3/734: mkdir db/d83/dfc 0 2024-09-10T04:05:25.461 INFO:tasks.workunit.client.1.smithi070.stdout:3/735: chown db/l64 48 1 2024-09-10T04:05:25.461 INFO:tasks.workunit.client.0.smithi046.stdout:5/762: dwrite d7/d11/f37 [0,4194304] 0 2024-09-10T04:05:25.463 INFO:tasks.workunit.client.0.smithi046.stdout:5/763: stat d7/d11/d3f/d5a/d63/f7f 0 2024-09-10T04:05:25.465 INFO:tasks.workunit.client.0.smithi046.stdout:5/764: fdatasync d7/d11/d14/d2f/d2b/f7a 0 2024-09-10T04:05:25.467 INFO:tasks.workunit.client.0.smithi046.stdout:6/579: mknod da/dc/d22/cc0 0 2024-09-10T04:05:25.467 INFO:tasks.workunit.client.0.smithi046.stdout:5/765: chown d7/d11/d14/d2f/deb/da5/dcc/dc0/fe0 47539682 1 2024-09-10T04:05:25.468 INFO:tasks.workunit.client.1.smithi070.stdout:3/736: dwrite db/d1c/d25/dd8/fe8 [0,4194304] 0 2024-09-10T04:05:25.474 INFO:tasks.workunit.client.0.smithi046.stdout:6/580: symlink da/db/d11/d13/d4f/lc1 0 2024-09-10T04:05:25.476 INFO:tasks.workunit.client.0.smithi046.stdout:5/766: mknod d7/d11/d14/d8e/d96/dce/c10f 0 2024-09-10T04:05:25.482 INFO:tasks.workunit.client.0.smithi046.stdout:5/767: dread d7/d11/d14/f25 [0,4194304] 0 2024-09-10T04:05:25.483 INFO:tasks.workunit.client.1.smithi070.stdout:5/704: truncate d17/d24/d45/f4f 1320615 0 2024-09-10T04:05:25.483 INFO:tasks.workunit.client.1.smithi070.stdout:5/705: fsync d17/d24/d77/db4/fed 0 2024-09-10T04:05:25.484 INFO:tasks.workunit.client.0.smithi046.stdout:5/768: creat d7/d11/d14/d2f/d2b/d92/f110 x:0 0 0 2024-09-10T04:05:25.486 INFO:tasks.workunit.client.1.smithi070.stdout:2/685: write d15/d18/d29/d2a/d2e/da6/dc0/dcb/fb9 [1021046,21081] 0 2024-09-10T04:05:25.487 INFO:tasks.workunit.client.0.smithi046.stdout:5/769: creat d7/d11/d14/d2f/deb/f111 x:0 0 0 2024-09-10T04:05:25.488 INFO:tasks.workunit.client.0.smithi046.stdout:3/625: truncate f4 1828443 0 2024-09-10T04:05:25.489 INFO:tasks.workunit.client.1.smithi070.stdout:5/706: rename d17/d19/d6a/d54/fef to d17/ff4 0 2024-09-10T04:05:25.490 INFO:tasks.workunit.client.1.smithi070.stdout:5/707: fsync d17/d19/d35/f72 0 2024-09-10T04:05:25.490 INFO:tasks.workunit.client.1.smithi070.stdout:2/686: creat d15/d59/ff4 x:0 0 0 2024-09-10T04:05:25.492 INFO:tasks.workunit.client.0.smithi046.stdout:5/770: read d7/d11/d14/d2f/d64/fc6 [117989,59411] 0 2024-09-10T04:05:25.495 INFO:tasks.workunit.client.1.smithi070.stdout:2/687: unlink d15/d2b/d6e/fd0 0 2024-09-10T04:05:25.495 INFO:tasks.workunit.client.1.smithi070.stdout:2/688: readlink d15/d18/d29/d2a/l50 0 2024-09-10T04:05:25.496 INFO:tasks.workunit.client.0.smithi046.stdout:3/626: symlink d12/d5d/lcc 0 2024-09-10T04:05:25.497 INFO:tasks.workunit.client.1.smithi070.stdout:0/824: write d8/d13/d1a/d75/f84 [1139826,89720] 0 2024-09-10T04:05:25.501 INFO:tasks.workunit.client.1.smithi070.stdout:5/708: creat d17/d6f/d32/ff5 x:0 0 0 2024-09-10T04:05:25.503 INFO:tasks.workunit.client.1.smithi070.stdout:1/620: write d3/d8/d13/d3f/f52 [2431644,83732] 0 2024-09-10T04:05:25.504 INFO:tasks.workunit.client.0.smithi046.stdout:8/606: truncate d1/f39 1179074 0 2024-09-10T04:05:25.505 INFO:tasks.workunit.client.1.smithi070.stdout:1/621: dread - d3/d8/d12/d2f/d32/f58 zero size 2024-09-10T04:05:25.507 INFO:tasks.workunit.client.1.smithi070.stdout:1/622: chown d3/d8/d13/d67/d83/fc5 45000660 1 2024-09-10T04:05:25.508 INFO:tasks.workunit.client.0.smithi046.stdout:3/627: creat d12/d5d/fcd x:0 0 0 2024-09-10T04:05:25.509 INFO:tasks.workunit.client.1.smithi070.stdout:7/720: dwrite d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:25.511 INFO:tasks.workunit.client.0.smithi046.stdout:8/607: dread d1/d29/d17/d1f/d46/f8b [0,4194304] 0 2024-09-10T04:05:25.511 INFO:tasks.workunit.client.1.smithi070.stdout:0/825: dread d8/d13/d1a/d8f/d95/d106/fd6 [0,4194304] 0 2024-09-10T04:05:25.511 INFO:tasks.workunit.client.1.smithi070.stdout:1/623: read d3/d8/d12/f4a [368216,66993] 0 2024-09-10T04:05:25.512 INFO:tasks.workunit.client.0.smithi046.stdout:8/608: chown d1/d29/d17/d1f/d46/d6a/d82/fc4 878842 1 2024-09-10T04:05:25.512 INFO:tasks.workunit.client.1.smithi070.stdout:1/624: write d3/d8/d13/d67/f8a [2717846,33273] 0 2024-09-10T04:05:25.513 INFO:tasks.workunit.client.0.smithi046.stdout:5/771: getdents d7/d11/d14/d2f/deb/da5 0 2024-09-10T04:05:25.516 INFO:tasks.workunit.client.1.smithi070.stdout:2/689: creat d15/d18/d56/d7c/ff5 x:0 0 0 2024-09-10T04:05:25.518 INFO:tasks.workunit.client.0.smithi046.stdout:3/628: fsync d12/d5d/f6e 0 2024-09-10T04:05:25.523 INFO:tasks.workunit.client.0.smithi046.stdout:4/783: write d3/fa [606289,58108] 0 2024-09-10T04:05:25.525 INFO:tasks.workunit.client.0.smithi046.stdout:5/772: dread d7/d11/d14/f25 [0,4194304] 0 2024-09-10T04:05:25.527 INFO:tasks.workunit.client.0.smithi046.stdout:4/784: chown d3/d12/d17/d105 4683040 1 2024-09-10T04:05:25.527 INFO:tasks.workunit.client.0.smithi046.stdout:8/609: dwrite d1/d29/d17/f9c [0,4194304] 0 2024-09-10T04:05:25.528 INFO:tasks.workunit.client.0.smithi046.stdout:9/586: dwrite d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:25.530 INFO:tasks.workunit.client.1.smithi070.stdout:7/721: mkdir d2/dc8/df9 0 2024-09-10T04:05:25.535 INFO:tasks.workunit.client.1.smithi070.stdout:0/826: creat d8/d24/d2d/d5f/f110 x:0 0 0 2024-09-10T04:05:25.536 INFO:tasks.workunit.client.1.smithi070.stdout:0/827: chown d8/d13/de2/d4f/f7f 46 1 2024-09-10T04:05:25.539 INFO:tasks.workunit.client.0.smithi046.stdout:6/581: dwrite da/db/d11/d2b/d2f/d3b/f5e [0,4194304] 0 2024-09-10T04:05:25.543 INFO:tasks.workunit.client.0.smithi046.stdout:5/773: creat d7/d11/d14/d2f/deb/da5/dcc/f112 x:0 0 0 2024-09-10T04:05:25.543 INFO:tasks.workunit.client.0.smithi046.stdout:4/785: dread d3/d12/d17/d22/d30/f3a [0,4194304] 0 2024-09-10T04:05:25.544 INFO:tasks.workunit.client.0.smithi046.stdout:8/610: creat d1/d29/d17/d1f/d46/fca x:0 0 0 2024-09-10T04:05:25.546 INFO:tasks.workunit.client.1.smithi070.stdout:7/722: rmdir d2/d3/dc/d13 39 2024-09-10T04:05:25.548 INFO:tasks.workunit.client.0.smithi046.stdout:9/587: mknod d4/d18/cb5 0 2024-09-10T04:05:25.550 INFO:tasks.workunit.client.1.smithi070.stdout:1/625: mkdir d3/d8/d12/d2f/dcc 0 2024-09-10T04:05:25.552 INFO:tasks.workunit.client.0.smithi046.stdout:6/582: rename da/dc/d79 to da/db/d11/d2b/d5a/dc2 0 2024-09-10T04:05:25.554 INFO:tasks.workunit.client.1.smithi070.stdout:0/828: rename d8/d13/d47/c109 to d8/d24/d2d/d5f/df6/c111 0 2024-09-10T04:05:25.555 INFO:tasks.workunit.client.1.smithi070.stdout:0/829: chown d8/f1b 7527299 1 2024-09-10T04:05:25.556 INFO:tasks.workunit.client.0.smithi046.stdout:9/588: dwrite d4/f50 [0,4194304] 0 2024-09-10T04:05:25.556 INFO:tasks.workunit.client.1.smithi070.stdout:2/690: mkdir d15/df6 0 2024-09-10T04:05:25.558 INFO:tasks.workunit.client.0.smithi046.stdout:6/583: dread da/f7b [0,4194304] 0 2024-09-10T04:05:25.558 INFO:tasks.workunit.client.0.smithi046.stdout:9/589: truncate d4/d18/d89/fa7 506838 0 2024-09-10T04:05:25.558 INFO:tasks.workunit.client.1.smithi070.stdout:2/691: chown d15/d18/d29/d2a/d2e/da6/dc0/dcb/fb9 35360 1 2024-09-10T04:05:25.558 INFO:tasks.workunit.client.0.smithi046.stdout:5/774: creat d7/d11/d3f/d53/d81/f113 x:0 0 0 2024-09-10T04:05:25.561 INFO:tasks.workunit.client.1.smithi070.stdout:0/830: dwrite d8/d24/d36/f46 [4194304,4194304] 0 2024-09-10T04:05:25.561 INFO:tasks.workunit.client.0.smithi046.stdout:5/775: write d7/d11/d3f/d53/fc7 [2889859,103893] 0 2024-09-10T04:05:25.562 INFO:tasks.workunit.client.1.smithi070.stdout:0/831: chown d8/d85/dca 18905520 1 2024-09-10T04:05:25.562 INFO:tasks.workunit.client.0.smithi046.stdout:3/629: getdents d12/d13/d37/d51/dbd 0 2024-09-10T04:05:25.564 INFO:tasks.workunit.client.1.smithi070.stdout:1/626: creat d3/d8/d13/d67/fcd x:0 0 0 2024-09-10T04:05:25.566 INFO:tasks.workunit.client.1.smithi070.stdout:1/627: fdatasync d3/d8/d13/d3f/f52 0 2024-09-10T04:05:25.569 INFO:tasks.workunit.client.1.smithi070.stdout:9/750: sync 2024-09-10T04:05:25.570 INFO:tasks.workunit.client.0.smithi046.stdout:4/786: mknod d3/d12/c110 0 2024-09-10T04:05:25.571 INFO:tasks.workunit.client.1.smithi070.stdout:5/709: dwrite d17/d6f/fc8 [0,4194304] 0 2024-09-10T04:05:25.575 INFO:tasks.workunit.client.0.smithi046.stdout:9/590: mknod d4/d3c/d4d/d5c/d59/cb6 0 2024-09-10T04:05:25.576 INFO:tasks.workunit.client.1.smithi070.stdout:3/737: dwrite db/d1c/d25/ffa [0,4194304] 0 2024-09-10T04:05:25.580 INFO:tasks.workunit.client.1.smithi070.stdout:9/751: creat d9/d1c/dac/ff2 x:0 0 0 2024-09-10T04:05:25.581 INFO:tasks.workunit.client.1.smithi070.stdout:9/752: chown d9/d7a/l53 450069 1 2024-09-10T04:05:25.583 INFO:tasks.workunit.client.0.smithi046.stdout:4/787: creat d3/d12/d14/f111 x:0 0 0 2024-09-10T04:05:25.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:25 smithi070 ceph-mon[35495]: pgmap v266: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 256 MiB/s wr, 458 op/s 2024-09-10T04:05:25.585 INFO:tasks.workunit.client.0.smithi046.stdout:2/618: sync 2024-09-10T04:05:25.585 INFO:tasks.workunit.client.0.smithi046.stdout:0/505: sync 2024-09-10T04:05:25.585 INFO:tasks.workunit.client.0.smithi046.stdout:1/492: sync 2024-09-10T04:05:25.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:25 smithi046 ceph-mon[28328]: pgmap v266: 65 pgs: 65 active+clean; 2.4 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 256 MiB/s wr, 458 op/s 2024-09-10T04:05:25.586 INFO:tasks.workunit.client.1.smithi070.stdout:5/710: fdatasync d17/d19/d6a/d54/d57/d63/f9a 0 2024-09-10T04:05:25.588 INFO:tasks.workunit.client.1.smithi070.stdout:4/671: sync 2024-09-10T04:05:25.588 INFO:tasks.workunit.client.1.smithi070.stdout:6/717: sync 2024-09-10T04:05:25.588 INFO:tasks.workunit.client.1.smithi070.stdout:8/765: sync 2024-09-10T04:05:25.589 INFO:tasks.workunit.client.0.smithi046.stdout:4/788: dread d3/d12/d17/d29/d58/f6d [4194304,4194304] 0 2024-09-10T04:05:25.590 INFO:tasks.workunit.client.1.smithi070.stdout:8/766: chown d4/d16/d73/d96/l85 859393390 1 2024-09-10T04:05:25.591 INFO:tasks.workunit.client.0.smithi046.stdout:4/789: read d3/d12/d17/d29/ddd/ff2 [1320025,62799] 0 2024-09-10T04:05:25.592 INFO:tasks.workunit.client.1.smithi070.stdout:3/738: rmdir db/d79/dd4 39 2024-09-10T04:05:25.593 INFO:tasks.workunit.client.1.smithi070.stdout:3/739: truncate db/d1c/ff6 154977 0 2024-09-10T04:05:25.593 INFO:tasks.workunit.client.0.smithi046.stdout:3/630: mknod d12/d13/d37/d51/db5/d50/d7a/dc1/cce 0 2024-09-10T04:05:25.595 INFO:tasks.workunit.client.0.smithi046.stdout:8/611: write d1/d29/d17/d1f/d2a/f58 [3933506,50469] 0 2024-09-10T04:05:25.598 INFO:tasks.workunit.client.1.smithi070.stdout:2/692: rename d15/d59/l84 to d15/d2b/d6e/lf7 0 2024-09-10T04:05:25.598 INFO:tasks.workunit.client.1.smithi070.stdout:7/723: dwrite d2/f55 [0,4194304] 0 2024-09-10T04:05:25.599 INFO:tasks.workunit.client.0.smithi046.stdout:1/493: rename d3/dc/f4b to d3/d27/d5d/d60/f9f 0 2024-09-10T04:05:25.600 INFO:tasks.workunit.client.1.smithi070.stdout:9/753: truncate d9/d1c/dac/d8a/dc4/de1/f67 1107310 0 2024-09-10T04:05:25.601 INFO:tasks.workunit.client.1.smithi070.stdout:9/754: readlink d9/d1c/dac/l79 0 2024-09-10T04:05:25.602 INFO:tasks.workunit.client.1.smithi070.stdout:9/755: write d9/d1c/dac/d8a/f3d [1134024,8051] 0 2024-09-10T04:05:25.603 INFO:tasks.workunit.client.0.smithi046.stdout:6/584: write da/db/d11/d2b/f9a [994522,59574] 0 2024-09-10T04:05:25.605 INFO:tasks.workunit.client.0.smithi046.stdout:5/776: write d7/d11/d14/f1d [4471040,116384] 0 2024-09-10T04:05:25.605 INFO:tasks.workunit.client.0.smithi046.stdout:4/790: dread - d3/d12/d17/fc5 zero size 2024-09-10T04:05:25.607 INFO:tasks.workunit.client.0.smithi046.stdout:5/777: readlink d7/d11/d14/d2f/d2b/l44 0 2024-09-10T04:05:25.608 INFO:tasks.workunit.client.1.smithi070.stdout:4/672: symlink de/d13/d18/d26/le3 0 2024-09-10T04:05:25.609 INFO:tasks.workunit.client.0.smithi046.stdout:3/631: read - d12/d13/d37/d51/db5/d50/d7a/dc1/f97 zero size 2024-09-10T04:05:25.614 INFO:tasks.workunit.client.0.smithi046.stdout:0/506: rename d1/dc/d25/d33/d1a/d64 to d1/d13/d37/d75/d94/d9a 0 2024-09-10T04:05:25.614 INFO:tasks.workunit.client.0.smithi046.stdout:9/591: dwrite d4/d3c/f6b [0,4194304] 0 2024-09-10T04:05:25.615 INFO:tasks.workunit.client.0.smithi046.stdout:1/494: dread - d3/dc/d34/d51/f83 zero size 2024-09-10T04:05:25.619 INFO:tasks.workunit.client.0.smithi046.stdout:6/585: creat da/db/d11/d2b/d2f/d3b/fc3 x:0 0 0 2024-09-10T04:05:25.619 INFO:tasks.workunit.client.0.smithi046.stdout:4/791: symlink d3/d12/d14/l112 0 2024-09-10T04:05:25.621 INFO:tasks.workunit.client.0.smithi046.stdout:5/778: creat d7/d11/d14/d2f/f114 x:0 0 0 2024-09-10T04:05:25.623 INFO:tasks.workunit.client.1.smithi070.stdout:9/756: unlink d9/d1c/dac/d8a/d1b/c28 0 2024-09-10T04:05:25.624 INFO:tasks.workunit.client.0.smithi046.stdout:3/632: mkdir d12/d5d/d71/d49/dcf 0 2024-09-10T04:05:25.626 INFO:tasks.workunit.client.0.smithi046.stdout:8/612: mknod d1/d29/d17/d1f/d46/d6a/d82/dc2/ccb 0 2024-09-10T04:05:25.628 INFO:tasks.workunit.client.1.smithi070.stdout:6/718: mknod d0/d3/d18/d1b/d17/d3f/ce0 0 2024-09-10T04:05:25.628 INFO:tasks.workunit.client.1.smithi070.stdout:6/719: chown d0/d3/d18/d1b/d33/c4e 2672394 1 2024-09-10T04:05:25.629 INFO:tasks.workunit.client.0.smithi046.stdout:0/507: mknod d1/dc/d25/d33/d1a/d41/c9b 0 2024-09-10T04:05:25.631 INFO:tasks.workunit.client.1.smithi070.stdout:0/832: getdents d8/d13/de2/d71 0 2024-09-10T04:05:25.632 INFO:tasks.workunit.client.0.smithi046.stdout:2/619: link d2/d6/d23/d5d/d77/d7b/d85/d8d/cdc d2/d6/d9d/ce4 0 2024-09-10T04:05:25.632 INFO:tasks.workunit.client.1.smithi070.stdout:8/767: mknod d4/d39/dfc/d43/d67/c102 0 2024-09-10T04:05:25.635 INFO:tasks.workunit.client.0.smithi046.stdout:4/792: read d3/dab/fe5 [670900,32475] 0 2024-09-10T04:05:25.636 INFO:tasks.workunit.client.1.smithi070.stdout:1/628: getdents d3/d8/d13/d67/d83 0 2024-09-10T04:05:25.636 INFO:tasks.workunit.client.0.smithi046.stdout:6/586: truncate da/db/d63/fa2 952716 0 2024-09-10T04:05:25.638 INFO:tasks.workunit.client.0.smithi046.stdout:0/508: dwrite d1/dc/d25/d33/d3/d29/f3c [0,4194304] 0 2024-09-10T04:05:25.638 INFO:tasks.workunit.client.1.smithi070.stdout:2/693: mkdir d15/d18/db2/ded/df8 0 2024-09-10T04:05:25.643 INFO:tasks.workunit.client.1.smithi070.stdout:7/724: unlink d2/d7/d9/d18/d49/l8b 0 2024-09-10T04:05:25.651 INFO:tasks.workunit.client.1.smithi070.stdout:3/740: truncate db/d1c/d25/dd8/fe8 88966 0 2024-09-10T04:05:25.651 INFO:tasks.workunit.client.0.smithi046.stdout:9/592: fdatasync d4/d3c/d4d/d5c/d59/f5d 0 2024-09-10T04:05:25.653 INFO:tasks.workunit.client.1.smithi070.stdout:0/833: mknod d8/d13/de2/d53/dc6/c112 0 2024-09-10T04:05:25.654 INFO:tasks.workunit.client.0.smithi046.stdout:5/779: write d7/d11/d3f/fd4 [516786,91009] 0 2024-09-10T04:05:25.654 INFO:tasks.workunit.client.0.smithi046.stdout:4/793: dread - d3/d12/d17/d29/d58/db2/fc4 zero size 2024-09-10T04:05:25.657 INFO:tasks.workunit.client.0.smithi046.stdout:6/587: dread - da/db/d63/fa1 zero size 2024-09-10T04:05:25.663 INFO:tasks.workunit.client.1.smithi070.stdout:9/757: mkdir d9/d1c/dac/d8a/dc4/dda/de0/d90/df0/df3 0 2024-09-10T04:05:25.663 INFO:tasks.workunit.client.0.smithi046.stdout:5/780: dwrite d7/d11/d14/d2f/ff1 [0,4194304] 0 2024-09-10T04:05:25.664 INFO:tasks.workunit.client.1.smithi070.stdout:5/711: link d17/d6f/l3e d17/d24/dcc/lf6 0 2024-09-10T04:05:25.666 INFO:tasks.workunit.client.1.smithi070.stdout:3/741: creat db/d1c/d3e/ffd x:0 0 0 2024-09-10T04:05:25.667 INFO:tasks.workunit.client.1.smithi070.stdout:6/720: rename d0/ce to d0/d3/d18/d1b/d17/d3f/d40/ce1 0 2024-09-10T04:05:25.668 INFO:tasks.workunit.client.0.smithi046.stdout:9/593: readlink d4/d7/d1f/l53 0 2024-09-10T04:05:25.668 INFO:tasks.workunit.client.0.smithi046.stdout:9/594: chown d4/d18/c6c 1 1 2024-09-10T04:05:25.669 INFO:tasks.workunit.client.1.smithi070.stdout:0/834: fdatasync d8/d24/d2d/d8d/f79 0 2024-09-10T04:05:25.669 INFO:tasks.workunit.client.0.smithi046.stdout:9/595: chown d4/d18/d89/l90 747281 1 2024-09-10T04:05:25.670 INFO:tasks.workunit.client.1.smithi070.stdout:8/768: mknod d4/d16/d17/d6f/de5/c103 0 2024-09-10T04:05:25.670 INFO:tasks.workunit.client.1.smithi070.stdout:0/835: dread - d8/d13/de2/fd0 zero size 2024-09-10T04:05:25.672 INFO:tasks.workunit.client.0.smithi046.stdout:3/633: creat d12/d13/fd0 x:0 0 0 2024-09-10T04:05:25.674 INFO:tasks.workunit.client.1.smithi070.stdout:2/694: link d15/d18/d29/f3d d15/df6/ff9 0 2024-09-10T04:05:25.675 INFO:tasks.workunit.client.0.smithi046.stdout:5/781: fsync d7/d11/d3f/d53/d75/f9f 0 2024-09-10T04:05:25.676 INFO:tasks.workunit.client.0.smithi046.stdout:8/613: link d1/d29/d19/d36/d4c/lac d1/d29/d17/lcc 0 2024-09-10T04:05:25.677 INFO:tasks.workunit.client.0.smithi046.stdout:5/782: write d7/d11/d14/d2f/deb/f111 [59969,100878] 0 2024-09-10T04:05:25.678 INFO:tasks.workunit.client.0.smithi046.stdout:1/495: getdents d3/d27/d5d/d60 0 2024-09-10T04:05:25.683 INFO:tasks.workunit.client.0.smithi046.stdout:4/794: creat d3/d12/d17/d105/f113 x:0 0 0 2024-09-10T04:05:25.685 INFO:tasks.workunit.client.1.smithi070.stdout:4/673: link de/d13/d18/d23/fc6 de/fe4 0 2024-09-10T04:05:25.685 INFO:tasks.workunit.client.0.smithi046.stdout:0/509: dread d1/dc/f21 [0,4194304] 0 2024-09-10T04:05:25.686 INFO:tasks.workunit.client.0.smithi046.stdout:8/614: dwrite d1/d29/d17/d1f/d46/d76/fab [0,4194304] 0 2024-09-10T04:05:25.687 INFO:tasks.workunit.client.1.smithi070.stdout:5/712: dwrite d17/ff4 [0,4194304] 0 2024-09-10T04:05:25.687 INFO:tasks.workunit.client.0.smithi046.stdout:9/596: mkdir d4/d7/d81/db7 0 2024-09-10T04:05:25.687 INFO:tasks.workunit.client.1.smithi070.stdout:4/674: truncate de/d13/d18/d26/f2b 1607431 0 2024-09-10T04:05:25.687 INFO:tasks.workunit.client.0.smithi046.stdout:0/510: write d1/dc/d49/f4a [8203808,124666] 0 2024-09-10T04:05:25.689 INFO:tasks.workunit.client.1.smithi070.stdout:3/742: mkdir db/d79/db6/dc3/de1/dfe 0 2024-09-10T04:05:25.690 INFO:tasks.workunit.client.0.smithi046.stdout:3/634: fsync d12/f5f 0 2024-09-10T04:05:25.694 INFO:tasks.workunit.client.1.smithi070.stdout:6/721: symlink d0/d3/d18/d1b/d17/d28/le2 0 2024-09-10T04:05:25.694 INFO:tasks.workunit.client.1.smithi070.stdout:6/722: read - d0/d7f/fc9 zero size 2024-09-10T04:05:25.696 INFO:tasks.workunit.client.1.smithi070.stdout:6/723: truncate d0/d3/d18/f4c 1746882 0 2024-09-10T04:05:25.698 INFO:tasks.workunit.client.0.smithi046.stdout:1/496: creat d3/d9b/fa0 x:0 0 0 2024-09-10T04:05:25.698 INFO:tasks.workunit.client.0.smithi046.stdout:1/497: readlink d3/d27/l49 0 2024-09-10T04:05:25.701 INFO:tasks.workunit.client.1.smithi070.stdout:8/769: symlink d4/d16/d17/d1a/d60/d7a/d84/l104 0 2024-09-10T04:05:25.703 INFO:tasks.workunit.client.1.smithi070.stdout:9/758: dwrite d9/f22 [0,4194304] 0 2024-09-10T04:05:25.704 INFO:tasks.workunit.client.1.smithi070.stdout:9/759: write d9/f9a [924315,17718] 0 2024-09-10T04:05:25.704 INFO:tasks.workunit.client.0.smithi046.stdout:4/795: mkdir d3/d12/d17/d29/d58/db2/d5f/d85/d114 0 2024-09-10T04:05:25.706 INFO:tasks.workunit.client.1.smithi070.stdout:7/725: creat d2/d7/db/d3a/ffa x:0 0 0 2024-09-10T04:05:25.708 INFO:tasks.workunit.client.0.smithi046.stdout:8/615: creat d1/d29/d17/fcd x:0 0 0 2024-09-10T04:05:25.710 INFO:tasks.workunit.client.0.smithi046.stdout:0/511: dread d1/dc/d25/d33/f3f [0,4194304] 0 2024-09-10T04:05:25.711 INFO:tasks.workunit.client.1.smithi070.stdout:5/713: read d17/d19/f2e [2045359,87132] 0 2024-09-10T04:05:25.712 INFO:tasks.workunit.client.0.smithi046.stdout:3/635: creat d12/d5d/d71/d49/d5c/fd1 x:0 0 0 2024-09-10T04:05:25.716 INFO:tasks.workunit.client.0.smithi046.stdout:1/498: unlink d3/d62/f73 0 2024-09-10T04:05:25.718 INFO:tasks.workunit.client.1.smithi070.stdout:0/836: symlink d8/d13/l113 0 2024-09-10T04:05:25.720 INFO:tasks.workunit.client.0.smithi046.stdout:4/796: creat d3/d12/d17/d29/d58/db2/d5f/d85/f115 x:0 0 0 2024-09-10T04:05:25.720 INFO:tasks.workunit.client.1.smithi070.stdout:6/724: read - d0/d3/d18/d62/d70/f7c zero size 2024-09-10T04:05:25.720 INFO:tasks.workunit.client.0.smithi046.stdout:9/597: mkdir d4/d3c/d4d/d5c/db8 0 2024-09-10T04:05:25.721 INFO:tasks.workunit.client.0.smithi046.stdout:4/797: stat d3/d12/d17/d22/d30/c9b 0 2024-09-10T04:05:25.721 INFO:tasks.workunit.client.1.smithi070.stdout:1/629: getdents d3/d8/d13/d3f 0 2024-09-10T04:05:25.722 INFO:tasks.workunit.client.0.smithi046.stdout:4/798: write d3/d12/f73 [1149493,72660] 0 2024-09-10T04:05:25.723 INFO:tasks.workunit.client.0.smithi046.stdout:4/799: chown d3/d12/d17/d105 0 1 2024-09-10T04:05:25.724 INFO:tasks.workunit.client.1.smithi070.stdout:2/695: mkdir d15/d18/d56/d7c/dfa 0 2024-09-10T04:05:25.726 INFO:tasks.workunit.client.0.smithi046.stdout:8/616: dread d1/d29/d19/fb1 [0,4194304] 0 2024-09-10T04:05:25.726 INFO:tasks.workunit.client.1.smithi070.stdout:2/696: write d15/d18/d56/d7c/fe7 [769381,73228] 0 2024-09-10T04:05:25.726 INFO:tasks.workunit.client.1.smithi070.stdout:8/770: symlink d4/d16/d17/d1a/df7/l105 0 2024-09-10T04:05:25.727 INFO:tasks.workunit.client.1.smithi070.stdout:2/697: dread d15/d18/d99/fe1 [0,4194304] 0 2024-09-10T04:05:25.728 INFO:tasks.workunit.client.0.smithi046.stdout:3/636: mkdir d12/d13/d14/d1e/dd2 0 2024-09-10T04:05:25.728 INFO:tasks.workunit.client.1.smithi070.stdout:8/771: truncate d4/d16/d17/dab/fce 1555119 0 2024-09-10T04:05:25.730 INFO:tasks.workunit.client.0.smithi046.stdout:5/783: truncate d7/ffd 3050813 0 2024-09-10T04:05:25.731 INFO:tasks.workunit.client.1.smithi070.stdout:7/726: symlink d2/d7/d9/d18/d3c/d52/lfb 0 2024-09-10T04:05:25.734 INFO:tasks.workunit.client.0.smithi046.stdout:3/637: dwrite d12/d13/d37/d94/fb3 [0,4194304] 0 2024-09-10T04:05:25.734 INFO:tasks.workunit.client.0.smithi046.stdout:9/598: symlink d4/d66/lb9 0 2024-09-10T04:05:25.737 INFO:tasks.workunit.client.0.smithi046.stdout:0/512: symlink d1/dc/d25/d33/d3/dd/d55/d4d/l9c 0 2024-09-10T04:05:25.741 INFO:tasks.workunit.client.0.smithi046.stdout:4/800: rename d3/d12/d17/d22/d30/f72 to d3/d12/d17/d29/ddd/f116 0 2024-09-10T04:05:25.741 INFO:tasks.workunit.client.1.smithi070.stdout:0/837: symlink d8/d13/de2/d4f/l114 0 2024-09-10T04:05:25.741 INFO:tasks.workunit.client.1.smithi070.stdout:0/838: read - d8/d85/dca/fe9 zero size 2024-09-10T04:05:25.741 INFO:tasks.workunit.client.1.smithi070.stdout:0/839: dread - d8/d13/d47/d7b/fd8 zero size 2024-09-10T04:05:25.741 INFO:tasks.workunit.client.1.smithi070.stdout:6/725: rmdir d0/d3/da4 39 2024-09-10T04:05:25.742 INFO:tasks.workunit.client.0.smithi046.stdout:4/801: write d3/d12/d17/d29/d58/db2/d5f/d85/da8/fbe [382749,25268] 0 2024-09-10T04:05:25.743 INFO:tasks.workunit.client.0.smithi046.stdout:8/617: symlink d1/d29/d17/d1f/d46/d6a/d82/dc2/lce 0 2024-09-10T04:05:25.743 INFO:tasks.workunit.client.0.smithi046.stdout:4/802: chown d3/d12/c81 11890 1 2024-09-10T04:05:25.745 INFO:tasks.workunit.client.1.smithi070.stdout:1/630: mknod d3/d8/d27/cce 0 2024-09-10T04:05:25.745 INFO:tasks.workunit.client.1.smithi070.stdout:1/631: stat d3/d8/d13/d3f/d6d 0 2024-09-10T04:05:25.746 INFO:tasks.workunit.client.1.smithi070.stdout:1/632: chown d3/d8/d13/d3f/d6d 1 1 2024-09-10T04:05:25.749 INFO:tasks.workunit.client.1.smithi070.stdout:4/675: truncate de/d13/d18/d23/f61 1688376 0 2024-09-10T04:05:25.750 INFO:tasks.workunit.client.1.smithi070.stdout:9/760: mkdir d9/d1c/dac/d8a/dc4/dda/de0/d90/df0/df3/df4 0 2024-09-10T04:05:25.755 INFO:tasks.workunit.client.1.smithi070.stdout:3/743: rename db/d79/fa8 to db/d79/fff 0 2024-09-10T04:05:25.756 INFO:tasks.workunit.client.1.smithi070.stdout:5/714: dwrite d17/d37/fa6 [0,4194304] 0 2024-09-10T04:05:25.757 INFO:tasks.workunit.client.1.smithi070.stdout:5/715: fsync d17/d37/f8d 0 2024-09-10T04:05:25.758 INFO:tasks.workunit.client.0.smithi046.stdout:0/513: truncate d1/dc/d25/d33/d1a/f31 1587959 0 2024-09-10T04:05:25.760 INFO:tasks.workunit.client.0.smithi046.stdout:4/803: mknod d3/d12/dec/c117 0 2024-09-10T04:05:25.761 INFO:tasks.workunit.client.0.smithi046.stdout:8/618: mknod d1/d29/d19/d36/d4c/d5e/ccf 0 2024-09-10T04:05:25.765 INFO:tasks.workunit.client.0.smithi046.stdout:5/784: mkdir d7/d11/d3f/d53/d99/d10d/d115 0 2024-09-10T04:05:25.766 INFO:tasks.workunit.client.1.smithi070.stdout:2/698: mknod d15/d18/d29/d2a/d2e/d32/cfb 0 2024-09-10T04:05:25.766 INFO:tasks.workunit.client.0.smithi046.stdout:9/599: write d4/d44/f85 [103786,8519] 0 2024-09-10T04:05:25.771 INFO:tasks.workunit.client.1.smithi070.stdout:8/772: mkdir d4/d39/dfc/d43/d67/dec/d106 0 2024-09-10T04:05:25.773 INFO:tasks.workunit.client.0.smithi046.stdout:0/514: creat d1/dc/d25/d33/d1a/d41/f9d x:0 0 0 2024-09-10T04:05:25.776 INFO:tasks.workunit.client.1.smithi070.stdout:3/744: rmdir db/d79/d94 39 2024-09-10T04:05:25.779 INFO:tasks.workunit.client.1.smithi070.stdout:5/716: rmdir d17/d19/d6a/d54/d57/d63 39 2024-09-10T04:05:25.781 INFO:tasks.workunit.client.0.smithi046.stdout:7/704: sync 2024-09-10T04:05:25.781 INFO:tasks.workunit.client.1.smithi070.stdout:3/745: dwrite db/d1c/d25/d28/fba [0,4194304] 0 2024-09-10T04:05:25.782 INFO:tasks.workunit.client.0.smithi046.stdout:3/638: dread d12/d13/d14/d1e/d2d/f88 [0,4194304] 0 2024-09-10T04:05:25.782 INFO:tasks.workunit.client.0.smithi046.stdout:9/600: mkdir d4/d18/d89/dba 0 2024-09-10T04:05:25.783 INFO:tasks.workunit.client.1.smithi070.stdout:3/746: write db/ff8 [732598,85134] 0 2024-09-10T04:05:25.783 INFO:tasks.workunit.client.1.smithi070.stdout:6/726: mkdir d0/d3/d18/de3 0 2024-09-10T04:05:25.784 INFO:tasks.workunit.client.0.smithi046.stdout:3/639: read d12/d13/d37/d94/fb3 [420680,88472] 0 2024-09-10T04:05:25.784 INFO:tasks.workunit.client.0.smithi046.stdout:8/619: getdents d1/d29/d19/db7 0 2024-09-10T04:05:25.785 INFO:tasks.workunit.client.1.smithi070.stdout:2/699: mknod d15/d59/cfc 0 2024-09-10T04:05:25.785 INFO:tasks.workunit.client.1.smithi070.stdout:9/761: dwrite d9/d1c/dac/d8a/d3c/d62/f97 [0,4194304] 0 2024-09-10T04:05:25.788 INFO:tasks.workunit.client.0.smithi046.stdout:7/705: fdatasync d5/d6/db/dd/d24/f31 0 2024-09-10T04:05:25.794 INFO:tasks.workunit.client.1.smithi070.stdout:2/700: dwrite d15/d18/f63 [4194304,4194304] 0 2024-09-10T04:05:25.797 INFO:tasks.workunit.client.0.smithi046.stdout:8/620: symlink d1/d29/d19/d36/d6e/d75/ld0 0 2024-09-10T04:05:25.797 INFO:tasks.workunit.client.1.smithi070.stdout:0/840: rename d8/d24/d2d/d5f/ld4 to d8/d13/de2/l115 0 2024-09-10T04:05:25.799 INFO:tasks.workunit.client.0.smithi046.stdout:3/640: rename d12/d13/d14/d1e/d2d/d39/d67/daf to d12/d13/d37/dab/dac/dd3 0 2024-09-10T04:05:25.802 INFO:tasks.workunit.client.1.smithi070.stdout:8/773: mkdir d4/d16/d17/d1a/d60/d7a/db8/d107 0 2024-09-10T04:05:25.804 INFO:tasks.workunit.client.1.smithi070.stdout:0/841: read d8/d13/de2/d4f/ff9 [1842240,91671] 0 2024-09-10T04:05:25.804 INFO:tasks.workunit.client.1.smithi070.stdout:7/727: getdents d2/d3/dc/d79 0 2024-09-10T04:05:25.806 INFO:tasks.workunit.client.1.smithi070.stdout:2/701: dwrite d15/d59/feb [0,4194304] 0 2024-09-10T04:05:25.807 INFO:tasks.workunit.client.1.smithi070.stdout:0/842: dread - d8/d13/de2/d53/f102 zero size 2024-09-10T04:05:25.808 INFO:tasks.workunit.client.1.smithi070.stdout:0/843: fsync d8/f1b 0 2024-09-10T04:05:25.808 INFO:tasks.workunit.client.1.smithi070.stdout:7/728: write d2/d7/d9/d18/d3c/fae [4998861,126332] 0 2024-09-10T04:05:25.808 INFO:tasks.workunit.client.1.smithi070.stdout:2/702: write d15/d18/d29/d2a/fc4 [4573296,38330] 0 2024-09-10T04:05:25.809 INFO:tasks.workunit.client.0.smithi046.stdout:8/621: unlink d1/d29/d17/c6f 0 2024-09-10T04:05:25.813 INFO:tasks.workunit.client.0.smithi046.stdout:8/622: dread d1/d29/d12/f45 [0,4194304] 0 2024-09-10T04:05:25.815 INFO:tasks.workunit.client.0.smithi046.stdout:7/706: rename d5/d6/dbe/dd7/l2b to d5/d6/db/dd/d24/d76/d9c/lf3 0 2024-09-10T04:05:25.817 INFO:tasks.workunit.client.0.smithi046.stdout:3/641: readlink d12/d13/d37/d51/db5/d50/l6a 0 2024-09-10T04:05:25.817 INFO:tasks.workunit.client.1.smithi070.stdout:5/717: write f16 [4431507,25972] 0 2024-09-10T04:05:25.817 INFO:tasks.workunit.client.0.smithi046.stdout:7/707: write d5/d9/d12/d72/d77/de0/fc6 [924750,124066] 0 2024-09-10T04:05:25.818 INFO:tasks.workunit.client.0.smithi046.stdout:3/642: chown d12/d13/d14/c84 46 1 2024-09-10T04:05:25.821 INFO:tasks.workunit.client.1.smithi070.stdout:9/762: mkdir d9/d1c/dac/d8a/dc4/dda/de0/dd5/df5 0 2024-09-10T04:05:25.821 INFO:tasks.workunit.client.0.smithi046.stdout:5/785: getdents d7/d11/d14/d2f/deb/da5/dcc/dc0/df3 0 2024-09-10T04:05:25.821 INFO:tasks.workunit.client.1.smithi070.stdout:9/763: truncate d9/d7a/d74/d8c/fd4 196131 0 2024-09-10T04:05:25.823 INFO:tasks.workunit.client.1.smithi070.stdout:6/727: write d0/fa7 [942102,65009] 0 2024-09-10T04:05:25.825 INFO:tasks.workunit.client.0.smithi046.stdout:7/708: creat d5/d6/d55/ff4 x:0 0 0 2024-09-10T04:05:25.826 INFO:tasks.workunit.client.1.smithi070.stdout:6/728: dread d0/d3/d18/d1b/d17/f47 [0,4194304] 0 2024-09-10T04:05:25.826 INFO:tasks.workunit.client.0.smithi046.stdout:9/601: write d4/d7/d1f/d54/f6e [221576,90110] 0 2024-09-10T04:05:25.827 INFO:tasks.workunit.client.0.smithi046.stdout:7/709: write d5/d6/d32/f74 [3197676,46065] 0 2024-09-10T04:05:25.830 INFO:tasks.workunit.client.0.smithi046.stdout:5/786: creat d7/d11/d14/d8e/d96/f116 x:0 0 0 2024-09-10T04:05:25.831 INFO:tasks.workunit.client.1.smithi070.stdout:2/703: mkdir d15/d18/db2/dc3/dfd 0 2024-09-10T04:05:25.831 INFO:tasks.workunit.client.0.smithi046.stdout:5/787: readlink d7/d11/d14/d2f/d64/lc1 0 2024-09-10T04:05:25.832 INFO:tasks.workunit.client.1.smithi070.stdout:0/844: truncate d8/d13/de2/d4f/f6e 1054659 0 2024-09-10T04:05:25.832 INFO:tasks.workunit.client.1.smithi070.stdout:6/729: dwrite d0/d3/d18/d1b/d17/d28/f5e [0,4194304] 0 2024-09-10T04:05:25.833 INFO:tasks.workunit.client.1.smithi070.stdout:3/747: link db/d1c/d25/d28/ld9 db/d79/db6/dc3/l100 0 2024-09-10T04:05:25.834 INFO:tasks.workunit.client.0.smithi046.stdout:9/602: mknod d4/d29/d32/cbb 0 2024-09-10T04:05:25.835 INFO:tasks.workunit.client.0.smithi046.stdout:7/710: truncate d5/d6/dbe/dd7/f92 428910 0 2024-09-10T04:05:25.836 INFO:tasks.workunit.client.0.smithi046.stdout:7/711: stat d5/d6/db/dd/d4b/fc7 0 2024-09-10T04:05:25.841 INFO:tasks.workunit.client.0.smithi046.stdout:8/623: rename d1/d29/d17/fb0 to d1/d29/d17/d1f/fd1 0 2024-09-10T04:05:25.844 INFO:tasks.workunit.client.1.smithi070.stdout:5/718: dread d17/d24/f2a [0,4194304] 0 2024-09-10T04:05:25.845 INFO:tasks.workunit.client.1.smithi070.stdout:7/729: symlink d2/d3/dc/d13/dbd/lfc 0 2024-09-10T04:05:25.849 INFO:tasks.workunit.client.1.smithi070.stdout:1/633: rename d3/d8/d13/d3f/f42 to d3/d8/d12/d2f/fcf 0 2024-09-10T04:05:25.850 INFO:tasks.workunit.client.0.smithi046.stdout:7/712: mknod d5/d9/d12/d72/d77/de0/d1f/cf5 0 2024-09-10T04:05:25.852 INFO:tasks.workunit.client.0.smithi046.stdout:8/624: mkdir d1/d29/d19/d73/dd2 0 2024-09-10T04:05:25.854 INFO:tasks.workunit.client.0.smithi046.stdout:3/643: truncate d12/d13/d37/f81 671081 0 2024-09-10T04:05:25.854 INFO:tasks.workunit.client.0.smithi046.stdout:8/625: chown d1/d29/d17/d1f/d46/l5d 97165649 1 2024-09-10T04:05:25.855 INFO:tasks.workunit.client.0.smithi046.stdout:9/603: dread d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:25.855 INFO:tasks.workunit.client.1.smithi070.stdout:1/634: dwrite d3/d8/d12/d2f/d32/f58 [0,4194304] 0 2024-09-10T04:05:25.856 INFO:tasks.workunit.client.0.smithi046.stdout:8/626: fsync d1/d29/d19/d36/d4c/fa9 0 2024-09-10T04:05:25.857 INFO:tasks.workunit.client.0.smithi046.stdout:7/713: rename d5/d6/db/dd/d24/db9/fde to d5/d6/d55/ff6 0 2024-09-10T04:05:25.857 INFO:tasks.workunit.client.0.smithi046.stdout:8/627: chown d1/d29/d19/d36/l61 3148634 1 2024-09-10T04:05:25.861 INFO:tasks.workunit.client.0.smithi046.stdout:5/788: dread d7/d11/d14/d45/d51/fbb [0,4194304] 0 2024-09-10T04:05:25.866 INFO:tasks.workunit.client.0.smithi046.stdout:7/714: rename d5/d6/d55/f99 to d5/ff7 0 2024-09-10T04:05:25.868 INFO:tasks.workunit.client.0.smithi046.stdout:7/715: chown d5/d9/d12/d72/d77/de0/d1f/fe1 108 1 2024-09-10T04:05:25.873 INFO:tasks.workunit.client.0.smithi046.stdout:9/604: unlink d4/d3c/c74 0 2024-09-10T04:05:25.873 INFO:tasks.workunit.client.0.smithi046.stdout:9/605: readlink d4/d7/l3f 0 2024-09-10T04:05:25.874 INFO:tasks.workunit.client.0.smithi046.stdout:5/789: stat d7/d11/d3f/d53/d81/lcf 0 2024-09-10T04:05:25.876 INFO:tasks.workunit.client.1.smithi070.stdout:8/774: rename d4/d16/d17/d1a/c37 to d4/d39/dfc/d43/d67/dec/c108 0 2024-09-10T04:05:25.877 INFO:tasks.workunit.client.1.smithi070.stdout:9/764: truncate d9/d59/f18 2254048 0 2024-09-10T04:05:25.905 INFO:tasks.workunit.client.0.smithi046.stdout:8/628: mkdir d1/d29/d19/d36/d4c/dd3 0 2024-09-10T04:05:25.907 INFO:tasks.workunit.client.0.smithi046.stdout:9/606: fsync d4/d7/f5e 0 2024-09-10T04:05:25.908 INFO:tasks.workunit.client.0.smithi046.stdout:5/790: mknod d7/d11/d3f/dd3/c117 0 2024-09-10T04:05:25.909 INFO:tasks.workunit.client.1.smithi070.stdout:5/719: dwrite d17/d19/d6a/d54/d57/f88 [0,4194304] 0 2024-09-10T04:05:25.910 INFO:tasks.workunit.client.1.smithi070.stdout:5/720: readlink d17/d19/d6a/d54/d57/da3/lea 0 2024-09-10T04:05:25.911 INFO:tasks.workunit.client.1.smithi070.stdout:1/635: creat d3/d8/d12/d2f/d32/da8/fd0 x:0 0 0 2024-09-10T04:05:25.911 INFO:tasks.workunit.client.1.smithi070.stdout:3/748: mkdir db/d79/d94/d101 0 2024-09-10T04:05:25.912 INFO:tasks.workunit.client.1.smithi070.stdout:5/721: write d17/d19/d35/f6c [1394987,116211] 0 2024-09-10T04:05:25.913 INFO:tasks.workunit.client.1.smithi070.stdout:1/636: write d3/d10/d1d/fb9 [994780,5702] 0 2024-09-10T04:05:25.914 INFO:tasks.workunit.client.0.smithi046.stdout:3/644: link cd d12/d5d/d71/d49/cd4 0 2024-09-10T04:05:25.914 INFO:tasks.workunit.client.1.smithi070.stdout:1/637: stat d3/d8/d13/d67/fcd 0 2024-09-10T04:05:25.922 INFO:tasks.workunit.client.1.smithi070.stdout:6/730: mknod d0/d3/da4/dc0/ce4 0 2024-09-10T04:05:25.926 INFO:tasks.workunit.client.1.smithi070.stdout:3/749: dread db/d1c/f4b [0,4194304] 0 2024-09-10T04:05:25.929 INFO:tasks.workunit.client.0.smithi046.stdout:5/791: unlink d7/d11/d3f/d53/da9/f83 0 2024-09-10T04:05:25.930 INFO:tasks.workunit.client.0.smithi046.stdout:3/645: symlink d12/d13/d14/d1e/d2d/d39/ld5 0 2024-09-10T04:05:25.931 INFO:tasks.workunit.client.1.smithi070.stdout:3/750: dwrite db/d83/fce [0,4194304] 0 2024-09-10T04:05:25.932 INFO:tasks.workunit.client.1.smithi070.stdout:3/751: stat db/d1c/fe2 0 2024-09-10T04:05:25.932 INFO:tasks.workunit.client.0.smithi046.stdout:7/716: link d5/d9/l8a d5/d9/d43/lf8 0 2024-09-10T04:05:25.933 INFO:tasks.workunit.client.1.smithi070.stdout:7/730: truncate d2/d7/db/d3a/d7e/f93 7276540 0 2024-09-10T04:05:25.933 INFO:tasks.workunit.client.1.smithi070.stdout:3/752: read f7 [1790997,44505] 0 2024-09-10T04:05:25.935 INFO:tasks.workunit.client.1.smithi070.stdout:7/731: chown d2/d3/dc/d79/fb9 14910 1 2024-09-10T04:05:25.936 INFO:tasks.workunit.client.1.smithi070.stdout:7/732: dread - d2/d7/db/d3a/ffa zero size 2024-09-10T04:05:25.938 INFO:tasks.workunit.client.0.smithi046.stdout:9/607: getdents d4/d18/da8 0 2024-09-10T04:05:25.940 INFO:tasks.workunit.client.1.smithi070.stdout:6/731: unlink d0/d3/d18/d1b/d17/d54/f6c 0 2024-09-10T04:05:25.943 INFO:tasks.workunit.client.1.smithi070.stdout:7/733: dread d2/d7/d9/d18/d3c/fae [4194304,4194304] 0 2024-09-10T04:05:25.943 INFO:tasks.workunit.client.1.smithi070.stdout:0/845: getdents d8/d13/de2/d74 0 2024-09-10T04:05:25.944 INFO:tasks.workunit.client.1.smithi070.stdout:0/846: write d8/f49 [6490910,2767] 0 2024-09-10T04:05:25.945 INFO:tasks.workunit.client.1.smithi070.stdout:0/847: stat d8/d24/d2d/d34/f5b 0 2024-09-10T04:05:25.945 INFO:tasks.workunit.client.1.smithi070.stdout:1/638: symlink d3/d8/d12/d2f/ld1 0 2024-09-10T04:05:25.947 INFO:tasks.workunit.client.1.smithi070.stdout:1/639: fdatasync d3/d8/d27/d7a/fa7 0 2024-09-10T04:05:25.949 INFO:tasks.workunit.client.1.smithi070.stdout:9/765: fdatasync d9/d7a/d74/f47 0 2024-09-10T04:05:25.949 INFO:tasks.workunit.client.1.smithi070.stdout:5/722: dread d17/d24/d45/f7e [0,4194304] 0 2024-09-10T04:05:25.950 INFO:tasks.workunit.client.1.smithi070.stdout:6/732: chown d0/d3/d1f/d81/ccf 105190 1 2024-09-10T04:05:25.953 INFO:tasks.workunit.client.1.smithi070.stdout:3/753: dread db/f74 [0,4194304] 0 2024-09-10T04:05:25.956 INFO:tasks.workunit.client.1.smithi070.stdout:7/734: creat d2/d3/dc/d5d/d41/dd6/ffd x:0 0 0 2024-09-10T04:05:25.957 INFO:tasks.workunit.client.1.smithi070.stdout:8/775: getdents d4/d16/d17/d1a/d60/d7a 0 2024-09-10T04:05:25.962 INFO:tasks.workunit.client.1.smithi070.stdout:0/848: rename d8/d24/d2d/d34/d6b/cb2 to d8/d24/da9/c116 0 2024-09-10T04:05:25.966 INFO:tasks.workunit.client.1.smithi070.stdout:9/766: creat d9/d1c/dac/d8a/d3c/d92/ff6 x:0 0 0 2024-09-10T04:05:25.968 INFO:tasks.workunit.client.1.smithi070.stdout:0/849: dwrite d8/d13/d1a/d75/ffb [0,4194304] 0 2024-09-10T04:05:25.968 INFO:tasks.workunit.client.1.smithi070.stdout:6/733: creat d0/d3/d18/d62/db8/fe5 x:0 0 0 2024-09-10T04:05:25.974 INFO:tasks.workunit.client.1.smithi070.stdout:7/735: rename d2/d3/d63/da9/dc1 to d2/d3/dc/d13/dfe 0 2024-09-10T04:05:25.976 INFO:tasks.workunit.client.1.smithi070.stdout:0/850: dwrite d8/d13/d1a/d8f/d95/fee [0,4194304] 0 2024-09-10T04:05:25.979 INFO:tasks.workunit.client.1.smithi070.stdout:9/767: truncate d9/d59/d2a/d3f/f91 532384 0 2024-09-10T04:05:25.983 INFO:tasks.workunit.client.1.smithi070.stdout:3/754: creat db/d79/db6/dc3/de1/dfe/f102 x:0 0 0 2024-09-10T04:05:25.986 INFO:tasks.workunit.client.1.smithi070.stdout:8/776: creat d4/d16/d17/d19/d62/df0/f109 x:0 0 0 2024-09-10T04:05:25.987 INFO:tasks.workunit.client.1.smithi070.stdout:8/777: dread - d4/d39/dfc/d43/d52/fc4 zero size 2024-09-10T04:05:25.988 INFO:tasks.workunit.client.1.smithi070.stdout:7/736: dread d2/d3/dc/d5d/d42/fa0 [0,4194304] 0 2024-09-10T04:05:25.991 INFO:tasks.workunit.client.1.smithi070.stdout:9/768: mknod d9/d1c/dac/d8a/dc4/dda/de0/d90/cf7 0 2024-09-10T04:05:25.993 INFO:tasks.workunit.client.1.smithi070.stdout:3/755: mkdir db/d1c/d3e/d103 0 2024-09-10T04:05:25.997 INFO:tasks.workunit.client.1.smithi070.stdout:7/737: mknod d2/d3/dc/d5d/d41/da8/cff 0 2024-09-10T04:05:25.999 INFO:tasks.workunit.client.1.smithi070.stdout:0/851: dread d8/d13/d1a/f30 [0,4194304] 0 2024-09-10T04:05:26.000 INFO:tasks.workunit.client.1.smithi070.stdout:1/640: truncate d3/d8/d13/d3f/f52 1988701 0 2024-09-10T04:05:26.002 INFO:tasks.workunit.client.1.smithi070.stdout:6/734: creat d0/d3/d1f/d81/fe6 x:0 0 0 2024-09-10T04:05:26.004 INFO:tasks.workunit.client.1.smithi070.stdout:8/778: symlink d4/l10a 0 2024-09-10T04:05:26.006 INFO:tasks.workunit.client.1.smithi070.stdout:7/738: creat d2/d3/dc/d5d/d41/da8/f100 x:0 0 0 2024-09-10T04:05:26.007 INFO:tasks.workunit.client.1.smithi070.stdout:7/739: write d2/d40/d6c/db2/db7/fc0 [1201326,8897] 0 2024-09-10T04:05:26.012 INFO:tasks.workunit.client.1.smithi070.stdout:6/735: fsync d0/d3/d18/d62/d70/fb1 0 2024-09-10T04:05:26.014 INFO:tasks.workunit.client.1.smithi070.stdout:1/641: dread d3/d10/f73 [0,4194304] 0 2024-09-10T04:05:26.016 INFO:tasks.workunit.client.1.smithi070.stdout:7/740: creat d2/dc8/f101 x:0 0 0 2024-09-10T04:05:26.016 INFO:tasks.workunit.client.0.smithi046.stdout:5/792: rmdir d7/d11/d3f/d53/d81 39 2024-09-10T04:05:26.017 INFO:tasks.workunit.client.0.smithi046.stdout:5/793: readlink d7/d11/d14/d2f/d2b/l46 0 2024-09-10T04:05:26.019 INFO:tasks.workunit.client.0.smithi046.stdout:3/646: unlink d12/d13/d14/d1e/d2d/d39/c46 0 2024-09-10T04:05:26.020 INFO:tasks.workunit.client.0.smithi046.stdout:7/717: mknod d5/d6/d55/d89/cf9 0 2024-09-10T04:05:26.021 INFO:tasks.workunit.client.1.smithi070.stdout:6/736: chown d0/d3/d18/d1b/c1e 13451918 1 2024-09-10T04:05:26.022 INFO:tasks.workunit.client.1.smithi070.stdout:8/779: dread d4/d16/d17/d19/f5b [0,4194304] 0 2024-09-10T04:05:26.022 INFO:tasks.workunit.client.0.smithi046.stdout:8/629: creat d1/d29/d19/d36/fd4 x:0 0 0 2024-09-10T04:05:26.024 INFO:tasks.workunit.client.1.smithi070.stdout:7/741: mkdir d2/d7/d9/dd5/d102 0 2024-09-10T04:05:26.026 INFO:tasks.workunit.client.1.smithi070.stdout:7/742: truncate d2/d3/d63/da9/ff3 538069 0 2024-09-10T04:05:26.026 INFO:tasks.workunit.client.1.smithi070.stdout:0/852: link d8/d13/de2/d53/dc6/c112 d8/d13/d1a/d8f/d95/c117 0 2024-09-10T04:05:26.031 INFO:tasks.workunit.client.1.smithi070.stdout:8/780: rename d4/dcf/fb2 to d4/d39/dc6/f10b 0 2024-09-10T04:05:26.032 INFO:tasks.workunit.client.0.smithi046.stdout:7/718: dwrite d5/d6/db/f38 [4194304,4194304] 0 2024-09-10T04:05:26.036 INFO:tasks.workunit.client.1.smithi070.stdout:0/853: mknod d8/d24/d97/c118 0 2024-09-10T04:05:26.037 INFO:tasks.workunit.client.1.smithi070.stdout:7/743: mkdir d2/d7/d9/d18/d3c/d52/d103 0 2024-09-10T04:05:26.039 INFO:tasks.workunit.client.0.smithi046.stdout:7/719: rmdir d5/d9/d12/d72/d77/de0/d1f/d7a 39 2024-09-10T04:05:26.041 INFO:tasks.workunit.client.1.smithi070.stdout:8/781: creat d4/dbf/d74/da4/f10c x:0 0 0 2024-09-10T04:05:26.042 INFO:tasks.workunit.client.0.smithi046.stdout:8/630: rmdir d1/d29/d17/d1f/d46/d67/dc1/da4 0 2024-09-10T04:05:26.042 INFO:tasks.workunit.client.1.smithi070.stdout:8/782: readlink d4/dcf/l75 0 2024-09-10T04:05:26.043 INFO:tasks.workunit.client.0.smithi046.stdout:8/631: fdatasync d1/d29/d12/f97 0 2024-09-10T04:05:26.043 INFO:tasks.workunit.client.0.smithi046.stdout:7/720: dread d5/d6/d32/dab/db8/fa6 [0,4194304] 0 2024-09-10T04:05:26.044 INFO:tasks.workunit.client.1.smithi070.stdout:7/744: dwrite d2/d3/d63/f87 [0,4194304] 0 2024-09-10T04:05:26.045 INFO:tasks.workunit.client.0.smithi046.stdout:8/632: truncate d1/d29/d19/d36/fd4 1024143 0 2024-09-10T04:05:26.045 INFO:tasks.workunit.client.1.smithi070.stdout:7/745: chown d2/d3/dc/d5d 1915812 1 2024-09-10T04:05:26.046 INFO:tasks.workunit.client.0.smithi046.stdout:8/633: chown d1/d29/d17/d1f/d46/d6a/d82/d99/fb3 28713551 1 2024-09-10T04:05:26.047 INFO:tasks.workunit.client.1.smithi070.stdout:0/854: unlink d8/d13/de2/d71/l98 0 2024-09-10T04:05:26.047 INFO:tasks.workunit.client.0.smithi046.stdout:8/634: chown d1/d29/d19/d36/d6e/d75/ld0 212547 1 2024-09-10T04:05:26.048 INFO:tasks.workunit.client.0.smithi046.stdout:7/721: rmdir d5/d6 39 2024-09-10T04:05:26.049 INFO:tasks.workunit.client.1.smithi070.stdout:1/642: write d3/d8/d12/d2f/d32/f41 [3089194,24036] 0 2024-09-10T04:05:26.050 INFO:tasks.workunit.client.1.smithi070.stdout:8/783: mknod d4/d39/dfc/d43/d52/c10d 0 2024-09-10T04:05:26.054 INFO:tasks.workunit.client.1.smithi070.stdout:7/746: mkdir d2/d40/d6c/db2/db7/d104 0 2024-09-10T04:05:26.055 INFO:tasks.workunit.client.0.smithi046.stdout:8/635: truncate d1/da2/faa 3442329 0 2024-09-10T04:05:26.056 INFO:tasks.workunit.client.1.smithi070.stdout:0/855: creat d8/d24/d2d/d34/f119 x:0 0 0 2024-09-10T04:05:26.058 INFO:tasks.workunit.client.1.smithi070.stdout:6/737: getdents d0/d3/d1f 0 2024-09-10T04:05:26.058 INFO:tasks.workunit.client.0.smithi046.stdout:9/608: dwrite d4/d18/f22 [4194304,4194304] 0 2024-09-10T04:05:26.059 INFO:tasks.workunit.client.1.smithi070.stdout:1/643: unlink d3/d10/f5e 0 2024-09-10T04:05:26.060 INFO:tasks.workunit.client.1.smithi070.stdout:1/644: chown d3/d10/d1d/f88 10508435 1 2024-09-10T04:05:26.062 INFO:tasks.workunit.client.0.smithi046.stdout:2/620: sync 2024-09-10T04:05:26.063 INFO:tasks.workunit.client.0.smithi046.stdout:5/794: dwrite d7/d11/d14/d2f/d2b/d6a/fd8 [0,4194304] 0 2024-09-10T04:05:26.065 INFO:tasks.workunit.client.1.smithi070.stdout:6/738: symlink d0/d3/d18/d1b/d17/d54/dc4/dd2/le7 0 2024-09-10T04:05:26.069 INFO:tasks.workunit.client.1.smithi070.stdout:7/747: creat d2/d40/d6c/db2/db7/d104/f105 x:0 0 0 2024-09-10T04:05:26.072 INFO:tasks.workunit.client.1.smithi070.stdout:0/856: rmdir d8/d13/d1a/da0 39 2024-09-10T04:05:26.072 INFO:tasks.workunit.client.0.smithi046.stdout:5/795: mknod d7/d11/d3f/d53/da9/c118 0 2024-09-10T04:05:26.073 INFO:tasks.workunit.client.1.smithi070.stdout:0/857: readlink d8/d13/d1a/d9a/la8 0 2024-09-10T04:05:26.074 INFO:tasks.workunit.client.0.smithi046.stdout:5/796: readlink d7/d11/d14/d8e/lbc 0 2024-09-10T04:05:26.074 INFO:tasks.workunit.client.1.smithi070.stdout:6/739: mkdir d0/d3/d1f/de8 0 2024-09-10T04:05:26.075 INFO:tasks.workunit.client.0.smithi046.stdout:5/797: chown d7/d11/d3f/dd3/c109 853340 1 2024-09-10T04:05:26.075 INFO:tasks.workunit.client.0.smithi046.stdout:9/609: dread d4/d3c/d4d/d5c/d59/f7f [0,4194304] 0 2024-09-10T04:05:26.076 INFO:tasks.workunit.client.1.smithi070.stdout:8/784: creat d4/d39/dfc/f10e x:0 0 0 2024-09-10T04:05:26.077 INFO:tasks.workunit.client.1.smithi070.stdout:6/740: truncate d0/d3/d18/d1b/d17/d28/f95 770704 0 2024-09-10T04:05:26.077 INFO:tasks.workunit.client.0.smithi046.stdout:5/798: unlink d7/c10c 0 2024-09-10T04:05:26.078 INFO:tasks.workunit.client.0.smithi046.stdout:5/799: stat d7/d11/d14/d8e/de2/le6 0 2024-09-10T04:05:26.078 INFO:tasks.workunit.client.1.smithi070.stdout:1/645: mknod d3/d8/d13/d3f/d6d/cd2 0 2024-09-10T04:05:26.079 INFO:tasks.workunit.client.0.smithi046.stdout:2/621: dread d2/d6/d23/d5d/d5e/f72 [0,4194304] 0 2024-09-10T04:05:26.081 INFO:tasks.workunit.client.0.smithi046.stdout:7/722: write d5/d9/d12/d72/d77/de0/d1f/f48 [5158277,67306] 0 2024-09-10T04:05:26.082 INFO:tasks.workunit.client.0.smithi046.stdout:9/610: unlink d4/d66/d7c/f7d 0 2024-09-10T04:05:26.085 INFO:tasks.workunit.client.0.smithi046.stdout:5/800: mknod d7/d11/d14/d45/c119 0 2024-09-10T04:05:26.087 INFO:tasks.workunit.client.1.smithi070.stdout:8/785: unlink d4/d16/d17/d1a/f32 0 2024-09-10T04:05:26.089 INFO:tasks.workunit.client.0.smithi046.stdout:8/636: dwrite d1/d29/d17/d1f/f31 [0,4194304] 0 2024-09-10T04:05:26.093 INFO:tasks.workunit.client.1.smithi070.stdout:1/646: write d3/d8/d13/f14 [599702,26626] 0 2024-09-10T04:05:26.095 INFO:tasks.workunit.client.0.smithi046.stdout:5/801: fdatasync d7/d11/d14/f95 0 2024-09-10T04:05:26.102 INFO:tasks.workunit.client.1.smithi070.stdout:6/741: getdents d0/d3/d18/de3 0 2024-09-10T04:05:26.103 INFO:tasks.workunit.client.0.smithi046.stdout:2/622: write d2/d6/f43 [966669,120176] 0 2024-09-10T04:05:26.104 INFO:tasks.workunit.client.1.smithi070.stdout:7/748: dwrite d2/d3/dc/d5d/d41/f50 [0,4194304] 0 2024-09-10T04:05:26.107 INFO:tasks.workunit.client.1.smithi070.stdout:0/858: link d8/d24/dce/le0 d8/d13/de2/d7d/l11a 0 2024-09-10T04:05:26.109 INFO:tasks.workunit.client.0.smithi046.stdout:5/802: dread d7/d11/d3f/d5a/fb2 [0,4194304] 0 2024-09-10T04:05:26.109 INFO:tasks.workunit.client.1.smithi070.stdout:1/647: fdatasync d3/d8/d13/d3f/d6d/f8e 0 2024-09-10T04:05:26.112 INFO:tasks.workunit.client.1.smithi070.stdout:8/786: creat d4/d39/dfc/d43/d67/dec/d106/f10f x:0 0 0 2024-09-10T04:05:26.113 INFO:tasks.workunit.client.0.smithi046.stdout:7/723: creat d5/d9/ffa x:0 0 0 2024-09-10T04:05:26.113 INFO:tasks.workunit.client.1.smithi070.stdout:8/787: chown d4/d39 0 1 2024-09-10T04:05:26.115 INFO:tasks.workunit.client.0.smithi046.stdout:2/623: mknod d2/d6/d23/d42/db4/dca/ce5 0 2024-09-10T04:05:26.116 INFO:tasks.workunit.client.0.smithi046.stdout:9/611: dread d4/f28 [0,4194304] 0 2024-09-10T04:05:26.116 INFO:tasks.workunit.client.0.smithi046.stdout:8/637: dwrite d1/d29/d19/d36/d4c/d5e/fa1 [0,4194304] 0 2024-09-10T04:05:26.117 INFO:tasks.workunit.client.0.smithi046.stdout:2/624: chown d2/d6/d9d/db6 185289 1 2024-09-10T04:05:26.119 INFO:tasks.workunit.client.1.smithi070.stdout:0/859: mkdir d8/d13/de2/d4f/d11b 0 2024-09-10T04:05:26.119 INFO:tasks.workunit.client.0.smithi046.stdout:8/638: chown d1/d29/d17/d1f/d2a/f58 604301163 1 2024-09-10T04:05:26.121 INFO:tasks.workunit.client.1.smithi070.stdout:4/676: sync 2024-09-10T04:05:26.122 INFO:tasks.workunit.client.1.smithi070.stdout:1/648: mkdir d3/dd3 0 2024-09-10T04:05:26.124 INFO:tasks.workunit.client.1.smithi070.stdout:1/649: truncate d3/d10/d1d/dbd/da2/fa6 722062 0 2024-09-10T04:05:26.124 INFO:tasks.workunit.client.1.smithi070.stdout:8/788: unlink d4/d16/d17/d6f/cde 0 2024-09-10T04:05:26.126 INFO:tasks.workunit.client.1.smithi070.stdout:8/789: dread d4/dbf/d87/fa2 [0,4194304] 0 2024-09-10T04:05:26.126 INFO:tasks.workunit.client.0.smithi046.stdout:2/625: dwrite d2/d49/f9b [0,4194304] 0 2024-09-10T04:05:26.131 INFO:tasks.workunit.client.1.smithi070.stdout:4/677: mknod de/d13/d18/d26/ce5 0 2024-09-10T04:05:26.136 INFO:tasks.workunit.client.0.smithi046.stdout:2/626: rename d2/d57 to d2/d6/d23/d42/de6 0 2024-09-10T04:05:26.138 INFO:tasks.workunit.client.0.smithi046.stdout:5/803: getdents d7/d11/d14/d45 0 2024-09-10T04:05:26.140 INFO:tasks.workunit.client.1.smithi070.stdout:6/742: rename d0/d3/d18/d1b/d17/c3e to d0/d3/d18/ce9 0 2024-09-10T04:05:26.143 INFO:tasks.workunit.client.0.smithi046.stdout:7/724: getdents d5/d9/d12/d72/d77/de0/d1f/d7a 0 2024-09-10T04:05:26.146 INFO:tasks.workunit.client.0.smithi046.stdout:5/804: unlink d7/d11/d14/d45/f4b 0 2024-09-10T04:05:26.147 INFO:tasks.workunit.client.0.smithi046.stdout:9/612: write d4/fb [411512,94810] 0 2024-09-10T04:05:26.150 INFO:tasks.workunit.client.0.smithi046.stdout:7/725: mkdir d5/d9/d12/d72/d77/da7/dfb 0 2024-09-10T04:05:26.152 INFO:tasks.workunit.client.0.smithi046.stdout:5/805: mknod d7/d11/d3f/d53/d75/c11a 0 2024-09-10T04:05:26.153 INFO:tasks.workunit.client.0.smithi046.stdout:9/613: rmdir d4/d29 39 2024-09-10T04:05:26.155 INFO:tasks.workunit.client.0.smithi046.stdout:2/627: getdents d2/d6/d23/d42/de6 0 2024-09-10T04:05:26.157 INFO:tasks.workunit.client.0.smithi046.stdout:9/614: fsync d4/d7/d1f/f15 0 2024-09-10T04:05:26.166 INFO:tasks.workunit.client.0.smithi046.stdout:9/615: creat d4/d18/da8/fbc x:0 0 0 2024-09-10T04:05:26.170 INFO:tasks.workunit.client.0.smithi046.stdout:9/616: mknod d4/d29/d32/cbd 0 2024-09-10T04:05:26.174 INFO:tasks.workunit.client.0.smithi046.stdout:9/617: getdents d4/d7/d61 0 2024-09-10T04:05:26.175 INFO:tasks.workunit.client.0.smithi046.stdout:2/628: write d2/d8/d37/d63/f75 [129563,9610] 0 2024-09-10T04:05:26.179 INFO:tasks.workunit.client.0.smithi046.stdout:2/629: chown d2/d8/d65/d90/fba 20 1 2024-09-10T04:05:26.180 INFO:tasks.workunit.client.0.smithi046.stdout:5/806: dwrite d7/d11/d3f/d5a/d63/f88 [4194304,4194304] 0 2024-09-10T04:05:26.183 INFO:tasks.workunit.client.0.smithi046.stdout:7/726: dwrite d5/d6/db/f2a [0,4194304] 0 2024-09-10T04:05:26.186 INFO:tasks.workunit.client.0.smithi046.stdout:2/630: creat d2/d6/d23/d5d/d77/d7b/dc4/fe7 x:0 0 0 2024-09-10T04:05:26.187 INFO:tasks.workunit.client.0.smithi046.stdout:9/618: dread d4/d3c/d4d/d5c/d87/faa [0,4194304] 0 2024-09-10T04:05:26.188 INFO:tasks.workunit.client.0.smithi046.stdout:2/631: chown d2/d8/d37 272 1 2024-09-10T04:05:26.192 INFO:tasks.workunit.client.1.smithi070.stdout:0/860: truncate d8/d13/d1a/d8f/d95/d106/fd6 3567698 0 2024-09-10T04:05:26.193 INFO:tasks.workunit.client.0.smithi046.stdout:5/807: creat d7/d11/d3f/d53/da9/f11b x:0 0 0 2024-09-10T04:05:26.196 INFO:tasks.workunit.client.0.smithi046.stdout:9/619: dwrite d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:26.198 INFO:tasks.workunit.client.0.smithi046.stdout:7/727: rename d5/d6/db/dd/d24/db9 to d5/d9/d12/d72/d77/de0/d1f/d7a/dfc 0 2024-09-10T04:05:26.200 INFO:tasks.workunit.client.0.smithi046.stdout:5/808: rmdir d7/d11/d14 39 2024-09-10T04:05:26.203 INFO:tasks.workunit.client.0.smithi046.stdout:7/728: mknod d5/d6/dbe/dd7/cfd 0 2024-09-10T04:05:26.211 INFO:tasks.workunit.client.0.smithi046.stdout:2/632: write d2/d6/d15/f70 [2976726,108510] 0 2024-09-10T04:05:26.211 INFO:tasks.workunit.client.0.smithi046.stdout:7/729: dwrite d5/d6/db/dd/d24/d76/fc3 [0,4194304] 0 2024-09-10T04:05:26.213 INFO:tasks.workunit.client.0.smithi046.stdout:9/620: creat d4/d3c/d4d/d5c/d87/fbe x:0 0 0 2024-09-10T04:05:26.219 INFO:tasks.workunit.client.0.smithi046.stdout:2/633: unlink d2/d6/d23/d5d/d77/d7b/f92 0 2024-09-10T04:05:26.223 INFO:tasks.workunit.client.0.smithi046.stdout:2/634: rename d2/d6/d23/d5d/d77/d7b/dc4/fe7 to d2/d6/d23/d42/db4/dca/fe8 0 2024-09-10T04:05:26.223 INFO:tasks.workunit.client.0.smithi046.stdout:2/635: chown d2/d8/cd 4078 1 2024-09-10T04:05:26.225 INFO:tasks.workunit.client.0.smithi046.stdout:9/621: creat d4/d3c/fbf x:0 0 0 2024-09-10T04:05:26.228 INFO:tasks.workunit.client.0.smithi046.stdout:9/622: mknod d4/d66/cc0 0 2024-09-10T04:05:26.229 INFO:tasks.workunit.client.0.smithi046.stdout:9/623: chown d4/d29/d65/f75 692063212 1 2024-09-10T04:05:26.230 INFO:tasks.workunit.client.0.smithi046.stdout:2/636: symlink d2/d6/d23/d5d/d77/d7b/dd8/le9 0 2024-09-10T04:05:26.232 INFO:tasks.workunit.client.0.smithi046.stdout:9/624: creat d4/d7/d81/fc1 x:0 0 0 2024-09-10T04:05:26.233 INFO:tasks.workunit.client.0.smithi046.stdout:9/625: stat d4/f28 0 2024-09-10T04:05:26.236 INFO:tasks.workunit.client.0.smithi046.stdout:9/626: symlink d4/d7/d61/lc2 0 2024-09-10T04:05:26.241 INFO:tasks.workunit.client.0.smithi046.stdout:5/809: dwrite d7/d11/d14/d2f/deb/da5/fb6 [0,4194304] 0 2024-09-10T04:05:26.243 INFO:tasks.workunit.client.0.smithi046.stdout:7/730: dwrite d5/d9/d12/d72/d77/da7/f75 [0,4194304] 0 2024-09-10T04:05:26.248 INFO:tasks.workunit.client.1.smithi070.stdout:2/704: sync 2024-09-10T04:05:26.248 INFO:tasks.workunit.client.1.smithi070.stdout:5/723: sync 2024-09-10T04:05:26.249 INFO:tasks.workunit.client.0.smithi046.stdout:5/810: mkdir d7/d11/d14/d8e/d96/d11c 0 2024-09-10T04:05:26.251 INFO:tasks.workunit.client.0.smithi046.stdout:2/637: dread d2/d6/d23/d55/f67 [0,4194304] 0 2024-09-10T04:05:26.254 INFO:tasks.workunit.client.1.smithi070.stdout:7/749: rename d2/d7/d9/d18/d3c/la6 to d2/d3/dc/d13/dfe/l106 0 2024-09-10T04:05:26.255 INFO:tasks.workunit.client.0.smithi046.stdout:2/638: mkdir d2/d6/d23/d5d/d77/dea 0 2024-09-10T04:05:26.258 INFO:tasks.workunit.client.1.smithi070.stdout:8/790: link d4/d16/d17/d1a/d60/d7a/f7b d4/d16/d17/d6f/db5/f110 0 2024-09-10T04:05:26.260 INFO:tasks.workunit.client.1.smithi070.stdout:1/650: dwrite d3/d8/f2d [4194304,4194304] 0 2024-09-10T04:05:26.261 INFO:tasks.workunit.client.1.smithi070.stdout:5/724: rmdir d17/d6f/d32/d90 39 2024-09-10T04:05:26.261 INFO:tasks.workunit.client.1.smithi070.stdout:1/651: chown d3/d10/db2/dc7/c55 3128 1 2024-09-10T04:05:26.262 INFO:tasks.workunit.client.0.smithi046.stdout:2/639: getdents d2/d6/d23/d42/db8 0 2024-09-10T04:05:26.262 INFO:tasks.workunit.client.1.smithi070.stdout:1/652: fdatasync d3/d8/d13/fc3 0 2024-09-10T04:05:26.263 INFO:tasks.workunit.client.1.smithi070.stdout:6/743: rename d0/d3/d18/d1b/d17/d28 to d0/d3/da4/dc0/dea 0 2024-09-10T04:05:26.265 INFO:tasks.workunit.client.1.smithi070.stdout:2/705: mknod d15/d18/d29/d2a/d2e/cfe 0 2024-09-10T04:05:26.265 INFO:tasks.workunit.client.0.smithi046.stdout:5/811: dread d7/d11/d14/f89 [0,4194304] 0 2024-09-10T04:05:26.267 INFO:tasks.workunit.client.1.smithi070.stdout:0/861: mknod d8/d24/da9/c11c 0 2024-09-10T04:05:26.268 INFO:tasks.workunit.client.1.smithi070.stdout:0/862: read - d8/d24/d97/fbe zero size 2024-09-10T04:05:26.269 INFO:tasks.workunit.client.1.smithi070.stdout:0/863: readlink d8/d13/de2/d7d/l83 0 2024-09-10T04:05:26.269 INFO:tasks.workunit.client.1.smithi070.stdout:7/750: mkdir d2/d7/d9/d18/d3c/d52/d107 0 2024-09-10T04:05:26.270 INFO:tasks.workunit.client.1.smithi070.stdout:8/791: chown d4/d16/d17/d1a/fa9 1641441 1 2024-09-10T04:05:26.274 INFO:tasks.workunit.client.1.smithi070.stdout:5/725: creat d17/d24/d77/ff7 x:0 0 0 2024-09-10T04:05:26.276 INFO:tasks.workunit.client.1.smithi070.stdout:1/653: write f1 [4682755,69022] 0 2024-09-10T04:05:26.276 INFO:tasks.workunit.client.1.smithi070.stdout:6/744: creat d0/d3/d5c/feb x:0 0 0 2024-09-10T04:05:26.278 INFO:tasks.workunit.client.1.smithi070.stdout:7/751: dwrite d2/d7/d9/f31 [0,4194304] 0 2024-09-10T04:05:26.279 INFO:tasks.workunit.client.1.smithi070.stdout:0/864: mknod d8/d13/de2/d7d/c11d 0 2024-09-10T04:05:26.281 INFO:tasks.workunit.client.0.smithi046.stdout:9/627: dwrite d4/d29/d32/f55 [0,4194304] 0 2024-09-10T04:05:26.281 INFO:tasks.workunit.client.1.smithi070.stdout:8/792: dwrite d4/d39/dfc/d43/d67/dec/d106/f10f [0,4194304] 0 2024-09-10T04:05:26.282 INFO:tasks.workunit.client.1.smithi070.stdout:5/726: mknod d17/d37/d93/cf8 0 2024-09-10T04:05:26.282 INFO:tasks.workunit.client.1.smithi070.stdout:0/865: fdatasync d8/d24/d2d/d5f/f60 0 2024-09-10T04:05:26.284 INFO:tasks.workunit.client.1.smithi070.stdout:1/654: write d3/d8/d12/d2f/fa9 [4609118,103937] 0 2024-09-10T04:05:26.285 INFO:tasks.workunit.client.1.smithi070.stdout:6/745: rmdir d0/d3/d18/d1b/d17/d3f/d40 39 2024-09-10T04:05:26.286 INFO:tasks.workunit.client.0.smithi046.stdout:2/640: dread d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:26.287 INFO:tasks.workunit.client.0.smithi046.stdout:9/628: mknod d4/d3c/d4d/cc3 0 2024-09-10T04:05:26.290 INFO:tasks.workunit.client.1.smithi070.stdout:0/866: mknod d8/d24/da9/c11e 0 2024-09-10T04:05:26.290 INFO:tasks.workunit.client.0.smithi046.stdout:9/629: write d4/d3c/d4d/d5c/f93 [1027448,49957] 0 2024-09-10T04:05:26.294 INFO:tasks.workunit.client.0.smithi046.stdout:9/630: write d4/d3c/d4d/d5c/f8d [650033,82965] 0 2024-09-10T04:05:26.296 INFO:tasks.workunit.client.1.smithi070.stdout:7/752: rename d2/d3/dc/d79/f9d to d2/d7/d9/dd5/f108 0 2024-09-10T04:05:26.296 INFO:tasks.workunit.client.1.smithi070.stdout:0/867: dwrite f5 [0,4194304] 0 2024-09-10T04:05:26.296 INFO:tasks.workunit.client.0.smithi046.stdout:7/731: dwrite d5/d6/dbe/dd7/f92 [0,4194304] 0 2024-09-10T04:05:26.297 INFO:tasks.workunit.client.0.smithi046.stdout:9/631: write d4/d3c/d4d/d5c/f4e [306392,37759] 0 2024-09-10T04:05:26.297 INFO:tasks.workunit.client.1.smithi070.stdout:7/753: chown d2/d3/dc/d13/fc7 0 1 2024-09-10T04:05:26.298 INFO:tasks.workunit.client.0.smithi046.stdout:2/641: symlink d2/d6/d9d/leb 0 2024-09-10T04:05:26.299 INFO:tasks.workunit.client.0.smithi046.stdout:2/642: stat d2/d86 0 2024-09-10T04:05:26.300 INFO:tasks.workunit.client.1.smithi070.stdout:5/727: creat d17/d24/d77/db4/ff9 x:0 0 0 2024-09-10T04:05:26.303 INFO:tasks.workunit.client.1.smithi070.stdout:8/793: mkdir d4/d39/dfc/d43/d67/dec/d111 0 2024-09-10T04:05:26.307 INFO:tasks.workunit.client.0.smithi046.stdout:9/632: mknod d4/d44/cc4 0 2024-09-10T04:05:26.307 INFO:tasks.workunit.client.0.smithi046.stdout:1/499: sync 2024-09-10T04:05:26.308 INFO:tasks.workunit.client.0.smithi046.stdout:0/515: sync 2024-09-10T04:05:26.308 INFO:tasks.workunit.client.0.smithi046.stdout:6/588: sync 2024-09-10T04:05:26.308 INFO:tasks.workunit.client.0.smithi046.stdout:4/804: sync 2024-09-10T04:05:26.308 INFO:tasks.workunit.client.0.smithi046.stdout:5/812: dwrite d7/d11/d14/d2f/deb/da5/dcc/dc0/fe0 [0,4194304] 0 2024-09-10T04:05:26.309 INFO:tasks.workunit.client.1.smithi070.stdout:9/769: sync 2024-09-10T04:05:26.310 INFO:tasks.workunit.client.0.smithi046.stdout:4/805: chown d3/d12/d17/de1 2590099 1 2024-09-10T04:05:26.314 INFO:tasks.workunit.client.0.smithi046.stdout:6/589: read da/db/d10/d27/f88 [2375696,11391] 0 2024-09-10T04:05:26.315 INFO:tasks.workunit.client.1.smithi070.stdout:0/868: mknod d8/d13/d1a/d8f/d95/c11f 0 2024-09-10T04:05:26.316 INFO:tasks.workunit.client.1.smithi070.stdout:0/869: readlink d8/d24/d36/l101 0 2024-09-10T04:05:26.318 INFO:tasks.workunit.client.1.smithi070.stdout:2/706: stat d15/d18/d29/d2a/d2e/da6/dc0/dcb 0 2024-09-10T04:05:26.318 INFO:tasks.workunit.client.0.smithi046.stdout:9/633: symlink d4/d7/d61/lc5 0 2024-09-10T04:05:26.319 INFO:tasks.workunit.client.1.smithi070.stdout:2/707: chown d15/d18/d56/d7c/d55/d6d 0 1 2024-09-10T04:05:26.319 INFO:tasks.workunit.client.0.smithi046.stdout:9/634: write d4/d3c/d4d/d5c/f4e [323411,30814] 0 2024-09-10T04:05:26.321 INFO:tasks.workunit.client.1.smithi070.stdout:7/754: creat d2/d40/d6c/db2/db7/d104/f109 x:0 0 0 2024-09-10T04:05:26.322 INFO:tasks.workunit.client.0.smithi046.stdout:7/732: symlink d5/d9/d12/dc5/lfe 0 2024-09-10T04:05:26.324 INFO:tasks.workunit.client.1.smithi070.stdout:5/728: creat d17/d24/d77/ffa x:0 0 0 2024-09-10T04:05:26.325 INFO:tasks.workunit.client.1.smithi070.stdout:8/794: rmdir d4/dbf/d74/da4 39 2024-09-10T04:05:26.326 INFO:tasks.workunit.client.1.smithi070.stdout:8/795: write d4/d16/dcc/fe8 [1272355,58362] 0 2024-09-10T04:05:26.327 INFO:tasks.workunit.client.1.smithi070.stdout:1/655: write d3/d8/d12/f4a [625525,41330] 0 2024-09-10T04:05:26.330 INFO:tasks.workunit.client.1.smithi070.stdout:0/870: dread d8/d13/f5a [0,4194304] 0 2024-09-10T04:05:26.331 INFO:tasks.workunit.client.0.smithi046.stdout:4/806: creat d3/d12/d17/d29/dba/f118 x:0 0 0 2024-09-10T04:05:26.331 INFO:tasks.workunit.client.1.smithi070.stdout:0/871: write d8/d24/d36/f46 [5553466,6518] 0 2024-09-10T04:05:26.333 INFO:tasks.workunit.client.1.smithi070.stdout:9/770: unlink d9/d1c/dac/d8a/d3c/f40 0 2024-09-10T04:05:26.336 INFO:tasks.workunit.client.0.smithi046.stdout:2/643: dwrite d2/d49/f74 [0,4194304] 0 2024-09-10T04:05:26.336 INFO:tasks.workunit.client.0.smithi046.stdout:3/647: sync 2024-09-10T04:05:26.336 INFO:tasks.workunit.client.0.smithi046.stdout:2/644: stat d2/d8/f29 0 2024-09-10T04:05:26.337 INFO:tasks.workunit.client.0.smithi046.stdout:3/648: stat d12/d13/d37/l34 0 2024-09-10T04:05:26.339 INFO:tasks.workunit.client.0.smithi046.stdout:3/649: stat d12/d5d/d71/d49 0 2024-09-10T04:05:26.339 INFO:tasks.workunit.client.1.smithi070.stdout:7/755: unlink d2/d7/db/d3a/le7 0 2024-09-10T04:05:26.340 INFO:tasks.workunit.client.1.smithi070.stdout:9/771: dwrite d9/d1c/dac/d8a/dc4/de1/fd8 [0,4194304] 0 2024-09-10T04:05:26.342 INFO:tasks.workunit.client.1.smithi070.stdout:9/772: write d9/d59/d2a/d3f/fe4 [863544,79712] 0 2024-09-10T04:05:26.344 INFO:tasks.workunit.client.0.smithi046.stdout:9/635: unlink d4/d7/l60 0 2024-09-10T04:05:26.350 INFO:tasks.workunit.client.0.smithi046.stdout:5/813: dwrite d7/d11/d14/d45/f6d [0,4194304] 0 2024-09-10T04:05:26.350 INFO:tasks.workunit.client.0.smithi046.stdout:4/807: creat d3/d12/d17/de1/ddf/f119 x:0 0 0 2024-09-10T04:05:26.351 INFO:tasks.workunit.client.1.smithi070.stdout:0/872: creat d8/d13/de2/d6f/f120 x:0 0 0 2024-09-10T04:05:26.354 INFO:tasks.workunit.client.0.smithi046.stdout:3/650: dwrite d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fb4 [0,4194304] 0 2024-09-10T04:05:26.355 INFO:tasks.workunit.client.0.smithi046.stdout:6/590: mknod da/d96/cc4 0 2024-09-10T04:05:26.356 INFO:tasks.workunit.client.0.smithi046.stdout:2/645: mkdir d2/d6/d23/d42/de6/dec 0 2024-09-10T04:05:26.357 INFO:tasks.workunit.client.1.smithi070.stdout:7/756: mkdir d2/dc8/d10a 0 2024-09-10T04:05:26.361 INFO:tasks.workunit.client.0.smithi046.stdout:1/500: rename d3/d27/d32 to d3/da1 0 2024-09-10T04:05:26.363 INFO:tasks.workunit.client.0.smithi046.stdout:1/501: readlink d3/d27/l49 0 2024-09-10T04:05:26.364 INFO:tasks.workunit.client.0.smithi046.stdout:9/636: dwrite d4/d3c/d4d/d5c/f8d [0,4194304] 0 2024-09-10T04:05:26.366 INFO:tasks.workunit.client.1.smithi070.stdout:3/756: sync 2024-09-10T04:05:26.366 INFO:tasks.workunit.client.1.smithi070.stdout:4/678: sync 2024-09-10T04:05:26.369 INFO:tasks.workunit.client.1.smithi070.stdout:9/773: read d9/f25 [652228,87331] 0 2024-09-10T04:05:26.373 INFO:tasks.workunit.client.1.smithi070.stdout:1/656: link d3/d8/fc8 d3/d10/d1d/dbd/fd4 0 2024-09-10T04:05:26.374 INFO:tasks.workunit.client.0.smithi046.stdout:5/814: dread d7/d11/d3f/fc8 [0,4194304] 0 2024-09-10T04:05:26.375 INFO:tasks.workunit.client.1.smithi070.stdout:6/746: dread d0/d3/d1f/d55/f6d [0,4194304] 0 2024-09-10T04:05:26.375 INFO:tasks.workunit.client.0.smithi046.stdout:5/815: readlink d7/d11/d14/d8e/de2/le6 0 2024-09-10T04:05:26.376 INFO:tasks.workunit.client.1.smithi070.stdout:0/873: rename d8/d24/d36/cfe to d8/d13/de2/d7d/c121 0 2024-09-10T04:05:26.377 INFO:tasks.workunit.client.1.smithi070.stdout:0/874: fdatasync d8/d13/de2/d4f/f7f 0 2024-09-10T04:05:26.377 INFO:tasks.workunit.client.0.smithi046.stdout:4/808: symlink d3/d12/d17/d91/dbf/l11a 0 2024-09-10T04:05:26.379 INFO:tasks.workunit.client.0.smithi046.stdout:2/646: mkdir d2/d86/d9c/dbc/ded 0 2024-09-10T04:05:26.381 INFO:tasks.workunit.client.1.smithi070.stdout:2/708: dwrite d15/d18/d29/d2a/d2e/d32/f33 [0,4194304] 0 2024-09-10T04:05:26.381 INFO:tasks.workunit.client.0.smithi046.stdout:0/516: getdents d1/dc/d25/d33/d3/d29 0 2024-09-10T04:05:26.384 INFO:tasks.workunit.client.0.smithi046.stdout:8/639: sync 2024-09-10T04:05:26.385 INFO:tasks.workunit.client.1.smithi070.stdout:8/796: creat d4/d16/d17/f112 x:0 0 0 2024-09-10T04:05:26.386 INFO:tasks.workunit.client.1.smithi070.stdout:5/729: dwrite d17/d19/d6a/d54/d57/d63/f9a [0,4194304] 0 2024-09-10T04:05:26.386 INFO:tasks.workunit.client.1.smithi070.stdout:8/797: chown d4/d16/d17 106424425 1 2024-09-10T04:05:26.387 INFO:tasks.workunit.client.1.smithi070.stdout:9/774: truncate d9/d59/d2a/d3f/f86 287982 0 2024-09-10T04:05:26.388 INFO:tasks.workunit.client.0.smithi046.stdout:6/591: mknod da/db/d11/d2b/cc5 0 2024-09-10T04:05:26.389 INFO:tasks.workunit.client.1.smithi070.stdout:6/747: mknod d0/d3/da4/cec 0 2024-09-10T04:05:26.390 INFO:tasks.workunit.client.0.smithi046.stdout:4/809: mkdir d3/d12/d17/d22/d51/d5c/d11b 0 2024-09-10T04:05:26.391 INFO:tasks.workunit.client.0.smithi046.stdout:2/647: fsync d2/f25 0 2024-09-10T04:05:26.393 INFO:tasks.workunit.client.1.smithi070.stdout:7/757: symlink d2/d7/d9/dd5/d102/l10b 0 2024-09-10T04:05:26.394 INFO:tasks.workunit.client.0.smithi046.stdout:0/517: creat d1/dc/d25/f9e x:0 0 0 2024-09-10T04:05:26.395 INFO:tasks.workunit.client.0.smithi046.stdout:0/518: chown d1/dc/d25/d33/d3/d29 12866 1 2024-09-10T04:05:26.399 INFO:tasks.workunit.client.0.smithi046.stdout:4/810: mkdir d3/dab/d11c 0 2024-09-10T04:05:26.400 INFO:tasks.workunit.client.1.smithi070.stdout:9/775: creat d9/d1c/dac/d8a/dc4/dda/de0/d90/df0/ff8 x:0 0 0 2024-09-10T04:05:26.401 INFO:tasks.workunit.client.1.smithi070.stdout:6/748: mknod d0/d3/d18/d1b/d17/d54/dc4/ced 0 2024-09-10T04:05:26.402 INFO:tasks.workunit.client.0.smithi046.stdout:7/733: rename d5/d6/db/dd/d24/db7/cdb to d5/d9/d12/d72/d77/cff 0 2024-09-10T04:05:26.403 INFO:tasks.workunit.client.1.smithi070.stdout:0/875: symlink d8/d13/d1a/l122 0 2024-09-10T04:05:26.404 INFO:tasks.workunit.client.0.smithi046.stdout:7/734: write d5/d6/db/dd/f2f [32902,126661] 0 2024-09-10T04:05:26.404 INFO:tasks.workunit.client.1.smithi070.stdout:0/876: chown d8/d85/df5/le1 6242020 1 2024-09-10T04:05:26.405 INFO:tasks.workunit.client.1.smithi070.stdout:0/877: chown d8/d13/de2/d81 130 1 2024-09-10T04:05:26.405 INFO:tasks.workunit.client.0.smithi046.stdout:2/648: symlink d2/d6/d15/lee 0 2024-09-10T04:05:26.407 INFO:tasks.workunit.client.1.smithi070.stdout:7/758: truncate d2/d40/d6c/db2/fc2 774633 0 2024-09-10T04:05:26.409 INFO:tasks.workunit.client.1.smithi070.stdout:2/709: mknod d15/cff 0 2024-09-10T04:05:26.410 INFO:tasks.workunit.client.0.smithi046.stdout:9/637: creat d4/d3c/d4d/d5c/fc6 x:0 0 0 2024-09-10T04:05:26.411 INFO:tasks.workunit.client.1.smithi070.stdout:3/757: creat db/d1c/d25/f104 x:0 0 0 2024-09-10T04:05:26.412 INFO:tasks.workunit.client.0.smithi046.stdout:8/640: creat d1/d29/d19/db7/fd5 x:0 0 0 2024-09-10T04:05:26.413 INFO:tasks.workunit.client.0.smithi046.stdout:6/592: dread da/db/d11/f32 [0,4194304] 0 2024-09-10T04:05:26.414 INFO:tasks.workunit.client.0.smithi046.stdout:0/519: symlink d1/dc/d25/d33/d1a/l9f 0 2024-09-10T04:05:26.416 INFO:tasks.workunit.client.1.smithi070.stdout:8/798: symlink d4/d16/d17/d19/d62/dd1/l113 0 2024-09-10T04:05:26.420 INFO:tasks.workunit.client.1.smithi070.stdout:1/657: creat d3/d8/d12/d2f/fd5 x:0 0 0 2024-09-10T04:05:26.421 INFO:tasks.workunit.client.1.smithi070.stdout:1/658: write d3/d8/d13/d67/d83/fc5 [8920706,39594] 0 2024-09-10T04:05:26.422 INFO:tasks.workunit.client.1.smithi070.stdout:1/659: fdatasync d3/d10/db2/fb7 0 2024-09-10T04:05:26.422 INFO:tasks.workunit.client.1.smithi070.stdout:6/749: creat d0/d3/d18/d2a/fee x:0 0 0 2024-09-10T04:05:26.425 INFO:tasks.workunit.client.1.smithi070.stdout:9/776: creat d9/d1c/dac/d8a/dc4/dda/de0/d90/d89/ddc/ff9 x:0 0 0 2024-09-10T04:05:26.425 INFO:tasks.workunit.client.1.smithi070.stdout:1/660: chown d3/d8/d27/d7a/d94/da5 1922032876 1 2024-09-10T04:05:26.426 INFO:tasks.workunit.client.0.smithi046.stdout:7/735: creat d5/d6/d55/d7d/f100 x:0 0 0 2024-09-10T04:05:26.428 INFO:tasks.workunit.client.1.smithi070.stdout:5/730: dwrite d17/d24/f27 [0,4194304] 0 2024-09-10T04:05:26.429 INFO:tasks.workunit.client.1.smithi070.stdout:7/759: creat d2/d7/d9/db4/f10c x:0 0 0 2024-09-10T04:05:26.430 INFO:tasks.workunit.client.0.smithi046.stdout:1/502: getdents d3 0 2024-09-10T04:05:26.431 INFO:tasks.workunit.client.0.smithi046.stdout:1/503: fsync d3/dc/d34/d51/f99 0 2024-09-10T04:05:26.433 INFO:tasks.workunit.client.0.smithi046.stdout:8/641: creat d1/d29/d17/d1f/d46/d6a/d82/fd6 x:0 0 0 2024-09-10T04:05:26.435 INFO:tasks.workunit.client.0.smithi046.stdout:6/593: dread - da/db/d11/d2b/d55/f81 zero size 2024-09-10T04:05:26.437 INFO:tasks.workunit.client.1.smithi070.stdout:4/679: getdents de/d13/d18/d26/d8c/d93 0 2024-09-10T04:05:26.438 INFO:tasks.workunit.client.0.smithi046.stdout:0/520: creat d1/dc/d25/d33/d1a/d41/fa0 x:0 0 0 2024-09-10T04:05:26.440 INFO:tasks.workunit.client.0.smithi046.stdout:5/816: getdents d7/d11/d14/d8e/de2/d108 0 2024-09-10T04:05:26.442 INFO:tasks.workunit.client.1.smithi070.stdout:4/680: dwrite de/d13/f45 [0,4194304] 0 2024-09-10T04:05:26.443 INFO:tasks.workunit.client.0.smithi046.stdout:4/811: dwrite d3/d12/d17/d22/d51/d5c/ff1 [0,4194304] 0 2024-09-10T04:05:26.448 INFO:tasks.workunit.client.0.smithi046.stdout:3/651: rename d12/f4d to d12/d13/d37/d51/db5/d4b/db7/dc0/fd6 0 2024-09-10T04:05:26.449 INFO:tasks.workunit.client.1.smithi070.stdout:9/777: write d9/d1c/f55 [471900,28048] 0 2024-09-10T04:05:26.451 INFO:tasks.workunit.client.0.smithi046.stdout:7/736: creat d5/d6/dbe/dd7/f101 x:0 0 0 2024-09-10T04:05:26.453 INFO:tasks.workunit.client.1.smithi070.stdout:9/778: dread d9/d1c/d34/fd9 [0,4194304] 0 2024-09-10T04:05:26.454 INFO:tasks.workunit.client.0.smithi046.stdout:7/737: dread d5/d9/d12/d72/d77/de0/d1f/f3b [0,4194304] 0 2024-09-10T04:05:26.455 INFO:tasks.workunit.client.1.smithi070.stdout:9/779: chown d9/d1c/dac/d8a/dc4/dda/de0/d90/d89 15297928 1 2024-09-10T04:05:26.456 INFO:tasks.workunit.client.1.smithi070.stdout:1/661: creat d3/d10/db2/dc7/d54/fd6 x:0 0 0 2024-09-10T04:05:26.456 INFO:tasks.workunit.client.1.smithi070.stdout:5/731: mkdir d17/d19/d6a/dfb 0 2024-09-10T04:05:26.457 INFO:tasks.workunit.client.1.smithi070.stdout:5/732: chown d17/d19/f1d 15371 1 2024-09-10T04:05:26.458 INFO:tasks.workunit.client.1.smithi070.stdout:5/733: chown d17/d19/l1f 426250362 1 2024-09-10T04:05:26.459 INFO:tasks.workunit.client.0.smithi046.stdout:8/642: creat d1/d29/fd7 x:0 0 0 2024-09-10T04:05:26.461 INFO:tasks.workunit.client.1.smithi070.stdout:9/780: dread d9/d1c/d34/fd9 [0,4194304] 0 2024-09-10T04:05:26.461 INFO:tasks.workunit.client.0.smithi046.stdout:5/817: dread d7/d11/d14/d2f/d2b/f3a [0,4194304] 0 2024-09-10T04:05:26.462 INFO:tasks.workunit.client.0.smithi046.stdout:0/521: unlink d1/d13/d37/f87 0 2024-09-10T04:05:26.462 INFO:tasks.workunit.client.1.smithi070.stdout:9/781: write d9/d1c/dac/d8a/d3c/d92/fec [480064,95630] 0 2024-09-10T04:05:26.463 INFO:tasks.workunit.client.0.smithi046.stdout:5/818: write d7/d11/f6e [2038360,5635] 0 2024-09-10T04:05:26.464 INFO:tasks.workunit.client.1.smithi070.stdout:3/758: unlink db/c15 0 2024-09-10T04:05:26.465 INFO:tasks.workunit.client.0.smithi046.stdout:5/819: readlink d7/d11/d14/d8e/la2 0 2024-09-10T04:05:26.468 INFO:tasks.workunit.client.0.smithi046.stdout:8/643: dwrite d1/d29/d17/d1f/d46/d76/fab [0,4194304] 0 2024-09-10T04:05:26.472 INFO:tasks.workunit.client.0.smithi046.stdout:4/812: unlink d3/d12/d17/cfd 0 2024-09-10T04:05:26.474 INFO:tasks.workunit.client.0.smithi046.stdout:9/638: write d4/d7/fe [831977,114172] 0 2024-09-10T04:05:26.476 INFO:tasks.workunit.client.1.smithi070.stdout:4/681: mknod de/d13/d18/d23/d47/d59/d79/d6a/ce6 0 2024-09-10T04:05:26.479 INFO:tasks.workunit.client.1.smithi070.stdout:0/878: dwrite d8/d85/fac [0,4194304] 0 2024-09-10T04:05:26.482 INFO:tasks.workunit.client.0.smithi046.stdout:7/738: rename d5/f56 to d5/d9/d12/dc5/f102 0 2024-09-10T04:05:26.486 INFO:tasks.workunit.client.1.smithi070.stdout:7/760: link d2/d7/d9/d18/d23/f91 d2/dc8/df9/f10d 0 2024-09-10T04:05:26.490 INFO:tasks.workunit.client.1.smithi070.stdout:1/662: creat d3/d10/db2/dc7/d54/fd7 x:0 0 0 2024-09-10T04:05:26.491 INFO:tasks.workunit.client.0.smithi046.stdout:6/594: symlink da/db/d11/d2b/lc6 0 2024-09-10T04:05:26.496 INFO:tasks.workunit.client.1.smithi070.stdout:9/782: unlink d9/d59/d2a/c2b 0 2024-09-10T04:05:26.498 INFO:tasks.workunit.client.0.smithi046.stdout:1/504: write d3/dc/d34/d51/f78 [1555886,93261] 0 2024-09-10T04:05:26.499 INFO:tasks.workunit.client.0.smithi046.stdout:0/522: dread d1/d13/d37/d75/f7f [0,4194304] 0 2024-09-10T04:05:26.500 INFO:tasks.workunit.client.0.smithi046.stdout:5/820: symlink d7/d11/d14/d45/l11d 0 2024-09-10T04:05:26.501 INFO:tasks.workunit.client.0.smithi046.stdout:0/523: stat d1/dc/d25/d33/d3/l30 0 2024-09-10T04:05:26.505 INFO:tasks.workunit.client.1.smithi070.stdout:4/682: fdatasync de/d13/d43/f76 0 2024-09-10T04:05:26.507 INFO:tasks.workunit.client.0.smithi046.stdout:3/652: mknod d12/d13/d14/d1e/dd2/cd7 0 2024-09-10T04:05:26.508 INFO:tasks.workunit.client.1.smithi070.stdout:0/879: creat d8/d13/d1a/d8f/d95/d106/f123 x:0 0 0 2024-09-10T04:05:26.514 INFO:tasks.workunit.client.1.smithi070.stdout:7/761: mkdir d2/d7/d9/d18/d3c/d52/d10e 0 2024-09-10T04:05:26.516 INFO:tasks.workunit.client.0.smithi046.stdout:7/739: chown d5/d9/d12/d72/d77/de0/c63 2 1 2024-09-10T04:05:26.517 INFO:tasks.workunit.client.1.smithi070.stdout:1/663: creat d3/d8/d13/d3f/d6d/d81/fd8 x:0 0 0 2024-09-10T04:05:26.518 INFO:tasks.workunit.client.0.smithi046.stdout:6/595: rmdir da/db/d11/d2b/d2f/d3b 39 2024-09-10T04:05:26.519 INFO:tasks.workunit.client.0.smithi046.stdout:6/596: chown da/db/l9c 160936 1 2024-09-10T04:05:26.519 INFO:tasks.workunit.client.0.smithi046.stdout:9/639: dread d4/d3c/f40 [0,4194304] 0 2024-09-10T04:05:26.521 INFO:tasks.workunit.client.0.smithi046.stdout:8/644: dwrite d1/d29/d12/f45 [0,4194304] 0 2024-09-10T04:05:26.521 INFO:tasks.workunit.client.0.smithi046.stdout:2/649: truncate d2/d6/d15/f70 1964765 0 2024-09-10T04:05:26.521 INFO:tasks.workunit.client.0.smithi046.stdout:6/597: chown da/f24 25 1 2024-09-10T04:05:26.521 INFO:tasks.workunit.client.0.smithi046.stdout:1/505: rename d3/l19 to d3/dc/d21/d3a/la2 0 2024-09-10T04:05:26.522 INFO:tasks.workunit.client.1.smithi070.stdout:7/762: dwrite d2/d7/d9/d18/fcc [0,4194304] 0 2024-09-10T04:05:26.524 INFO:tasks.workunit.client.0.smithi046.stdout:5/821: rmdir d7/d11/d14/d45 39 2024-09-10T04:05:26.525 INFO:tasks.workunit.client.1.smithi070.stdout:2/710: getdents d15/d18/d29/d2a/d2e/d32 0 2024-09-10T04:05:26.526 INFO:tasks.workunit.client.1.smithi070.stdout:1/664: dwrite d3/d8/d12/d2f/d32/f4c [8388608,4194304] 0 2024-09-10T04:05:26.527 INFO:tasks.workunit.client.1.smithi070.stdout:9/783: rename d9/d1c/dac/d8a/dc4/dda/de0/d90/d89/ddc to d9/d1c/d34/d8b/dcc/dfa 0 2024-09-10T04:05:26.529 INFO:tasks.workunit.client.1.smithi070.stdout:3/759: truncate db/d1c/d25/dd8/fe8 1053099 0 2024-09-10T04:05:26.530 INFO:tasks.workunit.client.0.smithi046.stdout:3/653: chown d12/d5d/l68 646 1 2024-09-10T04:05:26.530 INFO:tasks.workunit.client.1.smithi070.stdout:8/799: getdents d4/d16/dcc 0 2024-09-10T04:05:26.530 INFO:tasks.workunit.client.1.smithi070.stdout:6/750: getdents d0/d3/da4 0 2024-09-10T04:05:26.534 INFO:tasks.workunit.client.0.smithi046.stdout:0/524: rename d1/f6c to d1/dc/d49/fa1 0 2024-09-10T04:05:26.537 INFO:tasks.workunit.client.0.smithi046.stdout:6/598: creat da/d71/fc7 x:0 0 0 2024-09-10T04:05:26.537 INFO:tasks.workunit.client.0.smithi046.stdout:0/525: chown d1/dc/d25/d33/f11 28702 1 2024-09-10T04:05:26.539 INFO:tasks.workunit.client.0.smithi046.stdout:2/650: unlink d2/d6/l22 0 2024-09-10T04:05:26.540 INFO:tasks.workunit.client.1.smithi070.stdout:5/734: rmdir d17/d24/d45/dc0/de5 0 2024-09-10T04:05:26.542 INFO:tasks.workunit.client.0.smithi046.stdout:8/645: mknod d1/d29/d17/d1f/d46/d6a/cd8 0 2024-09-10T04:05:26.542 INFO:tasks.workunit.client.0.smithi046.stdout:9/640: truncate d4/d3c/f84 400221 0 2024-09-10T04:05:26.543 INFO:tasks.workunit.client.1.smithi070.stdout:2/711: creat d15/d18/d29/d2a/d2e/d32/f100 x:0 0 0 2024-09-10T04:05:26.545 INFO:tasks.workunit.client.0.smithi046.stdout:5/822: symlink d7/d11/d14/d2f/d2b/d92/l11e 0 2024-09-10T04:05:26.547 INFO:tasks.workunit.client.0.smithi046.stdout:9/641: chown d4/d29/c9c 48513 1 2024-09-10T04:05:26.550 INFO:tasks.workunit.client.0.smithi046.stdout:0/526: dwrite d1/dc/d25/d33/d4/d19/f58 [0,4194304] 0 2024-09-10T04:05:26.554 INFO:tasks.workunit.client.0.smithi046.stdout:0/527: dread - d1/dc/d25/d33/d3/dd/f96 zero size 2024-09-10T04:05:26.565 INFO:tasks.workunit.client.0.smithi046.stdout:9/642: dwrite d4/d7/f31 [0,4194304] 0 2024-09-10T04:05:26.565 INFO:tasks.workunit.client.0.smithi046.stdout:2/651: creat d2/d6/d23/d5d/fef x:0 0 0 2024-09-10T04:05:26.565 INFO:tasks.workunit.client.0.smithi046.stdout:6/599: dread da/dc/d22/f34 [0,4194304] 0 2024-09-10T04:05:26.567 INFO:tasks.workunit.client.0.smithi046.stdout:4/813: getdents d3/d12/d17/d105 0 2024-09-10T04:05:26.568 INFO:tasks.workunit.client.1.smithi070.stdout:0/880: fdatasync d8/d13/d1a/f45 0 2024-09-10T04:05:26.575 INFO:tasks.workunit.client.0.smithi046.stdout:8/646: rename d1/d29/d17/d1f/d46/d6a/d82/fc4 to d1/d29/d17/d1f/d46/d6a/fd9 0 2024-09-10T04:05:26.576 INFO:tasks.workunit.client.0.smithi046.stdout:3/654: write d12/f5f [1317343,106451] 0 2024-09-10T04:05:26.577 INFO:tasks.workunit.client.1.smithi070.stdout:7/763: mknod d2/d7/d9/d18/d3c/d52/d103/c10f 0 2024-09-10T04:05:26.577 INFO:tasks.workunit.client.0.smithi046.stdout:3/655: read - d12/d5d/f6e zero size 2024-09-10T04:05:26.580 INFO:tasks.workunit.client.1.smithi070.stdout:1/665: mknod d3/d90/dcb/cd9 0 2024-09-10T04:05:26.581 INFO:tasks.workunit.client.0.smithi046.stdout:9/643: read - d4/d3c/d4d/f76 zero size 2024-09-10T04:05:26.582 INFO:tasks.workunit.client.1.smithi070.stdout:1/666: readlink d3/d8/d12/d2f/d32/da8/lb6 0 2024-09-10T04:05:26.582 INFO:tasks.workunit.client.0.smithi046.stdout:2/652: creat d2/d6/d23/ff0 x:0 0 0 2024-09-10T04:05:26.583 INFO:tasks.workunit.client.1.smithi070.stdout:7/764: dwrite d2/d3/dc/d5d/d2d/f8e [0,4194304] 0 2024-09-10T04:05:26.587 INFO:tasks.workunit.client.0.smithi046.stdout:4/814: mkdir d3/d12/d14/d11d 0 2024-09-10T04:05:26.589 INFO:tasks.workunit.client.0.smithi046.stdout:5/823: dwrite d7/d11/d14/d2f/deb/fdc [0,4194304] 0 2024-09-10T04:05:26.589 INFO:tasks.workunit.client.0.smithi046.stdout:4/815: readlink d3/d12/d17/d22/d30/lb3 0 2024-09-10T04:05:26.589 INFO:tasks.workunit.client.1.smithi070.stdout:4/683: creat de/d13/d18/d23/d39/fe7 x:0 0 0 2024-09-10T04:05:26.590 INFO:tasks.workunit.client.1.smithi070.stdout:5/735: dread f11 [0,4194304] 0 2024-09-10T04:05:26.591 INFO:tasks.workunit.client.0.smithi046.stdout:4/816: write d3/d12/d17/d29/d58/ffe [3439396,38746] 0 2024-09-10T04:05:26.594 INFO:tasks.workunit.client.0.smithi046.stdout:4/817: chown d3/d12/d17/d22/d51/d94/fd6 53 1 2024-09-10T04:05:26.597 INFO:tasks.workunit.client.1.smithi070.stdout:7/765: dread d2/d3/dc/d5d/d2d/fba [0,4194304] 0 2024-09-10T04:05:26.599 INFO:tasks.workunit.client.1.smithi070.stdout:6/751: mknod d0/d3/d18/d1b/d17/d3f/d40/d94/cef 0 2024-09-10T04:05:26.599 INFO:tasks.workunit.client.0.smithi046.stdout:0/528: write d1/dc/d25/d33/d4/f6f [4552219,126573] 0 2024-09-10T04:05:26.601 INFO:tasks.workunit.client.1.smithi070.stdout:9/784: truncate d9/d1c/f2c 2804248 0 2024-09-10T04:05:26.601 INFO:tasks.workunit.client.1.smithi070.stdout:9/785: stat l4 0 2024-09-10T04:05:26.601 INFO:tasks.workunit.client.0.smithi046.stdout:8/647: dread d1/d29/d17/d1f/d46/d6a/f94 [0,4194304] 0 2024-09-10T04:05:26.602 INFO:tasks.workunit.client.1.smithi070.stdout:9/786: stat d9/d1c/dac/d8a/d3c 0 2024-09-10T04:05:26.603 INFO:tasks.workunit.client.1.smithi070.stdout:2/712: symlink d15/df6/l101 0 2024-09-10T04:05:26.606 INFO:tasks.workunit.client.1.smithi070.stdout:3/760: rmdir db/d83/dfc 0 2024-09-10T04:05:26.607 INFO:tasks.workunit.client.0.smithi046.stdout:7/740: link d5/d6/db/dd/d24/d76/d9c/lf3 d5/d9/d8b/l103 0 2024-09-10T04:05:26.608 INFO:tasks.workunit.client.0.smithi046.stdout:6/600: dwrite da/db/d11/d13/d1e/f4d [0,4194304] 0 2024-09-10T04:05:26.609 INFO:tasks.workunit.client.1.smithi070.stdout:1/667: mknod d3/d8/d13/d67/d83/cda 0 2024-09-10T04:05:26.610 INFO:tasks.workunit.client.0.smithi046.stdout:7/741: write d5/d6/d32/ff2 [616424,55975] 0 2024-09-10T04:05:26.611 INFO:tasks.workunit.client.0.smithi046.stdout:1/506: dwrite d3/d27/f18 [4194304,4194304] 0 2024-09-10T04:05:26.611 INFO:tasks.workunit.client.0.smithi046.stdout:9/644: mkdir d4/d7/d1f/d54/dc7 0 2024-09-10T04:05:26.616 INFO:tasks.workunit.client.1.smithi070.stdout:8/800: mkdir d4/dbf/d74/da4/d114 0 2024-09-10T04:05:26.623 INFO:tasks.workunit.client.0.smithi046.stdout:5/824: mknod d7/d11/d14/d8e/c11f 0 2024-09-10T04:05:26.624 INFO:tasks.workunit.client.1.smithi070.stdout:0/881: mknod d8/d85/c124 0 2024-09-10T04:05:26.625 INFO:tasks.workunit.client.1.smithi070.stdout:0/882: chown d8/d24/d36/c3b 10 1 2024-09-10T04:05:26.626 INFO:tasks.workunit.client.1.smithi070.stdout:0/883: chown d8/d24/d2d/d5f 76616 1 2024-09-10T04:05:26.627 INFO:tasks.workunit.client.0.smithi046.stdout:0/529: symlink d1/dc/d25/d33/d1a/la2 0 2024-09-10T04:05:26.628 INFO:tasks.workunit.client.1.smithi070.stdout:6/752: read d0/d3/d8c/d9a/fac [612412,82928] 0 2024-09-10T04:05:26.634 INFO:tasks.workunit.client.1.smithi070.stdout:4/684: dread de/d13/d18/d26/f42 [0,4194304] 0 2024-09-10T04:05:26.636 INFO:tasks.workunit.client.0.smithi046.stdout:8/648: write d1/f49 [97785,106388] 0 2024-09-10T04:05:26.637 INFO:tasks.workunit.client.0.smithi046.stdout:7/742: mkdir d5/d9/d12/d72/d104 0 2024-09-10T04:05:26.639 INFO:tasks.workunit.client.0.smithi046.stdout:2/653: fdatasync d2/d6/d15/f3e 0 2024-09-10T04:05:26.641 INFO:tasks.workunit.client.0.smithi046.stdout:5/825: creat d7/d11/d14/d2f/deb/da5/f120 x:0 0 0 2024-09-10T04:05:26.643 INFO:tasks.workunit.client.1.smithi070.stdout:0/884: mkdir d8/d13/de2/d4f/d125 0 2024-09-10T04:05:26.644 INFO:tasks.workunit.client.0.smithi046.stdout:0/530: rename d1/dc/d25/d33/d3/dd/l6b to d1/dc/d25/d72/la3 0 2024-09-10T04:05:26.646 INFO:tasks.workunit.client.0.smithi046.stdout:3/656: dwrite d12/d13/d37/d51/db5/d4b/db7/dc0/fd6 [0,4194304] 0 2024-09-10T04:05:26.647 INFO:tasks.workunit.client.1.smithi070.stdout:6/753: unlink d0/d7f/fc9 0 2024-09-10T04:05:26.649 INFO:tasks.workunit.client.0.smithi046.stdout:9/645: dread d4/d7/f1e [0,4194304] 0 2024-09-10T04:05:26.649 INFO:tasks.workunit.client.0.smithi046.stdout:9/646: chown d4/d7/d1f/f15 236213 1 2024-09-10T04:05:26.650 INFO:tasks.workunit.client.1.smithi070.stdout:3/761: link db/dcc/fd5 db/d1c/d53/f105 0 2024-09-10T04:05:26.651 INFO:tasks.workunit.client.1.smithi070.stdout:1/668: symlink d3/d8/d13/d3f/d5d/ldb 0 2024-09-10T04:05:26.653 INFO:tasks.workunit.client.0.smithi046.stdout:1/507: unlink d3/dc/c1c 0 2024-09-10T04:05:26.653 INFO:tasks.workunit.client.1.smithi070.stdout:1/669: fdatasync d3/d8/d13/fc4 0 2024-09-10T04:05:26.653 INFO:tasks.workunit.client.0.smithi046.stdout:1/508: stat d3/dc/d34/d51/f83 0 2024-09-10T04:05:26.654 INFO:tasks.workunit.client.1.smithi070.stdout:7/766: dwrite d2/d66/f78 [4194304,4194304] 0 2024-09-10T04:05:26.655 INFO:tasks.workunit.client.0.smithi046.stdout:8/649: write d1/d29/d19/fb1 [528032,40134] 0 2024-09-10T04:05:26.656 INFO:tasks.workunit.client.1.smithi070.stdout:7/767: read d2/d3/dc/d5d/d41/f50 [110270,99327] 0 2024-09-10T04:05:26.656 INFO:tasks.workunit.client.1.smithi070.stdout:5/736: creat d17/d6f/d32/ffc x:0 0 0 2024-09-10T04:05:26.657 INFO:tasks.workunit.client.0.smithi046.stdout:8/650: truncate d1/d29/d12/f97 172720 0 2024-09-10T04:05:26.658 INFO:tasks.workunit.client.1.smithi070.stdout:7/768: stat d2/d40/d6c/db2/db7/d104 0 2024-09-10T04:05:26.658 INFO:tasks.workunit.client.1.smithi070.stdout:5/737: read d17/d19/d35/f72 [3044756,87294] 0 2024-09-10T04:05:26.659 INFO:tasks.workunit.client.0.smithi046.stdout:4/818: creat d3/d12/d17/d29/d58/db2/f11e x:0 0 0 2024-09-10T04:05:26.661 INFO:tasks.workunit.client.1.smithi070.stdout:7/769: read d2/d40/d6c/db2/fe3 [235144,40972] 0 2024-09-10T04:05:26.665 INFO:tasks.workunit.client.0.smithi046.stdout:7/743: rename d5/d9/d12/d72/d77/de0/d1f/fe1 to d5/d9/d43/db4/def/f105 0 2024-09-10T04:05:26.667 INFO:tasks.workunit.client.0.smithi046.stdout:8/651: dwrite d1/d29/d19/fb1 [0,4194304] 0 2024-09-10T04:05:26.667 INFO:tasks.workunit.client.1.smithi070.stdout:7/770: dwrite d2/d3/d63/f87 [0,4194304] 0 2024-09-10T04:05:26.668 INFO:tasks.workunit.client.1.smithi070.stdout:8/801: link d4/d16/d73/fd0 d4/d39/dfc/d43/f115 0 2024-09-10T04:05:26.669 INFO:tasks.workunit.client.0.smithi046.stdout:8/652: chown d1/d29/d17/d1f/d2a/l37 824 1 2024-09-10T04:05:26.670 INFO:tasks.workunit.client.0.smithi046.stdout:3/657: symlink d12/d5d/d71/d49/d5c/ld8 0 2024-09-10T04:05:26.673 INFO:tasks.workunit.client.1.smithi070.stdout:7/771: dread d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:26.675 INFO:tasks.workunit.client.0.smithi046.stdout:6/601: rmdir da/db/d11/db4 0 2024-09-10T04:05:26.677 INFO:tasks.workunit.client.1.smithi070.stdout:1/670: rmdir d3/d8/d13/d67 39 2024-09-10T04:05:26.679 INFO:tasks.workunit.client.0.smithi046.stdout:1/509: creat d3/d27/d2a/fa3 x:0 0 0 2024-09-10T04:05:26.685 INFO:tasks.workunit.client.0.smithi046.stdout:2/654: unlink d2/d8/fb2 0 2024-09-10T04:05:26.687 INFO:tasks.workunit.client.1.smithi070.stdout:9/787: getdents d9/d1c/dac/d8a/dc4/dda/de0/d90/d89 0 2024-09-10T04:05:26.687 INFO:tasks.workunit.client.1.smithi070.stdout:0/885: write d8/d13/de2/d74/fa1 [649339,40253] 0 2024-09-10T04:05:26.688 INFO:tasks.workunit.client.1.smithi070.stdout:4/685: dwrite de/fe4 [0,4194304] 0 2024-09-10T04:05:26.690 INFO:tasks.workunit.client.1.smithi070.stdout:0/886: dread - d8/d13/de2/d6f/fec zero size 2024-09-10T04:05:26.692 INFO:tasks.workunit.client.1.smithi070.stdout:2/713: getdents d15/d2b/d6e 0 2024-09-10T04:05:26.695 INFO:tasks.workunit.client.1.smithi070.stdout:5/738: rename d17/d24/d77/db4/ff9 to d17/d19/d6a/d54/d57/da3/ffd 0 2024-09-10T04:05:26.696 INFO:tasks.workunit.client.0.smithi046.stdout:0/531: dwrite d1/dc/d25/d33/d3/dd/d55/f6d [0,4194304] 0 2024-09-10T04:05:26.699 INFO:tasks.workunit.client.0.smithi046.stdout:0/532: stat d1/dc/d25/d33/d3/dd/d55/d77/c92 0 2024-09-10T04:05:26.699 INFO:tasks.workunit.client.0.smithi046.stdout:0/533: fsync d1/dc/f17 0 2024-09-10T04:05:26.701 INFO:tasks.workunit.client.1.smithi070.stdout:3/762: dread db/d1c/d25/d28/f7d [0,4194304] 0 2024-09-10T04:05:26.705 INFO:tasks.workunit.client.0.smithi046.stdout:6/602: creat da/db/d11/d13/d1e/d23/d86/fc8 x:0 0 0 2024-09-10T04:05:26.706 INFO:tasks.workunit.client.0.smithi046.stdout:1/510: mkdir d3/d27/d5d/da4 0 2024-09-10T04:05:26.710 INFO:tasks.workunit.client.0.smithi046.stdout:5/826: rmdir d7/d11/d14/d8e/de2/d108/df5 0 2024-09-10T04:05:26.716 INFO:tasks.workunit.client.0.smithi046.stdout:7/744: symlink d5/d6/db/l106 0 2024-09-10T04:05:26.719 INFO:tasks.workunit.client.1.smithi070.stdout:5/739: rename d17/d37 to d17/d19/d6a/d54/de1/dfe 0 2024-09-10T04:05:26.720 INFO:tasks.workunit.client.0.smithi046.stdout:0/534: mkdir d1/d13/d37/d75/d94/d9a/da4 0 2024-09-10T04:05:26.720 INFO:tasks.workunit.client.1.smithi070.stdout:9/788: dread d9/d1c/dac/d8a/d3c/fa2 [0,4194304] 0 2024-09-10T04:05:26.721 INFO:tasks.workunit.client.1.smithi070.stdout:9/789: chown d9/d7a/d74/l46 2342 1 2024-09-10T04:05:26.723 INFO:tasks.workunit.client.0.smithi046.stdout:3/658: mkdir d12/dd9 0 2024-09-10T04:05:26.723 INFO:tasks.workunit.client.0.smithi046.stdout:7/745: dwrite d5/d6/d39/d5f/f96 [0,4194304] 0 2024-09-10T04:05:26.723 INFO:tasks.workunit.client.0.smithi046.stdout:4/819: write d3/dab/fbd [1235136,122605] 0 2024-09-10T04:05:26.724 INFO:tasks.workunit.client.1.smithi070.stdout:7/772: mknod d2/d7/c110 0 2024-09-10T04:05:26.726 INFO:tasks.workunit.client.0.smithi046.stdout:7/746: write d5/d6/dbe/dd7/fbb [227221,40363] 0 2024-09-10T04:05:26.727 INFO:tasks.workunit.client.1.smithi070.stdout:1/671: mkdir d3/d10/ddc 0 2024-09-10T04:05:26.731 INFO:tasks.workunit.client.1.smithi070.stdout:0/887: truncate d8/d13/d1a/f39 445195 0 2024-09-10T04:05:26.732 INFO:tasks.workunit.client.1.smithi070.stdout:2/714: symlink d15/d18/d56/d7c/dfa/l102 0 2024-09-10T04:05:26.735 INFO:tasks.workunit.client.1.smithi070.stdout:5/740: read d17/d24/d2b/f81 [80174,19851] 0 2024-09-10T04:05:26.736 INFO:tasks.workunit.client.1.smithi070.stdout:9/790: mkdir d9/d1c/dac/d8a/dfb 0 2024-09-10T04:05:26.738 INFO:tasks.workunit.client.1.smithi070.stdout:0/888: dwrite d8/d13/d47/d7b/fd8 [0,4194304] 0 2024-09-10T04:05:26.740 INFO:tasks.workunit.client.1.smithi070.stdout:7/773: creat d2/d40/d6c/db2/db7/d104/f111 x:0 0 0 2024-09-10T04:05:26.741 INFO:tasks.workunit.client.0.smithi046.stdout:3/659: mknod d12/d5d/d71/d49/d5c/d9f/cda 0 2024-09-10T04:05:26.743 INFO:tasks.workunit.client.1.smithi070.stdout:1/672: mknod d3/d8/d13/d3f/d5d/cdd 0 2024-09-10T04:05:26.743 INFO:tasks.workunit.client.0.smithi046.stdout:6/603: dwrite da/db/d11/d2b/f70 [0,4194304] 0 2024-09-10T04:05:26.744 INFO:tasks.workunit.client.0.smithi046.stdout:3/660: chown d12/d13/d14/d1e/d2d/d39/d67/d92 281560 1 2024-09-10T04:05:26.744 INFO:tasks.workunit.client.0.smithi046.stdout:9/647: getdents d4/d66/da6 0 2024-09-10T04:05:26.746 INFO:tasks.workunit.client.0.smithi046.stdout:9/648: write d4/d7/d1f/f21 [3786834,110339] 0 2024-09-10T04:05:26.747 INFO:tasks.workunit.client.0.smithi046.stdout:3/661: write d12/d5d/fcd [76140,81888] 0 2024-09-10T04:05:26.749 INFO:tasks.workunit.client.0.smithi046.stdout:9/649: chown d4/d29/d32/cbd 12 1 2024-09-10T04:05:26.751 INFO:tasks.workunit.client.0.smithi046.stdout:7/747: symlink d5/d6/l107 0 2024-09-10T04:05:26.752 INFO:tasks.workunit.client.0.smithi046.stdout:5/827: fdatasync d7/f43 0 2024-09-10T04:05:26.756 INFO:tasks.workunit.client.0.smithi046.stdout:8/653: getdents d1/d29/d19/d36/d6e 0 2024-09-10T04:05:26.756 INFO:tasks.workunit.client.1.smithi070.stdout:3/763: creat db/d1c/d25/d28/f106 x:0 0 0 2024-09-10T04:05:26.760 INFO:tasks.workunit.client.0.smithi046.stdout:0/535: dwrite d1/dc/d25/d33/d3/dd/d55/d4d/f4e [0,4194304] 0 2024-09-10T04:05:26.762 INFO:tasks.workunit.client.0.smithi046.stdout:0/536: write d1/dc/d25/d33/d3/dd/d55/f43 [913518,25716] 0 2024-09-10T04:05:26.764 INFO:tasks.workunit.client.1.smithi070.stdout:4/686: getdents de/d13/d18/d23/d47/d59/d79 0 2024-09-10T04:05:26.765 INFO:tasks.workunit.client.1.smithi070.stdout:4/687: dread - de/d13/d5b/d7b/fb9 zero size 2024-09-10T04:05:26.768 INFO:tasks.workunit.client.1.smithi070.stdout:5/741: rename c15 to d17/d19/d6a/cff 0 2024-09-10T04:05:26.769 INFO:tasks.workunit.client.0.smithi046.stdout:1/511: mknod d3/da1/d1f/ca5 0 2024-09-10T04:05:26.770 INFO:tasks.workunit.client.0.smithi046.stdout:4/820: dwrite d3/d12/d17/d29/dba/fd9 [0,4194304] 0 2024-09-10T04:05:26.773 INFO:tasks.workunit.client.0.smithi046.stdout:7/748: fsync d5/d9/f5b 0 2024-09-10T04:05:26.776 INFO:tasks.workunit.client.1.smithi070.stdout:7/774: write d2/d66/fa3 [322278,75279] 0 2024-09-10T04:05:26.777 INFO:tasks.workunit.client.1.smithi070.stdout:1/673: write d3/d8/fc8 [246797,73219] 0 2024-09-10T04:05:26.781 INFO:tasks.workunit.client.1.smithi070.stdout:9/791: dwrite d9/d7a/f44 [0,4194304] 0 2024-09-10T04:05:26.781 INFO:tasks.workunit.client.1.smithi070.stdout:4/688: creat de/d13/da6/ddb/fe8 x:0 0 0 2024-09-10T04:05:26.781 INFO:tasks.workunit.client.0.smithi046.stdout:8/654: mkdir d1/d29/d17/d1f/d46/d6a/d82/d99/dda 0 2024-09-10T04:05:26.782 INFO:tasks.workunit.client.1.smithi070.stdout:2/715: dread d15/d2b/d94/f9a [0,4194304] 0 2024-09-10T04:05:26.782 INFO:tasks.workunit.client.0.smithi046.stdout:0/537: rmdir d1/dc 39 2024-09-10T04:05:26.784 INFO:tasks.workunit.client.0.smithi046.stdout:9/650: dwrite d4/d3c/d4d/d5c/d87/faa [0,4194304] 0 2024-09-10T04:05:26.785 INFO:tasks.workunit.client.0.smithi046.stdout:9/651: fdatasync d4/d3c/f6b 0 2024-09-10T04:05:26.785 INFO:tasks.workunit.client.1.smithi070.stdout:9/792: dread - d9/d1c/dac/d8a/d3c/d92/ff6 zero size 2024-09-10T04:05:26.785 INFO:tasks.workunit.client.1.smithi070.stdout:3/764: mknod db/d79/c107 0 2024-09-10T04:05:26.787 INFO:tasks.workunit.client.1.smithi070.stdout:0/889: truncate d8/d13/d1a/f23 6226345 0 2024-09-10T04:05:26.788 INFO:tasks.workunit.client.0.smithi046.stdout:1/512: mkdir d3/dc/d34/d58/da6 0 2024-09-10T04:05:26.790 INFO:tasks.workunit.client.1.smithi070.stdout:1/674: symlink d3/d8/d13/d3f/d6d/d81/lde 0 2024-09-10T04:05:26.790 INFO:tasks.workunit.client.0.smithi046.stdout:4/821: creat d3/d12/d17/d29/d58/db2/d7a/f11f x:0 0 0 2024-09-10T04:05:26.791 INFO:tasks.workunit.client.1.smithi070.stdout:2/716: mknod d15/d18/d56/d7c/d55/c103 0 2024-09-10T04:05:26.791 INFO:tasks.workunit.client.1.smithi070.stdout:5/742: truncate d17/d24/f25 390974 0 2024-09-10T04:05:26.793 INFO:tasks.workunit.client.0.smithi046.stdout:7/749: mkdir d5/d6/d62/d108 0 2024-09-10T04:05:26.794 INFO:tasks.workunit.client.1.smithi070.stdout:3/765: truncate db/f3f 4534649 0 2024-09-10T04:05:26.794 INFO:tasks.workunit.client.0.smithi046.stdout:5/828: creat d7/d11/d3f/d53/d81/f121 x:0 0 0 2024-09-10T04:05:26.796 INFO:tasks.workunit.client.0.smithi046.stdout:3/662: mkdir d12/d13/d37/d51/db5/d50/d7a/dc1/ddb 0 2024-09-10T04:05:26.798 INFO:tasks.workunit.client.1.smithi070.stdout:4/689: dwrite de/d13/d5b/d7b/fe2 [0,4194304] 0 2024-09-10T04:05:26.798 INFO:tasks.workunit.client.1.smithi070.stdout:5/743: fsync d17/d24/d45/f7e 0 2024-09-10T04:05:26.800 INFO:tasks.workunit.client.1.smithi070.stdout:2/717: mknod d15/d2b/d6e/c104 0 2024-09-10T04:05:26.801 INFO:tasks.workunit.client.1.smithi070.stdout:9/793: dwrite d9/d1c/d34/d8b/dcc/f9d [8388608,4194304] 0 2024-09-10T04:05:26.802 INFO:tasks.workunit.client.1.smithi070.stdout:4/690: write de/d13/d43/f65 [2237507,61164] 0 2024-09-10T04:05:26.802 INFO:tasks.workunit.client.0.smithi046.stdout:8/655: creat d1/d29/d19/d36/d4c/d5e/fdb x:0 0 0 2024-09-10T04:05:26.803 INFO:tasks.workunit.client.1.smithi070.stdout:1/675: mknod d3/d90/cdf 0 2024-09-10T04:05:26.808 INFO:tasks.workunit.client.1.smithi070.stdout:0/890: symlink d8/d13/de2/d4f/d11b/l126 0 2024-09-10T04:05:26.815 INFO:tasks.workunit.client.1.smithi070.stdout:7/775: write d2/d7/d9/dd5/f108 [182281,85594] 0 2024-09-10T04:05:26.819 INFO:tasks.workunit.client.0.smithi046.stdout:7/750: fsync d5/d6/d55/d89/fa8 0 2024-09-10T04:05:26.822 INFO:tasks.workunit.client.1.smithi070.stdout:4/691: fdatasync de/d13/f38 0 2024-09-10T04:05:26.823 INFO:tasks.workunit.client.1.smithi070.stdout:4/692: fdatasync de/d13/d18/d26/d8c/f90 0 2024-09-10T04:05:26.823 INFO:tasks.workunit.client.0.smithi046.stdout:5/829: mkdir d7/d11/d14/d8e/d96/dce/d122 0 2024-09-10T04:05:26.824 INFO:tasks.workunit.client.1.smithi070.stdout:4/693: dread - de/d13/d18/d23/d39/fe7 zero size 2024-09-10T04:05:26.824 INFO:tasks.workunit.client.1.smithi070.stdout:9/794: rmdir d9/d59/d2a/d3f/daf 39 2024-09-10T04:05:26.825 INFO:tasks.workunit.client.0.smithi046.stdout:0/538: dread d1/dc/f17 [0,4194304] 0 2024-09-10T04:05:26.825 INFO:tasks.workunit.client.0.smithi046.stdout:6/604: getdents da/d45 0 2024-09-10T04:05:26.827 INFO:tasks.workunit.client.1.smithi070.stdout:0/891: mkdir d8/d13/de2/d53/dc6/d127 0 2024-09-10T04:05:26.830 INFO:tasks.workunit.client.0.smithi046.stdout:8/656: mkdir d1/d29/d19/d73/ddc 0 2024-09-10T04:05:26.831 INFO:tasks.workunit.client.1.smithi070.stdout:7/776: mkdir d2/d40/d6c/d112 0 2024-09-10T04:05:26.832 INFO:tasks.workunit.client.0.smithi046.stdout:9/652: mkdir d4/d7/d1f/d54/dc7/dc8 0 2024-09-10T04:05:26.833 INFO:tasks.workunit.client.0.smithi046.stdout:9/653: readlink d4/d3c/d4d/d80/la0 0 2024-09-10T04:05:26.834 INFO:tasks.workunit.client.1.smithi070.stdout:5/744: creat d17/d19/d6a/d54/de1/dfe/dce/f100 x:0 0 0 2024-09-10T04:05:26.834 INFO:tasks.workunit.client.0.smithi046.stdout:1/513: mkdir d3/da1/da7 0 2024-09-10T04:05:26.836 INFO:tasks.workunit.client.1.smithi070.stdout:5/745: read - d17/d19/d6a/d54/d57/da3/ffd zero size 2024-09-10T04:05:26.836 INFO:tasks.workunit.client.0.smithi046.stdout:4/822: mknod d3/d12/c120 0 2024-09-10T04:05:26.837 INFO:tasks.workunit.client.1.smithi070.stdout:7/777: dwrite d2/d7/ff4 [0,4194304] 0 2024-09-10T04:05:26.838 INFO:tasks.workunit.client.1.smithi070.stdout:4/694: truncate de/d13/d5b/d7b/dd5/f60 743521 0 2024-09-10T04:05:26.838 INFO:tasks.workunit.client.1.smithi070.stdout:5/746: fsync d17/d19/d6a/d54/d57/da3/f82 0 2024-09-10T04:05:26.838 INFO:tasks.workunit.client.1.smithi070.stdout:7/778: chown d2/d66 8156 1 2024-09-10T04:05:26.839 INFO:tasks.workunit.client.1.smithi070.stdout:9/795: symlink d9/d1c/dac/d8a/dc4/dda/de0/d90/df0/lfc 0 2024-09-10T04:05:26.840 INFO:tasks.workunit.client.0.smithi046.stdout:5/830: mkdir d7/d11/d14/d2f/deb/da5/d123 0 2024-09-10T04:05:26.841 INFO:tasks.workunit.client.0.smithi046.stdout:0/539: mknod d1/dc/d25/d33/d1a/d41/ca5 0 2024-09-10T04:05:26.841 INFO:tasks.workunit.client.1.smithi070.stdout:5/747: write d17/d19/d35/f3a [4101854,98998] 0 2024-09-10T04:05:26.841 INFO:tasks.workunit.client.1.smithi070.stdout:0/892: symlink d8/d13/de2/d4f/l128 0 2024-09-10T04:05:26.842 INFO:tasks.workunit.client.0.smithi046.stdout:6/605: mkdir da/db/d10/dc9 0 2024-09-10T04:05:26.842 INFO:tasks.workunit.client.0.smithi046.stdout:0/540: dread - d1/d13/d37/d75/d94/d9a/f90 zero size 2024-09-10T04:05:26.844 INFO:tasks.workunit.client.0.smithi046.stdout:3/663: mkdir d12/d13/d14/d1e/d2d/d39/d67/da0/ddc 0 2024-09-10T04:05:26.845 INFO:tasks.workunit.client.1.smithi070.stdout:4/695: dread de/d13/d18/d23/f52 [0,4194304] 0 2024-09-10T04:05:26.845 INFO:tasks.workunit.client.0.smithi046.stdout:3/664: chown d12/d13/d37/d94/dbf 202 1 2024-09-10T04:05:26.846 INFO:tasks.workunit.client.1.smithi070.stdout:4/696: write de/d13/d5b/d7b/dd5/d85/f9a [3794564,29819] 0 2024-09-10T04:05:26.848 INFO:tasks.workunit.client.0.smithi046.stdout:1/514: truncate d3/da1/d1f/f61 402827 0 2024-09-10T04:05:26.850 INFO:tasks.workunit.client.0.smithi046.stdout:4/823: creat d3/d12/d17/d29/f121 x:0 0 0 2024-09-10T04:05:26.854 INFO:tasks.workunit.client.0.smithi046.stdout:5/831: stat d7/d11/d3f/d5a/d63/fab 0 2024-09-10T04:05:26.855 INFO:tasks.workunit.client.1.smithi070.stdout:0/893: creat d8/d85/dca/f129 x:0 0 0 2024-09-10T04:05:26.859 INFO:tasks.workunit.client.0.smithi046.stdout:0/541: mkdir d1/dc/d25/d33/d3/dd/d55/d77/da6 0 2024-09-10T04:05:26.860 INFO:tasks.workunit.client.0.smithi046.stdout:4/824: dwrite d3/d12/d17/d29/d58/db2/d5f/d85/f115 [0,4194304] 0 2024-09-10T04:05:26.861 INFO:tasks.workunit.client.1.smithi070.stdout:5/748: symlink d17/d19/d6a/d54/d57/da8/l101 0 2024-09-10T04:05:26.862 INFO:tasks.workunit.client.0.smithi046.stdout:8/657: dread d1/fd [0,4194304] 0 2024-09-10T04:05:26.863 INFO:tasks.workunit.client.0.smithi046.stdout:8/658: chown d1/d29/d19/f34 0 1 2024-09-10T04:05:26.864 INFO:tasks.workunit.client.1.smithi070.stdout:4/697: rmdir de/d13/da6/ddb 39 2024-09-10T04:05:26.864 INFO:tasks.workunit.client.1.smithi070.stdout:4/698: stat de/d13/d18/d26/f2f 0 2024-09-10T04:05:26.865 INFO:tasks.workunit.client.0.smithi046.stdout:3/665: creat d12/d13/d37/fdd x:0 0 0 2024-09-10T04:05:26.867 INFO:tasks.workunit.client.0.smithi046.stdout:1/515: mknod d3/dc/d34/d58/ca8 0 2024-09-10T04:05:26.869 INFO:tasks.workunit.client.0.smithi046.stdout:5/832: rmdir d7/d11/d14/d8e/d96 39 2024-09-10T04:05:26.870 INFO:tasks.workunit.client.1.smithi070.stdout:0/894: rmdir d8/d13/de2/d6f 39 2024-09-10T04:05:26.873 INFO:tasks.workunit.client.1.smithi070.stdout:2/718: rename d15/d18/d29/f3d to d15/d2b/d6e/f105 0 2024-09-10T04:05:26.875 INFO:tasks.workunit.client.1.smithi070.stdout:5/749: creat d17/d19/d6a/d54/d57/da8/f102 x:0 0 0 2024-09-10T04:05:26.876 INFO:tasks.workunit.client.0.smithi046.stdout:9/654: creat d4/fc9 x:0 0 0 2024-09-10T04:05:26.877 INFO:tasks.workunit.client.0.smithi046.stdout:9/655: dread - d4/d3c/d4d/d80/fa4 zero size 2024-09-10T04:05:26.877 INFO:tasks.workunit.client.0.smithi046.stdout:9/656: chown d4/d7/l3f 19 1 2024-09-10T04:05:26.879 INFO:tasks.workunit.client.0.smithi046.stdout:4/825: rename d3/d12/d17/fc5 to d3/d12/dec/f122 0 2024-09-10T04:05:26.880 INFO:tasks.workunit.client.1.smithi070.stdout:4/699: mknod de/d13/d5b/da0/ce9 0 2024-09-10T04:05:26.880 INFO:tasks.workunit.client.0.smithi046.stdout:4/826: dread - d3/d12/d17/d29/d58/db2/f11e zero size 2024-09-10T04:05:26.882 INFO:tasks.workunit.client.0.smithi046.stdout:8/659: creat d1/d29/d17/d1f/d2a/fdd x:0 0 0 2024-09-10T04:05:26.883 INFO:tasks.workunit.client.1.smithi070.stdout:7/779: creat d2/d7/d9/f113 x:0 0 0 2024-09-10T04:05:26.884 INFO:tasks.workunit.client.1.smithi070.stdout:7/780: dread - d2/d7/d9/d18/d3c/d52/fb0 zero size 2024-09-10T04:05:26.884 INFO:tasks.workunit.client.0.smithi046.stdout:3/666: creat d12/d13/d37/dab/dac/dd3/fde x:0 0 0 2024-09-10T04:05:26.885 INFO:tasks.workunit.client.1.smithi070.stdout:0/895: write d8/d13/d1a/f3c [5388683,22126] 0 2024-09-10T04:05:26.889 INFO:tasks.workunit.client.0.smithi046.stdout:6/606: link da/db/d11/l47 da/db/d63/lca 0 2024-09-10T04:05:26.894 INFO:tasks.workunit.client.1.smithi070.stdout:4/700: unlink de/d13/d18/f21 0 2024-09-10T04:05:26.894 INFO:tasks.workunit.client.1.smithi070.stdout:4/701: chown de/d13/d18/d23/d39/d77 23111 1 2024-09-10T04:05:26.895 INFO:tasks.workunit.client.1.smithi070.stdout:0/896: dread d8/d13/d47/f5c [0,4194304] 0 2024-09-10T04:05:26.896 INFO:tasks.workunit.client.0.smithi046.stdout:9/657: creat d4/d18/d89/fca x:0 0 0 2024-09-10T04:05:26.896 INFO:tasks.workunit.client.0.smithi046.stdout:9/658: chown d4/d18/d89/fa9 1556649499 1 2024-09-10T04:05:26.896 INFO:tasks.workunit.client.0.smithi046.stdout:9/659: chown d4/d18/f27 16102 1 2024-09-10T04:05:26.899 INFO:tasks.workunit.client.0.smithi046.stdout:0/542: rename d1/l56 to d1/dc/d25/d33/d3/d29/d2d/la7 0 2024-09-10T04:05:26.901 INFO:tasks.workunit.client.1.smithi070.stdout:7/781: dwrite d2/d7/db/f37 [0,4194304] 0 2024-09-10T04:05:26.905 INFO:tasks.workunit.client.0.smithi046.stdout:1/516: mknod d3/dc/d34/d58/da6/ca9 0 2024-09-10T04:05:26.906 INFO:tasks.workunit.client.1.smithi070.stdout:2/719: write d15/d18/d29/d2a/d2e/d32/f33 [3932467,42814] 0 2024-09-10T04:05:26.907 INFO:tasks.workunit.client.0.smithi046.stdout:4/827: dwrite d3/d12/d17/d29/ddd/ff2 [0,4194304] 0 2024-09-10T04:05:26.909 INFO:tasks.workunit.client.1.smithi070.stdout:2/720: readlink d15/d18/d29/d2a/l3e 0 2024-09-10T04:05:26.909 INFO:tasks.workunit.client.0.smithi046.stdout:4/828: write d3/d12/d17/d22/d51/d5c/ff1 [1913637,39171] 0 2024-09-10T04:05:26.910 INFO:tasks.workunit.client.0.smithi046.stdout:9/660: creat d4/d66/da6/fcb x:0 0 0 2024-09-10T04:05:26.910 INFO:tasks.workunit.client.1.smithi070.stdout:7/782: dwrite d2/d3/dc/d5d/d2d/f8e [0,4194304] 0 2024-09-10T04:05:26.911 INFO:tasks.workunit.client.0.smithi046.stdout:9/661: fsync d4/f50 0 2024-09-10T04:05:26.911 INFO:tasks.workunit.client.1.smithi070.stdout:4/702: mkdir de/d13/d5b/d7b/dea 0 2024-09-10T04:05:26.912 INFO:tasks.workunit.client.1.smithi070.stdout:5/750: dwrite d17/d19/d6a/d54/de1/dfe/f91 [0,4194304] 0 2024-09-10T04:05:26.913 INFO:tasks.workunit.client.1.smithi070.stdout:5/751: chown d17/d24/d84/dab 43 1 2024-09-10T04:05:26.914 INFO:tasks.workunit.client.1.smithi070.stdout:0/897: creat d8/d13/d1a/d8f/d95/f12a x:0 0 0 2024-09-10T04:05:26.915 INFO:tasks.workunit.client.0.smithi046.stdout:3/667: mkdir d12/dd9/ddf 0 2024-09-10T04:05:26.918 INFO:tasks.workunit.client.0.smithi046.stdout:3/668: chown d12/d13/d37/d51/db5/d50/d7a/dc1/cce 440547764 1 2024-09-10T04:05:26.920 INFO:tasks.workunit.client.1.smithi070.stdout:9/796: rename d9/d1c/dac/d8a/dc4/dda/de0/d90/d89 to d9/d1c/dfd 0 2024-09-10T04:05:26.922 INFO:tasks.workunit.client.0.smithi046.stdout:4/829: dread d3/fa [0,4194304] 0 2024-09-10T04:05:26.923 INFO:tasks.workunit.client.1.smithi070.stdout:5/752: dwrite d17/d19/d6a/d54/d57/d63/fc6 [0,4194304] 0 2024-09-10T04:05:26.926 INFO:tasks.workunit.client.1.smithi070.stdout:5/753: dread - d17/d19/d6a/d54/de1/dfe/fcf zero size 2024-09-10T04:05:26.927 INFO:tasks.workunit.client.0.smithi046.stdout:9/662: mknod d4/d66/d7c/ccc 0 2024-09-10T04:05:26.928 INFO:tasks.workunit.client.0.smithi046.stdout:9/663: chown d4/d66/cc0 3140 1 2024-09-10T04:05:26.933 INFO:tasks.workunit.client.1.smithi070.stdout:9/797: mknod d9/d1c/dac/d8a/d3c/d92/da8/cfe 0 2024-09-10T04:05:26.935 INFO:tasks.workunit.client.1.smithi070.stdout:9/798: write d9/d1c/dac/d8a/dc4/de1/fd8 [724922,128204] 0 2024-09-10T04:05:26.935 INFO:tasks.workunit.client.0.smithi046.stdout:4/830: rmdir d3/d12 39 2024-09-10T04:05:26.939 INFO:tasks.workunit.client.1.smithi070.stdout:5/754: symlink d17/d24/d84/l103 0 2024-09-10T04:05:26.942 INFO:tasks.workunit.client.1.smithi070.stdout:0/898: truncate d8/d13/fe8 403831 0 2024-09-10T04:05:26.943 INFO:tasks.workunit.client.1.smithi070.stdout:0/899: chown d8/l15 7725449 1 2024-09-10T04:05:26.945 INFO:tasks.workunit.client.0.smithi046.stdout:1/517: dread d3/ff [0,4194304] 0 2024-09-10T04:05:26.948 INFO:tasks.workunit.client.1.smithi070.stdout:7/783: write d2/d40/d6c/f7f [7990842,10555] 0 2024-09-10T04:05:26.951 INFO:tasks.workunit.client.1.smithi070.stdout:2/721: write d15/d18/d29/d2a/d2e/d32/f73 [33537,85646] 0 2024-09-10T04:05:26.951 INFO:tasks.workunit.client.1.smithi070.stdout:0/900: creat d8/d24/d2d/d5f/f12b x:0 0 0 2024-09-10T04:05:26.954 INFO:tasks.workunit.client.1.smithi070.stdout:2/722: dread d15/d18/d56/d7c/fe7 [0,4194304] 0 2024-09-10T04:05:26.957 INFO:tasks.workunit.client.1.smithi070.stdout:6/754: sync 2024-09-10T04:05:26.958 INFO:tasks.workunit.client.1.smithi070.stdout:7/784: creat d2/dc5/f114 x:0 0 0 2024-09-10T04:05:26.958 INFO:tasks.workunit.client.1.smithi070.stdout:6/755: stat d0/d3/d18/d62/d70/f86 0 2024-09-10T04:05:26.959 INFO:tasks.workunit.client.1.smithi070.stdout:5/755: write d17/f20 [180650,82494] 0 2024-09-10T04:05:26.965 INFO:tasks.workunit.client.1.smithi070.stdout:6/756: dwrite d0/d3/d18/f4c [0,4194304] 0 2024-09-10T04:05:26.966 INFO:tasks.workunit.client.0.smithi046.stdout:8/660: rename d1/d29/d12/c4e to d1/d29/d17/d1f/cde 0 2024-09-10T04:05:26.967 INFO:tasks.workunit.client.1.smithi070.stdout:9/799: getdents d9/d1c/dfd 0 2024-09-10T04:05:26.967 INFO:tasks.workunit.client.0.smithi046.stdout:8/661: chown d1/d29/d12/f57 4 1 2024-09-10T04:05:26.969 INFO:tasks.workunit.client.0.smithi046.stdout:8/662: chown d1/d29/d17/d1f/f54 447 1 2024-09-10T04:05:26.969 INFO:tasks.workunit.client.0.smithi046.stdout:4/831: write d3/d12/d17/d29/d58/f75 [5148846,103099] 0 2024-09-10T04:05:26.970 INFO:tasks.workunit.client.0.smithi046.stdout:8/663: readlink d1/d29/d19/lae 0 2024-09-10T04:05:26.972 INFO:tasks.workunit.client.1.smithi070.stdout:7/785: mkdir d2/d7/d9/dd5/d115 0 2024-09-10T04:05:26.974 INFO:tasks.workunit.client.0.smithi046.stdout:0/543: getdents d1/dc/d25/d72 0 2024-09-10T04:05:26.976 INFO:tasks.workunit.client.0.smithi046.stdout:1/518: truncate d3/d27/d5d/d60/f9f 1052273 0 2024-09-10T04:05:26.978 INFO:tasks.workunit.client.0.smithi046.stdout:3/669: link d12/d13/d37/d51/f8e d12/d13/d37/d51/dbd/fe0 0 2024-09-10T04:05:26.978 INFO:tasks.workunit.client.1.smithi070.stdout:2/723: write d15/d18/f28 [5143181,78291] 0 2024-09-10T04:05:26.979 INFO:tasks.workunit.client.1.smithi070.stdout:0/901: dwrite d8/d13/f28 [0,4194304] 0 2024-09-10T04:05:26.979 INFO:tasks.workunit.client.0.smithi046.stdout:5/833: rename d7/d11/d3f/d5a/c8d to d7/d11/d14/d45/d51/c124 0 2024-09-10T04:05:26.982 INFO:tasks.workunit.client.1.smithi070.stdout:4/703: rename de/d13/d18/f4d to de/d13/d18/d23/d39/d48/feb 0 2024-09-10T04:05:26.982 INFO:tasks.workunit.client.0.smithi046.stdout:4/832: symlink d3/d12/d17/d91/l123 0 2024-09-10T04:05:26.984 INFO:tasks.workunit.client.0.smithi046.stdout:8/664: mknod d1/d29/d17/d1f/d46/cdf 0 2024-09-10T04:05:26.985 INFO:tasks.workunit.client.0.smithi046.stdout:8/665: chown d1/d29/d12/f9f 3024905 1 2024-09-10T04:05:26.987 INFO:tasks.workunit.client.1.smithi070.stdout:6/757: mkdir d0/d3/d18/d2a/df0 0 2024-09-10T04:05:26.988 INFO:tasks.workunit.client.1.smithi070.stdout:6/758: chown d0/d3/d18/d1b/d17/d54/dc4/dd2 88741 1 2024-09-10T04:05:26.988 INFO:tasks.workunit.client.0.smithi046.stdout:8/666: dread d1/d29/d17/d1f/d46/d6a/d82/d99/fb3 [0,4194304] 0 2024-09-10T04:05:26.990 INFO:tasks.workunit.client.1.smithi070.stdout:7/786: mknod d2/d7/d9/d18/d3c/d52/d103/c116 0 2024-09-10T04:05:26.994 INFO:tasks.workunit.client.0.smithi046.stdout:3/670: unlink d12/d13/l48 0 2024-09-10T04:05:26.999 INFO:tasks.workunit.client.0.smithi046.stdout:1/519: dwrite d3/d9b/f9e [0,4194304] 0 2024-09-10T04:05:26.999 INFO:tasks.workunit.client.1.smithi070.stdout:2/724: mkdir d15/d2b/d106 0 2024-09-10T04:05:27.007 INFO:tasks.workunit.client.0.smithi046.stdout:0/544: creat d1/dc/d25/d33/d3/dd/d55/d77/da6/fa8 x:0 0 0 2024-09-10T04:05:27.008 INFO:tasks.workunit.client.0.smithi046.stdout:1/520: dwrite d3/dc/f84 [0,4194304] 0 2024-09-10T04:05:27.009 INFO:tasks.workunit.client.0.smithi046.stdout:9/664: truncate d4/d3c/d4d/d5c/d87/faa 1153880 0 2024-09-10T04:05:27.010 INFO:tasks.workunit.client.0.smithi046.stdout:8/667: dread d1/d29/d17/f9c [0,4194304] 0 2024-09-10T04:05:27.010 INFO:tasks.workunit.client.1.smithi070.stdout:9/800: symlink d9/d1c/dac/d8a/dc4/dda/de0/d90/df0/df3/df4/lff 0 2024-09-10T04:05:27.016 INFO:tasks.workunit.client.1.smithi070.stdout:5/756: truncate d17/d19/d6a/d54/de1/dfe/d93/fdf 430883 0 2024-09-10T04:05:27.017 INFO:tasks.workunit.client.1.smithi070.stdout:7/787: symlink d2/d40/d6c/db2/l117 0 2024-09-10T04:05:27.018 INFO:tasks.workunit.client.1.smithi070.stdout:0/902: dread d8/f27 [0,4194304] 0 2024-09-10T04:05:27.019 INFO:tasks.workunit.client.1.smithi070.stdout:2/725: mknod d15/d18/d56/d7c/d55/d6d/c107 0 2024-09-10T04:05:27.022 INFO:tasks.workunit.client.0.smithi046.stdout:3/671: rename d12/d13/d37/d51/da7/fc5 to d12/d13/d14/d1e/d2d/fe1 0 2024-09-10T04:05:27.024 INFO:tasks.workunit.client.0.smithi046.stdout:5/834: dwrite d7/d11/d14/d2f/d64/fc6 [0,4194304] 0 2024-09-10T04:05:27.025 INFO:tasks.workunit.client.1.smithi070.stdout:0/903: dwrite d8/d13/d1a/d8f/d95/fee [0,4194304] 0 2024-09-10T04:05:27.025 INFO:tasks.workunit.client.1.smithi070.stdout:9/801: symlink d9/d1c/dac/d8a/d3c/l100 0 2024-09-10T04:05:27.028 INFO:tasks.workunit.client.1.smithi070.stdout:1/676: sync 2024-09-10T04:05:27.028 INFO:tasks.workunit.client.1.smithi070.stdout:3/766: sync 2024-09-10T04:05:27.028 INFO:tasks.workunit.client.1.smithi070.stdout:8/802: sync 2024-09-10T04:05:27.029 INFO:tasks.workunit.client.1.smithi070.stdout:8/803: chown d4/c89 15 1 2024-09-10T04:05:27.030 INFO:tasks.workunit.client.0.smithi046.stdout:4/833: dwrite d3/d12/d17/fe0 [0,4194304] 0 2024-09-10T04:05:27.032 INFO:tasks.workunit.client.0.smithi046.stdout:1/521: truncate d3/dc/f88 3788066 0 2024-09-10T04:05:27.038 INFO:tasks.workunit.client.1.smithi070.stdout:1/677: dwrite d3/d10/d1d/dbd/da2/fa6 [0,4194304] 0 2024-09-10T04:05:27.041 INFO:tasks.workunit.client.1.smithi070.stdout:7/788: truncate d2/d7/d9/d18/d3c/fae 4293923 0 2024-09-10T04:05:27.042 INFO:tasks.workunit.client.1.smithi070.stdout:4/704: dread - de/d13/d18/d26/fa4 zero size 2024-09-10T04:05:27.048 INFO:tasks.workunit.client.0.smithi046.stdout:0/545: dread d1/dc/d25/d33/d1a/f31 [0,4194304] 0 2024-09-10T04:05:27.048 INFO:tasks.workunit.client.1.smithi070.stdout:9/802: creat d9/d59/d2a/f101 x:0 0 0 2024-09-10T04:05:27.055 INFO:tasks.workunit.client.0.smithi046.stdout:9/665: rename d4/d7/l12 to d4/d18/d92/lcd 0 2024-09-10T04:05:27.060 INFO:tasks.workunit.client.1.smithi070.stdout:3/767: dread db/d1c/d25/d28/fba [0,4194304] 0 2024-09-10T04:05:27.064 INFO:tasks.workunit.client.1.smithi070.stdout:2/726: mknod d15/d18/d29/d2a/c108 0 2024-09-10T04:05:27.064 INFO:tasks.workunit.client.0.smithi046.stdout:8/668: getdents d1/d29/d19/d36/d4c/d5e/dc7 0 2024-09-10T04:05:27.065 INFO:tasks.workunit.client.1.smithi070.stdout:2/727: truncate d15/d18/db2/fd6 329 0 2024-09-10T04:05:27.067 INFO:tasks.workunit.client.1.smithi070.stdout:7/789: unlink d2/d7/d9/d18/d23/c26 0 2024-09-10T04:05:27.067 INFO:tasks.workunit.client.1.smithi070.stdout:7/790: stat d2/d7/d9/d18/d3c/d52/d103 0 2024-09-10T04:05:27.068 INFO:tasks.workunit.client.0.smithi046.stdout:0/546: creat d1/d13/fa9 x:0 0 0 2024-09-10T04:05:27.069 INFO:tasks.workunit.client.0.smithi046.stdout:3/672: dread - d12/d13/d37/d51/db5/d50/d7a/fa4 zero size 2024-09-10T04:05:27.070 INFO:tasks.workunit.client.1.smithi070.stdout:9/803: creat d9/d1c/dac/d8a/dc4/dda/de0/d90/df0/df3/df4/f102 x:0 0 0 2024-09-10T04:05:27.071 INFO:tasks.workunit.client.1.smithi070.stdout:6/759: link d0/d3/d18/d62/d70/f86 d0/d3/d8c/ff1 0 2024-09-10T04:05:27.072 INFO:tasks.workunit.client.0.smithi046.stdout:1/522: creat d3/d62/faa x:0 0 0 2024-09-10T04:05:27.074 INFO:tasks.workunit.client.0.smithi046.stdout:8/669: mkdir d1/d29/d17/d1f/d46/d67/de0 0 2024-09-10T04:05:27.074 INFO:tasks.workunit.client.1.smithi070.stdout:7/791: dwrite d2/d40/d6c/db2/db7/d104/f109 [0,4194304] 0 2024-09-10T04:05:27.075 INFO:tasks.workunit.client.0.smithi046.stdout:8/670: fdatasync d1/d29/d19/d36/f4f 0 2024-09-10T04:05:27.076 INFO:tasks.workunit.client.1.smithi070.stdout:5/757: creat d17/d19/d6a/d54/d57/d63/f104 x:0 0 0 2024-09-10T04:05:27.078 INFO:tasks.workunit.client.1.smithi070.stdout:5/758: chown d17/d24/d84/l103 914012211 1 2024-09-10T04:05:27.079 INFO:tasks.workunit.client.1.smithi070.stdout:5/759: chown d17/d19/f44 5399 1 2024-09-10T04:05:27.080 INFO:tasks.workunit.client.0.smithi046.stdout:3/673: unlink d12/d5d/l68 0 2024-09-10T04:05:27.080 INFO:tasks.workunit.client.1.smithi070.stdout:6/760: dwrite d0/d3/d18/d1b/d17/fa5 [4194304,4194304] 0 2024-09-10T04:05:27.080 INFO:tasks.workunit.client.1.smithi070.stdout:5/760: stat d17/d24/d77/c8b 0 2024-09-10T04:05:27.081 INFO:tasks.workunit.client.1.smithi070.stdout:5/761: fdatasync d17/ff4 0 2024-09-10T04:05:27.081 INFO:tasks.workunit.client.0.smithi046.stdout:3/674: write f3 [4075488,45750] 0 2024-09-10T04:05:27.083 INFO:tasks.workunit.client.1.smithi070.stdout:1/678: rename d3/ca4 to d3/d8/d27/ce0 0 2024-09-10T04:05:27.087 INFO:tasks.workunit.client.1.smithi070.stdout:2/728: read d15/d18/d29/d2a/d2e/f87 [414996,8482] 0 2024-09-10T04:05:27.088 INFO:tasks.workunit.client.0.smithi046.stdout:4/834: rmdir d3/d12/d17/d22/d51/d5c/d100 0 2024-09-10T04:05:27.089 INFO:tasks.workunit.client.1.smithi070.stdout:5/762: dwrite d17/d24/d77/db4/ff3 [0,4194304] 0 2024-09-10T04:05:27.091 INFO:tasks.workunit.client.1.smithi070.stdout:5/763: chown d17/d19/d6a/d54/l71 53952815 1 2024-09-10T04:05:27.092 INFO:tasks.workunit.client.1.smithi070.stdout:0/904: creat d8/f12c x:0 0 0 2024-09-10T04:05:27.093 INFO:tasks.workunit.client.0.smithi046.stdout:8/671: creat d1/d29/d19/db7/fe1 x:0 0 0 2024-09-10T04:05:27.096 INFO:tasks.workunit.client.1.smithi070.stdout:8/804: dwrite d4/f29 [0,4194304] 0 2024-09-10T04:05:27.097 INFO:tasks.workunit.client.1.smithi070.stdout:9/804: write d9/d1c/dac/d8a/dc4/de1/fc3 [179234,71253] 0 2024-09-10T04:05:27.098 INFO:tasks.workunit.client.0.smithi046.stdout:9/666: rename d4/d7/d81/db7 to d4/d7/d1f/dce 0 2024-09-10T04:05:27.102 INFO:tasks.workunit.client.1.smithi070.stdout:7/792: mknod d2/d7/d9/d18/d3c/d52/c118 0 2024-09-10T04:05:27.104 INFO:tasks.workunit.client.0.smithi046.stdout:5/835: getdents d7/d11/d3f/d53/d99 0 2024-09-10T04:05:27.106 INFO:tasks.workunit.client.1.smithi070.stdout:9/805: dwrite d9/d1c/dac/d8a/d3c/d92/da8/fe9 [0,4194304] 0 2024-09-10T04:05:27.107 INFO:tasks.workunit.client.1.smithi070.stdout:9/806: stat d9/l16 0 2024-09-10T04:05:27.107 INFO:tasks.workunit.client.1.smithi070.stdout:6/761: rmdir d0/d3/d18/d1b/d17/d3f/d40 39 2024-09-10T04:05:27.111 INFO:tasks.workunit.client.1.smithi070.stdout:3/768: rename db/d83/d92/fee to db/d79/dd4/f108 0 2024-09-10T04:05:27.113 INFO:tasks.workunit.client.1.smithi070.stdout:3/769: chown db/d1c/d25/fe6 452 1 2024-09-10T04:05:27.118 INFO:tasks.workunit.client.0.smithi046.stdout:3/675: creat d12/dd9/ddf/fe2 x:0 0 0 2024-09-10T04:05:27.119 INFO:tasks.workunit.client.0.smithi046.stdout:0/547: dwrite d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:27.121 INFO:tasks.workunit.client.1.smithi070.stdout:4/705: creat de/d13/fec x:0 0 0 2024-09-10T04:05:27.122 INFO:tasks.workunit.client.0.smithi046.stdout:3/676: rename d12/d13/d37/d51 to d12/d13/d37/d51/db5/d50/d7a/dc1/d96/de3 22 2024-09-10T04:05:27.123 INFO:tasks.workunit.client.1.smithi070.stdout:5/764: mknod d17/d19/d6a/c105 0 2024-09-10T04:05:27.124 INFO:tasks.workunit.client.1.smithi070.stdout:0/905: dread - d8/d13/d1a/f45 zero size 2024-09-10T04:05:27.125 INFO:tasks.workunit.client.0.smithi046.stdout:8/672: link d1/d29/d19/d36/fb6 d1/d29/d17/d1f/d46/d6a/d82/dc2/fe2 0 2024-09-10T04:05:27.129 INFO:tasks.workunit.client.0.smithi046.stdout:5/836: symlink d7/d11/d14/l125 0 2024-09-10T04:05:27.131 INFO:tasks.workunit.client.0.smithi046.stdout:1/523: getdents d3/d27/d2a 0 2024-09-10T04:05:27.132 INFO:tasks.workunit.client.1.smithi070.stdout:6/762: truncate d0/d3/d18/d1b/d17/d53/fb4 148923 0 2024-09-10T04:05:27.134 INFO:tasks.workunit.client.0.smithi046.stdout:8/673: rmdir d1/d29/d19/d36/d4c 39 2024-09-10T04:05:27.135 INFO:tasks.workunit.client.1.smithi070.stdout:4/706: dread de/d13/d18/d26/f2c [0,4194304] 0 2024-09-10T04:05:27.137 INFO:tasks.workunit.client.0.smithi046.stdout:5/837: dwrite d7/d11/d3f/d53/f47 [0,4194304] 0 2024-09-10T04:05:27.137 INFO:tasks.workunit.client.0.smithi046.stdout:9/667: write d4/d3c/d4d/d5c/d87/faa [997124,34076] 0 2024-09-10T04:05:27.139 INFO:tasks.workunit.client.1.smithi070.stdout:2/729: mknod d15/c109 0 2024-09-10T04:05:27.141 INFO:tasks.workunit.client.1.smithi070.stdout:0/906: symlink d8/d85/dca/l12d 0 2024-09-10T04:05:27.142 INFO:tasks.workunit.client.1.smithi070.stdout:4/707: dwrite de/d13/fec [0,4194304] 0 2024-09-10T04:05:27.143 INFO:tasks.workunit.client.1.smithi070.stdout:0/907: chown d8/d13/d47/d7b/fd8 0 1 2024-09-10T04:05:27.143 INFO:tasks.workunit.client.1.smithi070.stdout:0/908: dread - d8/f99 zero size 2024-09-10T04:05:27.144 INFO:tasks.workunit.client.0.smithi046.stdout:8/674: dwrite d1/f49 [0,4194304] 0 2024-09-10T04:05:27.146 INFO:tasks.workunit.client.0.smithi046.stdout:3/677: fsync d12/d13/d14/d1e/d2d/f4f 0 2024-09-10T04:05:27.150 INFO:tasks.workunit.client.0.smithi046.stdout:1/524: dread d3/d9b/f8d [0,4194304] 0 2024-09-10T04:05:27.150 INFO:tasks.workunit.client.0.smithi046.stdout:1/525: chown d3/dc/l96 3989090 1 2024-09-10T04:05:27.151 INFO:tasks.workunit.client.0.smithi046.stdout:1/526: chown d3/d27/d2a 12 1 2024-09-10T04:05:27.152 INFO:tasks.workunit.client.0.smithi046.stdout:4/835: getdents d3/d12/d17/d29/ddd 0 2024-09-10T04:05:27.154 INFO:tasks.workunit.client.0.smithi046.stdout:0/548: truncate d1/dc/f76 3098533 0 2024-09-10T04:05:27.159 INFO:tasks.workunit.client.0.smithi046.stdout:5/838: creat d7/d11/d14/d2f/f126 x:0 0 0 2024-09-10T04:05:27.162 INFO:tasks.workunit.client.1.smithi070.stdout:8/805: dwrite d4/d39/dc6/f10b [0,4194304] 0 2024-09-10T04:05:27.163 INFO:tasks.workunit.client.1.smithi070.stdout:7/793: truncate d2/d3/dc/d5d/d2d/f8e 3477285 0 2024-09-10T04:05:27.169 INFO:tasks.workunit.client.1.smithi070.stdout:5/765: link d17/d19/d6a/fd2 d17/d19/d6a/f106 0 2024-09-10T04:05:27.170 INFO:tasks.workunit.client.0.smithi046.stdout:3/678: unlink d12/d13/d37/d51/db5/d50/d7a/dc1/cce 0 2024-09-10T04:05:27.172 INFO:tasks.workunit.client.0.smithi046.stdout:3/679: read d12/f5f [1792836,26914] 0 2024-09-10T04:05:27.175 INFO:tasks.workunit.client.0.smithi046.stdout:4/836: creat d3/d12/d17/d29/d58/db2/d7a/f124 x:0 0 0 2024-09-10T04:05:27.177 INFO:tasks.workunit.client.0.smithi046.stdout:0/549: symlink d1/dc/d25/d33/d4/laa 0 2024-09-10T04:05:27.177 INFO:tasks.workunit.client.1.smithi070.stdout:9/807: link d9/d1c/l4f d9/d1c/dac/d8a/dfb/l103 0 2024-09-10T04:05:27.178 INFO:tasks.workunit.client.0.smithi046.stdout:0/550: read d1/dc/d25/d33/f3f [1756387,32508] 0 2024-09-10T04:05:27.180 INFO:tasks.workunit.client.1.smithi070.stdout:1/679: getdents d3/d10/db2/dc7/d54 0 2024-09-10T04:05:27.180 INFO:tasks.workunit.client.0.smithi046.stdout:0/551: write d1/dc/d25/d33/d4/d19/f58 [5203622,81045] 0 2024-09-10T04:05:27.180 INFO:tasks.workunit.client.0.smithi046.stdout:8/675: write d1/d29/d19/d73/fba [642323,99836] 0 2024-09-10T04:05:27.181 INFO:tasks.workunit.client.0.smithi046.stdout:5/839: dread d7/d11/d3f/d53/d81/fd0 [0,4194304] 0 2024-09-10T04:05:27.182 INFO:tasks.workunit.client.0.smithi046.stdout:5/840: fdatasync d7/d11/f6e 0 2024-09-10T04:05:27.182 INFO:tasks.workunit.client.1.smithi070.stdout:3/770: creat db/d1c/d25/d28/f109 x:0 0 0 2024-09-10T04:05:27.183 INFO:tasks.workunit.client.0.smithi046.stdout:0/552: chown d1/dc/d25/d33/d3/d29/f38 15070357 1 2024-09-10T04:05:27.184 INFO:tasks.workunit.client.1.smithi070.stdout:1/680: dread d3/d10/d1d/fb9 [0,4194304] 0 2024-09-10T04:05:27.185 INFO:tasks.workunit.client.0.smithi046.stdout:0/553: chown d1/dc/d25/d33/d4/d19/l5f 85 1 2024-09-10T04:05:27.185 INFO:tasks.workunit.client.1.smithi070.stdout:1/681: read - d3/d8/d12/d61/fc0 zero size 2024-09-10T04:05:27.187 INFO:tasks.workunit.client.1.smithi070.stdout:2/730: mkdir d15/d18/d29/d2a/d10a 0 2024-09-10T04:05:27.188 INFO:tasks.workunit.client.1.smithi070.stdout:8/806: unlink d4/d39/dfc/d43/d67/ld5 0 2024-09-10T04:05:27.189 INFO:tasks.workunit.client.0.smithi046.stdout:3/680: creat d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fe4 x:0 0 0 2024-09-10T04:05:27.193 INFO:tasks.workunit.client.0.smithi046.stdout:4/837: rmdir d3/dab 39 2024-09-10T04:05:27.196 INFO:tasks.workunit.client.1.smithi070.stdout:7/794: rename d2/d7/db/d3a to d2/dc8/d119 0 2024-09-10T04:05:27.196 INFO:tasks.workunit.client.1.smithi070.stdout:5/766: creat d17/d19/d6a/d54/de1/dfe/d93/f107 x:0 0 0 2024-09-10T04:05:27.199 INFO:tasks.workunit.client.1.smithi070.stdout:6/763: creat d0/d3/d8c/ff2 x:0 0 0 2024-09-10T04:05:27.200 INFO:tasks.workunit.client.1.smithi070.stdout:9/808: symlink d9/d1c/dac/d8a/l104 0 2024-09-10T04:05:27.203 INFO:tasks.workunit.client.1.smithi070.stdout:5/767: dwrite d17/d19/d6a/d54/d57/da8/fe2 [0,4194304] 0 2024-09-10T04:05:27.206 INFO:tasks.workunit.client.0.smithi046.stdout:5/841: fsync d7/f43 0 2024-09-10T04:05:27.208 INFO:tasks.workunit.client.1.smithi070.stdout:1/682: symlink d3/d10/db2/dc7/le1 0 2024-09-10T04:05:27.209 INFO:tasks.workunit.client.1.smithi070.stdout:1/683: chown d3/d8/fc8 106502699 1 2024-09-10T04:05:27.210 INFO:tasks.workunit.client.1.smithi070.stdout:0/909: truncate d8/fc 1516741 0 2024-09-10T04:05:27.213 INFO:tasks.workunit.client.0.smithi046.stdout:0/554: symlink d1/dc/d25/d72/lab 0 2024-09-10T04:05:27.215 INFO:tasks.workunit.client.1.smithi070.stdout:8/807: dread - d4/dbf/d87/fc0 zero size 2024-09-10T04:05:27.220 INFO:tasks.workunit.client.0.smithi046.stdout:4/838: symlink d3/d12/d17/d29/d58/db2/d5f/d85/l125 0 2024-09-10T04:05:27.222 INFO:tasks.workunit.client.0.smithi046.stdout:9/668: getdents d4/d18/d92 0 2024-09-10T04:05:27.222 INFO:tasks.workunit.client.0.smithi046.stdout:4/839: chown d3/d12/d14/f62 1226 1 2024-09-10T04:05:27.222 INFO:tasks.workunit.client.0.smithi046.stdout:8/676: dwrite d1/d29/d19/f56 [0,4194304] 0 2024-09-10T04:05:27.225 INFO:tasks.workunit.client.0.smithi046.stdout:8/677: readlink d1/d29/d17/d1f/d46/d6a/d82/dc2/lce 0 2024-09-10T04:05:27.226 INFO:tasks.workunit.client.0.smithi046.stdout:5/842: dread d7/d11/d14/f95 [0,4194304] 0 2024-09-10T04:05:27.226 INFO:tasks.workunit.client.0.smithi046.stdout:0/555: creat d1/d13/d37/d75/d94/d9a/fac x:0 0 0 2024-09-10T04:05:27.229 INFO:tasks.workunit.client.0.smithi046.stdout:1/527: rename d3/da1/d1f/c47 to d3/d27/cab 0 2024-09-10T04:05:27.229 INFO:tasks.workunit.client.0.smithi046.stdout:3/681: mknod d12/d13/ce5 0 2024-09-10T04:05:27.231 INFO:tasks.workunit.client.1.smithi070.stdout:1/684: dread d3/d8/f2d [4194304,4194304] 0 2024-09-10T04:05:27.232 INFO:tasks.workunit.client.0.smithi046.stdout:9/669: unlink d4/d7/d81/cb1 0 2024-09-10T04:05:27.236 INFO:tasks.workunit.client.0.smithi046.stdout:4/840: write d3/d12/d17/d29/d58/db2/fc4 [879448,16308] 0 2024-09-10T04:05:27.240 INFO:tasks.workunit.client.0.smithi046.stdout:5/843: write d7/d11/d3f/ffe [261828,106850] 0 2024-09-10T04:05:27.242 INFO:tasks.workunit.client.0.smithi046.stdout:5/844: stat d7/d11/d3f/d53/d75/f9f 0 2024-09-10T04:05:27.242 INFO:tasks.workunit.client.0.smithi046.stdout:8/678: creat d1/d29/d19/d36/d6e/d75/fe3 x:0 0 0 2024-09-10T04:05:27.243 INFO:tasks.workunit.client.0.smithi046.stdout:5/845: fsync d7/d11/d3f/d53/da9/f11b 0 2024-09-10T04:05:27.245 INFO:tasks.workunit.client.0.smithi046.stdout:5/846: write d7/d11/d3f/d53/f47 [609852,22497] 0 2024-09-10T04:05:27.245 INFO:tasks.workunit.client.0.smithi046.stdout:0/556: mknod d1/dc/d25/d33/d4/cad 0 2024-09-10T04:05:27.247 INFO:tasks.workunit.client.0.smithi046.stdout:8/679: dread d1/f79 [0,4194304] 0 2024-09-10T04:05:27.248 INFO:tasks.workunit.client.0.smithi046.stdout:1/528: dread d3/d27/f20 [0,4194304] 0 2024-09-10T04:05:27.249 INFO:tasks.workunit.client.0.smithi046.stdout:0/557: read d1/dc/d25/d33/d1a/d41/f84 [362151,28937] 0 2024-09-10T04:05:27.250 INFO:tasks.workunit.client.0.smithi046.stdout:9/670: rename d4/d66/da6/fcb to d4/d18/da8/fcf 0 2024-09-10T04:05:27.252 INFO:tasks.workunit.client.0.smithi046.stdout:9/671: dread d4/d18/d89/fa7 [0,4194304] 0 2024-09-10T04:05:27.256 INFO:tasks.workunit.client.0.smithi046.stdout:5/847: creat d7/d11/d14/d8e/de2/d108/f127 x:0 0 0 2024-09-10T04:05:27.259 INFO:tasks.workunit.client.0.smithi046.stdout:3/682: creat d12/d13/d14/d1e/dd2/fe6 x:0 0 0 2024-09-10T04:05:27.262 INFO:tasks.workunit.client.0.smithi046.stdout:8/680: mkdir d1/d29/d17/d1f/d46/d67/de4 0 2024-09-10T04:05:27.265 INFO:tasks.workunit.client.0.smithi046.stdout:0/558: creat d1/dc/d25/d33/d4/fae x:0 0 0 2024-09-10T04:05:27.266 INFO:tasks.workunit.client.0.smithi046.stdout:0/559: read - d1/dc/d25/d33/d3/dd/d55/d77/da6/fa8 zero size 2024-09-10T04:05:27.267 INFO:tasks.workunit.client.0.smithi046.stdout:4/841: rename d3/d12/d17/d29/d58/db2/d5f/d85/da8/lf4 to d3/d12/d17/d22/d51/d5c/d11b/l126 0 2024-09-10T04:05:27.269 INFO:tasks.workunit.client.0.smithi046.stdout:0/560: dread - d1/dc/d25/d33/d4/fae zero size 2024-09-10T04:05:27.273 INFO:tasks.workunit.client.0.smithi046.stdout:3/683: mknod d12/d13/d14/d1e/d2d/d6f/ce7 0 2024-09-10T04:05:27.274 INFO:tasks.workunit.client.0.smithi046.stdout:8/681: unlink d1/d29/d17/c42 0 2024-09-10T04:05:27.276 INFO:tasks.workunit.client.0.smithi046.stdout:9/672: rename d4/d3c/d4d/d5c/db8 to d4/d7/d61/dd0 0 2024-09-10T04:05:27.277 INFO:tasks.workunit.client.0.smithi046.stdout:9/673: write d4/fc9 [349102,59882] 0 2024-09-10T04:05:27.281 INFO:tasks.workunit.client.0.smithi046.stdout:5/848: link d7/d11/d14/f25 d7/d11/d14/d2f/da3/f128 0 2024-09-10T04:05:27.285 INFO:tasks.workunit.client.0.smithi046.stdout:0/561: fsync d1/dc/d49/f4a 0 2024-09-10T04:05:27.287 INFO:tasks.workunit.client.0.smithi046.stdout:9/674: symlink d4/d3c/d4d/d5c/d59/ld1 0 2024-09-10T04:05:27.293 INFO:tasks.workunit.client.0.smithi046.stdout:8/682: creat d1/d29/d17/d1f/d46/d67/de0/fe5 x:0 0 0 2024-09-10T04:05:27.293 INFO:tasks.workunit.client.0.smithi046.stdout:2/655: sync 2024-09-10T04:05:27.295 INFO:tasks.workunit.client.0.smithi046.stdout:5/849: dwrite d7/d11/d14/d2f/d2b/f3a [0,4194304] 0 2024-09-10T04:05:27.301 INFO:tasks.workunit.client.0.smithi046.stdout:8/683: dwrite d1/d29/d17/d1f/d46/fa8 [0,4194304] 0 2024-09-10T04:05:27.303 INFO:tasks.workunit.client.0.smithi046.stdout:1/529: link d3/f4d d3/d27/fac 0 2024-09-10T04:05:27.303 INFO:tasks.workunit.client.0.smithi046.stdout:8/684: write d1/d29/d17/d1f/d2a/fdd [271218,53331] 0 2024-09-10T04:05:27.305 INFO:tasks.workunit.client.0.smithi046.stdout:3/684: link d12/d13/ce5 d12/d13/d14/d1e/d2d/ce8 0 2024-09-10T04:05:27.313 INFO:tasks.workunit.client.1.smithi070.stdout:4/708: creat de/fed x:0 0 0 2024-09-10T04:05:27.314 INFO:tasks.workunit.client.0.smithi046.stdout:4/842: getdents d3/d12/d17/d22/d51/d94 0 2024-09-10T04:05:27.317 INFO:tasks.workunit.client.0.smithi046.stdout:0/562: link d1/d13/d37/f6a d1/d13/d37/d75/d94/d9a/da4/faf 0 2024-09-10T04:05:27.322 INFO:tasks.workunit.client.0.smithi046.stdout:4/843: dwrite d3/d12/d17/d29/d58/db2/d7a/f11f [0,4194304] 0 2024-09-10T04:05:27.322 INFO:tasks.workunit.client.1.smithi070.stdout:9/809: chown d9/d59/d2a/d3f/daf/ce8 129774420 1 2024-09-10T04:05:27.324 INFO:tasks.workunit.client.0.smithi046.stdout:4/844: write d3/d12/d17/d105/f113 [707235,2950] 0 2024-09-10T04:05:27.324 INFO:tasks.workunit.client.0.smithi046.stdout:0/563: dread d1/dc/f7b [0,4194304] 0 2024-09-10T04:05:27.324 INFO:tasks.workunit.client.0.smithi046.stdout:8/685: readlink d1/d29/d17/d1f/d46/d6a/d82/l98 0 2024-09-10T04:05:27.327 INFO:tasks.workunit.client.0.smithi046.stdout:7/751: sync 2024-09-10T04:05:27.327 INFO:tasks.workunit.client.0.smithi046.stdout:6/607: sync 2024-09-10T04:05:27.327 INFO:tasks.workunit.client.0.smithi046.stdout:0/564: dread d1/d13/d37/d75/f7f [0,4194304] 0 2024-09-10T04:05:27.328 INFO:tasks.workunit.client.1.smithi070.stdout:3/771: unlink db/d1c/d25/d28/c35 0 2024-09-10T04:05:27.330 INFO:tasks.workunit.client.0.smithi046.stdout:0/565: write d1/dc/f27 [1415761,71845] 0 2024-09-10T04:05:27.331 INFO:tasks.workunit.client.0.smithi046.stdout:6/608: write da/db/d11/d13/d1e/d23/d86/fc8 [288936,72451] 0 2024-09-10T04:05:27.334 INFO:tasks.workunit.client.0.smithi046.stdout:9/675: creat d4/d29/fd2 x:0 0 0 2024-09-10T04:05:27.334 INFO:tasks.workunit.client.1.smithi070.stdout:7/795: sync 2024-09-10T04:05:27.335 INFO:tasks.workunit.client.0.smithi046.stdout:4/845: dwrite d3/d12/d17/d29/d58/db2/d5f/d85/fd7 [0,4194304] 0 2024-09-10T04:05:27.336 INFO:tasks.workunit.client.1.smithi070.stdout:7/796: write d2/d7/d9/dd5/f108 [180068,45240] 0 2024-09-10T04:05:27.337 INFO:tasks.workunit.client.1.smithi070.stdout:7/797: chown d2/dc8/df9 3316 1 2024-09-10T04:05:27.340 INFO:tasks.workunit.client.0.smithi046.stdout:1/530: link d3/d9b/f9e d3/d27/d5d/fad 0 2024-09-10T04:05:27.345 INFO:tasks.workunit.client.1.smithi070.stdout:1/685: rename d3/d8/d27/d9a/cbf to d3/d8/d27/d7a/ce2 0 2024-09-10T04:05:27.346 INFO:tasks.workunit.client.1.smithi070.stdout:1/686: stat d3/la 0 2024-09-10T04:05:27.346 INFO:tasks.workunit.client.0.smithi046.stdout:3/685: truncate d12/d13/d14/f33 3554338 0 2024-09-10T04:05:27.350 INFO:tasks.workunit.client.0.smithi046.stdout:7/752: creat d5/d9/d12/f109 x:0 0 0 2024-09-10T04:05:27.351 INFO:tasks.workunit.client.1.smithi070.stdout:9/810: stat d9/d1c/cb9 0 2024-09-10T04:05:27.352 INFO:tasks.workunit.client.1.smithi070.stdout:5/768: mknod d17/d19/d6a/dfb/c108 0 2024-09-10T04:05:27.354 INFO:tasks.workunit.client.1.smithi070.stdout:3/772: mknod db/d79/db6/dc3/c10a 0 2024-09-10T04:05:27.354 INFO:tasks.workunit.client.1.smithi070.stdout:3/773: stat db/d1c/c42 0 2024-09-10T04:05:27.355 INFO:tasks.workunit.client.0.smithi046.stdout:6/609: fdatasync da/f2a 0 2024-09-10T04:05:27.358 INFO:tasks.workunit.client.0.smithi046.stdout:0/566: symlink d1/dc/d25/d33/d3/d29/d2d/d5e/lb0 0 2024-09-10T04:05:27.359 INFO:tasks.workunit.client.0.smithi046.stdout:2/656: dwrite d2/d8/d37/d63/f82 [0,4194304] 0 2024-09-10T04:05:27.361 INFO:tasks.workunit.client.0.smithi046.stdout:5/850: dwrite d7/d11/d14/f29 [0,4194304] 0 2024-09-10T04:05:27.364 INFO:tasks.workunit.client.0.smithi046.stdout:4/846: creat d3/d12/d17/d91/dbf/f127 x:0 0 0 2024-09-10T04:05:27.364 INFO:tasks.workunit.client.1.smithi070.stdout:6/764: dwrite d0/d3/f99 [0,4194304] 0 2024-09-10T04:05:27.365 INFO:tasks.workunit.client.1.smithi070.stdout:6/765: fsync d0/fa7 0 2024-09-10T04:05:27.368 INFO:tasks.workunit.client.1.smithi070.stdout:7/798: creat d2/d3/dc/d5d/d41/da8/f11a x:0 0 0 2024-09-10T04:05:27.369 INFO:tasks.workunit.client.1.smithi070.stdout:2/731: truncate f3 6667719 0 2024-09-10T04:05:27.370 INFO:tasks.workunit.client.1.smithi070.stdout:2/732: dread - d15/d18/d56/d7c/ff5 zero size 2024-09-10T04:05:27.371 INFO:tasks.workunit.client.1.smithi070.stdout:7/799: write d2/d7/d9/d18/d3c/f73 [8855026,445] 0 2024-09-10T04:05:27.372 INFO:tasks.workunit.client.1.smithi070.stdout:2/733: fsync d15/d18/d29/d2a/d2e/da6/ff0 0 2024-09-10T04:05:27.372 INFO:tasks.workunit.client.1.smithi070.stdout:4/709: unlink de/d13/d18/l63 0 2024-09-10T04:05:27.373 INFO:tasks.workunit.client.0.smithi046.stdout:1/531: read d3/f26 [878996,78874] 0 2024-09-10T04:05:27.374 INFO:tasks.workunit.client.1.smithi070.stdout:7/800: truncate d2/d7/db/f92 1887914 0 2024-09-10T04:05:27.375 INFO:tasks.workunit.client.1.smithi070.stdout:9/811: dread - d9/d59/d2a/d3f/daf/fc0 zero size 2024-09-10T04:05:27.375 INFO:tasks.workunit.client.1.smithi070.stdout:8/808: write d4/d16/f33 [3382701,20405] 0 2024-09-10T04:05:27.378 INFO:tasks.workunit.client.1.smithi070.stdout:9/812: dread d9/d1c/dac/d8a/d3c/d62/f8d [0,4194304] 0 2024-09-10T04:05:27.380 INFO:tasks.workunit.client.1.smithi070.stdout:9/813: write d9/d1c/dac/d8a/dc4/de1/fc3 [295853,47291] 0 2024-09-10T04:05:27.380 INFO:tasks.workunit.client.1.smithi070.stdout:3/774: fdatasync db/d1c/fe2 0 2024-09-10T04:05:27.381 INFO:tasks.workunit.client.1.smithi070.stdout:3/775: stat db/d1c/l7e 0 2024-09-10T04:05:27.383 INFO:tasks.workunit.client.0.smithi046.stdout:6/610: mknod da/db/d18/ccb 0 2024-09-10T04:05:27.385 INFO:tasks.workunit.client.0.smithi046.stdout:0/567: mknod d1/dc/d25/d72/cb1 0 2024-09-10T04:05:27.386 INFO:tasks.workunit.client.1.smithi070.stdout:6/766: stat d0/d3/l1c 0 2024-09-10T04:05:27.387 INFO:tasks.workunit.client.1.smithi070.stdout:6/767: write d0/d3/d18/d1b/fd3 [348441,55257] 0 2024-09-10T04:05:27.390 INFO:tasks.workunit.client.0.smithi046.stdout:9/676: unlink d4/d29/l35 0 2024-09-10T04:05:27.393 INFO:tasks.workunit.client.1.smithi070.stdout:2/734: readlink d15/d18/d29/d2a/ld8 0 2024-09-10T04:05:27.393 INFO:tasks.workunit.client.1.smithi070.stdout:0/910: truncate d8/d13/de2/d74/fa1 24570 0 2024-09-10T04:05:27.395 INFO:tasks.workunit.client.0.smithi046.stdout:4/847: creat d3/d12/d17/d29/d58/db2/d7a/f128 x:0 0 0 2024-09-10T04:05:27.396 INFO:tasks.workunit.client.1.smithi070.stdout:7/801: creat d2/d7/d9/d18/d3c/d52/d103/f11b x:0 0 0 2024-09-10T04:05:27.397 INFO:tasks.workunit.client.1.smithi070.stdout:7/802: truncate d2/d3/dc/d79/fed 5061844 0 2024-09-10T04:05:27.397 INFO:tasks.workunit.client.0.smithi046.stdout:8/686: creat d1/d29/d19/fe6 x:0 0 0 2024-09-10T04:05:27.398 INFO:tasks.workunit.client.1.smithi070.stdout:8/809: fdatasync d4/d39/dfc/da8/ffb 0 2024-09-10T04:05:27.400 INFO:tasks.workunit.client.0.smithi046.stdout:7/753: write d5/d9/d43/f51 [83630,38482] 0 2024-09-10T04:05:27.401 INFO:tasks.workunit.client.1.smithi070.stdout:5/769: write d17/d19/d6a/d54/d57/f65 [4787683,35139] 0 2024-09-10T04:05:27.402 INFO:tasks.workunit.client.0.smithi046.stdout:1/532: fsync d3/d9b/f8d 0 2024-09-10T04:05:27.406 INFO:tasks.workunit.client.0.smithi046.stdout:6/611: mknod da/db/d11/d2b/d5a/ccc 0 2024-09-10T04:05:27.406 INFO:tasks.workunit.client.0.smithi046.stdout:6/612: readlink da/db/d18/lba 0 2024-09-10T04:05:27.407 INFO:tasks.workunit.client.1.smithi070.stdout:3/776: creat db/d79/db6/dc3/de1/dfe/f10b x:0 0 0 2024-09-10T04:05:27.408 INFO:tasks.workunit.client.0.smithi046.stdout:0/568: mkdir d1/dc/d49/db2 0 2024-09-10T04:05:27.412 INFO:tasks.workunit.client.1.smithi070.stdout:4/710: unlink de/d13/d18/d23/d47/d59/d79/dbc/fd1 0 2024-09-10T04:05:27.416 INFO:tasks.workunit.client.1.smithi070.stdout:2/735: fdatasync d15/d18/f27 0 2024-09-10T04:05:27.416 INFO:tasks.workunit.client.0.smithi046.stdout:5/851: dwrite d7/d11/d14/d2f/da3/fbf [0,4194304] 0 2024-09-10T04:05:27.419 INFO:tasks.workunit.client.1.smithi070.stdout:0/911: symlink d8/d13/d1a/d75/l12e 0 2024-09-10T04:05:27.420 INFO:tasks.workunit.client.0.smithi046.stdout:4/848: mkdir d3/d12/dec/df0/d129 0 2024-09-10T04:05:27.421 INFO:tasks.workunit.client.1.smithi070.stdout:7/803: creat d2/d7/d9/db4/f11c x:0 0 0 2024-09-10T04:05:27.422 INFO:tasks.workunit.client.1.smithi070.stdout:6/768: dread d0/d3/d18/d1b/f11 [4194304,4194304] 0 2024-09-10T04:05:27.422 INFO:tasks.workunit.client.0.smithi046.stdout:8/687: truncate d1/d29/d17/d1f/d46/f8b 195476 0 2024-09-10T04:05:27.423 INFO:tasks.workunit.client.0.smithi046.stdout:3/686: link d12/d13/d37/dab/dac/dd3/fde d12/d5d/d71/d49/d5c/fe9 0 2024-09-10T04:05:27.425 INFO:tasks.workunit.client.0.smithi046.stdout:3/687: write d12/d13/d37/d51/db5/d4b/fb6 [178863,107547] 0 2024-09-10T04:05:27.426 INFO:tasks.workunit.client.1.smithi070.stdout:8/810: creat d4/d16/d17/d1a/d60/d7a/d84/f116 x:0 0 0 2024-09-10T04:05:27.429 INFO:tasks.workunit.client.1.smithi070.stdout:2/736: dread d15/df6/ff9 [0,4194304] 0 2024-09-10T04:05:27.430 INFO:tasks.workunit.client.0.smithi046.stdout:7/754: rename d5/d6/d39/d5f/f96 to d5/d6/d39/d5f/f10a 0 2024-09-10T04:05:27.430 INFO:tasks.workunit.client.0.smithi046.stdout:7/755: stat d5 0 2024-09-10T04:05:27.431 INFO:tasks.workunit.client.1.smithi070.stdout:5/770: symlink d17/d24/d2b/l109 0 2024-09-10T04:05:27.431 INFO:tasks.workunit.client.0.smithi046.stdout:7/756: dread - d5/d6/dbe/dd7/f101 zero size 2024-09-10T04:05:27.432 INFO:tasks.workunit.client.1.smithi070.stdout:9/814: symlink d9/d1c/dac/d8a/dc4/dda/de0/l105 0 2024-09-10T04:05:27.435 INFO:tasks.workunit.client.1.smithi070.stdout:4/711: rename de/d13/d43/f76 to de/d13/d18/d23/d39/d48/fee 0 2024-09-10T04:05:27.437 INFO:tasks.workunit.client.1.smithi070.stdout:4/712: chown de/d13/d18/d23/d39/d48/la8 648730 1 2024-09-10T04:05:27.439 INFO:tasks.workunit.client.0.smithi046.stdout:4/849: truncate d3/d12/d17/d29/f83 1985559 0 2024-09-10T04:05:27.440 INFO:tasks.workunit.client.0.smithi046.stdout:2/657: dwrite d2/d6/d23/d5d/f8f [0,4194304] 0 2024-09-10T04:05:27.441 INFO:tasks.workunit.client.1.smithi070.stdout:0/912: symlink d8/d13/de2/d53/d59/l12f 0 2024-09-10T04:05:27.441 INFO:tasks.workunit.client.0.smithi046.stdout:4/850: chown d3/d12/d17/d29/d58/db2/d7a/f11f 30 1 2024-09-10T04:05:27.442 INFO:tasks.workunit.client.0.smithi046.stdout:4/851: readlink d3/d12/d17/d22/d51/d5c/l78 0 2024-09-10T04:05:27.443 INFO:tasks.workunit.client.1.smithi070.stdout:8/811: dread d4/f77 [0,4194304] 0 2024-09-10T04:05:27.449 INFO:tasks.workunit.client.1.smithi070.stdout:7/804: mkdir d2/d3/dc/d79/d11d 0 2024-09-10T04:05:27.451 INFO:tasks.workunit.client.0.smithi046.stdout:3/688: creat d12/d13/d14/d1e/d2d/d39/fea x:0 0 0 2024-09-10T04:05:27.455 INFO:tasks.workunit.client.1.smithi070.stdout:7/805: dwrite d2/d3/dc/d13/fc7 [0,4194304] 0 2024-09-10T04:05:27.456 INFO:tasks.workunit.client.0.smithi046.stdout:7/757: rename d5/d6/dbe/dd7/fbb to d5/d9/d43/f10b 0 2024-09-10T04:05:27.456 INFO:tasks.workunit.client.0.smithi046.stdout:7/758: stat d5/d6/d32/f95 0 2024-09-10T04:05:27.458 INFO:tasks.workunit.client.0.smithi046.stdout:6/613: truncate da/fe 1470719 0 2024-09-10T04:05:27.459 INFO:tasks.workunit.client.0.smithi046.stdout:9/677: link d4/d3c/d4d/d5c/d59/l70 d4/d7/d61/dd0/ld3 0 2024-09-10T04:05:27.459 INFO:tasks.workunit.client.0.smithi046.stdout:6/614: chown da/db/d11/d2b/f70 89 1 2024-09-10T04:05:27.468 INFO:tasks.workunit.client.0.smithi046.stdout:4/852: mknod d3/d12/d17/d22/d30/dc7/c12a 0 2024-09-10T04:05:27.468 INFO:tasks.workunit.client.0.smithi046.stdout:9/678: dwrite d4/d7/d1f/f21 [0,4194304] 0 2024-09-10T04:05:27.471 INFO:tasks.workunit.client.0.smithi046.stdout:3/689: creat d12/d13/d37/dab/dac/dd3/feb x:0 0 0 2024-09-10T04:05:27.473 INFO:tasks.workunit.client.1.smithi070.stdout:6/769: rename d0/d3/da4/dc0/dea/f6b to d0/d3/d18/d1b/d33/ff3 0 2024-09-10T04:05:27.474 INFO:tasks.workunit.client.0.smithi046.stdout:8/688: dwrite d1/d29/d17/d1f/d46/d6a/f94 [0,4194304] 0 2024-09-10T04:05:27.476 INFO:tasks.workunit.client.1.smithi070.stdout:4/713: creat de/d13/d18/d23/d47/d59/fef x:0 0 0 2024-09-10T04:05:27.478 INFO:tasks.workunit.client.0.smithi046.stdout:7/759: mkdir d5/d6/db/dd/d10c 0 2024-09-10T04:05:27.480 INFO:tasks.workunit.client.0.smithi046.stdout:7/760: write d5/d9/d12/d72/d77/de0/f14 [2998115,70489] 0 2024-09-10T04:05:27.486 INFO:tasks.workunit.client.0.smithi046.stdout:2/658: unlink d2/d6/d23/d5d/d77/d7b/d85/lb0 0 2024-09-10T04:05:27.486 INFO:tasks.workunit.client.0.smithi046.stdout:2/659: fdatasync d2/f35 0 2024-09-10T04:05:27.488 INFO:tasks.workunit.client.0.smithi046.stdout:2/660: chown d2/d6/d23/d42/db4/ld3 3581469 1 2024-09-10T04:05:27.489 INFO:tasks.workunit.client.1.smithi070.stdout:2/737: truncate d15/d18/f21 795753 0 2024-09-10T04:05:27.492 INFO:tasks.workunit.client.0.smithi046.stdout:9/679: unlink d4/d18/c2d 0 2024-09-10T04:05:27.492 INFO:tasks.workunit.client.0.smithi046.stdout:8/689: fdatasync d1/d29/d17/d1f/d46/d6a/f94 0 2024-09-10T04:05:27.493 INFO:tasks.workunit.client.1.smithi070.stdout:9/815: dwrite d9/d1c/dac/d8a/d3c/d62/f8d [0,4194304] 0 2024-09-10T04:05:27.493 INFO:tasks.workunit.client.0.smithi046.stdout:4/853: creat d3/d12/d17/d29/ddd/f12b x:0 0 0 2024-09-10T04:05:27.494 INFO:tasks.workunit.client.0.smithi046.stdout:9/680: read - d4/d18/d89/fa9 zero size 2024-09-10T04:05:27.495 INFO:tasks.workunit.client.1.smithi070.stdout:3/777: creat db/d1c/d25/d28/f10c x:0 0 0 2024-09-10T04:05:27.497 INFO:tasks.workunit.client.0.smithi046.stdout:9/681: chown d4/d66/da6/f72 23913312 1 2024-09-10T04:05:27.500 INFO:tasks.workunit.client.0.smithi046.stdout:9/682: write d4/d3c/d4d/d5c/f8d [4023933,99578] 0 2024-09-10T04:05:27.505 INFO:tasks.workunit.client.0.smithi046.stdout:6/615: dread da/db/d18/f35 [0,4194304] 0 2024-09-10T04:05:27.505 INFO:tasks.workunit.client.0.smithi046.stdout:8/690: dwrite d1/d29/d19/d36/fd4 [0,4194304] 0 2024-09-10T04:05:27.506 INFO:tasks.workunit.client.0.smithi046.stdout:5/852: rename d7/d11/d14/d2f/df0/cf7 to d7/d11/d3f/dd3/c129 0 2024-09-10T04:05:27.507 INFO:tasks.workunit.client.1.smithi070.stdout:4/714: rmdir de/d13/d18/d23/d47/d59/d79/d6a 39 2024-09-10T04:05:27.507 INFO:tasks.workunit.client.0.smithi046.stdout:5/853: write d7/d11/d14/d2f/deb/f10e [305333,45463] 0 2024-09-10T04:05:27.508 INFO:tasks.workunit.client.1.smithi070.stdout:4/715: chown de/d13/d5b/d7b/dd5/f71 3477 1 2024-09-10T04:05:27.508 INFO:tasks.workunit.client.1.smithi070.stdout:1/687: sync 2024-09-10T04:05:27.510 INFO:tasks.workunit.client.1.smithi070.stdout:1/688: chown d3/d8/d12/d2f/f5b 2920 1 2024-09-10T04:05:27.511 INFO:tasks.workunit.client.1.smithi070.stdout:0/913: fdatasync d8/d13/de2/d4f/f6e 0 2024-09-10T04:05:27.512 INFO:tasks.workunit.client.1.smithi070.stdout:1/689: readlink d3/d8/l51 0 2024-09-10T04:05:27.515 INFO:tasks.workunit.client.1.smithi070.stdout:9/816: dread d9/fab [0,4194304] 0 2024-09-10T04:05:27.518 INFO:tasks.workunit.client.1.smithi070.stdout:5/771: truncate d17/d19/d35/f6c 893414 0 2024-09-10T04:05:27.518 INFO:tasks.workunit.client.1.smithi070.stdout:5/772: dread - d17/d6f/d32/ffc zero size 2024-09-10T04:05:27.520 INFO:tasks.workunit.client.1.smithi070.stdout:5/773: truncate d17/d19/d6a/d54/d57/da8/fdc 842303 0 2024-09-10T04:05:27.521 INFO:tasks.workunit.client.1.smithi070.stdout:5/774: write d17/d24/d84/d9b/fcb [1209411,40699] 0 2024-09-10T04:05:27.528 INFO:tasks.workunit.client.0.smithi046.stdout:4/854: rmdir d3/d12/d17/d29/d58/db2/d7a 39 2024-09-10T04:05:27.530 INFO:tasks.workunit.client.0.smithi046.stdout:4/855: truncate d3/d12/d17/d29/d58/db2/f11e 749573 0 2024-09-10T04:05:27.532 INFO:tasks.workunit.client.1.smithi070.stdout:8/812: creat d4/d16/d17/d19/f117 x:0 0 0 2024-09-10T04:05:27.533 INFO:tasks.workunit.client.0.smithi046.stdout:2/661: write d2/f73 [5495639,57479] 0 2024-09-10T04:05:27.534 INFO:tasks.workunit.client.1.smithi070.stdout:4/716: write de/d13/d18/d23/d47/d59/d79/f66 [706812,114312] 0 2024-09-10T04:05:27.534 INFO:tasks.workunit.client.0.smithi046.stdout:2/662: truncate d2/d6/f43 2022236 0 2024-09-10T04:05:27.538 INFO:tasks.workunit.client.1.smithi070.stdout:8/813: dwrite d4/d16/d17/d1a/d60/f9c [0,4194304] 0 2024-09-10T04:05:27.538 INFO:tasks.workunit.client.0.smithi046.stdout:6/616: creat da/dc/fcd x:0 0 0 2024-09-10T04:05:27.540 INFO:tasks.workunit.client.1.smithi070.stdout:2/738: mkdir d15/d18/d29/d2a/d2e/d32/d36/d10b 0 2024-09-10T04:05:27.540 INFO:tasks.workunit.client.0.smithi046.stdout:6/617: read da/db/d10/d27/f8a [2086171,63298] 0 2024-09-10T04:05:27.543 INFO:tasks.workunit.client.0.smithi046.stdout:5/854: creat d7/d11/d14/d2f/deb/f12a x:0 0 0 2024-09-10T04:05:27.544 INFO:tasks.workunit.client.0.smithi046.stdout:9/683: write d4/d18/d89/fa7 [451539,68284] 0 2024-09-10T04:05:27.545 INFO:tasks.workunit.client.0.smithi046.stdout:5/855: chown d7/d11/d14/d2f/deb/da5/f120 15255 1 2024-09-10T04:05:27.545 INFO:tasks.workunit.client.0.smithi046.stdout:7/761: symlink d5/d9/d12/d72/d77/de0/d1f/l10d 0 2024-09-10T04:05:27.546 INFO:tasks.workunit.client.1.smithi070.stdout:5/775: stat d17/d6f/d32/d90/d99/cac 0 2024-09-10T04:05:27.548 INFO:tasks.workunit.client.0.smithi046.stdout:9/684: truncate d4/d29/f6d 4216417 0 2024-09-10T04:05:27.548 INFO:tasks.workunit.client.0.smithi046.stdout:3/690: rmdir d12/d13/d14/d1e/d2d/d39/d67/da0/ddc 0 2024-09-10T04:05:27.551 INFO:tasks.workunit.client.0.smithi046.stdout:4/856: rename d3/d12/d17/d22/d51/d5c/d6c/ced to d3/d12/d17/d29/c12c 0 2024-09-10T04:05:27.552 INFO:tasks.workunit.client.0.smithi046.stdout:2/663: truncate d2/d6/d23/d55/fa3 381625 0 2024-09-10T04:05:27.554 INFO:tasks.workunit.client.0.smithi046.stdout:7/762: dwrite d5/d6/db/dd/d24/d76/d9c/fb2 [0,4194304] 0 2024-09-10T04:05:27.557 INFO:tasks.workunit.client.1.smithi070.stdout:0/914: mknod d8/c130 0 2024-09-10T04:05:27.558 INFO:tasks.workunit.client.1.smithi070.stdout:3/778: write db/d83/fac [161183,61520] 0 2024-09-10T04:05:27.559 INFO:tasks.workunit.client.1.smithi070.stdout:1/690: symlink d3/d8/d27/d7a/db4/le3 0 2024-09-10T04:05:27.560 INFO:tasks.workunit.client.1.smithi070.stdout:1/691: chown d3/d8/d27/d7a/db4 0 1 2024-09-10T04:05:27.561 INFO:tasks.workunit.client.0.smithi046.stdout:5/856: creat d7/d11/d14/d2f/deb/da5/dcc/f12b x:0 0 0 2024-09-10T04:05:27.562 INFO:tasks.workunit.client.1.smithi070.stdout:7/806: link d2/d7/d9/db4/cec d2/d3/dc/d79/d11d/c11e 0 2024-09-10T04:05:27.563 INFO:tasks.workunit.client.0.smithi046.stdout:3/691: mkdir d12/d5d/d71/d49/dec 0 2024-09-10T04:05:27.564 INFO:tasks.workunit.client.0.smithi046.stdout:9/685: truncate d4/d29/d32/f43 4842745 0 2024-09-10T04:05:27.568 INFO:tasks.workunit.client.0.smithi046.stdout:7/763: dread d5/d6/db/dd/d24/f31 [0,4194304] 0 2024-09-10T04:05:27.568 INFO:tasks.workunit.client.1.smithi070.stdout:5/776: mknod d17/d19/d35/c10a 0 2024-09-10T04:05:27.569 INFO:tasks.workunit.client.1.smithi070.stdout:5/777: chown d17/d6f/d32/c43 1805 1 2024-09-10T04:05:27.570 INFO:tasks.workunit.client.0.smithi046.stdout:7/764: dread d5/d6/db/f29 [4194304,4194304] 0 2024-09-10T04:05:27.570 INFO:tasks.workunit.client.0.smithi046.stdout:6/618: link da/db/d10/d27/f88 da/db/d11/d13/d4f/fce 0 2024-09-10T04:05:27.573 INFO:tasks.workunit.client.1.smithi070.stdout:6/770: getdents d0/d3/d8c/dab 0 2024-09-10T04:05:27.574 INFO:tasks.workunit.client.0.smithi046.stdout:9/686: mkdir d4/d7/d1f/dce/dd4 0 2024-09-10T04:05:27.575 INFO:tasks.workunit.client.0.smithi046.stdout:5/857: dread d7/f70 [0,4194304] 0 2024-09-10T04:05:27.576 INFO:tasks.workunit.client.0.smithi046.stdout:3/692: mkdir d12/d13/d37/d51/db5/d4b/db7/dc0/ded 0 2024-09-10T04:05:27.577 INFO:tasks.workunit.client.0.smithi046.stdout:3/693: chown d12/d13/d37/d51/db5/d50/d52/f65 1991 1 2024-09-10T04:05:27.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:27 smithi070 ceph-mon[35495]: pgmap v267: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 102 MiB/s rd, 280 MiB/s wr, 574 op/s 2024-09-10T04:05:27.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:27 smithi046 ceph-mon[28328]: pgmap v267: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 102 MiB/s rd, 280 MiB/s wr, 574 op/s 2024-09-10T04:05:27.598 INFO:tasks.workunit.client.0.smithi046.stdout:8/691: getdents d1/d29/d17/d1f 0 2024-09-10T04:05:27.601 INFO:tasks.workunit.client.1.smithi070.stdout:9/817: rename d9/d1c/dac/d8a/l80 to d9/d59/l106 0 2024-09-10T04:05:27.601 INFO:tasks.workunit.client.0.smithi046.stdout:2/664: write d2/d6/d23/f87 [1108674,761] 0 2024-09-10T04:05:27.605 INFO:tasks.workunit.client.1.smithi070.stdout:8/814: mknod d4/dbf/d74/da4/d114/c118 0 2024-09-10T04:05:27.608 INFO:tasks.workunit.client.1.smithi070.stdout:2/739: mkdir d15/d18/d56/d7c/d10c 0 2024-09-10T04:05:27.609 INFO:tasks.workunit.client.0.smithi046.stdout:2/665: dwrite d2/d6/d9d/db6/dbe/fde [0,4194304] 0 2024-09-10T04:05:27.609 INFO:tasks.workunit.client.1.smithi070.stdout:3/779: write db/d1c/d53/fa2 [3449634,99843] 0 2024-09-10T04:05:27.610 INFO:tasks.workunit.client.1.smithi070.stdout:3/780: stat db/d1c/d89/lc0 0 2024-09-10T04:05:27.611 INFO:tasks.workunit.client.0.smithi046.stdout:9/687: mkdir d4/d29/d65/dd5 0 2024-09-10T04:05:27.615 INFO:tasks.workunit.client.1.smithi070.stdout:1/692: mknod d3/d10/ce4 0 2024-09-10T04:05:27.618 INFO:tasks.workunit.client.0.smithi046.stdout:8/692: creat d1/d29/d19/d73/fe7 x:0 0 0 2024-09-10T04:05:27.625 INFO:tasks.workunit.client.0.smithi046.stdout:6/619: mknod da/db/d11/d13/ccf 0 2024-09-10T04:05:27.626 INFO:tasks.workunit.client.1.smithi070.stdout:8/815: unlink d4/f29 0 2024-09-10T04:05:27.626 INFO:tasks.workunit.client.1.smithi070.stdout:8/816: read - d4/dcf/f101 zero size 2024-09-10T04:05:27.627 INFO:tasks.workunit.client.0.smithi046.stdout:5/858: write d7/d11/d3f/d53/d75/f7c [90069,94157] 0 2024-09-10T04:05:27.628 INFO:tasks.workunit.client.1.smithi070.stdout:2/740: fsync d15/d18/d29/d64/f95 0 2024-09-10T04:05:27.629 INFO:tasks.workunit.client.0.smithi046.stdout:5/859: write d7/d11/d14/d2f/f22 [3310259,4550] 0 2024-09-10T04:05:27.631 INFO:tasks.workunit.client.0.smithi046.stdout:2/666: unlink d2/d6/d23/d42/f4e 0 2024-09-10T04:05:27.634 INFO:tasks.workunit.client.1.smithi070.stdout:7/807: truncate d2/d40/d6c/db2/db7/d104/f109 163035 0 2024-09-10T04:05:27.635 INFO:tasks.workunit.client.0.smithi046.stdout:7/765: write d5/f6b [478980,54589] 0 2024-09-10T04:05:27.637 INFO:tasks.workunit.client.0.smithi046.stdout:3/694: mkdir d12/d13/d14/d1e/d2d/d6f/d8f/dee 0 2024-09-10T04:05:27.644 INFO:tasks.workunit.client.1.smithi070.stdout:5/778: rename d17/d6f/d32/d90/fb6 to d17/d24/d84/d9b/f10b 0 2024-09-10T04:05:27.648 INFO:tasks.workunit.client.0.smithi046.stdout:9/688: dwrite d4/d18/f33 [0,4194304] 0 2024-09-10T04:05:27.648 INFO:tasks.workunit.client.0.smithi046.stdout:5/860: symlink d7/d11/d3f/d53/d75/l12c 0 2024-09-10T04:05:27.648 INFO:tasks.workunit.client.0.smithi046.stdout:2/667: dread d2/d6/d9d/fa5 [0,4194304] 0 2024-09-10T04:05:27.652 INFO:tasks.workunit.client.0.smithi046.stdout:7/766: dread d5/d9/d12/d72/d77/de0/f14 [0,4194304] 0 2024-09-10T04:05:27.653 INFO:tasks.workunit.client.0.smithi046.stdout:7/767: fsync d5/d6/dbe/dd7/f101 0 2024-09-10T04:05:27.653 INFO:tasks.workunit.client.1.smithi070.stdout:6/771: symlink d0/d3/d1f/d81/lf4 0 2024-09-10T04:05:27.660 INFO:tasks.workunit.client.1.smithi070.stdout:0/915: getdents d8/d13/d1a 0 2024-09-10T04:05:27.661 INFO:tasks.workunit.client.0.smithi046.stdout:9/689: creat d4/d3c/d4d/d80/fd6 x:0 0 0 2024-09-10T04:05:27.662 INFO:tasks.workunit.client.1.smithi070.stdout:1/693: mkdir d3/d8/d12/d2f/dcc/de5 0 2024-09-10T04:05:27.663 INFO:tasks.workunit.client.1.smithi070.stdout:3/781: truncate db/d79/db6/fbe 1642644 0 2024-09-10T04:05:27.663 INFO:tasks.workunit.client.1.smithi070.stdout:1/694: chown d3/d10/db2/dc7/l71 28 1 2024-09-10T04:05:27.663 INFO:tasks.workunit.client.1.smithi070.stdout:4/717: getdents de/d13/d18/d5c/db6 0 2024-09-10T04:05:27.664 INFO:tasks.workunit.client.1.smithi070.stdout:5/779: read - d17/f9e zero size 2024-09-10T04:05:27.664 INFO:tasks.workunit.client.1.smithi070.stdout:1/695: chown d3/d10/db2/dc7/d54/c56 421 1 2024-09-10T04:05:27.664 INFO:tasks.workunit.client.0.smithi046.stdout:2/668: mknod d2/d6/d23/d5d/d77/d7b/dc4/cf1 0 2024-09-10T04:05:27.665 INFO:tasks.workunit.client.0.smithi046.stdout:2/669: fsync d2/d6/d23/f4b 0 2024-09-10T04:05:27.666 INFO:tasks.workunit.client.1.smithi070.stdout:4/718: truncate de/d13/d18/d23/d39/d77/faa 720118 0 2024-09-10T04:05:27.666 INFO:tasks.workunit.client.1.smithi070.stdout:9/818: creat d9/d59/d2a/d3f/f107 x:0 0 0 2024-09-10T04:05:27.667 INFO:tasks.workunit.client.1.smithi070.stdout:8/817: link d4/d16/f34 d4/d39/dfc/d43/d52/f119 0 2024-09-10T04:05:27.668 INFO:tasks.workunit.client.0.smithi046.stdout:8/693: rmdir d1/d29/d17/d1f/d46/d6a/dbb 0 2024-09-10T04:05:27.670 INFO:tasks.workunit.client.1.smithi070.stdout:2/741: symlink d15/d2b/d106/l10d 0 2024-09-10T04:05:27.674 INFO:tasks.workunit.client.0.smithi046.stdout:5/861: rename d7/d11/d14/d8e/d96/f116 to d7/d11/d3f/d53/d75/f12d 0 2024-09-10T04:05:27.674 INFO:tasks.workunit.client.0.smithi046.stdout:5/862: dread - d7/d11/d14/d45/ffb zero size 2024-09-10T04:05:27.675 INFO:tasks.workunit.client.0.smithi046.stdout:9/690: creat d4/d7/d61/fd7 x:0 0 0 2024-09-10T04:05:27.680 INFO:tasks.workunit.client.0.smithi046.stdout:7/768: dwrite d5/d6/d32/dab/db6/f79 [4194304,4194304] 0 2024-09-10T04:05:27.681 INFO:tasks.workunit.client.0.smithi046.stdout:2/670: mkdir d2/d6/d9d/db6/dbe/df2 0 2024-09-10T04:05:27.684 INFO:tasks.workunit.client.0.smithi046.stdout:8/694: creat d1/d29/d17/d1f/d46/d6a/fe8 x:0 0 0 2024-09-10T04:05:27.684 INFO:tasks.workunit.client.0.smithi046.stdout:6/620: getdents da/db/d11/d2b/d55 0 2024-09-10T04:05:27.686 INFO:tasks.workunit.client.1.smithi070.stdout:1/696: chown d3/d8/d13/d67/d83/fc5 98 1 2024-09-10T04:05:27.687 INFO:tasks.workunit.client.0.smithi046.stdout:5/863: mknod d7/d11/d3f/d53/d75/c12e 0 2024-09-10T04:05:27.687 INFO:tasks.workunit.client.1.smithi070.stdout:4/719: read de/d13/d18/d23/d47/f51 [120060,39527] 0 2024-09-10T04:05:27.688 INFO:tasks.workunit.client.1.smithi070.stdout:4/720: fsync f0 0 2024-09-10T04:05:27.688 INFO:tasks.workunit.client.0.smithi046.stdout:3/695: getdents d12/d13/d37/dab/dac 0 2024-09-10T04:05:27.689 INFO:tasks.workunit.client.0.smithi046.stdout:5/864: write d7/d11/d14/d2f/d64/fc6 [4203735,82130] 0 2024-09-10T04:05:27.691 INFO:tasks.workunit.client.1.smithi070.stdout:8/818: creat d4/d16/d17/d1a/df7/f11a x:0 0 0 2024-09-10T04:05:27.692 INFO:tasks.workunit.client.1.smithi070.stdout:6/772: mkdir d0/d3/d18/df5 0 2024-09-10T04:05:27.693 INFO:tasks.workunit.client.1.smithi070.stdout:6/773: write d0/d3/d1f/d55/f8e [1148693,24310] 0 2024-09-10T04:05:27.693 INFO:tasks.workunit.client.1.smithi070.stdout:7/808: rmdir d2/dc8/d10a 0 2024-09-10T04:05:27.694 INFO:tasks.workunit.client.0.smithi046.stdout:2/671: mkdir d2/d6/d23/d5d/d77/d7b/d85/d8d/df3 0 2024-09-10T04:05:27.695 INFO:tasks.workunit.client.0.smithi046.stdout:2/672: readlink d2/d8/d37/d63/l88 0 2024-09-10T04:05:27.697 INFO:tasks.workunit.client.1.smithi070.stdout:0/916: link d8/d24/d2d/d34/d6b/ff3 d8/d13/de2/d7d/d8a/f131 0 2024-09-10T04:05:27.699 INFO:tasks.workunit.client.0.smithi046.stdout:6/621: truncate da/f2a 879465 0 2024-09-10T04:05:27.701 INFO:tasks.workunit.client.1.smithi070.stdout:7/809: dwrite d2/d7/d9/dd5/fe2 [0,4194304] 0 2024-09-10T04:05:27.702 INFO:tasks.workunit.client.1.smithi070.stdout:7/810: chown d2/d3/d63/fe0 263858003 1 2024-09-10T04:05:27.703 INFO:tasks.workunit.client.0.smithi046.stdout:6/622: dread da/db/d11/d13/d1e/d23/d86/fc8 [0,4194304] 0 2024-09-10T04:05:27.703 INFO:tasks.workunit.client.1.smithi070.stdout:5/780: symlink d17/l10c 0 2024-09-10T04:05:27.704 INFO:tasks.workunit.client.1.smithi070.stdout:1/697: dread d3/d8/d13/d67/d83/fc5 [4194304,4194304] 0 2024-09-10T04:05:27.704 INFO:tasks.workunit.client.1.smithi070.stdout:7/811: stat d2/d40/d6c/db2/db7/d104 0 2024-09-10T04:05:27.705 INFO:tasks.workunit.client.1.smithi070.stdout:5/781: readlink d17/d19/d6a/d54/d57/lad 0 2024-09-10T04:05:27.706 INFO:tasks.workunit.client.1.smithi070.stdout:7/812: readlink d2/d3/dc/d5d/l77 0 2024-09-10T04:05:27.707 INFO:tasks.workunit.client.0.smithi046.stdout:5/865: readlink d7/d11/d3f/d5a/l85 0 2024-09-10T04:05:27.708 INFO:tasks.workunit.client.1.smithi070.stdout:5/782: chown d17/d19/d6a/d54/de1/le6 3412 1 2024-09-10T04:05:27.708 INFO:tasks.workunit.client.1.smithi070.stdout:7/813: fdatasync d2/d7/d9/d18/d3c/d52/fb0 0 2024-09-10T04:05:27.708 INFO:tasks.workunit.client.0.smithi046.stdout:9/691: mknod d4/d7/d1f/d54/dc7/dc8/cd8 0 2024-09-10T04:05:27.708 INFO:tasks.workunit.client.1.smithi070.stdout:7/814: chown d2/d7/d9/l10 840877 1 2024-09-10T04:05:27.710 INFO:tasks.workunit.client.1.smithi070.stdout:9/819: rename d9/d59/d2a/d3f/f107 to d9/d7a/d72/f108 0 2024-09-10T04:05:27.710 INFO:tasks.workunit.client.0.smithi046.stdout:2/673: mknod d2/d6/d23/d55/cf4 0 2024-09-10T04:05:27.711 INFO:tasks.workunit.client.0.smithi046.stdout:6/623: dwrite da/db/d11/d2b/f9a [0,4194304] 0 2024-09-10T04:05:27.713 INFO:tasks.workunit.client.1.smithi070.stdout:4/721: symlink de/d13/d5b/d7b/lf0 0 2024-09-10T04:05:27.714 INFO:tasks.workunit.client.0.smithi046.stdout:5/866: dread d7/d11/d14/f29 [0,4194304] 0 2024-09-10T04:05:27.716 INFO:tasks.workunit.client.1.smithi070.stdout:8/819: rmdir d4/d16 39 2024-09-10T04:05:27.718 INFO:tasks.workunit.client.1.smithi070.stdout:8/820: truncate d4/d39/dfc/d43/d52/f119 515673 0 2024-09-10T04:05:27.719 INFO:tasks.workunit.client.1.smithi070.stdout:8/821: readlink d4/d39/dfc/d43/d52/l53 0 2024-09-10T04:05:27.722 INFO:tasks.workunit.client.1.smithi070.stdout:0/917: creat d8/d13/d1a/d9a/f132 x:0 0 0 2024-09-10T04:05:27.723 INFO:tasks.workunit.client.0.smithi046.stdout:3/696: write d12/d5d/d71/d49/d5c/d9f/fa3 [916120,72262] 0 2024-09-10T04:05:27.728 INFO:tasks.workunit.client.0.smithi046.stdout:9/692: creat d4/d29/d32/fd9 x:0 0 0 2024-09-10T04:05:27.729 INFO:tasks.workunit.client.0.smithi046.stdout:9/693: write d4/d18/d89/fa9 [540919,44720] 0 2024-09-10T04:05:27.730 INFO:tasks.workunit.client.0.smithi046.stdout:8/695: truncate d1/d29/d19/d36/fd4 3732434 0 2024-09-10T04:05:27.730 INFO:tasks.workunit.client.0.smithi046.stdout:9/694: chown d4/d3c/d4d/d5c/d59/ld1 9 1 2024-09-10T04:05:27.734 INFO:tasks.workunit.client.0.smithi046.stdout:2/674: fdatasync d2/d6/d23/d5d/d77/d7b/d85/d8d/fc5 0 2024-09-10T04:05:27.735 INFO:tasks.workunit.client.1.smithi070.stdout:7/815: rmdir d2/d3/dc/d79 39 2024-09-10T04:05:27.735 INFO:tasks.workunit.client.1.smithi070.stdout:6/774: dread d0/d3/d18/d1b/d33/ff3 [0,4194304] 0 2024-09-10T04:05:27.736 INFO:tasks.workunit.client.0.smithi046.stdout:7/769: link d5/d6/d62/l88 d5/d9/d43/db4/def/l10e 0 2024-09-10T04:05:27.737 INFO:tasks.workunit.client.1.smithi070.stdout:9/820: truncate d9/d59/d2a/fcf 437842 0 2024-09-10T04:05:27.738 INFO:tasks.workunit.client.0.smithi046.stdout:5/867: fdatasync d7/d11/d14/d8e/d96/fbe 0 2024-09-10T04:05:27.739 INFO:tasks.workunit.client.0.smithi046.stdout:5/868: readlink d7/d11/d14/d2f/d64/lc9 0 2024-09-10T04:05:27.743 INFO:tasks.workunit.client.1.smithi070.stdout:0/918: rmdir d8/d13/de2/d4f/d11b 39 2024-09-10T04:05:27.745 INFO:tasks.workunit.client.0.smithi046.stdout:3/697: chown d12/d13/d37/d51/db5/d50/c83 376985031 1 2024-09-10T04:05:27.746 INFO:tasks.workunit.client.1.smithi070.stdout:8/822: dread d4/d39/dfc/f31 [0,4194304] 0 2024-09-10T04:05:27.746 INFO:tasks.workunit.client.1.smithi070.stdout:3/782: getdents db/d1c 0 2024-09-10T04:05:27.747 INFO:tasks.workunit.client.0.smithi046.stdout:6/624: dwrite da/d59/f85 [0,4194304] 0 2024-09-10T04:05:27.747 INFO:tasks.workunit.client.1.smithi070.stdout:7/816: dread d2/d66/f78 [0,4194304] 0 2024-09-10T04:05:27.749 INFO:tasks.workunit.client.1.smithi070.stdout:5/783: mknod d17/d19/d6a/d54/d57/c10d 0 2024-09-10T04:05:27.750 INFO:tasks.workunit.client.0.smithi046.stdout:9/695: mknod d4/d7/d1f/d54/dc7/dc8/cda 0 2024-09-10T04:05:27.753 INFO:tasks.workunit.client.0.smithi046.stdout:2/675: write d2/d6/d23/d5d/d5e/f84 [5238213,123559] 0 2024-09-10T04:05:27.754 INFO:tasks.workunit.client.0.smithi046.stdout:7/770: symlink d5/d9/d12/d72/d77/de0/d1f/d7a/dfc/l10f 0 2024-09-10T04:05:27.756 INFO:tasks.workunit.client.0.smithi046.stdout:5/869: rename d7/d11/d14/c100 to d7/d11/d14/d2f/deb/c12f 0 2024-09-10T04:05:27.757 INFO:tasks.workunit.client.0.smithi046.stdout:5/870: write d7/d11/d14/d45/ffb [370483,2227] 0 2024-09-10T04:05:27.761 INFO:tasks.workunit.client.1.smithi070.stdout:3/783: creat db/d1c/f10d x:0 0 0 2024-09-10T04:05:27.763 INFO:tasks.workunit.client.1.smithi070.stdout:7/817: symlink d2/d3/dc/d13/dbd/l11f 0 2024-09-10T04:05:27.764 INFO:tasks.workunit.client.0.smithi046.stdout:6/625: truncate da/db/d11/d13/f57 4295125 0 2024-09-10T04:05:27.765 INFO:tasks.workunit.client.0.smithi046.stdout:6/626: chown da/db/d11/d13/d1e/lb5 1945 1 2024-09-10T04:05:27.765 INFO:tasks.workunit.client.1.smithi070.stdout:5/784: symlink d17/d6f/d32/d90/da5/l10e 0 2024-09-10T04:05:27.765 INFO:tasks.workunit.client.1.smithi070.stdout:9/821: write d9/d7a/d74/fc6 [705214,94414] 0 2024-09-10T04:05:27.766 INFO:tasks.workunit.client.1.smithi070.stdout:4/722: write de/d13/d18/d23/d39/f57 [654862,4707] 0 2024-09-10T04:05:27.766 INFO:tasks.workunit.client.1.smithi070.stdout:6/775: dread d0/d3/d18/f52 [0,4194304] 0 2024-09-10T04:05:27.768 INFO:tasks.workunit.client.0.smithi046.stdout:9/696: mkdir d4/d18/d89/ddb 0 2024-09-10T04:05:27.768 INFO:tasks.workunit.client.1.smithi070.stdout:0/919: write d8/d13/f3d [3184403,27123] 0 2024-09-10T04:05:27.770 INFO:tasks.workunit.client.1.smithi070.stdout:0/920: fsync d8/d85/df5/f10e 0 2024-09-10T04:05:27.772 INFO:tasks.workunit.client.1.smithi070.stdout:0/921: write d8/d13/f1f [2436315,3709] 0 2024-09-10T04:05:27.772 INFO:tasks.workunit.client.0.smithi046.stdout:2/676: creat d2/d6/d23/d5d/d77/ff5 x:0 0 0 2024-09-10T04:05:27.774 INFO:tasks.workunit.client.0.smithi046.stdout:7/771: symlink d5/d9/d43/db4/def/l110 0 2024-09-10T04:05:27.775 INFO:tasks.workunit.client.1.smithi070.stdout:1/698: getdents d3/d8/d12/d2f/d32/da8 0 2024-09-10T04:05:27.775 INFO:tasks.workunit.client.0.smithi046.stdout:7/772: write d5/d9/ffa [996294,63032] 0 2024-09-10T04:05:27.776 INFO:tasks.workunit.client.0.smithi046.stdout:5/871: creat d7/d11/d3f/d5a/f130 x:0 0 0 2024-09-10T04:05:27.776 INFO:tasks.workunit.client.1.smithi070.stdout:9/822: dwrite d9/d1c/dac/d8a/f20 [4194304,4194304] 0 2024-09-10T04:05:27.776 INFO:tasks.workunit.client.1.smithi070.stdout:6/776: dwrite d0/d3/f99 [0,4194304] 0 2024-09-10T04:05:27.777 INFO:tasks.workunit.client.1.smithi070.stdout:9/823: chown d9/d7a/f51 677 1 2024-09-10T04:05:27.778 INFO:tasks.workunit.client.0.smithi046.stdout:8/696: symlink d1/d29/d19/d36/d4c/d5e/dc7/le9 0 2024-09-10T04:05:27.778 INFO:tasks.workunit.client.0.smithi046.stdout:9/697: symlink d4/d3c/d4d/d80/ldc 0 2024-09-10T04:05:27.782 INFO:tasks.workunit.client.1.smithi070.stdout:5/785: creat d17/d19/d6a/d54/d57/da8/f10f x:0 0 0 2024-09-10T04:05:27.785 INFO:tasks.workunit.client.0.smithi046.stdout:3/698: creat d12/d13/d37/d51/db5/d4b/fef x:0 0 0 2024-09-10T04:05:27.788 INFO:tasks.workunit.client.0.smithi046.stdout:7/773: write d5/d6/d32/dab/fb1 [1602776,95002] 0 2024-09-10T04:05:27.795 INFO:tasks.workunit.client.1.smithi070.stdout:0/922: fdatasync d8/d24/d2d/d34/d6b/ff3 0 2024-09-10T04:05:27.797 INFO:tasks.workunit.client.1.smithi070.stdout:4/723: symlink de/d13/lf1 0 2024-09-10T04:05:27.798 INFO:tasks.workunit.client.1.smithi070.stdout:4/724: chown de/d13/d18/d26/f2d 28 1 2024-09-10T04:05:27.799 INFO:tasks.workunit.client.0.smithi046.stdout:9/698: dread d4/fa1 [0,4194304] 0 2024-09-10T04:05:27.800 INFO:tasks.workunit.client.1.smithi070.stdout:6/777: creat d0/d3/d18/d2a/ff6 x:0 0 0 2024-09-10T04:05:27.801 INFO:tasks.workunit.client.0.smithi046.stdout:3/699: creat d12/d13/d37/d51/db5/d50/d7a/dc1/d96/ff0 x:0 0 0 2024-09-10T04:05:27.802 INFO:tasks.workunit.client.0.smithi046.stdout:3/700: stat d12/d5d/d71/c66 0 2024-09-10T04:05:27.806 INFO:tasks.workunit.client.1.smithi070.stdout:9/824: dwrite d9/d1c/dac/d8a/d3c/d92/fb1 [4194304,4194304] 0 2024-09-10T04:05:27.807 INFO:tasks.workunit.client.0.smithi046.stdout:9/699: dread d4/d3c/f40 [0,4194304] 0 2024-09-10T04:05:27.808 INFO:tasks.workunit.client.1.smithi070.stdout:3/784: truncate db/d1c/d25/d28/f106 592062 0 2024-09-10T04:05:27.808 INFO:tasks.workunit.client.0.smithi046.stdout:9/700: readlink l1 0 2024-09-10T04:05:27.809 INFO:tasks.workunit.client.1.smithi070.stdout:5/786: creat d17/d19/d6a/d54/de1/dfe/dce/f110 x:0 0 0 2024-09-10T04:05:27.809 INFO:tasks.workunit.client.0.smithi046.stdout:2/677: write d2/d6/d23/d55/f7e [171188,39225] 0 2024-09-10T04:05:27.811 INFO:tasks.workunit.client.0.smithi046.stdout:8/697: dwrite d1/f2e [4194304,4194304] 0 2024-09-10T04:05:27.812 INFO:tasks.workunit.client.1.smithi070.stdout:8/823: rename d4/d16/d17 to d4/dbf/d74/d11b 0 2024-09-10T04:05:27.813 INFO:tasks.workunit.client.0.smithi046.stdout:8/698: write d1/d29/d17/d1f/d2a/f58 [2997398,3507] 0 2024-09-10T04:05:27.814 INFO:tasks.workunit.client.0.smithi046.stdout:8/699: readlink d1/d29/la0 0 2024-09-10T04:05:27.814 INFO:tasks.workunit.client.1.smithi070.stdout:0/923: creat d8/d24/d2d/d34/f133 x:0 0 0 2024-09-10T04:05:27.815 INFO:tasks.workunit.client.0.smithi046.stdout:8/700: chown d1/d29/fd7 192165461 1 2024-09-10T04:05:27.815 INFO:tasks.workunit.client.0.smithi046.stdout:5/872: link d7/d11/d14/d2f/l77 d7/d11/d3f/d5a/l131 0 2024-09-10T04:05:27.817 INFO:tasks.workunit.client.1.smithi070.stdout:1/699: symlink d3/d8/d13/d3f/d6d/dbe/le6 0 2024-09-10T04:05:27.818 INFO:tasks.workunit.client.0.smithi046.stdout:5/873: chown d7/d11/d3f/d5a/l61 244507048 1 2024-09-10T04:05:27.818 INFO:tasks.workunit.client.1.smithi070.stdout:1/700: fsync d3/d8/d12/d2f/d32/da8/fd0 0 2024-09-10T04:05:27.818 INFO:tasks.workunit.client.0.smithi046.stdout:9/701: creat d4/d7/d1f/d54/dc7/fdd x:0 0 0 2024-09-10T04:05:27.819 INFO:tasks.workunit.client.1.smithi070.stdout:4/725: read de/d13/d5b/d7b/dd5/f60 [562630,33813] 0 2024-09-10T04:05:27.820 INFO:tasks.workunit.client.1.smithi070.stdout:4/726: write de/d13/d18/d23/d39/d48/fab [3149764,43606] 0 2024-09-10T04:05:27.821 INFO:tasks.workunit.client.0.smithi046.stdout:2/678: symlink d2/d6/d23/d42/de6/lf6 0 2024-09-10T04:05:27.823 INFO:tasks.workunit.client.0.smithi046.stdout:3/701: creat d12/d5d/d71/db1/ff1 x:0 0 0 2024-09-10T04:05:27.825 INFO:tasks.workunit.client.1.smithi070.stdout:4/727: dread de/d13/d18/d23/fc6 [0,4194304] 0 2024-09-10T04:05:27.825 INFO:tasks.workunit.client.1.smithi070.stdout:9/825: unlink d9/d7a/f51 0 2024-09-10T04:05:27.826 INFO:tasks.workunit.client.1.smithi070.stdout:2/742: sync 2024-09-10T04:05:27.826 INFO:tasks.workunit.client.0.smithi046.stdout:7/774: mknod d5/d9/d12/d72/d77/de0/d9f/c111 0 2024-09-10T04:05:27.829 INFO:tasks.workunit.client.0.smithi046.stdout:8/701: mkdir d1/d29/d19/d36/d4c/d5e/dc7/dea 0 2024-09-10T04:05:27.831 INFO:tasks.workunit.client.0.smithi046.stdout:7/775: dread d5/d6/d32/dab/db6/f79 [4194304,4194304] 0 2024-09-10T04:05:27.832 INFO:tasks.workunit.client.0.smithi046.stdout:7/776: write d5/f6b [1576038,51039] 0 2024-09-10T04:05:27.834 INFO:tasks.workunit.client.1.smithi070.stdout:8/824: truncate d4/dbf/fb6 1024091 0 2024-09-10T04:05:27.834 INFO:tasks.workunit.client.1.smithi070.stdout:8/825: dread - d4/dbf/d87/ff2 zero size 2024-09-10T04:05:27.836 INFO:tasks.workunit.client.0.smithi046.stdout:2/679: truncate d2/d8/d37/f41 6973562 0 2024-09-10T04:05:27.837 INFO:tasks.workunit.client.1.smithi070.stdout:6/778: write d0/d3/d18/d1b/d17/d53/fa6 [3565365,38015] 0 2024-09-10T04:05:27.837 INFO:tasks.workunit.client.0.smithi046.stdout:2/680: fdatasync d2/d6/d9d/fa5 0 2024-09-10T04:05:27.839 INFO:tasks.workunit.client.0.smithi046.stdout:1/533: sync 2024-09-10T04:05:27.839 INFO:tasks.workunit.client.0.smithi046.stdout:4/857: sync 2024-09-10T04:05:27.840 INFO:tasks.workunit.client.0.smithi046.stdout:0/569: sync 2024-09-10T04:05:27.840 INFO:tasks.workunit.client.1.smithi070.stdout:1/701: fdatasync d3/d8/d12/f24 0 2024-09-10T04:05:27.843 INFO:tasks.workunit.client.0.smithi046.stdout:8/702: creat d1/d29/d19/db7/feb x:0 0 0 2024-09-10T04:05:27.847 INFO:tasks.workunit.client.1.smithi070.stdout:5/787: write d17/d6f/f52 [2639070,66966] 0 2024-09-10T04:05:27.847 INFO:tasks.workunit.client.0.smithi046.stdout:5/874: dread d7/d11/d3f/d53/f9b [0,4194304] 0 2024-09-10T04:05:27.848 INFO:tasks.workunit.client.1.smithi070.stdout:5/788: stat d17/d19/d6a/d54/d57/da3/ffd 0 2024-09-10T04:05:27.848 INFO:tasks.workunit.client.0.smithi046.stdout:9/702: mknod d4/d18/cde 0 2024-09-10T04:05:27.849 INFO:tasks.workunit.client.1.smithi070.stdout:7/818: link d2/d3/dc/d5d/d42/fc4 d2/d3/dc/d5d/d41/dd6/de6/f120 0 2024-09-10T04:05:27.851 INFO:tasks.workunit.client.0.smithi046.stdout:9/703: dread - d4/d18/da8/fbc zero size 2024-09-10T04:05:27.851 INFO:tasks.workunit.client.0.smithi046.stdout:5/875: write d7/d11/d14/d2f/deb/da5/fb6 [3307506,70642] 0 2024-09-10T04:05:27.851 INFO:tasks.workunit.client.1.smithi070.stdout:4/728: fdatasync de/d13/d18/d23/d39/d48/feb 0 2024-09-10T04:05:27.853 INFO:tasks.workunit.client.0.smithi046.stdout:8/703: dwrite d1/d29/d17/d1f/d46/d6a/d82/d99/fa5 [0,4194304] 0 2024-09-10T04:05:27.857 INFO:tasks.workunit.client.0.smithi046.stdout:2/681: chown d2/d6/d23/d5d/d5e/cd7 2278 1 2024-09-10T04:05:27.857 INFO:tasks.workunit.client.1.smithi070.stdout:2/743: unlink d15/d18/d29/d2a/d2e/d32/d36/l93 0 2024-09-10T04:05:27.857 INFO:tasks.workunit.client.1.smithi070.stdout:9/826: dwrite d9/d1c/dfd/fdd [0,4194304] 0 2024-09-10T04:05:27.860 INFO:tasks.workunit.client.1.smithi070.stdout:3/785: creat db/d1c/d53/f10e x:0 0 0 2024-09-10T04:05:27.860 INFO:tasks.workunit.client.0.smithi046.stdout:4/858: dread - d3/fe3 zero size 2024-09-10T04:05:27.864 INFO:tasks.workunit.client.0.smithi046.stdout:9/704: dread d4/d7/f8 [0,4194304] 0 2024-09-10T04:05:27.865 INFO:tasks.workunit.client.1.smithi070.stdout:8/826: rename d4/dbf/l7e to d4/dbf/d74/d11b/d19/d62/l11c 0 2024-09-10T04:05:27.868 INFO:tasks.workunit.client.0.smithi046.stdout:0/570: truncate d1/dc/d25/d33/d1a/d41/f84 436655 0 2024-09-10T04:05:27.869 INFO:tasks.workunit.client.0.smithi046.stdout:1/534: write d3/ff [4868783,107431] 0 2024-09-10T04:05:27.869 INFO:tasks.workunit.client.1.smithi070.stdout:6/779: chown d0/d3/d5c/c7a 0 1 2024-09-10T04:05:27.869 INFO:tasks.workunit.client.1.smithi070.stdout:9/827: dread d9/d1c/dac/d8a/dc4/de1/fc3 [0,4194304] 0 2024-09-10T04:05:27.871 INFO:tasks.workunit.client.1.smithi070.stdout:9/828: truncate d9/d59/d2a/f101 732190 0 2024-09-10T04:05:27.871 INFO:tasks.workunit.client.1.smithi070.stdout:1/702: fsync d3/d8/d27/f93 0 2024-09-10T04:05:27.872 INFO:tasks.workunit.client.0.smithi046.stdout:7/777: mkdir d5/d9/d112 0 2024-09-10T04:05:27.875 INFO:tasks.workunit.client.0.smithi046.stdout:9/705: dread d4/d7/fe [0,4194304] 0 2024-09-10T04:05:27.876 INFO:tasks.workunit.client.0.smithi046.stdout:3/702: rename d12/d13/d37/d51/f62 to d12/d13/ff2 0 2024-09-10T04:05:27.879 INFO:tasks.workunit.client.0.smithi046.stdout:2/682: mknod d2/d6/d23/d5d/d77/dac/cf7 0 2024-09-10T04:05:27.882 INFO:tasks.workunit.client.1.smithi070.stdout:4/729: read de/d13/d5b/da0/fb4 [1614719,42519] 0 2024-09-10T04:05:27.884 INFO:tasks.workunit.client.0.smithi046.stdout:4/859: creat d3/d12/d17/d29/d58/db2/d5f/d85/da8/f12d x:0 0 0 2024-09-10T04:05:27.885 INFO:tasks.workunit.client.1.smithi070.stdout:2/744: dread - d15/d18/d29/d2a/d2e/d32/fe5 zero size 2024-09-10T04:05:27.888 INFO:tasks.workunit.client.0.smithi046.stdout:5/876: dread d7/d11/f23 [0,4194304] 0 2024-09-10T04:05:27.891 INFO:tasks.workunit.client.1.smithi070.stdout:0/924: rename d8/d85/df5/fae to d8/d13/de2/d6f/f134 0 2024-09-10T04:05:27.892 INFO:tasks.workunit.client.1.smithi070.stdout:0/925: chown d8/d24/f6d 1077889 1 2024-09-10T04:05:27.893 INFO:tasks.workunit.client.0.smithi046.stdout:0/571: creat d1/dc/d25/d33/d3/dd/d55/d77/da6/fb3 x:0 0 0 2024-09-10T04:05:27.894 INFO:tasks.workunit.client.1.smithi070.stdout:8/827: symlink d4/d16/d73/d96/l11d 0 2024-09-10T04:05:27.895 INFO:tasks.workunit.client.0.smithi046.stdout:7/778: read d5/d9/d43/f5c [154481,5726] 0 2024-09-10T04:05:27.895 INFO:tasks.workunit.client.0.smithi046.stdout:8/704: creat d1/d29/d19/d36/d4c/d5e/dc0/fec x:0 0 0 2024-09-10T04:05:27.896 INFO:tasks.workunit.client.0.smithi046.stdout:7/779: stat d5/d6/db/dd/d4b/f52 0 2024-09-10T04:05:27.896 INFO:tasks.workunit.client.0.smithi046.stdout:8/705: stat d1/d29/d17/d1f/d46/l5d 0 2024-09-10T04:05:27.897 INFO:tasks.workunit.client.0.smithi046.stdout:3/703: truncate d12/d13/d14/d1e/d2d/f4f 1491812 0 2024-09-10T04:05:27.898 INFO:tasks.workunit.client.0.smithi046.stdout:2/683: truncate d2/d6/d23/d55/f67 2911898 0 2024-09-10T04:05:27.899 INFO:tasks.workunit.client.0.smithi046.stdout:3/704: chown d12/d13/d37/d51/db5/d50/d7a/dc1/f97 101696111 1 2024-09-10T04:05:27.900 INFO:tasks.workunit.client.0.smithi046.stdout:4/860: dread d3/d12/d17/d22/f70 [0,4194304] 0 2024-09-10T04:05:27.900 INFO:tasks.workunit.client.1.smithi070.stdout:9/829: creat d9/d1c/dac/d8a/d3c/d92/da8/f109 x:0 0 0 2024-09-10T04:05:27.901 INFO:tasks.workunit.client.1.smithi070.stdout:5/789: mknod d17/d19/d6a/d54/d57/df2/c111 0 2024-09-10T04:05:27.902 INFO:tasks.workunit.client.1.smithi070.stdout:7/819: mknod d2/d3/dc/d79/c121 0 2024-09-10T04:05:27.903 INFO:tasks.workunit.client.1.smithi070.stdout:2/745: mknod d15/d18/db2/c10e 0 2024-09-10T04:05:27.903 INFO:tasks.workunit.client.0.smithi046.stdout:0/572: write d1/dc/d49/fa1 [304924,123175] 0 2024-09-10T04:05:27.906 INFO:tasks.workunit.client.1.smithi070.stdout:2/746: dread d15/df6/ff9 [0,4194304] 0 2024-09-10T04:05:27.907 INFO:tasks.workunit.client.0.smithi046.stdout:8/706: symlink d1/d29/d17/d1f/d46/d6a/d82/dc2/led 0 2024-09-10T04:05:27.907 INFO:tasks.workunit.client.1.smithi070.stdout:2/747: chown d15/d18/db2/ce3 2 1 2024-09-10T04:05:27.907 INFO:tasks.workunit.client.1.smithi070.stdout:2/748: readlink d15/d18/d56/l8c 0 2024-09-10T04:05:27.908 INFO:tasks.workunit.client.0.smithi046.stdout:8/707: chown d1/d29/d19/d36/d4c/f5b 17 1 2024-09-10T04:05:27.908 INFO:tasks.workunit.client.1.smithi070.stdout:4/730: rename de/db0 to de/d13/d5b/d7b/dd5/daf/df2 0 2024-09-10T04:05:27.909 INFO:tasks.workunit.client.1.smithi070.stdout:4/731: dread - de/d13/d18/d23/d39/d77/fc2 zero size 2024-09-10T04:05:27.909 INFO:tasks.workunit.client.1.smithi070.stdout:0/926: symlink d8/d13/d1a/d75/l135 0 2024-09-10T04:05:27.910 INFO:tasks.workunit.client.1.smithi070.stdout:6/780: mknod d0/d3/d18/d62/cf7 0 2024-09-10T04:05:27.911 INFO:tasks.workunit.client.1.smithi070.stdout:6/781: write d0/d3/d8c/ff2 [1048259,103320] 0 2024-09-10T04:05:27.913 INFO:tasks.workunit.client.1.smithi070.stdout:8/828: symlink d4/dbf/d74/d11b/d1a/d60/l11e 0 2024-09-10T04:05:27.914 INFO:tasks.workunit.client.0.smithi046.stdout:1/535: creat d3/fae x:0 0 0 2024-09-10T04:05:27.916 INFO:tasks.workunit.client.0.smithi046.stdout:7/780: creat d5/d6/db/dd/d10c/f113 x:0 0 0 2024-09-10T04:05:27.918 INFO:tasks.workunit.client.0.smithi046.stdout:7/781: truncate d5/d9/d43/db4/def/f105 763962 0 2024-09-10T04:05:27.918 INFO:tasks.workunit.client.1.smithi070.stdout:1/703: link d3/d8/d27/f68 d3/d10/d1d/d79/fe7 0 2024-09-10T04:05:27.920 INFO:tasks.workunit.client.1.smithi070.stdout:9/830: readlink d9/d1c/dac/d8a/l2f 0 2024-09-10T04:05:27.923 INFO:tasks.workunit.client.0.smithi046.stdout:2/684: mkdir d2/d6/d23/dd5/df8 0 2024-09-10T04:05:27.924 INFO:tasks.workunit.client.1.smithi070.stdout:7/820: mkdir d2/dc5/d122 0 2024-09-10T04:05:27.925 INFO:tasks.workunit.client.1.smithi070.stdout:7/821: chown d2/d7/d9/l90 240338522 1 2024-09-10T04:05:27.927 INFO:tasks.workunit.client.1.smithi070.stdout:3/786: unlink db/d1c/d25/d28/d43/lab 0 2024-09-10T04:05:27.930 INFO:tasks.workunit.client.1.smithi070.stdout:2/749: creat d15/d18/d56/d7c/dfa/f10f x:0 0 0 2024-09-10T04:05:27.931 INFO:tasks.workunit.client.0.smithi046.stdout:2/685: dwrite d2/d6/d23/d42/de6/fcf [0,4194304] 0 2024-09-10T04:05:27.933 INFO:tasks.workunit.client.1.smithi070.stdout:0/927: creat d8/d24/d2d/d34/d6b/f136 x:0 0 0 2024-09-10T04:05:27.933 INFO:tasks.workunit.client.0.smithi046.stdout:1/536: chown d3/d27/d5d/d60/f65 3334 1 2024-09-10T04:05:27.934 INFO:tasks.workunit.client.1.smithi070.stdout:4/732: creat de/d13/d5b/d7b/dd5/daf/db7/ff3 x:0 0 0 2024-09-10T04:05:27.935 INFO:tasks.workunit.client.0.smithi046.stdout:0/573: mknod d1/d13/d37/d75/d94/d9a/da4/cb4 0 2024-09-10T04:05:27.941 INFO:tasks.workunit.client.1.smithi070.stdout:2/750: dwrite d15/d18/d29/f7f [0,4194304] 0 2024-09-10T04:05:27.941 INFO:tasks.workunit.client.1.smithi070.stdout:2/751: chown d15/d18/d29/d2a/d2e/da6/ff0 0 1 2024-09-10T04:05:27.942 INFO:tasks.workunit.client.0.smithi046.stdout:7/782: creat d5/d6/db/dd/d24/d76/d9c/f114 x:0 0 0 2024-09-10T04:05:27.943 INFO:tasks.workunit.client.1.smithi070.stdout:5/790: mknod d17/d19/d6a/d54/c112 0 2024-09-10T04:05:27.943 INFO:tasks.workunit.client.0.smithi046.stdout:3/705: write d12/d5d/fae [766211,125341] 0 2024-09-10T04:05:27.944 INFO:tasks.workunit.client.0.smithi046.stdout:0/574: read d1/f35 [1593942,89600] 0 2024-09-10T04:05:27.946 INFO:tasks.workunit.client.0.smithi046.stdout:2/686: readlink d2/d6/d15/l40 0 2024-09-10T04:05:27.947 INFO:tasks.workunit.client.1.smithi070.stdout:7/822: creat d2/d3/dc/d5d/d41/dd6/f123 x:0 0 0 2024-09-10T04:05:27.948 INFO:tasks.workunit.client.0.smithi046.stdout:0/575: fsync d1/dc/d49/fa1 0 2024-09-10T04:05:27.954 INFO:tasks.workunit.client.0.smithi046.stdout:4/861: dwrite d3/d12/d17/d29/f4f [0,4194304] 0 2024-09-10T04:05:27.958 INFO:tasks.workunit.client.1.smithi070.stdout:1/704: dread d3/d8/d13/f18 [0,4194304] 0 2024-09-10T04:05:27.958 INFO:tasks.workunit.client.1.smithi070.stdout:8/829: link d4/dbf/d74/d11b/d6f/db5/fea d4/dbf/d74/d11b/d1a/d60/f11f 0 2024-09-10T04:05:27.959 INFO:tasks.workunit.client.1.smithi070.stdout:8/830: write d4/d39/dfc/d43/d52/fc4 [332622,7105] 0 2024-09-10T04:05:27.962 INFO:tasks.workunit.client.0.smithi046.stdout:4/862: dwrite d3/d12/d17/d29/f38 [0,4194304] 0 2024-09-10T04:05:27.963 INFO:tasks.workunit.client.1.smithi070.stdout:5/791: rmdir d17/d24/d2b 39 2024-09-10T04:05:27.967 INFO:tasks.workunit.client.1.smithi070.stdout:7/823: mknod d2/d40/c124 0 2024-09-10T04:05:27.969 INFO:tasks.workunit.client.1.smithi070.stdout:5/792: dwrite d17/d6f/fc8 [4194304,4194304] 0 2024-09-10T04:05:27.969 INFO:tasks.workunit.client.0.smithi046.stdout:8/708: dwrite d1/f49 [4194304,4194304] 0 2024-09-10T04:05:27.984 INFO:tasks.workunit.client.1.smithi070.stdout:5/793: mknod d17/d19/d6a/d54/d57/d63/dcd/c113 0 2024-09-10T04:05:27.987 INFO:tasks.workunit.client.1.smithi070.stdout:4/733: rename de/d13/d18/d23/d47/d59/fcb to de/d13/d18/d23/d47/ff4 0 2024-09-10T04:05:27.987 INFO:tasks.workunit.client.0.smithi046.stdout:4/863: dread d3/d12/d14/f62 [0,4194304] 0 2024-09-10T04:05:27.988 INFO:tasks.workunit.client.1.smithi070.stdout:7/824: getdents d2/dc5/d122 0 2024-09-10T04:05:27.990 INFO:tasks.workunit.client.1.smithi070.stdout:3/787: getdents db/d79/db5 0 2024-09-10T04:05:27.991 INFO:tasks.workunit.client.1.smithi070.stdout:5/794: rmdir d17/d6f/d32/d90/d99 39 2024-09-10T04:05:27.994 INFO:tasks.workunit.client.1.smithi070.stdout:8/831: dread d4/f99 [0,4194304] 0 2024-09-10T04:05:27.995 INFO:tasks.workunit.client.1.smithi070.stdout:4/734: creat de/d13/d18/d23/d39/d48/ff5 x:0 0 0 2024-09-10T04:05:27.997 INFO:tasks.workunit.client.1.smithi070.stdout:7/825: rename d2/d7/d9/d18/d23/l88 to d2/d3/dc/d5d/d41/l125 0 2024-09-10T04:05:27.997 INFO:tasks.workunit.client.1.smithi070.stdout:0/928: dwrite d8/d24/d2d/d34/d6b/ff3 [0,4194304] 0 2024-09-10T04:05:27.998 INFO:tasks.workunit.client.1.smithi070.stdout:8/832: read d4/dbf/d74/d11b/d6f/db5/f110 [381094,43780] 0 2024-09-10T04:05:28.000 INFO:tasks.workunit.client.1.smithi070.stdout:5/795: rmdir d17/d19/d6a/d54/d57 39 2024-09-10T04:05:28.003 INFO:tasks.workunit.client.1.smithi070.stdout:4/735: creat de/d13/d18/d23/d47/d59/ff6 x:0 0 0 2024-09-10T04:05:28.005 INFO:tasks.workunit.client.1.smithi070.stdout:7/826: read - d2/d3/dc/d5d/d41/da8/fcf zero size 2024-09-10T04:05:28.005 INFO:tasks.workunit.client.1.smithi070.stdout:5/796: read d17/d19/f44 [543421,90225] 0 2024-09-10T04:05:28.005 INFO:tasks.workunit.client.1.smithi070.stdout:0/929: dread d8/d13/d47/f5c [0,4194304] 0 2024-09-10T04:05:28.006 INFO:tasks.workunit.client.1.smithi070.stdout:3/788: rename db/d79 to db/d1c/d25/d28/dd3/de5/d10f 0 2024-09-10T04:05:28.009 INFO:tasks.workunit.client.1.smithi070.stdout:1/705: dwrite d3/d10/f2c [0,4194304] 0 2024-09-10T04:05:28.013 INFO:tasks.workunit.client.1.smithi070.stdout:4/736: creat de/d13/d18/d5c/db6/ff7 x:0 0 0 2024-09-10T04:05:28.014 INFO:tasks.workunit.client.1.smithi070.stdout:0/930: dwrite d8/d24/d2d/d34/f133 [0,4194304] 0 2024-09-10T04:05:28.015 INFO:tasks.workunit.client.1.smithi070.stdout:4/737: chown de/d13/d5b/d7b/dd5/daf/db7/ff3 272 1 2024-09-10T04:05:28.019 INFO:tasks.workunit.client.1.smithi070.stdout:8/833: unlink d4/dbf/d74/d11b/d1a/d60/d7a/f7b 0 2024-09-10T04:05:28.020 INFO:tasks.workunit.client.0.smithi046.stdout:3/706: mknod d12/d13/d37/d51/cf3 0 2024-09-10T04:05:28.025 INFO:tasks.workunit.client.1.smithi070.stdout:1/706: rmdir d3/d10/d1d/dbd 39 2024-09-10T04:05:28.028 INFO:tasks.workunit.client.0.smithi046.stdout:2/687: creat d2/d6/d23/d5d/d77/d7b/dc4/ff9 x:0 0 0 2024-09-10T04:05:28.030 INFO:tasks.workunit.client.1.smithi070.stdout:0/931: mkdir d8/d24/d36/d137 0 2024-09-10T04:05:28.030 INFO:tasks.workunit.client.1.smithi070.stdout:0/932: chown d8/d13/l68 8543199 1 2024-09-10T04:05:28.032 INFO:tasks.workunit.client.0.smithi046.stdout:0/576: fdatasync d1/dc/d25/d33/d4/f36 0 2024-09-10T04:05:28.035 INFO:tasks.workunit.client.1.smithi070.stdout:0/933: dwrite f5 [4194304,4194304] 0 2024-09-10T04:05:28.038 INFO:tasks.workunit.client.1.smithi070.stdout:3/789: mkdir db/d1c/d25/d28/dd3/de5/d10f/d110 0 2024-09-10T04:05:28.041 INFO:tasks.workunit.client.1.smithi070.stdout:7/827: creat d2/dc8/d119/f126 x:0 0 0 2024-09-10T04:05:28.044 INFO:tasks.workunit.client.1.smithi070.stdout:4/738: symlink de/d13/lf8 0 2024-09-10T04:05:28.045 INFO:tasks.workunit.client.1.smithi070.stdout:0/934: dwrite d8/d13/d47/d7b/f8c [0,4194304] 0 2024-09-10T04:05:28.048 INFO:tasks.workunit.client.0.smithi046.stdout:3/707: symlink d12/d13/d37/dab/dac/dd3/lf4 0 2024-09-10T04:05:28.051 INFO:tasks.workunit.client.1.smithi070.stdout:5/797: dwrite d17/d19/d6a/fd2 [0,4194304] 0 2024-09-10T04:05:28.054 INFO:tasks.workunit.client.0.smithi046.stdout:0/577: dread d1/dc/d25/d33/d4/f36 [0,4194304] 0 2024-09-10T04:05:28.055 INFO:tasks.workunit.client.0.smithi046.stdout:1/537: rmdir d3/d27/d5d/da4 0 2024-09-10T04:05:28.061 INFO:tasks.workunit.client.1.smithi070.stdout:8/834: creat d4/d39/dfc/d43/d67/f120 x:0 0 0 2024-09-10T04:05:28.064 INFO:tasks.workunit.client.0.smithi046.stdout:8/709: symlink d1/d29/d19/d36/d4c/dd3/lee 0 2024-09-10T04:05:28.065 INFO:tasks.workunit.client.0.smithi046.stdout:4/864: write d3/d12/d17/d22/d51/d5c/f76 [2119427,7075] 0 2024-09-10T04:05:28.066 INFO:tasks.workunit.client.1.smithi070.stdout:1/707: dwrite d3/d8/d13/d3f/d6d/f8e [0,4194304] 0 2024-09-10T04:05:28.066 INFO:tasks.workunit.client.0.smithi046.stdout:7/783: link d5/d6/d39/d5f/cda d5/d9/d43/db4/c115 0 2024-09-10T04:05:28.069 INFO:tasks.workunit.client.1.smithi070.stdout:5/798: truncate d17/d24/d45/f98 1158406 0 2024-09-10T04:05:28.070 INFO:tasks.workunit.client.1.smithi070.stdout:7/828: mkdir d2/d3/dc/d79/d11d/d127 0 2024-09-10T04:05:28.071 INFO:tasks.workunit.client.1.smithi070.stdout:5/799: fdatasync d17/d6f/d32/f68 0 2024-09-10T04:05:28.072 INFO:tasks.workunit.client.1.smithi070.stdout:7/829: readlink d2/d7/d9/dd5/d102/l10b 0 2024-09-10T04:05:28.074 INFO:tasks.workunit.client.1.smithi070.stdout:7/830: write d2/d7/d9/d18/f99 [3974306,58220] 0 2024-09-10T04:05:28.075 INFO:tasks.workunit.client.1.smithi070.stdout:4/739: dread de/d13/d43/f6b [0,4194304] 0 2024-09-10T04:05:28.080 INFO:tasks.workunit.client.0.smithi046.stdout:0/578: mknod d1/dc/cb5 0 2024-09-10T04:05:28.081 INFO:tasks.workunit.client.1.smithi070.stdout:3/790: truncate db/d1c/fe2 783656 0 2024-09-10T04:05:28.083 INFO:tasks.workunit.client.0.smithi046.stdout:0/579: truncate d1/dc/d25/d33/d3/dd/d55/d77/da6/fb3 611794 0 2024-09-10T04:05:28.084 INFO:tasks.workunit.client.1.smithi070.stdout:6/782: sync 2024-09-10T04:05:28.084 INFO:tasks.workunit.client.0.smithi046.stdout:2/688: rename d2/d6/d23/d42/db4/dca/dd6/fe3 to d2/d6/d15/ffa 0 2024-09-10T04:05:28.085 INFO:tasks.workunit.client.1.smithi070.stdout:5/800: mkdir d17/d24/d84/dab/d114 0 2024-09-10T04:05:28.086 INFO:tasks.workunit.client.1.smithi070.stdout:5/801: readlink d17/l10c 0 2024-09-10T04:05:28.089 INFO:tasks.workunit.client.1.smithi070.stdout:8/835: mknod d4/d39/dfc/d43/d67/dec/d111/c121 0 2024-09-10T04:05:28.090 INFO:tasks.workunit.client.0.smithi046.stdout:1/538: dread d3/dc/f17 [4194304,4194304] 0 2024-09-10T04:05:28.090 INFO:tasks.workunit.client.0.smithi046.stdout:7/784: mkdir d5/d9/d12/d72/d77/de0/d1f/d116 0 2024-09-10T04:05:28.092 INFO:tasks.workunit.client.1.smithi070.stdout:5/802: dwrite d17/d19/fd1 [0,4194304] 0 2024-09-10T04:05:28.092 INFO:tasks.workunit.client.1.smithi070.stdout:7/831: write d2/d3/dc/d5d/d2d/fba [2839392,94032] 0 2024-09-10T04:05:28.094 INFO:tasks.workunit.client.0.smithi046.stdout:4/865: link d3/d12/d17/d22/d51/d5c/f76 d3/d12/d17/de1/ddf/dfc/f12e 0 2024-09-10T04:05:28.096 INFO:tasks.workunit.client.0.smithi046.stdout:4/866: dread d3/d12/d17/d22/f70 [0,4194304] 0 2024-09-10T04:05:28.099 INFO:tasks.workunit.client.1.smithi070.stdout:4/740: creat de/d13/d43/d88/ff9 x:0 0 0 2024-09-10T04:05:28.101 INFO:tasks.workunit.client.0.smithi046.stdout:8/710: truncate d1/d29/d12/f57 314848 0 2024-09-10T04:05:28.103 INFO:tasks.workunit.client.0.smithi046.stdout:8/711: write d1/d29/d19/d36/d4c/d5e/fdb [493602,70652] 0 2024-09-10T04:05:28.104 INFO:tasks.workunit.client.0.smithi046.stdout:2/689: mknod d2/d6/d9d/cfb 0 2024-09-10T04:05:28.107 INFO:tasks.workunit.client.0.smithi046.stdout:7/785: rmdir d5/d9/d12/d72/d77/de0/d1e 39 2024-09-10T04:05:28.111 INFO:tasks.workunit.client.1.smithi070.stdout:8/836: mkdir d4/d16/dcc/d122 0 2024-09-10T04:05:28.112 INFO:tasks.workunit.client.1.smithi070.stdout:8/837: write d4/d39/dfc/d43/f5d [3231647,6645] 0 2024-09-10T04:05:28.113 INFO:tasks.workunit.client.0.smithi046.stdout:4/867: dread d3/d12/d17/d29/dba/fd5 [0,4194304] 0 2024-09-10T04:05:28.113 INFO:tasks.workunit.client.0.smithi046.stdout:3/708: rename d12/d13/f18 to d12/d13/d37/d51/db5/d4b/db7/ff5 0 2024-09-10T04:05:28.119 INFO:tasks.workunit.client.0.smithi046.stdout:8/712: mkdir d1/d29/d17/d1f/d46/d67/dc1/d9d/def 0 2024-09-10T04:05:28.122 INFO:tasks.workunit.client.1.smithi070.stdout:6/783: write d0/d3/d18/d2a/f59 [2634265,34288] 0 2024-09-10T04:05:28.123 INFO:tasks.workunit.client.1.smithi070.stdout:4/741: mknod de/d13/d5b/da0/cfa 0 2024-09-10T04:05:28.124 INFO:tasks.workunit.client.0.smithi046.stdout:0/580: dwrite d1/d13/d37/d75/f7f [0,4194304] 0 2024-09-10T04:05:28.126 INFO:tasks.workunit.client.0.smithi046.stdout:7/786: truncate d5/d6/d32/dab/db6/fa5 556896 0 2024-09-10T04:05:28.128 INFO:tasks.workunit.client.0.smithi046.stdout:7/787: read - d5/d6/db/dd/d24/db7/fe9 zero size 2024-09-10T04:05:28.130 INFO:tasks.workunit.client.0.smithi046.stdout:4/868: dread d3/d12/d17/d29/d58/db2/d5f/d85/f115 [0,4194304] 0 2024-09-10T04:05:28.133 INFO:tasks.workunit.client.1.smithi070.stdout:5/803: dwrite f8 [0,4194304] 0 2024-09-10T04:05:28.134 INFO:tasks.workunit.client.1.smithi070.stdout:5/804: stat d17/d6f/d32/c43 0 2024-09-10T04:05:28.135 INFO:tasks.workunit.client.0.smithi046.stdout:8/713: dread d1/d29/d17/d1f/d46/f8b [0,4194304] 0 2024-09-10T04:05:28.135 INFO:tasks.workunit.client.0.smithi046.stdout:4/869: dread d3/d12/d17/d105/f113 [0,4194304] 0 2024-09-10T04:05:28.136 INFO:tasks.workunit.client.0.smithi046.stdout:8/714: chown d1/d29/d19/d36/d6e/d75/l92 8 1 2024-09-10T04:05:28.140 INFO:tasks.workunit.client.1.smithi070.stdout:0/935: getdents d8/d13/de2/d4f/d11b 0 2024-09-10T04:05:28.141 INFO:tasks.workunit.client.0.smithi046.stdout:0/581: fdatasync d1/dc/d25/d33/d1a/f66 0 2024-09-10T04:05:28.146 INFO:tasks.workunit.client.1.smithi070.stdout:4/742: unlink de/d13/d18/d23/d39/d77/faa 0 2024-09-10T04:05:28.146 INFO:tasks.workunit.client.0.smithi046.stdout:3/709: dwrite d12/d13/d14/f36 [0,4194304] 0 2024-09-10T04:05:28.147 INFO:tasks.workunit.client.1.smithi070.stdout:4/743: chown de/d13/d18/d26/d8c/d93/lb2 0 1 2024-09-10T04:05:28.149 INFO:tasks.workunit.client.1.smithi070.stdout:1/708: getdents d3/d8/d13/d67/d83 0 2024-09-10T04:05:28.149 INFO:tasks.workunit.client.1.smithi070.stdout:3/791: link db/d1c/d53/f95 db/d1c/d25/d28/dd3/f111 0 2024-09-10T04:05:28.152 INFO:tasks.workunit.client.1.smithi070.stdout:6/784: write d0/d3/d18/d62/db8/fc5 [318065,62764] 0 2024-09-10T04:05:28.152 INFO:tasks.workunit.client.1.smithi070.stdout:0/936: dread d8/d13/d1a/f7e [0,4194304] 0 2024-09-10T04:05:28.155 INFO:tasks.workunit.client.1.smithi070.stdout:3/792: dread db/ff8 [0,4194304] 0 2024-09-10T04:05:28.156 INFO:tasks.workunit.client.0.smithi046.stdout:1/539: link d3/dc/f88 d3/dc/d21/d3a/faf 0 2024-09-10T04:05:28.156 INFO:tasks.workunit.client.1.smithi070.stdout:3/793: chown db/d1c/d25 1451 1 2024-09-10T04:05:28.159 INFO:tasks.workunit.client.1.smithi070.stdout:3/794: chown db/d1c/d25/d28/dd3/de5/d10f/d94/d99/fa5 62933 1 2024-09-10T04:05:28.161 INFO:tasks.workunit.client.0.smithi046.stdout:2/690: link d2/d6/d23/d42/db4/ld3 d2/d6/d23/d5d/d77/d7b/dc4/lfc 0 2024-09-10T04:05:28.162 INFO:tasks.workunit.client.1.smithi070.stdout:8/838: dwrite d4/dbf/d74/d11b/d1a/d60/d7a/d84/ffe [0,4194304] 0 2024-09-10T04:05:28.162 INFO:tasks.workunit.client.0.smithi046.stdout:6/627: sync 2024-09-10T04:05:28.162 INFO:tasks.workunit.client.0.smithi046.stdout:8/715: read d1/d29/fe [4294906,74480] 0 2024-09-10T04:05:28.163 INFO:tasks.workunit.client.1.smithi070.stdout:1/709: mknod d3/d90/ce8 0 2024-09-10T04:05:28.164 INFO:tasks.workunit.client.1.smithi070.stdout:5/805: dread d17/d24/d45/f96 [0,4194304] 0 2024-09-10T04:05:28.166 INFO:tasks.workunit.client.1.smithi070.stdout:0/937: fdatasync d8/d13/d1a/d8f/d95/fcc 0 2024-09-10T04:05:28.168 INFO:tasks.workunit.client.1.smithi070.stdout:0/938: write d8/d85/df5/fef [271956,21059] 0 2024-09-10T04:05:28.170 INFO:tasks.workunit.client.0.smithi046.stdout:8/716: dwrite d1/f49 [0,4194304] 0 2024-09-10T04:05:28.171 INFO:tasks.workunit.client.1.smithi070.stdout:6/785: unlink d0/d3/da4/dc0/dea/c5d 0 2024-09-10T04:05:28.173 INFO:tasks.workunit.client.0.smithi046.stdout:7/788: mknod d5/d9/d112/c117 0 2024-09-10T04:05:28.174 INFO:tasks.workunit.client.1.smithi070.stdout:4/744: dread de/d13/d18/d23/d39/d48/fee [0,4194304] 0 2024-09-10T04:05:28.175 INFO:tasks.workunit.client.0.smithi046.stdout:7/789: truncate d5/d9/d43/db4/fc8 889206 0 2024-09-10T04:05:28.177 INFO:tasks.workunit.client.0.smithi046.stdout:7/790: write d5/d6/dbe/dd7/f101 [120323,70956] 0 2024-09-10T04:05:28.180 INFO:tasks.workunit.client.1.smithi070.stdout:3/795: write db/d1c/d89/f9f [5212416,49300] 0 2024-09-10T04:05:28.181 INFO:tasks.workunit.client.1.smithi070.stdout:1/710: dread - d3/d8/d13/d3f/f78 zero size 2024-09-10T04:05:28.184 INFO:tasks.workunit.client.1.smithi070.stdout:4/745: dwrite f0 [4194304,4194304] 0 2024-09-10T04:05:28.185 INFO:tasks.workunit.client.0.smithi046.stdout:2/691: mknod d2/d6/d23/d42/cfd 0 2024-09-10T04:05:28.188 INFO:tasks.workunit.client.0.smithi046.stdout:6/628: symlink da/d45/ld0 0 2024-09-10T04:05:28.192 INFO:tasks.workunit.client.1.smithi070.stdout:0/939: rename d8/d13/c70 to d8/d13/de2/d7d/d8a/c138 0 2024-09-10T04:05:28.193 INFO:tasks.workunit.client.1.smithi070.stdout:0/940: fsync d8/f49 0 2024-09-10T04:05:28.196 INFO:tasks.workunit.client.0.smithi046.stdout:0/582: dwrite d1/dc/d25/d33/d3/f46 [0,4194304] 0 2024-09-10T04:05:28.199 INFO:tasks.workunit.client.1.smithi070.stdout:6/786: unlink d0/d3/d18/d1b/d17/d54/faf 0 2024-09-10T04:05:28.201 INFO:tasks.workunit.client.0.smithi046.stdout:7/791: dread d5/d9/d12/d72/d77/de0/fe3 [4194304,4194304] 0 2024-09-10T04:05:28.201 INFO:tasks.workunit.client.1.smithi070.stdout:4/746: dread de/d13/d18/d23/d47/f51 [0,4194304] 0 2024-09-10T04:05:28.202 INFO:tasks.workunit.client.1.smithi070.stdout:3/796: unlink db/d1c/d25/f104 0 2024-09-10T04:05:28.205 INFO:tasks.workunit.client.0.smithi046.stdout:2/692: dread d2/d49/f9b [0,4194304] 0 2024-09-10T04:05:28.212 INFO:tasks.workunit.client.0.smithi046.stdout:4/870: dwrite d3/d12/d17/d22/f59 [0,4194304] 0 2024-09-10T04:05:28.212 INFO:tasks.workunit.client.0.smithi046.stdout:9/706: sync 2024-09-10T04:05:28.218 INFO:tasks.workunit.client.0.smithi046.stdout:8/717: link d1/d29/d19/d36/f4f d1/d29/d17/d1f/d46/d76/ff0 0 2024-09-10T04:05:28.219 INFO:tasks.workunit.client.1.smithi070.stdout:9/831: sync 2024-09-10T04:05:28.219 INFO:tasks.workunit.client.1.smithi070.stdout:2/752: sync 2024-09-10T04:05:28.223 INFO:tasks.workunit.client.1.smithi070.stdout:8/839: truncate d4/d39/dc6/f10b 2500382 0 2024-09-10T04:05:28.225 INFO:tasks.workunit.client.0.smithi046.stdout:2/693: truncate d2/d8/f29 2286685 0 2024-09-10T04:05:28.226 INFO:tasks.workunit.client.1.smithi070.stdout:5/806: dwrite d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:28.227 INFO:tasks.workunit.client.1.smithi070.stdout:5/807: write d17/d19/d6a/d54/de1/dfe/dce/f100 [1040000,120814] 0 2024-09-10T04:05:28.228 INFO:tasks.workunit.client.0.smithi046.stdout:6/629: dwrite da/fd [0,4194304] 0 2024-09-10T04:05:28.230 INFO:tasks.workunit.client.0.smithi046.stdout:0/583: dread d1/dc/d25/d33/d3/f14 [4194304,4194304] 0 2024-09-10T04:05:28.233 INFO:tasks.workunit.client.0.smithi046.stdout:0/584: write d1/d13/d37/d75/f83 [1323422,54452] 0 2024-09-10T04:05:28.234 INFO:tasks.workunit.client.0.smithi046.stdout:8/718: symlink d1/d29/d19/d36/d4c/d5e/lf1 0 2024-09-10T04:05:28.235 INFO:tasks.workunit.client.0.smithi046.stdout:4/871: dread d3/fa [0,4194304] 0 2024-09-10T04:05:28.236 INFO:tasks.workunit.client.0.smithi046.stdout:7/792: mkdir d5/d9/d12/d72/d77/de0/d1f/d116/d118 0 2024-09-10T04:05:28.237 INFO:tasks.workunit.client.0.smithi046.stdout:4/872: readlink d3/l7 0 2024-09-10T04:05:28.237 INFO:tasks.workunit.client.0.smithi046.stdout:8/719: dread - d1/d29/d19/d36/d4c/d5e/f5f zero size 2024-09-10T04:05:28.239 INFO:tasks.workunit.client.0.smithi046.stdout:0/585: chown d1/dc/d25/d33/d3/dd/d55/d4d/l9c 481915 1 2024-09-10T04:05:28.239 INFO:tasks.workunit.client.0.smithi046.stdout:4/873: stat d3/d12/d17/d22/f70 0 2024-09-10T04:05:28.241 INFO:tasks.workunit.client.0.smithi046.stdout:1/540: creat d3/da1/d1f/fb0 x:0 0 0 2024-09-10T04:05:28.244 INFO:tasks.workunit.client.0.smithi046.stdout:2/694: unlink d2/f8b 0 2024-09-10T04:05:28.250 INFO:tasks.workunit.client.0.smithi046.stdout:3/710: link d12/d13/d14/d1e/l79 d12/d13/d37/d51/db5/d50/lf6 0 2024-09-10T04:05:28.251 INFO:tasks.workunit.client.1.smithi070.stdout:7/832: sync 2024-09-10T04:05:28.252 INFO:tasks.workunit.client.0.smithi046.stdout:3/711: chown d12/d13/d37/d51/db5/d50/d7a/dc1/f97 34 1 2024-09-10T04:05:28.254 INFO:tasks.workunit.client.0.smithi046.stdout:7/793: mknod d5/d9/d12/d72/d77/de0/d1f/d7a/c119 0 2024-09-10T04:05:28.256 INFO:tasks.workunit.client.0.smithi046.stdout:9/707: dwrite d4/d3c/d4d/f76 [0,4194304] 0 2024-09-10T04:05:28.257 INFO:tasks.workunit.client.0.smithi046.stdout:0/586: unlink d1/dc/d25/d33/d3/f14 0 2024-09-10T04:05:28.257 INFO:tasks.workunit.client.0.smithi046.stdout:7/794: write d5/d9/d12/d72/d77/f82 [860074,23085] 0 2024-09-10T04:05:28.266 INFO:tasks.workunit.client.0.smithi046.stdout:3/712: truncate d12/d5d/d71/f5b 1190276 0 2024-09-10T04:05:28.271 INFO:tasks.workunit.client.0.smithi046.stdout:7/795: creat d5/d6/d39/f11a x:0 0 0 2024-09-10T04:05:28.272 INFO:tasks.workunit.client.0.smithi046.stdout:7/796: truncate d5/d6/d55/d7d/ff1 487962 0 2024-09-10T04:05:28.274 INFO:tasks.workunit.client.0.smithi046.stdout:0/587: unlink d1/d13/d37/d75/d94/d9a/f7c 0 2024-09-10T04:05:28.278 INFO:tasks.workunit.client.0.smithi046.stdout:8/720: getdents d1/d29/d19/d36/d6e 0 2024-09-10T04:05:28.279 INFO:tasks.workunit.client.0.smithi046.stdout:1/541: write d3/dc/f88 [4751721,104109] 0 2024-09-10T04:05:28.283 INFO:tasks.workunit.client.0.smithi046.stdout:3/713: mkdir d12/d5d/d71/df7 0 2024-09-10T04:05:28.284 INFO:tasks.workunit.client.0.smithi046.stdout:9/708: mknod d4/d18/d89/dba/cdf 0 2024-09-10T04:05:28.288 INFO:tasks.workunit.client.0.smithi046.stdout:0/588: mknod d1/dc/d25/cb6 0 2024-09-10T04:05:28.293 INFO:tasks.workunit.client.0.smithi046.stdout:1/542: dread d3/da1/d1f/f61 [0,4194304] 0 2024-09-10T04:05:28.295 INFO:tasks.workunit.client.0.smithi046.stdout:2/695: truncate d2/d49/f9b 1641087 0 2024-09-10T04:05:28.295 INFO:tasks.workunit.client.0.smithi046.stdout:1/543: read d3/dc/f10 [4040787,118855] 0 2024-09-10T04:05:28.297 INFO:tasks.workunit.client.0.smithi046.stdout:2/696: write d2/d6/f61 [606408,129092] 0 2024-09-10T04:05:28.297 INFO:tasks.workunit.client.0.smithi046.stdout:1/544: chown d3/da1/f2e 105805 1 2024-09-10T04:05:28.300 INFO:tasks.workunit.client.0.smithi046.stdout:0/589: creat d1/d13/d37/fb7 x:0 0 0 2024-09-10T04:05:28.302 INFO:tasks.workunit.client.0.smithi046.stdout:0/590: write d1/dc/f27 [2536633,103362] 0 2024-09-10T04:05:28.302 INFO:tasks.workunit.client.0.smithi046.stdout:8/721: mknod d1/d29/d12/cf2 0 2024-09-10T04:05:28.306 INFO:tasks.workunit.client.0.smithi046.stdout:0/591: chown d1/dc/d25/d33/d4/l82 521999 1 2024-09-10T04:05:28.309 INFO:tasks.workunit.client.0.smithi046.stdout:2/697: dwrite d2/d6/d23/d5d/d77/ff5 [0,4194304] 0 2024-09-10T04:05:28.309 INFO:tasks.workunit.client.0.smithi046.stdout:3/714: dread d12/d13/d14/d1e/d2d/d39/d67/fa1 [0,4194304] 0 2024-09-10T04:05:28.310 INFO:tasks.workunit.client.1.smithi070.stdout:9/832: dread d9/d7a/f3a [0,4194304] 0 2024-09-10T04:05:28.310 INFO:tasks.workunit.client.0.smithi046.stdout:0/592: write d1/d13/d37/f61 [2592206,107709] 0 2024-09-10T04:05:28.320 INFO:tasks.workunit.client.1.smithi070.stdout:1/711: dwrite d3/d8/d13/f25 [0,4194304] 0 2024-09-10T04:05:28.321 INFO:tasks.workunit.client.1.smithi070.stdout:5/808: creat d17/d19/d6a/d54/de1/dfe/f115 x:0 0 0 2024-09-10T04:05:28.322 INFO:tasks.workunit.client.1.smithi070.stdout:7/833: readlink d2/d3/d63/l67 0 2024-09-10T04:05:28.325 INFO:tasks.workunit.client.1.smithi070.stdout:0/941: truncate d8/d13/f1f 1919726 0 2024-09-10T04:05:28.326 INFO:tasks.workunit.client.1.smithi070.stdout:6/787: mkdir d0/d3/d18/d1b/d17/d3f/d40/d94/df8 0 2024-09-10T04:05:28.330 INFO:tasks.workunit.client.0.smithi046.stdout:9/709: rename d4/d29/d32 to d4/d3c/de0 0 2024-09-10T04:05:28.332 INFO:tasks.workunit.client.0.smithi046.stdout:9/710: fsync d4/d3c/de0/f55 0 2024-09-10T04:05:28.333 INFO:tasks.workunit.client.0.smithi046.stdout:7/797: getdents d5/d6/db/dd/d10c 0 2024-09-10T04:05:28.333 INFO:tasks.workunit.client.0.smithi046.stdout:0/593: dread d1/dc/d49/f4a [0,4194304] 0 2024-09-10T04:05:28.333 INFO:tasks.workunit.client.0.smithi046.stdout:9/711: readlink d4/d7/d61/lc2 0 2024-09-10T04:05:28.334 INFO:tasks.workunit.client.0.smithi046.stdout:0/594: write d1/dc/f27 [4877263,99676] 0 2024-09-10T04:05:28.335 INFO:tasks.workunit.client.0.smithi046.stdout:9/712: write d4/d3c/de0/fd9 [1043753,127746] 0 2024-09-10T04:05:28.339 INFO:tasks.workunit.client.1.smithi070.stdout:8/840: dwrite d4/d39/dfc/d43/d52/f6c [0,4194304] 0 2024-09-10T04:05:28.341 INFO:tasks.workunit.client.1.smithi070.stdout:1/712: fsync d3/d8/d13/d3f/f78 0 2024-09-10T04:05:28.342 INFO:tasks.workunit.client.1.smithi070.stdout:5/809: symlink d17/d24/d77/l116 0 2024-09-10T04:05:28.345 INFO:tasks.workunit.client.1.smithi070.stdout:9/833: dread d9/f73 [0,4194304] 0 2024-09-10T04:05:28.348 INFO:tasks.workunit.client.0.smithi046.stdout:7/798: creat d5/d9/d12/d72/d77/da7/f11b x:0 0 0 2024-09-10T04:05:28.349 INFO:tasks.workunit.client.0.smithi046.stdout:1/545: truncate d3/dc/f84 1329118 0 2024-09-10T04:05:28.349 INFO:tasks.workunit.client.0.smithi046.stdout:2/698: dread d2/d6/d15/fa4 [0,4194304] 0 2024-09-10T04:05:28.351 INFO:tasks.workunit.client.1.smithi070.stdout:0/942: dwrite d8/d13/d47/f5c [0,4194304] 0 2024-09-10T04:05:28.352 INFO:tasks.workunit.client.1.smithi070.stdout:4/747: rename de/d13/d18/d23/d47/d59/l69 to de/d13/lfb 0 2024-09-10T04:05:28.352 INFO:tasks.workunit.client.0.smithi046.stdout:0/595: mknod d1/dc/d25/cb8 0 2024-09-10T04:05:28.353 INFO:tasks.workunit.client.0.smithi046.stdout:8/722: truncate d1/f2e 3554847 0 2024-09-10T04:05:28.353 INFO:tasks.workunit.client.0.smithi046.stdout:8/723: fdatasync d1/f86 0 2024-09-10T04:05:28.354 INFO:tasks.workunit.client.1.smithi070.stdout:3/797: dwrite db/d1c/f6d [0,4194304] 0 2024-09-10T04:05:28.355 INFO:tasks.workunit.client.1.smithi070.stdout:4/748: readlink de/d13/d5b/d7b/dd5/daf/df2/ld8 0 2024-09-10T04:05:28.355 INFO:tasks.workunit.client.0.smithi046.stdout:9/713: creat d4/d3c/d4d/fe1 x:0 0 0 2024-09-10T04:05:28.355 INFO:tasks.workunit.client.0.smithi046.stdout:9/714: chown d4/d7/l3f 126210 1 2024-09-10T04:05:28.356 INFO:tasks.workunit.client.1.smithi070.stdout:2/753: creat d15/d18/d29/d2a/f110 x:0 0 0 2024-09-10T04:05:28.357 INFO:tasks.workunit.client.0.smithi046.stdout:1/546: readlink d3/d9b/l2f 0 2024-09-10T04:05:28.361 INFO:tasks.workunit.client.0.smithi046.stdout:0/596: unlink d1/f91 0 2024-09-10T04:05:28.362 INFO:tasks.workunit.client.1.smithi070.stdout:8/841: chown d4/dbf/d74/d11b/fd3 538 1 2024-09-10T04:05:28.363 INFO:tasks.workunit.client.1.smithi070.stdout:5/810: creat d17/d24/d77/f117 x:0 0 0 2024-09-10T04:05:28.364 INFO:tasks.workunit.client.1.smithi070.stdout:6/788: dwrite d0/d3/d18/d1b/d17/d3f/d40/f48 [4194304,4194304] 0 2024-09-10T04:05:28.365 INFO:tasks.workunit.client.0.smithi046.stdout:8/724: rmdir d1/d29/d17/d1f/d46/d67/dc1 39 2024-09-10T04:05:28.366 INFO:tasks.workunit.client.1.smithi070.stdout:1/713: creat d3/d8/d13/d3f/d6d/d81/fe9 x:0 0 0 2024-09-10T04:05:28.368 INFO:tasks.workunit.client.0.smithi046.stdout:9/715: symlink d4/d3c/d4d/d80/le2 0 2024-09-10T04:05:28.371 INFO:tasks.workunit.client.0.smithi046.stdout:1/547: chown d3/dc/d21/l5a 2345432 1 2024-09-10T04:05:28.374 INFO:tasks.workunit.client.1.smithi070.stdout:9/834: unlink d9/d1c/dac/d8a/dc4/dda/ccb 0 2024-09-10T04:05:28.378 INFO:tasks.workunit.client.0.smithi046.stdout:5/877: sync 2024-09-10T04:05:28.378 INFO:tasks.workunit.client.1.smithi070.stdout:7/834: rename d2/d7/db/c14 to d2/d3/dc/d79/d11d/c128 0 2024-09-10T04:05:28.383 INFO:tasks.workunit.client.0.smithi046.stdout:0/597: creat d1/dc/d25/d72/fb9 x:0 0 0 2024-09-10T04:05:28.383 INFO:tasks.workunit.client.0.smithi046.stdout:0/598: fsync d1/d13/fa9 0 2024-09-10T04:05:28.385 INFO:tasks.workunit.client.1.smithi070.stdout:7/835: dwrite d2/d3/dc/d79/fed [4194304,4194304] 0 2024-09-10T04:05:28.387 INFO:tasks.workunit.client.1.smithi070.stdout:3/798: read db/d1c/d25/d28/d43/f8b [4114438,6408] 0 2024-09-10T04:05:28.387 INFO:tasks.workunit.client.0.smithi046.stdout:8/725: dread d1/d29/d19/d36/f3f [0,4194304] 0 2024-09-10T04:05:28.390 INFO:tasks.workunit.client.1.smithi070.stdout:7/836: truncate d2/d3/d63/fe0 756297 0 2024-09-10T04:05:28.396 INFO:tasks.workunit.client.1.smithi070.stdout:5/811: chown d17/d19/d6a/d54/d57/fb0 286621849 1 2024-09-10T04:05:28.397 INFO:tasks.workunit.client.1.smithi070.stdout:5/812: stat d17/d19/d6a/d54/d57/da3/l5d 0 2024-09-10T04:05:28.399 INFO:tasks.workunit.client.0.smithi046.stdout:0/599: unlink d1/dc/d25/d33/d3/dd/c4b 0 2024-09-10T04:05:28.401 INFO:tasks.workunit.client.1.smithi070.stdout:0/943: creat d8/d24/d36/d137/f139 x:0 0 0 2024-09-10T04:05:28.402 INFO:tasks.workunit.client.1.smithi070.stdout:4/749: mknod de/d13/d5b/d7b/dea/cfc 0 2024-09-10T04:05:28.403 INFO:tasks.workunit.client.0.smithi046.stdout:1/548: dread d3/dc/d34/f76 [0,4194304] 0 2024-09-10T04:05:28.405 INFO:tasks.workunit.client.1.smithi070.stdout:8/842: mkdir d4/dbf/d74/d11b/d1a/d60/d7a/d123 0 2024-09-10T04:05:28.405 INFO:tasks.workunit.client.0.smithi046.stdout:7/799: getdents d5/d9/d12/d72/d77/de0/d1f/d7a 0 2024-09-10T04:05:28.406 INFO:tasks.workunit.client.0.smithi046.stdout:5/878: fdatasync d7/f1f 0 2024-09-10T04:05:28.406 INFO:tasks.workunit.client.1.smithi070.stdout:2/754: dwrite d15/d18/d56/f9c [0,4194304] 0 2024-09-10T04:05:28.408 INFO:tasks.workunit.client.1.smithi070.stdout:8/843: write d4/d39/dfc/f30 [9941974,127277] 0 2024-09-10T04:05:28.408 INFO:tasks.workunit.client.0.smithi046.stdout:2/699: getdents d2/d6/d23/d42/db4/dca 0 2024-09-10T04:05:28.409 INFO:tasks.workunit.client.0.smithi046.stdout:2/700: chown d2/d6/d23/d5d/d5e/cd7 59464417 1 2024-09-10T04:05:28.412 INFO:tasks.workunit.client.1.smithi070.stdout:1/714: unlink d3/d8/d12/d2f/fcf 0 2024-09-10T04:05:28.412 INFO:tasks.workunit.client.1.smithi070.stdout:8/844: write d4/dbf/d74/d11b/d1a/d60/d7a/d84/ff3 [178816,65136] 0 2024-09-10T04:05:28.413 INFO:tasks.workunit.client.0.smithi046.stdout:2/701: write d2/d6/d9d/fa5 [2676898,107617] 0 2024-09-10T04:05:28.413 INFO:tasks.workunit.client.0.smithi046.stdout:9/716: rename d4/d7/l3b to d4/le3 0 2024-09-10T04:05:28.416 INFO:tasks.workunit.client.0.smithi046.stdout:0/600: mknod d1/d13/d37/d75/d94/d9a/da4/cba 0 2024-09-10T04:05:28.417 INFO:tasks.workunit.client.1.smithi070.stdout:4/750: creat de/d13/d18/d26/ffd x:0 0 0 2024-09-10T04:05:28.417 INFO:tasks.workunit.client.0.smithi046.stdout:8/726: write d1/d29/d17/d1f/f54 [829443,37703] 0 2024-09-10T04:05:28.418 INFO:tasks.workunit.client.1.smithi070.stdout:0/944: truncate d8/d13/d1a/f23 2950562 0 2024-09-10T04:05:28.419 INFO:tasks.workunit.client.1.smithi070.stdout:0/945: readlink d8/d13/de2/d4f/l9f 0 2024-09-10T04:05:28.420 INFO:tasks.workunit.client.1.smithi070.stdout:5/813: symlink d17/d24/d2b/l118 0 2024-09-10T04:05:28.422 INFO:tasks.workunit.client.1.smithi070.stdout:5/814: write d17/d19/d6a/d54/de1/dfe/fcf [293513,124705] 0 2024-09-10T04:05:28.423 INFO:tasks.workunit.client.0.smithi046.stdout:2/702: symlink d2/d6/d23/d5d/d77/d7b/dd8/lfe 0 2024-09-10T04:05:28.425 INFO:tasks.workunit.client.0.smithi046.stdout:2/703: write d2/d6/d23/d5d/d5e/f84 [4913502,15032] 0 2024-09-10T04:05:28.426 INFO:tasks.workunit.client.0.smithi046.stdout:1/549: rename d3/d27/d5d/d60/c93 to d3/dc/d21/d3a/cb1 0 2024-09-10T04:05:28.428 INFO:tasks.workunit.client.1.smithi070.stdout:1/715: read d3/d8/d27/f93 [1368918,95566] 0 2024-09-10T04:05:28.428 INFO:tasks.workunit.client.1.smithi070.stdout:8/845: mkdir d4/d39/dfc/da8/d124 0 2024-09-10T04:05:28.429 INFO:tasks.workunit.client.0.smithi046.stdout:0/601: creat d1/dc/d25/d33/d3/dd/fbb x:0 0 0 2024-09-10T04:05:28.430 INFO:tasks.workunit.client.1.smithi070.stdout:5/815: dwrite d17/d6f/d32/ffc [0,4194304] 0 2024-09-10T04:05:28.431 INFO:tasks.workunit.client.0.smithi046.stdout:8/727: creat d1/d29/d17/d1f/d46/d6a/ff3 x:0 0 0 2024-09-10T04:05:28.433 INFO:tasks.workunit.client.0.smithi046.stdout:7/800: dread d5/d9/d12/d72/d77/de0/fd5 [0,4194304] 0 2024-09-10T04:05:28.434 INFO:tasks.workunit.client.1.smithi070.stdout:3/799: getdents db/d83/d92 0 2024-09-10T04:05:28.436 INFO:tasks.workunit.client.1.smithi070.stdout:8/846: rmdir d4/d16/d73 39 2024-09-10T04:05:28.437 INFO:tasks.workunit.client.1.smithi070.stdout:8/847: stat d4/dbf/d74/d11b/d19/f72 0 2024-09-10T04:05:28.438 INFO:tasks.workunit.client.0.smithi046.stdout:1/550: creat d3/dc/d21/d3a/fb2 x:0 0 0 2024-09-10T04:05:28.438 INFO:tasks.workunit.client.1.smithi070.stdout:1/716: rmdir d3/d8/d13/d67/d83 39 2024-09-10T04:05:28.444 INFO:tasks.workunit.client.1.smithi070.stdout:0/946: creat d8/d13/de2/d4f/d125/f13a x:0 0 0 2024-09-10T04:05:28.445 INFO:tasks.workunit.client.0.smithi046.stdout:0/602: mkdir d1/d13/d37/d75/d94/d9a/da4/dbc 0 2024-09-10T04:05:28.446 INFO:tasks.workunit.client.1.smithi070.stdout:5/816: symlink d17/d19/d6a/d54/de1/dfe/d93/l119 0 2024-09-10T04:05:28.447 INFO:tasks.workunit.client.0.smithi046.stdout:0/603: dread d1/dc/d25/d33/d3/dd/d55/d77/da6/fb3 [0,4194304] 0 2024-09-10T04:05:28.449 INFO:tasks.workunit.client.0.smithi046.stdout:5/879: link d7/d11/d14/d2f/d64/fc6 d7/d11/d14/d8e/f132 0 2024-09-10T04:05:28.453 INFO:tasks.workunit.client.1.smithi070.stdout:8/848: unlink d4/d39/dfc/d43/l95 0 2024-09-10T04:05:28.457 INFO:tasks.workunit.client.1.smithi070.stdout:4/751: mkdir de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe 0 2024-09-10T04:05:28.459 INFO:tasks.workunit.client.0.smithi046.stdout:9/717: rmdir d4/d29/d65/dd5 0 2024-09-10T04:05:28.460 INFO:tasks.workunit.client.0.smithi046.stdout:2/704: write d2/d6/d23/d5d/d77/d7b/d85/d8d/fc5 [157641,127291] 0 2024-09-10T04:05:28.462 INFO:tasks.workunit.client.0.smithi046.stdout:1/551: dwrite d3/f77 [0,4194304] 0 2024-09-10T04:05:28.462 INFO:tasks.workunit.client.0.smithi046.stdout:2/705: readlink d2/d6/d23/d5d/d77/l6e 0 2024-09-10T04:05:28.465 INFO:tasks.workunit.client.0.smithi046.stdout:8/728: mknod d1/cf4 0 2024-09-10T04:05:28.465 INFO:tasks.workunit.client.0.smithi046.stdout:1/552: read d3/d27/d2a/f8b [5416005,8528] 0 2024-09-10T04:05:28.466 INFO:tasks.workunit.client.1.smithi070.stdout:2/755: getdents d15/d18/d99 0 2024-09-10T04:05:28.467 INFO:tasks.workunit.client.0.smithi046.stdout:1/553: fdatasync d3/dc/d21/d3a/faf 0 2024-09-10T04:05:28.468 INFO:tasks.workunit.client.0.smithi046.stdout:0/604: truncate d1/dc/f21 4548499 0 2024-09-10T04:05:28.468 INFO:tasks.workunit.client.0.smithi046.stdout:9/718: dwrite d4/d7/f31 [0,4194304] 0 2024-09-10T04:05:28.470 INFO:tasks.workunit.client.0.smithi046.stdout:5/880: mknod d7/d11/d14/d2f/d2b/d92/c133 0 2024-09-10T04:05:28.470 INFO:tasks.workunit.client.0.smithi046.stdout:0/605: fsync d1/dc/d25/d33/d3/f46 0 2024-09-10T04:05:28.471 INFO:tasks.workunit.client.1.smithi070.stdout:3/800: write db/d1c/d25/d28/d43/f8b [2675230,65094] 0 2024-09-10T04:05:28.472 INFO:tasks.workunit.client.0.smithi046.stdout:0/606: chown d1/dc/d25/cb6 623766023 1 2024-09-10T04:05:28.472 INFO:tasks.workunit.client.0.smithi046.stdout:7/801: mknod d5/d9/d12/d72/d77/c11c 0 2024-09-10T04:05:28.477 INFO:tasks.workunit.client.1.smithi070.stdout:4/752: mkdir de/d13/d5b/d7b/dd5/dff 0 2024-09-10T04:05:28.478 INFO:tasks.workunit.client.1.smithi070.stdout:1/717: dwrite d3/d8/d12/d2f/f34 [0,4194304] 0 2024-09-10T04:05:28.478 INFO:tasks.workunit.client.1.smithi070.stdout:0/947: write d8/d13/de2/d4f/ff9 [2964291,34867] 0 2024-09-10T04:05:28.480 INFO:tasks.workunit.client.1.smithi070.stdout:5/817: symlink d17/l11a 0 2024-09-10T04:05:28.482 INFO:tasks.workunit.client.1.smithi070.stdout:8/849: mknod d4/d16/dcc/d122/c125 0 2024-09-10T04:05:28.484 INFO:tasks.workunit.client.0.smithi046.stdout:9/719: symlink d4/d66/da6/le4 0 2024-09-10T04:05:28.490 INFO:tasks.workunit.client.1.smithi070.stdout:5/818: symlink d17/d24/d45/dc0/l11b 0 2024-09-10T04:05:28.493 INFO:tasks.workunit.client.1.smithi070.stdout:1/718: rename d3/d10/d1d/l3a to d3/d8/d27/d7a/d94/lea 0 2024-09-10T04:05:28.494 INFO:tasks.workunit.client.0.smithi046.stdout:8/729: symlink d1/d29/d19/d36/lf5 0 2024-09-10T04:05:28.495 INFO:tasks.workunit.client.1.smithi070.stdout:8/850: dwrite d4/dbf/d74/d11b/d1a/fdf [0,4194304] 0 2024-09-10T04:05:28.495 INFO:tasks.workunit.client.0.smithi046.stdout:0/607: dread d1/dc/d25/d33/d1a/d41/f6e [0,4194304] 0 2024-09-10T04:05:28.496 INFO:tasks.workunit.client.1.smithi070.stdout:4/753: mknod de/d13/d18/d23/d39/d48/c100 0 2024-09-10T04:05:28.497 INFO:tasks.workunit.client.0.smithi046.stdout:7/802: dwrite d5/d6/d55/ff6 [0,4194304] 0 2024-09-10T04:05:28.498 INFO:tasks.workunit.client.1.smithi070.stdout:4/754: write f0 [6498204,19578] 0 2024-09-10T04:05:28.498 INFO:tasks.workunit.client.1.smithi070.stdout:4/755: readlink de/d13/d18/d23/l41 0 2024-09-10T04:05:28.499 INFO:tasks.workunit.client.0.smithi046.stdout:1/554: symlink d3/lb3 0 2024-09-10T04:05:28.499 INFO:tasks.workunit.client.0.smithi046.stdout:8/730: chown d1/d29/d17/d1f/d46/d67/caf 0 1 2024-09-10T04:05:28.499 INFO:tasks.workunit.client.0.smithi046.stdout:8/731: readlink d1/l28 0 2024-09-10T04:05:28.501 INFO:tasks.workunit.client.0.smithi046.stdout:1/555: fdatasync d3/d27/d2a/fa3 0 2024-09-10T04:05:28.501 INFO:tasks.workunit.client.0.smithi046.stdout:5/881: mkdir d7/d11/d14/d2f/deb/da5/dcc/dc0/d134 0 2024-09-10T04:05:28.502 INFO:tasks.workunit.client.1.smithi070.stdout:6/789: sync 2024-09-10T04:05:28.502 INFO:tasks.workunit.client.1.smithi070.stdout:9/835: sync 2024-09-10T04:05:28.502 INFO:tasks.workunit.client.1.smithi070.stdout:7/837: sync 2024-09-10T04:05:28.504 INFO:tasks.workunit.client.1.smithi070.stdout:6/790: write d0/d3/d18/d62/db8/fe5 [541036,100980] 0 2024-09-10T04:05:28.505 INFO:tasks.workunit.client.0.smithi046.stdout:9/720: dread d4/d3c/de0/f55 [0,4194304] 0 2024-09-10T04:05:28.505 INFO:tasks.workunit.client.0.smithi046.stdout:7/803: write d5/d6/db/dd/d24/d76/d9c/f114 [429238,66454] 0 2024-09-10T04:05:28.508 INFO:tasks.workunit.client.0.smithi046.stdout:5/882: chown d7/d11/d3f/d53/da9/f11b 272153 1 2024-09-10T04:05:28.508 INFO:tasks.workunit.client.0.smithi046.stdout:1/556: read d3/dc/f10 [1896655,59739] 0 2024-09-10T04:05:28.508 INFO:tasks.workunit.client.0.smithi046.stdout:9/721: chown d4/d3c/d4d/d5c/f93 210094 1 2024-09-10T04:05:28.509 INFO:tasks.workunit.client.0.smithi046.stdout:8/732: mkdir d1/d29/d17/d1f/d2a/df6 0 2024-09-10T04:05:28.511 INFO:tasks.workunit.client.1.smithi070.stdout:8/851: mkdir d4/d16/dcc/d122/d126 0 2024-09-10T04:05:28.511 INFO:tasks.workunit.client.0.smithi046.stdout:1/557: write d3/d27/f20 [4784597,126834] 0 2024-09-10T04:05:28.512 INFO:tasks.workunit.client.1.smithi070.stdout:8/852: chown d4/fe9 5685734 1 2024-09-10T04:05:28.512 INFO:tasks.workunit.client.0.smithi046.stdout:2/706: getdents d2/d6/d23/d55/d9f 0 2024-09-10T04:05:28.515 INFO:tasks.workunit.client.0.smithi046.stdout:0/608: link d1/d13/d37/d75/d94/d9a/fac d1/d13/d37/d75/d94/d9a/fbd 0 2024-09-10T04:05:28.516 INFO:tasks.workunit.client.1.smithi070.stdout:2/756: write d15/d2b/d6e/f89 [594478,102743] 0 2024-09-10T04:05:28.517 INFO:tasks.workunit.client.0.smithi046.stdout:9/722: fsync d4/d7/f5e 0 2024-09-10T04:05:28.518 INFO:tasks.workunit.client.0.smithi046.stdout:0/609: chown d1/dc/d25/d33/d3/dd/d55/d4d/f4e 0 1 2024-09-10T04:05:28.521 INFO:tasks.workunit.client.0.smithi046.stdout:7/804: rename d5/d6/d32/ca9 to d5/d6/d62/d108/c11d 0 2024-09-10T04:05:28.521 INFO:tasks.workunit.client.1.smithi070.stdout:7/838: mknod d2/d3/dc/d5d/d41/dd6/c129 0 2024-09-10T04:05:28.522 INFO:tasks.workunit.client.0.smithi046.stdout:2/707: dread d2/d6/d23/d5d/d5e/f72 [0,4194304] 0 2024-09-10T04:05:28.523 INFO:tasks.workunit.client.1.smithi070.stdout:9/836: dread d9/d7a/f44 [0,4194304] 0 2024-09-10T04:05:28.523 INFO:tasks.workunit.client.0.smithi046.stdout:5/883: truncate d7/f43 556914 0 2024-09-10T04:05:28.524 INFO:tasks.workunit.client.1.smithi070.stdout:2/757: write d15/d18/d29/d2a/d2e/d32/f33 [5136456,25637] 0 2024-09-10T04:05:28.524 INFO:tasks.workunit.client.1.smithi070.stdout:3/801: dwrite db/f12 [0,4194304] 0 2024-09-10T04:05:28.526 INFO:tasks.workunit.client.0.smithi046.stdout:0/610: dread d1/dc/d25/d33/d3/d29/d2d/d5e/f89 [0,4194304] 0 2024-09-10T04:05:28.526 INFO:tasks.workunit.client.1.smithi070.stdout:0/948: dwrite d8/d24/d2d/d5f/fe7 [0,4194304] 0 2024-09-10T04:05:28.529 INFO:tasks.workunit.client.0.smithi046.stdout:1/558: symlink d3/dc/d21/lb4 0 2024-09-10T04:05:28.530 INFO:tasks.workunit.client.1.smithi070.stdout:5/819: symlink d17/d6f/d32/d90/l11c 0 2024-09-10T04:05:28.531 INFO:tasks.workunit.client.0.smithi046.stdout:7/805: dwrite d5/d6/db/dd/d24/d76/d9c/f114 [0,4194304] 0 2024-09-10T04:05:28.532 INFO:tasks.workunit.client.0.smithi046.stdout:1/559: chown d3/d27/c94 0 1 2024-09-10T04:05:28.533 INFO:tasks.workunit.client.0.smithi046.stdout:8/733: dread d1/d29/d12/f45 [0,4194304] 0 2024-09-10T04:05:28.535 INFO:tasks.workunit.client.0.smithi046.stdout:8/734: chown d1/d29/d17/d1f/d46/d67/de0 977 1 2024-09-10T04:05:28.535 INFO:tasks.workunit.client.1.smithi070.stdout:8/853: unlink d4/d39/dfc/d43/d52/fc4 0 2024-09-10T04:05:28.541 INFO:tasks.workunit.client.0.smithi046.stdout:6/630: sync 2024-09-10T04:05:28.546 INFO:tasks.workunit.client.1.smithi070.stdout:2/758: dread - d15/d2b/d6e/fe0 zero size 2024-09-10T04:05:28.548 INFO:tasks.workunit.client.1.smithi070.stdout:9/837: creat d9/d1c/dac/d8a/d3c/d62/f10a x:0 0 0 2024-09-10T04:05:28.548 INFO:tasks.workunit.client.1.smithi070.stdout:2/759: write d15/d18/d29/d2a/d2e/da6/dc0/dcb/fb9 [1041876,76139] 0 2024-09-10T04:05:28.549 INFO:tasks.workunit.client.0.smithi046.stdout:5/884: chown d7/f19 3962 1 2024-09-10T04:05:28.549 INFO:tasks.workunit.client.1.smithi070.stdout:3/802: mkdir db/d1c/d89/d112 0 2024-09-10T04:05:28.551 INFO:tasks.workunit.client.1.smithi070.stdout:3/803: readlink db/d1c/d25/d28/d43/l6c 0 2024-09-10T04:05:28.551 INFO:tasks.workunit.client.1.smithi070.stdout:0/949: symlink d8/d24/d2d/d5f/l13b 0 2024-09-10T04:05:28.552 INFO:tasks.workunit.client.0.smithi046.stdout:0/611: creat d1/dc/d49/fbe x:0 0 0 2024-09-10T04:05:28.553 INFO:tasks.workunit.client.0.smithi046.stdout:0/612: fdatasync d1/dc/d25/d33/d3/f46 0 2024-09-10T04:05:28.554 INFO:tasks.workunit.client.1.smithi070.stdout:6/791: symlink d0/d3/d18/d2a/df0/lf9 0 2024-09-10T04:05:28.554 INFO:tasks.workunit.client.1.smithi070.stdout:6/792: fdatasync d0/d3/d1f/d81/fb7 0 2024-09-10T04:05:28.555 INFO:tasks.workunit.client.0.smithi046.stdout:1/560: creat d3/d9b/fb5 x:0 0 0 2024-09-10T04:05:28.555 INFO:tasks.workunit.client.1.smithi070.stdout:6/793: write d0/d3/d18/d2a/fdd [148358,17105] 0 2024-09-10T04:05:28.556 INFO:tasks.workunit.client.0.smithi046.stdout:9/723: write d4/d7/f4a [207144,39531] 0 2024-09-10T04:05:28.558 INFO:tasks.workunit.client.0.smithi046.stdout:9/724: chown d4/d18/d89/l90 56993726 1 2024-09-10T04:05:28.561 INFO:tasks.workunit.client.1.smithi070.stdout:4/756: creat de/d13/d18/d23/f101 x:0 0 0 2024-09-10T04:05:28.561 INFO:tasks.workunit.client.1.smithi070.stdout:9/838: dread d9/d1c/dac/d8a/dc4/de1/f66 [0,4194304] 0 2024-09-10T04:05:28.561 INFO:tasks.workunit.client.1.smithi070.stdout:4/757: readlink de/d13/l1d 0 2024-09-10T04:05:28.562 INFO:tasks.workunit.client.0.smithi046.stdout:0/613: dwrite d1/d13/d37/d75/f8b [0,4194304] 0 2024-09-10T04:05:28.567 INFO:tasks.workunit.client.0.smithi046.stdout:5/885: truncate d7/d11/d14/d8e/d96/fbe 5014667 0 2024-09-10T04:05:28.568 INFO:tasks.workunit.client.1.smithi070.stdout:0/950: dread - d8/d24/d2d/d8d/fe6 zero size 2024-09-10T04:05:28.572 INFO:tasks.workunit.client.1.smithi070.stdout:1/719: getdents d3/d8/d12/d2f 0 2024-09-10T04:05:28.574 INFO:tasks.workunit.client.0.smithi046.stdout:7/806: link d5/d6/d32/f74 d5/d9/d12/d72/d77/de0/d9f/f11e 0 2024-09-10T04:05:28.581 INFO:tasks.workunit.client.1.smithi070.stdout:9/839: symlink d9/d1c/dac/d8a/d3c/l10b 0 2024-09-10T04:05:28.582 INFO:tasks.workunit.client.0.smithi046.stdout:0/614: unlink d1/dc/d25/d33/d4/cad 0 2024-09-10T04:05:28.582 INFO:tasks.workunit.client.0.smithi046.stdout:8/735: dwrite d1/d29/d19/d36/d4c/f5b [0,4194304] 0 2024-09-10T04:05:28.583 INFO:tasks.workunit.client.1.smithi070.stdout:2/760: symlink d15/l111 0 2024-09-10T04:05:28.584 INFO:tasks.workunit.client.0.smithi046.stdout:2/708: write d2/d6/d15/f70 [1482234,48440] 0 2024-09-10T04:05:28.587 INFO:tasks.workunit.client.0.smithi046.stdout:6/631: dwrite da/db/d10/d27/f92 [0,4194304] 0 2024-09-10T04:05:28.589 INFO:tasks.workunit.client.1.smithi070.stdout:6/794: write d0/d3/d18/d62/d70/fba [1012310,25097] 0 2024-09-10T04:05:28.590 INFO:tasks.workunit.client.1.smithi070.stdout:8/854: rename d4/dbf/d74/d11b/d6f/db5/ff1 to d4/d39/dfc/f127 0 2024-09-10T04:05:28.591 INFO:tasks.workunit.client.1.smithi070.stdout:4/758: write fc [1176133,91347] 0 2024-09-10T04:05:28.593 INFO:tasks.workunit.client.0.smithi046.stdout:1/561: dwrite d3/d27/d2a/f7e [0,4194304] 0 2024-09-10T04:05:28.595 INFO:tasks.workunit.client.0.smithi046.stdout:0/615: read d1/dc/f7b [720083,41901] 0 2024-09-10T04:05:28.595 INFO:tasks.workunit.client.1.smithi070.stdout:2/761: mknod d15/d18/d29/ddb/c112 0 2024-09-10T04:05:28.596 INFO:tasks.workunit.client.1.smithi070.stdout:1/720: symlink d3/d8/d12/leb 0 2024-09-10T04:05:28.597 INFO:tasks.workunit.client.1.smithi070.stdout:1/721: chown d3/d8/d12/d2f/d32/da8/lb6 5109898 1 2024-09-10T04:05:28.597 INFO:tasks.workunit.client.0.smithi046.stdout:7/807: symlink d5/d9/l11f 0 2024-09-10T04:05:28.600 INFO:tasks.workunit.client.1.smithi070.stdout:3/804: rename db/d1c/d3e/d103 to db/d1c/d25/d28/dd3/de5/d10f/db5/d113 0 2024-09-10T04:05:28.603 INFO:tasks.workunit.client.1.smithi070.stdout:8/855: mknod d4/dbf/d74/d11b/dab/c128 0 2024-09-10T04:05:28.607 INFO:tasks.workunit.client.0.smithi046.stdout:9/725: rename d4/d66/da6/f96 to d4/d3c/d4d/d5c/d87/fe5 0 2024-09-10T04:05:28.609 INFO:tasks.workunit.client.0.smithi046.stdout:1/562: chown d3/d27/cab 1657590588 1 2024-09-10T04:05:28.610 INFO:tasks.workunit.client.1.smithi070.stdout:1/722: dwrite d3/d8/fc8 [0,4194304] 0 2024-09-10T04:05:28.610 INFO:tasks.workunit.client.1.smithi070.stdout:4/759: mknod de/d13/d18/d23/d39/d48/c102 0 2024-09-10T04:05:28.610 INFO:tasks.workunit.client.1.smithi070.stdout:4/760: chown de/d13/d18/d5c/cd3 1298 1 2024-09-10T04:05:28.610 INFO:tasks.workunit.client.0.smithi046.stdout:1/563: dread d3/da1/d1f/f61 [0,4194304] 0 2024-09-10T04:05:28.611 INFO:tasks.workunit.client.1.smithi070.stdout:2/762: symlink d15/d18/d29/d64/l113 0 2024-09-10T04:05:28.613 INFO:tasks.workunit.client.0.smithi046.stdout:0/616: symlink d1/dc/d25/d33/d4/lbf 0 2024-09-10T04:05:28.614 INFO:tasks.workunit.client.0.smithi046.stdout:7/808: stat d5/d6/db/f1d 0 2024-09-10T04:05:28.616 INFO:tasks.workunit.client.1.smithi070.stdout:0/951: rename d8/d13/de2/f65 to d8/d13/d1a/d9a/f13c 0 2024-09-10T04:05:28.617 INFO:tasks.workunit.client.1.smithi070.stdout:8/856: symlink d4/dbf/d74/da4/d114/l129 0 2024-09-10T04:05:28.621 INFO:tasks.workunit.client.0.smithi046.stdout:5/886: rename d7/d11/d14/d2f/deb/f111 to d7/d11/d14/d8e/d96/dce/d122/f135 0 2024-09-10T04:05:28.621 INFO:tasks.workunit.client.0.smithi046.stdout:9/726: rmdir d4/d3c/d4d 39 2024-09-10T04:05:28.621 INFO:tasks.workunit.client.1.smithi070.stdout:4/761: readlink de/d13/d18/d23/d39/lc3 0 2024-09-10T04:05:28.622 INFO:tasks.workunit.client.1.smithi070.stdout:2/763: mknod d15/d18/d56/c114 0 2024-09-10T04:05:28.622 INFO:tasks.workunit.client.1.smithi070.stdout:3/805: mknod db/d1c/d25/c114 0 2024-09-10T04:05:28.622 INFO:tasks.workunit.client.0.smithi046.stdout:1/564: symlink d3/dc/d34/d58/lb6 0 2024-09-10T04:05:28.624 INFO:tasks.workunit.client.1.smithi070.stdout:6/795: rename d0/l9 to d0/d3/d18/d62/d70/lfa 0 2024-09-10T04:05:28.626 INFO:tasks.workunit.client.1.smithi070.stdout:0/952: mknod d8/d85/df5/c13d 0 2024-09-10T04:05:28.627 INFO:tasks.workunit.client.0.smithi046.stdout:7/809: mkdir d5/d6/d32/dab/d120 0 2024-09-10T04:05:28.628 INFO:tasks.workunit.client.1.smithi070.stdout:1/723: mkdir d3/d8/d92/dec 0 2024-09-10T04:05:28.629 INFO:tasks.workunit.client.1.smithi070.stdout:9/840: dread d9/d1c/d34/d8b/f9e [0,4194304] 0 2024-09-10T04:05:28.630 INFO:tasks.workunit.client.1.smithi070.stdout:6/796: dread d0/d3/d18/d1b/d33/ff3 [0,4194304] 0 2024-09-10T04:05:28.630 INFO:tasks.workunit.client.1.smithi070.stdout:2/764: creat d15/d59/f115 x:0 0 0 2024-09-10T04:05:28.632 INFO:tasks.workunit.client.1.smithi070.stdout:6/797: chown d0/d3/d18/d62/f85 179 1 2024-09-10T04:05:28.634 INFO:tasks.workunit.client.1.smithi070.stdout:8/857: dread d4/dbf/d74/d11b/d19/f25 [0,4194304] 0 2024-09-10T04:05:28.635 INFO:tasks.workunit.client.1.smithi070.stdout:0/953: mkdir d8/deb/d13e 0 2024-09-10T04:05:28.636 INFO:tasks.workunit.client.0.smithi046.stdout:9/727: creat d4/d3c/de0/d95/fe6 x:0 0 0 2024-09-10T04:05:28.636 INFO:tasks.workunit.client.1.smithi070.stdout:8/858: write d4/dcf/f101 [437446,100991] 0 2024-09-10T04:05:28.636 INFO:tasks.workunit.client.1.smithi070.stdout:6/798: dread d0/f57 [0,4194304] 0 2024-09-10T04:05:28.639 INFO:tasks.workunit.client.0.smithi046.stdout:8/736: getdents d1/d29/d17/d1f 0 2024-09-10T04:05:28.639 INFO:tasks.workunit.client.0.smithi046.stdout:0/617: write d1/dc/d25/d33/d1a/d41/f6e [1632386,14810] 0 2024-09-10T04:05:28.640 INFO:tasks.workunit.client.1.smithi070.stdout:1/724: creat d3/d8/d13/d3f/d5d/fed x:0 0 0 2024-09-10T04:05:28.642 INFO:tasks.workunit.client.0.smithi046.stdout:1/565: rmdir d3/d27/d5d 39 2024-09-10T04:05:28.644 INFO:tasks.workunit.client.0.smithi046.stdout:1/566: stat d3/d27/d2a/f7e 0 2024-09-10T04:05:28.646 INFO:tasks.workunit.client.1.smithi070.stdout:3/806: truncate f3 349484 0 2024-09-10T04:05:28.648 INFO:tasks.workunit.client.0.smithi046.stdout:0/618: dwrite d1/dc/f95 [0,4194304] 0 2024-09-10T04:05:28.649 INFO:tasks.workunit.client.0.smithi046.stdout:7/810: creat d5/d6/db/dd/d24/db7/f121 x:0 0 0 2024-09-10T04:05:28.650 INFO:tasks.workunit.client.1.smithi070.stdout:0/954: dread d8/d13/f5a [4194304,4194304] 0 2024-09-10T04:05:28.650 INFO:tasks.workunit.client.1.smithi070.stdout:7/839: sync 2024-09-10T04:05:28.651 INFO:tasks.workunit.client.1.smithi070.stdout:0/955: dread - d8/d13/de2/d4f/fdc zero size 2024-09-10T04:05:28.651 INFO:tasks.workunit.client.1.smithi070.stdout:7/840: stat d2/d7/d9/d18/d49/l5b 0 2024-09-10T04:05:28.652 INFO:tasks.workunit.client.0.smithi046.stdout:6/632: getdents da/db/d10/d51 0 2024-09-10T04:05:28.653 INFO:tasks.workunit.client.0.smithi046.stdout:5/887: creat d7/d11/d14/d2f/deb/da5/d123/f136 x:0 0 0 2024-09-10T04:05:28.653 INFO:tasks.workunit.client.0.smithi046.stdout:9/728: dread d4/d29/f6d [0,4194304] 0 2024-09-10T04:05:28.654 INFO:tasks.workunit.client.0.smithi046.stdout:7/811: chown d5/d6/db/dd/d24/d76/d9c/ccb 258211 1 2024-09-10T04:05:28.656 INFO:tasks.workunit.client.1.smithi070.stdout:7/841: chown d2/d40/d6c/f7f 80414 1 2024-09-10T04:05:28.656 INFO:tasks.workunit.client.0.smithi046.stdout:9/729: write d4/d3c/de0/fd9 [2028948,37309] 0 2024-09-10T04:05:28.658 INFO:tasks.workunit.client.1.smithi070.stdout:9/841: dwrite d9/d1c/dac/d8a/d3c/f52 [0,4194304] 0 2024-09-10T04:05:28.659 INFO:tasks.workunit.client.1.smithi070.stdout:1/725: rmdir d3/d8/d27/d7a 39 2024-09-10T04:05:28.661 INFO:tasks.workunit.client.0.smithi046.stdout:0/619: symlink d1/dc/d25/d33/d1a/d41/lc0 0 2024-09-10T04:05:28.662 INFO:tasks.workunit.client.1.smithi070.stdout:3/807: mknod db/d1c/d25/d28/dd3/de5/d10f/db5/c115 0 2024-09-10T04:05:28.664 INFO:tasks.workunit.client.1.smithi070.stdout:0/956: dread d8/d13/de2/d7d/d8a/f131 [0,4194304] 0 2024-09-10T04:05:28.667 INFO:tasks.workunit.client.0.smithi046.stdout:7/812: creat d5/d6/d39/f122 x:0 0 0 2024-09-10T04:05:28.669 INFO:tasks.workunit.client.0.smithi046.stdout:7/813: write d5/d6/d32/dab/fb1 [5126250,50814] 0 2024-09-10T04:05:28.671 INFO:tasks.workunit.client.0.smithi046.stdout:7/814: truncate d5/fbf 845826 0 2024-09-10T04:05:28.672 INFO:tasks.workunit.client.0.smithi046.stdout:7/815: chown d5/d6/db/f1d 37476752 1 2024-09-10T04:05:28.673 INFO:tasks.workunit.client.0.smithi046.stdout:8/737: mknod d1/d29/d19/d73/dd2/cf7 0 2024-09-10T04:05:28.673 INFO:tasks.workunit.client.0.smithi046.stdout:7/816: write d5/d6/db/f38 [8421493,37374] 0 2024-09-10T04:05:28.676 INFO:tasks.workunit.client.0.smithi046.stdout:5/888: rmdir d7/d11/d14/d2f/deb/da5/dcc/de5 39 2024-09-10T04:05:28.677 INFO:tasks.workunit.client.1.smithi070.stdout:8/859: dwrite d4/dbf/d74/d11b/d19/d62/fbc [0,4194304] 0 2024-09-10T04:05:28.679 INFO:tasks.workunit.client.1.smithi070.stdout:8/860: dread d4/dcf/f9a [0,4194304] 0 2024-09-10T04:05:28.679 INFO:tasks.workunit.client.1.smithi070.stdout:2/765: mknod d15/d18/d29/d2a/d2e/d32/d9b/c116 0 2024-09-10T04:05:28.681 INFO:tasks.workunit.client.1.smithi070.stdout:1/726: write d3/d8/d27/d7a/fa7 [1005304,83311] 0 2024-09-10T04:05:28.686 INFO:tasks.workunit.client.1.smithi070.stdout:4/762: getdents de/d13/d18/d23/d39/d48 0 2024-09-10T04:05:28.686 INFO:tasks.workunit.client.0.smithi046.stdout:0/620: mknod d1/d13/d37/d75/d94/d9a/cc1 0 2024-09-10T04:05:28.686 INFO:tasks.workunit.client.0.smithi046.stdout:6/633: dwrite da/db/d11/d13/d1e/d23/d86/fc8 [0,4194304] 0 2024-09-10T04:05:28.687 INFO:tasks.workunit.client.1.smithi070.stdout:4/763: truncate de/d13/d18/d23/d47/d59/ff6 580330 0 2024-09-10T04:05:28.688 INFO:tasks.workunit.client.1.smithi070.stdout:9/842: truncate d9/fb7 627252 0 2024-09-10T04:05:28.688 INFO:tasks.workunit.client.1.smithi070.stdout:4/764: chown de/d13/d18/d23/d39/d48/ff5 58 1 2024-09-10T04:05:28.690 INFO:tasks.workunit.client.1.smithi070.stdout:4/765: chown de/d13/d18/d23/d39/d48/fee 53766349 1 2024-09-10T04:05:28.693 INFO:tasks.workunit.client.1.smithi070.stdout:5/820: sync 2024-09-10T04:05:28.697 INFO:tasks.workunit.client.1.smithi070.stdout:0/957: unlink d8/d13/de2/d4f/fbb 0 2024-09-10T04:05:28.701 INFO:tasks.workunit.client.0.smithi046.stdout:9/730: link d4/f99 d4/d18/d92/fe7 0 2024-09-10T04:05:28.704 INFO:tasks.workunit.client.1.smithi070.stdout:8/861: creat d4/dbf/d74/d11b/d1a/d60/d7a/d84/f12a x:0 0 0 2024-09-10T04:05:28.705 INFO:tasks.workunit.client.1.smithi070.stdout:0/958: dwrite d8/d13/de2/d6f/f120 [0,4194304] 0 2024-09-10T04:05:28.708 INFO:tasks.workunit.client.1.smithi070.stdout:0/959: read - d8/d13/de2/d6f/fec zero size 2024-09-10T04:05:28.713 INFO:tasks.workunit.client.1.smithi070.stdout:9/843: creat d9/d1c/dac/d8a/dc4/f10c x:0 0 0 2024-09-10T04:05:28.713 INFO:tasks.workunit.client.1.smithi070.stdout:8/862: dwrite d4/dbf/d74/d11b/d1a/f88 [0,4194304] 0 2024-09-10T04:05:28.716 INFO:tasks.workunit.client.1.smithi070.stdout:9/844: write d9/d7a/d74/d8c/fd4 [946410,118407] 0 2024-09-10T04:05:28.717 INFO:tasks.workunit.client.0.smithi046.stdout:7/817: dwrite d5/d9/d12/d72/d77/de0/fd5 [0,4194304] 0 2024-09-10T04:05:28.718 INFO:tasks.workunit.client.0.smithi046.stdout:5/889: write d7/d11/d3f/d53/d99/fae [3825270,21032] 0 2024-09-10T04:05:28.720 INFO:tasks.workunit.client.1.smithi070.stdout:5/821: mknod d17/d19/d6a/d54/de1/dfe/dce/c11d 0 2024-09-10T04:05:28.723 INFO:tasks.workunit.client.0.smithi046.stdout:1/567: rename d3/d27/d5d/d60/f65 to d3/d27/fb7 0 2024-09-10T04:05:28.723 INFO:tasks.workunit.client.1.smithi070.stdout:6/799: getdents d0/d3/d18/d1b/d17/d54/dc4 0 2024-09-10T04:05:28.725 INFO:tasks.workunit.client.0.smithi046.stdout:0/621: write d1/dc/d25/d33/d1a/d80/f8a [554421,104230] 0 2024-09-10T04:05:28.726 INFO:tasks.workunit.client.0.smithi046.stdout:0/622: chown d1/dc/d25/d33 4065 1 2024-09-10T04:05:28.727 INFO:tasks.workunit.client.0.smithi046.stdout:6/634: write da/db/d10/d27/f77 [3014955,49752] 0 2024-09-10T04:05:28.728 INFO:tasks.workunit.client.0.smithi046.stdout:0/623: write d1/dc/d25/d33/d4/fae [438440,62424] 0 2024-09-10T04:05:28.730 INFO:tasks.workunit.client.0.smithi046.stdout:4/874: sync 2024-09-10T04:05:28.730 INFO:tasks.workunit.client.0.smithi046.stdout:3/715: sync 2024-09-10T04:05:28.730 INFO:tasks.workunit.client.1.smithi070.stdout:5/822: dwrite d17/d19/d6a/d54/de1/dfe/dce/f110 [0,4194304] 0 2024-09-10T04:05:28.731 INFO:tasks.workunit.client.0.smithi046.stdout:9/731: unlink d4/d3c/d4d/d80/fd6 0 2024-09-10T04:05:28.734 INFO:tasks.workunit.client.0.smithi046.stdout:5/890: symlink d7/d11/d14/d2f/d2b/d92/l137 0 2024-09-10T04:05:28.737 INFO:tasks.workunit.client.0.smithi046.stdout:5/891: truncate d7/d11/d14/d2f/deb/ff8 952125 0 2024-09-10T04:05:28.738 INFO:tasks.workunit.client.0.smithi046.stdout:8/738: rename d1/d29/d19/d36/d6e/d75/ld0 to d1/d29/d19/d36/d4c/d5e/dc7/lf8 0 2024-09-10T04:05:28.741 INFO:tasks.workunit.client.1.smithi070.stdout:7/842: link d2/d7/d9/d18/d3c/d52/fb0 d2/f12a 0 2024-09-10T04:05:28.742 INFO:tasks.workunit.client.1.smithi070.stdout:7/843: dread - d2/d7/d9/d18/d23/fe5 zero size 2024-09-10T04:05:28.742 INFO:tasks.workunit.client.0.smithi046.stdout:9/732: dread d4/d3c/d4d/d5c/d59/f7f [0,4194304] 0 2024-09-10T04:05:28.743 INFO:tasks.workunit.client.0.smithi046.stdout:5/892: dread d7/d11/d14/d2f/deb/f10e [0,4194304] 0 2024-09-10T04:05:28.744 INFO:tasks.workunit.client.0.smithi046.stdout:5/893: chown d7/d11/d3f/d53/d81 2 1 2024-09-10T04:05:28.747 INFO:tasks.workunit.client.1.smithi070.stdout:0/960: mkdir d8/d13/d1a/d8f/d95/d106/d13f 0 2024-09-10T04:05:28.748 INFO:tasks.workunit.client.0.smithi046.stdout:6/635: mkdir da/db/d18/dd1 0 2024-09-10T04:05:28.750 INFO:tasks.workunit.client.0.smithi046.stdout:4/875: rmdir d3/d12/dec/df0 39 2024-09-10T04:05:28.751 INFO:tasks.workunit.client.1.smithi070.stdout:7/844: dwrite d2/d7/d9/f82 [4194304,4194304] 0 2024-09-10T04:05:28.751 INFO:tasks.workunit.client.1.smithi070.stdout:3/808: dread f3 [0,4194304] 0 2024-09-10T04:05:28.753 INFO:tasks.workunit.client.0.smithi046.stdout:7/818: mkdir d5/d9/d12/d72/d77/de0/d1f/d123 0 2024-09-10T04:05:28.754 INFO:tasks.workunit.client.1.smithi070.stdout:4/766: creat de/d13/da6/ddb/f103 x:0 0 0 2024-09-10T04:05:28.754 INFO:tasks.workunit.client.0.smithi046.stdout:3/716: mknod d12/d13/d37/d51/db5/cf8 0 2024-09-10T04:05:28.756 INFO:tasks.workunit.client.0.smithi046.stdout:8/739: fdatasync d1/d29/d17/d1f/d2a/f85 0 2024-09-10T04:05:28.760 INFO:tasks.workunit.client.0.smithi046.stdout:9/733: creat d4/d3c/d4d/d80/fe8 x:0 0 0 2024-09-10T04:05:28.761 INFO:tasks.workunit.client.1.smithi070.stdout:8/863: dread d4/dbf/d74/d11b/d1a/f78 [4194304,4194304] 0 2024-09-10T04:05:28.761 INFO:tasks.workunit.client.1.smithi070.stdout:2/766: dwrite f3 [4194304,4194304] 0 2024-09-10T04:05:28.762 INFO:tasks.workunit.client.1.smithi070.stdout:5/823: dread d17/d19/d35/f72 [0,4194304] 0 2024-09-10T04:05:28.764 INFO:tasks.workunit.client.1.smithi070.stdout:0/961: dwrite d8/d13/d1a/f3c [4194304,4194304] 0 2024-09-10T04:05:28.767 INFO:tasks.workunit.client.1.smithi070.stdout:5/824: readlink d17/d24/d2b/laf 0 2024-09-10T04:05:28.767 INFO:tasks.workunit.client.0.smithi046.stdout:5/894: creat d7/d11/d14/d2f/deb/f138 x:0 0 0 2024-09-10T04:05:28.767 INFO:tasks.workunit.client.1.smithi070.stdout:8/864: dread d4/dcf/f9a [0,4194304] 0 2024-09-10T04:05:28.768 INFO:tasks.workunit.client.0.smithi046.stdout:5/895: fsync d7/d11/d14/d45/ffb 0 2024-09-10T04:05:28.768 INFO:tasks.workunit.client.1.smithi070.stdout:1/727: getdents d3/d8/d12/d2f 0 2024-09-10T04:05:28.769 INFO:tasks.workunit.client.1.smithi070.stdout:2/767: read d15/d18/d29/d2a/f58 [669033,76873] 0 2024-09-10T04:05:28.770 INFO:tasks.workunit.client.0.smithi046.stdout:6/636: mknod da/db/d11/cd2 0 2024-09-10T04:05:28.772 INFO:tasks.workunit.client.0.smithi046.stdout:0/624: getdents d1/dc/d25/d33/d1a/d80 0 2024-09-10T04:05:28.773 INFO:tasks.workunit.client.0.smithi046.stdout:0/625: write d1/dc/f17 [1000459,3166] 0 2024-09-10T04:05:28.776 INFO:tasks.workunit.client.0.smithi046.stdout:4/876: symlink d3/d12/d17/d29/ddd/l12f 0 2024-09-10T04:05:28.777 INFO:tasks.workunit.client.1.smithi070.stdout:8/865: dwrite d4/dbf/d74/d11b/d19/d62/df0/f109 [0,4194304] 0 2024-09-10T04:05:28.779 INFO:tasks.workunit.client.0.smithi046.stdout:2/709: sync 2024-09-10T04:05:28.780 INFO:tasks.workunit.client.1.smithi070.stdout:8/866: dread - d4/dbf/d74/d11b/d19/f117 zero size 2024-09-10T04:05:28.781 INFO:tasks.workunit.client.0.smithi046.stdout:7/819: creat d5/d9/d112/f124 x:0 0 0 2024-09-10T04:05:28.781 INFO:tasks.workunit.client.0.smithi046.stdout:7/820: dread - d5/d6/d39/f11a zero size 2024-09-10T04:05:28.783 INFO:tasks.workunit.client.1.smithi070.stdout:4/767: truncate de/d13/d5b/d7b/dd5/f71 3857435 0 2024-09-10T04:05:28.784 INFO:tasks.workunit.client.1.smithi070.stdout:9/845: truncate d9/d59/d2a/f101 365124 0 2024-09-10T04:05:28.787 INFO:tasks.workunit.client.1.smithi070.stdout:4/768: dread fc [0,4194304] 0 2024-09-10T04:05:28.788 INFO:tasks.workunit.client.1.smithi070.stdout:5/825: unlink d17/d19/d6a/d54/de1/dfe/d93/lb7 0 2024-09-10T04:05:28.788 INFO:tasks.workunit.client.0.smithi046.stdout:3/717: rmdir d12/d13/d37/d94 39 2024-09-10T04:05:28.789 INFO:tasks.workunit.client.1.smithi070.stdout:5/826: stat d17/d19/d6a/d54/de1/dfe/ca7 0 2024-09-10T04:05:28.789 INFO:tasks.workunit.client.1.smithi070.stdout:1/728: mkdir d3/d10/db2/dee 0 2024-09-10T04:05:28.790 INFO:tasks.workunit.client.1.smithi070.stdout:2/768: chown d15/d18/d56/d7c/d55/d6d/c75 1947 1 2024-09-10T04:05:28.791 INFO:tasks.workunit.client.1.smithi070.stdout:2/769: stat d15/d18/d56/d7c/d55/d6d/lb4 0 2024-09-10T04:05:28.792 INFO:tasks.workunit.client.1.smithi070.stdout:3/809: symlink db/d1c/d25/d28/l116 0 2024-09-10T04:05:28.792 INFO:tasks.workunit.client.1.smithi070.stdout:3/810: readlink db/d1c/d25/d28/l116 0 2024-09-10T04:05:28.794 INFO:tasks.workunit.client.1.smithi070.stdout:7/845: creat d2/d7/d9/d18/d3c/d52/d10e/f12b x:0 0 0 2024-09-10T04:05:28.795 INFO:tasks.workunit.client.0.smithi046.stdout:9/734: unlink d4/d7/d1f/d54/dc7/dc8/cda 0 2024-09-10T04:05:28.801 INFO:tasks.workunit.client.0.smithi046.stdout:6/637: creat da/db/d11/d13/d1e/d23/d86/fd3 x:0 0 0 2024-09-10T04:05:28.804 INFO:tasks.workunit.client.0.smithi046.stdout:0/626: rename d1/dc/d25/d33/d3/f46 to d1/dc/d25/d33/d3/d29/fc2 0 2024-09-10T04:05:28.804 INFO:tasks.workunit.client.1.smithi070.stdout:4/769: rmdir de/d13/d5b 39 2024-09-10T04:05:28.805 INFO:tasks.workunit.client.1.smithi070.stdout:2/770: dread d15/d18/d29/d2a/f67 [0,4194304] 0 2024-09-10T04:05:28.806 INFO:tasks.workunit.client.0.smithi046.stdout:2/710: read - d2/d6/d9d/db6/fb7 zero size 2024-09-10T04:05:28.814 INFO:tasks.workunit.client.1.smithi070.stdout:3/811: creat db/d1c/d3e/f117 x:0 0 0 2024-09-10T04:05:28.814 INFO:tasks.workunit.client.0.smithi046.stdout:8/740: dread - d1/d29/d17/d1f/d46/d67/dc1/f60 zero size 2024-09-10T04:05:28.816 INFO:tasks.workunit.client.1.smithi070.stdout:7/846: creat d2/d40/d6c/db2/f12c x:0 0 0 2024-09-10T04:05:28.817 INFO:tasks.workunit.client.0.smithi046.stdout:1/568: link d3/dc/c11 d3/cb8 0 2024-09-10T04:05:28.823 INFO:tasks.workunit.client.1.smithi070.stdout:0/962: dwrite d8/d13/de2/d74/fa1 [0,4194304] 0 2024-09-10T04:05:28.823 INFO:tasks.workunit.client.0.smithi046.stdout:6/638: symlink da/db/d11/d13/d1e/d23/ld4 0 2024-09-10T04:05:28.824 INFO:tasks.workunit.client.1.smithi070.stdout:0/963: dread - d8/d13/de2/d4f/d125/f13a zero size 2024-09-10T04:05:28.828 INFO:tasks.workunit.client.0.smithi046.stdout:4/877: rename d3/d12/d17/d22/d30/l5d to d3/d12/d17/d22/d51/d94/l130 0 2024-09-10T04:05:28.830 INFO:tasks.workunit.client.1.smithi070.stdout:1/729: mkdir d3/d8/d13/d67/d83/def 0 2024-09-10T04:05:28.830 INFO:tasks.workunit.client.1.smithi070.stdout:5/827: dwrite f10 [0,4194304] 0 2024-09-10T04:05:28.830 INFO:tasks.workunit.client.0.smithi046.stdout:2/711: truncate d2/d8/d37/fc1 813760 0 2024-09-10T04:05:28.832 INFO:tasks.workunit.client.1.smithi070.stdout:5/828: readlink d17/d24/d84/lb3 0 2024-09-10T04:05:28.833 INFO:tasks.workunit.client.1.smithi070.stdout:5/829: chown d17/d24/f2f 3048634 1 2024-09-10T04:05:28.834 INFO:tasks.workunit.client.0.smithi046.stdout:7/821: unlink d5/d9/d12/d72/d77/de0/d1e/cd8 0 2024-09-10T04:05:28.834 INFO:tasks.workunit.client.1.smithi070.stdout:0/964: dwrite d8/d24/d2d/f51 [0,4194304] 0 2024-09-10T04:05:28.836 INFO:tasks.workunit.client.1.smithi070.stdout:5/830: dread - d17/d19/d6a/d54/de1/dfe/d93/f107 zero size 2024-09-10T04:05:28.836 INFO:tasks.workunit.client.1.smithi070.stdout:3/812: creat db/d1c/d25/d28/dd3/de5/f118 x:0 0 0 2024-09-10T04:05:28.836 INFO:tasks.workunit.client.0.smithi046.stdout:5/896: write d7/ffd [500743,17902] 0 2024-09-10T04:05:28.839 INFO:tasks.workunit.client.1.smithi070.stdout:7/847: mkdir d2/dc8/d119/d7e/d12d 0 2024-09-10T04:05:28.839 INFO:tasks.workunit.client.0.smithi046.stdout:1/569: mknod d3/d27/d5d/d60/cb9 0 2024-09-10T04:05:28.840 INFO:tasks.workunit.client.1.smithi070.stdout:7/848: write d2/d3/dc/d13/dfe/d9f/ff2 [671181,23646] 0 2024-09-10T04:05:28.841 INFO:tasks.workunit.client.0.smithi046.stdout:9/735: mknod d4/ce9 0 2024-09-10T04:05:28.843 INFO:tasks.workunit.client.1.smithi070.stdout:9/846: creat d9/d59/f10d x:0 0 0 2024-09-10T04:05:28.848 INFO:tasks.workunit.client.0.smithi046.stdout:8/741: rename d1/d29/d17/d1f/d46/d67/cb4 to d1/d29/d19/d36/d4c/d5e/cf9 0 2024-09-10T04:05:28.849 INFO:tasks.workunit.client.0.smithi046.stdout:0/627: mkdir d1/dc/d49/db2/dc3 0 2024-09-10T04:05:28.851 INFO:tasks.workunit.client.0.smithi046.stdout:0/628: fsync d1/dc/d25/d33/d1a/d41/f9d 0 2024-09-10T04:05:28.855 INFO:tasks.workunit.client.1.smithi070.stdout:6/800: sync 2024-09-10T04:05:28.855 INFO:tasks.workunit.client.0.smithi046.stdout:3/718: creat d12/d13/ff9 x:0 0 0 2024-09-10T04:05:28.857 INFO:tasks.workunit.client.0.smithi046.stdout:3/719: dread - d12/d13/d14/d1e/dd2/fe6 zero size 2024-09-10T04:05:28.857 INFO:tasks.workunit.client.1.smithi070.stdout:0/965: unlink d8/d24/d2d/d5f/c107 0 2024-09-10T04:05:28.857 INFO:tasks.workunit.client.1.smithi070.stdout:0/966: readlink d8/d24/l37 0 2024-09-10T04:05:28.858 INFO:tasks.workunit.client.1.smithi070.stdout:0/967: dread - d8/d24/ffd zero size 2024-09-10T04:05:28.858 INFO:tasks.workunit.client.0.smithi046.stdout:1/570: symlink d3/dc/d34/d58/lba 0 2024-09-10T04:05:28.859 INFO:tasks.workunit.client.0.smithi046.stdout:1/571: readlink d3/dc/d34/d58/lb6 0 2024-09-10T04:05:28.859 INFO:tasks.workunit.client.1.smithi070.stdout:3/813: symlink db/d1c/d25/dd8/l119 0 2024-09-10T04:05:28.860 INFO:tasks.workunit.client.0.smithi046.stdout:9/736: creat d4/d7/d81/fea x:0 0 0 2024-09-10T04:05:28.862 INFO:tasks.workunit.client.1.smithi070.stdout:5/831: symlink d17/d19/d6a/d54/d57/d63/db9/dc2/l11e 0 2024-09-10T04:05:28.864 INFO:tasks.workunit.client.1.smithi070.stdout:7/849: rename d2/d3/dc/d5d/d41/da8 to d2/d40/d12e 0 2024-09-10T04:05:28.865 INFO:tasks.workunit.client.1.smithi070.stdout:7/850: readlink d2/d3/dc/d13/dfe/l4b 0 2024-09-10T04:05:28.866 INFO:tasks.workunit.client.0.smithi046.stdout:4/878: rename d3/d12/d17/d22/d30/fff to d3/d12/d17/d22/d51/d94/f131 0 2024-09-10T04:05:28.867 INFO:tasks.workunit.client.1.smithi070.stdout:9/847: mkdir d9/d1c/d34/d8b/d10e 0 2024-09-10T04:05:28.868 INFO:tasks.workunit.client.0.smithi046.stdout:8/742: mknod d1/d29/d17/d1f/cfa 0 2024-09-10T04:05:28.869 INFO:tasks.workunit.client.0.smithi046.stdout:8/743: write d1/d29/d19/db7/feb [862939,18719] 0 2024-09-10T04:05:28.870 INFO:tasks.workunit.client.1.smithi070.stdout:4/770: dwrite de/d13/d5b/d7b/dd5/f60 [0,4194304] 0 2024-09-10T04:05:28.872 INFO:tasks.workunit.client.1.smithi070.stdout:9/848: dread d9/d1c/dac/d8a/d3c/d92/fec [0,4194304] 0 2024-09-10T04:05:28.875 INFO:tasks.workunit.client.1.smithi070.stdout:3/814: read - db/d1c/d25/d28/dd3/de5/d10f/fbb zero size 2024-09-10T04:05:28.875 INFO:tasks.workunit.client.1.smithi070.stdout:5/832: read - d17/fd7 zero size 2024-09-10T04:05:28.876 INFO:tasks.workunit.client.0.smithi046.stdout:6/639: link da/db/d18/c73 da/db/d11/d13/d4f/cd5 0 2024-09-10T04:05:28.879 INFO:tasks.workunit.client.1.smithi070.stdout:1/730: dwrite d3/d8/d13/d3f/f52 [0,4194304] 0 2024-09-10T04:05:28.880 INFO:tasks.workunit.client.0.smithi046.stdout:4/879: truncate d3/d12/d17/d29/d58/db2/d5f/d85/fa2 5004865 0 2024-09-10T04:05:28.882 INFO:tasks.workunit.client.0.smithi046.stdout:4/880: truncate d3/d12/d14/f111 731297 0 2024-09-10T04:05:28.882 INFO:tasks.workunit.client.0.smithi046.stdout:8/744: rename d1/d29/d17/d1f/d46/f8b to d1/d29/d17/d1f/d46/ffb 0 2024-09-10T04:05:28.883 INFO:tasks.workunit.client.1.smithi070.stdout:7/851: dwrite d2/d3/dc/d5d/d41/f50 [0,4194304] 0 2024-09-10T04:05:28.885 INFO:tasks.workunit.client.1.smithi070.stdout:4/771: mknod de/d13/d18/d26/d8c/d93/c104 0 2024-09-10T04:05:28.886 INFO:tasks.workunit.client.1.smithi070.stdout:7/852: write d2/d3/dc/d13/dfe/fe4 [732130,44068] 0 2024-09-10T04:05:28.887 INFO:tasks.workunit.client.0.smithi046.stdout:2/712: link d2/d6/d23/d5d/d77/d7b/d85/c6b d2/d6/d23/d42/cff 0 2024-09-10T04:05:28.889 INFO:tasks.workunit.client.1.smithi070.stdout:2/771: link d15/d18/d56/d7c/l83 d15/d18/l117 0 2024-09-10T04:05:28.889 INFO:tasks.workunit.client.1.smithi070.stdout:2/772: stat cd 0 2024-09-10T04:05:28.898 INFO:tasks.workunit.client.0.smithi046.stdout:3/720: symlink d12/d13/d37/d94/dbf/lfa 0 2024-09-10T04:05:28.900 INFO:tasks.workunit.client.1.smithi070.stdout:3/815: mkdir db/d83/d11a 0 2024-09-10T04:05:28.901 INFO:tasks.workunit.client.1.smithi070.stdout:5/833: mkdir d17/d19/d6a/d54/de1/d11f 0 2024-09-10T04:05:28.906 INFO:tasks.workunit.client.0.smithi046.stdout:2/713: dread d2/d6/d23/d5d/d77/f54 [0,4194304] 0 2024-09-10T04:05:28.908 INFO:tasks.workunit.client.0.smithi046.stdout:8/745: truncate d1/d29/d12/f1e 3920914 0 2024-09-10T04:05:28.909 INFO:tasks.workunit.client.0.smithi046.stdout:9/737: dwrite d4/d3c/d4d/d5c/d59/f5d [0,4194304] 0 2024-09-10T04:05:28.911 INFO:tasks.workunit.client.0.smithi046.stdout:0/629: truncate d1/dc/d25/d33/d3/dd/d55/f43 147909 0 2024-09-10T04:05:28.914 INFO:tasks.workunit.client.0.smithi046.stdout:7/822: getdents d5/d6/d32/dab 0 2024-09-10T04:05:28.916 INFO:tasks.workunit.client.1.smithi070.stdout:6/801: creat d0/d3/d18/d1b/d17/ffb x:0 0 0 2024-09-10T04:05:28.916 INFO:tasks.workunit.client.0.smithi046.stdout:7/823: write d5/d6/d32/f95 [339524,47457] 0 2024-09-10T04:05:28.917 INFO:tasks.workunit.client.1.smithi070.stdout:6/802: fsync d0/d3/d18/d2a/fdd 0 2024-09-10T04:05:28.918 INFO:tasks.workunit.client.0.smithi046.stdout:6/640: write da/db/d11/d2b/d55/f81 [998090,79212] 0 2024-09-10T04:05:28.920 INFO:tasks.workunit.client.1.smithi070.stdout:9/849: symlink d9/d1c/d34/d8b/d10e/l10f 0 2024-09-10T04:05:28.920 INFO:tasks.workunit.client.0.smithi046.stdout:8/746: dwrite d1/d29/fd7 [0,4194304] 0 2024-09-10T04:05:28.924 INFO:tasks.workunit.client.1.smithi070.stdout:9/850: read d9/d59/f1a [2123806,115357] 0 2024-09-10T04:05:28.924 INFO:tasks.workunit.client.0.smithi046.stdout:1/572: rename d3/d27/f18 to d3/da1/fbb 0 2024-09-10T04:05:28.927 INFO:tasks.workunit.client.1.smithi070.stdout:1/731: creat d3/d10/d1d/d79/ff0 x:0 0 0 2024-09-10T04:05:28.929 INFO:tasks.workunit.client.0.smithi046.stdout:2/714: creat d2/d8/d65/d90/f100 x:0 0 0 2024-09-10T04:05:28.929 INFO:tasks.workunit.client.0.smithi046.stdout:6/641: dwrite da/db/d11/d2b/d55/f81 [0,4194304] 0 2024-09-10T04:05:28.930 INFO:tasks.workunit.client.0.smithi046.stdout:4/881: truncate d3/d12/d17/d22/d51/d94/f131 330454 0 2024-09-10T04:05:28.931 INFO:tasks.workunit.client.1.smithi070.stdout:6/803: dread d0/d3/f98 [0,4194304] 0 2024-09-10T04:05:28.932 INFO:tasks.workunit.client.0.smithi046.stdout:2/715: readlink d2/d6/d23/d5d/d77/d7b/dd8/le9 0 2024-09-10T04:05:28.932 INFO:tasks.workunit.client.1.smithi070.stdout:6/804: chown d0/d3/d18/d1b/d17/c9c 247235 1 2024-09-10T04:05:28.933 INFO:tasks.workunit.client.0.smithi046.stdout:2/716: chown d2/l20 188827307 1 2024-09-10T04:05:28.934 INFO:tasks.workunit.client.1.smithi070.stdout:6/805: read d0/d3/f51 [1483963,126363] 0 2024-09-10T04:05:28.936 INFO:tasks.workunit.client.0.smithi046.stdout:3/721: dread d12/d13/d14/d1e/f3f [0,4194304] 0 2024-09-10T04:05:28.938 INFO:tasks.workunit.client.1.smithi070.stdout:0/968: getdents d8/d24/dce 0 2024-09-10T04:05:28.939 INFO:tasks.workunit.client.0.smithi046.stdout:0/630: mkdir d1/dc/d25/dc4 0 2024-09-10T04:05:28.944 INFO:tasks.workunit.client.1.smithi070.stdout:3/816: dwrite db/d1c/d53/f105 [0,4194304] 0 2024-09-10T04:05:28.950 INFO:tasks.workunit.client.0.smithi046.stdout:8/747: fdatasync d1/d29/d19/d36/d6e/f95 0 2024-09-10T04:05:28.953 INFO:tasks.workunit.client.0.smithi046.stdout:1/573: rename d3/dc/d21/d3a/fb2 to d3/d9b/fbc 0 2024-09-10T04:05:28.954 INFO:tasks.workunit.client.1.smithi070.stdout:7/853: link d2/d7/d9/l95 d2/d3/dc/d5d/l12f 0 2024-09-10T04:05:28.954 INFO:tasks.workunit.client.0.smithi046.stdout:1/574: readlink d3/d27/l49 0 2024-09-10T04:05:28.955 INFO:tasks.workunit.client.0.smithi046.stdout:6/642: mkdir da/db/d11/d9f/dd6 0 2024-09-10T04:05:28.956 INFO:tasks.workunit.client.0.smithi046.stdout:8/748: dread d1/d29/fd7 [0,4194304] 0 2024-09-10T04:05:28.957 INFO:tasks.workunit.client.1.smithi070.stdout:6/806: rmdir d0/d3/d18/d1b/d17/d53 39 2024-09-10T04:05:28.957 INFO:tasks.workunit.client.0.smithi046.stdout:6/643: chown da/db/d11/d9f 2185463 1 2024-09-10T04:05:28.957 INFO:tasks.workunit.client.1.smithi070.stdout:6/807: chown d0/d3/d18/d1b/d33 2088329 1 2024-09-10T04:05:28.958 INFO:tasks.workunit.client.0.smithi046.stdout:8/749: chown d1/d29/d17/d1f/d46/d76/fab 129991198 1 2024-09-10T04:05:28.959 INFO:tasks.workunit.client.1.smithi070.stdout:0/969: chown d8/d13/d47/c10d 10 1 2024-09-10T04:05:28.959 INFO:tasks.workunit.client.0.smithi046.stdout:8/750: readlink d1/d29/d17/d1f/d46/d6a/d82/dc2/led 0 2024-09-10T04:05:28.959 INFO:tasks.workunit.client.1.smithi070.stdout:5/834: truncate d17/d19/d6a/d54/d57/f88 3063170 0 2024-09-10T04:05:28.960 INFO:tasks.workunit.client.0.smithi046.stdout:7/824: write d5/d6/db/f29 [48867,103250] 0 2024-09-10T04:05:28.961 INFO:tasks.workunit.client.0.smithi046.stdout:2/717: dread d2/d8/d37/fc1 [0,4194304] 0 2024-09-10T04:05:28.962 INFO:tasks.workunit.client.0.smithi046.stdout:7/825: chown d5/d9/d12/d72/d77/de0/d1e 159773 1 2024-09-10T04:05:28.964 INFO:tasks.workunit.client.0.smithi046.stdout:9/738: truncate d4/d18/f38 5745510 0 2024-09-10T04:05:28.965 INFO:tasks.workunit.client.0.smithi046.stdout:9/739: chown d4/d7/d81/fea 62835937 1 2024-09-10T04:05:28.965 INFO:tasks.workunit.client.1.smithi070.stdout:7/854: dread d2/d7/d9/d18/f99 [0,4194304] 0 2024-09-10T04:05:28.966 INFO:tasks.workunit.client.0.smithi046.stdout:3/722: creat d12/d13/d37/d51/db5/d50/d52/ffb x:0 0 0 2024-09-10T04:05:28.967 INFO:tasks.workunit.client.1.smithi070.stdout:4/772: getdents de/d13/d18/d23/d47/d59/d79/dbc 0 2024-09-10T04:05:28.971 INFO:tasks.workunit.client.1.smithi070.stdout:6/808: creat d0/d3/d18/d1b/d17/d54/dc4/dd2/dde/ffc x:0 0 0 2024-09-10T04:05:28.972 INFO:tasks.workunit.client.1.smithi070.stdout:7/855: dwrite d2/f55 [0,4194304] 0 2024-09-10T04:05:28.975 INFO:tasks.workunit.client.1.smithi070.stdout:0/970: creat d8/d13/d47/d7b/f140 x:0 0 0 2024-09-10T04:05:28.975 INFO:tasks.workunit.client.1.smithi070.stdout:9/851: creat d9/d1c/f110 x:0 0 0 2024-09-10T04:05:28.975 INFO:tasks.workunit.client.1.smithi070.stdout:7/856: fdatasync d2/d40/d6c/db2/f12c 0 2024-09-10T04:05:28.976 INFO:tasks.workunit.client.1.smithi070.stdout:5/835: mkdir d17/d6f/d32/d90/da5/d120 0 2024-09-10T04:05:28.978 INFO:tasks.workunit.client.1.smithi070.stdout:3/817: dwrite db/d1c/d25/d28/dd3/ff9 [0,4194304] 0 2024-09-10T04:05:28.978 INFO:tasks.workunit.client.0.smithi046.stdout:7/826: dread d5/d6/fb5 [0,4194304] 0 2024-09-10T04:05:28.979 INFO:tasks.workunit.client.1.smithi070.stdout:8/867: sync 2024-09-10T04:05:28.979 INFO:tasks.workunit.client.1.smithi070.stdout:0/971: write d8/d13/de2/d4f/ff9 [2496551,18577] 0 2024-09-10T04:05:28.980 INFO:tasks.workunit.client.1.smithi070.stdout:1/732: creat d3/d8/d13/ff1 x:0 0 0 2024-09-10T04:05:28.981 INFO:tasks.workunit.client.1.smithi070.stdout:4/773: symlink de/d13/d18/d26/l105 0 2024-09-10T04:05:28.984 INFO:tasks.workunit.client.1.smithi070.stdout:6/809: fdatasync d0/d3/f98 0 2024-09-10T04:05:28.989 INFO:tasks.workunit.client.1.smithi070.stdout:4/774: dwrite de/d13/f45 [0,4194304] 0 2024-09-10T04:05:28.993 INFO:tasks.workunit.client.1.smithi070.stdout:5/836: symlink d17/d19/d6a/d54/de1/dfe/l121 0 2024-09-10T04:05:28.994 INFO:tasks.workunit.client.1.smithi070.stdout:7/857: dread d2/d7/d9/d18/f99 [0,4194304] 0 2024-09-10T04:05:29.004 INFO:tasks.workunit.client.1.smithi070.stdout:4/775: rmdir de/d13/d18/d26/d8c 39 2024-09-10T04:05:29.008 INFO:tasks.workunit.client.1.smithi070.stdout:7/858: creat d2/d66/f130 x:0 0 0 2024-09-10T04:05:29.013 INFO:tasks.workunit.client.1.smithi070.stdout:3/818: dwrite db/f14 [0,4194304] 0 2024-09-10T04:05:29.014 INFO:tasks.workunit.client.1.smithi070.stdout:9/852: creat d9/f111 x:0 0 0 2024-09-10T04:05:29.015 INFO:tasks.workunit.client.1.smithi070.stdout:4/776: mkdir de/d13/d18/d23/d47/d59/d106 0 2024-09-10T04:05:29.017 INFO:tasks.workunit.client.1.smithi070.stdout:7/859: creat d2/d7/d9/d18/d3c/d52/f131 x:0 0 0 2024-09-10T04:05:29.018 INFO:tasks.workunit.client.1.smithi070.stdout:1/733: dwrite d3/d8/d12/d2f/f7e [0,4194304] 0 2024-09-10T04:05:29.021 INFO:tasks.workunit.client.1.smithi070.stdout:8/868: write d4/d16/d73/fba [811177,94576] 0 2024-09-10T04:05:29.022 INFO:tasks.workunit.client.1.smithi070.stdout:3/819: mkdir db/d83/d92/d11b 0 2024-09-10T04:05:29.023 INFO:tasks.workunit.client.1.smithi070.stdout:5/837: link d17/d24/d84/l103 d17/d24/dcc/l122 0 2024-09-10T04:05:29.025 INFO:tasks.workunit.client.1.smithi070.stdout:1/734: creat d3/d10/db2/ff2 x:0 0 0 2024-09-10T04:05:29.027 INFO:tasks.workunit.client.1.smithi070.stdout:0/972: getdents d8/d24/d2d/d5f 0 2024-09-10T04:05:29.028 INFO:tasks.workunit.client.1.smithi070.stdout:1/735: stat d3/d8/d13/fc4 0 2024-09-10T04:05:29.028 INFO:tasks.workunit.client.0.smithi046.stdout:0/631: rename d1/d13/c18 to d1/d13/d37/d75/d94/d9a/da4/cc5 0 2024-09-10T04:05:29.029 INFO:tasks.workunit.client.1.smithi070.stdout:9/853: mkdir d9/d59/d112 0 2024-09-10T04:05:29.029 INFO:tasks.workunit.client.1.smithi070.stdout:0/973: dread - d8/d13/d1a/d8f/d95/d106/f123 zero size 2024-09-10T04:05:29.030 INFO:tasks.workunit.client.0.smithi046.stdout:1/575: symlink d3/dc/d34/d58/lbd 0 2024-09-10T04:05:29.030 INFO:tasks.workunit.client.1.smithi070.stdout:8/869: rmdir d4/dbf/d74/d11b/d1a/d60 39 2024-09-10T04:05:29.032 INFO:tasks.workunit.client.0.smithi046.stdout:4/882: symlink d3/d12/dec/d106/l132 0 2024-09-10T04:05:29.032 INFO:tasks.workunit.client.1.smithi070.stdout:4/777: dread de/d13/f1a [0,4194304] 0 2024-09-10T04:05:29.036 INFO:tasks.workunit.client.1.smithi070.stdout:0/974: chown d8/d24/d36/d137 1490661 1 2024-09-10T04:05:29.036 INFO:tasks.workunit.client.1.smithi070.stdout:3/820: unlink db/d1c/d25/d28/d43/fa4 0 2024-09-10T04:05:29.037 INFO:tasks.workunit.client.1.smithi070.stdout:8/870: chown d4/dbf/d74/da4/fbd 4518544 1 2024-09-10T04:05:29.037 INFO:tasks.workunit.client.1.smithi070.stdout:4/778: write de/d13/d5b/d7b/dd5/daf/db7/ff3 [495340,119698] 0 2024-09-10T04:05:29.037 INFO:tasks.workunit.client.0.smithi046.stdout:8/751: creat d1/d29/d17/d1f/d46/d6a/ffc x:0 0 0 2024-09-10T04:05:29.037 INFO:tasks.workunit.client.1.smithi070.stdout:7/860: dread d2/d7/d9/d18/d23/f47 [0,4194304] 0 2024-09-10T04:05:29.038 INFO:tasks.workunit.client.1.smithi070.stdout:7/861: chown d2/d40/c124 179 1 2024-09-10T04:05:29.042 INFO:tasks.workunit.client.0.smithi046.stdout:9/740: rmdir d4/d7/d1f/dce 39 2024-09-10T04:05:29.047 INFO:tasks.workunit.client.1.smithi070.stdout:1/736: dread d3/d8/d27/d7a/d94/f9e [0,4194304] 0 2024-09-10T04:05:29.048 INFO:tasks.workunit.client.0.smithi046.stdout:2/718: dread d2/d6/d23/d42/f53 [0,4194304] 0 2024-09-10T04:05:29.048 INFO:tasks.workunit.client.0.smithi046.stdout:1/576: dread d3/d9b/f9e [0,4194304] 0 2024-09-10T04:05:29.050 INFO:tasks.workunit.client.1.smithi070.stdout:8/871: truncate d4/d16/f22 2244461 0 2024-09-10T04:05:29.052 INFO:tasks.workunit.client.0.smithi046.stdout:6/644: rename da/dc/d22 to da/db/d11/dd7 0 2024-09-10T04:05:29.053 INFO:tasks.workunit.client.1.smithi070.stdout:7/862: readlink d2/d40/d6c/ld8 0 2024-09-10T04:05:29.055 INFO:tasks.workunit.client.1.smithi070.stdout:2/773: sync 2024-09-10T04:05:29.055 INFO:tasks.workunit.client.1.smithi070.stdout:9/854: rename d9/d1c/d34/l77 to d9/d1c/d34/d8b/dcc/l113 0 2024-09-10T04:05:29.059 INFO:tasks.workunit.client.1.smithi070.stdout:1/737: dread d3/d8/d13/fc4 [0,4194304] 0 2024-09-10T04:05:29.060 INFO:tasks.workunit.client.0.smithi046.stdout:8/752: rmdir d1/d29/d17/d1f/d46/d67/dc1/d9d 39 2024-09-10T04:05:29.060 INFO:tasks.workunit.client.1.smithi070.stdout:7/863: dread d2/d7/db/f92 [0,4194304] 0 2024-09-10T04:05:29.061 INFO:tasks.workunit.client.1.smithi070.stdout:5/838: link d17/d24/d84/d9b/cbb d17/d19/d6a/d54/d57/d63/db9/c123 0 2024-09-10T04:05:29.061 INFO:tasks.workunit.client.1.smithi070.stdout:5/839: dread - d17/d24/dbf/fd3 zero size 2024-09-10T04:05:29.064 INFO:tasks.workunit.client.0.smithi046.stdout:9/741: rmdir d4/d3c/d4d/d5c 39 2024-09-10T04:05:29.064 INFO:tasks.workunit.client.0.smithi046.stdout:4/883: dwrite d3/d12/d17/d29/ddd/f116 [4194304,4194304] 0 2024-09-10T04:05:29.064 INFO:tasks.workunit.client.1.smithi070.stdout:8/872: rmdir d4/dbf/d74/d11b/d19 39 2024-09-10T04:05:29.068 INFO:tasks.workunit.client.1.smithi070.stdout:4/779: dwrite de/f31 [0,4194304] 0 2024-09-10T04:05:29.070 INFO:tasks.workunit.client.0.smithi046.stdout:9/742: dread d4/d7/f31 [0,4194304] 0 2024-09-10T04:05:29.071 INFO:tasks.workunit.client.0.smithi046.stdout:9/743: write d4/d7/d1f/d54/dc7/fdd [146925,7501] 0 2024-09-10T04:05:29.074 INFO:tasks.workunit.client.1.smithi070.stdout:2/774: creat d15/d18/d29/d64/f118 x:0 0 0 2024-09-10T04:05:29.079 INFO:tasks.workunit.client.1.smithi070.stdout:0/975: dwrite d8/d13/d1a/f23 [0,4194304] 0 2024-09-10T04:05:29.079 INFO:tasks.workunit.client.1.smithi070.stdout:1/738: rmdir d3/d8/d92 39 2024-09-10T04:05:29.080 INFO:tasks.workunit.client.0.smithi046.stdout:3/723: write d12/d13/d37/f60 [1418424,100569] 0 2024-09-10T04:05:29.080 INFO:tasks.workunit.client.0.smithi046.stdout:1/577: symlink d3/d27/d5d/d60/lbe 0 2024-09-10T04:05:29.084 INFO:tasks.workunit.client.0.smithi046.stdout:7/827: truncate d5/d6/db/f38 1739827 0 2024-09-10T04:05:29.085 INFO:tasks.workunit.client.0.smithi046.stdout:7/828: readlink d5/d6/d39/d5f/l94 0 2024-09-10T04:05:29.087 INFO:tasks.workunit.client.1.smithi070.stdout:7/864: creat d2/d3/dc/d5d/d2d/f132 x:0 0 0 2024-09-10T04:05:29.088 INFO:tasks.workunit.client.0.smithi046.stdout:6/645: creat da/db/d11/d2b/d5a/fd8 x:0 0 0 2024-09-10T04:05:29.089 INFO:tasks.workunit.client.0.smithi046.stdout:5/897: sync 2024-09-10T04:05:29.091 INFO:tasks.workunit.client.0.smithi046.stdout:5/898: write d7/d11/d14/d2f/f126 [409449,35450] 0 2024-09-10T04:05:29.092 INFO:tasks.workunit.client.1.smithi070.stdout:8/873: mknod d4/dbf/d74/d11b/d1a/df7/c12b 0 2024-09-10T04:05:29.092 INFO:tasks.workunit.client.0.smithi046.stdout:5/899: readlink d7/d11/d3f/d53/d75/l12c 0 2024-09-10T04:05:29.092 INFO:tasks.workunit.client.0.smithi046.stdout:2/719: write d2/d6/d23/d5d/d77/d7b/d85/d8d/fc5 [1320517,104494] 0 2024-09-10T04:05:29.093 INFO:tasks.workunit.client.0.smithi046.stdout:8/753: mknod d1/d29/d19/d36/d6e/d75/cfd 0 2024-09-10T04:05:29.096 INFO:tasks.workunit.client.0.smithi046.stdout:4/884: creat d3/d12/d17/d22/d51/d5c/d11b/f133 x:0 0 0 2024-09-10T04:05:29.099 INFO:tasks.workunit.client.1.smithi070.stdout:2/775: symlink d15/d18/d56/l119 0 2024-09-10T04:05:29.099 INFO:tasks.workunit.client.0.smithi046.stdout:8/754: dread d1/d29/d17/d1f/d46/d6a/d82/d99/fb3 [0,4194304] 0 2024-09-10T04:05:29.100 INFO:tasks.workunit.client.0.smithi046.stdout:8/755: chown d1/c30 8604 1 2024-09-10T04:05:29.101 INFO:tasks.workunit.client.0.smithi046.stdout:8/756: write d1/d29/d19/d36/d4c/d5e/f7f [1498025,71347] 0 2024-09-10T04:05:29.103 INFO:tasks.workunit.client.1.smithi070.stdout:3/821: link db/d1c/d25/d28/c7b db/d83/d92/d11b/c11c 0 2024-09-10T04:05:29.104 INFO:tasks.workunit.client.0.smithi046.stdout:3/724: truncate d12/f74 1675355 0 2024-09-10T04:05:29.105 INFO:tasks.workunit.client.0.smithi046.stdout:3/725: write d12/dd9/ddf/fe2 [765499,88132] 0 2024-09-10T04:05:29.107 INFO:tasks.workunit.client.1.smithi070.stdout:4/780: dwrite de/d13/d18/d23/d39/f8e [0,4194304] 0 2024-09-10T04:05:29.109 INFO:tasks.workunit.client.0.smithi046.stdout:9/744: write d4/d7/f5e [1073355,60116] 0 2024-09-10T04:05:29.114 INFO:tasks.workunit.client.1.smithi070.stdout:7/865: mkdir d2/d3/dc/d13/dfe/d133 0 2024-09-10T04:05:29.115 INFO:tasks.workunit.client.0.smithi046.stdout:1/578: write d3/d27/f42 [719408,112472] 0 2024-09-10T04:05:29.116 INFO:tasks.workunit.client.0.smithi046.stdout:6/646: truncate da/db/d11/d2b/d2f/f66 2872527 0 2024-09-10T04:05:29.116 INFO:tasks.workunit.client.1.smithi070.stdout:7/866: write d2/d3/dc/d13/dfe/fe4 [1785721,17] 0 2024-09-10T04:05:29.127 INFO:tasks.workunit.client.1.smithi070.stdout:4/781: mkdir de/d13/d5b/da0/d107 0 2024-09-10T04:05:29.130 INFO:tasks.workunit.client.1.smithi070.stdout:5/840: write d17/d24/d45/f51 [1347806,31085] 0 2024-09-10T04:05:29.132 INFO:tasks.workunit.client.1.smithi070.stdout:9/855: link d9/d1c/dac/d8a/dc4/dda/de0/d90/df0/df3/df4/lff d9/d7a/d74/d8c/l114 0 2024-09-10T04:05:29.136 INFO:tasks.workunit.client.1.smithi070.stdout:7/867: creat d2/d3/dc/d5d/d41/f134 x:0 0 0 2024-09-10T04:05:29.136 INFO:tasks.workunit.client.0.smithi046.stdout:2/720: dwrite d2/d8/d37/fc1 [0,4194304] 0 2024-09-10T04:05:29.138 INFO:tasks.workunit.client.1.smithi070.stdout:5/841: dwrite d17/d24/d84/fee [0,4194304] 0 2024-09-10T04:05:29.140 INFO:tasks.workunit.client.0.smithi046.stdout:3/726: write d12/d5d/d71/d49/f4e [80693,21339] 0 2024-09-10T04:05:29.142 INFO:tasks.workunit.client.0.smithi046.stdout:3/727: chown d12/d13/d37/d94/dbf/lfa 337127882 1 2024-09-10T04:05:29.144 INFO:tasks.workunit.client.1.smithi070.stdout:5/842: chown d17/d19/d6a/d54/de1/dfe/dce 3225422 1 2024-09-10T04:05:29.145 INFO:tasks.workunit.client.0.smithi046.stdout:0/632: getdents d1/dc 0 2024-09-10T04:05:29.146 INFO:tasks.workunit.client.0.smithi046.stdout:3/728: truncate d12/d5d/fae 1815466 0 2024-09-10T04:05:29.146 INFO:tasks.workunit.client.0.smithi046.stdout:8/757: dwrite d1/d29/d19/d36/fb6 [0,4194304] 0 2024-09-10T04:05:29.146 INFO:tasks.workunit.client.1.smithi070.stdout:2/776: link d15/d59/da1/fde d15/d18/d56/d7c/d55/f11a 0 2024-09-10T04:05:29.148 INFO:tasks.workunit.client.0.smithi046.stdout:8/758: chown d1/d29/d19/d36/d8d/l90 203810 1 2024-09-10T04:05:29.149 INFO:tasks.workunit.client.1.smithi070.stdout:1/739: link d3/d8/d13/d3f/d5d/ldb d3/dd3/lf3 0 2024-09-10T04:05:29.150 INFO:tasks.workunit.client.1.smithi070.stdout:3/822: dwrite db/d1c/d53/f5d [0,4194304] 0 2024-09-10T04:05:29.154 INFO:tasks.workunit.client.1.smithi070.stdout:9/856: unlink d9/d59/l42 0 2024-09-10T04:05:29.155 INFO:tasks.workunit.client.1.smithi070.stdout:4/782: creat de/d13/d5b/da0/f108 x:0 0 0 2024-09-10T04:05:29.155 INFO:tasks.workunit.client.1.smithi070.stdout:6/810: sync 2024-09-10T04:05:29.156 INFO:tasks.workunit.client.1.smithi070.stdout:7/868: dwrite d2/d3/d63/fe0 [0,4194304] 0 2024-09-10T04:05:29.157 INFO:tasks.workunit.client.0.smithi046.stdout:0/633: dwrite d1/dc/d25/d33/d3/dd/fbb [0,4194304] 0 2024-09-10T04:05:29.159 INFO:tasks.workunit.client.1.smithi070.stdout:6/811: chown d0/d3/d18/d1b/d17/f90 2 1 2024-09-10T04:05:29.161 INFO:tasks.workunit.client.0.smithi046.stdout:3/729: read f7 [221370,10492] 0 2024-09-10T04:05:29.162 INFO:tasks.workunit.client.1.smithi070.stdout:8/874: dwrite d4/d39/dc6/f10b [0,4194304] 0 2024-09-10T04:05:29.163 INFO:tasks.workunit.client.0.smithi046.stdout:3/730: chown d12/d5d/d71/d49/c61 1 1 2024-09-10T04:05:29.166 INFO:tasks.workunit.client.1.smithi070.stdout:8/875: dread - d4/dbf/d74/d11b/d1a/df7/f11a zero size 2024-09-10T04:05:29.166 INFO:tasks.workunit.client.0.smithi046.stdout:2/721: creat d2/d49/f101 x:0 0 0 2024-09-10T04:05:29.167 INFO:tasks.workunit.client.1.smithi070.stdout:0/976: getdents d8/d24/d2d/d5f/df6 0 2024-09-10T04:05:29.169 INFO:tasks.workunit.client.0.smithi046.stdout:0/634: dwrite d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:29.170 INFO:tasks.workunit.client.0.smithi046.stdout:2/722: chown d2/d6/d9d/db6/fb7 515 1 2024-09-10T04:05:29.170 INFO:tasks.workunit.client.1.smithi070.stdout:7/869: dwrite d2/d3/dc/d13/fc7 [0,4194304] 0 2024-09-10T04:05:29.172 INFO:tasks.workunit.client.1.smithi070.stdout:1/740: mkdir d3/d8/d27/d7a/df4 0 2024-09-10T04:05:29.172 INFO:tasks.workunit.client.0.smithi046.stdout:2/723: read d2/d8/d37/fc1 [413445,41028] 0 2024-09-10T04:05:29.174 INFO:tasks.workunit.client.0.smithi046.stdout:2/724: dread - d2/d6/d15/ffa zero size 2024-09-10T04:05:29.176 INFO:tasks.workunit.client.1.smithi070.stdout:2/777: symlink d15/d18/d29/d2a/d2e/da6/l11b 0 2024-09-10T04:05:29.176 INFO:tasks.workunit.client.0.smithi046.stdout:6/647: write da/db/d11/d2b/f6d [1662963,808] 0 2024-09-10T04:05:29.177 INFO:tasks.workunit.client.1.smithi070.stdout:2/778: chown d15/d59/ff4 124666343 1 2024-09-10T04:05:29.179 INFO:tasks.workunit.client.0.smithi046.stdout:5/900: write d7/d11/f1a [2155250,6283] 0 2024-09-10T04:05:29.185 INFO:tasks.workunit.client.1.smithi070.stdout:6/812: dread d0/d3/d18/d1b/d17/d54/fc6 [0,4194304] 0 2024-09-10T04:05:29.188 INFO:tasks.workunit.client.0.smithi046.stdout:1/579: mkdir d3/dc/d21/d3a/d90/dbf 0 2024-09-10T04:05:29.193 INFO:tasks.workunit.client.1.smithi070.stdout:5/843: creat d17/d6f/d32/d90/da5/d120/f124 x:0 0 0 2024-09-10T04:05:29.193 INFO:tasks.workunit.client.1.smithi070.stdout:5/844: stat d17/d24/d84/dab/ff0 0 2024-09-10T04:05:29.195 INFO:tasks.workunit.client.1.smithi070.stdout:0/977: unlink d8/d24/d2d/c50 0 2024-09-10T04:05:29.199 INFO:tasks.workunit.client.1.smithi070.stdout:1/741: truncate d3/d8/f2d 6853833 0 2024-09-10T04:05:29.199 INFO:tasks.workunit.client.0.smithi046.stdout:7/829: getdents d5/d33 0 2024-09-10T04:05:29.206 INFO:tasks.workunit.client.0.smithi046.stdout:6/648: creat da/db/d11/d2b/d5a/fd9 x:0 0 0 2024-09-10T04:05:29.206 INFO:tasks.workunit.client.1.smithi070.stdout:2/779: symlink d15/d2b/d106/l11c 0 2024-09-10T04:05:29.207 INFO:tasks.workunit.client.0.smithi046.stdout:2/725: rmdir d2/d6/d9d/db6/dbe 39 2024-09-10T04:05:29.211 INFO:tasks.workunit.client.0.smithi046.stdout:5/901: symlink d7/d11/d14/d45/d51/l139 0 2024-09-10T04:05:29.211 INFO:tasks.workunit.client.1.smithi070.stdout:9/857: mknod d9/d1c/dac/d8a/dc4/dda/de0/c115 0 2024-09-10T04:05:29.213 INFO:tasks.workunit.client.0.smithi046.stdout:1/580: read d3/d27/d5d/f9a [1418869,81881] 0 2024-09-10T04:05:29.213 INFO:tasks.workunit.client.1.smithi070.stdout:5/845: truncate d17/d19/d6a/d54/fb1 1120938 0 2024-09-10T04:05:29.214 INFO:tasks.workunit.client.0.smithi046.stdout:1/581: write d3/d27/f20 [1913528,45080] 0 2024-09-10T04:05:29.215 INFO:tasks.workunit.client.0.smithi046.stdout:8/759: creat d1/d29/d17/d1f/d46/d6a/d82/d99/dda/ffe x:0 0 0 2024-09-10T04:05:29.215 INFO:tasks.workunit.client.1.smithi070.stdout:0/978: readlink d8/d13/de2/l115 0 2024-09-10T04:05:29.218 INFO:tasks.workunit.client.0.smithi046.stdout:7/830: mkdir d5/d9/d12/dc5/d125 0 2024-09-10T04:05:29.221 INFO:tasks.workunit.client.0.smithi046.stdout:6/649: fsync da/d45/f56 0 2024-09-10T04:05:29.223 INFO:tasks.workunit.client.1.smithi070.stdout:4/783: dwrite de/d13/f1a [0,4194304] 0 2024-09-10T04:05:29.224 INFO:tasks.workunit.client.0.smithi046.stdout:9/745: getdents d4/d18/da8 0 2024-09-10T04:05:29.225 INFO:tasks.workunit.client.1.smithi070.stdout:4/784: write de/fed [658407,10695] 0 2024-09-10T04:05:29.231 INFO:tasks.workunit.client.1.smithi070.stdout:6/813: dwrite d0/d3/d18/d1b/d17/d54/fc6 [0,4194304] 0 2024-09-10T04:05:29.233 INFO:tasks.workunit.client.1.smithi070.stdout:3/823: write db/f82 [737628,68317] 0 2024-09-10T04:05:29.233 INFO:tasks.workunit.client.1.smithi070.stdout:8/876: dwrite d4/dbf/d74/d11b/d1a/f2e [0,4194304] 0 2024-09-10T04:05:29.235 INFO:tasks.workunit.client.1.smithi070.stdout:8/877: read d4/dcf/f9a [198670,85883] 0 2024-09-10T04:05:29.236 INFO:tasks.workunit.client.0.smithi046.stdout:4/885: getdents d3/dab 0 2024-09-10T04:05:29.241 INFO:tasks.workunit.client.0.smithi046.stdout:7/831: write d5/d9/d12/d72/d77/de0/fe3 [4298793,81860] 0 2024-09-10T04:05:29.245 INFO:tasks.workunit.client.0.smithi046.stdout:4/886: dwrite d3/d12/d17/d29/ddd/f12b [0,4194304] 0 2024-09-10T04:05:29.245 INFO:tasks.workunit.client.0.smithi046.stdout:7/832: fsync d5/d9/d12/d72/d77/de0/fc6 0 2024-09-10T04:05:29.247 INFO:tasks.workunit.client.0.smithi046.stdout:4/887: stat d3/d12/d17/de1/ddf/dfc 0 2024-09-10T04:05:29.249 INFO:tasks.workunit.client.1.smithi070.stdout:1/742: write d3/d10/d1d/f46 [343540,111469] 0 2024-09-10T04:05:29.250 INFO:tasks.workunit.client.0.smithi046.stdout:6/650: truncate da/db/d11/d2b/d2f/d3b/fc3 616285 0 2024-09-10T04:05:29.255 INFO:tasks.workunit.client.0.smithi046.stdout:4/888: dwrite d3/d12/d17/d22/f59 [0,4194304] 0 2024-09-10T04:05:29.256 INFO:tasks.workunit.client.0.smithi046.stdout:9/746: read d4/d3c/d4d/d5c/f4e [261879,45802] 0 2024-09-10T04:05:29.257 INFO:tasks.workunit.client.0.smithi046.stdout:5/902: rename d7/d11/l2c to d7/d11/d14/d2f/deb/da5/dcc/dc0/d134/l13a 0 2024-09-10T04:05:29.258 INFO:tasks.workunit.client.0.smithi046.stdout:9/747: read - d4/d7/d61/fd7 zero size 2024-09-10T04:05:29.261 INFO:tasks.workunit.client.0.smithi046.stdout:8/760: getdents d1/d29/d19/d36/d8d 0 2024-09-10T04:05:29.261 INFO:tasks.workunit.client.0.smithi046.stdout:1/582: write d3/da1/f2e [2631865,78494] 0 2024-09-10T04:05:29.263 INFO:tasks.workunit.client.0.smithi046.stdout:8/761: chown d1/d29/d19/d73/fba 113 1 2024-09-10T04:05:29.263 INFO:tasks.workunit.client.0.smithi046.stdout:1/583: readlink d3/da1/d1f/l81 0 2024-09-10T04:05:29.263 INFO:tasks.workunit.client.0.smithi046.stdout:3/731: link d12/d13/d37/d94/fb3 d12/d5d/ffc 0 2024-09-10T04:05:29.264 INFO:tasks.workunit.client.0.smithi046.stdout:1/584: truncate d3/fae 1014844 0 2024-09-10T04:05:29.272 INFO:tasks.workunit.client.0.smithi046.stdout:2/726: truncate d2/d6/d23/f87 493231 0 2024-09-10T04:05:29.279 INFO:tasks.workunit.client.0.smithi046.stdout:5/903: creat d7/d11/d14/d2f/d2b/d92/f13b x:0 0 0 2024-09-10T04:05:29.281 INFO:tasks.workunit.client.0.smithi046.stdout:8/762: write d1/d29/d17/f9c [2509599,57400] 0 2024-09-10T04:05:29.281 INFO:tasks.workunit.client.0.smithi046.stdout:5/904: chown d7/d11/d14/d45/fc2 521164 1 2024-09-10T04:05:29.282 INFO:tasks.workunit.client.0.smithi046.stdout:4/889: dwrite d3/d12/d14/f62 [0,4194304] 0 2024-09-10T04:05:29.284 INFO:tasks.workunit.client.0.smithi046.stdout:3/732: mkdir d12/d13/d14/d1e/d2d/dfd 0 2024-09-10T04:05:29.286 INFO:tasks.workunit.client.0.smithi046.stdout:1/585: rename d3/dc/f10 to d3/d27/d5d/d60/fc0 0 2024-09-10T04:05:29.290 INFO:tasks.workunit.client.0.smithi046.stdout:7/833: dwrite d5/d6/d32/dab/db6/f64 [0,4194304] 0 2024-09-10T04:05:29.291 INFO:tasks.workunit.client.0.smithi046.stdout:2/727: symlink d2/d6/d23/d42/db4/dca/l102 0 2024-09-10T04:05:29.293 INFO:tasks.workunit.client.0.smithi046.stdout:2/728: stat d2/d6/d23/d5d/d77/d7b/d85 0 2024-09-10T04:05:29.294 INFO:tasks.workunit.client.0.smithi046.stdout:2/729: chown d2/d86/d9c/dbc/le2 2411 1 2024-09-10T04:05:29.296 INFO:tasks.workunit.client.0.smithi046.stdout:5/905: symlink d7/d11/d14/d8e/d96/dce/l13c 0 2024-09-10T04:05:29.299 INFO:tasks.workunit.client.0.smithi046.stdout:4/890: fsync d3/d12/d17/d29/d58/f71 0 2024-09-10T04:05:29.302 INFO:tasks.workunit.client.0.smithi046.stdout:6/651: creat da/db/d11/d13/fda x:0 0 0 2024-09-10T04:05:29.303 INFO:tasks.workunit.client.0.smithi046.stdout:4/891: write d3/d12/d14/f62 [5225434,54962] 0 2024-09-10T04:05:29.305 INFO:tasks.workunit.client.0.smithi046.stdout:1/586: rename d3/da1/d1f/l8e to d3/da1/da7/lc1 0 2024-09-10T04:05:29.309 INFO:tasks.workunit.client.0.smithi046.stdout:5/906: dwrite d7/d11/d14/d2f/deb/da5/d123/f136 [0,4194304] 0 2024-09-10T04:05:29.309 INFO:tasks.workunit.client.0.smithi046.stdout:9/748: creat d4/d7/d1f/feb x:0 0 0 2024-09-10T04:05:29.309 INFO:tasks.workunit.client.0.smithi046.stdout:6/652: truncate da/db/d11/d2b/d2f/d3b/fc3 1425055 0 2024-09-10T04:05:29.310 INFO:tasks.workunit.client.0.smithi046.stdout:4/892: dread d3/d12/d14/f111 [0,4194304] 0 2024-09-10T04:05:29.316 INFO:tasks.workunit.client.0.smithi046.stdout:3/733: unlink d12/d13/d37/d94/fb3 0 2024-09-10T04:05:29.316 INFO:tasks.workunit.client.0.smithi046.stdout:4/893: dread d3/d12/d14/f62 [0,4194304] 0 2024-09-10T04:05:29.324 INFO:tasks.workunit.client.0.smithi046.stdout:5/907: dread d7/d11/d14/f89 [0,4194304] 0 2024-09-10T04:05:29.332 INFO:tasks.workunit.client.0.smithi046.stdout:8/763: write d1/d29/d12/f6b [869808,27059] 0 2024-09-10T04:05:29.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: pgmap v268: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 94 MiB/s rd, 246 MiB/s wr, 541 op/s 2024-09-10T04:05:29.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: Standby manager daemon smithi070.tezbzn restarted 2024-09-10T04:05:29.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: Standby manager daemon smithi070.tezbzn started 2024-09-10T04:05:29.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T04:05:29.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:05:29.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T04:05:29.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:05:29.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:29.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:29 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:29.337 INFO:tasks.workunit.client.0.smithi046.stdout:1/587: dwrite d3/d9b/fbc [0,4194304] 0 2024-09-10T04:05:29.341 INFO:tasks.workunit.client.0.smithi046.stdout:3/734: creat d12/d13/d37/d51/dbd/ffe x:0 0 0 2024-09-10T04:05:29.344 INFO:tasks.workunit.client.0.smithi046.stdout:4/894: unlink d3/d12/d17/d22/d51/d94/fd6 0 2024-09-10T04:05:29.346 INFO:tasks.workunit.client.0.smithi046.stdout:5/908: stat d7/f97 0 2024-09-10T04:05:29.351 INFO:tasks.workunit.client.0.smithi046.stdout:7/834: dwrite d5/d9/f42 [0,4194304] 0 2024-09-10T04:05:29.352 INFO:tasks.workunit.client.0.smithi046.stdout:2/730: creat d2/d6/d15/f103 x:0 0 0 2024-09-10T04:05:29.354 INFO:tasks.workunit.client.1.smithi070.stdout:2/780: mknod d15/d18/d56/c11d 0 2024-09-10T04:05:29.355 INFO:tasks.workunit.client.0.smithi046.stdout:6/653: dread da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:29.355 INFO:tasks.workunit.client.0.smithi046.stdout:8/764: creat d1/d29/d17/d1f/d46/d67/de0/fff x:0 0 0 2024-09-10T04:05:29.357 INFO:tasks.workunit.client.0.smithi046.stdout:6/654: fdatasync da/fd 0 2024-09-10T04:05:29.360 INFO:tasks.workunit.client.1.smithi070.stdout:9/858: creat d9/d1c/dac/d8a/dc4/f116 x:0 0 0 2024-09-10T04:05:29.361 INFO:tasks.workunit.client.1.smithi070.stdout:9/859: readlink d9/d1c/dac/d8a/l2f 0 2024-09-10T04:05:29.362 INFO:tasks.workunit.client.1.smithi070.stdout:5/846: rename d17/l10c to d17/d24/d84/dab/d114/l125 0 2024-09-10T04:05:29.365 INFO:tasks.workunit.client.1.smithi070.stdout:0/979: truncate d8/d13/de2/d53/dc6/fd9 181767 0 2024-09-10T04:05:29.365 INFO:tasks.workunit.client.0.smithi046.stdout:3/735: chown d12/d13/f17 262789500 1 2024-09-10T04:05:29.368 INFO:tasks.workunit.client.0.smithi046.stdout:4/895: write d3/d12/d17/d29/d58/db2/d7a/f124 [63648,54280] 0 2024-09-10T04:05:29.370 INFO:tasks.workunit.client.0.smithi046.stdout:6/655: dread da/db/d11/d13/f39 [0,4194304] 0 2024-09-10T04:05:29.371 INFO:tasks.workunit.client.0.smithi046.stdout:6/656: chown da/db/d11/d2b/d55/l67 300904959 1 2024-09-10T04:05:29.372 INFO:tasks.workunit.client.1.smithi070.stdout:0/980: dwrite d8/d13/d1a/f31 [0,4194304] 0 2024-09-10T04:05:29.374 INFO:tasks.workunit.client.0.smithi046.stdout:2/731: rename d2/d8/d37/d63/f82 to d2/d6/d23/d5d/d77/d7b/dd8/f104 0 2024-09-10T04:05:29.374 INFO:tasks.workunit.client.1.smithi070.stdout:8/878: readlink d4/dbf/d74/d11b/d1a/d60/l6a 0 2024-09-10T04:05:29.377 INFO:tasks.workunit.client.1.smithi070.stdout:5/847: dread d17/d19/fd1 [0,4194304] 0 2024-09-10T04:05:29.379 INFO:tasks.workunit.client.1.smithi070.stdout:1/743: mknod d3/d90/dcb/cf5 0 2024-09-10T04:05:29.381 INFO:tasks.workunit.client.0.smithi046.stdout:8/765: creat d1/d29/d17/d1f/d46/d6a/d82/f100 x:0 0 0 2024-09-10T04:05:29.382 INFO:tasks.workunit.client.0.smithi046.stdout:8/766: stat d1/d29/d17/d1f/d2a/fdd 0 2024-09-10T04:05:29.383 INFO:tasks.workunit.client.0.smithi046.stdout:1/588: mknod d3/dc/d34/d51/cc2 0 2024-09-10T04:05:29.387 INFO:tasks.workunit.client.1.smithi070.stdout:2/781: unlink d15/d59/c6a 0 2024-09-10T04:05:29.388 INFO:tasks.workunit.client.0.smithi046.stdout:9/749: truncate d4/d3c/d4d/d5c/d59/f5d 2678843 0 2024-09-10T04:05:29.389 INFO:tasks.workunit.client.0.smithi046.stdout:5/909: unlink d7/ca6 0 2024-09-10T04:05:29.397 INFO:tasks.workunit.client.1.smithi070.stdout:7/870: getdents d2/d7/d9/d18/d23 0 2024-09-10T04:05:29.401 INFO:tasks.workunit.client.0.smithi046.stdout:1/589: creat d3/dc/d21/fc3 x:0 0 0 2024-09-10T04:05:29.403 INFO:tasks.workunit.client.0.smithi046.stdout:8/767: chown d1/d29/d17/d1f/d46/d67/dc1/d9d/def 1 1 2024-09-10T04:05:29.403 INFO:tasks.workunit.client.0.smithi046.stdout:4/896: dread d3/d12/d17/d29/d58/db2/d5f/d85/fa2 [0,4194304] 0 2024-09-10T04:05:29.406 INFO:tasks.workunit.client.1.smithi070.stdout:6/814: creat d0/d3/d18/df5/ffd x:0 0 0 2024-09-10T04:05:29.407 INFO:tasks.workunit.client.1.smithi070.stdout:0/981: fdatasync d8/d24/d97/fbe 0 2024-09-10T04:05:29.411 INFO:tasks.workunit.client.1.smithi070.stdout:5/848: mknod d17/d19/d6a/d54/d57/d63/db9/c126 0 2024-09-10T04:05:29.414 INFO:tasks.workunit.client.0.smithi046.stdout:4/897: fsync d3/d12/d14/f62 0 2024-09-10T04:05:29.417 INFO:tasks.workunit.client.0.smithi046.stdout:8/768: rename d1/d29/d19/d36/d8d/l90 to d1/d29/d17/d1f/l101 0 2024-09-10T04:05:29.418 INFO:tasks.workunit.client.1.smithi070.stdout:9/860: symlink d9/d1c/d34/d8b/dcc/de7/l117 0 2024-09-10T04:05:29.422 INFO:tasks.workunit.client.0.smithi046.stdout:9/750: creat d4/d7/d1f/dce/fec x:0 0 0 2024-09-10T04:05:29.424 INFO:tasks.workunit.client.0.smithi046.stdout:1/590: dread d3/dc/d34/f92 [0,4194304] 0 2024-09-10T04:05:29.425 INFO:tasks.workunit.client.0.smithi046.stdout:5/910: truncate d7/fc 5440500 0 2024-09-10T04:05:29.426 INFO:tasks.workunit.client.1.smithi070.stdout:2/782: dwrite d15/d18/d29/d2a/f67 [0,4194304] 0 2024-09-10T04:05:29.427 INFO:tasks.workunit.client.0.smithi046.stdout:6/657: creat da/db/fdb x:0 0 0 2024-09-10T04:05:29.430 INFO:tasks.workunit.client.1.smithi070.stdout:0/982: creat d8/d24/dce/f141 x:0 0 0 2024-09-10T04:05:29.431 INFO:tasks.workunit.client.0.smithi046.stdout:2/732: rmdir d2/d6/d23/d42/db4/dca/dd6 0 2024-09-10T04:05:29.433 INFO:tasks.workunit.client.0.smithi046.stdout:3/736: dwrite d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fca [0,4194304] 0 2024-09-10T04:05:29.433 INFO:tasks.workunit.client.1.smithi070.stdout:8/879: mkdir d4/dbf/d74/d11b/d19/d62/df0/d12c 0 2024-09-10T04:05:29.434 INFO:tasks.workunit.client.1.smithi070.stdout:8/880: stat d4/dbf/d74/d11b/d19 0 2024-09-10T04:05:29.435 INFO:tasks.workunit.client.1.smithi070.stdout:8/881: readlink d4/dbf/d74/d11b/d1a/d60/lb9 0 2024-09-10T04:05:29.436 INFO:tasks.workunit.client.1.smithi070.stdout:3/824: link db/d1c/d25/d28/dd3/de5/d10f/fc4 db/dcc/f11d 0 2024-09-10T04:05:29.437 INFO:tasks.workunit.client.0.smithi046.stdout:4/898: creat d3/dab/dca/de4/f134 x:0 0 0 2024-09-10T04:05:29.440 INFO:tasks.workunit.client.1.smithi070.stdout:5/849: rename d17/d6f/d32/f68 to d17/d24/d77/f127 0 2024-09-10T04:05:29.446 INFO:tasks.workunit.client.1.smithi070.stdout:1/744: unlink d3/d8/d13/d3f/l75 0 2024-09-10T04:05:29.446 INFO:tasks.workunit.client.1.smithi070.stdout:9/861: mkdir d9/d1c/dac/d8a/d3c/d92/d118 0 2024-09-10T04:05:29.446 INFO:tasks.workunit.client.0.smithi046.stdout:8/769: truncate d1/d29/d19/d36/d4c/d5e/f5f 525558 0 2024-09-10T04:05:29.446 INFO:tasks.workunit.client.0.smithi046.stdout:9/751: creat d4/d18/d89/fed x:0 0 0 2024-09-10T04:05:29.446 INFO:tasks.workunit.client.0.smithi046.stdout:9/752: chown d4/d18 28658 1 2024-09-10T04:05:29.447 INFO:tasks.workunit.client.1.smithi070.stdout:7/871: mkdir d2/dc8/df9/d135 0 2024-09-10T04:05:29.447 INFO:tasks.workunit.client.1.smithi070.stdout:4/785: link de/d13/d5b/d7b/dd5/daf/df2/ld8 de/d13/d18/d23/d39/l109 0 2024-09-10T04:05:29.448 INFO:tasks.workunit.client.1.smithi070.stdout:7/872: fsync d2/d7/d9/db4/f11c 0 2024-09-10T04:05:29.450 INFO:tasks.workunit.client.0.smithi046.stdout:5/911: dread d7/f10 [4194304,4194304] 0 2024-09-10T04:05:29.451 INFO:tasks.workunit.client.0.smithi046.stdout:5/912: readlink d7/d11/d14/d2f/da3/lde 0 2024-09-10T04:05:29.452 INFO:tasks.workunit.client.0.smithi046.stdout:3/737: mkdir d12/d13/d37/d94/dff 0 2024-09-10T04:05:29.454 INFO:tasks.workunit.client.0.smithi046.stdout:2/733: rename d2/d6/d23/d42/f45 to d2/d6/d23/d42/db4/f105 0 2024-09-10T04:05:29.455 INFO:tasks.workunit.client.1.smithi070.stdout:3/825: chown db/d1c/d25/f87 3 1 2024-09-10T04:05:29.455 INFO:tasks.workunit.client.0.smithi046.stdout:2/734: dread - d2/d6/d23/d5d/fef zero size 2024-09-10T04:05:29.457 INFO:tasks.workunit.client.0.smithi046.stdout:4/899: unlink d3/d12/d17/cda 0 2024-09-10T04:05:29.460 INFO:tasks.workunit.client.1.smithi070.stdout:5/850: read d17/d19/f1d [1210152,82341] 0 2024-09-10T04:05:29.461 INFO:tasks.workunit.client.1.smithi070.stdout:1/745: rmdir d3/d20 39 2024-09-10T04:05:29.464 INFO:tasks.workunit.client.1.smithi070.stdout:9/862: fdatasync d9/d1c/dac/d8a/dc4/de1/fc3 0 2024-09-10T04:05:29.465 INFO:tasks.workunit.client.0.smithi046.stdout:6/658: dread da/db/d10/f1a [0,4194304] 0 2024-09-10T04:05:29.465 INFO:tasks.workunit.client.1.smithi070.stdout:5/851: dwrite f8 [0,4194304] 0 2024-09-10T04:05:29.467 INFO:tasks.workunit.client.0.smithi046.stdout:1/591: dwrite d3/ff [4194304,4194304] 0 2024-09-10T04:05:29.468 INFO:tasks.workunit.client.1.smithi070.stdout:6/815: creat d0/d3/ffe x:0 0 0 2024-09-10T04:05:29.469 INFO:tasks.workunit.client.1.smithi070.stdout:6/816: chown d0/d3/c13 0 1 2024-09-10T04:05:29.472 INFO:tasks.workunit.client.1.smithi070.stdout:7/873: rmdir d2/d40/d6c/db2 39 2024-09-10T04:05:29.474 INFO:tasks.workunit.client.0.smithi046.stdout:6/659: dwrite da/f7b [0,4194304] 0 2024-09-10T04:05:29.474 INFO:tasks.workunit.client.0.smithi046.stdout:9/753: rename d4/d18/d89/dba to d4/d18/d89/ddb/dee 0 2024-09-10T04:05:29.476 INFO:tasks.workunit.client.1.smithi070.stdout:4/786: symlink de/d13/d5b/d7b/dea/l10a 0 2024-09-10T04:05:29.477 INFO:tasks.workunit.client.0.smithi046.stdout:4/900: dread d3/d12/d17/d29/f4f [0,4194304] 0 2024-09-10T04:05:29.478 INFO:tasks.workunit.client.0.smithi046.stdout:2/735: rmdir d2/d6/d23/d5d/d77/d7b/d85 39 2024-09-10T04:05:29.480 INFO:tasks.workunit.client.0.smithi046.stdout:2/736: read d2/f25 [461883,105036] 0 2024-09-10T04:05:29.480 INFO:tasks.workunit.client.1.smithi070.stdout:0/983: write d8/d13/de2/d7d/d8a/fbc [276419,63828] 0 2024-09-10T04:05:29.483 INFO:tasks.workunit.client.1.smithi070.stdout:8/882: symlink d4/d39/dfc/da8/d124/l12d 0 2024-09-10T04:05:29.484 INFO:tasks.workunit.client.0.smithi046.stdout:8/770: mkdir d1/d29/d17/d1f/d2a/df6/d102 0 2024-09-10T04:05:29.488 INFO:tasks.workunit.client.0.smithi046.stdout:5/913: truncate d7/ff9 1016370 0 2024-09-10T04:05:29.491 INFO:tasks.workunit.client.0.smithi046.stdout:0/635: sync 2024-09-10T04:05:29.494 INFO:tasks.workunit.client.1.smithi070.stdout:5/852: creat d17/d24/d84/dab/f128 x:0 0 0 2024-09-10T04:05:29.495 INFO:tasks.workunit.client.0.smithi046.stdout:0/636: dread d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:29.497 INFO:tasks.workunit.client.0.smithi046.stdout:0/637: stat d1/dc/d25/d33/d1a/d41/lc0 0 2024-09-10T04:05:29.498 INFO:tasks.workunit.client.1.smithi070.stdout:7/874: rename d2/d7/d9/d18/d3c/d52/d103/c116 to d2/d40/c136 0 2024-09-10T04:05:29.500 INFO:tasks.workunit.client.0.smithi046.stdout:4/901: readlink d3/d12/l32 0 2024-09-10T04:05:29.502 INFO:tasks.workunit.client.0.smithi046.stdout:2/737: creat d2/d6/d9d/f106 x:0 0 0 2024-09-10T04:05:29.503 INFO:tasks.workunit.client.1.smithi070.stdout:2/783: link d15/d18/d56/l119 d15/d18/d29/d2a/d2e/da6/dc0/l11e 0 2024-09-10T04:05:29.504 INFO:tasks.workunit.client.0.smithi046.stdout:9/754: dwrite d4/d66/da6/f72 [0,4194304] 0 2024-09-10T04:05:29.507 INFO:tasks.workunit.client.0.smithi046.stdout:9/755: dread d4/d7/d1f/d54/dc7/fdd [0,4194304] 0 2024-09-10T04:05:29.508 INFO:tasks.workunit.client.0.smithi046.stdout:5/914: mknod d7/d11/d14/d2f/da3/c13d 0 2024-09-10T04:05:29.509 INFO:tasks.workunit.client.1.smithi070.stdout:0/984: dwrite d8/d13/d1a/d9a/f13c [0,4194304] 0 2024-09-10T04:05:29.509 INFO:tasks.workunit.client.0.smithi046.stdout:5/915: fsync d7/d11/d14/d2f/d2b/d92/f13b 0 2024-09-10T04:05:29.509 INFO:tasks.workunit.client.1.smithi070.stdout:1/746: unlink d3/d10/d1d/f30 0 2024-09-10T04:05:29.511 INFO:tasks.workunit.client.0.smithi046.stdout:3/738: creat d12/d13/d37/f100 x:0 0 0 2024-09-10T04:05:29.513 INFO:tasks.workunit.client.1.smithi070.stdout:9/863: fsync d9/d59/d2a/d3f/f86 0 2024-09-10T04:05:29.513 INFO:tasks.workunit.client.0.smithi046.stdout:3/739: write d12/d13/d14/d1e/d2d/d39/fea [435343,82741] 0 2024-09-10T04:05:29.517 INFO:tasks.workunit.client.1.smithi070.stdout:9/864: read - d9/d1c/dac/d8a/d3c/d62/f10a zero size 2024-09-10T04:05:29.518 INFO:tasks.workunit.client.0.smithi046.stdout:4/902: rename d3/d12/d17/d29/ddd/f116 to d3/d12/d17/d29/d58/db2/d5f/d85/da8/f135 0 2024-09-10T04:05:29.519 INFO:tasks.workunit.client.1.smithi070.stdout:9/865: write d9/d1c/dac/ff2 [377424,67190] 0 2024-09-10T04:05:29.520 INFO:tasks.workunit.client.1.smithi070.stdout:3/826: dwrite f7 [0,4194304] 0 2024-09-10T04:05:29.520 INFO:tasks.workunit.client.1.smithi070.stdout:8/883: rename d4/dbf/d74 to d4/d16/dcc/d12e 0 2024-09-10T04:05:29.521 INFO:tasks.workunit.client.1.smithi070.stdout:1/747: dread d3/d8/d27/d7a/fa7 [0,4194304] 0 2024-09-10T04:05:29.521 INFO:tasks.workunit.client.0.smithi046.stdout:2/738: rename d2/d6/d23/d42/de6/fb5 to d2/d6/d23/d42/de6/f107 0 2024-09-10T04:05:29.522 INFO:tasks.workunit.client.1.smithi070.stdout:8/884: chown d4/d39/dfc/d43/d67/dec 187 1 2024-09-10T04:05:29.522 INFO:tasks.workunit.client.0.smithi046.stdout:4/903: write d3/d12/d17/d29/d58/db2/d7a/f11f [2048549,101023] 0 2024-09-10T04:05:29.523 INFO:tasks.workunit.client.0.smithi046.stdout:2/739: chown d2/d86/d9c/fd0 6802746 1 2024-09-10T04:05:29.523 INFO:tasks.workunit.client.0.smithi046.stdout:4/904: chown d3/d12/d17/d22/d30/dc7 1 1 2024-09-10T04:05:29.524 INFO:tasks.workunit.client.1.smithi070.stdout:7/875: dread - d2/d40/d6c/db2/db7/d104/f111 zero size 2024-09-10T04:05:29.524 INFO:tasks.workunit.client.0.smithi046.stdout:2/740: write d2/d6/d15/f103 [776791,120326] 0 2024-09-10T04:05:29.525 INFO:tasks.workunit.client.1.smithi070.stdout:7/876: chown d2/d3/cda 25718837 1 2024-09-10T04:05:29.526 INFO:tasks.workunit.client.0.smithi046.stdout:8/771: symlink d1/d29/d12/l103 0 2024-09-10T04:05:29.528 INFO:tasks.workunit.client.1.smithi070.stdout:5/853: write d17/d24/d77/fdd [829328,14642] 0 2024-09-10T04:05:29.529 INFO:tasks.workunit.client.1.smithi070.stdout:6/817: truncate d0/d3/d18/d1b/f7d 1666390 0 2024-09-10T04:05:29.530 INFO:tasks.workunit.client.0.smithi046.stdout:9/756: creat d4/d7/d1f/d54/fef x:0 0 0 2024-09-10T04:05:29.531 INFO:tasks.workunit.client.1.smithi070.stdout:5/854: dread - d17/d19/d6a/d54/de1/dfe/d93/f107 zero size 2024-09-10T04:05:29.532 INFO:tasks.workunit.client.1.smithi070.stdout:0/985: fsync d8/d24/d2d/d34/f5b 0 2024-09-10T04:05:29.534 INFO:tasks.workunit.client.0.smithi046.stdout:6/660: creat da/db/d11/d13/d1e/fdc x:0 0 0 2024-09-10T04:05:29.537 INFO:tasks.workunit.client.1.smithi070.stdout:7/877: dwrite d2/d3/dc/d13/dbd/ff6 [0,4194304] 0 2024-09-10T04:05:29.542 INFO:tasks.workunit.client.1.smithi070.stdout:6/818: dread d0/d3/d18/d1b/f11 [4194304,4194304] 0 2024-09-10T04:05:29.545 INFO:tasks.workunit.client.0.smithi046.stdout:4/905: creat d3/d12/d17/d22/d51/d94/f136 x:0 0 0 2024-09-10T04:05:29.546 INFO:tasks.workunit.client.1.smithi070.stdout:1/748: dread d3/f47 [4194304,4194304] 0 2024-09-10T04:05:29.550 INFO:tasks.workunit.client.1.smithi070.stdout:9/866: dread d9/d1c/f2c [0,4194304] 0 2024-09-10T04:05:29.550 INFO:tasks.workunit.client.1.smithi070.stdout:4/787: creat de/d13/f10b x:0 0 0 2024-09-10T04:05:29.554 INFO:tasks.workunit.client.0.smithi046.stdout:5/916: fsync d7/ff9 0 2024-09-10T04:05:29.558 INFO:tasks.workunit.client.0.smithi046.stdout:3/740: creat d12/d13/d37/d94/dff/f101 x:0 0 0 2024-09-10T04:05:29.558 INFO:tasks.workunit.client.1.smithi070.stdout:8/885: dread d4/d16/dcc/d12e/d11b/d1a/f2a [0,4194304] 0 2024-09-10T04:05:29.562 INFO:tasks.workunit.client.1.smithi070.stdout:2/784: write d15/d18/d29/d2a/d2e/da6/dc0/fce [1106325,116340] 0 2024-09-10T04:05:29.564 INFO:tasks.workunit.client.0.smithi046.stdout:4/906: rename d3/d12/d17/d91/dbf/f127 to d3/d12/d17/de1/ddf/f137 0 2024-09-10T04:05:29.566 INFO:tasks.workunit.client.0.smithi046.stdout:9/757: write d4/d7/f1e [1834239,88096] 0 2024-09-10T04:05:29.567 INFO:tasks.workunit.client.0.smithi046.stdout:2/741: creat d2/d6/d23/d5d/d77/d7b/d85/d8d/f108 x:0 0 0 2024-09-10T04:05:29.568 INFO:tasks.workunit.client.0.smithi046.stdout:2/742: chown d2/d8/d65/d90/f100 277 1 2024-09-10T04:05:29.571 INFO:tasks.workunit.client.0.smithi046.stdout:8/772: truncate d1/d29/d17/d1f/d46/d6a/f94 322050 0 2024-09-10T04:05:29.572 INFO:tasks.workunit.client.0.smithi046.stdout:6/661: write f6 [2022864,10215] 0 2024-09-10T04:05:29.572 INFO:tasks.workunit.client.1.smithi070.stdout:7/878: dread d2/d7/db/f92 [0,4194304] 0 2024-09-10T04:05:29.572 INFO:tasks.workunit.client.1.smithi070.stdout:6/819: chown d0/d3/l2c 15716 1 2024-09-10T04:05:29.575 INFO:tasks.workunit.client.0.smithi046.stdout:5/917: rmdir d7/d11/d14/d2f/deb/da5/dcc 39 2024-09-10T04:05:29.577 INFO:tasks.workunit.client.1.smithi070.stdout:0/986: write d8/d24/d2d/d8d/fe6 [149294,76937] 0 2024-09-10T04:05:29.579 INFO:tasks.workunit.client.0.smithi046.stdout:3/741: symlink d12/d13/d37/d94/dff/l102 0 2024-09-10T04:05:29.579 INFO:tasks.workunit.client.1.smithi070.stdout:4/788: stat de/d13/d18/l25 0 2024-09-10T04:05:29.580 INFO:tasks.workunit.client.1.smithi070.stdout:6/820: dwrite d0/d3/d1f/d81/fe6 [0,4194304] 0 2024-09-10T04:05:29.580 INFO:tasks.workunit.client.1.smithi070.stdout:0/987: fsync d8/d13/d1a/d8f/d95/f12a 0 2024-09-10T04:05:29.582 INFO:tasks.workunit.client.0.smithi046.stdout:9/758: creat d4/d7/d1f/d54/dc7/ff0 x:0 0 0 2024-09-10T04:05:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: pgmap v268: 65 pgs: 65 active+clean; 2.9 GiB data, 8.7 GiB used, 528 GiB / 536 GiB avail; 94 MiB/s rd, 246 MiB/s wr, 541 op/s 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: Standby manager daemon smithi070.tezbzn restarted 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: Standby manager daemon smithi070.tezbzn started 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:29 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:29.587 INFO:tasks.workunit.client.1.smithi070.stdout:2/785: creat d15/d18/db2/f11f x:0 0 0 2024-09-10T04:05:29.591 INFO:tasks.workunit.client.0.smithi046.stdout:8/773: dread d1/d29/d17/d1f/d46/d76/fab [0,4194304] 0 2024-09-10T04:05:29.591 INFO:tasks.workunit.client.0.smithi046.stdout:2/743: write d2/d49/f2e [1856575,113409] 0 2024-09-10T04:05:29.592 INFO:tasks.workunit.client.0.smithi046.stdout:6/662: fsync da/db/d11/d13/d1e/d23/d86/dbf/f95 0 2024-09-10T04:05:29.594 INFO:tasks.workunit.client.0.smithi046.stdout:2/744: write d2/d6/d23/ff0 [264787,46537] 0 2024-09-10T04:05:29.596 INFO:tasks.workunit.client.0.smithi046.stdout:4/907: dwrite d3/dab/fe5 [0,4194304] 0 2024-09-10T04:05:29.598 INFO:tasks.workunit.client.0.smithi046.stdout:4/908: write d3/d12/d14/f18 [653031,129011] 0 2024-09-10T04:05:29.602 INFO:tasks.workunit.client.1.smithi070.stdout:6/821: creat d0/d3/d18/d1b/d33/fff x:0 0 0 2024-09-10T04:05:29.604 INFO:tasks.workunit.client.1.smithi070.stdout:5/855: creat d17/d24/d45/f129 x:0 0 0 2024-09-10T04:05:29.611 INFO:tasks.workunit.client.0.smithi046.stdout:9/759: dwrite d4/d29/d65/f75 [0,4194304] 0 2024-09-10T04:05:29.614 INFO:tasks.workunit.client.0.smithi046.stdout:5/918: mknod d7/d11/d14/d2f/deb/da5/dcc/dc0/df3/c13e 0 2024-09-10T04:05:29.618 INFO:tasks.workunit.client.0.smithi046.stdout:3/742: symlink d12/d13/d37/d51/db5/d4b/db7/dc0/ded/l103 0 2024-09-10T04:05:29.620 INFO:tasks.workunit.client.1.smithi070.stdout:1/749: truncate d3/d8/d13/f18 448056 0 2024-09-10T04:05:29.620 INFO:tasks.workunit.client.0.smithi046.stdout:9/760: dwrite d4/d18/d89/fed [0,4194304] 0 2024-09-10T04:05:29.621 INFO:tasks.workunit.client.1.smithi070.stdout:7/879: creat d2/dc5/d122/f137 x:0 0 0 2024-09-10T04:05:29.622 INFO:tasks.workunit.client.1.smithi070.stdout:3/827: getdents db/d1c/d89 0 2024-09-10T04:05:29.622 INFO:tasks.workunit.client.0.smithi046.stdout:6/663: rename da/db/d11/d2b/d55/l67 to da/db/d11/d2b/d2f/d3b/ldd 0 2024-09-10T04:05:29.623 INFO:tasks.workunit.client.0.smithi046.stdout:9/761: truncate d4/d3c/d4d/d5c/d87/fbe 824778 0 2024-09-10T04:05:29.624 INFO:tasks.workunit.client.1.smithi070.stdout:2/786: write d15/d18/d56/d7c/d55/d6d/fad [513613,109590] 0 2024-09-10T04:05:29.625 INFO:tasks.workunit.client.0.smithi046.stdout:9/762: write d4/d3c/d4d/d5c/d87/fbe [717675,118884] 0 2024-09-10T04:05:29.626 INFO:tasks.workunit.client.1.smithi070.stdout:5/856: rename d17/d19/d6a/d54/de1/dfe/l39 to d17/d19/d6a/l12a 0 2024-09-10T04:05:29.629 INFO:tasks.workunit.client.1.smithi070.stdout:0/988: getdents d8/d13/d1a/d8f 0 2024-09-10T04:05:29.629 INFO:tasks.workunit.client.0.smithi046.stdout:8/774: fsync d1/d29/d19/d36/d4c/d5e/f5f 0 2024-09-10T04:05:29.630 INFO:tasks.workunit.client.1.smithi070.stdout:3/828: mknod db/d1c/d53/c11e 0 2024-09-10T04:05:29.631 INFO:tasks.workunit.client.1.smithi070.stdout:1/750: dwrite d3/d8/d27/f68 [0,4194304] 0 2024-09-10T04:05:29.632 INFO:tasks.workunit.client.0.smithi046.stdout:2/745: mkdir d2/d8/d109 0 2024-09-10T04:05:29.632 INFO:tasks.workunit.client.1.smithi070.stdout:5/857: write d17/d19/d35/f5f [199388,91508] 0 2024-09-10T04:05:29.633 INFO:tasks.workunit.client.1.smithi070.stdout:1/751: chown d3/d10/db2 7 1 2024-09-10T04:05:29.634 INFO:tasks.workunit.client.0.smithi046.stdout:5/919: rmdir d7/d11/d3f/d53/d81 39 2024-09-10T04:05:29.634 INFO:tasks.workunit.client.1.smithi070.stdout:5/858: dread - d17/fd7 zero size 2024-09-10T04:05:29.634 INFO:tasks.workunit.client.0.smithi046.stdout:5/920: dread - d7/d11/d3f/d53/d75/f9f zero size 2024-09-10T04:05:29.635 INFO:tasks.workunit.client.1.smithi070.stdout:6/822: dread d0/d3/d18/d1b/d17/d53/fb4 [0,4194304] 0 2024-09-10T04:05:29.638 INFO:tasks.workunit.client.1.smithi070.stdout:8/886: getdents d4/d39/dfc/d43/d67 0 2024-09-10T04:05:29.639 INFO:tasks.workunit.client.0.smithi046.stdout:4/909: truncate d3/d12/d17/d22/d51/d5c/f76 2503428 0 2024-09-10T04:05:29.641 INFO:tasks.workunit.client.0.smithi046.stdout:6/664: truncate da/f60 620554 0 2024-09-10T04:05:29.643 INFO:tasks.workunit.client.1.smithi070.stdout:1/752: dwrite d3/d8/d12/d2f/d32/f41 [0,4194304] 0 2024-09-10T04:05:29.647 INFO:tasks.workunit.client.0.smithi046.stdout:4/910: dwrite d3/d12/d17/d29/f38 [0,4194304] 0 2024-09-10T04:05:29.648 INFO:tasks.workunit.client.1.smithi070.stdout:3/829: fsync db/d1c/d25/f86 0 2024-09-10T04:05:29.652 INFO:tasks.workunit.client.1.smithi070.stdout:5/859: dread d17/d24/d84/d9b/fb5 [0,4194304] 0 2024-09-10T04:05:29.656 INFO:tasks.workunit.client.0.smithi046.stdout:2/746: dread d2/d6/d23/f4b [0,4194304] 0 2024-09-10T04:05:29.657 INFO:tasks.workunit.client.1.smithi070.stdout:6/823: mknod d0/d3/d18/d1b/d17/d53/c100 0 2024-09-10T04:05:29.661 INFO:tasks.workunit.client.1.smithi070.stdout:7/880: rename d2/d3/dc/d13/dbd/le9 to d2/d3/dc/d79/d11d/d127/l138 0 2024-09-10T04:05:29.662 INFO:tasks.workunit.client.0.smithi046.stdout:6/665: fdatasync da/db/d11/d13/d1e/f82 0 2024-09-10T04:05:29.662 INFO:tasks.workunit.client.1.smithi070.stdout:7/881: readlink d2/d40/d6c/ld8 0 2024-09-10T04:05:29.663 INFO:tasks.workunit.client.0.smithi046.stdout:9/763: write d4/d18/d92/fe7 [489002,84556] 0 2024-09-10T04:05:29.663 INFO:tasks.workunit.client.1.smithi070.stdout:8/887: truncate d4/dbf/d87/f9f 5216533 0 2024-09-10T04:05:29.664 INFO:tasks.workunit.client.0.smithi046.stdout:6/666: write da/db/d11/d2b/d2f/d3b/f5e [871704,100855] 0 2024-09-10T04:05:29.669 INFO:tasks.workunit.client.1.smithi070.stdout:1/753: symlink d3/d8/d13/d3f/d6d/d81/lf6 0 2024-09-10T04:05:29.676 INFO:tasks.workunit.client.0.smithi046.stdout:5/921: dwrite d7/d11/d14/d45/d51/f91 [0,4194304] 0 2024-09-10T04:05:29.677 INFO:tasks.workunit.client.0.smithi046.stdout:6/667: dread da/f24 [0,4194304] 0 2024-09-10T04:05:29.678 INFO:tasks.workunit.client.1.smithi070.stdout:5/860: symlink d17/d24/d77/l12b 0 2024-09-10T04:05:29.679 INFO:tasks.workunit.client.1.smithi070.stdout:5/861: write d17/d24/d84/d9b/fcb [960406,27811] 0 2024-09-10T04:05:29.680 INFO:tasks.workunit.client.0.smithi046.stdout:2/747: mkdir d2/d8/d37/d10a 0 2024-09-10T04:05:29.680 INFO:tasks.workunit.client.1.smithi070.stdout:6/824: mknod d0/d3/d18/d1b/d17/d3f/d40/c101 0 2024-09-10T04:05:29.681 INFO:tasks.workunit.client.0.smithi046.stdout:2/748: stat d2/d6/d23/d5d/d77/d7b/dd8/f104 0 2024-09-10T04:05:29.682 INFO:tasks.workunit.client.0.smithi046.stdout:9/764: write d4/d18/da8/fcf [985726,19058] 0 2024-09-10T04:05:29.683 INFO:tasks.workunit.client.0.smithi046.stdout:2/749: read d2/f73 [770252,2463] 0 2024-09-10T04:05:29.686 INFO:tasks.workunit.client.1.smithi070.stdout:8/888: mkdir d4/d16/dcc/d12f 0 2024-09-10T04:05:29.686 INFO:tasks.workunit.client.0.smithi046.stdout:2/750: dread - d2/d6/d9d/db6/fb7 zero size 2024-09-10T04:05:29.687 INFO:tasks.workunit.client.1.smithi070.stdout:2/787: link d15/d18/f20 d15/d2b/d94/f120 0 2024-09-10T04:05:29.689 INFO:tasks.workunit.client.0.smithi046.stdout:4/911: dwrite d3/fe3 [0,4194304] 0 2024-09-10T04:05:29.692 INFO:tasks.workunit.client.1.smithi070.stdout:1/754: rename d3/d8/c33 to d3/d8/d12/d2f/dcc/de5/cf7 0 2024-09-10T04:05:29.692 INFO:tasks.workunit.client.1.smithi070.stdout:8/889: dwrite d4/d39/dfc/d43/d67/dec/d106/f10f [0,4194304] 0 2024-09-10T04:05:29.692 INFO:tasks.workunit.client.0.smithi046.stdout:4/912: fsync d3/d12/d17/d29/d58/db2/d7a/f128 0 2024-09-10T04:05:29.693 INFO:tasks.workunit.client.0.smithi046.stdout:8/775: dwrite d1/f2e [0,4194304] 0 2024-09-10T04:05:29.694 INFO:tasks.workunit.client.0.smithi046.stdout:4/913: chown d3/d12/d17/d29/c3b 34 1 2024-09-10T04:05:29.694 INFO:tasks.workunit.client.0.smithi046.stdout:5/922: unlink d7/d11/d14/d2f/d2b/d92/f13b 0 2024-09-10T04:05:29.696 INFO:tasks.workunit.client.1.smithi070.stdout:3/830: creat db/d83/d11a/f11f x:0 0 0 2024-09-10T04:05:29.704 INFO:tasks.workunit.client.1.smithi070.stdout:1/755: dread f1 [0,4194304] 0 2024-09-10T04:05:29.712 INFO:tasks.workunit.client.0.smithi046.stdout:8/776: dwrite d1/d29/d17/d1f/d46/d6a/d82/f100 [0,4194304] 0 2024-09-10T04:05:29.720 INFO:tasks.workunit.client.0.smithi046.stdout:2/751: dread d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:29.763 INFO:tasks.workunit.client.1.smithi070.stdout:9/867: sync 2024-09-10T04:05:29.775 INFO:tasks.workunit.client.0.smithi046.stdout:6/668: fsync da/db/d11/dd7/f34 0 2024-09-10T04:05:29.793 INFO:tasks.workunit.client.0.smithi046.stdout:5/923: rename d7/d11/f23 to d7/d11/d14/d2f/df0/f13f 0 2024-09-10T04:05:29.806 INFO:tasks.workunit.client.1.smithi070.stdout:1/756: symlink d3/d8/d13/d3f/d6d/dbe/lf8 0 2024-09-10T04:05:29.808 INFO:tasks.workunit.client.1.smithi070.stdout:9/868: readlink d9/d1c/dac/d8a/d3c/d62/l94 0 2024-09-10T04:05:29.813 INFO:tasks.workunit.client.1.smithi070.stdout:9/869: dwrite d9/d7a/d74/fc6 [0,4194304] 0 2024-09-10T04:05:29.817 INFO:tasks.workunit.client.1.smithi070.stdout:7/882: creat d2/d7/d9/d18/f139 x:0 0 0 2024-09-10T04:05:29.818 INFO:tasks.workunit.client.0.smithi046.stdout:8/777: symlink d1/d29/d17/d1f/d46/d67/de4/l104 0 2024-09-10T04:05:29.819 INFO:tasks.workunit.client.0.smithi046.stdout:8/778: stat d1/d29/d17/f9c 0 2024-09-10T04:05:29.821 INFO:tasks.workunit.client.0.smithi046.stdout:8/779: readlink d1/d29/d19/d36/d4c/d5e/l8c 0 2024-09-10T04:05:29.822 INFO:tasks.workunit.client.1.smithi070.stdout:2/788: getdents d15/d18/d99 0 2024-09-10T04:05:29.824 INFO:tasks.workunit.client.0.smithi046.stdout:9/765: link d4/d3c/d4d/d5c/d59/l5a d4/d7/d1f/dce/dd4/lf1 0 2024-09-10T04:05:29.825 INFO:tasks.workunit.client.0.smithi046.stdout:9/766: fsync d4/d18/d89/fa7 0 2024-09-10T04:05:29.826 INFO:tasks.workunit.client.0.smithi046.stdout:9/767: write d4/d44/f85 [78238,9031] 0 2024-09-10T04:05:29.827 INFO:tasks.workunit.client.0.smithi046.stdout:9/768: stat d4/d3c/d4d/d80/fa4 0 2024-09-10T04:05:29.828 INFO:tasks.workunit.client.1.smithi070.stdout:1/757: write d3/d8/d13/d67/d83/fc5 [10130645,79655] 0 2024-09-10T04:05:29.833 INFO:tasks.workunit.client.0.smithi046.stdout:8/780: creat d1/d29/d19/d36/d6e/d75/f105 x:0 0 0 2024-09-10T04:05:29.840 INFO:tasks.workunit.client.0.smithi046.stdout:4/914: link d3/d12/d17/f6e d3/d12/dec/f138 0 2024-09-10T04:05:29.846 INFO:tasks.workunit.client.1.smithi070.stdout:4/789: sync 2024-09-10T04:05:29.846 INFO:tasks.workunit.client.1.smithi070.stdout:0/989: sync 2024-09-10T04:05:29.848 INFO:tasks.workunit.client.0.smithi046.stdout:8/781: creat d1/d29/d17/f106 x:0 0 0 2024-09-10T04:05:29.848 INFO:tasks.workunit.client.1.smithi070.stdout:7/883: dread d2/dc8/d119/d7e/f93 [0,4194304] 0 2024-09-10T04:05:29.849 INFO:tasks.workunit.client.1.smithi070.stdout:7/884: fdatasync d2/d7/d9/dd5/f108 0 2024-09-10T04:05:29.850 INFO:tasks.workunit.client.0.smithi046.stdout:4/915: mknod d3/dab/db9/c139 0 2024-09-10T04:05:29.857 INFO:tasks.workunit.client.1.smithi070.stdout:9/870: link d9/d1c/dac/d8a/dc4/dda/de0/d90/cf7 d9/d59/d2a/d3f/daf/c119 0 2024-09-10T04:05:29.864 INFO:tasks.workunit.client.0.smithi046.stdout:8/782: fsync d1/d29/d17/d1f/d46/ffb 0 2024-09-10T04:05:29.864 INFO:tasks.workunit.client.0.smithi046.stdout:8/783: chown d1/d29/d17/d1f 27774 1 2024-09-10T04:05:29.866 INFO:tasks.workunit.client.0.smithi046.stdout:8/784: chown d1/d29/d17/d1f/d2a/f2d 320735782 1 2024-09-10T04:05:29.866 INFO:tasks.workunit.client.1.smithi070.stdout:2/789: rename d15/d18/f1a to d15/d18/f121 0 2024-09-10T04:05:29.867 INFO:tasks.workunit.client.1.smithi070.stdout:2/790: chown d15/d18/d29/d2a/d2e/d32/f100 33 1 2024-09-10T04:05:29.868 INFO:tasks.workunit.client.0.smithi046.stdout:9/769: creat d4/d7/d1f/ff2 x:0 0 0 2024-09-10T04:05:29.870 INFO:tasks.workunit.client.1.smithi070.stdout:4/790: dread de/d13/d18/d23/f3c [0,4194304] 0 2024-09-10T04:05:29.875 INFO:tasks.workunit.client.0.smithi046.stdout:9/770: dwrite d4/d7/d1f/d54/dc7/ff0 [0,4194304] 0 2024-09-10T04:05:29.875 INFO:tasks.workunit.client.1.smithi070.stdout:7/885: read d2/d7/d9/d18/d3c/fae [3453121,75834] 0 2024-09-10T04:05:29.879 INFO:tasks.workunit.client.1.smithi070.stdout:9/871: mknod d9/d7a/d74/d8c/c11a 0 2024-09-10T04:05:29.880 INFO:tasks.workunit.client.1.smithi070.stdout:9/872: dread - d9/d1c/d34/d8b/fef zero size 2024-09-10T04:05:29.888 INFO:tasks.workunit.client.0.smithi046.stdout:8/785: link d1/d29/d19/d36/lb2 d1/d29/d19/d36/d4c/d5e/dc7/l107 0 2024-09-10T04:05:29.891 INFO:tasks.workunit.client.0.smithi046.stdout:9/771: fdatasync d4/d3c/d4d/d5c/d59/f5d 0 2024-09-10T04:05:29.893 INFO:tasks.workunit.client.1.smithi070.stdout:7/886: fdatasync d2/d7/d9/d18/d3c/d52/fc9 0 2024-09-10T04:05:29.902 INFO:tasks.workunit.client.0.smithi046.stdout:8/786: dread d1/d29/d17/d1f/d2a/f2d [0,4194304] 0 2024-09-10T04:05:29.904 INFO:tasks.workunit.client.0.smithi046.stdout:8/787: dread - d1/d29/d17/d1f/d46/d6a/fe8 zero size 2024-09-10T04:05:29.905 INFO:tasks.workunit.client.0.smithi046.stdout:8/788: truncate d1/d29/d19/d36/fb6 4503504 0 2024-09-10T04:05:29.906 INFO:tasks.workunit.client.1.smithi070.stdout:7/887: creat d2/d7/d9/d18/f13a x:0 0 0 2024-09-10T04:05:29.907 INFO:tasks.workunit.client.1.smithi070.stdout:4/791: getdents de/d13/d18/d26/d8c/d93 0 2024-09-10T04:05:29.907 INFO:tasks.workunit.client.1.smithi070.stdout:7/888: fsync d2/d7/d9/f82 0 2024-09-10T04:05:29.910 INFO:tasks.workunit.client.1.smithi070.stdout:7/889: fdatasync d2/d66/f78 0 2024-09-10T04:05:29.911 INFO:tasks.workunit.client.1.smithi070.stdout:5/862: symlink d17/d6f/d32/d90/l12c 0 2024-09-10T04:05:29.914 INFO:tasks.workunit.client.1.smithi070.stdout:4/792: dwrite de/d13/d5b/da0/f108 [0,4194304] 0 2024-09-10T04:05:29.916 INFO:tasks.workunit.client.0.smithi046.stdout:7/835: sync 2024-09-10T04:05:29.916 INFO:tasks.workunit.client.1.smithi070.stdout:4/793: readlink de/d13/d18/d23/l55 0 2024-09-10T04:05:29.919 INFO:tasks.workunit.client.0.smithi046.stdout:7/836: mkdir d5/d9/d12/d72/d77/de0/d9f/d126 0 2024-09-10T04:05:29.920 INFO:tasks.workunit.client.1.smithi070.stdout:5/863: dwrite d17/d19/d6a/d54/d57/d63/f9a [4194304,4194304] 0 2024-09-10T04:05:29.925 INFO:tasks.workunit.client.0.smithi046.stdout:7/837: dread d5/d9/d43/db4/def/f105 [0,4194304] 0 2024-09-10T04:05:29.926 INFO:tasks.workunit.client.1.smithi070.stdout:5/864: unlink d17/d24/d84/lb3 0 2024-09-10T04:05:29.927 INFO:tasks.workunit.client.1.smithi070.stdout:4/794: dwrite de/d13/da6/ddb/fe8 [0,4194304] 0 2024-09-10T04:05:29.927 INFO:tasks.workunit.client.1.smithi070.stdout:5/865: readlink d17/d24/d2b/l109 0 2024-09-10T04:05:29.931 INFO:tasks.workunit.client.1.smithi070.stdout:4/795: creat de/d13/d43/d88/db5/f10c x:0 0 0 2024-09-10T04:05:29.935 INFO:tasks.workunit.client.1.smithi070.stdout:5/866: getdents d17/d24/d45 0 2024-09-10T04:05:29.936 INFO:tasks.workunit.client.1.smithi070.stdout:5/867: write d17/d19/d35/f3a [1176654,33624] 0 2024-09-10T04:05:29.944 INFO:tasks.workunit.client.0.smithi046.stdout:7/838: dread d5/d6/d39/d5f/f10a [0,4194304] 0 2024-09-10T04:05:29.946 INFO:tasks.workunit.client.0.smithi046.stdout:7/839: truncate d5/d9/d12/f109 525389 0 2024-09-10T04:05:29.948 INFO:tasks.workunit.client.1.smithi070.stdout:6/825: dwrite d0/d3/d18/d1b/d17/d3f/d40/fd7 [0,4194304] 0 2024-09-10T04:05:29.951 INFO:tasks.workunit.client.1.smithi070.stdout:3/831: write db/f3c [1554658,18234] 0 2024-09-10T04:05:29.951 INFO:tasks.workunit.client.0.smithi046.stdout:2/752: rename d2/c68 to d2/d6/d23/d42/de6/dec/c10b 0 2024-09-10T04:05:29.952 INFO:tasks.workunit.client.0.smithi046.stdout:6/669: write da/db/d11/d2b/d55/fb2 [1311481,72020] 0 2024-09-10T04:05:29.954 INFO:tasks.workunit.client.0.smithi046.stdout:2/753: read d2/d6/d23/ff0 [156881,23733] 0 2024-09-10T04:05:29.955 INFO:tasks.workunit.client.1.smithi070.stdout:5/868: dread d17/d19/d35/f72 [0,4194304] 0 2024-09-10T04:05:29.955 INFO:tasks.workunit.client.1.smithi070.stdout:8/890: dwrite d4/d16/dcc/d12e/d11b/d1a/d60/f11f [0,4194304] 0 2024-09-10T04:05:29.958 INFO:tasks.workunit.client.0.smithi046.stdout:7/840: dwrite d5/d6/db/dd/d10c/f113 [0,4194304] 0 2024-09-10T04:05:29.961 INFO:tasks.workunit.client.1.smithi070.stdout:3/832: creat db/d83/d92/dc7/f120 x:0 0 0 2024-09-10T04:05:29.967 INFO:tasks.workunit.client.0.smithi046.stdout:7/841: read d5/d6/d32/dab/fb1 [519992,50072] 0 2024-09-10T04:05:29.970 INFO:tasks.workunit.client.1.smithi070.stdout:3/833: truncate db/f11 4764684 0 2024-09-10T04:05:29.974 INFO:tasks.workunit.client.1.smithi070.stdout:8/891: rmdir d4/d16/dcc/d12e/d11b/d1a/d60/d7a/db8 39 2024-09-10T04:05:29.975 INFO:tasks.workunit.client.1.smithi070.stdout:8/892: chown d4/d39/dfc/d43 1659736 1 2024-09-10T04:05:29.975 INFO:tasks.workunit.client.1.smithi070.stdout:3/834: creat db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/de1/dfe/f121 x:0 0 0 2024-09-10T04:05:29.978 INFO:tasks.workunit.client.1.smithi070.stdout:3/835: rmdir db/d83/d92 39 2024-09-10T04:05:29.981 INFO:tasks.workunit.client.1.smithi070.stdout:3/836: write db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/de1/dfe/f10b [1045154,66037] 0 2024-09-10T04:05:29.984 INFO:tasks.workunit.client.1.smithi070.stdout:5/869: dread d17/d19/d6a/d54/d57/da3/f55 [0,4194304] 0 2024-09-10T04:05:29.989 INFO:tasks.workunit.client.1.smithi070.stdout:8/893: dread d4/d16/dcc/d12e/d11b/d1a/f78 [0,4194304] 0 2024-09-10T04:05:30.003 INFO:tasks.workunit.client.1.smithi070.stdout:5/870: truncate d17/d19/fc3 3530720 0 2024-09-10T04:05:30.005 INFO:tasks.workunit.client.1.smithi070.stdout:5/871: write d17/d19/d6a/d54/d57/d63/fc6 [2215101,20704] 0 2024-09-10T04:05:30.010 INFO:tasks.workunit.client.1.smithi070.stdout:3/837: link db/d1c/d25/lde db/d1c/d25/d28/dd3/de5/d10f/d94/d99/l122 0 2024-09-10T04:05:30.010 INFO:tasks.workunit.client.1.smithi070.stdout:3/838: readlink db/l64 0 2024-09-10T04:05:30.010 INFO:tasks.workunit.client.1.smithi070.stdout:1/758: truncate d3/d10/d1d/f46 1423542 0 2024-09-10T04:05:30.011 INFO:tasks.workunit.client.1.smithi070.stdout:1/759: chown d3/d8/d27/cce 128736 1 2024-09-10T04:05:30.013 INFO:tasks.workunit.client.0.smithi046.stdout:7/842: mkdir d5/d6/d32/dab/d127 0 2024-09-10T04:05:30.015 INFO:tasks.workunit.client.0.smithi046.stdout:7/843: chown d5/d9/d12/d72/d77/de0/d1e 151910 1 2024-09-10T04:05:30.019 INFO:tasks.workunit.client.0.smithi046.stdout:4/916: dwrite d3/dab/dca/de4/fe9 [0,4194304] 0 2024-09-10T04:05:30.019 INFO:tasks.workunit.client.0.smithi046.stdout:2/754: getdents d2/d6/d23/d42/db8 0 2024-09-10T04:05:30.020 INFO:tasks.workunit.client.0.smithi046.stdout:2/755: chown d2/d6/d23/d42/de6 3 1 2024-09-10T04:05:30.021 INFO:tasks.workunit.client.0.smithi046.stdout:2/756: chown d2/d6/d23/d5d 240 1 2024-09-10T04:05:30.023 INFO:tasks.workunit.client.1.smithi070.stdout:5/872: getdents d17/d19/d6a/d54/d57/da8 0 2024-09-10T04:05:30.026 INFO:tasks.workunit.client.1.smithi070.stdout:3/839: getdents db/d1c/d25/d28/dd3/de5/d10f/db5 0 2024-09-10T04:05:30.029 INFO:tasks.workunit.client.0.smithi046.stdout:4/917: mkdir d3/d12/d17/d22/d30/dc7/d13a 0 2024-09-10T04:05:30.029 INFO:tasks.workunit.client.1.smithi070.stdout:5/873: dwrite d17/d6f/d32/ffc [0,4194304] 0 2024-09-10T04:05:30.031 INFO:tasks.workunit.client.1.smithi070.stdout:0/990: truncate d8/d13/f28 265776 0 2024-09-10T04:05:30.034 INFO:tasks.workunit.client.0.smithi046.stdout:2/757: creat d2/d6/d23/d42/db8/f10c x:0 0 0 2024-09-10T04:05:30.037 INFO:tasks.workunit.client.0.smithi046.stdout:4/918: dwrite d3/d12/d17/d29/dba/f118 [0,4194304] 0 2024-09-10T04:05:30.037 INFO:tasks.workunit.client.0.smithi046.stdout:7/844: getdents d5/d6/dbe 0 2024-09-10T04:05:30.040 INFO:tasks.workunit.client.1.smithi070.stdout:5/874: mkdir d17/d19/d12d 0 2024-09-10T04:05:30.040 INFO:tasks.workunit.client.0.smithi046.stdout:7/845: write d5/d6/db/dd/d4b/fc7 [114186,98445] 0 2024-09-10T04:05:30.042 INFO:tasks.workunit.client.0.smithi046.stdout:4/919: rmdir d3/d12/d17/d29/d58/db2 39 2024-09-10T04:05:30.042 INFO:tasks.workunit.client.0.smithi046.stdout:9/772: write d4/d44/f86 [1509593,43491] 0 2024-09-10T04:05:30.045 INFO:tasks.workunit.client.0.smithi046.stdout:7/846: truncate d5/d6/d32/dab/db6/f64 5850771 0 2024-09-10T04:05:30.047 INFO:tasks.workunit.client.1.smithi070.stdout:5/875: stat d17/d6f/d32/d90/d99 0 2024-09-10T04:05:30.049 INFO:tasks.workunit.client.0.smithi046.stdout:9/773: creat d4/d7/d61/ff3 x:0 0 0 2024-09-10T04:05:30.049 INFO:tasks.workunit.client.1.smithi070.stdout:9/873: dwrite d9/d1c/dac/fe2 [0,4194304] 0 2024-09-10T04:05:30.050 INFO:tasks.workunit.client.0.smithi046.stdout:2/758: dwrite d2/d6/d9d/db6/dbe/fde [0,4194304] 0 2024-09-10T04:05:30.051 INFO:tasks.workunit.client.0.smithi046.stdout:2/759: readlink d2/d6/l39 0 2024-09-10T04:05:30.052 INFO:tasks.workunit.client.0.smithi046.stdout:4/920: truncate d3/fc 7484281 0 2024-09-10T04:05:30.056 INFO:tasks.workunit.client.1.smithi070.stdout:2/791: dwrite d15/d18/f26 [0,4194304] 0 2024-09-10T04:05:30.062 INFO:tasks.workunit.client.1.smithi070.stdout:0/991: dread d8/d24/f43 [0,4194304] 0 2024-09-10T04:05:30.063 INFO:tasks.workunit.client.0.smithi046.stdout:2/760: mkdir d2/d8/d37/dc2/d10d 0 2024-09-10T04:05:30.064 INFO:tasks.workunit.client.1.smithi070.stdout:9/874: dwrite d9/d1c/dac/d8a/d3c/d92/ff6 [0,4194304] 0 2024-09-10T04:05:30.064 INFO:tasks.workunit.client.1.smithi070.stdout:2/792: mknod d15/d18/d29/d64/c122 0 2024-09-10T04:05:30.068 INFO:tasks.workunit.client.1.smithi070.stdout:7/890: write d2/d3/dc/d5d/f46 [123328,47379] 0 2024-09-10T04:05:30.073 INFO:tasks.workunit.client.1.smithi070.stdout:9/875: creat d9/d7a/d74/d8c/f11b x:0 0 0 2024-09-10T04:05:30.074 INFO:tasks.workunit.client.1.smithi070.stdout:4/796: rename de/d13/d5b/d7b/dd5/daf to de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/d10d 0 2024-09-10T04:05:30.075 INFO:tasks.workunit.client.1.smithi070.stdout:4/797: stat de/d13/d18/d23/d39/d77/fb1 0 2024-09-10T04:05:30.076 INFO:tasks.workunit.client.0.smithi046.stdout:7/847: dread d5/d6/db/dd/d24/d76/d9c/f114 [0,4194304] 0 2024-09-10T04:05:30.076 INFO:tasks.workunit.client.1.smithi070.stdout:4/798: readlink de/d13/d18/d23/d47/d59/l5f 0 2024-09-10T04:05:30.076 INFO:tasks.workunit.client.0.smithi046.stdout:7/848: stat d5/f60 0 2024-09-10T04:05:30.080 INFO:tasks.workunit.client.0.smithi046.stdout:9/774: dread d4/d44/f49 [0,4194304] 0 2024-09-10T04:05:30.082 INFO:tasks.workunit.client.0.smithi046.stdout:4/921: mkdir d3/d12/d17/d29/d58/db2/d5f/d13b 0 2024-09-10T04:05:30.086 INFO:tasks.workunit.client.1.smithi070.stdout:7/891: dwrite d2/d3/dc/d5d/d42/f7c [0,4194304] 0 2024-09-10T04:05:30.086 INFO:tasks.workunit.client.0.smithi046.stdout:5/924: rename d7/d11/d14/d2f/deb/da5/dcc/de5 to d7/d11/d14/d2f/deb/da5/dcc/dc0/df3/d140 0 2024-09-10T04:05:30.086 INFO:tasks.workunit.client.1.smithi070.stdout:7/892: stat d2/d7/l32 0 2024-09-10T04:05:30.087 INFO:tasks.workunit.client.1.smithi070.stdout:7/893: write d2/d7/d9/f113 [563272,124033] 0 2024-09-10T04:05:30.088 INFO:tasks.workunit.client.1.smithi070.stdout:7/894: readlink d2/d7/d9/l95 0 2024-09-10T04:05:30.088 INFO:tasks.workunit.client.1.smithi070.stdout:8/894: rename c1 to d4/d16/dcc/d12e/d11b/dab/c130 0 2024-09-10T04:05:30.089 INFO:tasks.workunit.client.1.smithi070.stdout:0/992: dread d8/f49 [4194304,4194304] 0 2024-09-10T04:05:30.093 INFO:tasks.workunit.client.0.smithi046.stdout:9/775: creat d4/d7/d61/ff4 x:0 0 0 2024-09-10T04:05:30.096 INFO:tasks.workunit.client.1.smithi070.stdout:0/993: dwrite d8/d13/de2/d7d/d8a/fbc [0,4194304] 0 2024-09-10T04:05:30.097 INFO:tasks.workunit.client.1.smithi070.stdout:1/760: rename d3/d8/d12/d2f/dcc/de5 to d3/d8/d27/d7a/df4/df9 0 2024-09-10T04:05:30.098 INFO:tasks.workunit.client.0.smithi046.stdout:9/776: dread d4/d7/d1f/d54/f6e [0,4194304] 0 2024-09-10T04:05:30.099 INFO:tasks.workunit.client.1.smithi070.stdout:0/994: write d8/d24/dce/f141 [714065,59211] 0 2024-09-10T04:05:30.101 INFO:tasks.workunit.client.1.smithi070.stdout:7/895: mknod d2/d3/dc/d5d/c13b 0 2024-09-10T04:05:30.102 INFO:tasks.workunit.client.1.smithi070.stdout:8/895: mknod d4/d39/dfc/d43/c131 0 2024-09-10T04:05:30.107 INFO:tasks.workunit.client.0.smithi046.stdout:9/777: symlink d4/d18/d89/ddb/lf5 0 2024-09-10T04:05:30.109 INFO:tasks.workunit.client.0.smithi046.stdout:9/778: truncate d4/d3c/d4d/d80/fe8 969989 0 2024-09-10T04:05:30.110 INFO:tasks.workunit.client.1.smithi070.stdout:4/799: rename de/d13/d18/d23/d39 to de/d13/d5b/d7b/dd5/d85/d10e 0 2024-09-10T04:05:30.113 INFO:tasks.workunit.client.0.smithi046.stdout:9/779: rename d4/d66/fae to d4/d29/d65/ff6 0 2024-09-10T04:05:30.114 INFO:tasks.workunit.client.1.smithi070.stdout:6/826: dwrite d0/d3/d18/d1b/d17/fb2 [0,4194304] 0 2024-09-10T04:05:30.114 INFO:tasks.workunit.client.1.smithi070.stdout:0/995: symlink d8/d85/df5/l142 0 2024-09-10T04:05:30.116 INFO:tasks.workunit.client.1.smithi070.stdout:6/827: fsync d0/d3/d1f/d81/fe6 0 2024-09-10T04:05:30.116 INFO:tasks.workunit.client.0.smithi046.stdout:9/780: symlink d4/d3c/d4d/d80/lf7 0 2024-09-10T04:05:30.117 INFO:tasks.workunit.client.1.smithi070.stdout:8/896: mknod d4/d16/dcc/d12e/d11b/d19/d62/c132 0 2024-09-10T04:05:30.119 INFO:tasks.workunit.client.0.smithi046.stdout:9/781: rmdir d4/d3c 39 2024-09-10T04:05:30.120 INFO:tasks.workunit.client.0.smithi046.stdout:9/782: chown d4/d7/d1f/d54/dc7/dc8/cd8 0 1 2024-09-10T04:05:30.121 INFO:tasks.workunit.client.0.smithi046.stdout:5/925: dread d7/f10 [0,4194304] 0 2024-09-10T04:05:30.124 INFO:tasks.workunit.client.1.smithi070.stdout:4/800: truncate de/d13/d18/d23/d47/d59/fc1 506181 0 2024-09-10T04:05:30.125 INFO:tasks.workunit.client.1.smithi070.stdout:8/897: dread d4/d16/dcc/d12e/d11b/d1a/fa9 [0,4194304] 0 2024-09-10T04:05:30.127 INFO:tasks.workunit.client.1.smithi070.stdout:8/898: write d4/d16/d73/fba [80469,29881] 0 2024-09-10T04:05:30.127 INFO:tasks.workunit.client.0.smithi046.stdout:9/783: readlink d4/d3c/d4d/d80/ldc 0 2024-09-10T04:05:30.128 INFO:tasks.workunit.client.1.smithi070.stdout:0/996: dread d8/d24/d2d/d5f/fe7 [0,4194304] 0 2024-09-10T04:05:30.132 INFO:tasks.workunit.client.0.smithi046.stdout:9/784: mknod d4/d18/d89/cf8 0 2024-09-10T04:05:30.133 INFO:tasks.workunit.client.0.smithi046.stdout:9/785: truncate d4/d7/d61/ff3 126740 0 2024-09-10T04:05:30.133 INFO:tasks.workunit.client.0.smithi046.stdout:5/926: unlink d7/d11/ce1 0 2024-09-10T04:05:30.135 INFO:tasks.workunit.client.1.smithi070.stdout:7/896: creat d2/d40/d6c/f13c x:0 0 0 2024-09-10T04:05:30.135 INFO:tasks.workunit.client.1.smithi070.stdout:7/897: chown d2/dc8/df9/f10d 4187158 1 2024-09-10T04:05:30.138 INFO:tasks.workunit.client.1.smithi070.stdout:4/801: mkdir de/d13/d18/d23/d47/d59/d10f 0 2024-09-10T04:05:30.141 INFO:tasks.workunit.client.0.smithi046.stdout:9/786: link d4/fc9 d4/d7/d1f/d54/ff9 0 2024-09-10T04:05:30.141 INFO:tasks.workunit.client.0.smithi046.stdout:6/670: write f1 [4826101,89885] 0 2024-09-10T04:05:30.142 INFO:tasks.workunit.client.0.smithi046.stdout:6/671: fsync da/db/d11/d2b/fbe 0 2024-09-10T04:05:30.142 INFO:tasks.workunit.client.1.smithi070.stdout:8/899: creat d4/d16/dcc/d12e/d11b/d6f/db5/f133 x:0 0 0 2024-09-10T04:05:30.143 INFO:tasks.workunit.client.0.smithi046.stdout:6/672: readlink da/db/d11/d13/d1e/d23/d54/lb7 0 2024-09-10T04:05:30.145 INFO:tasks.workunit.client.0.smithi046.stdout:9/787: dread d4/d18/d92/fe7 [0,4194304] 0 2024-09-10T04:05:30.145 INFO:tasks.workunit.client.1.smithi070.stdout:0/997: rename d8/d24/dce/f141 to d8/d13/de2/d53/d59/f143 0 2024-09-10T04:05:30.146 INFO:tasks.workunit.client.1.smithi070.stdout:0/998: truncate d8/d13/de2/d6f/fec 818219 0 2024-09-10T04:05:30.148 INFO:tasks.workunit.client.0.smithi046.stdout:1/592: sync 2024-09-10T04:05:30.150 INFO:tasks.workunit.client.0.smithi046.stdout:1/593: read d3/da1/f2e [4574064,63540] 0 2024-09-10T04:05:30.150 INFO:tasks.workunit.client.1.smithi070.stdout:6/828: truncate d0/fa7 416803 0 2024-09-10T04:05:30.152 INFO:tasks.workunit.client.0.smithi046.stdout:9/788: rename d4/d3c/d4d/d80/fa4 to d4/d18/d89/ffa 0 2024-09-10T04:05:30.153 INFO:tasks.workunit.client.0.smithi046.stdout:9/789: chown d4/d7/d81/fea 62 1 2024-09-10T04:05:30.154 INFO:tasks.workunit.client.1.smithi070.stdout:7/898: unlink d2/d40/c124 0 2024-09-10T04:05:30.155 INFO:tasks.workunit.client.1.smithi070.stdout:4/802: mknod de/d13/d5b/d7b/c110 0 2024-09-10T04:05:30.160 INFO:tasks.workunit.client.1.smithi070.stdout:3/840: write db/d1c/d25/d28/fba [256162,118549] 0 2024-09-10T04:05:30.163 INFO:tasks.workunit.client.1.smithi070.stdout:0/999: creat d8/d13/d1a/d75/f144 x:0 0 0 2024-09-10T04:05:30.165 INFO:tasks.workunit.client.0.smithi046.stdout:1/594: symlink d3/dc/d34/d58/lc4 0 2024-09-10T04:05:30.167 INFO:tasks.workunit.client.1.smithi070.stdout:6/829: rename d0/d3/d18/d2a/f59 to d0/d3/d18/d1b/d33/f102 0 2024-09-10T04:05:30.174 INFO:tasks.workunit.client.0.smithi046.stdout:6/673: rmdir da/db/d11/d13/d1e/d23/d54/d90/da3 0 2024-09-10T04:05:30.176 INFO:tasks.workunit.client.0.smithi046.stdout:1/595: creat d3/dc/d21/fc5 x:0 0 0 2024-09-10T04:05:30.177 INFO:tasks.workunit.client.1.smithi070.stdout:8/900: truncate d4/d16/dcc/d12e/fee 1775295 0 2024-09-10T04:05:30.183 INFO:tasks.workunit.client.1.smithi070.stdout:5/876: dwrite d17/f75 [0,4194304] 0 2024-09-10T04:05:30.184 INFO:tasks.workunit.client.1.smithi070.stdout:3/841: dwrite db/d1c/d25/d28/f73 [0,4194304] 0 2024-09-10T04:05:30.185 INFO:tasks.workunit.client.1.smithi070.stdout:7/899: mkdir d2/d7/d9/d18/d3c/d52/d107/d13d 0 2024-09-10T04:05:30.187 INFO:tasks.workunit.client.1.smithi070.stdout:7/900: write d2/d40/d6c/db2/db7/d104/f111 [785752,89032] 0 2024-09-10T04:05:30.187 INFO:tasks.workunit.client.0.smithi046.stdout:6/674: mkdir da/db/d11/d2b/d2f/dde 0 2024-09-10T04:05:30.188 INFO:tasks.workunit.client.0.smithi046.stdout:2/761: write d2/d6/d23/d42/db4/f105 [115613,36210] 0 2024-09-10T04:05:30.190 INFO:tasks.workunit.client.1.smithi070.stdout:4/803: mkdir de/d13/d18/d26/d111 0 2024-09-10T04:05:30.191 INFO:tasks.workunit.client.1.smithi070.stdout:4/804: write de/d13/d5b/da0/f108 [4116747,36432] 0 2024-09-10T04:05:30.194 INFO:tasks.workunit.client.0.smithi046.stdout:6/675: fdatasync da/db/d18/f49 0 2024-09-10T04:05:30.197 INFO:tasks.workunit.client.1.smithi070.stdout:9/876: write d9/f25 [1528670,49821] 0 2024-09-10T04:05:30.197 INFO:tasks.workunit.client.1.smithi070.stdout:8/901: mknod d4/d16/d73/d96/c134 0 2024-09-10T04:05:30.198 INFO:tasks.workunit.client.1.smithi070.stdout:8/902: write d4/d16/dcc/d12e/da4/fbd [799432,118762] 0 2024-09-10T04:05:30.200 INFO:tasks.workunit.client.1.smithi070.stdout:9/877: truncate d9/d1c/dac/d8a/d3c/d92/da8/f109 190816 0 2024-09-10T04:05:30.202 INFO:tasks.workunit.client.1.smithi070.stdout:2/793: write d15/d18/d29/d2a/d2e/d32/fc9 [207951,30948] 0 2024-09-10T04:05:30.204 INFO:tasks.workunit.client.1.smithi070.stdout:2/794: chown d15/d18/d29/d2a/d2e/da6/dc0/fce 49 1 2024-09-10T04:05:30.205 INFO:tasks.workunit.client.0.smithi046.stdout:4/922: write d3/d12/d17/d29/d58/db2/d5f/d85/da8/fce [1017557,89371] 0 2024-09-10T04:05:30.210 INFO:tasks.workunit.client.0.smithi046.stdout:2/762: rename d2/d8/d37/f66 to d2/d6/d23/d5d/f10e 0 2024-09-10T04:05:30.211 INFO:tasks.workunit.client.1.smithi070.stdout:4/805: fdatasync de/d13/d5b/d7b/dd5/d85/d10e/f67 0 2024-09-10T04:05:30.212 INFO:tasks.workunit.client.1.smithi070.stdout:4/806: dread - de/d13/d5b/d7b/dd5/d85/d10e/f67 zero size 2024-09-10T04:05:30.213 INFO:tasks.workunit.client.0.smithi046.stdout:7/849: dwrite d5/ff7 [0,4194304] 0 2024-09-10T04:05:30.218 INFO:tasks.workunit.client.0.smithi046.stdout:7/850: mkdir d5/d9/d12/d128 0 2024-09-10T04:05:30.222 INFO:tasks.workunit.client.0.smithi046.stdout:4/923: rename d3/d12/la5 to d3/d12/d14/d11d/l13c 0 2024-09-10T04:05:30.225 INFO:tasks.workunit.client.1.smithi070.stdout:2/795: mkdir d15/d2b/d106/d123 0 2024-09-10T04:05:30.228 INFO:tasks.workunit.client.0.smithi046.stdout:2/763: rename d2/d6/d23/l32 to d2/d8/d37/l10f 0 2024-09-10T04:05:30.228 INFO:tasks.workunit.client.1.smithi070.stdout:1/761: dwrite d3/d10/d1d/dbd/fd4 [4194304,4194304] 0 2024-09-10T04:05:30.230 INFO:tasks.workunit.client.0.smithi046.stdout:4/924: dwrite d3/d12/d17/d29/dba/fd9 [0,4194304] 0 2024-09-10T04:05:30.231 INFO:tasks.workunit.client.1.smithi070.stdout:7/901: unlink d2/d40/d6c/db2/fc2 0 2024-09-10T04:05:30.234 INFO:tasks.workunit.client.1.smithi070.stdout:7/902: dread d2/d7/d9/f113 [0,4194304] 0 2024-09-10T04:05:30.240 INFO:tasks.workunit.client.0.smithi046.stdout:4/925: mknod d3/dab/dca/de4/c13d 0 2024-09-10T04:05:30.241 INFO:tasks.workunit.client.0.smithi046.stdout:4/926: write d3/dab/dca/de4/f134 [480889,46027] 0 2024-09-10T04:05:30.247 INFO:tasks.workunit.client.0.smithi046.stdout:7/851: rename d5/d9/d12/d72/d77/de0/d1f/d7a/dfc to d5/d7f/d129 0 2024-09-10T04:05:30.250 INFO:tasks.workunit.client.0.smithi046.stdout:4/927: unlink d3/d12/d17/d29/d58/db2/f11e 0 2024-09-10T04:05:30.255 INFO:tasks.workunit.client.0.smithi046.stdout:7/852: rename d5/d6/db/dd/d4b/fc7 to d5/d9/d12/d72/d77/de0/d1f/d7a/f12a 0 2024-09-10T04:05:30.263 INFO:tasks.workunit.client.0.smithi046.stdout:7/853: creat d5/d9/d12/d72/d77/de0/d1f/f12b x:0 0 0 2024-09-10T04:05:30.268 INFO:tasks.workunit.client.0.smithi046.stdout:7/854: rename d5/d6/d55/d89/fe5 to d5/d6/d32/f12c 0 2024-09-10T04:05:30.270 INFO:tasks.workunit.client.0.smithi046.stdout:7/855: fsync d5/d9/d12/d72/d77/de0/d1f/f48 0 2024-09-10T04:05:30.270 INFO:tasks.workunit.client.0.smithi046.stdout:9/790: dwrite d4/d7/f7e [0,4194304] 0 2024-09-10T04:05:30.272 INFO:tasks.workunit.client.0.smithi046.stdout:9/791: truncate d4/d7/d1f/f21 5550081 0 2024-09-10T04:05:30.280 INFO:tasks.workunit.client.0.smithi046.stdout:7/856: dwrite d5/d9/d12/d72/d77/de0/d1f/d7a/f12a [0,4194304] 0 2024-09-10T04:05:30.286 INFO:tasks.workunit.client.0.smithi046.stdout:9/792: creat d4/d29/ffb x:0 0 0 2024-09-10T04:05:30.287 INFO:tasks.workunit.client.1.smithi070.stdout:6/830: dwrite d0/f8 [0,4194304] 0 2024-09-10T04:05:30.291 INFO:tasks.workunit.client.0.smithi046.stdout:7/857: truncate d5/d9/d12/d72/d77/de0/d9f/f11e 6788067 0 2024-09-10T04:05:30.293 INFO:tasks.workunit.client.0.smithi046.stdout:6/676: write da/db/f78 [3977322,7097] 0 2024-09-10T04:05:30.293 INFO:tasks.workunit.client.1.smithi070.stdout:4/807: symlink de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/d10d/df2/l112 0 2024-09-10T04:05:30.293 INFO:tasks.workunit.client.1.smithi070.stdout:6/831: dread d0/d3/d18/d1b/d17/d54/fc6 [0,4194304] 0 2024-09-10T04:05:30.294 INFO:tasks.workunit.client.1.smithi070.stdout:5/877: write d17/d24/d45/f98 [1423926,56727] 0 2024-09-10T04:05:30.295 INFO:tasks.workunit.client.1.smithi070.stdout:6/832: chown d0/d3/d1f/d81/la8 42187343 1 2024-09-10T04:05:30.298 INFO:tasks.workunit.client.0.smithi046.stdout:6/677: truncate da/db/d11/f8c 2148114 0 2024-09-10T04:05:30.299 INFO:tasks.workunit.client.0.smithi046.stdout:7/858: dwrite d5/d6/dbe/dd7/f92 [4194304,4194304] 0 2024-09-10T04:05:30.301 INFO:tasks.workunit.client.1.smithi070.stdout:3/842: dwrite db/d1c/d25/d28/dd3/de5/d10f/fa9 [0,4194304] 0 2024-09-10T04:05:30.305 INFO:tasks.workunit.client.0.smithi046.stdout:7/859: mknod d5/d6/db/dd/d10c/c12d 0 2024-09-10T04:05:30.308 INFO:tasks.workunit.client.1.smithi070.stdout:8/903: dwrite d4/d16/dcc/d12e/d11b/d19/f72 [0,4194304] 0 2024-09-10T04:05:30.309 INFO:tasks.workunit.client.0.smithi046.stdout:6/678: mkdir da/db/d11/d13/d1e/d69/ddf 0 2024-09-10T04:05:30.310 INFO:tasks.workunit.client.0.smithi046.stdout:0/638: sync 2024-09-10T04:05:30.310 INFO:tasks.workunit.client.0.smithi046.stdout:3/743: sync 2024-09-10T04:05:30.310 INFO:tasks.workunit.client.0.smithi046.stdout:6/679: chown da/db/d11/d13/d1e/f82 773615 1 2024-09-10T04:05:30.311 INFO:tasks.workunit.client.1.smithi070.stdout:9/878: dwrite d9/d1c/dac/d8a/d3c/d92/fa9 [0,4194304] 0 2024-09-10T04:05:30.311 INFO:tasks.workunit.client.0.smithi046.stdout:7/860: creat d5/d6/db/dd/d24/d76/f12e x:0 0 0 2024-09-10T04:05:30.312 INFO:tasks.workunit.client.0.smithi046.stdout:3/744: stat d12/d13/d37/d51/db5/d4b/d91/f9a 0 2024-09-10T04:05:30.313 INFO:tasks.workunit.client.0.smithi046.stdout:3/745: stat d12/d13/d37/l64 0 2024-09-10T04:05:30.315 INFO:tasks.workunit.client.0.smithi046.stdout:0/639: unlink d1/d13/d37/d75/f83 0 2024-09-10T04:05:30.317 INFO:tasks.workunit.client.0.smithi046.stdout:0/640: chown d1/dc/d25/d33/d3/dd/d55/f4c 1 1 2024-09-10T04:05:30.318 INFO:tasks.workunit.client.0.smithi046.stdout:6/680: unlink da/db/d11/d2b/d2f/d3b/f5e 0 2024-09-10T04:05:30.323 INFO:tasks.workunit.client.0.smithi046.stdout:2/764: truncate d2/d6/d9d/fa5 911468 0 2024-09-10T04:05:30.324 INFO:tasks.workunit.client.0.smithi046.stdout:7/861: symlink d5/d6/dbe/l12f 0 2024-09-10T04:05:30.329 INFO:tasks.workunit.client.1.smithi070.stdout:2/796: mknod d15/d18/db2/ded/df8/c124 0 2024-09-10T04:05:30.329 INFO:tasks.workunit.client.0.smithi046.stdout:2/765: dwrite d2/d6/d23/d42/de6/fcf [0,4194304] 0 2024-09-10T04:05:30.330 INFO:tasks.workunit.client.1.smithi070.stdout:5/878: dread d17/d24/d2b/f9d [0,4194304] 0 2024-09-10T04:05:30.336 INFO:tasks.workunit.client.0.smithi046.stdout:6/681: rename da/db/d11/d13/d4f/lc1 to da/db/d18/dd1/le0 0 2024-09-10T04:05:30.339 INFO:tasks.workunit.client.0.smithi046.stdout:4/928: write d3/d12/d17/d29/d58/db2/d5f/fb4 [1013195,75809] 0 2024-09-10T04:05:30.339 INFO:tasks.workunit.client.1.smithi070.stdout:3/843: read db/d1c/d3e/f54 [6383528,113857] 0 2024-09-10T04:05:30.344 INFO:tasks.workunit.client.0.smithi046.stdout:3/746: chown d12/d5d/d71/d49/f8d 8552985 1 2024-09-10T04:05:30.346 INFO:tasks.workunit.client.1.smithi070.stdout:9/879: creat d9/d59/d2a/f11c x:0 0 0 2024-09-10T04:05:30.347 INFO:tasks.workunit.client.0.smithi046.stdout:4/929: dwrite d3/d12/d17/d29/d58/db2/fc4 [0,4194304] 0 2024-09-10T04:05:30.348 INFO:tasks.workunit.client.0.smithi046.stdout:4/930: dread - d3/d12/d17/d22/d51/d94/faf zero size 2024-09-10T04:05:30.351 INFO:tasks.workunit.client.0.smithi046.stdout:6/682: read f9 [584352,48354] 0 2024-09-10T04:05:30.351 INFO:tasks.workunit.client.0.smithi046.stdout:7/862: getdents d5/d33 0 2024-09-10T04:05:30.352 INFO:tasks.workunit.client.0.smithi046.stdout:7/863: chown d5/d6/db/dd 33484412 1 2024-09-10T04:05:30.353 INFO:tasks.workunit.client.0.smithi046.stdout:6/683: chown da/db/d11/d2b/d2f/f72 219893825 1 2024-09-10T04:05:30.358 INFO:tasks.workunit.client.1.smithi070.stdout:5/879: rename d17/d19/d6a/d54/d57/d63/db9/dc2 to d17/d6f/d32/d90/da5/d12e 0 2024-09-10T04:05:30.359 INFO:tasks.workunit.client.0.smithi046.stdout:3/747: creat d12/d5d/d71/d49/f104 x:0 0 0 2024-09-10T04:05:30.362 INFO:tasks.workunit.client.1.smithi070.stdout:8/904: mkdir d4/d16/dcc/d12e/d135 0 2024-09-10T04:05:30.365 INFO:tasks.workunit.client.1.smithi070.stdout:9/880: rename d9/d1c/dac/d8a to d9/d1c/d34/d8b/d11d 0 2024-09-10T04:05:30.366 INFO:tasks.workunit.client.1.smithi070.stdout:9/881: chown d9/d1c/d34/d8b/ld1 183 1 2024-09-10T04:05:30.366 INFO:tasks.workunit.client.1.smithi070.stdout:5/880: mknod d17/d24/dcc/c12f 0 2024-09-10T04:05:30.367 INFO:tasks.workunit.client.0.smithi046.stdout:2/766: link d2/d8/d65/fa9 d2/d6/d23/d42/de6/f110 0 2024-09-10T04:05:30.367 INFO:tasks.workunit.client.1.smithi070.stdout:5/881: chown d17/d19/d6a/d54/de1/dfe/f91 11011 1 2024-09-10T04:05:30.368 INFO:tasks.workunit.client.1.smithi070.stdout:5/882: write d17/d19/d6a/d54/d57/da8/fe2 [4594605,1092] 0 2024-09-10T04:05:30.369 INFO:tasks.workunit.client.1.smithi070.stdout:5/883: readlink d17/d24/d84/la0 0 2024-09-10T04:05:30.370 INFO:tasks.workunit.client.1.smithi070.stdout:5/884: chown d17/d19/d6a/d54/d57/da3/c7a 403 1 2024-09-10T04:05:30.371 INFO:tasks.workunit.client.0.smithi046.stdout:4/931: symlink d3/d12/d17/d29/d58/db2/l13e 0 2024-09-10T04:05:30.373 INFO:tasks.workunit.client.0.smithi046.stdout:9/793: write d4/d18/f27 [1759591,44444] 0 2024-09-10T04:05:30.374 INFO:tasks.workunit.client.0.smithi046.stdout:7/864: creat d5/d9/d12/dc5/d125/f130 x:0 0 0 2024-09-10T04:05:30.377 INFO:tasks.workunit.client.0.smithi046.stdout:2/767: read d2/d8/d65/f7d [447987,117183] 0 2024-09-10T04:05:30.378 INFO:tasks.workunit.client.0.smithi046.stdout:2/768: fdatasync d2/d8/d65/d90/f100 0 2024-09-10T04:05:30.380 INFO:tasks.workunit.client.1.smithi070.stdout:8/905: unlink d4/f77 0 2024-09-10T04:05:30.381 INFO:tasks.workunit.client.1.smithi070.stdout:5/885: write d17/d19/d6a/d54/d57/fb0 [5059598,72331] 0 2024-09-10T04:05:30.382 INFO:tasks.workunit.client.0.smithi046.stdout:9/794: readlink d4/d3c/d4d/d5c/d59/l5a 0 2024-09-10T04:05:30.385 INFO:tasks.workunit.client.0.smithi046.stdout:9/795: read d4/d3c/d4d/d5c/f82 [1519267,119403] 0 2024-09-10T04:05:30.386 INFO:tasks.workunit.client.1.smithi070.stdout:7/903: dwrite d2/fbe [0,4194304] 0 2024-09-10T04:05:30.387 INFO:tasks.workunit.client.0.smithi046.stdout:4/932: dwrite d3/d12/d17/d29/d58/db2/d5f/d85/da8/f135 [4194304,4194304] 0 2024-09-10T04:05:30.389 INFO:tasks.workunit.client.0.smithi046.stdout:3/748: creat d12/d13/f105 x:0 0 0 2024-09-10T04:05:30.391 INFO:tasks.workunit.client.0.smithi046.stdout:3/749: fsync d12/d5d/d71/d49/d5c/fd1 0 2024-09-10T04:05:30.399 INFO:tasks.workunit.client.0.smithi046.stdout:4/933: dwrite d3/d12/d17/d29/d58/db2/d7a/f128 [0,4194304] 0 2024-09-10T04:05:30.400 INFO:tasks.workunit.client.1.smithi070.stdout:5/886: link d17/d19/d6a/d54/de1/dfe/f3d d17/d19/d6a/dfb/f130 0 2024-09-10T04:05:30.401 INFO:tasks.workunit.client.1.smithi070.stdout:6/833: write d0/d3/d18/d62/d70/f7c [91669,29060] 0 2024-09-10T04:05:30.401 INFO:tasks.workunit.client.0.smithi046.stdout:7/865: mknod d5/d9/d12/d128/c131 0 2024-09-10T04:05:30.404 INFO:tasks.workunit.client.1.smithi070.stdout:4/808: dwrite de/d13/d5b/d7b/dd5/d85/d10e/f6d [0,4194304] 0 2024-09-10T04:05:30.405 INFO:tasks.workunit.client.0.smithi046.stdout:7/866: read d5/d9/d12/d72/d77/f82 [3178504,76413] 0 2024-09-10T04:05:30.406 INFO:tasks.workunit.client.1.smithi070.stdout:7/904: getdents d2/d7/d9/dd5/d115 0 2024-09-10T04:05:30.409 INFO:tasks.workunit.client.0.smithi046.stdout:8/789: sync 2024-09-10T04:05:30.409 INFO:tasks.workunit.client.0.smithi046.stdout:8/790: stat d1/d29/d19/d36/f4f 0 2024-09-10T04:05:30.411 INFO:tasks.workunit.client.0.smithi046.stdout:8/791: write d1/d29/d12/f97 [315529,14409] 0 2024-09-10T04:05:30.411 INFO:tasks.workunit.client.1.smithi070.stdout:6/834: mknod d0/d3/da4/dc0/c103 0 2024-09-10T04:05:30.411 INFO:tasks.workunit.client.0.smithi046.stdout:0/641: write d1/dc/d25/d33/d3/d29/d2d/f4f [361297,18581] 0 2024-09-10T04:05:30.415 INFO:tasks.workunit.client.1.smithi070.stdout:4/809: creat de/d13/d18/d23/d47/d59/d79/dbc/f113 x:0 0 0 2024-09-10T04:05:30.418 INFO:tasks.workunit.client.0.smithi046.stdout:3/750: dread d12/d13/d14/d1e/d2d/f2e [0,4194304] 0 2024-09-10T04:05:30.419 INFO:tasks.workunit.client.0.smithi046.stdout:3/751: fdatasync d12/dd9/ddf/fe2 0 2024-09-10T04:05:30.421 INFO:tasks.workunit.client.1.smithi070.stdout:5/887: dread d17/d19/d6a/d54/de1/dfe/dce/f110 [0,4194304] 0 2024-09-10T04:05:30.424 INFO:tasks.workunit.client.1.smithi070.stdout:7/905: creat d2/d40/d6c/db2/db7/f13e x:0 0 0 2024-09-10T04:05:30.428 INFO:tasks.workunit.client.1.smithi070.stdout:6/835: unlink d0/d3/c13 0 2024-09-10T04:05:30.428 INFO:tasks.workunit.client.0.smithi046.stdout:0/642: dread d1/dc/f76 [0,4194304] 0 2024-09-10T04:05:30.429 INFO:tasks.workunit.client.0.smithi046.stdout:8/792: dread d1/d29/d19/d36/f3f [0,4194304] 0 2024-09-10T04:05:30.429 INFO:tasks.workunit.client.0.smithi046.stdout:3/752: dwrite d12/dd9/ddf/fe2 [0,4194304] 0 2024-09-10T04:05:30.431 INFO:tasks.workunit.client.1.smithi070.stdout:4/810: stat de/d13/d5b/d7b/dd5/d85/d10e/l109 0 2024-09-10T04:05:30.434 INFO:tasks.workunit.client.0.smithi046.stdout:4/934: mknod d3/d12/d17/d22/d51/d94/d107/c13f 0 2024-09-10T04:05:30.435 INFO:tasks.workunit.client.1.smithi070.stdout:5/888: unlink d17/d19/d6a/d54/ca2 0 2024-09-10T04:05:30.437 INFO:tasks.workunit.client.0.smithi046.stdout:6/684: write da/d45/f56 [2267128,110602] 0 2024-09-10T04:05:30.437 INFO:tasks.workunit.client.1.smithi070.stdout:7/906: creat d2/d3/d63/f13f x:0 0 0 2024-09-10T04:05:30.442 INFO:tasks.workunit.client.1.smithi070.stdout:4/811: creat de/d13/d5b/d7b/f114 x:0 0 0 2024-09-10T04:05:30.445 INFO:tasks.workunit.client.0.smithi046.stdout:7/867: rename d5/d9/d8b/l97 to d5/d7f/d129/l132 0 2024-09-10T04:05:30.446 INFO:tasks.workunit.client.1.smithi070.stdout:3/844: dwrite db/d1c/d25/d28/f37 [0,4194304] 0 2024-09-10T04:05:30.447 INFO:tasks.workunit.client.0.smithi046.stdout:8/793: unlink d1/d29/d17/d1f/d2a/f33 0 2024-09-10T04:05:30.448 INFO:tasks.workunit.client.1.smithi070.stdout:9/882: dwrite d9/d1c/d34/d8b/d11d/d1b/fb6 [0,4194304] 0 2024-09-10T04:05:30.448 INFO:tasks.workunit.client.1.smithi070.stdout:7/907: mknod d2/dc8/df9/c140 0 2024-09-10T04:05:30.449 INFO:tasks.workunit.client.0.smithi046.stdout:0/643: fsync d1/dc/d25/d33/d4/f36 0 2024-09-10T04:05:30.452 INFO:tasks.workunit.client.1.smithi070.stdout:8/906: write d4/d16/dcc/d12e/d11b/d1a/f2a [3438655,24174] 0 2024-09-10T04:05:30.454 INFO:tasks.workunit.client.1.smithi070.stdout:4/812: dwrite de/d13/d5b/d7b/fe2 [0,4194304] 0 2024-09-10T04:05:30.455 INFO:tasks.workunit.client.0.smithi046.stdout:8/794: dwrite d1/d29/d19/db7/fe1 [0,4194304] 0 2024-09-10T04:05:30.455 INFO:tasks.workunit.client.0.smithi046.stdout:4/935: chown d3/fc 309 1 2024-09-10T04:05:30.457 INFO:tasks.workunit.client.0.smithi046.stdout:8/795: chown d1/d29/d19/d36/d6e/d75/l92 222 1 2024-09-10T04:05:30.458 INFO:tasks.workunit.client.0.smithi046.stdout:2/769: write d2/f25 [1603518,16064] 0 2024-09-10T04:05:30.459 INFO:tasks.workunit.client.0.smithi046.stdout:1/596: sync 2024-09-10T04:05:30.459 INFO:tasks.workunit.client.0.smithi046.stdout:5/927: sync 2024-09-10T04:05:30.465 INFO:tasks.workunit.client.0.smithi046.stdout:1/597: chown d3/dc/d34/d58/f63 364 1 2024-09-10T04:05:30.467 INFO:tasks.workunit.client.1.smithi070.stdout:3/845: rmdir db/dcc 39 2024-09-10T04:05:30.467 INFO:tasks.workunit.client.0.smithi046.stdout:7/868: mkdir d5/d6/d62/d133 0 2024-09-10T04:05:30.470 INFO:tasks.workunit.client.1.smithi070.stdout:6/836: rename d0/d3/d18/d1b/d17/d54/db5/fbb to d0/d3/d18/d1b/d17/f104 0 2024-09-10T04:05:30.473 INFO:tasks.workunit.client.0.smithi046.stdout:0/644: creat d1/dc/d25/d33/d1a/d41/fc6 x:0 0 0 2024-09-10T04:05:30.474 INFO:tasks.workunit.client.1.smithi070.stdout:7/908: creat d2/d7/d9/d18/d3c/f141 x:0 0 0 2024-09-10T04:05:30.474 INFO:tasks.workunit.client.0.smithi046.stdout:0/645: readlink d1/dc/d25/d33/d1a/l9f 0 2024-09-10T04:05:30.477 INFO:tasks.workunit.client.0.smithi046.stdout:8/796: dread d1/d29/d19/f34 [0,4194304] 0 2024-09-10T04:05:30.479 INFO:tasks.workunit.client.0.smithi046.stdout:9/796: dwrite d4/d29/f3d [0,4194304] 0 2024-09-10T04:05:30.479 INFO:tasks.workunit.client.0.smithi046.stdout:4/936: creat d3/dab/dca/de4/f140 x:0 0 0 2024-09-10T04:05:30.485 INFO:tasks.workunit.client.0.smithi046.stdout:1/598: rmdir d3/d27 39 2024-09-10T04:05:30.489 INFO:tasks.workunit.client.1.smithi070.stdout:6/837: readlink d0/d3/d18/d62/l64 0 2024-09-10T04:05:30.489 INFO:tasks.workunit.client.0.smithi046.stdout:1/599: dread d3/d9b/fbc [0,4194304] 0 2024-09-10T04:05:30.490 INFO:tasks.workunit.client.0.smithi046.stdout:2/770: rmdir d2/d86 39 2024-09-10T04:05:30.490 INFO:tasks.workunit.client.1.smithi070.stdout:6/838: write d0/d3/d18/d1b/d17/d53/fa6 [2228452,112766] 0 2024-09-10T04:05:30.493 INFO:tasks.workunit.client.0.smithi046.stdout:6/685: write da/db/d11/d2b/d2f/f72 [993370,19964] 0 2024-09-10T04:05:30.493 INFO:tasks.workunit.client.1.smithi070.stdout:7/909: fsync d2/d40/d6c/db2/fe3 0 2024-09-10T04:05:30.494 INFO:tasks.workunit.client.0.smithi046.stdout:7/869: read - d5/d9/d43/db4/fca zero size 2024-09-10T04:05:30.495 INFO:tasks.workunit.client.1.smithi070.stdout:8/907: truncate d4/dbf/d87/f9f 5563265 0 2024-09-10T04:05:30.496 INFO:tasks.workunit.client.0.smithi046.stdout:3/753: creat d12/d13/d37/d51/db5/d50/f106 x:0 0 0 2024-09-10T04:05:30.498 INFO:tasks.workunit.client.1.smithi070.stdout:4/813: mkdir de/d13/da6/d115 0 2024-09-10T04:05:30.500 INFO:tasks.workunit.client.1.smithi070.stdout:5/889: getdents d17/d24/d84/dab 0 2024-09-10T04:05:30.503 INFO:tasks.workunit.client.1.smithi070.stdout:3/846: symlink db/dcc/l123 0 2024-09-10T04:05:30.504 INFO:tasks.workunit.client.1.smithi070.stdout:3/847: chown db/d1c/d25/d28/dd3/de5/d10f/d94 115022343 1 2024-09-10T04:05:30.505 INFO:tasks.workunit.client.0.smithi046.stdout:9/797: mknod d4/d66/cfc 0 2024-09-10T04:05:30.506 INFO:tasks.workunit.client.0.smithi046.stdout:9/798: write d4/d3c/d4d/f76 [2320807,88141] 0 2024-09-10T04:05:30.508 INFO:tasks.workunit.client.1.smithi070.stdout:4/814: dread de/d13/fec [0,4194304] 0 2024-09-10T04:05:30.509 INFO:tasks.workunit.client.1.smithi070.stdout:4/815: write de/d13/f10b [1032598,35566] 0 2024-09-10T04:05:30.509 INFO:tasks.workunit.client.1.smithi070.stdout:6/839: creat d0/d3/d18/d62/d70/f105 x:0 0 0 2024-09-10T04:05:30.512 INFO:tasks.workunit.client.1.smithi070.stdout:7/910: unlink d2/d7/db/f92 0 2024-09-10T04:05:30.513 INFO:tasks.workunit.client.0.smithi046.stdout:8/797: dread d1/d29/d12/f9f [0,4194304] 0 2024-09-10T04:05:30.514 INFO:tasks.workunit.client.1.smithi070.stdout:8/908: truncate d4/d39/dfc/f31 1811426 0 2024-09-10T04:05:30.515 INFO:tasks.workunit.client.0.smithi046.stdout:6/686: mkdir da/db/d63/de1 0 2024-09-10T04:05:30.518 INFO:tasks.workunit.client.0.smithi046.stdout:3/754: mknod d12/d13/d14/d1e/d2d/d6f/d8f/c107 0 2024-09-10T04:05:30.518 INFO:tasks.workunit.client.0.smithi046.stdout:3/755: stat d12/d5d/d71/d49/d5c/d9f 0 2024-09-10T04:05:30.519 INFO:tasks.workunit.client.1.smithi070.stdout:3/848: dread - db/d83/d92/dc7/f120 zero size 2024-09-10T04:05:30.521 INFO:tasks.workunit.client.1.smithi070.stdout:9/883: link d9/d7a/lc8 d9/d1c/d34/d63/l11e 0 2024-09-10T04:05:30.523 INFO:tasks.workunit.client.0.smithi046.stdout:4/937: symlink d3/d12/d17/d29/d58/db2/d5f/l141 0 2024-09-10T04:05:30.524 INFO:tasks.workunit.client.0.smithi046.stdout:4/938: chown d3/d12/d17/d22/f4b 1 1 2024-09-10T04:05:30.525 INFO:tasks.workunit.client.0.smithi046.stdout:3/756: dwrite d12/d13/d37/d51/db5/d4b/db7/dc0/fd6 [0,4194304] 0 2024-09-10T04:05:30.528 INFO:tasks.workunit.client.0.smithi046.stdout:3/757: chown d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fb4 108970 1 2024-09-10T04:05:30.530 INFO:tasks.workunit.client.0.smithi046.stdout:3/758: read d12/d13/d14/f24 [320639,113667] 0 2024-09-10T04:05:30.531 INFO:tasks.workunit.client.1.smithi070.stdout:7/911: creat d2/d7/d9/d18/d3c/d52/d10e/f142 x:0 0 0 2024-09-10T04:05:30.532 INFO:tasks.workunit.client.0.smithi046.stdout:3/759: fdatasync d12/d13/d14/f36 0 2024-09-10T04:05:30.534 INFO:tasks.workunit.client.0.smithi046.stdout:8/798: creat d1/d29/d17/f108 x:0 0 0 2024-09-10T04:05:30.535 INFO:tasks.workunit.client.0.smithi046.stdout:9/799: dwrite d4/d7/d1f/d54/f6e [0,4194304] 0 2024-09-10T04:05:30.537 INFO:tasks.workunit.client.1.smithi070.stdout:3/849: creat db/d1c/d25/d28/dd3/de5/d10f/db6/f124 x:0 0 0 2024-09-10T04:05:30.539 INFO:tasks.workunit.client.0.smithi046.stdout:5/928: link d7/d11/d14/d2f/d2b/d92/cd1 d7/d11/d14/d2f/deb/da5/dcc/c141 0 2024-09-10T04:05:30.541 INFO:tasks.workunit.client.0.smithi046.stdout:5/929: chown d7/d11/d14/d2f/deb/da5/dcc/cef 84776678 1 2024-09-10T04:05:30.541 INFO:tasks.workunit.client.0.smithi046.stdout:1/600: creat d3/dc/d34/fc6 x:0 0 0 2024-09-10T04:05:30.543 INFO:tasks.workunit.client.1.smithi070.stdout:7/912: fsync d2/d40/d6c/db2/fe3 0 2024-09-10T04:05:30.544 INFO:tasks.workunit.client.1.smithi070.stdout:7/913: chown d2/d3/dc/d5d/d2d/c4d 522 1 2024-09-10T04:05:30.549 INFO:tasks.workunit.client.0.smithi046.stdout:2/771: link d2/d6/d23/d5d/d77/d7b/d85/d8d/fc5 d2/d6/d23/d5d/d77/d7b/f111 0 2024-09-10T04:05:30.549 INFO:tasks.workunit.client.0.smithi046.stdout:6/687: write da/db/d11/d13/d1e/f3e [202158,99323] 0 2024-09-10T04:05:30.551 INFO:tasks.workunit.client.1.smithi070.stdout:5/890: link d17/d19/d6a/d54/d57/fb0 d17/d6f/d32/f131 0 2024-09-10T04:05:30.553 INFO:tasks.workunit.client.0.smithi046.stdout:6/688: dread da/f24 [0,4194304] 0 2024-09-10T04:05:30.554 INFO:tasks.workunit.client.1.smithi070.stdout:9/884: creat d9/d1c/d34/d8b/d11d/dc4/dda/de0/dd5/df5/f11f x:0 0 0 2024-09-10T04:05:30.555 INFO:tasks.workunit.client.1.smithi070.stdout:6/840: write d0/d3/d18/d2a/f4f [3998607,93176] 0 2024-09-10T04:05:30.555 INFO:tasks.workunit.client.1.smithi070.stdout:3/850: rmdir db/d1c/d25/d28/dd3/de5/d10f/dd4 39 2024-09-10T04:05:30.556 INFO:tasks.workunit.client.1.smithi070.stdout:9/885: fdatasync d9/d7a/d74/fc6 0 2024-09-10T04:05:30.558 INFO:tasks.workunit.client.1.smithi070.stdout:5/891: dwrite d17/d24/d77/ffa [0,4194304] 0 2024-09-10T04:05:30.558 INFO:tasks.workunit.client.1.smithi070.stdout:4/816: creat de/d13/d18/d23/f116 x:0 0 0 2024-09-10T04:05:30.558 INFO:tasks.workunit.client.1.smithi070.stdout:3/851: chown db/d1c/d53/fa2 1436131343 1 2024-09-10T04:05:30.558 INFO:tasks.workunit.client.0.smithi046.stdout:7/870: creat d5/d6/db/f134 x:0 0 0 2024-09-10T04:05:30.559 INFO:tasks.workunit.client.1.smithi070.stdout:5/892: readlink d17/d6f/l3e 0 2024-09-10T04:05:30.559 INFO:tasks.workunit.client.1.smithi070.stdout:9/886: chown d9/d1c/d34/d8b/d11d/dc4/f116 1266029 1 2024-09-10T04:05:30.559 INFO:tasks.workunit.client.1.smithi070.stdout:4/817: chown de/d13/d43/d88/ff9 349 1 2024-09-10T04:05:30.561 INFO:tasks.workunit.client.1.smithi070.stdout:3/852: dread - db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/de1/dfe/f102 zero size 2024-09-10T04:05:30.561 INFO:tasks.workunit.client.0.smithi046.stdout:8/799: creat d1/d29/d17/d1f/d46/d67/de4/f109 x:0 0 0 2024-09-10T04:05:30.562 INFO:tasks.workunit.client.0.smithi046.stdout:8/800: chown d1/d29/d17/d1f/f54 1 1 2024-09-10T04:05:30.564 INFO:tasks.workunit.client.0.smithi046.stdout:9/800: creat d4/d3c/d4d/d5c/d59/ffd x:0 0 0 2024-09-10T04:05:30.566 INFO:tasks.workunit.client.1.smithi070.stdout:4/818: dwrite de/d13/d5b/d7b/dd5/d85/d10e/f6d [0,4194304] 0 2024-09-10T04:05:30.568 INFO:tasks.workunit.client.0.smithi046.stdout:1/601: creat d3/dc/d21/d3a/fc7 x:0 0 0 2024-09-10T04:05:30.570 INFO:tasks.workunit.client.0.smithi046.stdout:2/772: unlink l0 0 2024-09-10T04:05:30.571 INFO:tasks.workunit.client.1.smithi070.stdout:6/841: mkdir d0/d3/d8c/d9a/d106 0 2024-09-10T04:05:30.573 INFO:tasks.workunit.client.1.smithi070.stdout:5/893: creat d17/d6f/d32/d90/da5/d120/f132 x:0 0 0 2024-09-10T04:05:30.576 INFO:tasks.workunit.client.1.smithi070.stdout:9/887: symlink d9/d1c/d34/d8b/d11d/dc4/dda/l120 0 2024-09-10T04:05:30.581 INFO:tasks.workunit.client.0.smithi046.stdout:3/760: fdatasync d12/d13/d14/d1e/d2d/f4f 0 2024-09-10T04:05:30.582 INFO:tasks.workunit.client.0.smithi046.stdout:6/689: dread da/fe [0,4194304] 0 2024-09-10T04:05:30.582 INFO:tasks.workunit.client.0.smithi046.stdout:6/690: dread - da/d71/fc7 zero size 2024-09-10T04:05:30.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:30 smithi070 ceph-mon[35495]: mgrmap e20: smithi046.zvckjg(active, since 7m), standbys: smithi070.tezbzn 2024-09-10T04:05:30.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:30 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:30.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:30 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:30.585 INFO:tasks.workunit.client.0.smithi046.stdout:8/801: mknod d1/d29/d19/db7/c10a 0 2024-09-10T04:05:30.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:30 smithi046 ceph-mon[28328]: mgrmap e20: smithi046.zvckjg(active, since 7m), standbys: smithi070.tezbzn 2024-09-10T04:05:30.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:30 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:30.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:30 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:30.586 INFO:tasks.workunit.client.0.smithi046.stdout:8/802: readlink d1/d29/d19/l3d 0 2024-09-10T04:05:30.588 INFO:tasks.workunit.client.1.smithi070.stdout:8/909: getdents d4/d16/d73 0 2024-09-10T04:05:30.588 INFO:tasks.workunit.client.1.smithi070.stdout:7/914: dwrite d2/dc8/d119/f9a [0,4194304] 0 2024-09-10T04:05:30.588 INFO:tasks.workunit.client.0.smithi046.stdout:5/930: dwrite d7/f19 [0,4194304] 0 2024-09-10T04:05:30.592 INFO:tasks.workunit.client.0.smithi046.stdout:1/602: symlink d3/dc/d21/lc8 0 2024-09-10T04:05:30.593 INFO:tasks.workunit.client.1.smithi070.stdout:6/842: mknod d0/d3/d18/d1b/d17/d54/dc4/dd2/c107 0 2024-09-10T04:05:30.594 INFO:tasks.workunit.client.0.smithi046.stdout:2/773: mknod d2/d6/d23/d42/db4/dca/c112 0 2024-09-10T04:05:30.601 INFO:tasks.workunit.client.1.smithi070.stdout:7/915: rename d2/d3/dc/d5d/d42/fa0 to d2/d3/dc/d13/dbd/f143 0 2024-09-10T04:05:30.602 INFO:tasks.workunit.client.0.smithi046.stdout:4/939: link d3/d12/d14/c8b d3/c142 0 2024-09-10T04:05:30.604 INFO:tasks.workunit.client.1.smithi070.stdout:5/894: link d17/d19/d35/f5f d17/d19/d6a/d54/d57/da8/f133 0 2024-09-10T04:05:30.605 INFO:tasks.workunit.client.1.smithi070.stdout:4/819: dwrite de/d13/d5b/d7b/dd5/d85/d10e/f5a [0,4194304] 0 2024-09-10T04:05:30.606 INFO:tasks.workunit.client.0.smithi046.stdout:8/803: mkdir d1/d29/d17/d1f/d2a/d10b 0 2024-09-10T04:05:30.607 INFO:tasks.workunit.client.0.smithi046.stdout:5/931: rmdir d7/d11/d14/d2f/d2b 39 2024-09-10T04:05:30.608 INFO:tasks.workunit.client.1.smithi070.stdout:9/888: dread d9/d1c/dfd/fa3 [0,4194304] 0 2024-09-10T04:05:30.611 INFO:tasks.workunit.client.1.smithi070.stdout:7/916: mknod d2/d7/d9/d18/d23/c144 0 2024-09-10T04:05:30.614 INFO:tasks.workunit.client.1.smithi070.stdout:6/843: dread d0/d3/f51 [0,4194304] 0 2024-09-10T04:05:30.616 INFO:tasks.workunit.client.0.smithi046.stdout:5/932: dwrite d7/d11/d14/d2f/f48 [0,4194304] 0 2024-09-10T04:05:30.617 INFO:tasks.workunit.client.1.smithi070.stdout:9/889: dwrite d9/d1c/d34/d8b/d11d/d3c/d92/ff6 [4194304,4194304] 0 2024-09-10T04:05:30.624 INFO:tasks.workunit.client.1.smithi070.stdout:7/917: mknod d2/d40/d6c/db2/db7/d104/c145 0 2024-09-10T04:05:30.627 INFO:tasks.workunit.client.0.smithi046.stdout:8/804: mkdir d1/d29/d19/d36/d4c/dd3/d10c 0 2024-09-10T04:05:30.628 INFO:tasks.workunit.client.1.smithi070.stdout:7/918: stat d2/d7/d9/d18/d3c/fae 0 2024-09-10T04:05:30.628 INFO:tasks.workunit.client.1.smithi070.stdout:4/820: unlink de/d13/d5b/da0/ld6 0 2024-09-10T04:05:30.628 INFO:tasks.workunit.client.1.smithi070.stdout:4/821: chown de/d13/d18/d5c/cd3 145858 1 2024-09-10T04:05:30.628 INFO:tasks.workunit.client.1.smithi070.stdout:7/919: read d2/d7/d9/d18/fcc [3975370,84396] 0 2024-09-10T04:05:30.628 INFO:tasks.workunit.client.1.smithi070.stdout:6/844: mknod d0/d3/d8c/dab/c108 0 2024-09-10T04:05:30.631 INFO:tasks.workunit.client.1.smithi070.stdout:7/920: read d2/d7/ff4 [820566,72609] 0 2024-09-10T04:05:30.632 INFO:tasks.workunit.client.1.smithi070.stdout:9/890: dread d9/d59/d2a/f101 [0,4194304] 0 2024-09-10T04:05:30.632 INFO:tasks.workunit.client.1.smithi070.stdout:7/921: chown d2/d40/d6c/db2/db7/d104/f111 61 1 2024-09-10T04:05:30.633 INFO:tasks.workunit.client.0.smithi046.stdout:5/933: unlink d7/d11/d3f/d5a/f72 0 2024-09-10T04:05:30.633 INFO:tasks.workunit.client.1.smithi070.stdout:5/895: write d17/d19/fc3 [4571161,84779] 0 2024-09-10T04:05:30.633 INFO:tasks.workunit.client.0.smithi046.stdout:5/934: readlink d7/d11/d14/d2f/deb/da5/dcc/lb4 0 2024-09-10T04:05:30.634 INFO:tasks.workunit.client.0.smithi046.stdout:3/761: link f4 d12/d13/d14/f108 0 2024-09-10T04:05:30.636 INFO:tasks.workunit.client.0.smithi046.stdout:4/940: write d3/d12/d17/d29/d58/db2/d5f/d85/da8/fea [1019832,15498] 0 2024-09-10T04:05:30.637 INFO:tasks.workunit.client.0.smithi046.stdout:2/774: chown d2/d6/d23/d5d/d77/d7b/d85/d8d/l98 45127615 1 2024-09-10T04:05:30.639 INFO:tasks.workunit.client.1.smithi070.stdout:4/822: mknod de/d13/d43/c117 0 2024-09-10T04:05:30.639 INFO:tasks.workunit.client.0.smithi046.stdout:2/775: write d2/d6/d23/d55/f7e [194445,19630] 0 2024-09-10T04:05:30.640 INFO:tasks.workunit.client.0.smithi046.stdout:3/762: dread d12/d13/d14/d1e/d2d/f2e [0,4194304] 0 2024-09-10T04:05:30.640 INFO:tasks.workunit.client.1.smithi070.stdout:7/922: dwrite d2/d3/dc/d5d/d41/f50 [0,4194304] 0 2024-09-10T04:05:30.640 INFO:tasks.workunit.client.0.smithi046.stdout:1/603: write d3/d27/f33 [760260,14117] 0 2024-09-10T04:05:30.641 INFO:tasks.workunit.client.1.smithi070.stdout:4/823: readlink de/d13/d5b/d7b/dd5/d85/d10e/d48/l83 0 2024-09-10T04:05:30.642 INFO:tasks.workunit.client.0.smithi046.stdout:7/871: getdents d5/d6/d32/dab/db6 0 2024-09-10T04:05:30.642 INFO:tasks.workunit.client.1.smithi070.stdout:8/910: write d4/d16/dcc/d12e/d11b/d1a/d60/d7a/fcb [717166,45262] 0 2024-09-10T04:05:30.643 INFO:tasks.workunit.client.1.smithi070.stdout:4/824: chown de/d13/d18/d23/f116 93131522 1 2024-09-10T04:05:30.643 INFO:tasks.workunit.client.1.smithi070.stdout:7/923: chown d2/d40/d6c/db2/db7/f13e 12 1 2024-09-10T04:05:30.643 INFO:tasks.workunit.client.0.smithi046.stdout:2/776: truncate d2/d6/d23/d42/db4/dca/fe8 309010 0 2024-09-10T04:05:30.644 INFO:tasks.workunit.client.0.smithi046.stdout:7/872: chown d5/d6/d32/dab 837 1 2024-09-10T04:05:30.644 INFO:tasks.workunit.client.0.smithi046.stdout:2/777: fdatasync d2/f25 0 2024-09-10T04:05:30.646 INFO:tasks.workunit.client.1.smithi070.stdout:5/896: dread f16 [0,4194304] 0 2024-09-10T04:05:30.647 INFO:tasks.workunit.client.0.smithi046.stdout:8/805: mkdir d1/d29/d17/d1f/d2a/df6/d10d 0 2024-09-10T04:05:30.650 INFO:tasks.workunit.client.1.smithi070.stdout:9/891: mkdir d9/d1c/d34/d8b/d11d/dc4/dda/d121 0 2024-09-10T04:05:30.653 INFO:tasks.workunit.client.0.smithi046.stdout:6/691: getdents da/db/d11/d9f 0 2024-09-10T04:05:30.654 INFO:tasks.workunit.client.1.smithi070.stdout:6/845: mknod d0/d3/d1f/de8/c109 0 2024-09-10T04:05:30.655 INFO:tasks.workunit.client.1.smithi070.stdout:8/911: read - d4/d16/dcc/d12e/da4/f10c zero size 2024-09-10T04:05:30.657 INFO:tasks.workunit.client.1.smithi070.stdout:7/924: symlink d2/dc5/l146 0 2024-09-10T04:05:30.657 INFO:tasks.workunit.client.0.smithi046.stdout:2/778: chown d2/d86 10047 1 2024-09-10T04:05:30.659 INFO:tasks.workunit.client.1.smithi070.stdout:7/925: fdatasync d2/d7/d9/d18/f13a 0 2024-09-10T04:05:30.659 INFO:tasks.workunit.client.0.smithi046.stdout:2/779: chown d2/d6/d23/d5d/d77/d7b/dd8/le9 48 1 2024-09-10T04:05:30.660 INFO:tasks.workunit.client.0.smithi046.stdout:8/806: rename d1/d29/d17/d1f/d46/d67/l7a to d1/d29/d17/d1f/d2a/l10e 0 2024-09-10T04:05:30.661 INFO:tasks.workunit.client.0.smithi046.stdout:8/807: read d1/f2e [4106073,102855] 0 2024-09-10T04:05:30.662 INFO:tasks.workunit.client.1.smithi070.stdout:9/892: rename d9/d1c/d34/d8b/d11d/d1b/cc9 to d9/d1c/d34/d8b/d11d/dfb/c122 0 2024-09-10T04:05:30.664 INFO:tasks.workunit.client.1.smithi070.stdout:6/846: dwrite d0/d3/d18/d2a/fee [0,4194304] 0 2024-09-10T04:05:30.664 INFO:tasks.workunit.client.0.smithi046.stdout:6/692: fsync da/db/d11/d2b/d2f/f65 0 2024-09-10T04:05:30.669 INFO:tasks.workunit.client.0.smithi046.stdout:6/693: readlink da/db/d11/d13/d1e/d23/d86/dbf/la6 0 2024-09-10T04:05:30.669 INFO:tasks.workunit.client.1.smithi070.stdout:5/897: dread d17/d19/d6a/f106 [0,4194304] 0 2024-09-10T04:05:30.671 INFO:tasks.workunit.client.0.smithi046.stdout:4/941: dread d3/d12/d17/f1b [0,4194304] 0 2024-09-10T04:05:30.672 INFO:tasks.workunit.client.0.smithi046.stdout:4/942: readlink d3/d12/d17/d22/d51/d5c/lf6 0 2024-09-10T04:05:30.673 INFO:tasks.workunit.client.0.smithi046.stdout:5/935: rename d7/d11/d14/d8e/de2 to d7/d11/d14/d2f/d2b/d92/d142 0 2024-09-10T04:05:30.674 INFO:tasks.workunit.client.0.smithi046.stdout:4/943: stat d3/d12/d17/d29/ddd 0 2024-09-10T04:05:30.674 INFO:tasks.workunit.client.1.smithi070.stdout:5/898: dwrite d17/d19/d6a/d54/de1/dfe/fa6 [0,4194304] 0 2024-09-10T04:05:30.675 INFO:tasks.workunit.client.0.smithi046.stdout:4/944: readlink d3/d12/d17/de1/ddf/leb 0 2024-09-10T04:05:30.675 INFO:tasks.workunit.client.1.smithi070.stdout:8/912: creat d4/d16/dcc/d122/d126/f136 x:0 0 0 2024-09-10T04:05:30.681 INFO:tasks.workunit.client.1.smithi070.stdout:4/825: dread de/d13/d5b/d7b/dd5/d85/d10e/d48/f7e [0,4194304] 0 2024-09-10T04:05:30.682 INFO:tasks.workunit.client.1.smithi070.stdout:6/847: rename d0/d3/d8c/d9a/fac to d0/d3/da4/f10a 0 2024-09-10T04:05:30.685 INFO:tasks.workunit.client.0.smithi046.stdout:8/808: dread d1/f39 [0,4194304] 0 2024-09-10T04:05:30.688 INFO:tasks.workunit.client.1.smithi070.stdout:5/899: mknod d17/d19/d6a/d54/d57/df2/c134 0 2024-09-10T04:05:30.690 INFO:tasks.workunit.client.1.smithi070.stdout:8/913: creat d4/d39/dfc/d43/d67/dec/d106/f137 x:0 0 0 2024-09-10T04:05:30.690 INFO:tasks.workunit.client.1.smithi070.stdout:5/900: truncate d17/d19/d6a/d54/de1/dfe/f8d 786156 0 2024-09-10T04:05:30.691 INFO:tasks.workunit.client.0.smithi046.stdout:2/780: creat d2/d6/d9d/db6/dbe/df2/f113 x:0 0 0 2024-09-10T04:05:30.692 INFO:tasks.workunit.client.0.smithi046.stdout:2/781: chown d2/d6/d9d/la2 37191 1 2024-09-10T04:05:30.694 INFO:tasks.workunit.client.1.smithi070.stdout:7/926: creat d2/d7/db/f147 x:0 0 0 2024-09-10T04:05:30.694 INFO:tasks.workunit.client.0.smithi046.stdout:3/763: getdents d12/d13/d37/d51/db5/d50/d7a/dc1 0 2024-09-10T04:05:30.696 INFO:tasks.workunit.client.1.smithi070.stdout:5/901: fsync d17/d24/d77/f127 0 2024-09-10T04:05:30.697 INFO:tasks.workunit.client.1.smithi070.stdout:9/893: link d9/d1c/d34/d8b/d11d/d1b/fba d9/d1c/d34/d8b/d11d/d1b/f123 0 2024-09-10T04:05:30.698 INFO:tasks.workunit.client.1.smithi070.stdout:9/894: write d9/d1c/d34/d8b/d11d/dc4/dda/de0/d90/df0/df3/df4/f102 [165991,43993] 0 2024-09-10T04:05:30.705 INFO:tasks.workunit.client.1.smithi070.stdout:7/927: symlink d2/d7/d9/d18/d3c/l148 0 2024-09-10T04:05:30.707 INFO:tasks.workunit.client.1.smithi070.stdout:8/914: write d4/d16/f36 [383987,41408] 0 2024-09-10T04:05:30.708 INFO:tasks.workunit.client.1.smithi070.stdout:5/902: symlink d17/d6f/l135 0 2024-09-10T04:05:30.709 INFO:tasks.workunit.client.1.smithi070.stdout:6/848: symlink d0/d3/da4/dc0/dea/l10b 0 2024-09-10T04:05:30.710 INFO:tasks.workunit.client.1.smithi070.stdout:9/895: creat d9/d1c/d34/d8b/d11d/dc4/dda/de0/d90/f124 x:0 0 0 2024-09-10T04:05:30.710 INFO:tasks.workunit.client.1.smithi070.stdout:4/826: link de/d13/d5b/d7b/dd5/d85/d10e/d48/l83 de/d13/d5b/d7b/dea/l118 0 2024-09-10T04:05:30.711 INFO:tasks.workunit.client.1.smithi070.stdout:7/928: rename d2/d3/d63/l67 to d2/dc5/d122/l149 0 2024-09-10T04:05:30.712 INFO:tasks.workunit.client.1.smithi070.stdout:8/915: mknod d4/d16/dcc/d12e/d11b/d1a/d60/c138 0 2024-09-10T04:05:30.714 INFO:tasks.workunit.client.1.smithi070.stdout:8/916: fsync d4/d16/dcc/d12e/d11b/d19/f117 0 2024-09-10T04:05:30.714 INFO:tasks.workunit.client.1.smithi070.stdout:5/903: rename d17/d19/d6a/d54/de1/dfe/faa to d17/d19/d12d/f136 0 2024-09-10T04:05:30.714 INFO:tasks.workunit.client.1.smithi070.stdout:7/929: write d2/d7/d9/d18/d3c/d52/d103/f11b [71717,49134] 0 2024-09-10T04:05:30.715 INFO:tasks.workunit.client.1.smithi070.stdout:4/827: creat de/d13/d5b/d7b/dd5/d85/d10e/d48/f119 x:0 0 0 2024-09-10T04:05:30.715 INFO:tasks.workunit.client.0.smithi046.stdout:1/604: link d3/d27/d5d/d60/f8a d3/d62/fc9 0 2024-09-10T04:05:30.717 INFO:tasks.workunit.client.1.smithi070.stdout:4/828: write de/d13/d18/d26/f2f [382173,13171] 0 2024-09-10T04:05:30.717 INFO:tasks.workunit.client.1.smithi070.stdout:8/917: rmdir d4/d16/dcc/d122 39 2024-09-10T04:05:30.718 INFO:tasks.workunit.client.0.smithi046.stdout:2/782: dread d2/d6/d15/f103 [0,4194304] 0 2024-09-10T04:05:30.718 INFO:tasks.workunit.client.0.smithi046.stdout:4/945: creat d3/d12/d17/d29/d58/db2/d5f/d85/d114/f143 x:0 0 0 2024-09-10T04:05:30.718 INFO:tasks.workunit.client.0.smithi046.stdout:7/873: getdents d5/d9/d12/d72 0 2024-09-10T04:05:30.719 INFO:tasks.workunit.client.1.smithi070.stdout:4/829: write de/d13/d43/d88/ff9 [279879,63565] 0 2024-09-10T04:05:30.720 INFO:tasks.workunit.client.1.smithi070.stdout:9/896: creat d9/d1c/d34/d8b/d11d/dc4/dda/de0/f125 x:0 0 0 2024-09-10T04:05:30.722 INFO:tasks.workunit.client.0.smithi046.stdout:8/809: truncate d1/d29/d12/f57 499283 0 2024-09-10T04:05:30.725 INFO:tasks.workunit.client.0.smithi046.stdout:8/810: fsync d1/d29/d17/d1f/d46/d6a/d82/fd6 0 2024-09-10T04:05:30.726 INFO:tasks.workunit.client.1.smithi070.stdout:5/904: unlink d17/d24/d84/l103 0 2024-09-10T04:05:30.726 INFO:tasks.workunit.client.1.smithi070.stdout:5/905: write d17/d19/fc3 [2467066,67021] 0 2024-09-10T04:05:30.732 INFO:tasks.workunit.client.0.smithi046.stdout:6/694: getdents da/db/d11/d13 0 2024-09-10T04:05:30.732 INFO:tasks.workunit.client.0.smithi046.stdout:4/946: dread d3/d12/f73 [0,4194304] 0 2024-09-10T04:05:30.734 INFO:tasks.workunit.client.0.smithi046.stdout:4/947: write d3/d12/d14/f2b [2862574,66106] 0 2024-09-10T04:05:30.734 INFO:tasks.workunit.client.0.smithi046.stdout:7/874: mkdir d5/d33/d135 0 2024-09-10T04:05:30.736 INFO:tasks.workunit.client.1.smithi070.stdout:7/930: mknod d2/d40/d6c/d112/c14a 0 2024-09-10T04:05:30.737 INFO:tasks.workunit.client.1.smithi070.stdout:8/918: rename d4/d16/dcc/d12e/d11b/d1a/d60 to d4/d16/d73/d139 0 2024-09-10T04:05:30.738 INFO:tasks.workunit.client.1.smithi070.stdout:7/931: read d2/d3/dc/d13/fc7 [3766899,121575] 0 2024-09-10T04:05:30.738 INFO:tasks.workunit.client.0.smithi046.stdout:2/783: write d2/d6/d23/d42/db4/dca/fe8 [356053,29118] 0 2024-09-10T04:05:30.741 INFO:tasks.workunit.client.1.smithi070.stdout:6/849: truncate d0/d3/d18/d1b/d33/f102 2282136 0 2024-09-10T04:05:30.741 INFO:tasks.workunit.client.1.smithi070.stdout:5/906: mkdir d17/d19/d12d/d137 0 2024-09-10T04:05:30.742 INFO:tasks.workunit.client.0.smithi046.stdout:1/605: write d3/da1/d1f/f35 [3119969,130504] 0 2024-09-10T04:05:30.743 INFO:tasks.workunit.client.1.smithi070.stdout:4/830: symlink de/d13/d18/d23/d47/d59/d79/l11a 0 2024-09-10T04:05:30.750 INFO:tasks.workunit.client.0.smithi046.stdout:3/764: dwrite d12/d13/d14/d1e/d2d/d39/d67/f6b [0,4194304] 0 2024-09-10T04:05:30.751 INFO:tasks.workunit.client.1.smithi070.stdout:7/932: mkdir d2/d3/dc/d5d/d41/dd6/de6/d14b 0 2024-09-10T04:05:30.754 INFO:tasks.workunit.client.0.smithi046.stdout:7/875: symlink d5/d6/db/dd/d24/d76/d9c/l136 0 2024-09-10T04:05:30.757 INFO:tasks.workunit.client.0.smithi046.stdout:1/606: mkdir d3/d62/dca 0 2024-09-10T04:05:30.758 INFO:tasks.workunit.client.0.smithi046.stdout:7/876: read d5/d6/db/dd/f2f [2670582,98756] 0 2024-09-10T04:05:30.758 INFO:tasks.workunit.client.1.smithi070.stdout:9/897: creat d9/d59/f126 x:0 0 0 2024-09-10T04:05:30.759 INFO:tasks.workunit.client.0.smithi046.stdout:1/607: dread d3/fae [0,4194304] 0 2024-09-10T04:05:30.761 INFO:tasks.workunit.client.0.smithi046.stdout:8/811: rename d1/d29/d17/lcc to d1/d29/d19/d36/d4c/dd3/d10c/l10f 0 2024-09-10T04:05:30.763 INFO:tasks.workunit.client.1.smithi070.stdout:5/907: dwrite f16 [4194304,4194304] 0 2024-09-10T04:05:30.765 INFO:tasks.workunit.client.1.smithi070.stdout:7/933: dread d2/d3/dc/d5d/d2d/fba [0,4194304] 0 2024-09-10T04:05:30.766 INFO:tasks.workunit.client.0.smithi046.stdout:7/877: dread d5/d6/d32/f95 [4194304,4194304] 0 2024-09-10T04:05:30.768 INFO:tasks.workunit.client.0.smithi046.stdout:3/765: creat d12/d13/d14/d1e/d2d/d39/d67/da0/f109 x:0 0 0 2024-09-10T04:05:30.768 INFO:tasks.workunit.client.0.smithi046.stdout:4/948: dread d3/d12/d17/d29/f99 [0,4194304] 0 2024-09-10T04:05:30.768 INFO:tasks.workunit.client.0.smithi046.stdout:7/878: chown d5/d6/fd2 754 1 2024-09-10T04:05:30.769 INFO:tasks.workunit.client.0.smithi046.stdout:2/784: creat d2/d86/d9c/dbc/ded/f114 x:0 0 0 2024-09-10T04:05:30.770 INFO:tasks.workunit.client.0.smithi046.stdout:7/879: write d5/fbf [123961,103480] 0 2024-09-10T04:05:30.772 INFO:tasks.workunit.client.0.smithi046.stdout:2/785: fdatasync d2/d6/d23/d5d/d5e/f84 0 2024-09-10T04:05:30.774 INFO:tasks.workunit.client.1.smithi070.stdout:6/850: dwrite d0/d3/da4/dc0/dea/f5e [0,4194304] 0 2024-09-10T04:05:30.776 INFO:tasks.workunit.client.1.smithi070.stdout:8/919: mkdir d4/d16/dcc/d12e/d135/d13a 0 2024-09-10T04:05:30.781 INFO:tasks.workunit.client.0.smithi046.stdout:1/608: mkdir d3/d9b/dcb 0 2024-09-10T04:05:30.784 INFO:tasks.workunit.client.0.smithi046.stdout:8/812: creat d1/d29/d17/d1f/d2a/f110 x:0 0 0 2024-09-10T04:05:30.786 INFO:tasks.workunit.client.1.smithi070.stdout:6/851: rmdir d0/d3/d18/d1b 39 2024-09-10T04:05:30.791 INFO:tasks.workunit.client.0.smithi046.stdout:6/695: dwrite da/db/d11/d13/d1e/d69/f87 [0,4194304] 0 2024-09-10T04:05:30.791 INFO:tasks.workunit.client.1.smithi070.stdout:8/920: creat d4/d16/dcc/d12e/d11b/d19/d62/dd1/f13b x:0 0 0 2024-09-10T04:05:30.794 INFO:tasks.workunit.client.0.smithi046.stdout:4/949: dread d3/d12/d17/d29/fbb [0,4194304] 0 2024-09-10T04:05:30.798 INFO:tasks.workunit.client.1.smithi070.stdout:7/934: rename d2/d40/d6c/d112 to d2/dc8/d119/d14c 0 2024-09-10T04:05:30.799 INFO:tasks.workunit.client.0.smithi046.stdout:2/786: symlink d2/d6/d23/d5d/d77/d7b/d85/l115 0 2024-09-10T04:05:30.800 INFO:tasks.workunit.client.1.smithi070.stdout:4/831: dwrite de/d13/d5b/d7b/dd5/fa1 [0,4194304] 0 2024-09-10T04:05:30.800 INFO:tasks.workunit.client.1.smithi070.stdout:1/762: sync 2024-09-10T04:05:30.802 INFO:tasks.workunit.client.1.smithi070.stdout:9/898: dwrite d9/d7a/f3a [0,4194304] 0 2024-09-10T04:05:30.804 INFO:tasks.workunit.client.0.smithi046.stdout:6/696: unlink da/db/d11/d13/d1e/laf 0 2024-09-10T04:05:30.804 INFO:tasks.workunit.client.0.smithi046.stdout:8/813: dread d1/d29/d17/d1f/d46/d67/dc1/fc6 [0,4194304] 0 2024-09-10T04:05:30.806 INFO:tasks.workunit.client.1.smithi070.stdout:6/852: mkdir d0/d3/d18/d1b/d17/d53/d10c 0 2024-09-10T04:05:30.808 INFO:tasks.workunit.client.1.smithi070.stdout:6/853: chown d0/d3 6326354 1 2024-09-10T04:05:30.808 INFO:tasks.workunit.client.1.smithi070.stdout:9/899: read d9/f22 [2318877,14089] 0 2024-09-10T04:05:30.809 INFO:tasks.workunit.client.1.smithi070.stdout:4/832: truncate de/d13/d5b/d7b/dd5/d85/d10e/d48/feb 975998 0 2024-09-10T04:05:30.816 INFO:tasks.workunit.client.1.smithi070.stdout:1/763: rmdir d3/d10/d1d/d79 39 2024-09-10T04:05:30.817 INFO:tasks.workunit.client.1.smithi070.stdout:1/764: chown d3/d8/d27/d7a/d94/da5/lbb 1 1 2024-09-10T04:05:30.819 INFO:tasks.workunit.client.0.smithi046.stdout:4/950: mknod d3/d12/d17/d29/d58/db2/d5f/d85/c144 0 2024-09-10T04:05:30.822 INFO:tasks.workunit.client.0.smithi046.stdout:8/814: fsync d1/d29/d17/d1f/d46/f4b 0 2024-09-10T04:05:30.824 INFO:tasks.workunit.client.1.smithi070.stdout:8/921: link d4/d39/dfc/d43/l4c d4/d16/dcc/d12e/d11b/d19/d62/dd1/l13c 0 2024-09-10T04:05:30.828 INFO:tasks.workunit.client.0.smithi046.stdout:8/815: dwrite d1/d29/d17/f106 [0,4194304] 0 2024-09-10T04:05:30.828 INFO:tasks.workunit.client.0.smithi046.stdout:9/801: sync 2024-09-10T04:05:30.829 INFO:tasks.workunit.client.0.smithi046.stdout:0/646: sync 2024-09-10T04:05:30.829 INFO:tasks.workunit.client.0.smithi046.stdout:7/880: rename d5/d6/db/dd/d24/d76/d9c/f114 to d5/d9/f137 0 2024-09-10T04:05:30.831 INFO:tasks.workunit.client.0.smithi046.stdout:7/881: stat d5/d6/d55/d7d/f100 0 2024-09-10T04:05:30.833 INFO:tasks.workunit.client.0.smithi046.stdout:4/951: mkdir d3/d12/d17/d29/d58/db2/d5f/d85/d114/d145 0 2024-09-10T04:05:30.836 INFO:tasks.workunit.client.1.smithi070.stdout:9/900: symlink d9/l127 0 2024-09-10T04:05:30.838 INFO:tasks.workunit.client.0.smithi046.stdout:3/766: dwrite d12/d13/d37/d51/db5/d4b/db7/ff5 [0,4194304] 0 2024-09-10T04:05:30.843 INFO:tasks.workunit.client.1.smithi070.stdout:8/922: unlink d4/d39/dfc/f46 0 2024-09-10T04:05:30.844 INFO:tasks.workunit.client.0.smithi046.stdout:9/802: mknod d4/d7/d1f/d54/dc7/dc8/cfe 0 2024-09-10T04:05:30.845 INFO:tasks.workunit.client.1.smithi070.stdout:9/901: dread d9/d1c/d34/d8b/d11d/d1b/f45 [0,4194304] 0 2024-09-10T04:05:30.846 INFO:tasks.workunit.client.0.smithi046.stdout:6/697: dwrite da/db/d10/f17 [8388608,4194304] 0 2024-09-10T04:05:30.847 INFO:tasks.workunit.client.0.smithi046.stdout:1/609: rename d3/dc/d34/d58/lb6 to d3/d27/d5d/d60/lcc 0 2024-09-10T04:05:30.847 INFO:tasks.workunit.client.1.smithi070.stdout:4/833: dwrite de/d13/d5b/d7b/dd5/fc7 [0,4194304] 0 2024-09-10T04:05:30.848 INFO:tasks.workunit.client.1.smithi070.stdout:1/765: mkdir d3/d10/dfa 0 2024-09-10T04:05:30.850 INFO:tasks.workunit.client.0.smithi046.stdout:8/816: symlink d1/d29/d17/d1f/d46/d6a/l111 0 2024-09-10T04:05:30.854 INFO:tasks.workunit.client.0.smithi046.stdout:0/647: truncate d1/dc/d25/d33/d1a/f66 632821 0 2024-09-10T04:05:30.856 INFO:tasks.workunit.client.0.smithi046.stdout:0/648: dread - d1/dc/d25/d33/d1a/d41/fc6 zero size 2024-09-10T04:05:30.857 INFO:tasks.workunit.client.1.smithi070.stdout:6/854: write d0/d3/da4/dc0/dea/f76 [1103179,98860] 0 2024-09-10T04:05:30.857 INFO:tasks.workunit.client.1.smithi070.stdout:6/855: chown d0 372914 1 2024-09-10T04:05:30.860 INFO:tasks.workunit.client.1.smithi070.stdout:9/902: fsync d9/d1c/d34/d8b/dcc/dfa/fea 0 2024-09-10T04:05:30.863 INFO:tasks.workunit.client.0.smithi046.stdout:6/698: rmdir da/db/d18/dd1 39 2024-09-10T04:05:30.865 INFO:tasks.workunit.client.1.smithi070.stdout:8/923: truncate d4/fa 132780 0 2024-09-10T04:05:30.865 INFO:tasks.workunit.client.0.smithi046.stdout:8/817: stat d1/d29/d19/d36/d4c/l59 0 2024-09-10T04:05:30.867 INFO:tasks.workunit.client.1.smithi070.stdout:6/856: symlink d0/d3/d18/d62/d70/l10d 0 2024-09-10T04:05:30.870 INFO:tasks.workunit.client.0.smithi046.stdout:9/803: link d4/d7/d61/fd7 d4/d7/d1f/d54/dc7/fff 0 2024-09-10T04:05:30.872 INFO:tasks.workunit.client.1.smithi070.stdout:9/903: write d9/d7a/f44 [2369954,84464] 0 2024-09-10T04:05:30.872 INFO:tasks.workunit.client.0.smithi046.stdout:8/818: dwrite d1/d29/d19/db7/fd5 [0,4194304] 0 2024-09-10T04:05:30.876 INFO:tasks.workunit.client.0.smithi046.stdout:6/699: rmdir da/db/d11/dd7 39 2024-09-10T04:05:30.877 INFO:tasks.workunit.client.0.smithi046.stdout:8/819: write d1/d29/d17/d1f/d2a/f58 [290991,25611] 0 2024-09-10T04:05:30.878 INFO:tasks.workunit.client.1.smithi070.stdout:1/766: mkdir d3/d8/d13/d67/d83/def/dfb 0 2024-09-10T04:05:30.880 INFO:tasks.workunit.client.0.smithi046.stdout:3/767: dwrite d12/d13/d14/d1e/d2d/f4f [0,4194304] 0 2024-09-10T04:05:30.960 INFO:tasks.workunit.client.1.smithi070.stdout:2/797: sync 2024-09-10T04:05:30.966 INFO:tasks.workunit.client.0.smithi046.stdout:9/804: symlink d4/d3c/de0/l100 0 2024-09-10T04:05:30.969 INFO:tasks.workunit.client.1.smithi070.stdout:1/767: fsync d3/f60 0 2024-09-10T04:05:30.972 INFO:tasks.workunit.client.0.smithi046.stdout:1/610: dwrite d3/f4a [0,4194304] 0 2024-09-10T04:05:30.972 INFO:tasks.workunit.client.1.smithi070.stdout:4/834: dwrite de/d13/d18/f3b [0,4194304] 0 2024-09-10T04:05:30.972 INFO:tasks.workunit.client.0.smithi046.stdout:6/700: creat da/db/d10/fe2 x:0 0 0 2024-09-10T04:05:30.974 INFO:tasks.workunit.client.1.smithi070.stdout:1/768: dread d3/d8/d27/d7a/d94/f9e [4194304,4194304] 0 2024-09-10T04:05:30.976 INFO:tasks.workunit.client.1.smithi070.stdout:2/798: fdatasync d15/d18/d29/d2a/d2e/da6/dc0/dcb/ff2 0 2024-09-10T04:05:30.976 INFO:tasks.workunit.client.1.smithi070.stdout:6/857: write d0/d3/d18/d1b/f11 [1583405,31113] 0 2024-09-10T04:05:30.977 INFO:tasks.workunit.client.0.smithi046.stdout:8/820: symlink d1/d29/d19/db7/l112 0 2024-09-10T04:05:30.978 INFO:tasks.workunit.client.1.smithi070.stdout:6/858: write d0/d3/d18/d1b/d17/ffb [932078,64560] 0 2024-09-10T04:05:30.980 INFO:tasks.workunit.client.1.smithi070.stdout:6/859: stat d0/d3/d18/d2a/ff6 0 2024-09-10T04:05:30.981 INFO:tasks.workunit.client.1.smithi070.stdout:6/860: chown d0/d3/da4/dc0/dea/f5e 90 1 2024-09-10T04:05:30.986 INFO:tasks.workunit.client.1.smithi070.stdout:1/769: mkdir d3/d8/d27/d7a/d94/dfc 0 2024-09-10T04:05:30.987 INFO:tasks.workunit.client.1.smithi070.stdout:2/799: mkdir d15/d18/d56/d7c/d55/d6d/d125 0 2024-09-10T04:05:30.991 INFO:tasks.workunit.client.0.smithi046.stdout:3/768: write d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fa6 [290380,25305] 0 2024-09-10T04:05:30.993 INFO:tasks.workunit.client.1.smithi070.stdout:6/861: read d0/d3/d18/d62/f85 [30935,72559] 0 2024-09-10T04:05:30.994 INFO:tasks.workunit.client.1.smithi070.stdout:6/862: chown d0/d3/d18/d2a 7578 1 2024-09-10T04:05:30.994 INFO:tasks.workunit.client.0.smithi046.stdout:3/769: write d12/d13/d37/d51/db5/d50/d7a/dc1/f97 [660059,49078] 0 2024-09-10T04:05:31.000 INFO:tasks.workunit.client.1.smithi070.stdout:4/835: truncate de/d13/d5b/d7b/fe2 3357135 0 2024-09-10T04:05:31.005 INFO:tasks.workunit.client.1.smithi070.stdout:2/800: mknod d15/d2b/d6e/db8/c126 0 2024-09-10T04:05:31.006 INFO:tasks.workunit.client.1.smithi070.stdout:8/924: rmdir d4/d16/d73/d139/d7a/db8/d107 0 2024-09-10T04:05:31.008 INFO:tasks.workunit.client.1.smithi070.stdout:1/770: truncate d3/d8/fc8 3260230 0 2024-09-10T04:05:31.011 INFO:tasks.workunit.client.1.smithi070.stdout:2/801: dread d15/d18/d56/d7c/f92 [0,4194304] 0 2024-09-10T04:05:31.016 INFO:tasks.workunit.client.1.smithi070.stdout:6/863: dwrite d0/d3/d18/d1b/d17/d53/f61 [4194304,4194304] 0 2024-09-10T04:05:31.017 INFO:tasks.workunit.client.1.smithi070.stdout:6/864: chown d0/d3/d1f/d81/la8 132275 1 2024-09-10T04:05:31.021 INFO:tasks.workunit.client.1.smithi070.stdout:2/802: symlink d15/d18/db2/de9/l127 0 2024-09-10T04:05:31.028 INFO:tasks.workunit.client.1.smithi070.stdout:1/771: write d3/d10/f72 [1144731,27475] 0 2024-09-10T04:05:31.029 INFO:tasks.workunit.client.1.smithi070.stdout:8/925: dread d4/d16/f3f [4194304,4194304] 0 2024-09-10T04:05:31.029 INFO:tasks.workunit.client.1.smithi070.stdout:6/865: dread d0/d3/d1f/d81/fb7 [0,4194304] 0 2024-09-10T04:05:31.035 INFO:tasks.workunit.client.1.smithi070.stdout:8/926: dwrite d4/dbf/d87/ff2 [0,4194304] 0 2024-09-10T04:05:31.037 INFO:tasks.workunit.client.1.smithi070.stdout:6/866: mknod d0/d3/d18/d1b/d17/d3f/d40/d94/c10e 0 2024-09-10T04:05:31.037 INFO:tasks.workunit.client.1.smithi070.stdout:8/927: dread - d4/d16/dcc/d12e/d11b/d19/d62/dd1/f13b zero size 2024-09-10T04:05:31.039 INFO:tasks.workunit.client.0.smithi046.stdout:9/805: unlink d4/d66/da6/f72 0 2024-09-10T04:05:31.040 INFO:tasks.workunit.client.1.smithi070.stdout:8/928: readlink d4/dcf/le1 0 2024-09-10T04:05:31.042 INFO:tasks.workunit.client.1.smithi070.stdout:2/803: dwrite d15/d18/d29/d2a/d2e/d32/f82 [0,4194304] 0 2024-09-10T04:05:31.043 INFO:tasks.workunit.client.0.smithi046.stdout:1/611: rename d3/da1/d1f/l81 to d3/dc/d34/d58/lcd 0 2024-09-10T04:05:31.044 INFO:tasks.workunit.client.0.smithi046.stdout:1/612: chown d3/d27/d5d/d60/fc0 2260 1 2024-09-10T04:05:31.047 INFO:tasks.workunit.client.1.smithi070.stdout:8/929: creat d4/d16/dcc/d12e/d11b/d19/d62/f13d x:0 0 0 2024-09-10T04:05:31.048 INFO:tasks.workunit.client.1.smithi070.stdout:2/804: truncate d15/d18/f1c 2123543 0 2024-09-10T04:05:31.049 INFO:tasks.workunit.client.1.smithi070.stdout:6/867: dread d0/d3/d18/d1b/d33/f65 [0,4194304] 0 2024-09-10T04:05:31.050 INFO:tasks.workunit.client.0.smithi046.stdout:1/613: dread d3/d27/f33 [0,4194304] 0 2024-09-10T04:05:31.052 INFO:tasks.workunit.client.0.smithi046.stdout:1/614: read d3/dc/f17 [5197413,29428] 0 2024-09-10T04:05:31.053 INFO:tasks.workunit.client.1.smithi070.stdout:8/930: creat d4/d39/dfc/d43/d67/dec/f13e x:0 0 0 2024-09-10T04:05:31.055 INFO:tasks.workunit.client.1.smithi070.stdout:8/931: dread - d4/d39/dfc/f10e zero size 2024-09-10T04:05:31.055 INFO:tasks.workunit.client.1.smithi070.stdout:2/805: dwrite d15/d18/d29/d64/f118 [0,4194304] 0 2024-09-10T04:05:31.055 INFO:tasks.workunit.client.0.smithi046.stdout:9/806: unlink d4/d7/f5e 0 2024-09-10T04:05:31.056 INFO:tasks.workunit.client.1.smithi070.stdout:8/932: write d4/d39/dc6/f10b [3949175,84992] 0 2024-09-10T04:05:31.058 INFO:tasks.workunit.client.1.smithi070.stdout:6/868: rename d0/d3/d18/d62/d70/fba to d0/d3/d18/d62/d70/f10f 0 2024-09-10T04:05:31.062 INFO:tasks.workunit.client.1.smithi070.stdout:2/806: creat d15/df6/f128 x:0 0 0 2024-09-10T04:05:31.064 INFO:tasks.workunit.client.0.smithi046.stdout:1/615: unlink d3/d27/d2a/f8b 0 2024-09-10T04:05:31.064 INFO:tasks.workunit.client.0.smithi046.stdout:1/616: readlink d3/d27/d5d/l79 0 2024-09-10T04:05:31.066 INFO:tasks.workunit.client.0.smithi046.stdout:3/770: dread d12/d13/d37/d51/db5/f4c [0,4194304] 0 2024-09-10T04:05:31.074 INFO:tasks.workunit.client.1.smithi070.stdout:2/807: unlink d15/d2b/d6e/f89 0 2024-09-10T04:05:31.076 INFO:tasks.workunit.client.0.smithi046.stdout:1/617: creat d3/dc/d34/d58/fce x:0 0 0 2024-09-10T04:05:31.076 INFO:tasks.workunit.client.0.smithi046.stdout:8/821: write d1/da2/faa [125866,64002] 0 2024-09-10T04:05:31.078 INFO:tasks.workunit.client.0.smithi046.stdout:8/822: chown d1/d29/d19/l3d 276749 1 2024-09-10T04:05:31.078 INFO:tasks.workunit.client.0.smithi046.stdout:3/771: chown d12/d13/c77 610797 1 2024-09-10T04:05:31.078 INFO:tasks.workunit.client.1.smithi070.stdout:3/853: sync 2024-09-10T04:05:31.078 INFO:tasks.workunit.client.1.smithi070.stdout:5/908: sync 2024-09-10T04:05:31.079 INFO:tasks.workunit.client.1.smithi070.stdout:3/854: chown db/d1c/d25/dd8/l119 3800971 1 2024-09-10T04:05:31.079 INFO:tasks.workunit.client.1.smithi070.stdout:5/909: chown d17/d19/d6a/d54/d57/da3/f97 49 1 2024-09-10T04:05:31.081 INFO:tasks.workunit.client.0.smithi046.stdout:6/701: rename da/db/d11/d13/d1e/d23/d54/d90 to da/de3 0 2024-09-10T04:05:31.082 INFO:tasks.workunit.client.0.smithi046.stdout:6/702: write da/db/f78 [489051,100553] 0 2024-09-10T04:05:31.083 INFO:tasks.workunit.client.1.smithi070.stdout:8/933: rename d4/d39/dfc/c9e to d4/dcf/c13f 0 2024-09-10T04:05:31.085 INFO:tasks.workunit.client.1.smithi070.stdout:6/869: write d0/d3/d8c/ff1 [482927,65528] 0 2024-09-10T04:05:31.086 INFO:tasks.workunit.client.1.smithi070.stdout:5/910: dwrite d17/d19/d6a/d54/d57/da8/fdc [0,4194304] 0 2024-09-10T04:05:31.089 INFO:tasks.workunit.client.1.smithi070.stdout:3/855: rename db/d1c/d3e/l52 to db/d1c/d25/d28/dd3/de5/d10f/d94/d99/l125 0 2024-09-10T04:05:31.091 INFO:tasks.workunit.client.1.smithi070.stdout:2/808: creat d15/d18/d29/d2a/d2e/da6/dc0/dca/f129 x:0 0 0 2024-09-10T04:05:31.092 INFO:tasks.workunit.client.0.smithi046.stdout:6/703: symlink da/d96/le4 0 2024-09-10T04:05:31.093 INFO:tasks.workunit.client.0.smithi046.stdout:9/807: link d4/d7/ca d4/d7/d81/c101 0 2024-09-10T04:05:31.094 INFO:tasks.workunit.client.0.smithi046.stdout:9/808: chown d4/f28 797 1 2024-09-10T04:05:31.095 INFO:tasks.workunit.client.0.smithi046.stdout:9/809: stat d4/d7/d61/dd0/ld3 0 2024-09-10T04:05:31.096 INFO:tasks.workunit.client.1.smithi070.stdout:8/934: mkdir d4/d16/dcc/d12e/da4/d140 0 2024-09-10T04:05:31.097 INFO:tasks.workunit.client.1.smithi070.stdout:8/935: stat d4/d16/dcc/d12e/d11b/d19/d62/fd4 0 2024-09-10T04:05:31.099 INFO:tasks.workunit.client.0.smithi046.stdout:8/823: write d1/d29/d17/d1f/d46/fca [31224,64508] 0 2024-09-10T04:05:31.107 INFO:tasks.workunit.client.0.smithi046.stdout:9/810: creat d4/d18/d89/f102 x:0 0 0 2024-09-10T04:05:31.107 INFO:tasks.workunit.client.0.smithi046.stdout:8/824: dwrite d1/d29/d17/d1f/f54 [0,4194304] 0 2024-09-10T04:05:31.109 INFO:tasks.workunit.client.0.smithi046.stdout:1/618: write d3/d62/fc9 [711891,112450] 0 2024-09-10T04:05:31.110 INFO:tasks.workunit.client.0.smithi046.stdout:9/811: dread d4/d7/d61/ff3 [0,4194304] 0 2024-09-10T04:05:31.111 INFO:tasks.workunit.client.0.smithi046.stdout:9/812: stat d4/d3c/de0/l3e 0 2024-09-10T04:05:31.111 INFO:tasks.workunit.client.0.smithi046.stdout:6/704: write da/db/d11/d13/d1e/f6a [882573,114302] 0 2024-09-10T04:05:31.114 INFO:tasks.workunit.client.1.smithi070.stdout:5/911: rename l1 to d17/d24/d84/dab/d114/l138 0 2024-09-10T04:05:31.114 INFO:tasks.workunit.client.0.smithi046.stdout:3/772: link d12/d13/d37/d51/db5/d50/d52/f65 d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/f10a 0 2024-09-10T04:05:31.116 INFO:tasks.workunit.client.1.smithi070.stdout:3/856: mkdir db/d83/d92/d11b/d126 0 2024-09-10T04:05:31.119 INFO:tasks.workunit.client.1.smithi070.stdout:5/912: dwrite d17/d6f/fc8 [0,4194304] 0 2024-09-10T04:05:31.119 INFO:tasks.workunit.client.0.smithi046.stdout:6/705: dwrite da/db/d11/d2b/f70 [0,4194304] 0 2024-09-10T04:05:31.120 INFO:tasks.workunit.client.1.smithi070.stdout:5/913: write d17/d24/d45/f129 [720279,102008] 0 2024-09-10T04:05:31.121 INFO:tasks.workunit.client.1.smithi070.stdout:6/870: creat d0/d3/d8c/f110 x:0 0 0 2024-09-10T04:05:31.122 INFO:tasks.workunit.client.1.smithi070.stdout:5/914: chown d17/d19/f2e 345724 1 2024-09-10T04:05:31.123 INFO:tasks.workunit.client.1.smithi070.stdout:5/915: stat d17/d24/d2b/f81 0 2024-09-10T04:05:31.123 INFO:tasks.workunit.client.1.smithi070.stdout:6/871: write d0/d3/da4/dc0/dea/f5e [4011615,11015] 0 2024-09-10T04:05:31.125 INFO:tasks.workunit.client.1.smithi070.stdout:6/872: stat d0/d3/d5c/cd5 0 2024-09-10T04:05:31.125 INFO:tasks.workunit.client.1.smithi070.stdout:3/857: mknod db/d83/d92/c127 0 2024-09-10T04:05:31.126 INFO:tasks.workunit.client.0.smithi046.stdout:5/936: sync 2024-09-10T04:05:31.126 INFO:tasks.workunit.client.0.smithi046.stdout:2/787: sync 2024-09-10T04:05:31.129 INFO:tasks.workunit.client.0.smithi046.stdout:9/813: dread d4/d3c/d4d/f76 [0,4194304] 0 2024-09-10T04:05:31.130 INFO:tasks.workunit.client.1.smithi070.stdout:2/809: getdents d15/d18/d29/ddb 0 2024-09-10T04:05:31.131 INFO:tasks.workunit.client.0.smithi046.stdout:6/706: rmdir da/d71 39 2024-09-10T04:05:31.132 INFO:tasks.workunit.client.1.smithi070.stdout:5/916: rmdir d17/d24/d84/dab/d114 39 2024-09-10T04:05:31.135 INFO:tasks.workunit.client.0.smithi046.stdout:1/619: rename d3/da1/fbb to d3/da1/da7/fcf 0 2024-09-10T04:05:31.136 INFO:tasks.workunit.client.1.smithi070.stdout:3/858: creat db/d83/d92/d11b/f128 x:0 0 0 2024-09-10T04:05:31.137 INFO:tasks.workunit.client.1.smithi070.stdout:8/936: write d4/dbf/d87/f9f [6551093,128425] 0 2024-09-10T04:05:31.138 INFO:tasks.workunit.client.1.smithi070.stdout:8/937: write d4/d39/dc6/f10b [3637856,62521] 0 2024-09-10T04:05:31.138 INFO:tasks.workunit.client.1.smithi070.stdout:2/810: rmdir d15/d18/d29/d2a/d2e/da6/dc0/dca 39 2024-09-10T04:05:31.139 INFO:tasks.workunit.client.1.smithi070.stdout:2/811: fdatasync d15/d18/d56/f9c 0 2024-09-10T04:05:31.145 INFO:tasks.workunit.client.0.smithi046.stdout:9/814: mkdir d4/d29/d65/d103 0 2024-09-10T04:05:31.147 INFO:tasks.workunit.client.0.smithi046.stdout:6/707: creat da/db/d10/fe5 x:0 0 0 2024-09-10T04:05:31.147 INFO:tasks.workunit.client.0.smithi046.stdout:3/773: dwrite f7 [0,4194304] 0 2024-09-10T04:05:31.149 INFO:tasks.workunit.client.1.smithi070.stdout:2/812: symlink d15/d2b/d6e/db8/l12a 0 2024-09-10T04:05:31.149 INFO:tasks.workunit.client.0.smithi046.stdout:1/620: creat d3/d27/d2a/fd0 x:0 0 0 2024-09-10T04:05:31.151 INFO:tasks.workunit.client.0.smithi046.stdout:9/815: read - d4/d7/d1f/dce/fec zero size 2024-09-10T04:05:31.151 INFO:tasks.workunit.client.1.smithi070.stdout:5/917: fsync d17/d24/d84/d9b/f10b 0 2024-09-10T04:05:31.154 INFO:tasks.workunit.client.1.smithi070.stdout:2/813: dread d15/d18/f7e [0,4194304] 0 2024-09-10T04:05:31.154 INFO:tasks.workunit.client.0.smithi046.stdout:9/816: chown d4/d7/d1f/dce/dd4/lf1 23 1 2024-09-10T04:05:31.154 INFO:tasks.workunit.client.0.smithi046.stdout:2/788: dwrite d2/d86/d9c/fe1 [0,4194304] 0 2024-09-10T04:05:31.155 INFO:tasks.workunit.client.0.smithi046.stdout:3/774: truncate d12/d13/d14/d1e/dd2/fe6 50426 0 2024-09-10T04:05:31.156 INFO:tasks.workunit.client.1.smithi070.stdout:2/814: stat d15/d18/f26 0 2024-09-10T04:05:31.156 INFO:tasks.workunit.client.1.smithi070.stdout:8/938: dread d4/d16/dcc/d12e/fee [0,4194304] 0 2024-09-10T04:05:31.156 INFO:tasks.workunit.client.0.smithi046.stdout:9/817: dread - d4/d29/ffb zero size 2024-09-10T04:05:31.156 INFO:tasks.workunit.client.0.smithi046.stdout:8/825: getdents d1/d29/d17/d1f/d46/d6a/d82/dc2 0 2024-09-10T04:05:31.158 INFO:tasks.workunit.client.0.smithi046.stdout:3/775: chown d12/d13/d14/d1e/d2d/d6f/d8f 12 1 2024-09-10T04:05:31.160 INFO:tasks.workunit.client.1.smithi070.stdout:5/918: dwrite d17/d19/d6a/d54/f62 [4194304,4194304] 0 2024-09-10T04:05:31.163 INFO:tasks.workunit.client.1.smithi070.stdout:5/919: truncate d17/d19/d6a/d54/d57/da8/f102 183901 0 2024-09-10T04:05:31.165 INFO:tasks.workunit.client.1.smithi070.stdout:5/920: dread - d17/d19/d6a/d54/de1/dfe/d93/f107 zero size 2024-09-10T04:05:31.166 INFO:tasks.workunit.client.1.smithi070.stdout:6/873: link d0/d3/da4/dc0/dea/f43 d0/d3/da4/f111 0 2024-09-10T04:05:31.168 INFO:tasks.workunit.client.0.smithi046.stdout:4/952: sync 2024-09-10T04:05:31.168 INFO:tasks.workunit.client.0.smithi046.stdout:0/649: sync 2024-09-10T04:05:31.168 INFO:tasks.workunit.client.0.smithi046.stdout:7/882: sync 2024-09-10T04:05:31.168 INFO:tasks.workunit.client.1.smithi070.stdout:6/874: truncate d0/d3/d18/d62/d70/f105 201213 0 2024-09-10T04:05:31.169 INFO:tasks.workunit.client.0.smithi046.stdout:5/937: creat d7/d11/d14/d2f/deb/da5/dcc/f143 x:0 0 0 2024-09-10T04:05:31.170 INFO:tasks.workunit.client.0.smithi046.stdout:6/708: dread da/f60 [0,4194304] 0 2024-09-10T04:05:31.172 INFO:tasks.workunit.client.0.smithi046.stdout:2/789: creat d2/d8/d37/dc2/f116 x:0 0 0 2024-09-10T04:05:31.174 INFO:tasks.workunit.client.0.smithi046.stdout:9/818: read - d4/d3c/de0/d95/fab zero size 2024-09-10T04:05:31.175 INFO:tasks.workunit.client.1.smithi070.stdout:5/921: mkdir d17/d6f/d139 0 2024-09-10T04:05:31.176 INFO:tasks.workunit.client.0.smithi046.stdout:8/826: readlink d1/d29/d19/d36/d4c/dd3/d10c/l10f 0 2024-09-10T04:05:31.181 INFO:tasks.workunit.client.0.smithi046.stdout:9/819: dread d4/fc9 [0,4194304] 0 2024-09-10T04:05:31.184 INFO:tasks.workunit.client.1.smithi070.stdout:8/939: mkdir d4/d16/d73/d139/d7a/d141 0 2024-09-10T04:05:31.187 INFO:tasks.workunit.client.1.smithi070.stdout:5/922: mknod d17/d24/d77/db4/c13a 0 2024-09-10T04:05:31.189 INFO:tasks.workunit.client.0.smithi046.stdout:6/709: dread da/f7b [0,4194304] 0 2024-09-10T04:05:31.189 INFO:tasks.workunit.client.1.smithi070.stdout:6/875: mknod d0/d3/d18/de3/c112 0 2024-09-10T04:05:31.191 INFO:tasks.workunit.client.0.smithi046.stdout:5/938: dread d7/d11/f1a [0,4194304] 0 2024-09-10T04:05:31.192 INFO:tasks.workunit.client.1.smithi070.stdout:5/923: creat d17/d19/d6a/dfb/f13b x:0 0 0 2024-09-10T04:05:31.192 INFO:tasks.workunit.client.1.smithi070.stdout:2/815: dwrite d15/d18/d29/d64/f95 [0,4194304] 0 2024-09-10T04:05:31.192 INFO:tasks.workunit.client.0.smithi046.stdout:5/939: readlink d7/d11/d14/d8e/d96/le7 0 2024-09-10T04:05:31.193 INFO:tasks.workunit.client.1.smithi070.stdout:6/876: readlink d0/d3/d18/d1b/d17/l49 0 2024-09-10T04:05:31.194 INFO:tasks.workunit.client.1.smithi070.stdout:6/877: write d0/d3/d18/d2a/ff6 [408989,119904] 0 2024-09-10T04:05:31.197 INFO:tasks.workunit.client.0.smithi046.stdout:2/790: creat d2/d6/d23/d5d/d5e/f117 x:0 0 0 2024-09-10T04:05:31.197 INFO:tasks.workunit.client.0.smithi046.stdout:5/940: dread d7/ffd [0,4194304] 0 2024-09-10T04:05:31.197 INFO:tasks.workunit.client.0.smithi046.stdout:2/791: dread - d2/d6/f4f zero size 2024-09-10T04:05:31.198 INFO:tasks.workunit.client.0.smithi046.stdout:5/941: write d7/f1f [900894,123264] 0 2024-09-10T04:05:31.199 INFO:tasks.workunit.client.1.smithi070.stdout:6/878: fdatasync d0/d3/d18/d1b/d17/d3f/fa9 0 2024-09-10T04:05:31.200 INFO:tasks.workunit.client.0.smithi046.stdout:5/942: chown d7/d11/d3f/d53/da9 0 1 2024-09-10T04:05:31.200 INFO:tasks.workunit.client.1.smithi070.stdout:5/924: dwrite d17/d19/d6a/d54/de1/dfe/dce/f100 [0,4194304] 0 2024-09-10T04:05:31.201 INFO:tasks.workunit.client.0.smithi046.stdout:8/827: symlink d1/d29/d19/d73/dd2/l113 0 2024-09-10T04:05:31.207 INFO:tasks.workunit.client.1.smithi070.stdout:5/925: readlink d17/d19/d6a/l12a 0 2024-09-10T04:05:31.208 INFO:tasks.workunit.client.0.smithi046.stdout:1/621: rmdir d3/d62/dca 0 2024-09-10T04:05:31.208 INFO:tasks.workunit.client.1.smithi070.stdout:5/926: chown d17/d19/d6a/d54/de1/dfe/f115 1349 1 2024-09-10T04:05:31.209 INFO:tasks.workunit.client.1.smithi070.stdout:8/940: write d4/d39/dc6/fe3 [514352,47178] 0 2024-09-10T04:05:31.210 INFO:tasks.workunit.client.0.smithi046.stdout:4/953: mkdir d3/d12/d146 0 2024-09-10T04:05:31.211 INFO:tasks.workunit.client.0.smithi046.stdout:1/622: read d3/d27/d5d/d60/fc0 [8025732,84700] 0 2024-09-10T04:05:31.212 INFO:tasks.workunit.client.0.smithi046.stdout:4/954: stat d3/d12/d17/d22/d51/d5c 0 2024-09-10T04:05:31.212 INFO:tasks.workunit.client.0.smithi046.stdout:0/650: write d1/dc/f7b [24531,117162] 0 2024-09-10T04:05:31.213 INFO:tasks.workunit.client.0.smithi046.stdout:6/710: creat da/d45/fe6 x:0 0 0 2024-09-10T04:05:31.216 INFO:tasks.workunit.client.1.smithi070.stdout:5/927: read f12 [4811933,78647] 0 2024-09-10T04:05:31.218 INFO:tasks.workunit.client.1.smithi070.stdout:6/879: truncate d0/d3/d1f/d81/fb7 1356496 0 2024-09-10T04:05:31.223 INFO:tasks.workunit.client.0.smithi046.stdout:7/883: rename d5/d9/d12/d72/d77/de0/c15 to d5/d9/d12/dc5/c138 0 2024-09-10T04:05:31.223 INFO:tasks.workunit.client.1.smithi070.stdout:2/816: dwrite d15/d2b/d6e/fd4 [0,4194304] 0 2024-09-10T04:05:31.224 INFO:tasks.workunit.client.1.smithi070.stdout:2/817: chown d15/d18/d56/cab 68239 1 2024-09-10T04:05:31.227 INFO:tasks.workunit.client.0.smithi046.stdout:2/792: dread d2/d6/d23/d5d/d77/f54 [4194304,4194304] 0 2024-09-10T04:05:31.228 INFO:tasks.workunit.client.0.smithi046.stdout:4/955: dread d3/d12/d17/f20 [4194304,4194304] 0 2024-09-10T04:05:31.230 INFO:tasks.workunit.client.0.smithi046.stdout:3/776: link d12/d5d/d71/c66 d12/d13/d37/d51/db5/d4b/db7/c10b 0 2024-09-10T04:05:31.230 INFO:tasks.workunit.client.0.smithi046.stdout:4/956: readlink d3/d12/d14/l112 0 2024-09-10T04:05:31.230 INFO:tasks.workunit.client.0.smithi046.stdout:2/793: write d2/d49/f2e [183298,130299] 0 2024-09-10T04:05:31.234 INFO:tasks.workunit.client.0.smithi046.stdout:1/623: truncate d3/dc/d34/d58/f59 304169 0 2024-09-10T04:05:31.242 INFO:tasks.workunit.client.0.smithi046.stdout:7/884: mkdir d5/d9/d43/db4/def/d139 0 2024-09-10T04:05:31.242 INFO:tasks.workunit.client.0.smithi046.stdout:1/624: dwrite d3/da1/d1f/fb0 [0,4194304] 0 2024-09-10T04:05:31.244 INFO:tasks.workunit.client.1.smithi070.stdout:7/935: sync 2024-09-10T04:05:31.244 INFO:tasks.workunit.client.1.smithi070.stdout:9/904: sync 2024-09-10T04:05:31.248 INFO:tasks.workunit.client.0.smithi046.stdout:8/828: creat d1/d29/d17/d1f/d2a/df6/d102/f114 x:0 0 0 2024-09-10T04:05:31.252 INFO:tasks.workunit.client.0.smithi046.stdout:7/885: dwrite d5/d6/dbe/dd7/f101 [0,4194304] 0 2024-09-10T04:05:31.252 INFO:tasks.workunit.client.0.smithi046.stdout:4/957: write d3/d12/f4d [962528,4060] 0 2024-09-10T04:05:31.252 INFO:tasks.workunit.client.0.smithi046.stdout:3/777: mknod d12/d13/d14/d1e/d2d/c10c 0 2024-09-10T04:05:31.254 INFO:tasks.workunit.client.0.smithi046.stdout:7/886: stat d5/d6/d62/l7e 0 2024-09-10T04:05:31.254 INFO:tasks.workunit.client.0.smithi046.stdout:6/711: dread da/db/d11/dd7/f34 [0,4194304] 0 2024-09-10T04:05:31.254 INFO:tasks.workunit.client.0.smithi046.stdout:2/794: rmdir d2/d6/d23/d55/d9f 39 2024-09-10T04:05:31.256 INFO:tasks.workunit.client.0.smithi046.stdout:3/778: truncate d12/d5d/d71/db1/ff1 94065 0 2024-09-10T04:05:31.257 INFO:tasks.workunit.client.1.smithi070.stdout:7/936: dread d2/d7/d9/d18/fcc [0,4194304] 0 2024-09-10T04:05:31.262 INFO:tasks.workunit.client.0.smithi046.stdout:8/829: readlink d1/d29/d17/d1f/l101 0 2024-09-10T04:05:31.266 INFO:tasks.workunit.client.0.smithi046.stdout:2/795: dwrite d2/d6/d23/d42/db4/dca/fe8 [0,4194304] 0 2024-09-10T04:05:31.272 INFO:tasks.workunit.client.0.smithi046.stdout:9/820: rename d4/c5f to d4/d3c/c104 0 2024-09-10T04:05:31.273 INFO:tasks.workunit.client.0.smithi046.stdout:9/821: readlink d4/d3c/d4d/d5c/d59/ld1 0 2024-09-10T04:05:31.276 INFO:tasks.workunit.client.0.smithi046.stdout:6/712: symlink da/d9b/le7 0 2024-09-10T04:05:31.281 INFO:tasks.workunit.client.0.smithi046.stdout:8/830: creat d1/d29/d19/d36/d6e/f115 x:0 0 0 2024-09-10T04:05:31.281 INFO:tasks.workunit.client.0.smithi046.stdout:1/625: dread d3/d27/f41 [0,4194304] 0 2024-09-10T04:05:31.281 INFO:tasks.workunit.client.0.smithi046.stdout:1/626: write d3/da1/d1f/f35 [153902,54750] 0 2024-09-10T04:05:31.281 INFO:tasks.workunit.client.0.smithi046.stdout:8/831: write d1/d29/d19/d36/d6e/d75/fe3 [1000594,20567] 0 2024-09-10T04:05:31.282 INFO:tasks.workunit.client.0.smithi046.stdout:1/627: dread - d3/d9b/fb5 zero size 2024-09-10T04:05:31.284 INFO:tasks.workunit.client.0.smithi046.stdout:2/796: symlink d2/d6/d23/d5d/d77/d7b/d85/d8d/l118 0 2024-09-10T04:05:31.286 INFO:tasks.workunit.client.0.smithi046.stdout:2/797: truncate d2/d6/d9d/f106 335285 0 2024-09-10T04:05:31.286 INFO:tasks.workunit.client.0.smithi046.stdout:0/651: rename d1/dc/d25/cb6 to d1/dc/d25/d33/d3/dd/d55/cc7 0 2024-09-10T04:05:31.288 INFO:tasks.workunit.client.0.smithi046.stdout:3/779: symlink d12/d13/d37/d51/db5/d50/l10d 0 2024-09-10T04:05:31.289 INFO:tasks.workunit.client.0.smithi046.stdout:9/822: mkdir d4/d18/d89/d105 0 2024-09-10T04:05:31.290 INFO:tasks.workunit.client.0.smithi046.stdout:9/823: chown d4/d18/d92/lb2 4545 1 2024-09-10T04:05:31.290 INFO:tasks.workunit.client.0.smithi046.stdout:6/713: symlink da/db/d11/d2b/d55/le8 0 2024-09-10T04:05:31.295 INFO:tasks.workunit.client.1.smithi070.stdout:5/928: creat d17/d24/d2b/f13c x:0 0 0 2024-09-10T04:05:31.296 INFO:tasks.workunit.client.0.smithi046.stdout:7/887: write d5/d6/db/dd/d24/d76/fe6 [1001537,1849] 0 2024-09-10T04:05:31.296 INFO:tasks.workunit.client.1.smithi070.stdout:5/929: write d17/d19/d35/f3a [1573142,84788] 0 2024-09-10T04:05:31.298 INFO:tasks.workunit.client.1.smithi070.stdout:6/880: rename d0/d3/d18/d62/f68 to d0/d3/d8c/d9a/f113 0 2024-09-10T04:05:31.302 INFO:tasks.workunit.client.0.smithi046.stdout:3/780: creat d12/d5d/d71/d49/f10e x:0 0 0 2024-09-10T04:05:31.303 INFO:tasks.workunit.client.0.smithi046.stdout:9/824: rename d4/d3c/d4d/d5c/f4e to d4/d66/d7c/f106 0 2024-09-10T04:05:31.308 INFO:tasks.workunit.client.0.smithi046.stdout:8/832: mkdir d1/d29/d17/d1f/d46/d67/d116 0 2024-09-10T04:05:31.309 INFO:tasks.workunit.client.0.smithi046.stdout:7/888: chown d5/d9/d12/d72/d77/ca2 26743 1 2024-09-10T04:05:31.332 INFO:tasks.workunit.client.1.smithi070.stdout:4/836: sync 2024-09-10T04:05:31.332 INFO:tasks.workunit.client.1.smithi070.stdout:1/772: sync 2024-09-10T04:05:31.333 INFO:tasks.workunit.client.1.smithi070.stdout:3/859: sync 2024-09-10T04:05:31.333 INFO:tasks.workunit.client.1.smithi070.stdout:3/860: chown db/d1c/d25/d28/dd3/de5/d10f/d110 51161354 1 2024-09-10T04:05:31.334 INFO:tasks.workunit.client.1.smithi070.stdout:3/861: truncate db/f3c 5041044 0 2024-09-10T04:05:31.348 INFO:tasks.workunit.client.1.smithi070.stdout:6/881: rmdir d0/d3/d5c 39 2024-09-10T04:05:31.350 INFO:tasks.workunit.client.1.smithi070.stdout:2/818: creat d15/d2b/d106/d123/f12b x:0 0 0 2024-09-10T04:05:31.374 INFO:tasks.workunit.client.1.smithi070.stdout:8/941: sync 2024-09-10T04:05:31.375 INFO:tasks.workunit.client.0.smithi046.stdout:9/825: unlink d4/d44/f86 0 2024-09-10T04:05:31.375 INFO:tasks.workunit.client.1.smithi070.stdout:7/937: mkdir d2/dc8/d119/d7e/d12d/d14d 0 2024-09-10T04:05:31.376 INFO:tasks.workunit.client.0.smithi046.stdout:6/714: mkdir da/db/d11/d9f/dd6/de9 0 2024-09-10T04:05:31.380 INFO:tasks.workunit.client.1.smithi070.stdout:1/773: mknod d3/d90/dcb/cfd 0 2024-09-10T04:05:31.380 INFO:tasks.workunit.client.0.smithi046.stdout:6/715: truncate da/db/d11/d13/d1e/f3e 519841 0 2024-09-10T04:05:31.381 INFO:tasks.workunit.client.1.smithi070.stdout:1/774: truncate d3/d8/d13/d3f/d6d/d81/fe9 663033 0 2024-09-10T04:05:31.382 INFO:tasks.workunit.client.1.smithi070.stdout:4/837: rename de/d13/d5b/d7b/dd5/fc7 to de/d13/d18/d5c/db6/f11b 0 2024-09-10T04:05:31.383 INFO:tasks.workunit.client.0.smithi046.stdout:1/628: dwrite d3/dc/d34/f72 [0,4194304] 0 2024-09-10T04:05:31.384 INFO:tasks.workunit.client.1.smithi070.stdout:3/862: truncate db/d1c/d25/fe6 35636 0 2024-09-10T04:05:31.385 INFO:tasks.workunit.client.0.smithi046.stdout:6/716: fdatasync da/db/d11/d2b/d2f/d3b/fc3 0 2024-09-10T04:05:31.386 INFO:tasks.workunit.client.0.smithi046.stdout:0/652: link d1/dc/d25/d72/cb1 d1/dc/d25/d33/d3/dd/d55/cc8 0 2024-09-10T04:05:31.388 INFO:tasks.workunit.client.0.smithi046.stdout:8/833: dwrite d1/f2e [4194304,4194304] 0 2024-09-10T04:05:31.389 INFO:tasks.workunit.client.1.smithi070.stdout:2/819: unlink d15/d18/d29/d2a/d2e/d32/d9b/c116 0 2024-09-10T04:05:31.390 INFO:tasks.workunit.client.1.smithi070.stdout:5/930: dwrite d17/d24/d84/d9b/fcb [0,4194304] 0 2024-09-10T04:05:31.392 INFO:tasks.workunit.client.0.smithi046.stdout:3/781: write d12/d13/d14/d1e/d2d/f2e [79041,62758] 0 2024-09-10T04:05:31.395 INFO:tasks.workunit.client.1.smithi070.stdout:9/905: creat d9/d59/d2a/d3f/f128 x:0 0 0 2024-09-10T04:05:31.395 INFO:tasks.workunit.client.0.smithi046.stdout:0/653: read d1/dc/d25/d33/d4/d19/f58 [2391674,129995] 0 2024-09-10T04:05:31.395 INFO:tasks.workunit.client.0.smithi046.stdout:2/798: dwrite d2/d6/d23/d55/fa3 [0,4194304] 0 2024-09-10T04:05:31.398 INFO:tasks.workunit.client.1.smithi070.stdout:9/906: write d9/d1c/d34/d8b/d11d/f20 [6098153,62027] 0 2024-09-10T04:05:31.398 INFO:tasks.workunit.client.1.smithi070.stdout:2/820: dwrite d15/df6/f128 [0,4194304] 0 2024-09-10T04:05:31.402 INFO:tasks.workunit.client.1.smithi070.stdout:1/775: creat d3/d8/d27/d7a/df4/ffe x:0 0 0 2024-09-10T04:05:31.403 INFO:tasks.workunit.client.0.smithi046.stdout:1/629: symlink d3/da1/da7/ld1 0 2024-09-10T04:05:31.405 INFO:tasks.workunit.client.1.smithi070.stdout:4/838: fdatasync de/d13/d43/f6b 0 2024-09-10T04:05:31.407 INFO:tasks.workunit.client.1.smithi070.stdout:4/839: dread de/d13/d5b/d7b/dd5/d85/d10e/d48/feb [0,4194304] 0 2024-09-10T04:05:31.410 INFO:tasks.workunit.client.1.smithi070.stdout:6/882: symlink d0/d3/d18/d1b/d17/d53/d10c/l114 0 2024-09-10T04:05:31.412 INFO:tasks.workunit.client.0.smithi046.stdout:8/834: readlink d1/d29/d19/d36/lb2 0 2024-09-10T04:05:31.417 INFO:tasks.workunit.client.1.smithi070.stdout:7/938: write d2/d3/dc/d5d/d42/fb5 [4337046,74710] 0 2024-09-10T04:05:31.417 INFO:tasks.workunit.client.0.smithi046.stdout:8/835: dread d1/d29/d17/d1f/d46/d67/dc1/fc6 [0,4194304] 0 2024-09-10T04:05:31.422 INFO:tasks.workunit.client.1.smithi070.stdout:8/942: mkdir d4/d16/dcc/d12e/d11b/d6f/d142 0 2024-09-10T04:05:31.423 INFO:tasks.workunit.client.0.smithi046.stdout:3/782: mknod d12/d13/d37/d51/db5/d4b/d91/c10f 0 2024-09-10T04:05:31.425 INFO:tasks.workunit.client.0.smithi046.stdout:6/717: write da/db/d10/d27/f88 [1413973,23082] 0 2024-09-10T04:05:31.426 INFO:tasks.workunit.client.1.smithi070.stdout:3/863: write db/d1c/d53/f95 [855326,31266] 0 2024-09-10T04:05:31.427 INFO:tasks.workunit.client.0.smithi046.stdout:0/654: mkdir d1/dc/d25/d33/d1a/dc9 0 2024-09-10T04:05:31.428 INFO:tasks.workunit.client.1.smithi070.stdout:9/907: rmdir d9/d59 39 2024-09-10T04:05:31.428 INFO:tasks.workunit.client.1.smithi070.stdout:3/864: truncate db/d1c/d25/d28/dd3/de5/f118 1027504 0 2024-09-10T04:05:31.429 INFO:tasks.workunit.client.0.smithi046.stdout:8/836: dread d1/d29/d17/d1f/d2a/f2d [0,4194304] 0 2024-09-10T04:05:31.429 INFO:tasks.workunit.client.0.smithi046.stdout:7/889: link d5/d9/l8a d5/d6/db/dd/d4b/l13a 0 2024-09-10T04:05:31.430 INFO:tasks.workunit.client.0.smithi046.stdout:1/630: mknod d3/da1/da7/cd2 0 2024-09-10T04:05:31.434 INFO:tasks.workunit.client.0.smithi046.stdout:9/826: write d4/d7/d1f/f26 [2749439,115476] 0 2024-09-10T04:05:31.435 INFO:tasks.workunit.client.1.smithi070.stdout:4/840: chown de/d13/d18/l49 12963775 1 2024-09-10T04:05:31.436 INFO:tasks.workunit.client.0.smithi046.stdout:9/827: write d4/d44/f85 [1126474,17863] 0 2024-09-10T04:05:31.436 INFO:tasks.workunit.client.0.smithi046.stdout:8/837: dwrite d1/d29/d19/d36/d4c/d5e/fa1 [0,4194304] 0 2024-09-10T04:05:31.438 INFO:tasks.workunit.client.0.smithi046.stdout:3/783: unlink d12/d13/d14/f25 0 2024-09-10T04:05:31.439 INFO:tasks.workunit.client.0.smithi046.stdout:3/784: readlink d12/d13/d37/d51/db5/d50/l10d 0 2024-09-10T04:05:31.439 INFO:tasks.workunit.client.1.smithi070.stdout:6/883: unlink d0/d3/d18/d62/c67 0 2024-09-10T04:05:31.440 INFO:tasks.workunit.client.0.smithi046.stdout:2/799: mknod d2/d8/d37/dc2/d10d/c119 0 2024-09-10T04:05:31.443 INFO:tasks.workunit.client.1.smithi070.stdout:5/931: creat d17/d24/d84/dab/d114/f13d x:0 0 0 2024-09-10T04:05:31.443 INFO:tasks.workunit.client.1.smithi070.stdout:1/776: write d3/d8/f49 [2834997,59474] 0 2024-09-10T04:05:31.444 INFO:tasks.workunit.client.0.smithi046.stdout:6/718: rename da/dc/ca4 to da/db/d11/d13/d1e/d23/d54/dad/cea 0 2024-09-10T04:05:31.446 INFO:tasks.workunit.client.1.smithi070.stdout:7/939: rename d2/d3/dc/d5d/d41/f134 to d2/d7/d9/dd5/d102/f14e 0 2024-09-10T04:05:31.447 INFO:tasks.workunit.client.1.smithi070.stdout:7/940: chown d2/d7/d9/f82 627 1 2024-09-10T04:05:31.447 INFO:tasks.workunit.client.0.smithi046.stdout:0/655: unlink d1/dc/d25/d33/d1a/d41/f9d 0 2024-09-10T04:05:31.448 INFO:tasks.workunit.client.1.smithi070.stdout:7/941: chown d2/d3/d63/da9/ff3 46 1 2024-09-10T04:05:31.448 INFO:tasks.workunit.client.0.smithi046.stdout:0/656: fsync d1/dc/d25/d72/fb9 0 2024-09-10T04:05:31.449 INFO:tasks.workunit.client.0.smithi046.stdout:7/890: creat d5/d6/d32/f13b x:0 0 0 2024-09-10T04:05:31.451 INFO:tasks.workunit.client.1.smithi070.stdout:9/908: mkdir d9/d1c/d34/d8b/d11d/d3c/d92/d129 0 2024-09-10T04:05:31.452 INFO:tasks.workunit.client.0.smithi046.stdout:1/631: fsync d3/dc/f17 0 2024-09-10T04:05:31.452 INFO:tasks.workunit.client.1.smithi070.stdout:2/821: symlink d15/l12c 0 2024-09-10T04:05:31.454 INFO:tasks.workunit.client.0.smithi046.stdout:8/838: chown d1/d29/d19/d36/c51 78 1 2024-09-10T04:05:31.456 INFO:tasks.workunit.client.0.smithi046.stdout:3/785: chown d12/d5d/lcc 614 1 2024-09-10T04:05:31.462 INFO:tasks.workunit.client.0.smithi046.stdout:6/719: creat da/dc/da5/feb x:0 0 0 2024-09-10T04:05:31.466 INFO:tasks.workunit.client.0.smithi046.stdout:0/657: rename d1/d13/d37/d75/d94/d9a/da4/dbc to d1/d13/d37/d75/d94/dca 0 2024-09-10T04:05:31.468 INFO:tasks.workunit.client.0.smithi046.stdout:0/658: write d1/dc/d25/d33/d3/dd/f96 [893337,74997] 0 2024-09-10T04:05:31.472 INFO:tasks.workunit.client.0.smithi046.stdout:8/839: mkdir d1/d29/d17/d1f/d2a/df6/d117 0 2024-09-10T04:05:31.473 INFO:tasks.workunit.client.0.smithi046.stdout:6/720: creat da/db/d10/d51/fec x:0 0 0 2024-09-10T04:05:31.474 INFO:tasks.workunit.client.0.smithi046.stdout:1/632: rename d3/dc/d34/d58/lba to d3/d27/d5d/d60/ld3 0 2024-09-10T04:05:31.476 INFO:tasks.workunit.client.0.smithi046.stdout:0/659: unlink d1/dc/d25/d33/d1a/d41/fa0 0 2024-09-10T04:05:31.476 INFO:tasks.workunit.client.0.smithi046.stdout:2/800: dwrite d2/d8/f6f [0,4194304] 0 2024-09-10T04:05:31.479 INFO:tasks.workunit.client.0.smithi046.stdout:2/801: read d2/d6/d15/fa4 [1671048,125709] 0 2024-09-10T04:05:31.481 INFO:tasks.workunit.client.0.smithi046.stdout:6/721: creat da/db/d11/d9f/fed x:0 0 0 2024-09-10T04:05:31.485 INFO:tasks.workunit.client.0.smithi046.stdout:2/802: symlink d2/d6/d23/d42/de6/l11a 0 2024-09-10T04:05:31.488 INFO:tasks.workunit.client.0.smithi046.stdout:8/840: dwrite d1/d29/d19/d36/f3f [0,4194304] 0 2024-09-10T04:05:31.489 INFO:tasks.workunit.client.0.smithi046.stdout:0/660: truncate d1/dc/d25/d33/d3/d29/f38 756485 0 2024-09-10T04:05:31.490 INFO:tasks.workunit.client.0.smithi046.stdout:1/633: symlink d3/dc/d34/ld4 0 2024-09-10T04:05:31.490 INFO:tasks.workunit.client.0.smithi046.stdout:7/891: link d5/d9/d43/db4/c115 d5/d6/dbe/dd7/c13c 0 2024-09-10T04:05:31.490 INFO:tasks.workunit.client.0.smithi046.stdout:8/841: fdatasync d1/d29/d17/d1f/d46/d6a/d82/d99/fa5 0 2024-09-10T04:05:31.492 INFO:tasks.workunit.client.0.smithi046.stdout:2/803: mknod d2/d86/d9c/dbc/ded/c11b 0 2024-09-10T04:05:31.497 INFO:tasks.workunit.client.0.smithi046.stdout:0/661: mkdir d1/dc/d49/db2/dcb 0 2024-09-10T04:05:31.498 INFO:tasks.workunit.client.0.smithi046.stdout:2/804: fsync d2/d8/d65/f7d 0 2024-09-10T04:05:31.501 INFO:tasks.workunit.client.0.smithi046.stdout:8/842: symlink d1/d29/d17/d1f/d46/d67/l118 0 2024-09-10T04:05:31.503 INFO:tasks.workunit.client.0.smithi046.stdout:8/843: write d1/d29/d19/db7/fd5 [2309283,7020] 0 2024-09-10T04:05:31.505 INFO:tasks.workunit.client.1.smithi070.stdout:4/841: symlink de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/d10d/df2/l11c 0 2024-09-10T04:05:31.505 INFO:tasks.workunit.client.1.smithi070.stdout:1/777: mknod d3/d90/dcb/cff 0 2024-09-10T04:05:31.505 INFO:tasks.workunit.client.1.smithi070.stdout:7/942: dread - d2/d4e/f74 zero size 2024-09-10T04:05:31.506 INFO:tasks.workunit.client.1.smithi070.stdout:8/943: rename d4/fe9 to d4/d16/dcc/d122/d126/f143 0 2024-09-10T04:05:31.506 INFO:tasks.workunit.client.0.smithi046.stdout:6/722: dread da/db/d10/d27/f36 [0,4194304] 0 2024-09-10T04:05:31.507 INFO:tasks.workunit.client.1.smithi070.stdout:1/778: chown d3/d8/d12/caa 0 1 2024-09-10T04:05:31.507 INFO:tasks.workunit.client.1.smithi070.stdout:9/909: write d9/d59/d2a/d3f/f128 [879273,112625] 0 2024-09-10T04:05:31.508 INFO:tasks.workunit.client.1.smithi070.stdout:1/779: dread - d3/d8/d27/d7a/df4/ffe zero size 2024-09-10T04:05:31.508 INFO:tasks.workunit.client.0.smithi046.stdout:6/723: readlink da/db/d11/d13/d1e/d23/d54/lb7 0 2024-09-10T04:05:31.509 INFO:tasks.workunit.client.0.smithi046.stdout:2/805: dread d2/d6/d23/d42/db4/f105 [0,4194304] 0 2024-09-10T04:05:31.510 INFO:tasks.workunit.client.0.smithi046.stdout:1/634: getdents d3/da1 0 2024-09-10T04:05:31.514 INFO:tasks.workunit.client.1.smithi070.stdout:8/944: fsync d4/fcd 0 2024-09-10T04:05:31.515 INFO:tasks.workunit.client.0.smithi046.stdout:7/892: dwrite d5/d6/db/f1d [0,4194304] 0 2024-09-10T04:05:31.516 INFO:tasks.workunit.client.1.smithi070.stdout:9/910: read d9/d59/d2a/d3f/f86 [285058,92005] 0 2024-09-10T04:05:31.519 INFO:tasks.workunit.client.1.smithi070.stdout:5/932: link d17/d24/f2f d17/d6f/d32/d90/d99/f13e 0 2024-09-10T04:05:31.520 INFO:tasks.workunit.client.0.smithi046.stdout:6/724: symlink da/de3/lee 0 2024-09-10T04:05:31.520 INFO:tasks.workunit.client.1.smithi070.stdout:5/933: chown f12 261 1 2024-09-10T04:05:31.520 INFO:tasks.workunit.client.0.smithi046.stdout:6/725: readlink da/db/d11/d13/d1e/d23/l8d 0 2024-09-10T04:05:31.521 INFO:tasks.workunit.client.1.smithi070.stdout:7/943: write d2/d66/f78 [5582271,80931] 0 2024-09-10T04:05:31.521 INFO:tasks.workunit.client.0.smithi046.stdout:6/726: chown da/db/d11 14414113 1 2024-09-10T04:05:31.522 INFO:tasks.workunit.client.0.smithi046.stdout:1/635: creat d3/dc/d34/d58/fd5 x:0 0 0 2024-09-10T04:05:31.523 INFO:tasks.workunit.client.0.smithi046.stdout:8/844: getdents d1/d29/d17/d1f/d46/dc9 0 2024-09-10T04:05:31.524 INFO:tasks.workunit.client.0.smithi046.stdout:7/893: symlink d5/d6/d32/l13d 0 2024-09-10T04:05:31.524 INFO:tasks.workunit.client.1.smithi070.stdout:4/842: dwrite de/d13/d5b/da0/fb4 [0,4194304] 0 2024-09-10T04:05:31.525 INFO:tasks.workunit.client.0.smithi046.stdout:0/662: getdents d1/dc/d25/d33/d4 0 2024-09-10T04:05:31.527 INFO:tasks.workunit.client.1.smithi070.stdout:8/945: unlink d4/d16/dcc/fe8 0 2024-09-10T04:05:31.528 INFO:tasks.workunit.client.0.smithi046.stdout:2/806: dread d2/d6/d9d/fa5 [0,4194304] 0 2024-09-10T04:05:31.529 INFO:tasks.workunit.client.0.smithi046.stdout:6/727: rename da/f40 to da/db/d11/d13/d1e/d23/d86/dbf/fef 0 2024-09-10T04:05:31.530 INFO:tasks.workunit.client.0.smithi046.stdout:1/636: stat d3/d27/d5d/d60/lcc 0 2024-09-10T04:05:31.532 INFO:tasks.workunit.client.0.smithi046.stdout:1/637: fdatasync d3/dc/d21/fc5 0 2024-09-10T04:05:31.532 INFO:tasks.workunit.client.0.smithi046.stdout:6/728: truncate da/db/d11/d9f/fed 126106 0 2024-09-10T04:05:31.534 INFO:tasks.workunit.client.0.smithi046.stdout:7/894: creat d5/d6/d62/f13e x:0 0 0 2024-09-10T04:05:31.535 INFO:tasks.workunit.client.1.smithi070.stdout:5/934: unlink d17/d6f/l135 0 2024-09-10T04:05:31.536 INFO:tasks.workunit.client.0.smithi046.stdout:7/895: dread - d5/d6/db/dd/d24/db7/f121 zero size 2024-09-10T04:05:31.536 INFO:tasks.workunit.client.0.smithi046.stdout:8/845: rmdir d1/d29/d17/d1f/d2a/df6/d102 39 2024-09-10T04:05:31.536 INFO:tasks.workunit.client.0.smithi046.stdout:5/943: sync 2024-09-10T04:05:31.536 INFO:tasks.workunit.client.0.smithi046.stdout:4/958: sync 2024-09-10T04:05:31.538 INFO:tasks.workunit.client.1.smithi070.stdout:1/780: truncate d3/d8/fc8 3485239 0 2024-09-10T04:05:31.541 INFO:tasks.workunit.client.0.smithi046.stdout:5/944: chown d7/d11/d14/d45/c55 0 1 2024-09-10T04:05:31.543 INFO:tasks.workunit.client.0.smithi046.stdout:6/729: dwrite da/db/d11/d13/d1e/f3e [0,4194304] 0 2024-09-10T04:05:31.545 INFO:tasks.workunit.client.0.smithi046.stdout:5/945: write d7/d11/d14/d45/f6d [4532427,11738] 0 2024-09-10T04:05:31.545 INFO:tasks.workunit.client.1.smithi070.stdout:4/843: fdatasync de/d13/d18/d5c/fd7 0 2024-09-10T04:05:31.547 INFO:tasks.workunit.client.0.smithi046.stdout:6/730: stat da/db/d11/d2b/d2f/d3b 0 2024-09-10T04:05:31.547 INFO:tasks.workunit.client.0.smithi046.stdout:5/946: stat d7/d11/d14/d2f/d2b/d6a/fa1 0 2024-09-10T04:05:31.550 INFO:tasks.workunit.client.1.smithi070.stdout:3/865: sync 2024-09-10T04:05:31.550 INFO:tasks.workunit.client.0.smithi046.stdout:7/896: dwrite d5/d6/d55/ff6 [0,4194304] 0 2024-09-10T04:05:31.550 INFO:tasks.workunit.client.1.smithi070.stdout:3/866: stat f7 0 2024-09-10T04:05:31.553 INFO:tasks.workunit.client.1.smithi070.stdout:4/844: dread de/d13/d5b/d7b/dd5/d85/d10e/f57 [0,4194304] 0 2024-09-10T04:05:31.553 INFO:tasks.workunit.client.1.smithi070.stdout:7/944: write d2/dc8/d119/d7e/f93 [3692682,91008] 0 2024-09-10T04:05:31.557 INFO:tasks.workunit.client.1.smithi070.stdout:5/935: rmdir d17/d19/d67 39 2024-09-10T04:05:31.557 INFO:tasks.workunit.client.0.smithi046.stdout:4/959: unlink d3/d12/d17/de1/ld1 0 2024-09-10T04:05:31.560 INFO:tasks.workunit.client.1.smithi070.stdout:7/945: dwrite d2/d7/d9/dd5/fe2 [0,4194304] 0 2024-09-10T04:05:31.563 INFO:tasks.workunit.client.1.smithi070.stdout:2/822: rename d15/d18/f1c to d15/d18/d29/d2a/d2e/d32/d9b/f12d 0 2024-09-10T04:05:31.564 INFO:tasks.workunit.client.0.smithi046.stdout:8/846: creat d1/d29/d17/d1f/d46/d6a/d82/dc2/f119 x:0 0 0 2024-09-10T04:05:31.568 INFO:tasks.workunit.client.0.smithi046.stdout:0/663: rmdir d1/dc/d25/dc4 0 2024-09-10T04:05:31.569 INFO:tasks.workunit.client.1.smithi070.stdout:9/911: dwrite d9/d59/d2a/d3f/f91 [0,4194304] 0 2024-09-10T04:05:31.569 INFO:tasks.workunit.client.1.smithi070.stdout:9/912: readlink d9/d7a/l53 0 2024-09-10T04:05:31.571 INFO:tasks.workunit.client.0.smithi046.stdout:6/731: read da/db/d11/d13/d1e/f82 [2636223,17671] 0 2024-09-10T04:05:31.572 INFO:tasks.workunit.client.0.smithi046.stdout:2/807: dwrite d2/d6/d15/fbb [0,4194304] 0 2024-09-10T04:05:31.572 INFO:tasks.workunit.client.1.smithi070.stdout:8/946: truncate d4/d16/dcc/d122/d126/f143 780862 0 2024-09-10T04:05:31.574 INFO:tasks.workunit.client.0.smithi046.stdout:2/808: write d2/d49/f101 [142565,105162] 0 2024-09-10T04:05:31.574 INFO:tasks.workunit.client.1.smithi070.stdout:4/845: chown de/c10 212025 1 2024-09-10T04:05:31.579 INFO:tasks.workunit.client.0.smithi046.stdout:1/638: creat d3/dc/d34/fd6 x:0 0 0 2024-09-10T04:05:31.583 INFO:tasks.workunit.client.0.smithi046.stdout:6/732: dwrite da/db/d11/d13/d1e/fdc [0,4194304] 0 2024-09-10T04:05:31.584 INFO:tasks.workunit.client.0.smithi046.stdout:0/664: mknod d1/dc/d25/d33/d3/dd/d55/d77/da6/ccc 0 2024-09-10T04:05:31.585 INFO:tasks.workunit.client.0.smithi046.stdout:1/639: read d3/d27/d2a/f7e [1995021,59802] 0 2024-09-10T04:05:31.586 INFO:tasks.workunit.client.0.smithi046.stdout:0/665: fdatasync d1/dc/d25/d33/d4/f36 0 2024-09-10T04:05:31.591 INFO:tasks.workunit.client.0.smithi046.stdout:2/809: creat d2/d49/f11c x:0 0 0 2024-09-10T04:05:31.591 INFO:tasks.workunit.client.0.smithi046.stdout:7/897: rename d5/d6/dbe/dd7/c13c to d5/d6/d55/d7d/c13f 0 2024-09-10T04:05:31.594 INFO:tasks.workunit.client.1.smithi070.stdout:9/913: mknod d9/d1c/d34/d8b/d11d/dc4/dda/de0/d90/df0/df3/df4/c12a 0 2024-09-10T04:05:31.597 INFO:tasks.workunit.client.0.smithi046.stdout:1/640: dwrite d3/da1/f2b [0,4194304] 0 2024-09-10T04:05:31.599 INFO:tasks.workunit.client.1.smithi070.stdout:4/846: mknod de/d13/d5b/d7b/dd5/d85/d10e/d77/c11d 0 2024-09-10T04:05:31.599 INFO:tasks.workunit.client.0.smithi046.stdout:8/847: dread d1/d29/d19/d36/f3c [0,4194304] 0 2024-09-10T04:05:31.600 INFO:tasks.workunit.client.1.smithi070.stdout:4/847: chown de/d13/d5b/d7b/dd5/d85 276027 1 2024-09-10T04:05:31.601 INFO:tasks.workunit.client.1.smithi070.stdout:4/848: dread - de/d13/d5b/d7b/dd5/d85/d10e/d48/ff5 zero size 2024-09-10T04:05:31.602 INFO:tasks.workunit.client.0.smithi046.stdout:5/947: link d7/d11/d3f/d5a/cb7 d7/d11/d3f/d53/d99/c144 0 2024-09-10T04:05:31.603 INFO:tasks.workunit.client.1.smithi070.stdout:1/781: truncate d3/d8/d27/fac 2271024 0 2024-09-10T04:05:31.606 INFO:tasks.workunit.client.1.smithi070.stdout:3/867: rename db/d1c/fcf to db/dcc/f129 0 2024-09-10T04:05:31.607 INFO:tasks.workunit.client.0.smithi046.stdout:6/733: rename da/fd to da/d96/ff0 0 2024-09-10T04:05:31.609 INFO:tasks.workunit.client.0.smithi046.stdout:4/960: link d3/d12/d17/d29/dba/fd9 d3/d12/dec/f147 0 2024-09-10T04:05:31.611 INFO:tasks.workunit.client.1.smithi070.stdout:9/914: unlink d9/d1c/l4f 0 2024-09-10T04:05:31.612 INFO:tasks.workunit.client.0.smithi046.stdout:4/961: read d3/d12/d17/d29/ddd/f12b [1703783,116657] 0 2024-09-10T04:05:31.613 INFO:tasks.workunit.client.1.smithi070.stdout:9/915: write d9/d59/d2a/d3f/fe4 [3597337,62694] 0 2024-09-10T04:05:31.614 INFO:tasks.workunit.client.0.smithi046.stdout:1/641: mkdir d3/d27/d5d/d60/dd7 0 2024-09-10T04:05:31.615 INFO:tasks.workunit.client.0.smithi046.stdout:8/848: creat d1/d29/d17/d1f/d46/d6a/d82/dc2/f11a x:0 0 0 2024-09-10T04:05:31.616 INFO:tasks.workunit.client.1.smithi070.stdout:5/936: rmdir d17/d6f/d139 0 2024-09-10T04:05:31.617 INFO:tasks.workunit.client.0.smithi046.stdout:0/666: mkdir d1/dc/d25/d33/d3/dcd 0 2024-09-10T04:05:31.618 INFO:tasks.workunit.client.1.smithi070.stdout:8/947: dwrite d4/dbf/d87/fa2 [0,4194304] 0 2024-09-10T04:05:31.619 INFO:tasks.workunit.client.1.smithi070.stdout:8/948: dread - d4/d16/dcc/d122/d126/f136 zero size 2024-09-10T04:05:31.619 INFO:tasks.workunit.client.1.smithi070.stdout:1/782: truncate d3/d8/d27/f93 1556102 0 2024-09-10T04:05:31.620 INFO:tasks.workunit.client.0.smithi046.stdout:1/642: dwrite d3/dc/d21/fc3 [0,4194304] 0 2024-09-10T04:05:31.621 INFO:tasks.workunit.client.0.smithi046.stdout:2/810: rename d2/d6/d15/c78 to d2/d8/d37/d10a/c11d 0 2024-09-10T04:05:31.623 INFO:tasks.workunit.client.0.smithi046.stdout:6/734: fsync da/db/d11/d2b/d5a/f97 0 2024-09-10T04:05:31.624 INFO:tasks.workunit.client.0.smithi046.stdout:2/811: write d2/d6/d15/f70 [588955,130657] 0 2024-09-10T04:05:31.624 INFO:tasks.workunit.client.1.smithi070.stdout:2/823: creat d15/f12e x:0 0 0 2024-09-10T04:05:31.626 INFO:tasks.workunit.client.0.smithi046.stdout:6/735: chown da/db/d10/d27/f88 247 1 2024-09-10T04:05:31.627 INFO:tasks.workunit.client.0.smithi046.stdout:2/812: fdatasync d2/d6/d23/d5d/fef 0 2024-09-10T04:05:31.628 INFO:tasks.workunit.client.0.smithi046.stdout:7/898: symlink d5/d9/d43/db4/def/d139/l140 0 2024-09-10T04:05:31.629 INFO:tasks.workunit.client.1.smithi070.stdout:4/849: symlink de/d13/d18/d26/l11e 0 2024-09-10T04:05:31.630 INFO:tasks.workunit.client.1.smithi070.stdout:4/850: stat de/d13/d18/d23/d47/d59 0 2024-09-10T04:05:31.631 INFO:tasks.workunit.client.0.smithi046.stdout:4/962: creat d3/d12/d17/d29/d58/db2/d5f/d85/f148 x:0 0 0 2024-09-10T04:05:31.631 INFO:tasks.workunit.client.1.smithi070.stdout:9/916: unlink d9/d1c/d34/d8b/d11d/d1b/lc2 0 2024-09-10T04:05:31.632 INFO:tasks.workunit.client.0.smithi046.stdout:8/849: creat d1/d29/d19/d36/d6e/d75/f11b x:0 0 0 2024-09-10T04:05:31.634 INFO:tasks.workunit.client.1.smithi070.stdout:5/937: creat d17/d19/d6a/d54/d57/da8/f13f x:0 0 0 2024-09-10T04:05:31.636 INFO:tasks.workunit.client.1.smithi070.stdout:5/938: write d17/d19/d6a/d54/de1/dfe/d93/f107 [930209,31792] 0 2024-09-10T04:05:31.637 INFO:tasks.workunit.client.1.smithi070.stdout:7/946: rename d2/d7/d9/d18/d3c/d52/lcd to d2/dc8/d119/d7e/l14f 0 2024-09-10T04:05:31.639 INFO:tasks.workunit.client.0.smithi046.stdout:0/667: truncate d1/dc/d25/d33/d1a/f31 2582007 0 2024-09-10T04:05:31.642 INFO:tasks.workunit.client.0.smithi046.stdout:1/643: rmdir d3/d27/d2a 39 2024-09-10T04:05:31.644 INFO:tasks.workunit.client.1.smithi070.stdout:7/947: dwrite d2/dc5/f114 [0,4194304] 0 2024-09-10T04:05:31.646 INFO:tasks.workunit.client.1.smithi070.stdout:4/851: dread de/d13/f38 [0,4194304] 0 2024-09-10T04:05:31.649 INFO:tasks.workunit.client.1.smithi070.stdout:4/852: chown de/d13/d5b/d7b/dd5/d85/d10e/l109 1 1 2024-09-10T04:05:31.650 INFO:tasks.workunit.client.1.smithi070.stdout:9/917: dread d9/d1c/d34/d8b/d11d/dc4/de1/f67 [0,4194304] 0 2024-09-10T04:05:31.655 INFO:tasks.workunit.client.1.smithi070.stdout:8/949: dread d4/d16/d73/d96/fb0 [0,4194304] 0 2024-09-10T04:05:31.659 INFO:tasks.workunit.client.0.smithi046.stdout:1/644: fsync d3/d9b/f8d 0 2024-09-10T04:05:31.660 INFO:tasks.workunit.client.0.smithi046.stdout:1/645: readlink d3/dc/d21/l64 0 2024-09-10T04:05:31.664 INFO:tasks.workunit.client.1.smithi070.stdout:2/824: mknod d15/d18/d29/d2a/d2e/da6/dc0/dca/c12f 0 2024-09-10T04:05:31.665 INFO:tasks.workunit.client.0.smithi046.stdout:7/899: dread d5/d9/d12/d72/d77/de0/d9f/f11e [4194304,4194304] 0 2024-09-10T04:05:31.666 INFO:tasks.workunit.client.1.smithi070.stdout:2/825: truncate d15/d18/d29/d2a/d2e/da6/ff0 370762 0 2024-09-10T04:05:31.669 INFO:tasks.workunit.client.1.smithi070.stdout:4/853: unlink de/d13/f1a 0 2024-09-10T04:05:31.671 INFO:tasks.workunit.client.1.smithi070.stdout:5/939: symlink d17/d19/d67/l140 0 2024-09-10T04:05:31.672 INFO:tasks.workunit.client.1.smithi070.stdout:5/940: write d17/d24/d84/d9b/fcb [4620078,30175] 0 2024-09-10T04:05:31.673 INFO:tasks.workunit.client.1.smithi070.stdout:5/941: chown d17/d6f/d32/d90/l11c 12955107 1 2024-09-10T04:05:31.674 INFO:tasks.workunit.client.1.smithi070.stdout:9/918: unlink d9/d1c/d34/d8b/dcc/f9d 0 2024-09-10T04:05:31.676 INFO:tasks.workunit.client.1.smithi070.stdout:3/868: symlink db/d1c/d25/d28/dd3/de5/l12a 0 2024-09-10T04:05:31.678 INFO:tasks.workunit.client.0.smithi046.stdout:2/813: creat d2/d6/d23/d5d/d77/d7b/f11e x:0 0 0 2024-09-10T04:05:31.679 INFO:tasks.workunit.client.1.smithi070.stdout:8/950: mkdir d4/d39/dfc/d43/d144 0 2024-09-10T04:05:31.681 INFO:tasks.workunit.client.0.smithi046.stdout:0/668: link d1/d13/d37/d75/d94/d9a/cc1 d1/dc/d25/d33/d3/dd/cce 0 2024-09-10T04:05:31.683 INFO:tasks.workunit.client.0.smithi046.stdout:0/669: write d1/d13/d37/d75/f7f [2432016,59989] 0 2024-09-10T04:05:31.686 INFO:tasks.workunit.client.0.smithi046.stdout:6/736: getdents da/db/d11/d13/d1e 0 2024-09-10T04:05:31.687 INFO:tasks.workunit.client.0.smithi046.stdout:2/814: symlink d2/d6/d23/d5d/d77/d7b/dc4/l11f 0 2024-09-10T04:05:31.687 INFO:tasks.workunit.client.1.smithi070.stdout:4/854: rmdir de/d13/d5b/d7b/dea 39 2024-09-10T04:05:31.689 INFO:tasks.workunit.client.0.smithi046.stdout:7/900: creat d5/d9/d12/d72/d77/de0/d9f/d126/f141 x:0 0 0 2024-09-10T04:05:31.694 INFO:tasks.workunit.client.0.smithi046.stdout:1/646: creat d3/d27/fd8 x:0 0 0 2024-09-10T04:05:31.695 INFO:tasks.workunit.client.0.smithi046.stdout:7/901: dwrite d5/d9/d12/d72/d77/de0/fe3 [0,4194304] 0 2024-09-10T04:05:31.697 INFO:tasks.workunit.client.0.smithi046.stdout:0/670: rmdir d1/dc/d25/d33/d3 39 2024-09-10T04:05:31.698 INFO:tasks.workunit.client.1.smithi070.stdout:5/942: rmdir d17/d19/d12d 39 2024-09-10T04:05:31.699 INFO:tasks.workunit.client.1.smithi070.stdout:5/943: dread - d17/d6f/d32/d90/da5/d120/f132 zero size 2024-09-10T04:05:31.709 INFO:tasks.workunit.client.0.smithi046.stdout:0/671: mkdir d1/d13/d37/d75/d94/d9a/dcf 0 2024-09-10T04:05:31.710 INFO:tasks.workunit.client.0.smithi046.stdout:2/815: truncate d2/d49/f9b 23335 0 2024-09-10T04:05:31.711 INFO:tasks.workunit.client.0.smithi046.stdout:2/816: chown d2/d86/d9c/dbc 17 1 2024-09-10T04:05:31.711 INFO:tasks.workunit.client.0.smithi046.stdout:1/647: mknod d3/d27/cd9 0 2024-09-10T04:05:31.713 INFO:tasks.workunit.client.0.smithi046.stdout:6/737: creat da/db/d11/d2b/d2f/ff1 x:0 0 0 2024-09-10T04:05:31.714 INFO:tasks.workunit.client.0.smithi046.stdout:0/672: creat d1/d13/d37/d75/fd0 x:0 0 0 2024-09-10T04:05:31.715 INFO:tasks.workunit.client.0.smithi046.stdout:1/648: getdents d3/d27/d5d/d60/dd7 0 2024-09-10T04:05:31.716 INFO:tasks.workunit.client.0.smithi046.stdout:6/738: symlink da/db/d11/d13/d1e/lf2 0 2024-09-10T04:05:31.718 INFO:tasks.workunit.client.0.smithi046.stdout:6/739: chown da/db/d10/d27 253968 1 2024-09-10T04:05:31.718 INFO:tasks.workunit.client.1.smithi070.stdout:6/884: sync 2024-09-10T04:05:31.719 INFO:tasks.workunit.client.1.smithi070.stdout:8/951: rmdir d4/dbf 39 2024-09-10T04:05:31.720 INFO:tasks.workunit.client.0.smithi046.stdout:2/817: dwrite d2/d6/d23/d5d/d77/d7b/dc4/ff9 [0,4194304] 0 2024-09-10T04:05:31.720 INFO:tasks.workunit.client.1.smithi070.stdout:7/948: rmdir d2/dc8/d119/d7e/d12d/d14d 0 2024-09-10T04:05:31.720 INFO:tasks.workunit.client.1.smithi070.stdout:1/783: getdents d3/d8/d12/d61 0 2024-09-10T04:05:31.720 INFO:tasks.workunit.client.0.smithi046.stdout:1/649: symlink d3/dc/d21/d3a/d90/dbf/lda 0 2024-09-10T04:05:31.721 INFO:tasks.workunit.client.0.smithi046.stdout:6/740: write da/db/d10/fe2 [624602,42116] 0 2024-09-10T04:05:31.722 INFO:tasks.workunit.client.1.smithi070.stdout:5/944: creat d17/d19/d6a/dfb/f141 x:0 0 0 2024-09-10T04:05:31.722 INFO:tasks.workunit.client.1.smithi070.stdout:2/826: dwrite d15/d18/d56/d7c/d55/f11a [0,4194304] 0 2024-09-10T04:05:31.723 INFO:tasks.workunit.client.1.smithi070.stdout:2/827: chown d15/d18/d56 45 1 2024-09-10T04:05:31.724 INFO:tasks.workunit.client.0.smithi046.stdout:0/673: dread d1/dc/d25/d33/d3/d29/f38 [0,4194304] 0 2024-09-10T04:05:31.726 INFO:tasks.workunit.client.0.smithi046.stdout:0/674: readlink d1/dc/d25/d33/d3/d29/l59 0 2024-09-10T04:05:31.730 INFO:tasks.workunit.client.0.smithi046.stdout:2/818: truncate d2/d8/d37/f6c 1236312 0 2024-09-10T04:05:31.732 INFO:tasks.workunit.client.0.smithi046.stdout:1/650: truncate d3/da1/f43 2092118 0 2024-09-10T04:05:31.734 INFO:tasks.workunit.client.0.smithi046.stdout:0/675: mknod d1/d13/d37/cd1 0 2024-09-10T04:05:31.736 INFO:tasks.workunit.client.0.smithi046.stdout:2/819: creat d2/d6/d9d/f120 x:0 0 0 2024-09-10T04:05:31.739 INFO:tasks.workunit.client.0.smithi046.stdout:7/902: dwrite d5/d6/db/dd/d24/d76/d9c/fa3 [0,4194304] 0 2024-09-10T04:05:31.748 INFO:tasks.workunit.client.0.smithi046.stdout:2/820: unlink d2/d6/d23/d5d/f8f 0 2024-09-10T04:05:31.748 INFO:tasks.workunit.client.0.smithi046.stdout:7/903: dwrite d5/d9/ffa [0,4194304] 0 2024-09-10T04:05:31.750 INFO:tasks.workunit.client.0.smithi046.stdout:7/904: write d5/fbf [1850566,120762] 0 2024-09-10T04:05:31.757 INFO:tasks.workunit.client.0.smithi046.stdout:6/741: readlink da/db/d11/d13/d1e/lb5 0 2024-09-10T04:05:31.758 INFO:tasks.workunit.client.0.smithi046.stdout:6/742: dread - da/db/d10/fe5 zero size 2024-09-10T04:05:31.763 INFO:tasks.workunit.client.0.smithi046.stdout:1/651: write d3/dc/d34/f76 [1139714,35359] 0 2024-09-10T04:05:31.764 INFO:tasks.workunit.client.0.smithi046.stdout:0/676: write d1/dc/d25/d33/d3/d29/d2d/f32 [1955597,60294] 0 2024-09-10T04:05:31.767 INFO:tasks.workunit.client.0.smithi046.stdout:6/743: dwrite da/db/d11/d2b/f70 [0,4194304] 0 2024-09-10T04:05:31.768 INFO:tasks.workunit.client.0.smithi046.stdout:1/652: symlink d3/da1/d1f/ldb 0 2024-09-10T04:05:31.770 INFO:tasks.workunit.client.0.smithi046.stdout:0/677: creat d1/dc/d49/db2/fd2 x:0 0 0 2024-09-10T04:05:31.771 INFO:tasks.workunit.client.0.smithi046.stdout:1/653: fsync d3/dc/d21/fc3 0 2024-09-10T04:05:31.775 INFO:tasks.workunit.client.0.smithi046.stdout:6/744: creat da/db/d11/d13/d1e/d23/d86/dbf/ff3 x:0 0 0 2024-09-10T04:05:31.777 INFO:tasks.workunit.client.0.smithi046.stdout:0/678: creat d1/d13/d37/d75/d94/fd3 x:0 0 0 2024-09-10T04:05:31.777 INFO:tasks.workunit.client.0.smithi046.stdout:2/821: dwrite d2/d6/d23/d42/f53 [0,4194304] 0 2024-09-10T04:05:31.778 INFO:tasks.workunit.client.0.smithi046.stdout:1/654: mknod d3/d27/d5d/d60/cdc 0 2024-09-10T04:05:31.781 INFO:tasks.workunit.client.0.smithi046.stdout:1/655: chown d3/d27/fb7 37 1 2024-09-10T04:05:31.785 INFO:tasks.workunit.client.0.smithi046.stdout:2/822: creat d2/d6/d23/dd5/f121 x:0 0 0 2024-09-10T04:05:31.787 INFO:tasks.workunit.client.0.smithi046.stdout:7/905: read d5/d9/d12/dc5/f102 [402851,85400] 0 2024-09-10T04:05:31.787 INFO:tasks.workunit.client.0.smithi046.stdout:2/823: fsync d2/d6/f61 0 2024-09-10T04:05:31.790 INFO:tasks.workunit.client.0.smithi046.stdout:1/656: dwrite d3/da1/d1f/f29 [0,4194304] 0 2024-09-10T04:05:31.791 INFO:tasks.workunit.client.0.smithi046.stdout:1/657: chown d3/d27/l4f 33205894 1 2024-09-10T04:05:31.792 INFO:tasks.workunit.client.0.smithi046.stdout:7/906: unlink d5/d6/d32/f13b 0 2024-09-10T04:05:31.792 INFO:tasks.workunit.client.0.smithi046.stdout:0/679: write d1/dc/d25/d33/d3/dd/d55/d4d/f71 [3445963,128074] 0 2024-09-10T04:05:31.793 INFO:tasks.workunit.client.0.smithi046.stdout:7/907: fdatasync d5/f6b 0 2024-09-10T04:05:31.793 INFO:tasks.workunit.client.0.smithi046.stdout:2/824: creat d2/d8/d109/f122 x:0 0 0 2024-09-10T04:05:31.798 INFO:tasks.workunit.client.0.smithi046.stdout:0/680: write d1/dc/f7b [604946,66739] 0 2024-09-10T04:05:31.801 INFO:tasks.workunit.client.0.smithi046.stdout:7/908: unlink d5/d6/db/dd/d24/d76/f12e 0 2024-09-10T04:05:31.803 INFO:tasks.workunit.client.0.smithi046.stdout:0/681: rmdir d1/dc/d25/d33/d3 39 2024-09-10T04:05:31.803 INFO:tasks.workunit.client.1.smithi070.stdout:9/919: creat d9/d59/f12b x:0 0 0 2024-09-10T04:05:31.803 INFO:tasks.workunit.client.0.smithi046.stdout:1/658: dwrite d3/dc/d34/d58/fd5 [0,4194304] 0 2024-09-10T04:05:31.804 INFO:tasks.workunit.client.1.smithi070.stdout:6/885: mknod d0/d3/d18/d1b/d17/d54/dc4/dd2/dde/c115 0 2024-09-10T04:05:31.806 INFO:tasks.workunit.client.0.smithi046.stdout:1/659: write d3/d27/f42 [1580727,115085] 0 2024-09-10T04:05:31.807 INFO:tasks.workunit.client.1.smithi070.stdout:5/945: rmdir d17/d19/d6a/d54/d57/d63/dcd 39 2024-09-10T04:05:31.811 INFO:tasks.workunit.client.1.smithi070.stdout:1/784: truncate d3/d8/d13/d3f/f78 339750 0 2024-09-10T04:05:31.813 INFO:tasks.workunit.client.1.smithi070.stdout:2/828: fdatasync d15/d18/d56/d7c/d55/d6d/fba 0 2024-09-10T04:05:31.813 INFO:tasks.workunit.client.0.smithi046.stdout:1/660: rename d3/da1/d1f/f61 to d3/d27/d5d/d60/fdd 0 2024-09-10T04:05:31.814 INFO:tasks.workunit.client.1.smithi070.stdout:9/920: truncate d9/d59/d2a/d3f/f86 1011598 0 2024-09-10T04:05:31.816 INFO:tasks.workunit.client.1.smithi070.stdout:8/952: dwrite d4/dcf/f9a [0,4194304] 0 2024-09-10T04:05:31.817 INFO:tasks.workunit.client.0.smithi046.stdout:3/786: sync 2024-09-10T04:05:31.817 INFO:tasks.workunit.client.0.smithi046.stdout:9/828: sync 2024-09-10T04:05:31.819 INFO:tasks.workunit.client.0.smithi046.stdout:7/909: dwrite d5/d9/d12/d72/d77/da7/f11b [0,4194304] 0 2024-09-10T04:05:31.820 INFO:tasks.workunit.client.1.smithi070.stdout:4/855: dwrite de/d13/d5b/d7b/fe2 [0,4194304] 0 2024-09-10T04:05:31.820 INFO:tasks.workunit.client.0.smithi046.stdout:0/682: dwrite d1/d13/d37/d75/d94/fd3 [0,4194304] 0 2024-09-10T04:05:31.822 INFO:tasks.workunit.client.1.smithi070.stdout:1/785: mkdir d3/d8/d13/d67/d83/def/d100 0 2024-09-10T04:05:31.825 INFO:tasks.workunit.client.1.smithi070.stdout:5/946: mkdir d17/d19/d6a/d54/de1/d11f/d142 0 2024-09-10T04:05:31.829 INFO:tasks.workunit.client.1.smithi070.stdout:9/921: symlink d9/d1c/d34/l12c 0 2024-09-10T04:05:31.829 INFO:tasks.workunit.client.1.smithi070.stdout:7/949: dread d2/d7/d9/d18/d23/f47 [0,4194304] 0 2024-09-10T04:05:31.851 INFO:tasks.workunit.client.0.smithi046.stdout:2/825: dwrite d2/d6/d9d/db6/fb7 [0,4194304] 0 2024-09-10T04:05:31.851 INFO:tasks.workunit.client.0.smithi046.stdout:1/661: dread d3/dc/f23 [0,4194304] 0 2024-09-10T04:05:31.851 INFO:tasks.workunit.client.0.smithi046.stdout:3/787: creat d12/d5d/d71/d49/f110 x:0 0 0 2024-09-10T04:05:31.853 INFO:tasks.workunit.client.0.smithi046.stdout:1/662: fsync d3/dc/d34/d58/fd5 0 2024-09-10T04:05:31.855 INFO:tasks.workunit.client.0.smithi046.stdout:3/788: fdatasync d12/d5d/d71/d49/f4e 0 2024-09-10T04:05:31.856 INFO:tasks.workunit.client.0.smithi046.stdout:2/826: write d2/d6/d23/d55/f7e [59068,121719] 0 2024-09-10T04:05:31.856 INFO:tasks.workunit.client.1.smithi070.stdout:6/886: rename d0/d3/d18/d62/d70/f10f to d0/d3/d18/d62/f116 0 2024-09-10T04:05:31.857 INFO:tasks.workunit.client.0.smithi046.stdout:3/789: readlink d12/d13/d37/d94/dbf/lfa 0 2024-09-10T04:05:31.857 INFO:tasks.workunit.client.0.smithi046.stdout:0/683: mkdir d1/d13/d37/d75/d94/dd4 0 2024-09-10T04:05:31.859 INFO:tasks.workunit.client.1.smithi070.stdout:1/786: stat d3/d8/c2e 0 2024-09-10T04:05:31.860 INFO:tasks.workunit.client.0.smithi046.stdout:7/910: truncate d5/f35 2760939 0 2024-09-10T04:05:31.861 INFO:tasks.workunit.client.1.smithi070.stdout:4/856: mkdir de/d13/d18/d23/d47/d59/d79/d6a/dce/d11f 0 2024-09-10T04:05:31.864 INFO:tasks.workunit.client.1.smithi070.stdout:2/829: dwrite f10 [0,4194304] 0 2024-09-10T04:05:31.865 INFO:tasks.workunit.client.1.smithi070.stdout:5/947: mkdir d17/d24/d84/d143 0 2024-09-10T04:05:31.872 INFO:tasks.workunit.client.1.smithi070.stdout:8/953: dwrite d4/d16/d73/d139/d7a/f9d [0,4194304] 0 2024-09-10T04:05:31.872 INFO:tasks.workunit.client.0.smithi046.stdout:1/663: mkdir d3/dc/d21/dde 0 2024-09-10T04:05:31.874 INFO:tasks.workunit.client.0.smithi046.stdout:2/827: mkdir d2/d8/d37/dc2/d123 0 2024-09-10T04:05:31.876 INFO:tasks.workunit.client.0.smithi046.stdout:3/790: mknod d12/d13/d14/d1e/d2d/c111 0 2024-09-10T04:05:31.877 INFO:tasks.workunit.client.1.smithi070.stdout:1/787: creat d3/d8/d27/d7a/d94/f101 x:0 0 0 2024-09-10T04:05:31.880 INFO:tasks.workunit.client.0.smithi046.stdout:7/911: rename d5/d6/d32/dab/feb to d5/d6/db/dd/d4b/f142 0 2024-09-10T04:05:31.882 INFO:tasks.workunit.client.1.smithi070.stdout:5/948: mkdir d17/d24/d84/d144 0 2024-09-10T04:05:31.883 INFO:tasks.workunit.client.1.smithi070.stdout:4/857: read de/d13/d18/d23/f61 [1137182,5469] 0 2024-09-10T04:05:31.884 INFO:tasks.workunit.client.0.smithi046.stdout:2/828: readlink d2/d6/d23/d55/l6d 0 2024-09-10T04:05:31.885 INFO:tasks.workunit.client.0.smithi046.stdout:0/684: mknod d1/dc/d25/d33/d3/d29/d2d/d5e/cd5 0 2024-09-10T04:05:31.887 INFO:tasks.workunit.client.0.smithi046.stdout:3/791: rename d12/d5d/d71/d49/d5c/d9f to d12/d13/d37/d51/db5/d4b/d91/d112 0 2024-09-10T04:05:31.889 INFO:tasks.workunit.client.0.smithi046.stdout:9/829: getdents d4/d7/d1f/dce/dd4 0 2024-09-10T04:05:31.890 INFO:tasks.workunit.client.0.smithi046.stdout:7/912: truncate d5/d6/d32/dab/db6/f79 8453290 0 2024-09-10T04:05:31.898 INFO:tasks.workunit.client.1.smithi070.stdout:7/950: truncate d2/d66/f78 8014243 0 2024-09-10T04:05:31.898 INFO:tasks.workunit.client.1.smithi070.stdout:7/951: chown d2/d7/d9/d18/d23 14202154 1 2024-09-10T04:05:31.899 INFO:tasks.workunit.client.0.smithi046.stdout:9/830: chown d4/f6 150583 1 2024-09-10T04:05:31.899 INFO:tasks.workunit.client.1.smithi070.stdout:7/952: chown d2/d7/d9/d18/lb8 344 1 2024-09-10T04:05:31.900 INFO:tasks.workunit.client.1.smithi070.stdout:5/949: creat d17/d24/d84/f145 x:0 0 0 2024-09-10T04:05:31.901 INFO:tasks.workunit.client.1.smithi070.stdout:5/950: dread - d17/fd7 zero size 2024-09-10T04:05:31.901 INFO:tasks.workunit.client.1.smithi070.stdout:5/951: fsync d17/d6f/d32/d90/da5/d120/f132 0 2024-09-10T04:05:31.902 INFO:tasks.workunit.client.1.smithi070.stdout:5/952: write d17/d24/d84/fee [1312035,2073] 0 2024-09-10T04:05:31.904 INFO:tasks.workunit.client.1.smithi070.stdout:4/858: mknod de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/c120 0 2024-09-10T04:05:31.905 INFO:tasks.workunit.client.0.smithi046.stdout:3/792: link d12/d13/d37/d51/db5/d50/d7a/dc1/d96/ff0 d12/d13/d14/d1e/dd2/f113 0 2024-09-10T04:05:31.905 INFO:tasks.workunit.client.1.smithi070.stdout:1/788: write d3/d8/d13/d3f/f53 [285307,112212] 0 2024-09-10T04:05:31.906 INFO:tasks.workunit.client.1.smithi070.stdout:9/922: link d9/d1c/d34/d8b/d11d/dc4/dda/l120 d9/d1c/d34/d8b/d11d/l12d 0 2024-09-10T04:05:31.906 INFO:tasks.workunit.client.1.smithi070.stdout:4/859: chown de/d13/d43/c94 7052 1 2024-09-10T04:05:31.907 INFO:tasks.workunit.client.1.smithi070.stdout:8/954: mkdir d4/d16/dcc/d12e/da4/d140/d145 0 2024-09-10T04:05:31.908 INFO:tasks.workunit.client.1.smithi070.stdout:6/887: dwrite d0/d3/d1f/f46 [0,4194304] 0 2024-09-10T04:05:31.910 INFO:tasks.workunit.client.1.smithi070.stdout:7/953: dread - d2/d40/d6c/db2/db7/d104/f105 zero size 2024-09-10T04:05:31.910 INFO:tasks.workunit.client.0.smithi046.stdout:2/829: dread d2/d6/d23/d5d/fae [0,4194304] 0 2024-09-10T04:05:31.911 INFO:tasks.workunit.client.0.smithi046.stdout:1/664: dwrite d3/da1/f43 [0,4194304] 0 2024-09-10T04:05:31.913 INFO:tasks.workunit.client.0.smithi046.stdout:3/793: fdatasync d12/d13/d37/f45 0 2024-09-10T04:05:31.913 INFO:tasks.workunit.client.1.smithi070.stdout:5/953: creat d17/d19/d6a/d54/de1/dfe/d93/f146 x:0 0 0 2024-09-10T04:05:31.919 INFO:tasks.workunit.client.0.smithi046.stdout:2/830: chown d2/d6/d23/d5d/d77/d7b/dd8/f104 14 1 2024-09-10T04:05:31.920 INFO:tasks.workunit.client.0.smithi046.stdout:3/794: write d12/d13/d14/f36 [311398,113801] 0 2024-09-10T04:05:31.921 INFO:tasks.workunit.client.1.smithi070.stdout:1/789: chown d3/d8/d12/f45 58252 1 2024-09-10T04:05:31.921 INFO:tasks.workunit.client.0.smithi046.stdout:1/665: read d3/d62/fc9 [230452,125736] 0 2024-09-10T04:05:31.922 INFO:tasks.workunit.client.1.smithi070.stdout:3/869: sync 2024-09-10T04:05:31.924 INFO:tasks.workunit.client.0.smithi046.stdout:3/795: truncate d12/d13/d37/f100 294679 0 2024-09-10T04:05:31.924 INFO:tasks.workunit.client.0.smithi046.stdout:9/831: link d4/d3c/d4d/d80/c8c d4/d3c/c107 0 2024-09-10T04:05:31.924 INFO:tasks.workunit.client.1.smithi070.stdout:9/923: mkdir d9/d1c/d34/d8b/dcc/de7/d12e 0 2024-09-10T04:05:31.924 INFO:tasks.workunit.client.1.smithi070.stdout:9/924: write d9/f111 [931507,107792] 0 2024-09-10T04:05:31.929 INFO:tasks.workunit.client.0.smithi046.stdout:3/796: chown d12/d13/d37/d51/db5/d50/c9d 125 1 2024-09-10T04:05:31.929 INFO:tasks.workunit.client.1.smithi070.stdout:2/830: getdents d15/d2b 0 2024-09-10T04:05:31.931 INFO:tasks.workunit.client.0.smithi046.stdout:2/831: mknod d2/d86/d9c/c124 0 2024-09-10T04:05:31.932 INFO:tasks.workunit.client.0.smithi046.stdout:1/666: dwrite d3/d62/faa [0,4194304] 0 2024-09-10T04:05:31.932 INFO:tasks.workunit.client.0.smithi046.stdout:9/832: dread d4/d3c/d4d/d5c/d87/fbe [0,4194304] 0 2024-09-10T04:05:31.934 INFO:tasks.workunit.client.0.smithi046.stdout:3/797: chown d12/d13/d37/d51/db5/l28 1821 1 2024-09-10T04:05:31.934 INFO:tasks.workunit.client.1.smithi070.stdout:2/831: dwrite d15/d18/d29/d2a/d2e/d32/f82 [0,4194304] 0 2024-09-10T04:05:31.935 INFO:tasks.workunit.client.1.smithi070.stdout:2/832: truncate d15/d59/f115 136741 0 2024-09-10T04:05:31.936 INFO:tasks.workunit.client.1.smithi070.stdout:2/833: truncate d15/d18/d29/d2a/f58 6217685 0 2024-09-10T04:05:31.937 INFO:tasks.workunit.client.1.smithi070.stdout:6/888: rename d0/d3/d18/d1b/d17/d3f/d40/c101 to d0/d3/d18/d2a/c117 0 2024-09-10T04:05:31.939 INFO:tasks.workunit.client.0.smithi046.stdout:3/798: write d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fa6 [105733,86578] 0 2024-09-10T04:05:31.939 INFO:tasks.workunit.client.0.smithi046.stdout:7/913: dread d5/d6/d32/dab/db6/f79 [0,4194304] 0 2024-09-10T04:05:31.940 INFO:tasks.workunit.client.1.smithi070.stdout:5/954: symlink d17/d6f/l147 0 2024-09-10T04:05:31.941 INFO:tasks.workunit.client.1.smithi070.stdout:5/955: write d17/d24/d77/f127 [1025767,105795] 0 2024-09-10T04:05:31.943 INFO:tasks.workunit.client.1.smithi070.stdout:1/790: write d3/d8/d27/d7a/d94/f9e [4842803,56414] 0 2024-09-10T04:05:31.944 INFO:tasks.workunit.client.1.smithi070.stdout:4/860: write de/d13/d43/d88/fcf [2048343,11509] 0 2024-09-10T04:05:31.945 INFO:tasks.workunit.client.1.smithi070.stdout:4/861: chown de/d13/d5b 30045701 1 2024-09-10T04:05:31.947 INFO:tasks.workunit.client.0.smithi046.stdout:2/832: creat d2/d6/d23/d42/de6/f125 x:0 0 0 2024-09-10T04:05:31.947 INFO:tasks.workunit.client.1.smithi070.stdout:9/925: chown d9/d1c/d34/d8b/d11d/d1b/c50 865769972 1 2024-09-10T04:05:31.949 INFO:tasks.workunit.client.0.smithi046.stdout:9/833: truncate d4/d3c/f40 209878 0 2024-09-10T04:05:31.950 INFO:tasks.workunit.client.0.smithi046.stdout:3/799: dwrite d12/d13/d37/d51/f8e [0,4194304] 0 2024-09-10T04:05:31.953 INFO:tasks.workunit.client.1.smithi070.stdout:8/955: creat d4/d16/dcc/d12e/d135/d13a/f146 x:0 0 0 2024-09-10T04:05:31.954 INFO:tasks.workunit.client.0.smithi046.stdout:7/914: creat d5/d6/d39/d5f/f143 x:0 0 0 2024-09-10T04:05:31.956 INFO:tasks.workunit.client.1.smithi070.stdout:2/834: creat d15/d2b/d94/f130 x:0 0 0 2024-09-10T04:05:31.959 INFO:tasks.workunit.client.1.smithi070.stdout:7/954: rename d2/d3/dc/d5d/d41/dd6/de6/d14b to d2/d3/dc/d5d/d41/d150 0 2024-09-10T04:05:31.965 INFO:tasks.workunit.client.0.smithi046.stdout:2/833: mknod d2/d6/d23/d42/de6/dec/c126 0 2024-09-10T04:05:31.968 INFO:tasks.workunit.client.0.smithi046.stdout:3/800: mkdir d12/d13/d14/d1e/d2d/d39/d67/da0/d114 0 2024-09-10T04:05:31.970 INFO:tasks.workunit.client.1.smithi070.stdout:4/862: mknod de/d13/d5b/d7b/dd5/d85/c121 0 2024-09-10T04:05:31.971 INFO:tasks.workunit.client.1.smithi070.stdout:4/863: write de/d13/d5b/d7b/dd5/d85/d10e/d77/fc2 [164239,22279] 0 2024-09-10T04:05:31.978 INFO:tasks.workunit.client.0.smithi046.stdout:9/834: write d4/f79 [4935121,107282] 0 2024-09-10T04:05:31.978 INFO:tasks.workunit.client.1.smithi070.stdout:6/889: symlink d0/d3/d8c/l118 0 2024-09-10T04:05:31.979 INFO:tasks.workunit.client.0.smithi046.stdout:9/835: fsync d4/d7/d1f/dce/fec 0 2024-09-10T04:05:31.984 INFO:tasks.workunit.client.1.smithi070.stdout:6/890: dwrite d0/f57 [0,4194304] 0 2024-09-10T04:05:31.987 INFO:tasks.workunit.client.1.smithi070.stdout:5/956: symlink d17/d24/dcc/l148 0 2024-09-10T04:05:31.988 INFO:tasks.workunit.client.1.smithi070.stdout:1/791: creat d3/d8/d13/d67/f102 x:0 0 0 2024-09-10T04:05:31.989 INFO:tasks.workunit.client.1.smithi070.stdout:3/870: getdents db/d1c/d25/d28/d43 0 2024-09-10T04:05:31.991 INFO:tasks.workunit.client.1.smithi070.stdout:4/864: symlink de/d13/d5b/d7b/dea/l122 0 2024-09-10T04:05:31.992 INFO:tasks.workunit.client.1.smithi070.stdout:6/891: creat d0/d3/da4/dc0/f119 x:0 0 0 2024-09-10T04:05:31.992 INFO:tasks.workunit.client.1.smithi070.stdout:2/835: link d15/d18/d29/d2a/d2e/da6/l11b d15/d18/db2/l131 0 2024-09-10T04:05:31.995 INFO:tasks.workunit.client.0.smithi046.stdout:5/948: sync 2024-09-10T04:05:31.997 INFO:tasks.workunit.client.1.smithi070.stdout:3/871: symlink db/d1c/d25/d28/dd3/de5/d10f/db5/d113/l12b 0 2024-09-10T04:05:32.001 INFO:tasks.workunit.client.1.smithi070.stdout:6/892: fsync d0/d3/f16 0 2024-09-10T04:05:32.003 INFO:tasks.workunit.client.1.smithi070.stdout:7/955: getdents d2/d7/d9/d18/d23 0 2024-09-10T04:05:32.007 INFO:tasks.workunit.client.1.smithi070.stdout:5/957: mknod d17/d6f/d32/c149 0 2024-09-10T04:05:32.007 INFO:tasks.workunit.client.1.smithi070.stdout:5/958: readlink d17/d6f/l147 0 2024-09-10T04:05:32.013 INFO:tasks.workunit.client.1.smithi070.stdout:7/956: dread d2/d3/dc/d13/dbd/ff6 [0,4194304] 0 2024-09-10T04:05:32.013 INFO:tasks.workunit.client.1.smithi070.stdout:6/893: unlink d0/d3/d1f/d81/lf4 0 2024-09-10T04:05:32.014 INFO:tasks.workunit.client.1.smithi070.stdout:7/957: write d2/d3/dc/d79/fed [8964425,58557] 0 2024-09-10T04:05:32.015 INFO:tasks.workunit.client.1.smithi070.stdout:7/958: chown d2/d3/dc/d13/dfe/l4b 11095 1 2024-09-10T04:05:32.016 INFO:tasks.workunit.client.1.smithi070.stdout:7/959: chown d2/dc8/df9/d135 6 1 2024-09-10T04:05:32.017 INFO:tasks.workunit.client.1.smithi070.stdout:2/836: link d15/d18/d29/f7f d15/d18/db2/f132 0 2024-09-10T04:05:32.018 INFO:tasks.workunit.client.1.smithi070.stdout:2/837: chown d15/d18/d29/d2a/d2e/f87 1860 1 2024-09-10T04:05:32.018 INFO:tasks.workunit.client.1.smithi070.stdout:1/792: write d3/d8/d13/d67/fcd [894764,48652] 0 2024-09-10T04:05:32.024 INFO:tasks.workunit.client.1.smithi070.stdout:4/865: write de/d13/d5b/d7b/dd5/d85/d10e/d48/f7e [961865,91771] 0 2024-09-10T04:05:32.026 INFO:tasks.workunit.client.1.smithi070.stdout:2/838: dwrite d15/d18/d29/d2a/d2e/d32/fe5 [0,4194304] 0 2024-09-10T04:05:32.026 INFO:tasks.workunit.client.1.smithi070.stdout:6/894: mkdir d0/d3/d18/d1b/d17/d53/d11a 0 2024-09-10T04:05:32.028 INFO:tasks.workunit.client.0.smithi046.stdout:3/801: mkdir d12/d13/d37/dab/d115 0 2024-09-10T04:05:32.030 INFO:tasks.workunit.client.1.smithi070.stdout:5/959: dread d17/d19/fd1 [0,4194304] 0 2024-09-10T04:05:32.034 INFO:tasks.workunit.client.0.smithi046.stdout:9/836: mkdir d4/d44/d108 0 2024-09-10T04:05:32.036 INFO:tasks.workunit.client.1.smithi070.stdout:7/960: dwrite d2/d3/dc/d13/dbd/ff6 [4194304,4194304] 0 2024-09-10T04:05:32.038 INFO:tasks.workunit.client.0.smithi046.stdout:5/949: unlink d7/d11/d3f/d53/f47 0 2024-09-10T04:05:32.039 INFO:tasks.workunit.client.0.smithi046.stdout:5/950: fdatasync d7/d11/d3f/d5a/f130 0 2024-09-10T04:05:32.039 INFO:tasks.workunit.client.1.smithi070.stdout:3/872: dread db/d1c/d3e/f9d [0,4194304] 0 2024-09-10T04:05:32.045 INFO:tasks.workunit.client.0.smithi046.stdout:2/834: symlink d2/d8/d37/d95/l127 0 2024-09-10T04:05:32.047 INFO:tasks.workunit.client.1.smithi070.stdout:4/866: rename de/d13/d18/ld9 to de/d13/d5b/d7b/dd5/d85/l123 0 2024-09-10T04:05:32.050 INFO:tasks.workunit.client.0.smithi046.stdout:7/915: dwrite d5/d6/d32/dab/db8/fa6 [0,4194304] 0 2024-09-10T04:05:32.054 INFO:tasks.workunit.client.0.smithi046.stdout:1/667: link d3/da1/f69 d3/dc/d34/d51/fdf 0 2024-09-10T04:05:32.056 INFO:tasks.workunit.client.1.smithi070.stdout:1/793: creat d3/d8/d13/d3f/f103 x:0 0 0 2024-09-10T04:05:32.057 INFO:tasks.workunit.client.0.smithi046.stdout:3/802: mkdir d12/d13/d37/d51/db5/d4b/d116 0 2024-09-10T04:05:32.060 INFO:tasks.workunit.client.1.smithi070.stdout:7/961: rename d2/d7/d9/d18/d3c/d52/c118 to d2/dc8/d119/d7e/c151 0 2024-09-10T04:05:32.060 INFO:tasks.workunit.client.1.smithi070.stdout:7/962: chown d2/d7/d9/d18/d49/l5b 1870787361 1 2024-09-10T04:05:32.063 INFO:tasks.workunit.client.0.smithi046.stdout:4/963: sync 2024-09-10T04:05:32.067 INFO:tasks.workunit.client.0.smithi046.stdout:3/803: symlink d12/dd9/ddf/l117 0 2024-09-10T04:05:32.069 INFO:tasks.workunit.client.0.smithi046.stdout:9/837: dwrite d4/d7/d1f/d54/fb0 [0,4194304] 0 2024-09-10T04:05:32.070 INFO:tasks.workunit.client.0.smithi046.stdout:3/804: readlink d12/d13/d37/d51/db5/d50/l7f 0 2024-09-10T04:05:32.071 INFO:tasks.workunit.client.0.smithi046.stdout:9/838: write d4/d3c/d4d/d5c/d59/ffd [265793,35855] 0 2024-09-10T04:05:32.074 INFO:tasks.workunit.client.0.smithi046.stdout:2/835: dread d2/d6/f80 [0,4194304] 0 2024-09-10T04:05:32.074 INFO:tasks.workunit.client.0.smithi046.stdout:9/839: readlink d4/d3c/d4d/d80/lf7 0 2024-09-10T04:05:32.074 INFO:tasks.workunit.client.1.smithi070.stdout:6/895: link d0/d3/d18/d62/db8/fd0 d0/d3/da4/dc0/dea/f11b 0 2024-09-10T04:05:32.076 INFO:tasks.workunit.client.1.smithi070.stdout:5/960: getdents d17/d19/d35 0 2024-09-10T04:05:32.078 INFO:tasks.workunit.client.1.smithi070.stdout:1/794: rename d3/d8/d27/d7a/db4/le3 to d3/d8/d92/l104 0 2024-09-10T04:05:32.079 INFO:tasks.workunit.client.0.smithi046.stdout:7/916: creat d5/d9/d12/d72/d77/f144 x:0 0 0 2024-09-10T04:05:32.081 INFO:tasks.workunit.client.0.smithi046.stdout:5/951: dwrite d7/d11/d3f/d53/d81/f113 [0,4194304] 0 2024-09-10T04:05:32.084 INFO:tasks.workunit.client.0.smithi046.stdout:9/840: mkdir d4/d18/d89/ddb/dee/d109 0 2024-09-10T04:05:32.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:31 smithi070 ceph-mon[35495]: pgmap v269: 65 pgs: 65 active+clean; 3.1 GiB data, 8.9 GiB used, 528 GiB / 536 GiB avail; 90 MiB/s rd, 221 MiB/s wr, 441 op/s 2024-09-10T04:05:32.086 INFO:tasks.workunit.client.0.smithi046.stdout:2/836: dwrite d2/d6/d23/d5d/d5e/f84 [0,4194304] 0 2024-09-10T04:05:32.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:31 smithi046 ceph-mon[28328]: pgmap v269: 65 pgs: 65 active+clean; 3.1 GiB data, 8.9 GiB used, 528 GiB / 536 GiB avail; 90 MiB/s rd, 221 MiB/s wr, 441 op/s 2024-09-10T04:05:32.086 INFO:tasks.workunit.client.1.smithi070.stdout:3/873: getdents db/d1c/d3e 0 2024-09-10T04:05:32.089 INFO:tasks.workunit.client.0.smithi046.stdout:1/668: creat d3/dc/d34/fe0 x:0 0 0 2024-09-10T04:05:32.090 INFO:tasks.workunit.client.1.smithi070.stdout:5/961: dread - d17/d24/d77/db4/fed zero size 2024-09-10T04:05:32.094 INFO:tasks.workunit.client.0.smithi046.stdout:9/841: dwrite d4/d7/d1f/f26 [0,4194304] 0 2024-09-10T04:05:32.097 INFO:tasks.workunit.client.0.smithi046.stdout:3/805: write d12/d13/d37/d51/db5/d50/d52/f65 [3703727,102735] 0 2024-09-10T04:05:32.097 INFO:tasks.workunit.client.1.smithi070.stdout:4/867: getdents de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/d10d/db7 0 2024-09-10T04:05:32.100 INFO:tasks.workunit.client.1.smithi070.stdout:7/963: creat d2/f152 x:0 0 0 2024-09-10T04:05:32.102 INFO:tasks.workunit.client.1.smithi070.stdout:6/896: mknod d0/d3/d1f/d93/c11c 0 2024-09-10T04:05:32.104 INFO:tasks.workunit.client.1.smithi070.stdout:3/874: mkdir db/d1c/d25/d28/dd3/de5/d12c 0 2024-09-10T04:05:32.105 INFO:tasks.workunit.client.0.smithi046.stdout:4/964: dwrite d3/d12/d17/f6e [0,4194304] 0 2024-09-10T04:05:32.107 INFO:tasks.workunit.client.0.smithi046.stdout:4/965: chown d3/d12/d14/f111 428493820 1 2024-09-10T04:05:32.108 INFO:tasks.workunit.client.1.smithi070.stdout:1/795: write d3/d8/fbc [267862,13513] 0 2024-09-10T04:05:32.111 INFO:tasks.workunit.client.1.smithi070.stdout:7/964: creat d2/d7/d9/d18/d3c/d52/f153 x:0 0 0 2024-09-10T04:05:32.115 INFO:tasks.workunit.client.1.smithi070.stdout:4/868: write de/d13/d5b/da0/fc9 [777064,67040] 0 2024-09-10T04:05:32.115 INFO:tasks.workunit.client.1.smithi070.stdout:3/875: rmdir db/d1c/d25/d28/dd3/de5/d10f/db6 39 2024-09-10T04:05:32.116 INFO:tasks.workunit.client.0.smithi046.stdout:7/917: dwrite d5/d9/d12/d72/d77/de0/d1f/f3b [0,4194304] 0 2024-09-10T04:05:32.116 INFO:tasks.workunit.client.0.smithi046.stdout:2/837: creat d2/d6/d23/dd5/df8/f128 x:0 0 0 2024-09-10T04:05:32.118 INFO:tasks.workunit.client.0.smithi046.stdout:2/838: dread - d2/d8/d65/d90/f100 zero size 2024-09-10T04:05:32.118 INFO:tasks.workunit.client.0.smithi046.stdout:2/839: chown d2/d49/c4c 684863351 1 2024-09-10T04:05:32.119 INFO:tasks.workunit.client.0.smithi046.stdout:3/806: creat d12/d13/d37/d51/db5/d4b/d116/f118 x:0 0 0 2024-09-10T04:05:32.122 INFO:tasks.workunit.client.1.smithi070.stdout:7/965: symlink d2/d7/d9/d18/d3c/d52/l154 0 2024-09-10T04:05:32.126 INFO:tasks.workunit.client.1.smithi070.stdout:3/876: mknod db/d83/d92/c12d 0 2024-09-10T04:05:32.128 INFO:tasks.workunit.client.1.smithi070.stdout:4/869: mkdir de/d13/d18/d26/d8c/d124 0 2024-09-10T04:05:32.129 INFO:tasks.workunit.client.1.smithi070.stdout:1/796: write d3/d8/d27/f95 [3385087,75517] 0 2024-09-10T04:05:32.133 INFO:tasks.workunit.client.1.smithi070.stdout:7/966: read - d2/dc5/fe1 zero size 2024-09-10T04:05:32.135 INFO:tasks.workunit.client.0.smithi046.stdout:5/952: getdents d7/d11/d14/d45/d51 0 2024-09-10T04:05:32.135 INFO:tasks.workunit.client.0.smithi046.stdout:4/966: dwrite d3/d12/d17/f1b [0,4194304] 0 2024-09-10T04:05:32.138 INFO:tasks.workunit.client.0.smithi046.stdout:7/918: mknod d5/d9/d12/d72/d77/de0/c145 0 2024-09-10T04:05:32.139 INFO:tasks.workunit.client.0.smithi046.stdout:4/967: fsync d3/d12/d17/d29/d58/db2/d5f/d85/da8/fce 0 2024-09-10T04:05:32.139 INFO:tasks.workunit.client.0.smithi046.stdout:3/807: dread f4 [0,4194304] 0 2024-09-10T04:05:32.139 INFO:tasks.workunit.client.0.smithi046.stdout:1/669: getdents d3/da1 0 2024-09-10T04:05:32.141 INFO:tasks.workunit.client.1.smithi070.stdout:4/870: read de/d13/d5b/d7b/dd5/f71 [611882,77828] 0 2024-09-10T04:05:32.141 INFO:tasks.workunit.client.1.smithi070.stdout:7/967: dwrite d2/d40/d12e/f11a [0,4194304] 0 2024-09-10T04:05:32.142 INFO:tasks.workunit.client.1.smithi070.stdout:1/797: unlink d3/d8/d13/d67/c9f 0 2024-09-10T04:05:32.144 INFO:tasks.workunit.client.1.smithi070.stdout:3/877: dread db/f3c [0,4194304] 0 2024-09-10T04:05:32.145 INFO:tasks.workunit.client.0.smithi046.stdout:5/953: dwrite d7/d11/d14/d2f/f48 [0,4194304] 0 2024-09-10T04:05:32.146 INFO:tasks.workunit.client.1.smithi070.stdout:4/871: dread - de/d13/d18/d23/d47/ff4 zero size 2024-09-10T04:05:32.147 INFO:tasks.workunit.client.1.smithi070.stdout:6/897: link d0/d3/d8c/d9a/f113 d0/d3/d8c/d9a/d106/f11d 0 2024-09-10T04:05:32.149 INFO:tasks.workunit.client.1.smithi070.stdout:5/962: link c9 d17/d19/d6a/d54/d57/d63/c14a 0 2024-09-10T04:05:32.150 INFO:tasks.workunit.client.0.smithi046.stdout:2/840: creat d2/d6/d23/d5d/f129 x:0 0 0 2024-09-10T04:05:32.150 INFO:tasks.workunit.client.0.smithi046.stdout:1/670: creat d3/d27/d5d/fe1 x:0 0 0 2024-09-10T04:05:32.153 INFO:tasks.workunit.client.0.smithi046.stdout:1/671: readlink d3/d27/d5d/d60/lbe 0 2024-09-10T04:05:32.153 INFO:tasks.workunit.client.0.smithi046.stdout:7/919: dwrite f4 [0,4194304] 0 2024-09-10T04:05:32.154 INFO:tasks.workunit.client.1.smithi070.stdout:7/968: creat d2/dc8/d119/d14c/f155 x:0 0 0 2024-09-10T04:05:32.154 INFO:tasks.workunit.client.0.smithi046.stdout:1/672: readlink d3/dc/d34/d58/lc4 0 2024-09-10T04:05:32.154 INFO:tasks.workunit.client.0.smithi046.stdout:7/920: readlink d5/d9/d12/d72/d77/de0/d1f/l10d 0 2024-09-10T04:05:32.156 INFO:tasks.workunit.client.1.smithi070.stdout:4/872: dwrite de/d13/d5b/d7b/dd5/f60 [0,4194304] 0 2024-09-10T04:05:32.157 INFO:tasks.workunit.client.1.smithi070.stdout:1/798: chown d3/d10/db2/dc7/lb3 6042165 1 2024-09-10T04:05:32.162 INFO:tasks.workunit.client.0.smithi046.stdout:1/673: mknod d3/d9b/ce2 0 2024-09-10T04:05:32.168 INFO:tasks.workunit.client.0.smithi046.stdout:5/954: symlink d7/d11/d14/d2f/l145 0 2024-09-10T04:05:32.171 INFO:tasks.workunit.client.1.smithi070.stdout:5/963: creat d17/d24/d84/d9b/f14b x:0 0 0 2024-09-10T04:05:32.172 INFO:tasks.workunit.client.0.smithi046.stdout:1/674: fdatasync d3/d27/f66 0 2024-09-10T04:05:32.174 INFO:tasks.workunit.client.0.smithi046.stdout:3/808: dread d12/d13/d37/d51/db5/d50/f55 [0,4194304] 0 2024-09-10T04:05:32.175 INFO:tasks.workunit.client.1.smithi070.stdout:5/964: dwrite d17/d19/fc3 [0,4194304] 0 2024-09-10T04:05:32.177 INFO:tasks.workunit.client.1.smithi070.stdout:5/965: chown d17/d19/d6a/d54/d57/df2/c111 9878820 1 2024-09-10T04:05:32.177 INFO:tasks.workunit.client.0.smithi046.stdout:3/809: chown d12/d13/d37/d51/cbe 250721069 1 2024-09-10T04:05:32.177 INFO:tasks.workunit.client.0.smithi046.stdout:2/841: write d2/d6/f1a [5179031,94059] 0 2024-09-10T04:05:32.179 INFO:tasks.workunit.client.1.smithi070.stdout:9/926: sync 2024-09-10T04:05:32.179 INFO:tasks.workunit.client.0.smithi046.stdout:1/675: creat d3/d62/fe3 x:0 0 0 2024-09-10T04:05:32.180 INFO:tasks.workunit.client.1.smithi070.stdout:1/799: creat d3/d90/dcb/f105 x:0 0 0 2024-09-10T04:05:32.181 INFO:tasks.workunit.client.1.smithi070.stdout:9/927: fdatasync d9/d1c/d34/d8b/d11d/d3c/d92/da8/f109 0 2024-09-10T04:05:32.182 INFO:tasks.workunit.client.0.smithi046.stdout:5/955: symlink d7/d11/d14/d8e/d96/d11c/l146 0 2024-09-10T04:05:32.183 INFO:tasks.workunit.client.0.smithi046.stdout:4/968: dwrite d3/d12/d17/d22/d51/d5c/fde [0,4194304] 0 2024-09-10T04:05:32.184 INFO:tasks.workunit.client.1.smithi070.stdout:3/878: creat db/d1c/d25/d28/dd3/de5/d10f/db6/f12e x:0 0 0 2024-09-10T04:05:32.189 INFO:tasks.workunit.client.0.smithi046.stdout:3/810: dread d12/d13/d37/f60 [0,4194304] 0 2024-09-10T04:05:32.193 INFO:tasks.workunit.client.0.smithi046.stdout:1/676: chown d3/c1b 236463 1 2024-09-10T04:05:32.196 INFO:tasks.workunit.client.1.smithi070.stdout:4/873: mkdir de/d13/d125 0 2024-09-10T04:05:32.197 INFO:tasks.workunit.client.1.smithi070.stdout:7/969: dwrite d2/d4e/f74 [0,4194304] 0 2024-09-10T04:05:32.198 INFO:tasks.workunit.client.0.smithi046.stdout:1/677: dread d3/dc/d34/d58/fd5 [0,4194304] 0 2024-09-10T04:05:32.200 INFO:tasks.workunit.client.0.smithi046.stdout:4/969: dread d3/d12/d14/f111 [0,4194304] 0 2024-09-10T04:05:32.201 INFO:tasks.workunit.client.0.smithi046.stdout:5/956: creat d7/d11/d3f/d53/d75/f147 x:0 0 0 2024-09-10T04:05:32.204 INFO:tasks.workunit.client.1.smithi070.stdout:1/800: readlink d3/d8/d92/l104 0 2024-09-10T04:05:32.205 INFO:tasks.workunit.client.1.smithi070.stdout:1/801: read - d3/d10/db2/dc7/d54/fd7 zero size 2024-09-10T04:05:32.205 INFO:tasks.workunit.client.0.smithi046.stdout:3/811: mknod d12/d13/d14/d1e/d2d/d39/c119 0 2024-09-10T04:05:32.206 INFO:tasks.workunit.client.1.smithi070.stdout:9/928: rmdir d9/d1c/d34 39 2024-09-10T04:05:32.207 INFO:tasks.workunit.client.1.smithi070.stdout:6/898: write d0/d3/da4/dc0/dea/f3b [2505248,84510] 0 2024-09-10T04:05:32.208 INFO:tasks.workunit.client.1.smithi070.stdout:3/879: rmdir db/dcc 39 2024-09-10T04:05:32.212 INFO:tasks.workunit.client.0.smithi046.stdout:5/957: creat d7/d11/d14/d2f/d2b/d92/d142/f148 x:0 0 0 2024-09-10T04:05:32.213 INFO:tasks.workunit.client.0.smithi046.stdout:3/812: dwrite d12/d13/d37/d94/dff/f101 [0,4194304] 0 2024-09-10T04:05:32.214 INFO:tasks.workunit.client.0.smithi046.stdout:4/970: rename d3/d12/d14/d11d to d3/d12/d17/d29/d58/db2/d5f/d85/d114/d149 0 2024-09-10T04:05:32.216 INFO:tasks.workunit.client.0.smithi046.stdout:2/842: link d2/d6/d23/d5d/d77/d7b/dd8/le9 d2/d6/d23/d5d/d77/d7b/d85/d8d/l12a 0 2024-09-10T04:05:32.222 INFO:tasks.workunit.client.1.smithi070.stdout:7/970: symlink d2/dc8/l156 0 2024-09-10T04:05:32.222 INFO:tasks.workunit.client.1.smithi070.stdout:1/802: mkdir d3/d10/db2/dc7/d54/d106 0 2024-09-10T04:05:32.223 INFO:tasks.workunit.client.0.smithi046.stdout:5/958: mknod d7/d11/d14/d2f/d2b/d92/d142/c149 0 2024-09-10T04:05:32.225 INFO:tasks.workunit.client.1.smithi070.stdout:6/899: creat d0/d3/d1f/de8/f11e x:0 0 0 2024-09-10T04:05:32.226 INFO:tasks.workunit.client.0.smithi046.stdout:3/813: symlink d12/d13/d37/d94/dbf/l11a 0 2024-09-10T04:05:32.227 INFO:tasks.workunit.client.1.smithi070.stdout:3/880: symlink db/d1c/d25/d28/dd3/l12f 0 2024-09-10T04:05:32.228 INFO:tasks.workunit.client.1.smithi070.stdout:5/966: dwrite d17/d24/d45/f4f [0,4194304] 0 2024-09-10T04:05:32.230 INFO:tasks.workunit.client.0.smithi046.stdout:4/971: creat d3/d12/d17/d22/d30/dc7/f14a x:0 0 0 2024-09-10T04:05:32.230 INFO:tasks.workunit.client.0.smithi046.stdout:2/843: dread d2/d6/f18 [0,4194304] 0 2024-09-10T04:05:32.232 INFO:tasks.workunit.client.0.smithi046.stdout:5/959: dread - d7/d11/d14/d2f/deb/da5/dcc/fec zero size 2024-09-10T04:05:32.233 INFO:tasks.workunit.client.0.smithi046.stdout:2/844: chown d2/d6/d23/d42/db4/ld3 731 1 2024-09-10T04:05:32.234 INFO:tasks.workunit.client.0.smithi046.stdout:1/678: dwrite d3/d27/d5d/d60/f8a [0,4194304] 0 2024-09-10T04:05:32.234 INFO:tasks.workunit.client.0.smithi046.stdout:5/960: stat d7/d11/d14/d45/d51/f86 0 2024-09-10T04:05:32.235 INFO:tasks.workunit.client.1.smithi070.stdout:5/967: dwrite d17/d24/d45/f76 [0,4194304] 0 2024-09-10T04:05:32.237 INFO:tasks.workunit.client.0.smithi046.stdout:1/679: truncate d3/da1/d1f/fb0 5088406 0 2024-09-10T04:05:32.237 INFO:tasks.workunit.client.0.smithi046.stdout:3/814: unlink d12/d13/d37/d51/db5/d4b/d91/f9a 0 2024-09-10T04:05:32.239 INFO:tasks.workunit.client.0.smithi046.stdout:3/815: read - d12/d5d/d71/d49/f104 zero size 2024-09-10T04:05:32.240 INFO:tasks.workunit.client.0.smithi046.stdout:1/680: readlink d3/dc/d21/lb4 0 2024-09-10T04:05:32.242 INFO:tasks.workunit.client.0.smithi046.stdout:2/845: mknod d2/d8/d37/d63/c12b 0 2024-09-10T04:05:32.244 INFO:tasks.workunit.client.0.smithi046.stdout:5/961: rmdir d7/d11/d14/d2f/d2b 39 2024-09-10T04:05:32.246 INFO:tasks.workunit.client.0.smithi046.stdout:3/816: mknod d12/dd9/ddf/c11b 0 2024-09-10T04:05:32.249 INFO:tasks.workunit.client.0.smithi046.stdout:3/817: truncate d12/d13/d14/d1e/dd2/fe6 679714 0 2024-09-10T04:05:32.256 INFO:tasks.workunit.client.0.smithi046.stdout:1/681: dwrite d3/d27/d2a/fd0 [0,4194304] 0 2024-09-10T04:05:32.261 INFO:tasks.workunit.client.0.smithi046.stdout:6/745: sync 2024-09-10T04:05:32.261 INFO:tasks.workunit.client.0.smithi046.stdout:0/685: sync 2024-09-10T04:05:32.261 INFO:tasks.workunit.client.0.smithi046.stdout:8/850: sync 2024-09-10T04:05:32.265 INFO:tasks.workunit.client.0.smithi046.stdout:2/846: dread d2/d6/d23/d42/db4/dca/fe8 [0,4194304] 0 2024-09-10T04:05:32.266 INFO:tasks.workunit.client.0.smithi046.stdout:6/746: chown da/db/d11/d13/d1e/d23/d54/l62 3933383 1 2024-09-10T04:05:32.267 INFO:tasks.workunit.client.0.smithi046.stdout:1/682: dwrite d3/dc/d34/fe0 [0,4194304] 0 2024-09-10T04:05:32.273 INFO:tasks.workunit.client.0.smithi046.stdout:3/818: rmdir d12/d5d/d71/d49/dec 0 2024-09-10T04:05:32.273 INFO:tasks.workunit.client.0.smithi046.stdout:0/686: fdatasync d1/dc/d25/d33/d1a/f47 0 2024-09-10T04:05:32.275 INFO:tasks.workunit.client.0.smithi046.stdout:5/962: dwrite d7/ffd [0,4194304] 0 2024-09-10T04:05:32.276 INFO:tasks.workunit.client.0.smithi046.stdout:3/819: fsync d12/d13/d37/d94/dff/f101 0 2024-09-10T04:05:32.282 INFO:tasks.workunit.client.0.smithi046.stdout:5/963: dread d7/d11/d14/d45/f6d [4194304,4194304] 0 2024-09-10T04:05:32.283 INFO:tasks.workunit.client.0.smithi046.stdout:8/851: creat d1/f11c x:0 0 0 2024-09-10T04:05:32.292 INFO:tasks.workunit.client.0.smithi046.stdout:3/820: creat d12/d13/d14/d1e/d2d/d39/d67/f11c x:0 0 0 2024-09-10T04:05:32.295 INFO:tasks.workunit.client.0.smithi046.stdout:8/852: mknod d1/d29/d17/d1f/d46/d6a/c11d 0 2024-09-10T04:05:32.296 INFO:tasks.workunit.client.0.smithi046.stdout:8/853: chown d1/d29/d17/d1f/d46/d6a/ff3 0 1 2024-09-10T04:05:32.298 INFO:tasks.workunit.client.0.smithi046.stdout:0/687: symlink d1/dc/d25/d33/d3/ld6 0 2024-09-10T04:05:32.301 INFO:tasks.workunit.client.0.smithi046.stdout:3/821: fdatasync d12/d13/d37/d51/db5/d50/f55 0 2024-09-10T04:05:32.302 INFO:tasks.workunit.client.0.smithi046.stdout:6/747: truncate da/d96/ff0 1755257 0 2024-09-10T04:05:32.302 INFO:tasks.workunit.client.0.smithi046.stdout:8/854: dwrite d1/d29/d17/f106 [0,4194304] 0 2024-09-10T04:05:32.304 INFO:tasks.workunit.client.0.smithi046.stdout:2/847: getdents d2/d6/d23 0 2024-09-10T04:05:32.305 INFO:tasks.workunit.client.1.smithi070.stdout:8/956: sync 2024-09-10T04:05:32.305 INFO:tasks.workunit.client.1.smithi070.stdout:2/839: sync 2024-09-10T04:05:32.310 INFO:tasks.workunit.client.0.smithi046.stdout:0/688: mkdir d1/d13/d37/d75/d94/d9a/dd7 0 2024-09-10T04:05:32.317 INFO:tasks.workunit.client.0.smithi046.stdout:3/822: symlink d12/d13/d37/d51/dbd/l11d 0 2024-09-10T04:05:32.317 INFO:tasks.workunit.client.0.smithi046.stdout:3/823: chown d12 868429819 1 2024-09-10T04:05:32.319 INFO:tasks.workunit.client.0.smithi046.stdout:6/748: dwrite da/db/d11/d13/d1e/f3e [0,4194304] 0 2024-09-10T04:05:32.327 INFO:tasks.workunit.client.0.smithi046.stdout:8/855: truncate d1/f86 2975361 0 2024-09-10T04:05:32.328 INFO:tasks.workunit.client.0.smithi046.stdout:2/848: symlink d2/d6/l12c 0 2024-09-10T04:05:32.330 INFO:tasks.workunit.client.0.smithi046.stdout:2/849: write d2/d8/d37/dc2/f116 [343762,58812] 0 2024-09-10T04:05:32.331 INFO:tasks.workunit.client.0.smithi046.stdout:2/850: read - d2/d6/d9d/f120 zero size 2024-09-10T04:05:32.331 INFO:tasks.workunit.client.1.smithi070.stdout:9/929: rename d9/d1c/dac to d9/d1c/d34/d8b/dcc/de7/d12f 0 2024-09-10T04:05:32.332 INFO:tasks.workunit.client.0.smithi046.stdout:3/824: rmdir d12/d13/d14/d1e/d2d/d39 39 2024-09-10T04:05:32.335 INFO:tasks.workunit.client.0.smithi046.stdout:6/749: rename da/db/d11/d13/d1e/d23/f2e to da/db/d11/d2b/d5a/ff4 0 2024-09-10T04:05:32.337 INFO:tasks.workunit.client.1.smithi070.stdout:6/900: mkdir d0/d3/d8c/d9a/d11f 0 2024-09-10T04:05:32.338 INFO:tasks.workunit.client.0.smithi046.stdout:0/689: dwrite d1/dc/d25/d33/d3/d29/d2d/d5e/f89 [0,4194304] 0 2024-09-10T04:05:32.343 INFO:tasks.workunit.client.0.smithi046.stdout:8/856: creat d1/d29/d19/d36/d6e/f11e x:0 0 0 2024-09-10T04:05:32.344 INFO:tasks.workunit.client.1.smithi070.stdout:2/840: mknod d15/d18/d56/d7c/d55/c133 0 2024-09-10T04:05:32.344 INFO:tasks.workunit.client.1.smithi070.stdout:2/841: truncate d15/f12e 645469 0 2024-09-10T04:05:32.345 INFO:tasks.workunit.client.0.smithi046.stdout:2/851: truncate d2/d6/d23/f87 935566 0 2024-09-10T04:05:32.345 INFO:tasks.workunit.client.0.smithi046.stdout:8/857: chown d1/d29/d19/d36/d6e/f115 924 1 2024-09-10T04:05:32.348 INFO:tasks.workunit.client.0.smithi046.stdout:3/825: write d12/d13/d14/d1e/d2d/d39/fea [843757,72742] 0 2024-09-10T04:05:32.348 INFO:tasks.workunit.client.1.smithi070.stdout:9/930: creat d9/d1c/d34/d8b/d11d/d1b/f130 x:0 0 0 2024-09-10T04:05:32.353 INFO:tasks.workunit.client.0.smithi046.stdout:8/858: dwrite d1/d29/d17/d1f/d46/d67/de0/fe5 [0,4194304] 0 2024-09-10T04:05:32.354 INFO:tasks.workunit.client.0.smithi046.stdout:8/859: stat d1/f11c 0 2024-09-10T04:05:32.354 INFO:tasks.workunit.client.1.smithi070.stdout:5/968: write d17/d19/f2e [3548528,108387] 0 2024-09-10T04:05:32.356 INFO:tasks.workunit.client.0.smithi046.stdout:8/860: stat d1/d29/d19/d36/c51 0 2024-09-10T04:05:32.357 INFO:tasks.workunit.client.1.smithi070.stdout:2/842: fdatasync d15/d18/d29/d2a/d2e/d32/f100 0 2024-09-10T04:05:32.361 INFO:tasks.workunit.client.0.smithi046.stdout:2/852: readlink d2/d6/d23/d5d/d77/d7b/dd8/le9 0 2024-09-10T04:05:32.363 INFO:tasks.workunit.client.0.smithi046.stdout:0/690: write d1/d13/d37/d75/d94/d9a/f90 [87811,58919] 0 2024-09-10T04:05:32.367 INFO:tasks.workunit.client.1.smithi070.stdout:5/969: unlink d17/d19/d35/l7f 0 2024-09-10T04:05:32.368 INFO:tasks.workunit.client.0.smithi046.stdout:0/691: truncate d1/d13/d37/d75/d94/d9a/fac 1020926 0 2024-09-10T04:05:32.368 INFO:tasks.workunit.client.0.smithi046.stdout:2/853: dread d2/d8/d65/f7d [0,4194304] 0 2024-09-10T04:05:32.368 INFO:tasks.workunit.client.0.smithi046.stdout:6/750: mknod da/cf5 0 2024-09-10T04:05:32.369 INFO:tasks.workunit.client.0.smithi046.stdout:2/854: dread - d2/d6/d9d/db6/dbe/df2/f113 zero size 2024-09-10T04:05:32.369 INFO:tasks.workunit.client.0.smithi046.stdout:0/692: truncate d1/d13/d37/d75/fd0 1007421 0 2024-09-10T04:05:32.370 INFO:tasks.workunit.client.0.smithi046.stdout:2/855: fdatasync d2/d6/d9d/f106 0 2024-09-10T04:05:32.372 INFO:tasks.workunit.client.1.smithi070.stdout:1/803: rename f1 to d3/d8/d27/f107 0 2024-09-10T04:05:32.374 INFO:tasks.workunit.client.1.smithi070.stdout:7/971: getdents d2/d7/d9/d18/d3c/d52/d103 0 2024-09-10T04:05:32.374 INFO:tasks.workunit.client.1.smithi070.stdout:1/804: readlink d3/d8/d13/d3f/d5d/ldb 0 2024-09-10T04:05:32.374 INFO:tasks.workunit.client.1.smithi070.stdout:2/843: dwrite d15/d18/d29/d2a/d2e/d32/f100 [0,4194304] 0 2024-09-10T04:05:32.375 INFO:tasks.workunit.client.1.smithi070.stdout:6/901: creat d0/d3/d8c/f120 x:0 0 0 2024-09-10T04:05:32.376 INFO:tasks.workunit.client.0.smithi046.stdout:3/826: mkdir d12/d5d/d71/df7/d11e 0 2024-09-10T04:05:32.377 INFO:tasks.workunit.client.1.smithi070.stdout:1/805: truncate d3/d10/db2/dc7/d54/fd7 1031331 0 2024-09-10T04:05:32.377 INFO:tasks.workunit.client.0.smithi046.stdout:3/827: write f3 [3821479,32646] 0 2024-09-10T04:05:32.379 INFO:tasks.workunit.client.1.smithi070.stdout:5/970: fdatasync d17/d19/f44 0 2024-09-10T04:05:32.380 INFO:tasks.workunit.client.1.smithi070.stdout:9/931: write d9/fb7 [950613,111269] 0 2024-09-10T04:05:32.382 INFO:tasks.workunit.client.0.smithi046.stdout:6/751: symlink da/dc/da5/lf6 0 2024-09-10T04:05:32.383 INFO:tasks.workunit.client.1.smithi070.stdout:2/844: dwrite d15/d18/d29/d2a/f110 [0,4194304] 0 2024-09-10T04:05:32.385 INFO:tasks.workunit.client.0.smithi046.stdout:0/693: creat d1/dc/d25/d33/d4/fd8 x:0 0 0 2024-09-10T04:05:32.387 INFO:tasks.workunit.client.0.smithi046.stdout:8/861: truncate d1/d29/d17/d1f/d46/d6a/f94 1363867 0 2024-09-10T04:05:32.388 INFO:tasks.workunit.client.0.smithi046.stdout:8/862: dread - d1/d29/d19/d36/d6e/f115 zero size 2024-09-10T04:05:32.389 INFO:tasks.workunit.client.1.smithi070.stdout:7/972: creat d2/d3/dc/d5d/d41/dd6/de6/f157 x:0 0 0 2024-09-10T04:05:32.391 INFO:tasks.workunit.client.0.smithi046.stdout:3/828: rename d12/d13/d37/d51/db5/d4b/dc4 to d12/d13/d14/d1e/d2d/d6f/d11f 0 2024-09-10T04:05:32.393 INFO:tasks.workunit.client.0.smithi046.stdout:2/856: dread d2/d8/d37/d63/f75 [0,4194304] 0 2024-09-10T04:05:32.393 INFO:tasks.workunit.client.0.smithi046.stdout:6/752: symlink da/db/d10/lf7 0 2024-09-10T04:05:32.395 INFO:tasks.workunit.client.0.smithi046.stdout:6/753: fsync da/db/d11/dd7/f34 0 2024-09-10T04:05:32.397 INFO:tasks.workunit.client.0.smithi046.stdout:8/863: chown d1/d29/d12/f1e 1276 1 2024-09-10T04:05:32.398 INFO:tasks.workunit.client.1.smithi070.stdout:1/806: truncate d3/d8/d12/d2f/f5b 1349760 0 2024-09-10T04:05:32.399 INFO:tasks.workunit.client.0.smithi046.stdout:8/864: write d1/d29/d17/d1f/d46/d6a/ff3 [826908,40439] 0 2024-09-10T04:05:32.408 INFO:tasks.workunit.client.0.smithi046.stdout:8/865: unlink d1/f11c 0 2024-09-10T04:05:32.409 INFO:tasks.workunit.client.0.smithi046.stdout:2/857: fdatasync d2/d6/d23/d5d/d77/f69 0 2024-09-10T04:05:32.410 INFO:tasks.workunit.client.0.smithi046.stdout:0/694: dread d1/dc/d25/d33/d4/f6f [4194304,4194304] 0 2024-09-10T04:05:32.410 INFO:tasks.workunit.client.1.smithi070.stdout:9/932: dwrite d9/f68 [0,4194304] 0 2024-09-10T04:05:32.411 INFO:tasks.workunit.client.0.smithi046.stdout:0/695: dread - d1/d13/fa9 zero size 2024-09-10T04:05:32.411 INFO:tasks.workunit.client.1.smithi070.stdout:9/933: read - d9/d1c/d34/d8b/fef zero size 2024-09-10T04:05:32.412 INFO:tasks.workunit.client.1.smithi070.stdout:9/934: dread - d9/d7a/d72/f108 zero size 2024-09-10T04:05:32.413 INFO:tasks.workunit.client.1.smithi070.stdout:9/935: readlink d9/d1c/d34/d8b/lee 0 2024-09-10T04:05:32.413 INFO:tasks.workunit.client.1.smithi070.stdout:4/874: sync 2024-09-10T04:05:32.419 INFO:tasks.workunit.client.1.smithi070.stdout:9/936: dwrite d9/d1c/d34/d8b/dcc/de7/d12f/fe2 [0,4194304] 0 2024-09-10T04:05:32.419 INFO:tasks.workunit.client.1.smithi070.stdout:3/881: rename db/d1c/d25/d28/dd3/de5/d10f/db5/lb8 to db/d1c/d25/l130 0 2024-09-10T04:05:32.419 INFO:tasks.workunit.client.0.smithi046.stdout:0/696: dwrite d1/d13/d37/d75/d94/fd3 [0,4194304] 0 2024-09-10T04:05:32.420 INFO:tasks.workunit.client.0.smithi046.stdout:2/858: mkdir d2/d6/d23/d5d/d77/d7b/dc4/d12d 0 2024-09-10T04:05:32.422 INFO:tasks.workunit.client.1.smithi070.stdout:7/973: mkdir d2/d40/d6c/d158 0 2024-09-10T04:05:32.422 INFO:tasks.workunit.client.0.smithi046.stdout:6/754: dwrite da/db/d11/d13/d1e/f82 [0,4194304] 0 2024-09-10T04:05:32.424 INFO:tasks.workunit.client.1.smithi070.stdout:6/902: creat d0/d3/da4/dc0/dea/f121 x:0 0 0 2024-09-10T04:05:32.429 INFO:tasks.workunit.client.1.smithi070.stdout:5/971: truncate d17/d19/d12d/f136 428856 0 2024-09-10T04:05:32.429 INFO:tasks.workunit.client.0.smithi046.stdout:3/829: link d12/d13/d37/d51/db5/d50/d7a/dc1/fbc d12/d13/d14/f120 0 2024-09-10T04:05:32.430 INFO:tasks.workunit.client.0.smithi046.stdout:0/697: mknod d1/dc/d25/d33/d4/d19/cd9 0 2024-09-10T04:05:32.432 INFO:tasks.workunit.client.0.smithi046.stdout:2/859: mknod d2/d6/d9d/db6/dbe/c12e 0 2024-09-10T04:05:32.433 INFO:tasks.workunit.client.1.smithi070.stdout:4/875: creat de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/f126 x:0 0 0 2024-09-10T04:05:32.440 INFO:tasks.workunit.client.1.smithi070.stdout:8/957: rename d4/d16/dcc/d12e/d11b/dab/fdc to d4/d16/d73/d139/d7a/db8/f147 0 2024-09-10T04:05:32.441 INFO:tasks.workunit.client.0.smithi046.stdout:3/830: mknod d12/d13/d14/d1e/d2d/d6f/d11f/c121 0 2024-09-10T04:05:32.444 INFO:tasks.workunit.client.0.smithi046.stdout:8/866: write d1/d29/d19/d36/fd4 [4747387,98872] 0 2024-09-10T04:05:32.444 INFO:tasks.workunit.client.1.smithi070.stdout:9/937: read d9/d1c/d34/d8b/d11d/f81 [1789525,65235] 0 2024-09-10T04:05:32.447 INFO:tasks.workunit.client.1.smithi070.stdout:1/807: write d3/d8/f2d [283876,81373] 0 2024-09-10T04:05:32.449 INFO:tasks.workunit.client.1.smithi070.stdout:6/903: symlink d0/d7f/l122 0 2024-09-10T04:05:32.453 INFO:tasks.workunit.client.0.smithi046.stdout:6/755: dread da/db/d11/d13/d1e/d23/d86/dbf/fef [4194304,4194304] 0 2024-09-10T04:05:32.453 INFO:tasks.workunit.client.1.smithi070.stdout:5/972: creat d17/d6f/d32/d90/d99/f14c x:0 0 0 2024-09-10T04:05:32.455 INFO:tasks.workunit.client.0.smithi046.stdout:0/698: write d1/dc/d25/f9e [664341,67569] 0 2024-09-10T04:05:32.458 INFO:tasks.workunit.client.0.smithi046.stdout:3/831: link d12/d13/d37/d51/db5/d4b/fb6 d12/d13/d37/d51/db5/d4b/d116/f122 0 2024-09-10T04:05:32.458 INFO:tasks.workunit.client.0.smithi046.stdout:2/860: write d2/d6/d23/d5d/d77/d7b/d85/d8d/fc5 [621245,2498] 0 2024-09-10T04:05:32.459 INFO:tasks.workunit.client.1.smithi070.stdout:2/845: getdents d15/d2b/d106 0 2024-09-10T04:05:32.459 INFO:tasks.workunit.client.0.smithi046.stdout:8/867: dread d1/d29/d19/d36/f3f [0,4194304] 0 2024-09-10T04:05:32.460 INFO:tasks.workunit.client.0.smithi046.stdout:9/842: sync 2024-09-10T04:05:32.461 INFO:tasks.workunit.client.0.smithi046.stdout:6/756: chown da/d96/ff0 2 1 2024-09-10T04:05:32.469 INFO:tasks.workunit.client.0.smithi046.stdout:0/699: mknod d1/d13/d37/d75/d94/dd4/cda 0 2024-09-10T04:05:32.471 INFO:tasks.workunit.client.1.smithi070.stdout:5/973: unlink d17/d24/d77/l116 0 2024-09-10T04:05:32.471 INFO:tasks.workunit.client.1.smithi070.stdout:5/974: readlink d17/l11a 0 2024-09-10T04:05:32.473 INFO:tasks.workunit.client.1.smithi070.stdout:9/938: write d9/d1c/d34/d8b/d11d/fbf [1000923,34877] 0 2024-09-10T04:05:32.474 INFO:tasks.workunit.client.1.smithi070.stdout:4/876: mkdir de/d13/d18/d26/d8c/d124/d127 0 2024-09-10T04:05:32.474 INFO:tasks.workunit.client.0.smithi046.stdout:2/861: dread d2/d49/f74 [0,4194304] 0 2024-09-10T04:05:32.475 INFO:tasks.workunit.client.0.smithi046.stdout:8/868: mknod d1/d29/d19/d36/d4c/d5e/c11f 0 2024-09-10T04:05:32.478 INFO:tasks.workunit.client.0.smithi046.stdout:9/843: write d4/d3c/de0/d95/fab [45182,125344] 0 2024-09-10T04:05:32.478 INFO:tasks.workunit.client.1.smithi070.stdout:8/958: symlink d4/d16/d73/l148 0 2024-09-10T04:05:32.479 INFO:tasks.workunit.client.0.smithi046.stdout:8/869: chown d1/d29/d19/d36/d4c/d5e/fdb 40483 1 2024-09-10T04:05:32.480 INFO:tasks.workunit.client.1.smithi070.stdout:8/959: chown d4/d16/dcc/d12e/d11b/d19/cc1 1939 1 2024-09-10T04:05:32.480 INFO:tasks.workunit.client.1.smithi070.stdout:8/960: chown d4/c8a 36 1 2024-09-10T04:05:32.481 INFO:tasks.workunit.client.1.smithi070.stdout:3/882: creat db/d1c/d25/f131 x:0 0 0 2024-09-10T04:05:32.482 INFO:tasks.workunit.client.0.smithi046.stdout:6/757: truncate da/f2a 1725925 0 2024-09-10T04:05:32.483 INFO:tasks.workunit.client.0.smithi046.stdout:3/832: dwrite d12/d13/d14/f24 [0,4194304] 0 2024-09-10T04:05:32.483 INFO:tasks.workunit.client.1.smithi070.stdout:7/974: link d2/d40/d6c/db2/fce d2/d40/d6c/db2/db7/f159 0 2024-09-10T04:05:32.484 INFO:tasks.workunit.client.0.smithi046.stdout:9/844: mkdir d4/d7/d61/d10a 0 2024-09-10T04:05:32.485 INFO:tasks.workunit.client.0.smithi046.stdout:9/845: chown d4/d66/da6/le4 205175 1 2024-09-10T04:05:32.486 INFO:tasks.workunit.client.1.smithi070.stdout:1/808: chown d3/d10/d1d/d79/fe7 1688 1 2024-09-10T04:05:32.489 INFO:tasks.workunit.client.1.smithi070.stdout:9/939: dread d9/d1c/d34/d8b/d11d/f20 [0,4194304] 0 2024-09-10T04:05:32.490 INFO:tasks.workunit.client.0.smithi046.stdout:8/870: mknod d1/d29/d17/d1f/d46/d76/c120 0 2024-09-10T04:05:32.491 INFO:tasks.workunit.client.1.smithi070.stdout:5/975: truncate d17/d24/d77/db4/fed 805396 0 2024-09-10T04:05:32.491 INFO:tasks.workunit.client.0.smithi046.stdout:8/871: chown d1/d29/d17/d1f/d46/fa8 4041 1 2024-09-10T04:05:32.493 INFO:tasks.workunit.client.1.smithi070.stdout:5/976: write d17/d19/d6a/d54/de1/dfe/f3d [2420770,43776] 0 2024-09-10T04:05:32.493 INFO:tasks.workunit.client.0.smithi046.stdout:6/758: stat da/db/d10/d27/f8a 0 2024-09-10T04:05:32.493 INFO:tasks.workunit.client.0.smithi046.stdout:8/872: stat d1/d29/d19/d36/d4c/d5e/f7f 0 2024-09-10T04:05:32.495 INFO:tasks.workunit.client.1.smithi070.stdout:4/877: unlink fc 0 2024-09-10T04:05:32.496 INFO:tasks.workunit.client.0.smithi046.stdout:2/862: truncate d2/d8/f29 524774 0 2024-09-10T04:05:32.497 INFO:tasks.workunit.client.0.smithi046.stdout:3/833: fsync d12/d13/d14/d1e/d2d/fe1 0 2024-09-10T04:05:32.497 INFO:tasks.workunit.client.1.smithi070.stdout:2/846: getdents d15/d18/d56/d7c/d55/d6d/d125 0 2024-09-10T04:05:32.499 INFO:tasks.workunit.client.0.smithi046.stdout:0/700: dwrite d1/dc/d49/f4a [0,4194304] 0 2024-09-10T04:05:32.499 INFO:tasks.workunit.client.1.smithi070.stdout:8/961: mknod d4/d16/dcc/d12e/d11b/d19/d62/d8d/c149 0 2024-09-10T04:05:32.500 INFO:tasks.workunit.client.0.smithi046.stdout:9/846: dread d4/d7/d61/ff3 [0,4194304] 0 2024-09-10T04:05:32.500 INFO:tasks.workunit.client.0.smithi046.stdout:0/701: chown d1/dc/d25/d33/d4/fae 1106 1 2024-09-10T04:05:32.502 INFO:tasks.workunit.client.0.smithi046.stdout:6/759: mkdir da/db/d11/d9f/dd6/df8 0 2024-09-10T04:05:32.502 INFO:tasks.workunit.client.0.smithi046.stdout:9/847: dread - d4/d7/d1f/feb zero size 2024-09-10T04:05:32.503 INFO:tasks.workunit.client.1.smithi070.stdout:3/883: read db/d1c/f6d [3956136,85442] 0 2024-09-10T04:05:32.504 INFO:tasks.workunit.client.1.smithi070.stdout:1/809: symlink d3/d8/d27/d7a/d94/l108 0 2024-09-10T04:05:32.505 INFO:tasks.workunit.client.1.smithi070.stdout:6/904: creat d0/f123 x:0 0 0 2024-09-10T04:05:32.506 INFO:tasks.workunit.client.0.smithi046.stdout:6/760: dread da/db/f16 [0,4194304] 0 2024-09-10T04:05:32.508 INFO:tasks.workunit.client.1.smithi070.stdout:9/940: fsync d9/d1c/dfd/fbd 0 2024-09-10T04:05:32.509 INFO:tasks.workunit.client.1.smithi070.stdout:9/941: chown d9/d1c/d34/d8b/dcc/de7/d12f/fe2 0 1 2024-09-10T04:05:32.511 INFO:tasks.workunit.client.1.smithi070.stdout:7/975: dread d2/d3/dc/d5d/d2d/fba [0,4194304] 0 2024-09-10T04:05:32.512 INFO:tasks.workunit.client.0.smithi046.stdout:9/848: symlink d4/d18/d92/l10b 0 2024-09-10T04:05:32.516 INFO:tasks.workunit.client.1.smithi070.stdout:7/976: dread - d2/d7/d9/d18/d3c/d52/fc9 zero size 2024-09-10T04:05:32.517 INFO:tasks.workunit.client.0.smithi046.stdout:9/849: write d4/d7/d1f/dce/fec [948432,53978] 0 2024-09-10T04:05:32.517 INFO:tasks.workunit.client.0.smithi046.stdout:3/834: mkdir d12/d13/d14/d1e/d123 0 2024-09-10T04:05:32.517 INFO:tasks.workunit.client.1.smithi070.stdout:4/878: rename de/d13/d18/d23/d47/d59/ff6 to de/d13/d18/d26/d8c/d124/d127/f128 0 2024-09-10T04:05:32.517 INFO:tasks.workunit.client.1.smithi070.stdout:9/942: dwrite d9/d1c/d34/d8b/d11d/d3c/d62/f10a [0,4194304] 0 2024-09-10T04:05:32.518 INFO:tasks.workunit.client.0.smithi046.stdout:3/835: readlink d12/d5d/d71/d49/l69 0 2024-09-10T04:05:32.519 INFO:tasks.workunit.client.0.smithi046.stdout:8/873: creat d1/d29/d17/d1f/d2a/df6/f121 x:0 0 0 2024-09-10T04:05:32.523 INFO:tasks.workunit.client.0.smithi046.stdout:2/863: dwrite d2/d6/d23/d5d/d77/d7b/dc4/fc6 [0,4194304] 0 2024-09-10T04:05:32.528 INFO:tasks.workunit.client.0.smithi046.stdout:6/761: unlink da/f7b 0 2024-09-10T04:05:32.531 INFO:tasks.workunit.client.0.smithi046.stdout:0/702: link d1/dc/d25/d33/d3/dd/f96 d1/d13/fdb 0 2024-09-10T04:05:32.532 INFO:tasks.workunit.client.1.smithi070.stdout:6/905: chown d0/d3/d18/d1b/d17/l20 42957 1 2024-09-10T04:05:32.533 INFO:tasks.workunit.client.1.smithi070.stdout:2/847: dread d15/d18/d29/d2a/d2e/d32/f49 [4194304,4194304] 0 2024-09-10T04:05:32.533 INFO:tasks.workunit.client.1.smithi070.stdout:6/906: chown d0/d3/d18/d1b/d17/l20 958900089 1 2024-09-10T04:05:32.534 INFO:tasks.workunit.client.1.smithi070.stdout:8/962: write d4/d16/f34 [1090966,97903] 0 2024-09-10T04:05:32.535 INFO:tasks.workunit.client.1.smithi070.stdout:6/907: read d0/d3/d18/d1b/d17/ffb [244553,6120] 0 2024-09-10T04:05:32.537 INFO:tasks.workunit.client.0.smithi046.stdout:3/836: mkdir d12/d5d/d71/df7/d124 0 2024-09-10T04:05:32.537 INFO:tasks.workunit.client.1.smithi070.stdout:5/977: stat d17/d24/d2b/f9d 0 2024-09-10T04:05:32.539 INFO:tasks.workunit.client.1.smithi070.stdout:3/884: write db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/de1/ff2 [760203,119700] 0 2024-09-10T04:05:32.541 INFO:tasks.workunit.client.1.smithi070.stdout:7/977: creat d2/dc5/f15a x:0 0 0 2024-09-10T04:05:32.545 INFO:tasks.workunit.client.0.smithi046.stdout:6/762: creat da/db/d11/d13/d1e/d23/d86/dbf/ff9 x:0 0 0 2024-09-10T04:05:32.547 INFO:tasks.workunit.client.0.smithi046.stdout:6/763: chown da/db/d11/d13/d1e/d23/ld4 326210 1 2024-09-10T04:05:32.548 INFO:tasks.workunit.client.0.smithi046.stdout:9/850: dwrite d4/f6 [0,4194304] 0 2024-09-10T04:05:32.549 INFO:tasks.workunit.client.1.smithi070.stdout:9/943: dread d9/d59/d2a/fcf [0,4194304] 0 2024-09-10T04:05:32.552 INFO:tasks.workunit.client.0.smithi046.stdout:3/837: dread d12/d5d/fcd [0,4194304] 0 2024-09-10T04:05:32.554 INFO:tasks.workunit.client.0.smithi046.stdout:2/864: fsync d2/d8/d37/f41 0 2024-09-10T04:05:32.557 INFO:tasks.workunit.client.1.smithi070.stdout:2/848: mknod d15/d59/da1/c134 0 2024-09-10T04:05:32.558 INFO:tasks.workunit.client.0.smithi046.stdout:6/764: creat da/db/d11/d9f/ffa x:0 0 0 2024-09-10T04:05:32.564 INFO:tasks.workunit.client.1.smithi070.stdout:8/963: rmdir d4/d39/dfc/d43/d52 39 2024-09-10T04:05:32.568 INFO:tasks.workunit.client.0.smithi046.stdout:8/874: creat d1/d29/d17/d1f/d2a/df6/f122 x:0 0 0 2024-09-10T04:05:32.569 INFO:tasks.workunit.client.1.smithi070.stdout:6/908: symlink d0/d3/d18/d2a/df0/l124 0 2024-09-10T04:05:32.569 INFO:tasks.workunit.client.0.smithi046.stdout:8/875: readlink d1/d29/d17/d1f/d46/d6a/d82/dc2/led 0 2024-09-10T04:05:32.571 INFO:tasks.workunit.client.1.smithi070.stdout:1/810: dwrite d3/d8/d12/d2f/f5b [0,4194304] 0 2024-09-10T04:05:32.571 INFO:tasks.workunit.client.0.smithi046.stdout:0/703: creat d1/fdc x:0 0 0 2024-09-10T04:05:32.573 INFO:tasks.workunit.client.1.smithi070.stdout:7/978: creat d2/d40/d12e/f15b x:0 0 0 2024-09-10T04:05:32.574 INFO:tasks.workunit.client.0.smithi046.stdout:3/838: truncate d12/d13/d37/d51/db5/d50/d7a/dc1/f93 1006535 0 2024-09-10T04:05:32.574 INFO:tasks.workunit.client.0.smithi046.stdout:2/865: dread d2/d6/d15/fa4 [0,4194304] 0 2024-09-10T04:05:32.575 INFO:tasks.workunit.client.0.smithi046.stdout:9/851: fdatasync d4/d29/d65/ff6 0 2024-09-10T04:05:32.577 INFO:tasks.workunit.client.1.smithi070.stdout:9/944: fsync d9/f9a 0 2024-09-10T04:05:32.577 INFO:tasks.workunit.client.0.smithi046.stdout:6/765: truncate da/db/d11/d13/d1e/d23/d86/dbf/f95 4982178 0 2024-09-10T04:05:32.578 INFO:tasks.workunit.client.0.smithi046.stdout:2/866: dread d2/d8/d37/dc2/f116 [0,4194304] 0 2024-09-10T04:05:32.579 INFO:tasks.workunit.client.0.smithi046.stdout:6/766: dread - da/db/d11/d13/d1e/d23/d86/fd3 zero size 2024-09-10T04:05:32.580 INFO:tasks.workunit.client.1.smithi070.stdout:7/979: dwrite d2/d7/d9/dd5/fe2 [0,4194304] 0 2024-09-10T04:05:32.581 INFO:tasks.workunit.client.1.smithi070.stdout:8/964: mknod d4/d16/dcc/d12e/d11b/d19/d62/d8d/c14a 0 2024-09-10T04:05:32.582 INFO:tasks.workunit.client.1.smithi070.stdout:5/978: mknod d17/d6f/d32/c14d 0 2024-09-10T04:05:32.584 INFO:tasks.workunit.client.1.smithi070.stdout:3/885: creat db/d1c/d25/d28/dd3/de5/d10f/d110/f132 x:0 0 0 2024-09-10T04:05:32.584 INFO:tasks.workunit.client.0.smithi046.stdout:9/852: truncate d4/d44/f94 1712532 0 2024-09-10T04:05:32.585 INFO:tasks.workunit.client.1.smithi070.stdout:6/909: mkdir d0/d3/d18/d1b/d17/d54/dc4/d125 0 2024-09-10T04:05:32.589 INFO:tasks.workunit.client.0.smithi046.stdout:2/867: creat d2/d8/d65/d90/f12f x:0 0 0 2024-09-10T04:05:32.591 INFO:tasks.workunit.client.0.smithi046.stdout:0/704: mknod d1/dc/d25/d33/cdd 0 2024-09-10T04:05:32.596 INFO:tasks.workunit.client.1.smithi070.stdout:2/849: symlink d15/d18/d29/d2a/d2e/d32/l135 0 2024-09-10T04:05:32.597 INFO:tasks.workunit.client.1.smithi070.stdout:7/980: symlink d2/d66/l15c 0 2024-09-10T04:05:32.599 INFO:tasks.workunit.client.0.smithi046.stdout:2/868: creat d2/d6/d23/d42/db4/f130 x:0 0 0 2024-09-10T04:05:32.600 INFO:tasks.workunit.client.0.smithi046.stdout:2/869: readlink d2/d86/d9c/dbc/le2 0 2024-09-10T04:05:32.602 INFO:tasks.workunit.client.1.smithi070.stdout:8/965: dread d4/d39/dfc/d43/d52/f119 [0,4194304] 0 2024-09-10T04:05:32.603 INFO:tasks.workunit.client.1.smithi070.stdout:3/886: creat db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/f133 x:0 0 0 2024-09-10T04:05:32.604 INFO:tasks.workunit.client.0.smithi046.stdout:0/705: creat d1/dc/d25/d33/d3/d29/d2d/d5e/fde x:0 0 0 2024-09-10T04:05:32.606 INFO:tasks.workunit.client.1.smithi070.stdout:6/910: creat d0/d7f/f126 x:0 0 0 2024-09-10T04:05:32.607 INFO:tasks.workunit.client.0.smithi046.stdout:6/767: write da/d96/ff0 [2688638,39557] 0 2024-09-10T04:05:32.607 INFO:tasks.workunit.client.1.smithi070.stdout:1/811: dwrite d3/f47 [4194304,4194304] 0 2024-09-10T04:05:32.608 INFO:tasks.workunit.client.1.smithi070.stdout:4/879: getdents de/d13/d18 0 2024-09-10T04:05:32.609 INFO:tasks.workunit.client.0.smithi046.stdout:9/853: symlink d4/d3c/d4d/d5c/l10c 0 2024-09-10T04:05:32.610 INFO:tasks.workunit.client.0.smithi046.stdout:8/876: getdents d1/d29/d17/d1f/d2a/df6 0 2024-09-10T04:05:32.611 INFO:tasks.workunit.client.0.smithi046.stdout:8/877: readlink d1/d29/d17/d1f/d46/d67/l6d 0 2024-09-10T04:05:32.611 INFO:tasks.workunit.client.1.smithi070.stdout:2/850: chown d15/d18/d29/d2a/d2e/d32/d9b/f12d 237120 1 2024-09-10T04:05:32.613 INFO:tasks.workunit.client.0.smithi046.stdout:2/870: rename d2/d6/d15/lee to d2/d6/d23/l131 0 2024-09-10T04:05:32.613 INFO:tasks.workunit.client.1.smithi070.stdout:9/945: dwrite d9/f73 [0,4194304] 0 2024-09-10T04:05:32.616 INFO:tasks.workunit.client.1.smithi070.stdout:5/979: dread d17/d19/d6a/d54/d57/da3/f82 [0,4194304] 0 2024-09-10T04:05:32.616 INFO:tasks.workunit.client.0.smithi046.stdout:3/839: getdents d12/d13 0 2024-09-10T04:05:32.617 INFO:tasks.workunit.client.1.smithi070.stdout:3/887: symlink db/d1c/d25/d28/dd3/l134 0 2024-09-10T04:05:32.618 INFO:tasks.workunit.client.0.smithi046.stdout:3/840: chown d12/d13/d37/d51/db5/d50/f55 1183088 1 2024-09-10T04:05:32.618 INFO:tasks.workunit.client.1.smithi070.stdout:3/888: dread - db/d1c/d3e/ff1 zero size 2024-09-10T04:05:32.624 INFO:tasks.workunit.client.1.smithi070.stdout:7/981: dread d2/d3/dc/d13/fc7 [0,4194304] 0 2024-09-10T04:05:32.624 INFO:tasks.workunit.client.0.smithi046.stdout:9/854: dread d4/d3c/de0/f63 [0,4194304] 0 2024-09-10T04:05:32.625 INFO:tasks.workunit.client.0.smithi046.stdout:8/878: mknod d1/d29/d19/d36/c123 0 2024-09-10T04:05:32.627 INFO:tasks.workunit.client.1.smithi070.stdout:9/946: chown d9/d1c/d34/d8b/d11d/d1b/f123 1046 1 2024-09-10T04:05:32.627 INFO:tasks.workunit.client.0.smithi046.stdout:3/841: rmdir d12/d13/d37/d51/db5/d50/d7a/dc1/d96 39 2024-09-10T04:05:32.629 INFO:tasks.workunit.client.0.smithi046.stdout:6/768: rename da/db/d11/d13/d4f/cd5 to da/db/d11/d9f/dd6/cfb 0 2024-09-10T04:05:32.630 INFO:tasks.workunit.client.0.smithi046.stdout:2/871: mknod d2/d6/d23/d5d/c132 0 2024-09-10T04:05:32.635 INFO:tasks.workunit.client.0.smithi046.stdout:9/855: symlink d4/d66/l10d 0 2024-09-10T04:05:32.636 INFO:tasks.workunit.client.1.smithi070.stdout:2/851: dread d15/d18/d29/d2a/d2e/d32/f33 [0,4194304] 0 2024-09-10T04:05:32.637 INFO:tasks.workunit.client.1.smithi070.stdout:2/852: write d15/d18/f26 [1172310,45206] 0 2024-09-10T04:05:32.638 INFO:tasks.workunit.client.1.smithi070.stdout:2/853: stat d15/d18/d56/d7c/d55/c133 0 2024-09-10T04:05:32.639 INFO:tasks.workunit.client.0.smithi046.stdout:7/921: sync 2024-09-10T04:05:32.641 INFO:tasks.workunit.client.0.smithi046.stdout:7/922: chown d5/d6/d62/dc1 1257625 1 2024-09-10T04:05:32.641 INFO:tasks.workunit.client.1.smithi070.stdout:9/947: mkdir d9/d7a/d74/d8c/d131 0 2024-09-10T04:05:32.643 INFO:tasks.workunit.client.0.smithi046.stdout:8/879: dwrite d1/d29/d17/d1f/d46/d76/fab [0,4194304] 0 2024-09-10T04:05:32.643 INFO:tasks.workunit.client.1.smithi070.stdout:4/880: write de/d13/d18/d26/d8c/f95 [900863,50846] 0 2024-09-10T04:05:32.644 INFO:tasks.workunit.client.1.smithi070.stdout:9/948: write d9/d59/d2a/d3f/fe4 [45994,117723] 0 2024-09-10T04:05:32.646 INFO:tasks.workunit.client.0.smithi046.stdout:9/856: write d4/d18/d89/ffa [260548,19095] 0 2024-09-10T04:05:32.648 INFO:tasks.workunit.client.1.smithi070.stdout:3/889: creat db/d1c/d89/f135 x:0 0 0 2024-09-10T04:05:32.649 INFO:tasks.workunit.client.1.smithi070.stdout:8/966: dwrite d4/d39/dfc/da8/ffb [0,4194304] 0 2024-09-10T04:05:32.652 INFO:tasks.workunit.client.0.smithi046.stdout:9/857: write d4/d18/da8/fcf [1418212,104296] 0 2024-09-10T04:05:32.652 INFO:tasks.workunit.client.0.smithi046.stdout:3/842: dread - d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fe4 zero size 2024-09-10T04:05:32.654 INFO:tasks.workunit.client.0.smithi046.stdout:7/923: stat d5/d6/db/f38 0 2024-09-10T04:05:32.654 INFO:tasks.workunit.client.1.smithi070.stdout:5/980: write d17/d19/d6a/d54/d57/d63/db9/fc4 [972690,32002] 0 2024-09-10T04:05:32.656 INFO:tasks.workunit.client.0.smithi046.stdout:0/706: dwrite d1/dc/d25/d33/d3/dd/d55/f43 [0,4194304] 0 2024-09-10T04:05:32.657 INFO:tasks.workunit.client.1.smithi070.stdout:6/911: write d0/d3/d8c/d9a/f113 [1179569,49241] 0 2024-09-10T04:05:32.658 INFO:tasks.workunit.client.1.smithi070.stdout:2/854: dread d15/d18/d29/f7f [0,4194304] 0 2024-09-10T04:05:32.661 INFO:tasks.workunit.client.1.smithi070.stdout:7/982: write d2/d40/d6c/db2/db7/d104/f105 [929445,105905] 0 2024-09-10T04:05:32.664 INFO:tasks.workunit.client.1.smithi070.stdout:9/949: dread d9/d59/d2a/d3f/f86 [0,4194304] 0 2024-09-10T04:05:32.664 INFO:tasks.workunit.client.1.smithi070.stdout:4/881: symlink de/d13/d5b/d7b/dd5/d85/d10e/d48/l129 0 2024-09-10T04:05:32.665 INFO:tasks.workunit.client.1.smithi070.stdout:3/890: stat db/d1c/c49 0 2024-09-10T04:05:32.667 INFO:tasks.workunit.client.1.smithi070.stdout:4/882: write de/d13/d43/d88/db5/f10c [656910,122715] 0 2024-09-10T04:05:32.667 INFO:tasks.workunit.client.1.smithi070.stdout:9/950: read d9/d59/f1a [3970401,64078] 0 2024-09-10T04:05:32.668 INFO:tasks.workunit.client.1.smithi070.stdout:5/981: stat d17/d19/d6a/cff 0 2024-09-10T04:05:32.669 INFO:tasks.workunit.client.0.smithi046.stdout:6/769: creat da/db/d11/ffc x:0 0 0 2024-09-10T04:05:32.671 INFO:tasks.workunit.client.0.smithi046.stdout:2/872: dread d2/d6/d23/d5d/d77/ff5 [0,4194304] 0 2024-09-10T04:05:32.671 INFO:tasks.workunit.client.0.smithi046.stdout:4/972: sync 2024-09-10T04:05:32.673 INFO:tasks.workunit.client.1.smithi070.stdout:2/855: fdatasync d15/d18/d29/d2a/d2e/d32/d36/f5b 0 2024-09-10T04:05:32.674 INFO:tasks.workunit.client.0.smithi046.stdout:8/880: dread d1/d29/d19/f56 [0,4194304] 0 2024-09-10T04:05:32.674 INFO:tasks.workunit.client.0.smithi046.stdout:2/873: read d2/d6/f61 [108023,38525] 0 2024-09-10T04:05:32.675 INFO:tasks.workunit.client.1.smithi070.stdout:9/951: dread d9/fb7 [0,4194304] 0 2024-09-10T04:05:32.675 INFO:tasks.workunit.client.0.smithi046.stdout:8/881: chown d1/d29/f27 124241871 1 2024-09-10T04:05:32.676 INFO:tasks.workunit.client.1.smithi070.stdout:9/952: chown d9/d59/fb5 1402 1 2024-09-10T04:05:32.679 INFO:tasks.workunit.client.0.smithi046.stdout:9/858: mkdir d4/d29/d10e 0 2024-09-10T04:05:32.681 INFO:tasks.workunit.client.1.smithi070.stdout:3/891: truncate db/f4e 3650215 0 2024-09-10T04:05:32.683 INFO:tasks.workunit.client.0.smithi046.stdout:7/924: write d5/d9/d43/db4/fca [731555,128716] 0 2024-09-10T04:05:32.684 INFO:tasks.workunit.client.0.smithi046.stdout:3/843: dread d12/d13/d37/d51/db5/d50/d7a/dc1/fbb [0,4194304] 0 2024-09-10T04:05:32.684 INFO:tasks.workunit.client.0.smithi046.stdout:7/925: dread - d5/d6/d39/f11a zero size 2024-09-10T04:05:32.686 INFO:tasks.workunit.client.0.smithi046.stdout:7/926: chown d5/d33/d135 680 1 2024-09-10T04:05:32.686 INFO:tasks.workunit.client.0.smithi046.stdout:3/844: chown d12/d13/d37/d51/db5/d4b/c89 1928 1 2024-09-10T04:05:32.688 INFO:tasks.workunit.client.0.smithi046.stdout:4/973: rmdir d3/d12/d17/d22/d51 39 2024-09-10T04:05:32.689 INFO:tasks.workunit.client.1.smithi070.stdout:6/912: mkdir d0/d3/d18/d1b/d17/d127 0 2024-09-10T04:05:32.691 INFO:tasks.workunit.client.1.smithi070.stdout:6/913: chown d0/d3/d1f/d93 568 1 2024-09-10T04:05:32.693 INFO:tasks.workunit.client.1.smithi070.stdout:2/856: creat d15/d18/d29/d2a/d2e/da6/dc0/dca/f136 x:0 0 0 2024-09-10T04:05:32.693 INFO:tasks.workunit.client.1.smithi070.stdout:6/914: fdatasync d0/d3/da4/dc0/f119 0 2024-09-10T04:05:32.695 INFO:tasks.workunit.client.1.smithi070.stdout:9/953: creat d9/d1c/d34/d8b/d11d/d3c/d92/f132 x:0 0 0 2024-09-10T04:05:32.695 INFO:tasks.workunit.client.1.smithi070.stdout:7/983: dwrite d2/dc5/fe1 [0,4194304] 0 2024-09-10T04:05:32.696 INFO:tasks.workunit.client.0.smithi046.stdout:0/707: creat d1/dc/d25/d33/d3/fdf x:0 0 0 2024-09-10T04:05:32.697 INFO:tasks.workunit.client.1.smithi070.stdout:8/967: dwrite d4/d39/dfc/d43/d67/d97/f82 [0,4194304] 0 2024-09-10T04:05:32.697 INFO:tasks.workunit.client.0.smithi046.stdout:3/845: rmdir d12/d13 39 2024-09-10T04:05:32.699 INFO:tasks.workunit.client.1.smithi070.stdout:7/984: write d2/d7/d9/f82 [8659655,40210] 0 2024-09-10T04:05:32.701 INFO:tasks.workunit.client.1.smithi070.stdout:4/883: dwrite de/d13/d18/d23/f52 [0,4194304] 0 2024-09-10T04:05:32.703 INFO:tasks.workunit.client.0.smithi046.stdout:2/874: symlink d2/d8/d37/dc2/d123/l133 0 2024-09-10T04:05:32.704 INFO:tasks.workunit.client.0.smithi046.stdout:2/875: chown d2/d6/d23/dd5/df8 716 1 2024-09-10T04:05:32.714 INFO:tasks.workunit.client.0.smithi046.stdout:6/770: creat da/db/d11/d13/d1e/d69/ddf/ffd x:0 0 0 2024-09-10T04:05:32.716 INFO:tasks.workunit.client.0.smithi046.stdout:7/927: dwrite d5/d9/d12/d72/d77/de0/d1e/f30 [0,4194304] 0 2024-09-10T04:05:32.717 INFO:tasks.workunit.client.0.smithi046.stdout:3/846: chown d12/d13 0 1 2024-09-10T04:05:32.718 INFO:tasks.workunit.client.0.smithi046.stdout:3/847: readlink d12/d13/d37/d51/dbd/l11d 0 2024-09-10T04:05:32.719 INFO:tasks.workunit.client.0.smithi046.stdout:4/974: mkdir d3/d12/d17/d14b 0 2024-09-10T04:05:32.720 INFO:tasks.workunit.client.1.smithi070.stdout:8/968: mknod d4/d16/d73/d139/c14b 0 2024-09-10T04:05:32.720 INFO:tasks.workunit.client.0.smithi046.stdout:1/683: sync 2024-09-10T04:05:32.720 INFO:tasks.workunit.client.0.smithi046.stdout:5/964: sync 2024-09-10T04:05:32.721 INFO:tasks.workunit.client.0.smithi046.stdout:2/876: fdatasync d2/d8/d65/fc7 0 2024-09-10T04:05:32.723 INFO:tasks.workunit.client.0.smithi046.stdout:5/965: truncate d7/d11/d14/d2f/deb/f12a 471923 0 2024-09-10T04:05:32.726 INFO:tasks.workunit.client.0.smithi046.stdout:9/859: truncate d4/d7/d1f/dce/fec 706872 0 2024-09-10T04:05:32.726 INFO:tasks.workunit.client.0.smithi046.stdout:9/860: stat d4/d3c/l73 0 2024-09-10T04:05:32.727 INFO:tasks.workunit.client.0.smithi046.stdout:0/708: mkdir d1/d13/d37/d75/d94/d9a/dd7/de0 0 2024-09-10T04:05:32.729 INFO:tasks.workunit.client.1.smithi070.stdout:4/884: creat de/d13/d18/d26/d8c/f12a x:0 0 0 2024-09-10T04:05:32.729 INFO:tasks.workunit.client.0.smithi046.stdout:6/771: unlink da/d59/l5b 0 2024-09-10T04:05:32.731 INFO:tasks.workunit.client.1.smithi070.stdout:5/982: mknod d17/d24/dcc/c14e 0 2024-09-10T04:05:32.732 INFO:tasks.workunit.client.1.smithi070.stdout:6/915: truncate d0/d3/d18/d1b/f7d 1958595 0 2024-09-10T04:05:32.735 INFO:tasks.workunit.client.1.smithi070.stdout:2/857: unlink d15/d18/d29/d2a/d2e/d32/d36/c47 0 2024-09-10T04:05:32.736 INFO:tasks.workunit.client.1.smithi070.stdout:9/954: write d9/d1c/d34/d8b/d11d/dc4/de1/fd8 [262358,26261] 0 2024-09-10T04:05:32.736 INFO:tasks.workunit.client.1.smithi070.stdout:7/985: write d2/d3/dc/d5d/d42/fc4 [29129,28082] 0 2024-09-10T04:05:32.737 INFO:tasks.workunit.client.0.smithi046.stdout:1/684: creat d3/da1/fe4 x:0 0 0 2024-09-10T04:05:32.738 INFO:tasks.workunit.client.1.smithi070.stdout:7/986: read - d2/d7/d9/d18/d3c/d52/fc9 zero size 2024-09-10T04:05:32.740 INFO:tasks.workunit.client.1.smithi070.stdout:7/987: fsync d2/d7/d9/d18/d3c/d52/f153 0 2024-09-10T04:05:32.741 INFO:tasks.workunit.client.0.smithi046.stdout:8/882: getdents d1/d29/d19/d36/d4c/d5e 0 2024-09-10T04:05:32.742 INFO:tasks.workunit.client.1.smithi070.stdout:9/955: dwrite d9/d59/f126 [0,4194304] 0 2024-09-10T04:05:32.743 INFO:tasks.workunit.client.0.smithi046.stdout:2/877: mknod d2/d6/d23/d42/dc8/c134 0 2024-09-10T04:05:32.745 INFO:tasks.workunit.client.0.smithi046.stdout:4/975: write d3/d12/d17/d29/fbb [1392435,109303] 0 2024-09-10T04:05:32.746 INFO:tasks.workunit.client.1.smithi070.stdout:7/988: dwrite d2/d3/dc/d79/fed [4194304,4194304] 0 2024-09-10T04:05:32.747 INFO:tasks.workunit.client.0.smithi046.stdout:9/861: symlink d4/d3c/de0/l10f 0 2024-09-10T04:05:32.748 INFO:tasks.workunit.client.1.smithi070.stdout:6/916: symlink d0/d3/d18/df5/l128 0 2024-09-10T04:05:32.751 INFO:tasks.workunit.client.1.smithi070.stdout:2/858: write d15/d18/d56/d7c/f9e [1075404,30625] 0 2024-09-10T04:05:32.752 INFO:tasks.workunit.client.0.smithi046.stdout:6/772: mkdir da/db/d11/d13/d1e/d23/d54/dad/dfe 0 2024-09-10T04:05:32.754 INFO:tasks.workunit.client.0.smithi046.stdout:6/773: write da/db/d11/d2b/d5a/fd9 [355735,108076] 0 2024-09-10T04:05:32.754 INFO:tasks.workunit.client.1.smithi070.stdout:8/969: truncate d4/d39/dfc/d43/f115 4414555 0 2024-09-10T04:05:32.758 INFO:tasks.workunit.client.1.smithi070.stdout:2/859: dwrite d15/d2b/d6e/fe0 [0,4194304] 0 2024-09-10T04:05:32.759 INFO:tasks.workunit.client.1.smithi070.stdout:1/812: sync 2024-09-10T04:05:32.760 INFO:tasks.workunit.client.0.smithi046.stdout:4/976: dread d3/d12/d17/d29/d58/db2/d5f/d85/da8/fea [0,4194304] 0 2024-09-10T04:05:32.760 INFO:tasks.workunit.client.1.smithi070.stdout:3/892: link db/d1c/d25/d28/f7c db/d1c/d25/d28/dd3/de5/d10f/d110/f136 0 2024-09-10T04:05:32.762 INFO:tasks.workunit.client.0.smithi046.stdout:1/685: fsync d3/d9b/f9e 0 2024-09-10T04:05:32.762 INFO:tasks.workunit.client.1.smithi070.stdout:2/860: chown d15/d18/d56/d7c/dfa/f10f 8 1 2024-09-10T04:05:32.763 INFO:tasks.workunit.client.1.smithi070.stdout:5/983: dwrite f12 [0,4194304] 0 2024-09-10T04:05:32.764 INFO:tasks.workunit.client.0.smithi046.stdout:0/709: write d1/dc/d25/d33/d1a/f47 [829306,22366] 0 2024-09-10T04:05:32.767 INFO:tasks.workunit.client.1.smithi070.stdout:4/885: mkdir de/d13/d18/d26/d111/d12b 0 2024-09-10T04:05:32.768 INFO:tasks.workunit.client.0.smithi046.stdout:7/928: write d5/d6/d32/f12c [580061,2329] 0 2024-09-10T04:05:32.768 INFO:tasks.workunit.client.1.smithi070.stdout:7/989: rmdir d2/d40/d12e 39 2024-09-10T04:05:32.771 INFO:tasks.workunit.client.0.smithi046.stdout:3/848: write d12/f74 [367384,129845] 0 2024-09-10T04:05:32.772 INFO:tasks.workunit.client.0.smithi046.stdout:3/849: dread - d12/d13/ff9 zero size 2024-09-10T04:05:32.773 INFO:tasks.workunit.client.1.smithi070.stdout:6/917: mkdir d0/d3/d1f/d55/d129 0 2024-09-10T04:05:32.774 INFO:tasks.workunit.client.0.smithi046.stdout:9/862: creat d4/d18/d92/f110 x:0 0 0 2024-09-10T04:05:32.777 INFO:tasks.workunit.client.1.smithi070.stdout:1/813: symlink d3/d10/db2/dc7/d54/l109 0 2024-09-10T04:05:32.779 INFO:tasks.workunit.client.0.smithi046.stdout:6/774: truncate da/db/d11/f93 70237 0 2024-09-10T04:05:32.780 INFO:tasks.workunit.client.1.smithi070.stdout:3/893: truncate db/d83/d92/faa 1175885 0 2024-09-10T04:05:32.786 INFO:tasks.workunit.client.0.smithi046.stdout:1/686: creat d3/dc/d21/d3a/d90/dbf/fe5 x:0 0 0 2024-09-10T04:05:32.786 INFO:tasks.workunit.client.1.smithi070.stdout:8/970: write d4/d39/f57 [44301,22204] 0 2024-09-10T04:05:32.786 INFO:tasks.workunit.client.1.smithi070.stdout:3/894: dwrite db/d1c/d25/d28/f37 [4194304,4194304] 0 2024-09-10T04:05:32.787 INFO:tasks.workunit.client.0.smithi046.stdout:1/687: write d3/dc/d21/d3a/faf [602284,76238] 0 2024-09-10T04:05:32.787 INFO:tasks.workunit.client.0.smithi046.stdout:8/883: mkdir d1/d29/d17/d1f/d46/d67/dc1/d124 0 2024-09-10T04:05:32.790 INFO:tasks.workunit.client.1.smithi070.stdout:5/984: write d17/ff4 [2621340,80209] 0 2024-09-10T04:05:32.791 INFO:tasks.workunit.client.1.smithi070.stdout:5/985: chown d17/d19/d6a/d54/d57/da3/f97 80 1 2024-09-10T04:05:32.791 INFO:tasks.workunit.client.1.smithi070.stdout:7/990: mknod d2/d66/c15d 0 2024-09-10T04:05:32.792 INFO:tasks.workunit.client.1.smithi070.stdout:5/986: chown d17/d19/d6a/d54/d57/d63/fc6 2088344 1 2024-09-10T04:05:32.798 INFO:tasks.workunit.client.0.smithi046.stdout:5/966: creat d7/d11/d14/d2f/deb/da5/dcc/dc0/f14a x:0 0 0 2024-09-10T04:05:32.799 INFO:tasks.workunit.client.0.smithi046.stdout:3/850: mknod d12/d13/d37/d51/db5/d4b/d91/c125 0 2024-09-10T04:05:32.800 INFO:tasks.workunit.client.1.smithi070.stdout:2/861: dwrite d15/d18/d29/d2a/d2e/d32/d9b/f12d [0,4194304] 0 2024-09-10T04:05:32.800 INFO:tasks.workunit.client.0.smithi046.stdout:4/977: dwrite d3/d12/d14/fd8 [0,4194304] 0 2024-09-10T04:05:32.801 INFO:tasks.workunit.client.1.smithi070.stdout:9/956: creat d9/d7a/f133 x:0 0 0 2024-09-10T04:05:32.802 INFO:tasks.workunit.client.1.smithi070.stdout:2/862: dread d15/d18/d99/fe1 [0,4194304] 0 2024-09-10T04:05:32.803 INFO:tasks.workunit.client.0.smithi046.stdout:1/688: symlink d3/dc/d21/d3a/le6 0 2024-09-10T04:05:32.805 INFO:tasks.workunit.client.1.smithi070.stdout:3/895: unlink db/d1c/d89/ce4 0 2024-09-10T04:05:32.810 INFO:tasks.workunit.client.1.smithi070.stdout:8/971: mknod d4/d16/d73/d139/d7a/d84/c14c 0 2024-09-10T04:05:32.811 INFO:tasks.workunit.client.1.smithi070.stdout:8/972: readlink d4/d16/dcc/d12e/d11b/l2d 0 2024-09-10T04:05:32.813 INFO:tasks.workunit.client.0.smithi046.stdout:7/929: write d5/d9/d12/d72/fed [406303,72816] 0 2024-09-10T04:05:32.814 INFO:tasks.workunit.client.1.smithi070.stdout:6/918: dread d0/d3/d1f/fc8 [0,4194304] 0 2024-09-10T04:05:32.814 INFO:tasks.workunit.client.0.smithi046.stdout:5/967: dwrite d7/d11/d14/d2f/deb/da5/d123/f136 [0,4194304] 0 2024-09-10T04:05:32.817 INFO:tasks.workunit.client.0.smithi046.stdout:7/930: chown d5/d6/d32/dab/d127 48967 1 2024-09-10T04:05:32.817 INFO:tasks.workunit.client.0.smithi046.stdout:9/863: dwrite d4/d7/d61/f8e [0,4194304] 0 2024-09-10T04:05:32.817 INFO:tasks.workunit.client.1.smithi070.stdout:5/987: truncate fd 3765744 0 2024-09-10T04:05:32.821 INFO:tasks.workunit.client.0.smithi046.stdout:0/710: dwrite d1/dc/f21 [0,4194304] 0 2024-09-10T04:05:32.822 INFO:tasks.workunit.client.1.smithi070.stdout:7/991: truncate d2/d3/dc/d5d/ff1 822896 0 2024-09-10T04:05:32.827 INFO:tasks.workunit.client.1.smithi070.stdout:4/886: truncate de/d13/fec 2717710 0 2024-09-10T04:05:32.831 INFO:tasks.workunit.client.1.smithi070.stdout:8/973: chown d4/d39/dfc/da8/caf 124071 1 2024-09-10T04:05:32.832 INFO:tasks.workunit.client.0.smithi046.stdout:3/851: creat d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/f126 x:0 0 0 2024-09-10T04:05:32.833 INFO:tasks.workunit.client.1.smithi070.stdout:6/919: stat d0/d3/d18/d62/db8/lc2 0 2024-09-10T04:05:32.836 INFO:tasks.workunit.client.0.smithi046.stdout:8/884: symlink d1/d29/d17/d1f/d46/d67/dc1/d9d/def/l125 0 2024-09-10T04:05:32.838 INFO:tasks.workunit.client.0.smithi046.stdout:1/689: creat d3/dc/d21/d3a/d90/fe7 x:0 0 0 2024-09-10T04:05:32.839 INFO:tasks.workunit.client.1.smithi070.stdout:7/992: truncate d2/d7/d9/d18/d23/f47 5096963 0 2024-09-10T04:05:32.839 INFO:tasks.workunit.client.0.smithi046.stdout:1/690: fdatasync d3/dc/d21/d3a/faf 0 2024-09-10T04:05:32.840 INFO:tasks.workunit.client.0.smithi046.stdout:2/878: link d2/d8/d37/d63/ld9 d2/d6/d23/d55/l135 0 2024-09-10T04:05:32.841 INFO:tasks.workunit.client.0.smithi046.stdout:0/711: fsync d1/dc/f21 0 2024-09-10T04:05:32.843 INFO:tasks.workunit.client.1.smithi070.stdout:1/814: dwrite d3/d8/d13/f18 [0,4194304] 0 2024-09-10T04:05:32.846 INFO:tasks.workunit.client.0.smithi046.stdout:9/864: rmdir d4/d7/d81 39 2024-09-10T04:05:32.848 INFO:tasks.workunit.client.1.smithi070.stdout:8/974: rename d4/fe to d4/d39/f14d 0 2024-09-10T04:05:32.850 INFO:tasks.workunit.client.0.smithi046.stdout:7/931: fsync d5/d6/d32/dab/db6/f79 0 2024-09-10T04:05:32.851 INFO:tasks.workunit.client.0.smithi046.stdout:7/932: readlink d5/d6/d39/d5f/l94 0 2024-09-10T04:05:32.855 INFO:tasks.workunit.client.0.smithi046.stdout:4/978: creat d3/dab/d11c/f14c x:0 0 0 2024-09-10T04:05:32.857 INFO:tasks.workunit.client.1.smithi070.stdout:7/993: creat d2/d7/d9/dd5/d102/f15e x:0 0 0 2024-09-10T04:05:32.859 INFO:tasks.workunit.client.1.smithi070.stdout:7/994: fsync d2/d3/d63/fef 0 2024-09-10T04:05:32.860 INFO:tasks.workunit.client.1.smithi070.stdout:7/995: chown d2/d3/dc/d13 166990218 1 2024-09-10T04:05:32.863 INFO:tasks.workunit.client.0.smithi046.stdout:1/691: unlink d3/d27/d5d/f9a 0 2024-09-10T04:05:32.863 INFO:tasks.workunit.client.1.smithi070.stdout:4/887: dwrite de/d13/d18/d23/d47/d59/f87 [0,4194304] 0 2024-09-10T04:05:32.865 INFO:tasks.workunit.client.1.smithi070.stdout:6/920: dread d0/d3/d18/d1b/d17/f104 [0,4194304] 0 2024-09-10T04:05:32.865 INFO:tasks.workunit.client.1.smithi070.stdout:1/815: stat d3/c1a 0 2024-09-10T04:05:32.865 INFO:tasks.workunit.client.0.smithi046.stdout:5/968: mkdir d7/d11/d3f/d14b 0 2024-09-10T04:05:32.866 INFO:tasks.workunit.client.0.smithi046.stdout:8/885: write d1/d29/d19/d36/d4c/fa9 [2991557,54791] 0 2024-09-10T04:05:32.867 INFO:tasks.workunit.client.1.smithi070.stdout:1/816: dread - d3/d10/d1d/d79/ff0 zero size 2024-09-10T04:05:32.867 INFO:tasks.workunit.client.1.smithi070.stdout:6/921: truncate d0/d3/d8c/f110 946079 0 2024-09-10T04:05:32.868 INFO:tasks.workunit.client.1.smithi070.stdout:6/922: write d0/d3/d8c/ff1 [906479,25875] 0 2024-09-10T04:05:32.871 INFO:tasks.workunit.client.1.smithi070.stdout:8/975: truncate d4/d16/f34 1374364 0 2024-09-10T04:05:32.873 INFO:tasks.workunit.client.0.smithi046.stdout:3/852: dread d12/d13/d14/d1e/d2d/f88 [0,4194304] 0 2024-09-10T04:05:32.874 INFO:tasks.workunit.client.1.smithi070.stdout:5/988: creat d17/d6f/d32/f14f x:0 0 0 2024-09-10T04:05:32.875 INFO:tasks.workunit.client.0.smithi046.stdout:9/865: write d4/d44/f49 [2926602,2339] 0 2024-09-10T04:05:32.878 INFO:tasks.workunit.client.0.smithi046.stdout:0/712: creat d1/dc/d25/d33/d3/dcd/fe1 x:0 0 0 2024-09-10T04:05:32.879 INFO:tasks.workunit.client.0.smithi046.stdout:8/886: fdatasync d1/d29/d19/d36/f4f 0 2024-09-10T04:05:32.879 INFO:tasks.workunit.client.0.smithi046.stdout:0/713: fdatasync d1/dc/d25/d33/d1a/d41/f6e 0 2024-09-10T04:05:32.880 INFO:tasks.workunit.client.1.smithi070.stdout:4/888: creat de/d13/d18/d26/d8c/d124/f12c x:0 0 0 2024-09-10T04:05:32.882 INFO:tasks.workunit.client.0.smithi046.stdout:7/933: mknod d5/d9/d12/d72/d77/de0/d1e/c146 0 2024-09-10T04:05:32.883 INFO:tasks.workunit.client.1.smithi070.stdout:6/923: truncate d0/d3/d8c/d9a/fd9 632426 0 2024-09-10T04:05:32.884 INFO:tasks.workunit.client.0.smithi046.stdout:4/979: symlink d3/d12/d146/l14d 0 2024-09-10T04:05:32.886 INFO:tasks.workunit.client.0.smithi046.stdout:3/853: mknod d12/d13/c127 0 2024-09-10T04:05:32.887 INFO:tasks.workunit.client.1.smithi070.stdout:5/989: creat d17/d19/d67/f150 x:0 0 0 2024-09-10T04:05:32.888 INFO:tasks.workunit.client.1.smithi070.stdout:3/896: getdents db/dcc 0 2024-09-10T04:05:32.889 INFO:tasks.workunit.client.0.smithi046.stdout:5/969: symlink d7/l14c 0 2024-09-10T04:05:32.890 INFO:tasks.workunit.client.0.smithi046.stdout:9/866: creat d4/d3c/de0/d95/f111 x:0 0 0 2024-09-10T04:05:32.891 INFO:tasks.workunit.client.1.smithi070.stdout:4/889: unlink de/d13/da6/ddb/f103 0 2024-09-10T04:05:32.892 INFO:tasks.workunit.client.0.smithi046.stdout:0/714: rmdir d1/dc/d25/d33/d3/d29/d2d/d5e 39 2024-09-10T04:05:32.894 INFO:tasks.workunit.client.1.smithi070.stdout:7/996: creat d2/d7/d9/f15f x:0 0 0 2024-09-10T04:05:32.895 INFO:tasks.workunit.client.1.smithi070.stdout:7/997: write d2/d3/dc/d5d/d42/fc4 [1454758,20061] 0 2024-09-10T04:05:32.896 INFO:tasks.workunit.client.1.smithi070.stdout:7/998: readlink d2/d7/la1 0 2024-09-10T04:05:32.897 INFO:tasks.workunit.client.0.smithi046.stdout:4/980: creat d3/d12/d17/d29/d58/db2/d5f/d85/f14e x:0 0 0 2024-09-10T04:05:32.898 INFO:tasks.workunit.client.1.smithi070.stdout:8/976: dread d4/d39/f14d [0,4194304] 0 2024-09-10T04:05:32.898 INFO:tasks.workunit.client.0.smithi046.stdout:4/981: read d3/d12/d17/f20 [2406210,33824] 0 2024-09-10T04:05:32.900 INFO:tasks.workunit.client.1.smithi070.stdout:6/924: symlink d0/d3/d8c/d9a/d11f/l12a 0 2024-09-10T04:05:32.902 INFO:tasks.workunit.client.1.smithi070.stdout:4/890: creat de/d13/d5b/da0/f12d x:0 0 0 2024-09-10T04:05:32.903 INFO:tasks.workunit.client.0.smithi046.stdout:5/970: mknod d7/d11/d14/d45/d51/c14d 0 2024-09-10T04:05:32.903 INFO:tasks.workunit.client.0.smithi046.stdout:5/971: stat d7/f97 0 2024-09-10T04:05:32.905 INFO:tasks.workunit.client.0.smithi046.stdout:5/972: write d7/d11/d3f/d53/d81/f113 [2900710,42712] 0 2024-09-10T04:05:32.906 INFO:tasks.workunit.client.0.smithi046.stdout:9/867: mkdir d4/d7/d1f/dce/dd4/d112 0 2024-09-10T04:05:32.907 INFO:tasks.workunit.client.1.smithi070.stdout:4/891: dwrite de/d13/d5b/d7b/dd5/d85/d10e/d48/ff5 [0,4194304] 0 2024-09-10T04:05:32.910 INFO:tasks.workunit.client.0.smithi046.stdout:8/887: getdents d1/d29/d19/d36/d4c/d5e/dc7/dea 0 2024-09-10T04:05:32.912 INFO:tasks.workunit.client.1.smithi070.stdout:7/999: symlink d2/d3/dc/d5d/l160 0 2024-09-10T04:05:32.912 INFO:tasks.workunit.client.0.smithi046.stdout:5/973: dwrite d7/d11/d3f/d5a/f130 [0,4194304] 0 2024-09-10T04:05:32.918 INFO:tasks.workunit.client.1.smithi070.stdout:3/897: dwrite db/d1c/d25/f55 [4194304,4194304] 0 2024-09-10T04:05:32.919 INFO:tasks.workunit.client.1.smithi070.stdout:6/925: rename d0/d3/d18/d2a/fee to d0/d3/d18/d1b/d33/f12b 0 2024-09-10T04:05:32.919 INFO:tasks.workunit.client.1.smithi070.stdout:3/898: chown db/d83/d92/f96 4366 1 2024-09-10T04:05:32.919 INFO:tasks.workunit.client.1.smithi070.stdout:3/899: chown db/d1c 15 1 2024-09-10T04:05:32.920 INFO:tasks.workunit.client.0.smithi046.stdout:1/692: getdents d3/dc/d21/d3a/d90 0 2024-09-10T04:05:32.925 INFO:tasks.workunit.client.0.smithi046.stdout:1/693: fsync d3/da1/fe4 0 2024-09-10T04:05:32.925 INFO:tasks.workunit.client.0.smithi046.stdout:1/694: readlink d3/dc/d21/lb4 0 2024-09-10T04:05:32.927 INFO:tasks.workunit.client.1.smithi070.stdout:5/990: getdents d17/d6f/d32/d90/d99 0 2024-09-10T04:05:32.929 INFO:tasks.workunit.client.0.smithi046.stdout:8/888: creat d1/d29/d17/d1f/d46/d76/f126 x:0 0 0 2024-09-10T04:05:32.930 INFO:tasks.workunit.client.1.smithi070.stdout:3/900: creat db/d1c/d25/d28/dd3/de5/d10f/db6/f137 x:0 0 0 2024-09-10T04:05:32.931 INFO:tasks.workunit.client.1.smithi070.stdout:3/901: truncate db/d83/d92/dc7/f120 891287 0 2024-09-10T04:05:32.933 INFO:tasks.workunit.client.1.smithi070.stdout:6/926: truncate d0/d3/d18/d1b/d17/d54/dc4/dd2/dde/fce 591744 0 2024-09-10T04:05:32.935 INFO:tasks.workunit.client.1.smithi070.stdout:8/977: mkdir d4/dbf/d14e 0 2024-09-10T04:05:32.935 INFO:tasks.workunit.client.0.smithi046.stdout:7/934: link d5/d7f/d129/lc0 d5/d6/d32/l147 0 2024-09-10T04:05:32.939 INFO:tasks.workunit.client.0.smithi046.stdout:4/982: write d3/d12/d17/d29/ddd/f12b [368709,21883] 0 2024-09-10T04:05:32.940 INFO:tasks.workunit.client.1.smithi070.stdout:3/902: mknod db/d83/d92/dc7/c138 0 2024-09-10T04:05:32.944 INFO:tasks.workunit.client.1.smithi070.stdout:8/978: rmdir d4/d16/dcc/d12e/d135/d13a 39 2024-09-10T04:05:32.945 INFO:tasks.workunit.client.0.smithi046.stdout:0/715: truncate d1/d13/d37/d75/fd0 394460 0 2024-09-10T04:05:32.946 INFO:tasks.workunit.client.0.smithi046.stdout:0/716: chown d1/dc/f21 166 1 2024-09-10T04:05:32.946 INFO:tasks.workunit.client.1.smithi070.stdout:4/892: dwrite de/f9b [0,4194304] 0 2024-09-10T04:05:32.947 INFO:tasks.workunit.client.0.smithi046.stdout:5/974: symlink d7/d11/d14/d2f/deb/dd5/l14e 0 2024-09-10T04:05:32.955 INFO:tasks.workunit.client.0.smithi046.stdout:9/868: link d4/d3c/d4d/d80/le2 d4/d18/d89/ddb/l113 0 2024-09-10T04:05:32.957 INFO:tasks.workunit.client.1.smithi070.stdout:4/893: unlink de/d13/d18/d5c/db6/f11b 0 2024-09-10T04:05:32.959 INFO:tasks.workunit.client.0.smithi046.stdout:8/889: mknod d1/c127 0 2024-09-10T04:05:32.961 INFO:tasks.workunit.client.1.smithi070.stdout:5/991: write f13 [3626678,11454] 0 2024-09-10T04:05:32.962 INFO:tasks.workunit.client.0.smithi046.stdout:0/717: symlink d1/dc/d25/d72/le2 0 2024-09-10T04:05:32.964 INFO:tasks.workunit.client.1.smithi070.stdout:3/903: write db/d1c/d25/d28/dd3/de5/d10f/db6/fc6 [958274,92304] 0 2024-09-10T04:05:32.964 INFO:tasks.workunit.client.0.smithi046.stdout:8/890: dread d1/d29/d17/d1f/d46/d6a/ff3 [0,4194304] 0 2024-09-10T04:05:32.964 INFO:tasks.workunit.client.0.smithi046.stdout:5/975: read - d7/d11/d14/d2f/d2b/fad zero size 2024-09-10T04:05:32.965 INFO:tasks.workunit.client.1.smithi070.stdout:3/904: write db/d83/d11a/f11f [919379,28964] 0 2024-09-10T04:05:32.971 INFO:tasks.workunit.client.1.smithi070.stdout:4/894: mkdir de/d13/d18/d23/d47/d59/d79/d6a/d12e 0 2024-09-10T04:05:32.972 INFO:tasks.workunit.client.0.smithi046.stdout:7/935: creat d5/d9/d12/d72/d104/f148 x:0 0 0 2024-09-10T04:05:32.973 INFO:tasks.workunit.client.1.smithi070.stdout:6/927: getdents d0/d3/d18/d1b/d17/d53/d10c 0 2024-09-10T04:05:32.975 INFO:tasks.workunit.client.1.smithi070.stdout:5/992: dread d17/d19/d6a/d54/d57/da8/fe2 [0,4194304] 0 2024-09-10T04:05:32.976 INFO:tasks.workunit.client.0.smithi046.stdout:9/869: read d4/d3c/f40 [168359,48018] 0 2024-09-10T04:05:32.977 INFO:tasks.workunit.client.1.smithi070.stdout:4/895: dread de/d13/d5b/da0/fc9 [0,4194304] 0 2024-09-10T04:05:32.982 INFO:tasks.workunit.client.1.smithi070.stdout:8/979: getdents d4/dbf 0 2024-09-10T04:05:32.983 INFO:tasks.workunit.client.0.smithi046.stdout:4/983: truncate d3/d12/d17/f20 4890097 0 2024-09-10T04:05:32.984 INFO:tasks.workunit.client.1.smithi070.stdout:3/905: write db/d1c/d3e/f50 [4389724,18877] 0 2024-09-10T04:05:32.985 INFO:tasks.workunit.client.1.smithi070.stdout:5/993: dread d17/d19/f23 [0,4194304] 0 2024-09-10T04:05:32.989 INFO:tasks.workunit.client.1.smithi070.stdout:6/928: symlink d0/d3/d1f/d81/l12c 0 2024-09-10T04:05:32.989 INFO:tasks.workunit.client.0.smithi046.stdout:0/718: write d1/dc/d25/d33/d1a/d41/f5a [841020,63267] 0 2024-09-10T04:05:32.992 INFO:tasks.workunit.client.1.smithi070.stdout:8/980: unlink d4/d39/dfc/d43/d67/dec/f13e 0 2024-09-10T04:05:32.998 INFO:tasks.workunit.client.1.smithi070.stdout:4/896: write de/d13/d5b/d7b/dd5/d85/d10e/f57 [1767632,10676] 0 2024-09-10T04:05:33.005 INFO:tasks.workunit.client.1.smithi070.stdout:3/906: write db/d1c/d25/f86 [469565,99506] 0 2024-09-10T04:05:33.005 INFO:tasks.workunit.client.1.smithi070.stdout:8/981: dread d4/d16/d73/d139/f9c [0,4194304] 0 2024-09-10T04:05:33.006 INFO:tasks.workunit.client.1.smithi070.stdout:5/994: mknod d17/d19/d6a/d54/d57/c151 0 2024-09-10T04:05:33.007 INFO:tasks.workunit.client.1.smithi070.stdout:8/982: readlink d4/d16/d73/d139/d7a/d84/l104 0 2024-09-10T04:05:33.007 INFO:tasks.workunit.client.1.smithi070.stdout:4/897: creat de/d13/d18/d26/d8c/f12f x:0 0 0 2024-09-10T04:05:33.008 INFO:tasks.workunit.client.1.smithi070.stdout:6/929: symlink d0/d3/d5c/l12d 0 2024-09-10T04:05:33.009 INFO:tasks.workunit.client.1.smithi070.stdout:3/907: mknod db/d1c/d25/d28/dd3/de5/d10f/d94/d99/c139 0 2024-09-10T04:05:33.011 INFO:tasks.workunit.client.1.smithi070.stdout:5/995: rmdir d17/d19/d6a/d54/de1/dfe/dce 39 2024-09-10T04:05:33.013 INFO:tasks.workunit.client.1.smithi070.stdout:6/930: read d0/d3/d18/d1b/d17/d53/fa3 [19761,28569] 0 2024-09-10T04:05:33.013 INFO:tasks.workunit.client.1.smithi070.stdout:3/908: readlink db/d1c/l88 0 2024-09-10T04:05:33.014 INFO:tasks.workunit.client.1.smithi070.stdout:5/996: mknod d17/d19/d6a/dfb/c152 0 2024-09-10T04:05:33.015 INFO:tasks.workunit.client.1.smithi070.stdout:3/909: dread db/d1c/d25/d28/f7d [0,4194304] 0 2024-09-10T04:05:33.015 INFO:tasks.workunit.client.1.smithi070.stdout:3/910: chown db/d1c/d25/ce3 6399 1 2024-09-10T04:05:33.016 INFO:tasks.workunit.client.1.smithi070.stdout:6/931: mknod d0/d3/d18/d2a/c12e 0 2024-09-10T04:05:33.018 INFO:tasks.workunit.client.1.smithi070.stdout:5/997: symlink d17/d19/d12d/l153 0 2024-09-10T04:05:33.018 INFO:tasks.workunit.client.1.smithi070.stdout:6/932: read d0/d3/da4/dc0/dea/f11b [135245,34289] 0 2024-09-10T04:05:33.019 INFO:tasks.workunit.client.1.smithi070.stdout:6/933: fsync d0/d3/da4/dc0/dea/f76 0 2024-09-10T04:05:33.019 INFO:tasks.workunit.client.1.smithi070.stdout:3/911: mknod db/d1c/d25/d28/dd3/de5/d10f/d110/c13a 0 2024-09-10T04:05:33.020 INFO:tasks.workunit.client.1.smithi070.stdout:3/912: readlink db/d1c/d25/d28/dd3/de5/l12a 0 2024-09-10T04:05:33.021 INFO:tasks.workunit.client.1.smithi070.stdout:8/983: rename d4/d16/dcc/d12e/d11b/d1a/df7/cf9 to d4/d39/dfc/d43/c14f 0 2024-09-10T04:05:33.025 INFO:tasks.workunit.client.1.smithi070.stdout:3/913: dwrite db/f33 [4194304,4194304] 0 2024-09-10T04:05:33.025 INFO:tasks.workunit.client.1.smithi070.stdout:5/998: fsync d17/d19/d6a/d54/de1/dfe/d93/fdf 0 2024-09-10T04:05:33.027 INFO:tasks.workunit.client.0.smithi046.stdout:1/695: link d3/da1/d1f/c50 d3/dc/d21/d3a/d90/dbf/ce8 0 2024-09-10T04:05:33.028 INFO:tasks.workunit.client.0.smithi046.stdout:1/696: write d3/da1/d1f/f29 [3425349,94963] 0 2024-09-10T04:05:33.029 INFO:tasks.workunit.client.1.smithi070.stdout:5/999: creat d17/d19/d35/f154 x:0 0 0 2024-09-10T04:05:33.031 INFO:tasks.workunit.client.0.smithi046.stdout:3/854: rmdir d12/d13/d37/d51/db5/d50/d7a/dc1/ddb 0 2024-09-10T04:05:33.031 INFO:tasks.workunit.client.1.smithi070.stdout:9/957: sync 2024-09-10T04:05:33.033 INFO:tasks.workunit.client.1.smithi070.stdout:3/914: creat db/d1c/d25/d28/dd3/de5/d10f/db5/d113/f13b x:0 0 0 2024-09-10T04:05:33.035 INFO:tasks.workunit.client.1.smithi070.stdout:6/934: rename d0/d3/d18/ce9 to d0/d3/d18/c12f 0 2024-09-10T04:05:33.038 INFO:tasks.workunit.client.0.smithi046.stdout:9/870: dread - d4/d3c/d4d/d5c/fc6 zero size 2024-09-10T04:05:33.039 INFO:tasks.workunit.client.1.smithi070.stdout:3/915: rename db/d1c/d25/d28/dd3/de5/d10f/c107 to db/d1c/d25/d28/dd3/de5/d10f/d94/d99/c13c 0 2024-09-10T04:05:33.041 INFO:tasks.workunit.client.0.smithi046.stdout:4/984: read - d3/dab/dca/de8/ff8 zero size 2024-09-10T04:05:33.045 INFO:tasks.workunit.client.1.smithi070.stdout:3/916: creat db/d83/d92/d11b/f13d x:0 0 0 2024-09-10T04:05:33.046 INFO:tasks.workunit.client.0.smithi046.stdout:5/976: mkdir d7/d11/d14/d2f/d14f 0 2024-09-10T04:05:33.047 INFO:tasks.workunit.client.0.smithi046.stdout:5/977: chown d7/d11/d14/d45/d51/f86 524 1 2024-09-10T04:05:33.050 INFO:tasks.workunit.client.0.smithi046.stdout:7/936: mkdir d5/d9/d12/d72/d77/de0/d1e/d149 0 2024-09-10T04:05:33.053 INFO:tasks.workunit.client.0.smithi046.stdout:4/985: creat d3/d12/d17/d29/dba/f14f x:0 0 0 2024-09-10T04:05:33.055 INFO:tasks.workunit.client.0.smithi046.stdout:7/937: dread d5/d9/d43/db4/fca [0,4194304] 0 2024-09-10T04:05:33.055 INFO:tasks.workunit.client.0.smithi046.stdout:8/891: creat d1/d29/d19/d36/f128 x:0 0 0 2024-09-10T04:05:33.057 INFO:tasks.workunit.client.0.smithi046.stdout:7/938: dread - d5/d9/d12/dc5/d125/f130 zero size 2024-09-10T04:05:33.063 INFO:tasks.workunit.client.0.smithi046.stdout:8/892: truncate d1/f79 305296 0 2024-09-10T04:05:33.065 INFO:tasks.workunit.client.0.smithi046.stdout:7/939: dwrite d5/d6/db/dd/d10c/f113 [0,4194304] 0 2024-09-10T04:05:33.067 INFO:tasks.workunit.client.0.smithi046.stdout:1/697: rename d3/ff to d3/d27/fe9 0 2024-09-10T04:05:33.073 INFO:tasks.workunit.client.0.smithi046.stdout:5/978: dread d7/d11/d14/d2f/d2b/d6a/fa1 [0,4194304] 0 2024-09-10T04:05:33.074 INFO:tasks.workunit.client.0.smithi046.stdout:4/986: unlink d3/d12/d17/de1/ddf/dfc/f12e 0 2024-09-10T04:05:33.078 INFO:tasks.workunit.client.0.smithi046.stdout:1/698: dwrite d3/dc/d21/d3a/d90/dbf/fe5 [0,4194304] 0 2024-09-10T04:05:33.081 INFO:tasks.workunit.client.0.smithi046.stdout:8/893: dread d1/d29/d19/d36/d4c/f7d [0,4194304] 0 2024-09-10T04:05:33.086 INFO:tasks.workunit.client.0.smithi046.stdout:1/699: unlink d3/dc/d34/d58/f63 0 2024-09-10T04:05:33.089 INFO:tasks.workunit.client.0.smithi046.stdout:4/987: truncate d3/d12/d17/d22/d51/fcc 2505393 0 2024-09-10T04:05:33.091 INFO:tasks.workunit.client.0.smithi046.stdout:7/940: getdents d5/d9/d12/dc5/d125 0 2024-09-10T04:05:33.092 INFO:tasks.workunit.client.0.smithi046.stdout:4/988: read d3/d12/d17/d29/d58/db2/d5f/f10a [2327751,118335] 0 2024-09-10T04:05:33.093 INFO:tasks.workunit.client.0.smithi046.stdout:1/700: mkdir d3/dc/d21/d3a/d90/dea 0 2024-09-10T04:05:33.094 INFO:tasks.workunit.client.0.smithi046.stdout:0/719: write d1/dc/d25/d33/d3/d29/f3c [1836510,31693] 0 2024-09-10T04:05:33.095 INFO:tasks.workunit.client.0.smithi046.stdout:8/894: mknod d1/d29/d19/d36/d4c/d5e/dc7/dea/c129 0 2024-09-10T04:05:33.097 INFO:tasks.workunit.client.0.smithi046.stdout:1/701: read d3/d27/fe9 [6540248,108617] 0 2024-09-10T04:05:33.098 INFO:tasks.workunit.client.0.smithi046.stdout:1/702: fdatasync d3/dc/d34/f72 0 2024-09-10T04:05:33.102 INFO:tasks.workunit.client.0.smithi046.stdout:4/989: unlink d3/d12/f4d 0 2024-09-10T04:05:33.107 INFO:tasks.workunit.client.0.smithi046.stdout:8/895: mknod d1/d29/d17/d1f/d46/d6a/d82/dc2/c12a 0 2024-09-10T04:05:33.108 INFO:tasks.workunit.client.0.smithi046.stdout:3/855: write d12/d13/f70 [475292,111742] 0 2024-09-10T04:05:33.109 INFO:tasks.workunit.client.0.smithi046.stdout:9/871: dwrite d4/fc9 [0,4194304] 0 2024-09-10T04:05:33.112 INFO:tasks.workunit.client.1.smithi070.stdout:9/958: truncate d9/d59/f126 3315078 0 2024-09-10T04:05:33.115 INFO:tasks.workunit.client.0.smithi046.stdout:4/990: fsync d3/d12/d17/d29/d58/db2/d5f/d85/fd7 0 2024-09-10T04:05:33.118 INFO:tasks.workunit.client.1.smithi070.stdout:6/935: creat d0/d3/d1f/d81/f130 x:0 0 0 2024-09-10T04:05:33.119 INFO:tasks.workunit.client.1.smithi070.stdout:6/936: write d0/d3/d8c/f120 [101627,126183] 0 2024-09-10T04:05:33.120 INFO:tasks.workunit.client.1.smithi070.stdout:6/937: stat d0/d3/da4/dc0/dea/f76 0 2024-09-10T04:05:33.120 INFO:tasks.workunit.client.1.smithi070.stdout:6/938: chown d0/f10 597928 1 2024-09-10T04:05:33.121 INFO:tasks.workunit.client.1.smithi070.stdout:3/917: truncate db/d1c/d25/d28/dd3/de5/d10f/db6/fbe 460682 0 2024-09-10T04:05:33.122 INFO:tasks.workunit.client.1.smithi070.stdout:3/918: dread - db/d1c/d25/d28/dd3/de5/d10f/db5/d113/f13b zero size 2024-09-10T04:05:33.123 INFO:tasks.workunit.client.0.smithi046.stdout:2/879: sync 2024-09-10T04:05:33.123 INFO:tasks.workunit.client.0.smithi046.stdout:6/775: sync 2024-09-10T04:05:33.123 INFO:tasks.workunit.client.0.smithi046.stdout:1/703: mkdir d3/d27/d5d/d60/dd7/deb 0 2024-09-10T04:05:33.123 INFO:tasks.workunit.client.1.smithi070.stdout:9/959: rmdir d9/d1c/d34/d8b/dcc 39 2024-09-10T04:05:33.129 INFO:tasks.workunit.client.0.smithi046.stdout:5/979: write d7/d11/d14/d8e/d96/fbe [4676094,1171] 0 2024-09-10T04:05:33.129 INFO:tasks.workunit.client.0.smithi046.stdout:4/991: dwrite d3/d12/d17/d29/d58/db2/d5f/d85/f14e [0,4194304] 0 2024-09-10T04:05:33.129 INFO:tasks.workunit.client.0.smithi046.stdout:4/992: chown d3/f9 158 1 2024-09-10T04:05:33.131 INFO:tasks.workunit.client.1.smithi070.stdout:9/960: creat d9/d7a/d74/f134 x:0 0 0 2024-09-10T04:05:33.131 INFO:tasks.workunit.client.0.smithi046.stdout:3/856: rmdir d12/d13/d37/d51/db5/d50/d7a/dc1/d96 39 2024-09-10T04:05:33.133 INFO:tasks.workunit.client.1.smithi070.stdout:9/961: write d9/d1c/d34/d8b/d11d/dc4/dda/de0/d90/f124 [336087,94081] 0 2024-09-10T04:05:33.133 INFO:tasks.workunit.client.1.smithi070.stdout:3/919: truncate db/d1c/d25/d28/dd3/de5/d10f/d94/d99/fa5 1171079 0 2024-09-10T04:05:33.133 INFO:tasks.workunit.client.0.smithi046.stdout:8/896: dread d1/d29/d17/d1f/fa3 [0,4194304] 0 2024-09-10T04:05:33.135 INFO:tasks.workunit.client.0.smithi046.stdout:9/872: dread d4/d18/d89/fa2 [0,4194304] 0 2024-09-10T04:05:33.135 INFO:tasks.workunit.client.1.smithi070.stdout:6/939: mknod d0/c131 0 2024-09-10T04:05:33.136 INFO:tasks.workunit.client.0.smithi046.stdout:3/857: truncate d12/d13/d37/d51/db5/d50/d52/f65 8945660 0 2024-09-10T04:05:33.136 INFO:tasks.workunit.client.1.smithi070.stdout:3/920: truncate db/d1c/d25/d28/dd3/de5/f118 1374529 0 2024-09-10T04:05:33.136 INFO:tasks.workunit.client.1.smithi070.stdout:6/940: stat d0/d3/d18/d1b/l31 0 2024-09-10T04:05:33.138 INFO:tasks.workunit.client.1.smithi070.stdout:9/962: creat d9/d1c/d34/d8b/d11d/dc4/dda/f135 x:0 0 0 2024-09-10T04:05:33.138 INFO:tasks.workunit.client.0.smithi046.stdout:7/941: write d5/d6/d32/f74 [2115401,115313] 0 2024-09-10T04:05:33.139 INFO:tasks.workunit.client.0.smithi046.stdout:2/880: chown d2/d8/d37/f6c 750848624 1 2024-09-10T04:05:33.140 INFO:tasks.workunit.client.1.smithi070.stdout:3/921: chown db/d1c/c42 3 1 2024-09-10T04:05:33.140 INFO:tasks.workunit.client.0.smithi046.stdout:0/720: write d1/dc/d25/d33/d3/dd/d55/d77/da6/fa8 [486828,120408] 0 2024-09-10T04:05:33.142 INFO:tasks.workunit.client.0.smithi046.stdout:1/704: creat d3/dc/fec x:0 0 0 2024-09-10T04:05:33.143 INFO:tasks.workunit.client.1.smithi070.stdout:1/817: sync 2024-09-10T04:05:33.143 INFO:tasks.workunit.client.1.smithi070.stdout:2/863: sync 2024-09-10T04:05:33.144 INFO:tasks.workunit.client.1.smithi070.stdout:1/818: fdatasync d3/d8/d13/f25 0 2024-09-10T04:05:33.144 INFO:tasks.workunit.client.0.smithi046.stdout:5/980: mknod d7/d11/d3f/d53/d99/c150 0 2024-09-10T04:05:33.146 INFO:tasks.workunit.client.1.smithi070.stdout:9/963: rename d9/d1c/d34/d8b/d11d/caa to d9/d7a/d74/c136 0 2024-09-10T04:05:33.148 INFO:tasks.workunit.client.0.smithi046.stdout:4/993: chown d3/d12/dec/df0/cd4 7353 1 2024-09-10T04:05:33.156 INFO:tasks.workunit.client.0.smithi046.stdout:7/942: mkdir d5/d9/d112/d14a 0 2024-09-10T04:05:33.158 INFO:tasks.workunit.client.0.smithi046.stdout:3/858: dwrite d12/d13/d14/d1e/dd2/fe6 [0,4194304] 0 2024-09-10T04:05:33.163 INFO:tasks.workunit.client.0.smithi046.stdout:5/981: truncate d7/d11/d14/d2f/d2b/fad 535097 0 2024-09-10T04:05:33.166 INFO:tasks.workunit.client.0.smithi046.stdout:6/776: dwrite da/d59/f85 [0,4194304] 0 2024-09-10T04:05:33.167 INFO:tasks.workunit.client.1.smithi070.stdout:1/819: creat d3/d8/d13/d67/f10a x:0 0 0 2024-09-10T04:05:33.167 INFO:tasks.workunit.client.1.smithi070.stdout:6/941: dwrite d0/f10 [0,4194304] 0 2024-09-10T04:05:33.168 INFO:tasks.workunit.client.1.smithi070.stdout:1/820: write d3/d8/f2d [5574522,92922] 0 2024-09-10T04:05:33.171 INFO:tasks.workunit.client.1.smithi070.stdout:1/821: chown d3/d8/d13/d3f/d5d/cdd 23908506 1 2024-09-10T04:05:33.175 INFO:tasks.workunit.client.1.smithi070.stdout:2/864: rename d15/d2b/cdf to d15/d18/d29/d2a/d2e/c137 0 2024-09-10T04:05:33.176 INFO:tasks.workunit.client.1.smithi070.stdout:6/942: dwrite d0/d3/ffe [0,4194304] 0 2024-09-10T04:05:33.178 INFO:tasks.workunit.client.0.smithi046.stdout:4/994: mknod d3/d12/d17/d29/d58/db2/d5f/d85/d114/d149/c150 0 2024-09-10T04:05:33.179 INFO:tasks.workunit.client.0.smithi046.stdout:4/995: read d3/d12/d14/fd8 [1781947,4972] 0 2024-09-10T04:05:33.181 INFO:tasks.workunit.client.0.smithi046.stdout:7/943: dread d5/d6/f1c [0,4194304] 0 2024-09-10T04:05:33.181 INFO:tasks.workunit.client.0.smithi046.stdout:9/873: symlink d4/d3c/d4d/l114 0 2024-09-10T04:05:33.182 INFO:tasks.workunit.client.1.smithi070.stdout:1/822: unlink d3/d10/d1d/fb9 0 2024-09-10T04:05:33.183 INFO:tasks.workunit.client.1.smithi070.stdout:9/964: dwrite d9/f22 [0,4194304] 0 2024-09-10T04:05:33.184 INFO:tasks.workunit.client.0.smithi046.stdout:7/944: write d5/d9/d12/d72/d77/de0/fe3 [585133,124771] 0 2024-09-10T04:05:33.187 INFO:tasks.workunit.client.0.smithi046.stdout:3/859: readlink d12/d13/d37/d51/db5/d50/lf6 0 2024-09-10T04:05:33.188 INFO:tasks.workunit.client.0.smithi046.stdout:3/860: chown d12/d13/d37/d51/dbd/fe0 1631829 1 2024-09-10T04:05:33.189 INFO:tasks.workunit.client.1.smithi070.stdout:2/865: mknod d15/d18/d56/d7c/c138 0 2024-09-10T04:05:33.189 INFO:tasks.workunit.client.0.smithi046.stdout:3/861: stat d12/d13/d37/d51/db5/d4b/d116 0 2024-09-10T04:05:33.190 INFO:tasks.workunit.client.1.smithi070.stdout:8/984: sync 2024-09-10T04:05:33.190 INFO:tasks.workunit.client.1.smithi070.stdout:4/898: sync 2024-09-10T04:05:33.191 INFO:tasks.workunit.client.1.smithi070.stdout:1/823: dwrite d3/d8/d13/f25 [0,4194304] 0 2024-09-10T04:05:33.192 INFO:tasks.workunit.client.1.smithi070.stdout:2/866: chown d15/d18/db2/c10e 0 1 2024-09-10T04:05:33.194 INFO:tasks.workunit.client.1.smithi070.stdout:1/824: readlink d3/d10/db2/dc7/l71 0 2024-09-10T04:05:33.196 INFO:tasks.workunit.client.0.smithi046.stdout:8/897: rmdir d1/d29/d17/d1f/d46/dc9 0 2024-09-10T04:05:33.198 INFO:tasks.workunit.client.0.smithi046.stdout:8/898: fsync d1/d29/d19/d36/d6e/f115 0 2024-09-10T04:05:33.199 INFO:tasks.workunit.client.1.smithi070.stdout:9/965: creat d9/f137 x:0 0 0 2024-09-10T04:05:33.201 INFO:tasks.workunit.client.1.smithi070.stdout:4/899: truncate de/d13/d5b/d7b/dd5/d85/d10e/f57 2673177 0 2024-09-10T04:05:33.202 INFO:tasks.workunit.client.1.smithi070.stdout:8/985: symlink d4/d16/dcc/d12e/da4/d114/l150 0 2024-09-10T04:05:33.204 INFO:tasks.workunit.client.1.smithi070.stdout:2/867: dwrite d15/d18/d56/d7c/ff5 [0,4194304] 0 2024-09-10T04:05:33.204 INFO:tasks.workunit.client.0.smithi046.stdout:9/874: creat d4/d3c/d4d/d80/f115 x:0 0 0 2024-09-10T04:05:33.205 INFO:tasks.workunit.client.1.smithi070.stdout:9/966: rmdir d9/d1c/d34 39 2024-09-10T04:05:33.206 INFO:tasks.workunit.client.0.smithi046.stdout:7/945: dread - d5/d6/d55/ff4 zero size 2024-09-10T04:05:33.207 INFO:tasks.workunit.client.1.smithi070.stdout:1/825: creat d3/dd3/f10b x:0 0 0 2024-09-10T04:05:33.209 INFO:tasks.workunit.client.0.smithi046.stdout:0/721: rename d1/dc/d25/d33/d1a/d41/ca5 to d1/ce3 0 2024-09-10T04:05:33.213 INFO:tasks.workunit.client.0.smithi046.stdout:3/862: stat d12/d13/d14/d1e/l56 0 2024-09-10T04:05:33.215 INFO:tasks.workunit.client.1.smithi070.stdout:8/986: unlink d4/d16/d73/d96/fb0 0 2024-09-10T04:05:33.216 INFO:tasks.workunit.client.0.smithi046.stdout:3/863: write d12/d13/d14/d1e/d2d/f4f [46531,75545] 0 2024-09-10T04:05:33.216 INFO:tasks.workunit.client.0.smithi046.stdout:1/705: link d3/lb3 d3/dc/d34/d58/led 0 2024-09-10T04:05:33.220 INFO:tasks.workunit.client.0.smithi046.stdout:0/722: dwrite d1/dc/d25/d33/d3/d29/d2d/f4f [0,4194304] 0 2024-09-10T04:05:33.221 INFO:tasks.workunit.client.1.smithi070.stdout:4/900: symlink de/d13/l130 0 2024-09-10T04:05:33.228 INFO:tasks.workunit.client.1.smithi070.stdout:2/868: mknod d15/d18/c139 0 2024-09-10T04:05:33.228 INFO:tasks.workunit.client.1.smithi070.stdout:3/922: sync 2024-09-10T04:05:33.228 INFO:tasks.workunit.client.1.smithi070.stdout:1/826: symlink d3/l10c 0 2024-09-10T04:05:33.228 INFO:tasks.workunit.client.0.smithi046.stdout:7/946: creat d5/d6/db/dd/d24/d76/f14b x:0 0 0 2024-09-10T04:05:33.229 INFO:tasks.workunit.client.1.smithi070.stdout:9/967: dwrite d9/d1c/d34/d8b/d11d/d3c/d92/da8/f109 [0,4194304] 0 2024-09-10T04:05:33.232 INFO:tasks.workunit.client.0.smithi046.stdout:2/881: rename d2/d8/d37/f6c to d2/d6/d23/d55/d9f/f136 0 2024-09-10T04:05:33.233 INFO:tasks.workunit.client.0.smithi046.stdout:2/882: stat d2/d6/d15/fa4 0 2024-09-10T04:05:33.237 INFO:tasks.workunit.client.1.smithi070.stdout:6/943: dwrite d0/d3/d18/d1b/d33/f65 [0,4194304] 0 2024-09-10T04:05:33.241 INFO:tasks.workunit.client.0.smithi046.stdout:3/864: truncate d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fa6 974302 0 2024-09-10T04:05:33.241 INFO:tasks.workunit.client.0.smithi046.stdout:8/899: mknod d1/c12b 0 2024-09-10T04:05:33.243 INFO:tasks.workunit.client.0.smithi046.stdout:5/982: write d7/d11/d14/d2f/d64/fd7 [1079182,128125] 0 2024-09-10T04:05:33.247 INFO:tasks.workunit.client.0.smithi046.stdout:7/947: creat d5/d6/db/dd/d10c/f14c x:0 0 0 2024-09-10T04:05:33.249 INFO:tasks.workunit.client.0.smithi046.stdout:6/777: dwrite da/db/d11/d2b/d2f/f74 [0,4194304] 0 2024-09-10T04:05:33.255 INFO:tasks.workunit.client.0.smithi046.stdout:2/883: dread d2/d8/f29 [0,4194304] 0 2024-09-10T04:05:33.256 INFO:tasks.workunit.client.0.smithi046.stdout:8/900: creat d1/d29/d19/d36/d8d/f12c x:0 0 0 2024-09-10T04:05:33.258 INFO:tasks.workunit.client.0.smithi046.stdout:4/996: dwrite d3/d12/d17/d29/f121 [0,4194304] 0 2024-09-10T04:05:33.262 INFO:tasks.workunit.client.0.smithi046.stdout:5/983: rmdir d7/d11/d3f/d53 39 2024-09-10T04:05:33.264 INFO:tasks.workunit.client.0.smithi046.stdout:9/875: write d4/d18/d89/fa9 [93335,18158] 0 2024-09-10T04:05:33.267 INFO:tasks.workunit.client.0.smithi046.stdout:7/948: readlink d5/d9/l4c 0 2024-09-10T04:05:33.267 INFO:tasks.workunit.client.0.smithi046.stdout:7/949: stat d5/d9/d43 0 2024-09-10T04:05:33.269 INFO:tasks.workunit.client.0.smithi046.stdout:1/706: creat d3/fee x:0 0 0 2024-09-10T04:05:33.270 INFO:tasks.workunit.client.0.smithi046.stdout:6/778: truncate da/db/d11/d13/f57 4478805 0 2024-09-10T04:05:33.292 INFO:tasks.workunit.client.1.smithi070.stdout:3/923: creat db/d1c/d89/f13e x:0 0 0 2024-09-10T04:05:33.293 INFO:tasks.workunit.client.1.smithi070.stdout:2/869: truncate d15/d18/d29/d2a/d2e/d32/f33 6071081 0 2024-09-10T04:05:33.294 INFO:tasks.workunit.client.1.smithi070.stdout:4/901: creat de/d13/da6/d115/f131 x:0 0 0 2024-09-10T04:05:33.295 INFO:tasks.workunit.client.1.smithi070.stdout:1/827: rmdir d3/d20 39 2024-09-10T04:05:33.296 INFO:tasks.workunit.client.1.smithi070.stdout:3/924: symlink db/d1c/d25/dd8/l13f 0 2024-09-10T04:05:33.298 INFO:tasks.workunit.client.1.smithi070.stdout:6/944: mkdir d0/d3/d18/d1b/d17/d54/dc4/d125/d132 0 2024-09-10T04:05:33.299 INFO:tasks.workunit.client.1.smithi070.stdout:8/987: getdents d4/d16/dcc/d12e/d11b/d1a/df7 0 2024-09-10T04:05:33.302 INFO:tasks.workunit.client.1.smithi070.stdout:4/902: mknod de/d13/d43/d88/db5/c132 0 2024-09-10T04:05:33.307 INFO:tasks.workunit.client.1.smithi070.stdout:1/828: mknod d3/d8/d92/dec/c10d 0 2024-09-10T04:05:33.310 INFO:tasks.workunit.client.1.smithi070.stdout:8/988: mkdir d4/d16/dcc/d12e/d11b/d6f/d151 0 2024-09-10T04:05:33.310 INFO:tasks.workunit.client.1.smithi070.stdout:2/870: mknod d15/d18/d29/d2a/d2e/da6/dc0/c13a 0 2024-09-10T04:05:33.311 INFO:tasks.workunit.client.1.smithi070.stdout:6/945: link d0/d3/f51 d0/d3/d18/df5/f133 0 2024-09-10T04:05:33.311 INFO:tasks.workunit.client.1.smithi070.stdout:2/871: chown d15/d18/d29/d2a/d2e/da6/dc0/dcb/cda 46417 1 2024-09-10T04:05:33.312 INFO:tasks.workunit.client.1.smithi070.stdout:8/989: creat d4/d16/d73/d139/d7a/db8/f152 x:0 0 0 2024-09-10T04:05:33.315 INFO:tasks.workunit.client.1.smithi070.stdout:9/968: write d9/d1c/d34/d8b/dcc/dfa/ff9 [972354,5447] 0 2024-09-10T04:05:33.316 INFO:tasks.workunit.client.1.smithi070.stdout:1/829: getdents d3/d8 0 2024-09-10T04:05:33.317 INFO:tasks.workunit.client.1.smithi070.stdout:1/830: write d3/d8/d13/d67/fcd [1666290,122270] 0 2024-09-10T04:05:33.318 INFO:tasks.workunit.client.1.smithi070.stdout:6/946: creat d0/d3/d18/d62/f134 x:0 0 0 2024-09-10T04:05:33.319 INFO:tasks.workunit.client.1.smithi070.stdout:2/872: rename d15/d18/d29/d2a/fc4 to d15/d18/d29/d2a/d2e/d32/f13b 0 2024-09-10T04:05:33.321 INFO:tasks.workunit.client.1.smithi070.stdout:6/947: creat d0/d3/d1f/d55/f135 x:0 0 0 2024-09-10T04:05:33.321 INFO:tasks.workunit.client.1.smithi070.stdout:1/831: rmdir d3/d8/d12/d2f/d32/da8 39 2024-09-10T04:05:33.322 INFO:tasks.workunit.client.1.smithi070.stdout:9/969: read d9/d1c/d34/f76 [5284999,69253] 0 2024-09-10T04:05:33.322 INFO:tasks.workunit.client.1.smithi070.stdout:2/873: dread d15/d18/d56/d7c/ff5 [0,4194304] 9 2024-09-10T04:05:33.322 INFO:tasks.workunit.client.1.smithi070.stdout:8/990: rename d4/d16/d73/d96/c8f to d4/d39/dfc/d43/d52/c153 0 2024-09-10T04:05:33.324 INFO:tasks.workunit.client.1.smithi070.stdout:3/925: dwrite db/d1c/d25/d28/d43/f77 [0,4194304] 0 2024-09-10T04:05:33.325 INFO:tasks.workunit.client.1.smithi070.stdout:2/874: creat d15/d18/d29/ddb/f13c x:0 0 0 2024-09-10T04:05:33.326 INFO:tasks.workunit.client.1.smithi070.stdout:8/991: readlink d4/d39/dfc/d43/d67/d97/l76 0 2024-09-10T04:05:33.329 INFO:tasks.workunit.client.1.smithi070.stdout:2/875: fsync d15/d18/d56/d7c/d55/d6d/fad 0 2024-09-10T04:05:33.331 INFO:tasks.workunit.client.1.smithi070.stdout:6/948: dwrite d0/d3/d8c/f120 [0,4194304] 0 2024-09-10T04:05:33.333 INFO:tasks.workunit.client.1.smithi070.stdout:1/832: link d3/d8/d12/d2f/fd5 d3/d8/d27/d7a/f10e 0 2024-09-10T04:05:33.333 INFO:tasks.workunit.client.1.smithi070.stdout:1/833: chown d3/d8 18098 1 2024-09-10T04:05:33.336 INFO:tasks.workunit.client.1.smithi070.stdout:8/992: creat d4/d16/dcc/d12e/d11b/dab/f154 x:0 0 0 2024-09-10T04:05:33.340 INFO:tasks.workunit.client.1.smithi070.stdout:6/949: rename d0/d3/d18/d1b/l31 to d0/d3/d1f/d55/l136 0 2024-09-10T04:05:33.341 INFO:tasks.workunit.client.1.smithi070.stdout:1/834: creat d3/d8/f10f x:0 0 0 2024-09-10T04:05:33.344 INFO:tasks.workunit.client.1.smithi070.stdout:6/950: creat d0/d3/d18/d1b/d17/d3f/d40/f137 x:0 0 0 2024-09-10T04:05:33.348 INFO:tasks.workunit.client.1.smithi070.stdout:6/951: creat d0/d3/da4/dc0/dea/f138 x:0 0 0 2024-09-10T04:05:33.348 INFO:tasks.workunit.client.1.smithi070.stdout:9/970: dread d9/d1c/d34/d8b/d11d/d1b/f123 [0,4194304] 0 2024-09-10T04:05:33.349 INFO:tasks.workunit.client.1.smithi070.stdout:9/971: dread - d9/f137 zero size 2024-09-10T04:05:33.350 INFO:tasks.workunit.client.1.smithi070.stdout:9/972: chown d9/d1c/f55 423069 1 2024-09-10T04:05:33.351 INFO:tasks.workunit.client.1.smithi070.stdout:9/973: chown d9/d1c/dfd/cde 69446721 1 2024-09-10T04:05:33.351 INFO:tasks.workunit.client.1.smithi070.stdout:9/974: chown d9/d59/d2a/f101 50 1 2024-09-10T04:05:33.352 INFO:tasks.workunit.client.1.smithi070.stdout:6/952: truncate d0/d3/d18/d62/db8/fd0 3675155 0 2024-09-10T04:05:33.353 INFO:tasks.workunit.client.1.smithi070.stdout:2/876: fsync d15/d18/d29/d2a/d2e/d32/f33 0 2024-09-10T04:05:33.355 INFO:tasks.workunit.client.1.smithi070.stdout:3/926: dread db/d1c/d25/d28/dd3/de5/d10f/fa3 [0,4194304] 0 2024-09-10T04:05:33.358 INFO:tasks.workunit.client.1.smithi070.stdout:1/835: truncate d3/d8/f5c 702783 0 2024-09-10T04:05:33.359 INFO:tasks.workunit.client.1.smithi070.stdout:8/993: dwrite d4/d39/dfc/d43/d67/f120 [0,4194304] 0 2024-09-10T04:05:33.361 INFO:tasks.workunit.client.1.smithi070.stdout:1/836: unlink d3/d10/c4d 0 2024-09-10T04:05:33.362 INFO:tasks.workunit.client.1.smithi070.stdout:1/837: readlink d3/d10/db2/dc7/le1 0 2024-09-10T04:05:33.363 INFO:tasks.workunit.client.1.smithi070.stdout:2/877: dwrite d15/d18/d29/d64/f95 [0,4194304] 0 2024-09-10T04:05:33.365 INFO:tasks.workunit.client.1.smithi070.stdout:3/927: rmdir db/d83/d92/d11b/d126 0 2024-09-10T04:05:33.366 INFO:tasks.workunit.client.1.smithi070.stdout:1/838: readlink d3/d8/d13/d3f/d6d/d81/lf6 0 2024-09-10T04:05:33.366 INFO:tasks.workunit.client.1.smithi070.stdout:9/975: truncate d9/d1c/d34/d8b/d11d/d1b/fb6 3357819 0 2024-09-10T04:05:33.366 INFO:tasks.workunit.client.1.smithi070.stdout:1/839: dread - d3/dd3/f10b zero size 2024-09-10T04:05:33.368 INFO:tasks.workunit.client.1.smithi070.stdout:1/840: truncate d3/d10/db2/ff2 130382 0 2024-09-10T04:05:33.373 INFO:tasks.workunit.client.0.smithi046.stdout:5/984: symlink d7/d11/d14/d2f/deb/da5/dcc/dc0/d134/l151 0 2024-09-10T04:05:33.375 INFO:tasks.workunit.client.0.smithi046.stdout:7/950: readlink d5/d7f/d129/l132 0 2024-09-10T04:05:33.375 INFO:tasks.workunit.client.0.smithi046.stdout:9/876: chown d4/d7/ca 52981 1 2024-09-10T04:05:33.377 INFO:tasks.workunit.client.1.smithi070.stdout:3/928: creat db/d1c/d25/f140 x:0 0 0 2024-09-10T04:05:33.378 INFO:tasks.workunit.client.1.smithi070.stdout:3/929: readlink db/d1c/d25/l2f 0 2024-09-10T04:05:33.379 INFO:tasks.workunit.client.0.smithi046.stdout:3/865: rmdir d12/d13/d14/d1e/d2d/d6f/d8f/dee 0 2024-09-10T04:05:33.380 INFO:tasks.workunit.client.1.smithi070.stdout:1/841: rmdir d3/d10/db2/dc7/d54/d106 0 2024-09-10T04:05:33.381 INFO:tasks.workunit.client.0.smithi046.stdout:9/877: dread - d4/d3c/de0/d95/f111 zero size 2024-09-10T04:05:33.382 INFO:tasks.workunit.client.0.smithi046.stdout:1/707: mknod d3/d27/d5d/d60/dd7/deb/cef 0 2024-09-10T04:05:33.382 INFO:tasks.workunit.client.1.smithi070.stdout:9/976: getdents d9/d1c/d34/d8b/d11d/dc4/dda/de0/d90/df0/df3/df4 0 2024-09-10T04:05:33.384 INFO:tasks.workunit.client.0.smithi046.stdout:8/901: dwrite d1/d29/d12/f45 [0,4194304] 0 2024-09-10T04:05:33.385 INFO:tasks.workunit.client.0.smithi046.stdout:9/878: chown d4/d66/lb9 7654640 1 2024-09-10T04:05:33.386 INFO:tasks.workunit.client.0.smithi046.stdout:2/884: dwrite d2/d6/d15/ffa [0,4194304] 0 2024-09-10T04:05:33.390 INFO:tasks.workunit.client.1.smithi070.stdout:2/878: truncate d15/d59/da1/fde 1064393 0 2024-09-10T04:05:33.390 INFO:tasks.workunit.client.1.smithi070.stdout:8/994: dwrite d4/d39/dfc/d43/f115 [0,4194304] 0 2024-09-10T04:05:33.392 INFO:tasks.workunit.client.0.smithi046.stdout:7/951: unlink d5/d9/d12/d72/d77/de0/d1f/c28 0 2024-09-10T04:05:33.395 INFO:tasks.workunit.client.1.smithi070.stdout:1/842: dwrite d3/d8/d27/d7a/d94/f101 [0,4194304] 0 2024-09-10T04:05:33.395 INFO:tasks.workunit.client.0.smithi046.stdout:0/723: getdents d1/dc/d25/d33/d3/d29/d2d/d5e 0 2024-09-10T04:05:33.396 INFO:tasks.workunit.client.1.smithi070.stdout:1/843: stat d3/dd3/lf3 0 2024-09-10T04:05:33.398 INFO:tasks.workunit.client.0.smithi046.stdout:5/985: rename d7/d11/d14/d8e/d96 to d7/d11/d14/d2f/deb/da5/d123/d152 0 2024-09-10T04:05:33.400 INFO:tasks.workunit.client.0.smithi046.stdout:6/779: link da/db/d11/d2b/d2f/d3b/l3d da/db/d11/d13/d1e/d23/d54/dad/dfe/lff 0 2024-09-10T04:05:33.402 INFO:tasks.workunit.client.0.smithi046.stdout:6/780: stat da/db/d11/d13/d1e/d69/cbd 0 2024-09-10T04:05:33.403 INFO:tasks.workunit.client.0.smithi046.stdout:3/866: mkdir d12/d5d/d71/d49/d128 0 2024-09-10T04:05:33.407 INFO:tasks.workunit.client.0.smithi046.stdout:9/879: unlink d4/d3c/d4d/d5c/f93 0 2024-09-10T04:05:33.407 INFO:tasks.workunit.client.0.smithi046.stdout:9/880: chown d4/l45 77 1 2024-09-10T04:05:33.408 INFO:tasks.workunit.client.1.smithi070.stdout:1/844: rename d3/d8/d13/fc3 to d3/d8/d13/d67/d83/f110 0 2024-09-10T04:05:33.409 INFO:tasks.workunit.client.0.smithi046.stdout:2/885: creat d2/d8/d37/dc2/f137 x:0 0 0 2024-09-10T04:05:33.413 INFO:tasks.workunit.client.0.smithi046.stdout:7/952: fdatasync d5/d6/db/dd/f2f 0 2024-09-10T04:05:33.418 INFO:tasks.workunit.client.0.smithi046.stdout:8/902: rename d1/d29/d17/d1f/d46/d6a/d82/dc2/lce to d1/d29/d17/d1f/d46/d67/d116/l12d 0 2024-09-10T04:05:33.419 INFO:tasks.workunit.client.1.smithi070.stdout:8/995: write d4/fb7 [356791,124073] 0 2024-09-10T04:05:33.420 INFO:tasks.workunit.client.0.smithi046.stdout:5/986: mkdir d7/d11/d14/d2f/d2b/d153 0 2024-09-10T04:05:33.421 INFO:tasks.workunit.client.0.smithi046.stdout:5/987: readlink d7/d11/d14/d2f/d64/le4 0 2024-09-10T04:05:33.421 INFO:tasks.workunit.client.1.smithi070.stdout:2/879: truncate d15/d18/d29/d2a/d2e/d32/f13b 1003821 0 2024-09-10T04:05:33.422 INFO:tasks.workunit.client.1.smithi070.stdout:2/880: write d15/d18/d29/d64/f118 [222466,98939] 0 2024-09-10T04:05:33.425 INFO:tasks.workunit.client.1.smithi070.stdout:9/977: link d9/d1c/d34/d8b/dcc/le5 d9/d1c/d34/d8b/d11d/dc4/dda/de0/d90/df0/df3/df4/l138 0 2024-09-10T04:05:33.426 INFO:tasks.workunit.client.1.smithi070.stdout:9/978: dread - d9/d59/f12b zero size 2024-09-10T04:05:33.428 INFO:tasks.workunit.client.1.smithi070.stdout:3/930: getdents db/d1c/d25/d28/dd3/de5/d10f/db5 0 2024-09-10T04:05:33.429 INFO:tasks.workunit.client.0.smithi046.stdout:2/886: rmdir d2/d6/d23/d5d/d77/d7b/dc4 39 2024-09-10T04:05:33.429 INFO:tasks.workunit.client.0.smithi046.stdout:0/724: write d1/dc/d49/fa1 [772795,47053] 0 2024-09-10T04:05:33.430 INFO:tasks.workunit.client.1.smithi070.stdout:8/996: symlink d4/d16/d73/d96/l155 0 2024-09-10T04:05:33.431 INFO:tasks.workunit.client.0.smithi046.stdout:0/725: write d1/dc/d25/d33/d1a/f97 [364083,46701] 0 2024-09-10T04:05:33.432 INFO:tasks.workunit.client.0.smithi046.stdout:0/726: chown d1/dc/d25/d33/d3/dd 109971 1 2024-09-10T04:05:33.433 INFO:tasks.workunit.client.0.smithi046.stdout:7/953: mkdir d5/d6/d62/d108/d14d 0 2024-09-10T04:05:33.434 INFO:tasks.workunit.client.1.smithi070.stdout:2/881: symlink d15/d59/l13d 0 2024-09-10T04:05:33.434 INFO:tasks.workunit.client.1.smithi070.stdout:1/845: symlink d3/d8/d13/d3f/l111 0 2024-09-10T04:05:33.435 INFO:tasks.workunit.client.0.smithi046.stdout:0/727: fdatasync d1/dc/d25/d33/d3/dd/d55/d77/da6/fa8 0 2024-09-10T04:05:33.435 INFO:tasks.workunit.client.1.smithi070.stdout:1/846: read d3/d8/f49 [1332225,114667] 0 2024-09-10T04:05:33.437 INFO:tasks.workunit.client.0.smithi046.stdout:8/903: readlink d1/d29/d19/d36/d4c/d5e/dc7/lf8 0 2024-09-10T04:05:33.441 INFO:tasks.workunit.client.0.smithi046.stdout:5/988: stat d7/d11/d3f/d53/d75 0 2024-09-10T04:05:33.445 INFO:tasks.workunit.client.1.smithi070.stdout:3/931: dread db/d1c/fe2 [0,4194304] 0 2024-09-10T04:05:33.445 INFO:tasks.workunit.client.0.smithi046.stdout:1/708: creat d3/dc/d34/ff0 x:0 0 0 2024-09-10T04:05:33.446 INFO:tasks.workunit.client.1.smithi070.stdout:3/932: fsync db/d1c/d89/f135 0 2024-09-10T04:05:33.451 INFO:tasks.workunit.client.1.smithi070.stdout:8/997: write d4/d16/d73/d139/d7a/db8/fe2 [1396861,106634] 0 2024-09-10T04:05:33.453 INFO:tasks.workunit.client.1.smithi070.stdout:1/847: read - d3/d8/d13/d3f/d6d/d81/fd8 zero size 2024-09-10T04:05:33.453 INFO:tasks.workunit.client.0.smithi046.stdout:3/867: truncate d12/d13/d14/f36 2912337 0 2024-09-10T04:05:33.454 INFO:tasks.workunit.client.0.smithi046.stdout:3/868: readlink d12/d13/d37/d51/db5/d50/l10d 0 2024-09-10T04:05:33.455 INFO:tasks.workunit.client.0.smithi046.stdout:7/954: creat d5/d9/d43/f14e x:0 0 0 2024-09-10T04:05:33.461 INFO:tasks.workunit.client.0.smithi046.stdout:3/869: dwrite d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/f126 [0,4194304] 0 2024-09-10T04:05:33.462 INFO:tasks.workunit.client.0.smithi046.stdout:3/870: chown d12/d13/d14/d1e/d2d/d39/d67 0 1 2024-09-10T04:05:33.489 INFO:tasks.workunit.client.1.smithi070.stdout:9/979: truncate d9/d1c/d34/d8b/dcc/dfa/ff9 110958 0 2024-09-10T04:05:33.492 INFO:tasks.workunit.client.1.smithi070.stdout:2/882: dwrite f11 [0,4194304] 0 2024-09-10T04:05:33.514 INFO:tasks.workunit.client.1.smithi070.stdout:8/998: symlink d4/d16/d73/d139/l156 0 2024-09-10T04:05:33.515 INFO:tasks.workunit.client.0.smithi046.stdout:6/781: link da/db/d11/cd2 da/db/d10/d51/c100 0 2024-09-10T04:05:33.515 INFO:tasks.workunit.client.0.smithi046.stdout:6/782: chown da/db/d63/l7f 224100646 1 2024-09-10T04:05:33.518 INFO:tasks.workunit.client.0.smithi046.stdout:2/887: rename d2/c4 to d2/d6/d23/d5d/d77/d7b/d85/c138 0 2024-09-10T04:05:33.522 INFO:tasks.workunit.client.0.smithi046.stdout:7/955: unlink d5/d9/d12/d72/d77/de0/d9f/d126/f141 0 2024-09-10T04:05:33.523 INFO:tasks.workunit.client.1.smithi070.stdout:9/980: rmdir d9/d1c/d34/d8b 39 2024-09-10T04:05:33.525 INFO:tasks.workunit.client.0.smithi046.stdout:0/728: creat d1/d13/d37/d75/d94/d9a/dcf/fe4 x:0 0 0 2024-09-10T04:05:33.529 INFO:tasks.workunit.client.1.smithi070.stdout:2/883: mknod d15/d18/d29/d2a/d2e/da6/dc0/dca/c13e 0 2024-09-10T04:05:33.530 INFO:tasks.workunit.client.0.smithi046.stdout:8/904: dwrite d1/d29/d17/d1f/f31 [0,4194304] 0 2024-09-10T04:05:33.531 INFO:tasks.workunit.client.0.smithi046.stdout:8/905: readlink d1/d29/d17/d1f/d46/d6a/d82/dc2/led 0 2024-09-10T04:05:33.533 INFO:tasks.workunit.client.0.smithi046.stdout:5/989: creat d7/d11/d3f/d14b/f154 x:0 0 0 2024-09-10T04:05:33.534 INFO:tasks.workunit.client.1.smithi070.stdout:1/848: fsync d3/d8/d27/fac 0 2024-09-10T04:05:33.534 INFO:tasks.workunit.client.1.smithi070.stdout:3/933: write db/d1c/d25/d28/dd3/de5/d10f/d94/d99/fa5 [1983518,67897] 0 2024-09-10T04:05:33.537 INFO:tasks.workunit.client.0.smithi046.stdout:6/783: unlink da/db/d11/d13/d1e/d23/d54/dad/dfe/lff 0 2024-09-10T04:05:33.538 INFO:tasks.workunit.client.0.smithi046.stdout:6/784: write da/db/d11/dd7/f34 [4474109,93623] 0 2024-09-10T04:05:33.540 INFO:tasks.workunit.client.0.smithi046.stdout:2/888: truncate d2/d6/d15/fa4 482325 0 2024-09-10T04:05:33.541 INFO:tasks.workunit.client.1.smithi070.stdout:2/884: unlink d15/d59/cfc 0 2024-09-10T04:05:33.543 INFO:tasks.workunit.client.1.smithi070.stdout:8/999: truncate d4/d39/dfc/f31 613107 0 2024-09-10T04:05:33.548 INFO:tasks.workunit.client.1.smithi070.stdout:1/849: creat d3/d90/f112 x:0 0 0 2024-09-10T04:05:33.550 INFO:tasks.workunit.client.1.smithi070.stdout:3/934: dread db/d1c/d25/d28/fba [0,4194304] 0 2024-09-10T04:05:33.554 INFO:tasks.workunit.client.1.smithi070.stdout:9/981: dwrite d9/d59/d2a/f101 [0,4194304] 0 2024-09-10T04:05:33.557 INFO:tasks.workunit.client.1.smithi070.stdout:2/885: dread - d15/d18/d29/d2a/d2e/d32/d36/f8a zero size 2024-09-10T04:05:33.559 INFO:tasks.workunit.client.1.smithi070.stdout:1/850: creat d3/d90/f113 x:0 0 0 2024-09-10T04:05:33.561 INFO:tasks.workunit.client.0.smithi046.stdout:0/729: dwrite d1/dc/d25/d33/d3/d29/f38 [0,4194304] 0 2024-09-10T04:05:33.562 INFO:tasks.workunit.client.1.smithi070.stdout:6/953: sync 2024-09-10T04:05:33.562 INFO:tasks.workunit.client.1.smithi070.stdout:4/903: sync 2024-09-10T04:05:33.563 INFO:tasks.workunit.client.1.smithi070.stdout:4/904: chown de/d13/d18/d26/c3d 46130458 1 2024-09-10T04:05:33.568 INFO:tasks.workunit.client.0.smithi046.stdout:2/889: mknod d2/d8/d65/d90/c139 0 2024-09-10T04:05:33.571 INFO:tasks.workunit.client.0.smithi046.stdout:5/990: write d7/d11/d3f/d53/f9b [3262920,56355] 0 2024-09-10T04:05:33.573 INFO:tasks.workunit.client.0.smithi046.stdout:3/871: link d12/d13/d14/d1e/d2d/c10c d12/d13/d14/d1e/d2d/dfd/c129 0 2024-09-10T04:05:33.575 INFO:tasks.workunit.client.1.smithi070.stdout:2/886: creat d15/d2b/d6e/db8/f13f x:0 0 0 2024-09-10T04:05:33.575 INFO:tasks.workunit.client.1.smithi070.stdout:2/887: chown d15/d18/d56/ca5 7 1 2024-09-10T04:05:33.576 INFO:tasks.workunit.client.0.smithi046.stdout:5/991: readlink d7/d11/d14/d2f/deb/da5/d123/d152/lfa 0 2024-09-10T04:05:33.576 INFO:tasks.workunit.client.0.smithi046.stdout:2/890: dwrite d2/f25 [0,4194304] 0 2024-09-10T04:05:33.578 INFO:tasks.workunit.client.1.smithi070.stdout:1/851: symlink d3/d8/d13/d3f/d5d/l114 0 2024-09-10T04:05:33.579 INFO:tasks.workunit.client.0.smithi046.stdout:2/891: chown d2/d6/d23/d5d/d77/d7b/d85 7280255 1 2024-09-10T04:05:33.580 INFO:tasks.workunit.client.1.smithi070.stdout:6/954: creat d0/d3/d18/d1b/d17/d54/dc4/d125/f139 x:0 0 0 2024-09-10T04:05:33.583 INFO:tasks.workunit.client.0.smithi046.stdout:6/785: dwrite da/db/d11/d13/f7c [0,4194304] 0 2024-09-10T04:05:33.584 INFO:tasks.workunit.client.1.smithi070.stdout:3/935: dwrite db/d1c/d25/d28/f106 [0,4194304] 0 2024-09-10T04:05:33.585 INFO:tasks.workunit.client.0.smithi046.stdout:8/906: write d1/f86 [3632358,94774] 0 2024-09-10T04:05:33.593 INFO:tasks.workunit.client.0.smithi046.stdout:3/872: creat d12/d13/d37/d51/db5/d4b/d91/d112/f12a x:0 0 0 2024-09-10T04:05:33.597 INFO:tasks.workunit.client.1.smithi070.stdout:4/905: link de/fe4 de/d13/d5b/d7b/dd5/d85/f133 0 2024-09-10T04:05:33.598 INFO:tasks.workunit.client.1.smithi070.stdout:9/982: dread d9/d1c/d34/d8b/d11d/d3c/d62/f97 [0,4194304] 0 2024-09-10T04:05:33.599 INFO:tasks.workunit.client.1.smithi070.stdout:2/888: dread d15/d18/d56/d7c/d55/f11a [0,4194304] 0 2024-09-10T04:05:33.601 INFO:tasks.workunit.client.0.smithi046.stdout:6/786: mknod da/db/d11/d13/d1e/d23/c101 0 2024-09-10T04:05:33.602 INFO:tasks.workunit.client.0.smithi046.stdout:0/730: creat d1/dc/d49/db2/dcb/fe5 x:0 0 0 2024-09-10T04:05:33.604 INFO:tasks.workunit.client.0.smithi046.stdout:0/731: fdatasync d1/dc/d25/d33/d3/dcd/fe1 0 2024-09-10T04:05:33.605 INFO:tasks.workunit.client.0.smithi046.stdout:8/907: mkdir d1/d29/d17/d1f/d2a/df6/d102/d12e 0 2024-09-10T04:05:33.606 INFO:tasks.workunit.client.1.smithi070.stdout:3/936: symlink db/d1c/d25/d28/dd3/de5/d10f/dd4/l141 0 2024-09-10T04:05:33.607 INFO:tasks.workunit.client.0.smithi046.stdout:7/956: getdents d5/d9/d12/d72 0 2024-09-10T04:05:33.607 INFO:tasks.workunit.client.0.smithi046.stdout:7/957: stat d5/d9/d43/db4 0 2024-09-10T04:05:33.608 INFO:tasks.workunit.client.0.smithi046.stdout:3/873: rmdir d12/d13/d37/d51/db5/d50 39 2024-09-10T04:05:33.609 INFO:tasks.workunit.client.1.smithi070.stdout:6/955: dwrite d0/d3/d18/fa2 [0,4194304] 0 2024-09-10T04:05:33.612 INFO:tasks.workunit.client.0.smithi046.stdout:5/992: write d7/d11/d14/d2f/da3/fbf [4427291,117810] 0 2024-09-10T04:05:33.612 INFO:tasks.workunit.client.0.smithi046.stdout:2/892: write d2/d8/d37/dc2/f116 [612775,55626] 0 2024-09-10T04:05:33.614 INFO:tasks.workunit.client.1.smithi070.stdout:3/937: mknod db/d1c/d25/d28/dd3/de5/c142 0 2024-09-10T04:05:33.615 INFO:tasks.workunit.client.0.smithi046.stdout:5/993: chown d7/d11/d3f/d53/d75/f9f 173 1 2024-09-10T04:05:33.616 INFO:tasks.workunit.client.1.smithi070.stdout:1/852: getdents d3/d10/db2 0 2024-09-10T04:05:33.617 INFO:tasks.workunit.client.0.smithi046.stdout:2/893: write d2/d6/d23/d5d/d77/d7b/d85/d8d/f108 [556151,121662] 0 2024-09-10T04:05:33.618 INFO:tasks.workunit.client.1.smithi070.stdout:9/983: symlink d9/d1c/d34/d8b/dcc/de7/d12e/l139 0 2024-09-10T04:05:33.619 INFO:tasks.workunit.client.1.smithi070.stdout:9/984: readlink d9/d1c/d34/d8b/d11d/d3c/l100 0 2024-09-10T04:05:33.623 INFO:tasks.workunit.client.0.smithi046.stdout:0/732: creat d1/dc/d25/d33/d1a/dc9/fe6 x:0 0 0 2024-09-10T04:05:33.623 INFO:tasks.workunit.client.1.smithi070.stdout:4/906: creat de/d13/d18/d26/f134 x:0 0 0 2024-09-10T04:05:33.624 INFO:tasks.workunit.client.0.smithi046.stdout:8/908: dread d1/fd [0,4194304] 0 2024-09-10T04:05:33.625 INFO:tasks.workunit.client.1.smithi070.stdout:3/938: dread - db/d1c/d25/d28/dd3/de5/d10f/fed zero size 2024-09-10T04:05:33.625 INFO:tasks.workunit.client.1.smithi070.stdout:4/907: chown de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/c120 58718 1 2024-09-10T04:05:33.626 INFO:tasks.workunit.client.0.smithi046.stdout:8/909: readlink d1/d29/d17/d1f/d46/d6a/d82/dc2/led 0 2024-09-10T04:05:33.627 INFO:tasks.workunit.client.0.smithi046.stdout:8/910: stat d1/d29/d17/d1f/d46/fca 0 2024-09-10T04:05:33.627 INFO:tasks.workunit.client.1.smithi070.stdout:6/956: unlink d0/d3/da4/dc0/dea/c77 0 2024-09-10T04:05:33.629 INFO:tasks.workunit.client.1.smithi070.stdout:9/985: chown d9/d1c/d34/d8b/d11d/dc4/de1/lb4 7757 1 2024-09-10T04:05:33.632 INFO:tasks.workunit.client.0.smithi046.stdout:7/958: write d5/d9/d12/d72/d77/de0/f14 [827049,4956] 0 2024-09-10T04:05:33.632 INFO:tasks.workunit.client.1.smithi070.stdout:1/853: mkdir d3/d10/d115 0 2024-09-10T04:05:33.634 INFO:tasks.workunit.client.1.smithi070.stdout:4/908: creat de/d13/d18/d26/d8c/d124/f135 x:0 0 0 2024-09-10T04:05:33.634 INFO:tasks.workunit.client.1.smithi070.stdout:2/889: truncate d15/d18/f24 1723745 0 2024-09-10T04:05:33.635 INFO:tasks.workunit.client.0.smithi046.stdout:2/894: rename d2/d6/d15/c71 to d2/d8/d37/d63/c13a 0 2024-09-10T04:05:33.636 INFO:tasks.workunit.client.1.smithi070.stdout:4/909: write de/d13/d5b/d7b/dd5/d85/d10e/f6d [366814,78560] 0 2024-09-10T04:05:33.636 INFO:tasks.workunit.client.0.smithi046.stdout:2/895: chown d2/d6/d23/d42/de6/c5f 176085885 1 2024-09-10T04:05:33.636 INFO:tasks.workunit.client.1.smithi070.stdout:2/890: write d15/d18/d29/d2a/d2e/d32/fe5 [2051714,64287] 0 2024-09-10T04:05:33.637 INFO:tasks.workunit.client.0.smithi046.stdout:3/874: rmdir d12/d13/d14/d1e/d123 0 2024-09-10T04:05:33.637 INFO:tasks.workunit.client.0.smithi046.stdout:2/896: chown d2/d6/d23/dd5/df8/f128 49 1 2024-09-10T04:05:33.638 INFO:tasks.workunit.client.1.smithi070.stdout:4/910: write de/d13/f10b [818288,4495] 0 2024-09-10T04:05:33.638 INFO:tasks.workunit.client.1.smithi070.stdout:6/957: dread d0/d3/da4/dc0/dea/f3b [0,4194304] 0 2024-09-10T04:05:33.639 INFO:tasks.workunit.client.0.smithi046.stdout:8/911: symlink d1/d29/d19/d36/d6e/d75/l12f 0 2024-09-10T04:05:33.640 INFO:tasks.workunit.client.0.smithi046.stdout:6/787: link da/db/d11/d2b/f2c da/db/d11/d13/d1e/d9d/f102 0 2024-09-10T04:05:33.641 INFO:tasks.workunit.client.0.smithi046.stdout:6/788: stat da/db/d11/d2b/d2f/f72 0 2024-09-10T04:05:33.642 INFO:tasks.workunit.client.1.smithi070.stdout:6/958: write d0/d3/d18/d1b/f11 [6518528,8129] 0 2024-09-10T04:05:33.644 INFO:tasks.workunit.client.1.smithi070.stdout:2/891: dwrite d15/d18/db2/f11f [0,4194304] 0 2024-09-10T04:05:33.645 INFO:tasks.workunit.client.0.smithi046.stdout:5/994: creat d7/d11/d14/d2f/deb/da5/dcc/dc0/f155 x:0 0 0 2024-09-10T04:05:33.649 INFO:tasks.workunit.client.0.smithi046.stdout:2/897: stat d2/d6/d15/l48 0 2024-09-10T04:05:33.650 INFO:tasks.workunit.client.0.smithi046.stdout:2/898: chown d2/d6/d23/ff0 1 1 2024-09-10T04:05:33.655 INFO:tasks.workunit.client.0.smithi046.stdout:8/912: fsync d1/d29/d17/f8e 0 2024-09-10T04:05:33.656 INFO:tasks.workunit.client.0.smithi046.stdout:8/913: stat d1/d29/d17/d1f/d46/d67/dc1/d9d 0 2024-09-10T04:05:33.658 INFO:tasks.workunit.client.1.smithi070.stdout:3/939: write db/f11 [2332406,122607] 0 2024-09-10T04:05:33.660 INFO:tasks.workunit.client.0.smithi046.stdout:5/995: truncate d7/d11/d14/d2f/deb/da5/dcc/dc0/fe0 2128603 0 2024-09-10T04:05:33.660 INFO:tasks.workunit.client.1.smithi070.stdout:9/986: dwrite d9/d59/d2a/fcf [0,4194304] 0 2024-09-10T04:05:33.662 INFO:tasks.workunit.client.1.smithi070.stdout:4/911: creat de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/d10d/df2/f136 x:0 0 0 2024-09-10T04:05:33.663 INFO:tasks.workunit.client.1.smithi070.stdout:4/912: chown de/lf 0 1 2024-09-10T04:05:33.664 INFO:tasks.workunit.client.0.smithi046.stdout:0/733: dwrite d1/d13/d37/d75/fd0 [0,4194304] 0 2024-09-10T04:05:33.667 INFO:tasks.workunit.client.1.smithi070.stdout:1/854: fdatasync d3/d8/d12/f24 0 2024-09-10T04:05:33.669 INFO:tasks.workunit.client.1.smithi070.stdout:4/913: dwrite de/d13/d18/d26/d8c/f12f [0,4194304] 0 2024-09-10T04:05:33.669 INFO:tasks.workunit.client.1.smithi070.stdout:1/855: write d3/d8/d27/d7a/d94/f9e [2329965,63402] 0 2024-09-10T04:05:33.672 INFO:tasks.workunit.client.1.smithi070.stdout:2/892: chown d15/d18/d29/l9d 3411385 1 2024-09-10T04:05:33.673 INFO:tasks.workunit.client.1.smithi070.stdout:2/893: fsync d15/d2b/d106/d123/f12b 0 2024-09-10T04:05:33.673 INFO:tasks.workunit.client.0.smithi046.stdout:8/914: creat d1/d29/d17/d1f/d46/d67/de0/f130 x:0 0 0 2024-09-10T04:05:33.674 INFO:tasks.workunit.client.0.smithi046.stdout:0/734: dwrite d1/dc/d25/d72/fb9 [0,4194304] 0 2024-09-10T04:05:33.675 INFO:tasks.workunit.client.1.smithi070.stdout:3/940: mknod db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/c143 0 2024-09-10T04:05:33.677 INFO:tasks.workunit.client.0.smithi046.stdout:0/735: chown d1/d13/d37/d75/d94/d9a/fbd 36 1 2024-09-10T04:05:33.678 INFO:tasks.workunit.client.0.smithi046.stdout:0/736: stat d1/dc/d25/d33/d3/d29/f3c 0 2024-09-10T04:05:33.678 INFO:tasks.workunit.client.1.smithi070.stdout:3/941: dread db/d1c/d25/d28/f37 [4194304,4194304] 0 2024-09-10T04:05:33.679 INFO:tasks.workunit.client.1.smithi070.stdout:9/987: unlink d9/d1c/l4c 0 2024-09-10T04:05:33.680 INFO:tasks.workunit.client.1.smithi070.stdout:9/988: chown d9/d1c/cb9 7 1 2024-09-10T04:05:33.684 INFO:tasks.workunit.client.1.smithi070.stdout:6/959: mkdir d0/d3/d18/d1b/d17/d3f/d40/d94/df8/d13a 0 2024-09-10T04:05:33.692 INFO:tasks.workunit.client.0.smithi046.stdout:7/959: write d5/d6/db/dd/d4b/f142 [1017626,72362] 0 2024-09-10T04:05:33.697 INFO:tasks.workunit.client.0.smithi046.stdout:0/737: rename d1/dc/d25/d33/d3/dd/fbb to d1/dc/fe7 0 2024-09-10T04:05:33.697 INFO:tasks.workunit.client.1.smithi070.stdout:9/989: truncate d9/d1c/d34/d8b/d11d/d3c/fd6 216323 0 2024-09-10T04:05:33.698 INFO:tasks.workunit.client.1.smithi070.stdout:9/990: dread - d9/f137 zero size 2024-09-10T04:05:33.699 INFO:tasks.workunit.client.1.smithi070.stdout:9/991: write d9/d59/f10d [750047,32839] 0 2024-09-10T04:05:33.700 INFO:tasks.workunit.client.0.smithi046.stdout:2/899: creat d2/d8/f13b x:0 0 0 2024-09-10T04:05:33.701 INFO:tasks.workunit.client.0.smithi046.stdout:3/875: getdents d12/dd9 0 2024-09-10T04:05:33.702 INFO:tasks.workunit.client.1.smithi070.stdout:1/856: truncate d3/d8/d27/d7a/f10e 915622 0 2024-09-10T04:05:33.702 INFO:tasks.workunit.client.0.smithi046.stdout:5/996: write d7/d11/d3f/d53/d75/fbd [211374,121138] 0 2024-09-10T04:05:33.703 INFO:tasks.workunit.client.1.smithi070.stdout:1/857: write d3/d8/d13/fc4 [1385015,15055] 0 2024-09-10T04:05:33.703 INFO:tasks.workunit.client.1.smithi070.stdout:2/894: mknod d15/c140 0 2024-09-10T04:05:33.704 INFO:tasks.workunit.client.0.smithi046.stdout:6/789: link da/db/d11/d13/d1e/d23/d54/dad/cea da/db/d11/d13/d1e/d23/d86/dbf/c103 0 2024-09-10T04:05:33.707 INFO:tasks.workunit.client.1.smithi070.stdout:4/914: write f8 [836801,59314] 0 2024-09-10T04:05:33.710 INFO:tasks.workunit.client.1.smithi070.stdout:9/992: truncate d9/fb3 1948049 0 2024-09-10T04:05:33.712 INFO:tasks.workunit.client.0.smithi046.stdout:8/915: write d1/d29/d17/d1f/d46/d67/dc1/f60 [652047,57647] 0 2024-09-10T04:05:33.714 INFO:tasks.workunit.client.1.smithi070.stdout:4/915: dwrite de/d13/d5b/d7b/dd5/f60 [4194304,4194304] 0 2024-09-10T04:05:33.716 INFO:tasks.workunit.client.1.smithi070.stdout:9/993: dread d9/d1c/d34/d8b/d11d/d1b/f123 [0,4194304] 0 2024-09-10T04:05:33.717 INFO:tasks.workunit.client.1.smithi070.stdout:9/994: fsync d9/f68 0 2024-09-10T04:05:33.718 INFO:tasks.workunit.client.0.smithi046.stdout:6/790: dread da/db/f78 [0,4194304] 0 2024-09-10T04:05:33.719 INFO:tasks.workunit.client.1.smithi070.stdout:6/960: link d0/d3/d1f/de8/c109 d0/d3/da4/c13b 0 2024-09-10T04:05:33.720 INFO:tasks.workunit.client.0.smithi046.stdout:7/960: dread d5/d6/d32/dab/db6/f79 [0,4194304] 0 2024-09-10T04:05:33.721 INFO:tasks.workunit.client.1.smithi070.stdout:9/995: dread d9/d59/d2a/f101 [0,4194304] 0 2024-09-10T04:05:33.725 INFO:tasks.workunit.client.1.smithi070.stdout:1/858: mkdir d3/d116 0 2024-09-10T04:05:33.731 INFO:tasks.workunit.client.0.smithi046.stdout:2/900: write d2/d6/f18 [2718694,6252] 0 2024-09-10T04:05:33.732 INFO:tasks.workunit.client.0.smithi046.stdout:5/997: creat d7/d11/d14/d2f/d14f/f156 x:0 0 0 2024-09-10T04:05:33.732 INFO:tasks.workunit.client.0.smithi046.stdout:2/901: readlink d2/d86/lb1 0 2024-09-10T04:05:33.737 INFO:tasks.workunit.client.0.smithi046.stdout:6/791: rename da/db/d11/d13/d1e/d23/d86/dbf/fef to da/db/d11/d13/d1e/d23/f104 0 2024-09-10T04:05:33.738 INFO:tasks.workunit.client.0.smithi046.stdout:7/961: dread d5/d6/d55/ff6 [0,4194304] 0 2024-09-10T04:05:33.738 INFO:tasks.workunit.client.0.smithi046.stdout:6/792: dread - da/db/d10/fe5 zero size 2024-09-10T04:05:33.739 INFO:tasks.workunit.client.1.smithi070.stdout:6/961: mknod d0/d3/d8c/d9a/c13c 0 2024-09-10T04:05:33.740 INFO:tasks.workunit.client.1.smithi070.stdout:9/996: dread - d9/d1c/d34/d8b/d11d/dc4/dda/de0/d90/df0/ff8 zero size 2024-09-10T04:05:33.743 INFO:tasks.workunit.client.0.smithi046.stdout:5/998: mknod d7/d11/d14/d2f/deb/da5/dcc/dc0/d134/c157 0 2024-09-10T04:05:33.744 INFO:tasks.workunit.client.0.smithi046.stdout:2/902: truncate d2/d6/d9d/fa5 1233758 0 2024-09-10T04:05:33.746 INFO:tasks.workunit.client.0.smithi046.stdout:7/962: rmdir d5/d9/d12/d72 39 2024-09-10T04:05:33.747 INFO:tasks.workunit.client.0.smithi046.stdout:3/876: getdents d12/d13/d37/d51/db5/d4b/db7 0 2024-09-10T04:05:33.748 INFO:tasks.workunit.client.1.smithi070.stdout:2/895: dwrite d15/d18/f24 [0,4194304] 0 2024-09-10T04:05:33.748 INFO:tasks.workunit.client.0.smithi046.stdout:6/793: rename da/db/d11/d13/d1e/d23/d86/cb0 to da/db/d11/d13/d1e/d23/d54/dad/c105 0 2024-09-10T04:05:33.752 INFO:tasks.workunit.client.0.smithi046.stdout:5/999: dwrite d7/d11/d14/d8e/f132 [0,4194304] 0 2024-09-10T04:05:33.753 INFO:tasks.workunit.client.1.smithi070.stdout:9/997: mknod d9/d1c/d34/d8b/dcc/dfa/c13a 0 2024-09-10T04:05:33.753 INFO:tasks.workunit.client.1.smithi070.stdout:4/916: rmdir de/d13/d18/d23/d47/d59/d10f 0 2024-09-10T04:05:33.756 INFO:tasks.workunit.client.1.smithi070.stdout:6/962: mknod d0/d3/d8c/d9a/d106/c13d 0 2024-09-10T04:05:33.758 INFO:tasks.workunit.client.1.smithi070.stdout:1/859: rename d3/d8/d27/d7a to d3/d8/d117 0 2024-09-10T04:05:33.761 INFO:tasks.workunit.client.1.smithi070.stdout:9/998: symlink d9/d1c/d34/d8b/d10e/l13b 0 2024-09-10T04:05:33.761 INFO:tasks.workunit.client.1.smithi070.stdout:1/860: chown d3/d90/dcb/f105 1902569 1 2024-09-10T04:05:33.762 INFO:tasks.workunit.client.1.smithi070.stdout:4/917: dwrite de/d13/d5b/d7b/dd5/d85/d10e/f57 [0,4194304] 0 2024-09-10T04:05:33.764 INFO:tasks.workunit.client.1.smithi070.stdout:4/918: fsync de/d13/d5b/d7b/fe2 0 2024-09-10T04:05:33.764 INFO:tasks.workunit.client.0.smithi046.stdout:3/877: symlink d12/d13/d14/d1e/d2d/d39/d67/da0/d114/l12b 0 2024-09-10T04:05:33.767 INFO:tasks.workunit.client.0.smithi046.stdout:7/963: mknod d5/d9/d12/d72/d77/de0/d1f/d7a/c14f 0 2024-09-10T04:05:33.771 INFO:tasks.workunit.client.1.smithi070.stdout:2/896: rename d15/df6/ff9 to d15/d18/d56/d7c/d55/f141 0 2024-09-10T04:05:33.772 INFO:tasks.workunit.client.0.smithi046.stdout:2/903: getdents d2/d6/d9d/db6 0 2024-09-10T04:05:33.775 INFO:tasks.workunit.client.0.smithi046.stdout:6/794: creat da/db/d18/f106 x:0 0 0 2024-09-10T04:05:33.778 INFO:tasks.workunit.client.1.smithi070.stdout:1/861: dwrite d3/d8/d12/d2f/d32/f6b [0,4194304] 0 2024-09-10T04:05:33.780 INFO:tasks.workunit.client.0.smithi046.stdout:2/904: dwrite d2/d6/d23/d42/f53 [0,4194304] 0 2024-09-10T04:05:33.783 INFO:tasks.workunit.client.1.smithi070.stdout:4/919: unlink de/d13/da6/d115/f131 0 2024-09-10T04:05:33.784 INFO:tasks.workunit.client.1.smithi070.stdout:4/920: stat de/d13/d18/d23/f61 0 2024-09-10T04:05:33.785 INFO:tasks.workunit.client.0.smithi046.stdout:7/964: truncate d5/d9/f5b 922984 0 2024-09-10T04:05:33.790 INFO:tasks.workunit.client.1.smithi070.stdout:2/897: creat d15/d18/db2/de9/f142 x:0 0 0 2024-09-10T04:05:33.793 INFO:tasks.workunit.client.1.smithi070.stdout:6/963: dwrite d0/d3/d18/fd4 [0,4194304] 0 2024-09-10T04:05:33.801 INFO:tasks.workunit.client.1.smithi070.stdout:4/921: mkdir de/d13/d18/d23/d47/d59/d79/d6a/d137 0 2024-09-10T04:05:33.801 INFO:tasks.workunit.client.0.smithi046.stdout:6/795: write da/db/d11/d2b/d2f/f37 [939875,38892] 0 2024-09-10T04:05:33.802 INFO:tasks.workunit.client.0.smithi046.stdout:3/878: dwrite d12/d13/d37/d51/db5/d50/d7a/dc1/f97 [0,4194304] 0 2024-09-10T04:05:33.804 INFO:tasks.workunit.client.1.smithi070.stdout:3/942: sync 2024-09-10T04:05:33.805 INFO:tasks.workunit.client.0.smithi046.stdout:7/965: creat d5/d33/d135/f150 x:0 0 0 2024-09-10T04:05:33.807 INFO:tasks.workunit.client.0.smithi046.stdout:7/966: readlink d5/d9/d12/d72/d77/de0/d1e/l9e 0 2024-09-10T04:05:33.809 INFO:tasks.workunit.client.1.smithi070.stdout:9/999: link d9/f35 d9/d1c/d34/d8b/dcc/de7/f13c 0 2024-09-10T04:05:33.809 INFO:tasks.workunit.client.0.smithi046.stdout:3/879: creat d12/d13/d37/d51/f12c x:0 0 0 2024-09-10T04:05:33.811 INFO:tasks.workunit.client.0.smithi046.stdout:6/796: rename da/db/d11/d13/d1e/d23/d86/dbf/ff9 to da/d45/f107 0 2024-09-10T04:05:33.812 INFO:tasks.workunit.client.1.smithi070.stdout:4/922: mknod de/d13/d5b/d7b/dd5/d85/d10e/d48/c138 0 2024-09-10T04:05:33.813 INFO:tasks.workunit.client.1.smithi070.stdout:4/923: chown de/d13/d5b/da0/f108 17 1 2024-09-10T04:05:33.815 INFO:tasks.workunit.client.0.smithi046.stdout:7/967: dwrite d5/d9/f42 [0,4194304] 0 2024-09-10T04:05:33.816 INFO:tasks.workunit.client.0.smithi046.stdout:2/905: getdents d2/d6/d23/d55 0 2024-09-10T04:05:33.819 INFO:tasks.workunit.client.0.smithi046.stdout:3/880: mknod d12/d13/d37/d94/dff/c12d 0 2024-09-10T04:05:33.823 INFO:tasks.workunit.client.1.smithi070.stdout:2/898: creat d15/d18/d29/d2a/d2e/da6/dc0/f143 x:0 0 0 2024-09-10T04:05:33.825 INFO:tasks.workunit.client.1.smithi070.stdout:1/862: write d3/d8/d13/d3f/d6d/d81/fd8 [1033783,76341] 0 2024-09-10T04:05:33.826 INFO:tasks.workunit.client.0.smithi046.stdout:2/906: rename d2/d6/c7a to d2/d8/d37/d10a/c13c 0 2024-09-10T04:05:33.833 INFO:tasks.workunit.client.1.smithi070.stdout:6/964: truncate d0/d3/d1f/f46 2066810 0 2024-09-10T04:05:33.835 INFO:tasks.workunit.client.1.smithi070.stdout:1/863: creat d3/d8/d117/d94/f118 x:0 0 0 2024-09-10T04:05:33.839 INFO:tasks.workunit.client.1.smithi070.stdout:3/943: creat db/d1c/d25/f144 x:0 0 0 2024-09-10T04:05:33.839 INFO:tasks.workunit.client.1.smithi070.stdout:4/924: write de/d13/d5b/d7b/dd5/d85/d10e/d48/fee [1152651,46409] 0 2024-09-10T04:05:33.840 INFO:tasks.workunit.client.0.smithi046.stdout:2/907: creat d2/d8/d37/dc2/f13d x:0 0 0 2024-09-10T04:05:33.841 INFO:tasks.workunit.client.0.smithi046.stdout:3/881: write d12/d5d/d71/d49/d5c/fe9 [695998,49526] 0 2024-09-10T04:05:33.841 INFO:tasks.workunit.client.1.smithi070.stdout:2/899: write d15/d18/d56/d7c/dfa/f10f [299060,89874] 0 2024-09-10T04:05:33.842 INFO:tasks.workunit.client.0.smithi046.stdout:3/882: readlink d12/d13/d14/d1e/l79 0 2024-09-10T04:05:33.842 INFO:tasks.workunit.client.1.smithi070.stdout:6/965: fsync d0/d3/d8c/fad 0 2024-09-10T04:05:33.843 INFO:tasks.workunit.client.0.smithi046.stdout:6/797: truncate da/db/d11/d13/d1e/f3e 3383855 0 2024-09-10T04:05:33.844 INFO:tasks.workunit.client.0.smithi046.stdout:7/968: link d5/d6/d55/d89/f9d d5/d9/d43/db4/f151 0 2024-09-10T04:05:33.845 INFO:tasks.workunit.client.1.smithi070.stdout:1/864: symlink d3/d8/d13/d67/d83/def/l119 0 2024-09-10T04:05:33.849 INFO:tasks.workunit.client.0.smithi046.stdout:2/908: rename d2/d8/d65/c7f to d2/d6/d9d/c13e 0 2024-09-10T04:05:33.851 INFO:tasks.workunit.client.0.smithi046.stdout:2/909: dread - d2/d6/d23/d5d/d77/d7b/f11e zero size 2024-09-10T04:05:33.852 INFO:tasks.workunit.client.0.smithi046.stdout:6/798: symlink da/db/d11/d13/d1e/d23/d54/dad/dfe/l108 0 2024-09-10T04:05:33.852 INFO:tasks.workunit.client.0.smithi046.stdout:2/910: read d2/d6/d23/d42/f53 [2716530,75346] 0 2024-09-10T04:05:33.853 INFO:tasks.workunit.client.0.smithi046.stdout:6/799: write da/db/d11/d2b/d55/fb2 [2568644,96106] 0 2024-09-10T04:05:33.858 INFO:tasks.workunit.client.1.smithi070.stdout:6/966: mkdir d0/d3/d5c/d13e 0 2024-09-10T04:05:33.858 INFO:tasks.workunit.client.1.smithi070.stdout:6/967: chown d0/f8 30 1 2024-09-10T04:05:33.865 INFO:tasks.workunit.client.1.smithi070.stdout:3/944: dwrite db/d1c/d25/d28/dd3/de5/d10f/fa3 [0,4194304] 0 2024-09-10T04:05:33.867 INFO:tasks.workunit.client.0.smithi046.stdout:3/883: rename d12/d13/f105 to d12/d13/d37/d51/db5/d4b/db7/dc0/f12e 0 2024-09-10T04:05:33.871 INFO:tasks.workunit.client.0.smithi046.stdout:2/911: fdatasync d2/d6/d23/d42/de6/f107 0 2024-09-10T04:05:33.874 INFO:tasks.workunit.client.1.smithi070.stdout:2/900: symlink d15/d18/d29/d2a/d2e/d32/d36/d10b/l144 0 2024-09-10T04:05:33.874 INFO:tasks.workunit.client.1.smithi070.stdout:2/901: dread - d15/d2b/d94/f130 zero size 2024-09-10T04:05:33.879 INFO:tasks.workunit.client.0.smithi046.stdout:7/969: rename d5/d6/db/dd/ld9 to d5/d6/d32/dab/l152 0 2024-09-10T04:05:33.881 INFO:tasks.workunit.client.1.smithi070.stdout:3/945: rename db/d83/d92/c127 to db/d1c/d25/d28/dd3/de5/d12c/c145 0 2024-09-10T04:05:33.885 INFO:tasks.workunit.client.0.smithi046.stdout:2/912: mkdir d2/d6/d23/d42/db4/dca/d13f 0 2024-09-10T04:05:33.886 INFO:tasks.workunit.client.1.smithi070.stdout:6/968: getdents d0/d3/d18/d1b/d17/d127 0 2024-09-10T04:05:33.887 INFO:tasks.workunit.client.1.smithi070.stdout:1/865: link d3/d8/d12/leb d3/d8/d13/d67/d83/def/dfb/l11a 0 2024-09-10T04:05:33.889 INFO:tasks.workunit.client.1.smithi070.stdout:4/925: getdents de/d13/d5b/d7b 0 2024-09-10T04:05:33.893 INFO:tasks.workunit.client.1.smithi070.stdout:2/902: dread d15/d18/d29/d2a/fbe [0,4194304] 0 2024-09-10T04:05:33.894 INFO:tasks.workunit.client.0.smithi046.stdout:2/913: creat d2/d6/d23/dd5/df8/f140 x:0 0 0 2024-09-10T04:05:33.894 INFO:tasks.workunit.client.1.smithi070.stdout:3/946: fsync db/dcc/f129 0 2024-09-10T04:05:33.895 INFO:tasks.workunit.client.0.smithi046.stdout:6/800: dwrite da/db/d11/d13/d1e/d9d/f102 [0,4194304] 0 2024-09-10T04:05:33.895 INFO:tasks.workunit.client.1.smithi070.stdout:4/926: dwrite de/d13/d5b/d7b/f114 [0,4194304] 0 2024-09-10T04:05:33.896 INFO:tasks.workunit.client.1.smithi070.stdout:4/927: write de/d13/d18/d26/d8c/f95 [1116422,115931] 0 2024-09-10T04:05:33.897 INFO:tasks.workunit.client.0.smithi046.stdout:7/970: dwrite d5/d9/d43/db4/def/f105 [0,4194304] 0 2024-09-10T04:05:33.897 INFO:tasks.workunit.client.1.smithi070.stdout:4/928: chown de/d13/d5b/d7b/la2 157391 1 2024-09-10T04:05:33.898 INFO:tasks.workunit.client.0.smithi046.stdout:3/884: creat d12/d13/d14/d1e/d2d/d39/d67/f12f x:0 0 0 2024-09-10T04:05:33.898 INFO:tasks.workunit.client.0.smithi046.stdout:6/801: chown da/db/d11/d2b/d5a/fd9 5 1 2024-09-10T04:05:33.903 INFO:tasks.workunit.client.1.smithi070.stdout:6/969: symlink d0/d3/d1f/d81/l13f 0 2024-09-10T04:05:33.903 INFO:tasks.workunit.client.1.smithi070.stdout:1/866: mkdir d3/d8/d27/d9a/d11b 0 2024-09-10T04:05:33.905 INFO:tasks.workunit.client.1.smithi070.stdout:1/867: fdatasync d3/dd3/f10b 0 2024-09-10T04:05:33.905 INFO:tasks.workunit.client.1.smithi070.stdout:6/970: chown d0/d3/d18/d1b/c9f 0 1 2024-09-10T04:05:33.905 INFO:tasks.workunit.client.1.smithi070.stdout:4/929: mkdir de/d13/d43/d88/db5/d139 0 2024-09-10T04:05:33.907 INFO:tasks.workunit.client.1.smithi070.stdout:2/903: mkdir d15/d18/d56/d7c/d10c/d145 0 2024-09-10T04:05:33.909 INFO:tasks.workunit.client.0.smithi046.stdout:7/971: rmdir d5/d9/d43/db4/def/d139 39 2024-09-10T04:05:33.909 INFO:tasks.workunit.client.0.smithi046.stdout:2/914: dwrite d2/d8/d37/dc2/f137 [0,4194304] 0 2024-09-10T04:05:33.909 INFO:tasks.workunit.client.0.smithi046.stdout:6/802: dread da/db/d10/fe2 [0,4194304] 0 2024-09-10T04:05:33.912 INFO:tasks.workunit.client.0.smithi046.stdout:3/885: mknod d12/d13/d37/dab/dac/c130 0 2024-09-10T04:05:33.914 INFO:tasks.workunit.client.1.smithi070.stdout:6/971: rmdir d0/d3/da4 39 2024-09-10T04:05:33.914 INFO:tasks.workunit.client.1.smithi070.stdout:4/930: symlink de/d13/d18/d23/d47/d59/d79/d6a/dce/dfe/d10d/l13a 0 2024-09-10T04:05:33.915 INFO:tasks.workunit.client.1.smithi070.stdout:1/868: read d3/d10/f35 [2755542,124533] 0 2024-09-10T04:05:33.916 INFO:tasks.workunit.client.1.smithi070.stdout:1/869: dread - d3/dd3/f10b zero size 2024-09-10T04:05:33.919 INFO:tasks.workunit.client.0.smithi046.stdout:6/803: dwrite da/db/d11/ffc [0,4194304] 0 2024-09-10T04:05:33.922 INFO:tasks.workunit.client.1.smithi070.stdout:3/947: creat db/d1c/d25/d28/f146 x:0 0 0 2024-09-10T04:05:33.922 INFO:tasks.workunit.client.1.smithi070.stdout:4/931: dwrite de/d13/d18/d26/d8c/d124/f135 [0,4194304] 0 2024-09-10T04:05:33.925 INFO:tasks.workunit.client.0.smithi046.stdout:3/886: rmdir d12/d13 39 2024-09-10T04:05:33.926 INFO:tasks.workunit.client.1.smithi070.stdout:6/972: creat d0/d3/d18/d1b/d17/d54/dc4/d125/f140 x:0 0 0 2024-09-10T04:05:33.928 INFO:tasks.workunit.client.1.smithi070.stdout:6/973: read - d0/d3/d18/d1b/d33/fff zero size 2024-09-10T04:05:33.931 INFO:tasks.workunit.client.0.smithi046.stdout:7/972: mknod d5/d9/d43/db4/def/c153 0 2024-09-10T04:05:33.933 INFO:tasks.workunit.client.0.smithi046.stdout:6/804: rmdir da 39 2024-09-10T04:05:33.936 INFO:tasks.workunit.client.1.smithi070.stdout:4/932: symlink de/d13/d5b/d7b/dd5/d85/l13b 0 2024-09-10T04:05:33.938 INFO:tasks.workunit.client.1.smithi070.stdout:6/974: symlink d0/d3/d5c/l141 0 2024-09-10T04:05:33.939 INFO:tasks.workunit.client.0.smithi046.stdout:7/973: chown d5/d6/d55/d7d/c13f 9123 1 2024-09-10T04:05:33.939 INFO:tasks.workunit.client.1.smithi070.stdout:2/904: write d15/d18/d56/d7c/d55/d6d/fa8 [1475257,8289] 0 2024-09-10T04:05:33.940 INFO:tasks.workunit.client.1.smithi070.stdout:6/975: chown d0/d3/d18/d1b/d17/d3f/d40/d94/df8 558877193 1 2024-09-10T04:05:33.940 INFO:tasks.workunit.client.1.smithi070.stdout:1/870: mknod d3/d10/c11c 0 2024-09-10T04:05:33.940 INFO:tasks.workunit.client.1.smithi070.stdout:3/948: dread db/d83/d92/faa [0,4194304] 0 2024-09-10T04:05:33.940 INFO:tasks.workunit.client.0.smithi046.stdout:7/974: write d5/d6/d32/f12c [1385115,24644] 0 2024-09-10T04:05:33.944 INFO:tasks.workunit.client.0.smithi046.stdout:3/887: creat d12/d13/d37/d51/db5/d4b/db7/dc0/ded/f131 x:0 0 0 2024-09-10T04:05:33.948 INFO:tasks.workunit.client.0.smithi046.stdout:2/915: dwrite d2/d6/d23/f87 [0,4194304] 0 2024-09-10T04:05:33.949 INFO:tasks.workunit.client.1.smithi070.stdout:1/871: creat d3/d8/d13/d3f/d6d/f11d x:0 0 0 2024-09-10T04:05:33.950 INFO:tasks.workunit.client.1.smithi070.stdout:6/976: dread d0/d3/d18/d1b/d17/d54/f87 [0,4194304] 0 2024-09-10T04:05:33.951 INFO:tasks.workunit.client.0.smithi046.stdout:2/916: write d2/d6/d23/dd5/df8/f140 [252120,116113] 0 2024-09-10T04:05:33.952 INFO:tasks.workunit.client.0.smithi046.stdout:3/888: mknod d12/d13/d14/d1e/d2d/d39/d67/d92/c132 0 2024-09-10T04:05:33.957 INFO:tasks.workunit.client.0.smithi046.stdout:1/709: sync 2024-09-10T04:05:33.957 INFO:tasks.workunit.client.0.smithi046.stdout:9/881: sync 2024-09-10T04:05:33.957 INFO:tasks.workunit.client.0.smithi046.stdout:4/997: sync 2024-09-10T04:05:33.957 INFO:tasks.workunit.client.0.smithi046.stdout:0/738: sync 2024-09-10T04:05:33.959 INFO:tasks.workunit.client.0.smithi046.stdout:6/805: rmdir da/db/d63/de1 0 2024-09-10T04:05:33.961 INFO:tasks.workunit.client.0.smithi046.stdout:2/917: dwrite d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:33.962 INFO:tasks.workunit.client.0.smithi046.stdout:4/998: chown d3/d12/d17/d22/d51/d94/d107/c13f 90973 1 2024-09-10T04:05:33.966 INFO:tasks.workunit.client.0.smithi046.stdout:2/918: write d2/d6/d9d/db6/dbe/fde [563286,51885] 0 2024-09-10T04:05:33.966 INFO:tasks.workunit.client.0.smithi046.stdout:3/889: symlink d12/d13/l133 0 2024-09-10T04:05:33.966 INFO:tasks.workunit.client.1.smithi070.stdout:6/977: creat d0/d3/d1f/d55/d129/f142 x:0 0 0 2024-09-10T04:05:33.967 INFO:tasks.workunit.client.0.smithi046.stdout:7/975: write d5/d9/d12/fa4 [1362430,2461] 0 2024-09-10T04:05:33.968 INFO:tasks.workunit.client.1.smithi070.stdout:6/978: dread d0/d3/d8c/f110 [0,4194304] 0 2024-09-10T04:05:33.970 INFO:tasks.workunit.client.1.smithi070.stdout:3/949: link db/d1c/d25/d28/dd3/de5/d10f/d110/f132 db/d1c/d25/d28/f147 0 2024-09-10T04:05:33.971 INFO:tasks.workunit.client.1.smithi070.stdout:4/933: truncate f8 215506 0 2024-09-10T04:05:33.972 INFO:tasks.workunit.client.1.smithi070.stdout:4/934: dread - de/d13/d43/d88/fa3 zero size 2024-09-10T04:05:33.973 INFO:tasks.workunit.client.1.smithi070.stdout:2/905: getdents d15 0 2024-09-10T04:05:33.975 INFO:tasks.workunit.client.1.smithi070.stdout:1/872: creat d3/d8/d13/d3f/f11e x:0 0 0 2024-09-10T04:05:33.979 INFO:tasks.workunit.client.0.smithi046.stdout:1/710: dwrite d3/dc/d34/d58/fce [0,4194304] 0 2024-09-10T04:05:33.980 INFO:tasks.workunit.client.0.smithi046.stdout:2/919: dread d2/d49/f101 [0,4194304] 0 2024-09-10T04:05:33.980 INFO:tasks.workunit.client.1.smithi070.stdout:1/873: dwrite d3/d8/f10f [0,4194304] 0 2024-09-10T04:05:33.982 INFO:tasks.workunit.client.0.smithi046.stdout:6/806: symlink da/db/d11/l109 0 2024-09-10T04:05:33.984 INFO:tasks.workunit.client.0.smithi046.stdout:0/739: chown d1/dc/d25/d72 358859181 1 2024-09-10T04:05:33.984 INFO:tasks.workunit.client.0.smithi046.stdout:7/976: dread - d5/d6/d62/dc1/fc4 zero size 2024-09-10T04:05:33.985 INFO:tasks.workunit.client.0.smithi046.stdout:7/977: chown d5/d6/db/dd/d24/d76/l8f 14922021 1 2024-09-10T04:05:33.990 INFO:tasks.workunit.client.1.smithi070.stdout:3/950: symlink db/d1c/d89/d112/l148 0 2024-09-10T04:05:33.994 INFO:tasks.workunit.client.1.smithi070.stdout:4/935: mknod de/d13/d5b/da0/d107/c13c 0 2024-09-10T04:05:33.995 INFO:tasks.workunit.client.1.smithi070.stdout:4/936: write de/d13/d18/d23/f116 [270297,91045] 0 2024-09-10T04:05:33.995 INFO:tasks.workunit.client.1.smithi070.stdout:4/937: readlink de/d13/d18/d5c/db6/lc0 0 2024-09-10T04:05:33.997 INFO:tasks.workunit.client.1.smithi070.stdout:4/938: read - de/d13/d18/d23/d47/ff4 zero size 2024-09-10T04:05:33.997 INFO:tasks.workunit.client.1.smithi070.stdout:1/874: dread d3/d8/d12/d2f/fd5 [0,4194304] 0 2024-09-10T04:05:34.002 INFO:tasks.workunit.client.0.smithi046.stdout:4/999: dread d3/dab/dca/de4/fe9 [0,4194304] 0 2024-09-10T04:05:34.002 INFO:tasks.workunit.client.1.smithi070.stdout:3/951: rmdir db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/de1 39 2024-09-10T04:05:34.004 INFO:tasks.workunit.client.1.smithi070.stdout:4/939: unlink f0 0 2024-09-10T04:05:34.007 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:33 smithi070 ceph-mon[35495]: pgmap v270: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 107 MiB/s rd, 254 MiB/s wr, 540 op/s 2024-09-10T04:05:34.008 INFO:tasks.workunit.client.1.smithi070.stdout:6/979: truncate d0/d3/d18/d1b/d17/d3f/d40/f48 2839737 0 2024-09-10T04:05:34.009 INFO:tasks.workunit.client.1.smithi070.stdout:3/952: creat db/d83/d92/d11b/f149 x:0 0 0 2024-09-10T04:05:34.010 INFO:tasks.workunit.client.1.smithi070.stdout:1/875: dwrite d3/d8/fbc [0,4194304] 0 2024-09-10T04:05:34.012 INFO:tasks.workunit.client.1.smithi070.stdout:4/940: symlink de/d13/d18/l13d 0 2024-09-10T04:05:34.015 INFO:tasks.workunit.client.1.smithi070.stdout:2/906: link d15/d18/d29/d2a/d2e/ldd d15/d18/d29/d2a/d10a/l146 0 2024-09-10T04:05:34.020 INFO:tasks.workunit.client.1.smithi070.stdout:1/876: rename d3/f47 to d3/d10/d1d/d79/f11f 0 2024-09-10T04:05:34.022 INFO:tasks.workunit.client.1.smithi070.stdout:2/907: read d15/d18/d29/d2a/d2e/da6/dc0/dcb/fb9 [740379,74231] 0 2024-09-10T04:05:34.025 INFO:tasks.workunit.client.1.smithi070.stdout:3/953: mkdir db/d1c/d14a 0 2024-09-10T04:05:34.032 INFO:tasks.workunit.client.1.smithi070.stdout:2/908: rename d15/d18/d56/d7c/d55/d6d/lb4 to d15/d18/d29/d2a/d10a/l147 0 2024-09-10T04:05:34.039 INFO:tasks.workunit.client.1.smithi070.stdout:2/909: mkdir d15/d18/d29/ddb/d148 0 2024-09-10T04:05:34.040 INFO:tasks.workunit.client.1.smithi070.stdout:2/910: write d15/d18/db2/f11f [1870333,18020] 0 2024-09-10T04:05:34.040 INFO:tasks.workunit.client.1.smithi070.stdout:3/954: rmdir db/d1c/d25/d28/dd3/de5/d10f/d110 39 2024-09-10T04:05:34.042 INFO:tasks.workunit.client.1.smithi070.stdout:3/955: chown db/d1c/d25/l81 178384753 1 2024-09-10T04:05:34.045 INFO:tasks.workunit.client.1.smithi070.stdout:6/980: dwrite d0/d3/d8c/ff2 [0,4194304] 0 2024-09-10T04:05:34.047 INFO:tasks.workunit.client.1.smithi070.stdout:2/911: rename d15/d18/d29/d2a/fbe to d15/d18/d29/d2a/d2e/d32/d36/f149 0 2024-09-10T04:05:34.048 INFO:tasks.workunit.client.1.smithi070.stdout:2/912: chown d15/d18/d29/d2a/d2e/d32/fc9 7932398 1 2024-09-10T04:05:34.050 INFO:tasks.workunit.client.0.smithi046.stdout:2/920: creat d2/d8/d65/d90/f141 x:0 0 0 2024-09-10T04:05:34.051 INFO:tasks.workunit.client.1.smithi070.stdout:4/941: mknod de/d13/d18/d5c/db6/c13e 0 2024-09-10T04:05:34.054 INFO:tasks.workunit.client.1.smithi070.stdout:3/956: truncate db/d1c/d25/d28/f109 683877 0 2024-09-10T04:05:34.055 INFO:tasks.workunit.client.0.smithi046.stdout:6/807: rmdir da/d59 39 2024-09-10T04:05:34.056 INFO:tasks.workunit.client.0.smithi046.stdout:6/808: chown da/de3/lee 1221579390 1 2024-09-10T04:05:34.061 INFO:tasks.workunit.client.0.smithi046.stdout:7/978: write d5/d9/f137 [343338,37166] 0 2024-09-10T04:05:34.061 INFO:tasks.workunit.client.1.smithi070.stdout:6/981: truncate d0/fa7 1345259 0 2024-09-10T04:05:34.062 INFO:tasks.workunit.client.0.smithi046.stdout:7/979: dread - d5/d6/d62/f13e zero size 2024-09-10T04:05:34.062 INFO:tasks.workunit.client.1.smithi070.stdout:1/877: write d3/d8/d13/d3f/f78 [1367485,120042] 0 2024-09-10T04:05:34.063 INFO:tasks.workunit.client.1.smithi070.stdout:4/942: symlink de/d13/d43/d88/db5/d139/l13f 0 2024-09-10T04:05:34.064 INFO:tasks.workunit.client.0.smithi046.stdout:1/711: mkdir d3/d27/df1 0 2024-09-10T04:05:34.067 INFO:tasks.workunit.client.0.smithi046.stdout:2/921: mkdir d2/d6/d23/d42/dc8/d142 0 2024-09-10T04:05:34.067 INFO:tasks.workunit.client.0.smithi046.stdout:2/922: readlink d2/d86/lb1 0 2024-09-10T04:05:34.067 INFO:tasks.workunit.client.0.smithi046.stdout:2/923: readlink d2/d8/l21 0 2024-09-10T04:05:34.068 INFO:tasks.workunit.client.1.smithi070.stdout:3/957: creat db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/de1/f14b x:0 0 0 2024-09-10T04:05:34.069 INFO:tasks.workunit.client.0.smithi046.stdout:6/809: mknod da/de3/c10a 0 2024-09-10T04:05:34.071 INFO:tasks.workunit.client.1.smithi070.stdout:6/982: symlink d0/d3/d18/d1b/d17/d3f/d40/d94/df8/l143 0 2024-09-10T04:05:34.072 INFO:tasks.workunit.client.0.smithi046.stdout:7/980: rmdir d5/d33 39 2024-09-10T04:05:34.074 INFO:tasks.workunit.client.0.smithi046.stdout:3/890: mknod d12/d13/d14/d1e/d2d/d39/d67/da0/d114/c134 0 2024-09-10T04:05:34.075 INFO:tasks.workunit.client.1.smithi070.stdout:3/958: write db/d1c/d25/d28/dd3/de5/d10f/fff [183752,91865] 0 2024-09-10T04:05:34.076 INFO:tasks.workunit.client.0.smithi046.stdout:1/712: unlink d3/da1/fe4 0 2024-09-10T04:05:34.077 INFO:tasks.workunit.client.0.smithi046.stdout:1/713: chown d3/d9b/c44 207264 1 2024-09-10T04:05:34.078 INFO:tasks.workunit.client.0.smithi046.stdout:2/924: fdatasync d2/d6/f80 0 2024-09-10T04:05:34.079 INFO:tasks.workunit.client.0.smithi046.stdout:0/740: write d1/d13/d37/fb7 [529335,86024] 0 2024-09-10T04:05:34.081 INFO:tasks.workunit.client.0.smithi046.stdout:6/810: symlink da/db/d11/d9f/dd6/l10b 0 2024-09-10T04:05:34.083 INFO:tasks.workunit.client.0.smithi046.stdout:7/981: mknod d5/d6/d55/d7d/c154 0 2024-09-10T04:05:34.085 INFO:tasks.workunit.client.1.smithi070.stdout:3/959: chown db/d1c/d25/d28/dd3/de5/d10f/d110/f136 7 1 2024-09-10T04:05:34.086 INFO:tasks.workunit.client.0.smithi046.stdout:3/891: read d12/f5f [3766315,14207] 0 2024-09-10T04:05:34.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:33 smithi046 ceph-mon[28328]: pgmap v270: 65 pgs: 65 active+clean; 3.4 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 107 MiB/s rd, 254 MiB/s wr, 540 op/s 2024-09-10T04:05:34.088 INFO:tasks.workunit.client.1.smithi070.stdout:1/878: mknod d3/d20/c120 0 2024-09-10T04:05:34.092 INFO:tasks.workunit.client.1.smithi070.stdout:4/943: write de/d13/d18/d26/fa4 [957834,127491] 0 2024-09-10T04:05:34.097 INFO:tasks.workunit.client.1.smithi070.stdout:6/983: rmdir d0/d3/d18/d1b/d17/d3f/d40/d94/df8/d13a 0 2024-09-10T04:05:34.099 INFO:tasks.workunit.client.0.smithi046.stdout:1/714: mknod d3/dc/d21/d3a/d90/dea/cf2 0 2024-09-10T04:05:34.101 INFO:tasks.workunit.client.1.smithi070.stdout:3/960: mknod db/d1c/d25/d28/dd3/de5/d10f/d94/d101/c14c 0 2024-09-10T04:05:34.102 INFO:tasks.workunit.client.1.smithi070.stdout:1/879: getdents d3/d116 0 2024-09-10T04:05:34.102 INFO:tasks.workunit.client.1.smithi070.stdout:4/944: dread de/d13/d5b/d7b/dd5/d85/f9a [0,4194304] 0 2024-09-10T04:05:34.105 INFO:tasks.workunit.client.0.smithi046.stdout:2/925: dwrite d2/d8/d37/d63/f75 [0,4194304] 0 2024-09-10T04:05:34.105 INFO:tasks.workunit.client.0.smithi046.stdout:1/715: symlink d3/dc/d34/d58/da6/lf3 0 2024-09-10T04:05:34.107 INFO:tasks.workunit.client.0.smithi046.stdout:0/741: dread d1/dc/d25/d33/d1a/f8e [0,4194304] 0 2024-09-10T04:05:34.108 INFO:tasks.workunit.client.1.smithi070.stdout:6/984: fdatasync d0/d3/d1f/f46 0 2024-09-10T04:05:34.108 INFO:tasks.workunit.client.1.smithi070.stdout:1/880: dwrite d3/d8/d13/d3f/d6d/d81/fe9 [0,4194304] 0 2024-09-10T04:05:34.109 INFO:tasks.workunit.client.1.smithi070.stdout:6/985: chown d0/d3/d1f/d81/f96 111 1 2024-09-10T04:05:34.109 INFO:tasks.workunit.client.1.smithi070.stdout:1/881: readlink d3/d8/d13/d3f/d5d/l114 0 2024-09-10T04:05:34.111 INFO:tasks.workunit.client.0.smithi046.stdout:7/982: dread d5/f60 [0,4194304] 0 2024-09-10T04:05:34.113 INFO:tasks.workunit.client.0.smithi046.stdout:3/892: link d12/d13/d37/faa d12/d13/d14/d1e/d2d/d39/d67/da0/f135 0 2024-09-10T04:05:34.113 INFO:tasks.workunit.client.0.smithi046.stdout:7/983: stat d5/d9/d12/d72/d77/de0/d1f/d7a/c119 0 2024-09-10T04:05:34.117 INFO:tasks.workunit.client.0.smithi046.stdout:6/811: truncate da/d96/ff0 836478 0 2024-09-10T04:05:34.118 INFO:tasks.workunit.client.0.smithi046.stdout:6/812: dread - da/db/d18/f106 zero size 2024-09-10T04:05:34.120 INFO:tasks.workunit.client.0.smithi046.stdout:2/926: chown d2/d6/d23/d5d/d77/fc3 365 1 2024-09-10T04:05:34.121 INFO:tasks.workunit.client.0.smithi046.stdout:2/927: dread d2/d6/d9d/f106 [0,4194304] 0 2024-09-10T04:05:34.122 INFO:tasks.workunit.client.1.smithi070.stdout:3/961: truncate db/d1c/d3e/f59 1486490 0 2024-09-10T04:05:34.122 INFO:tasks.workunit.client.0.smithi046.stdout:0/742: dread - d1/f8f zero size 2024-09-10T04:05:34.124 INFO:tasks.workunit.client.0.smithi046.stdout:7/984: truncate d5/d6/db/dd/d24/db7/fe9 987005 0 2024-09-10T04:05:34.124 INFO:tasks.workunit.client.1.smithi070.stdout:1/882: readlink d3/d8/d92/l104 0 2024-09-10T04:05:34.126 INFO:tasks.workunit.client.0.smithi046.stdout:1/716: unlink d3/dc/d34/d51/l6f 0 2024-09-10T04:05:34.127 INFO:tasks.workunit.client.1.smithi070.stdout:6/986: read d0/d3/d18/d1b/fd3 [120045,13216] 0 2024-09-10T04:05:34.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/743: dread d1/dc/d25/d33/d3/dd/d55/d77/da6/fa8 [0,4194304] 0 2024-09-10T04:05:34.130 INFO:tasks.workunit.client.1.smithi070.stdout:4/945: write de/d13/d5b/d7b/dd5/d85/d10e/d77/fb1 [185255,67799] 0 2024-09-10T04:05:34.132 INFO:tasks.workunit.client.0.smithi046.stdout:3/893: unlink d12/d13/f17 0 2024-09-10T04:05:34.134 INFO:tasks.workunit.client.0.smithi046.stdout:3/894: fsync d12/d5d/d71/d49/f10e 0 2024-09-10T04:05:34.135 INFO:tasks.workunit.client.0.smithi046.stdout:7/985: rmdir d5/d6/db/dd/d24 39 2024-09-10T04:05:34.140 INFO:tasks.workunit.client.0.smithi046.stdout:2/928: rename d2/d8/d37/dc2/d123 to d2/d6/d23/d5d/d77/d7b/d85/d8d/d143 0 2024-09-10T04:05:34.142 INFO:tasks.workunit.client.1.smithi070.stdout:1/883: rename d3/d90/ce8 to d3/d8/d117/c121 0 2024-09-10T04:05:34.143 INFO:tasks.workunit.client.0.smithi046.stdout:6/813: write f9 [371337,95876] 0 2024-09-10T04:05:34.145 INFO:tasks.workunit.client.0.smithi046.stdout:0/744: mknod d1/dc/d25/d33/d3/dd/d55/d77/da6/ce8 0 2024-09-10T04:05:34.146 INFO:tasks.workunit.client.0.smithi046.stdout:0/745: readlink d1/dc/d25/d33/d1a/d41/l50 0 2024-09-10T04:05:34.147 INFO:tasks.workunit.client.0.smithi046.stdout:8/916: sync 2024-09-10T04:05:34.147 INFO:tasks.workunit.client.1.smithi070.stdout:1/884: dwrite d3/d8/d13/d67/f10a [0,4194304] 0 2024-09-10T04:05:34.147 INFO:tasks.workunit.client.1.smithi070.stdout:6/987: creat d0/d3/d1f/de8/f144 x:0 0 0 2024-09-10T04:05:34.147 INFO:tasks.workunit.client.0.smithi046.stdout:0/746: chown d1/dc/d25/d33/cdd 6298483 1 2024-09-10T04:05:34.148 INFO:tasks.workunit.client.0.smithi046.stdout:0/747: chown d1/d13/d37/d75/d94/d9a/da4 187323934 1 2024-09-10T04:05:34.149 INFO:tasks.workunit.client.0.smithi046.stdout:3/895: dread - d12/d13/ff9 zero size 2024-09-10T04:05:34.151 INFO:tasks.workunit.client.0.smithi046.stdout:7/986: creat d5/d6/d55/f155 x:0 0 0 2024-09-10T04:05:34.154 INFO:tasks.workunit.client.0.smithi046.stdout:2/929: symlink d2/d8/d37/l144 0 2024-09-10T04:05:34.154 INFO:tasks.workunit.client.1.smithi070.stdout:3/962: dwrite db/d1c/d25/d28/fba [4194304,4194304] 0 2024-09-10T04:05:34.156 INFO:tasks.workunit.client.0.smithi046.stdout:6/814: mknod da/db/d11/d2b/d2f/d3b/c10c 0 2024-09-10T04:05:34.158 INFO:tasks.workunit.client.0.smithi046.stdout:6/815: chown da/db/d11/d2b/d5a 3 1 2024-09-10T04:05:34.158 INFO:tasks.workunit.client.0.smithi046.stdout:2/930: dread d2/d6/d23/ff0 [0,4194304] 0 2024-09-10T04:05:34.158 INFO:tasks.workunit.client.0.smithi046.stdout:8/917: creat d1/d29/f131 x:0 0 0 2024-09-10T04:05:34.161 INFO:tasks.workunit.client.0.smithi046.stdout:8/918: write d1/d29/d17/d1f/d46/d6a/d82/dc2/fe2 [252500,101309] 0 2024-09-10T04:05:34.161 INFO:tasks.workunit.client.1.smithi070.stdout:4/946: creat de/d13/d18/f140 x:0 0 0 2024-09-10T04:05:34.169 INFO:tasks.workunit.client.1.smithi070.stdout:3/963: creat db/d1c/d25/d28/d43/f14d x:0 0 0 2024-09-10T04:05:34.170 INFO:tasks.workunit.client.1.smithi070.stdout:3/964: dread - db/d1c/d25/d28/dd3/de5/d10f/db6/f137 zero size 2024-09-10T04:05:34.170 INFO:tasks.workunit.client.0.smithi046.stdout:7/987: mkdir d5/d6/db/dd/d4b/d156 0 2024-09-10T04:05:34.171 INFO:tasks.workunit.client.1.smithi070.stdout:2/913: sync 2024-09-10T04:05:34.172 INFO:tasks.workunit.client.0.smithi046.stdout:7/988: dread - d5/d9/d12/d72/d77/de0/d1f/f12b zero size 2024-09-10T04:05:34.177 INFO:tasks.workunit.client.1.smithi070.stdout:6/988: creat d0/d3/d18/d1b/d17/d3f/f145 x:0 0 0 2024-09-10T04:05:34.180 INFO:tasks.workunit.client.1.smithi070.stdout:3/965: truncate db/d1c/d25/ffa 5161915 0 2024-09-10T04:05:34.183 INFO:tasks.workunit.client.0.smithi046.stdout:0/748: mkdir d1/dc/de9 0 2024-09-10T04:05:34.188 INFO:tasks.workunit.client.1.smithi070.stdout:4/947: rename de/d13/d18/d26/ffd to de/d13/d5b/d7b/dd5/dff/f141 0 2024-09-10T04:05:34.192 INFO:tasks.workunit.client.1.smithi070.stdout:3/966: mknod db/d1c/d25/dd8/c14e 0 2024-09-10T04:05:34.193 INFO:tasks.workunit.client.0.smithi046.stdout:7/989: mkdir d5/d9/d12/dc5/d157 0 2024-09-10T04:05:34.194 INFO:tasks.workunit.client.1.smithi070.stdout:1/885: getdents d3/d8/d12/d2f 0 2024-09-10T04:05:34.197 INFO:tasks.workunit.client.1.smithi070.stdout:1/886: readlink d3/d8/d13/d3f/d6d/d81/lde 0 2024-09-10T04:05:34.197 INFO:tasks.workunit.client.0.smithi046.stdout:2/931: mknod d2/d6/c145 0 2024-09-10T04:05:34.199 INFO:tasks.workunit.client.1.smithi070.stdout:3/967: dwrite db/f11 [0,4194304] 0 2024-09-10T04:05:34.202 INFO:tasks.workunit.client.0.smithi046.stdout:8/919: rename d1/d29/d17/d1f/d46/d67/dc1/c81 to d1/d29/d19/db7/c132 0 2024-09-10T04:05:34.204 INFO:tasks.workunit.client.0.smithi046.stdout:6/816: dwrite da/db/d63/fa1 [0,4194304] 0 2024-09-10T04:05:34.208 INFO:tasks.workunit.client.0.smithi046.stdout:7/990: mkdir d5/d6/d32/d158 0 2024-09-10T04:05:34.209 INFO:tasks.workunit.client.0.smithi046.stdout:2/932: mknod d2/d8/d37/d10a/c146 0 2024-09-10T04:05:34.210 INFO:tasks.workunit.client.0.smithi046.stdout:2/933: chown d2/d6/d23/d42/de6/dec 62448 1 2024-09-10T04:05:34.212 INFO:tasks.workunit.client.1.smithi070.stdout:2/914: dwrite d15/d18/d99/fe1 [0,4194304] 0 2024-09-10T04:05:34.214 INFO:tasks.workunit.client.1.smithi070.stdout:3/968: mknod db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/de1/c14f 0 2024-09-10T04:05:34.216 INFO:tasks.workunit.client.0.smithi046.stdout:6/817: unlink da/db/d10/f5d 0 2024-09-10T04:05:34.217 INFO:tasks.workunit.client.0.smithi046.stdout:7/991: dwrite d5/d6/d32/dab/db8/fa6 [0,4194304] 0 2024-09-10T04:05:34.219 INFO:tasks.workunit.client.1.smithi070.stdout:1/887: chown d3/d20/l77 1563 1 2024-09-10T04:05:34.225 INFO:tasks.workunit.client.1.smithi070.stdout:6/989: write d0/fbd [553636,31308] 0 2024-09-10T04:05:34.225 INFO:tasks.workunit.client.1.smithi070.stdout:6/990: dread - d0/d3/d18/df5/ffd zero size 2024-09-10T04:05:34.227 INFO:tasks.workunit.client.1.smithi070.stdout:6/991: dread d0/d3/d18/d62/d70/f105 [0,4194304] 0 2024-09-10T04:05:34.227 INFO:tasks.workunit.client.1.smithi070.stdout:3/969: creat db/d1c/d25/d28/dd3/de5/d10f/d94/f150 x:0 0 0 2024-09-10T04:05:34.233 INFO:tasks.workunit.client.1.smithi070.stdout:2/915: mknod d15/d18/c14a 0 2024-09-10T04:05:34.233 INFO:tasks.workunit.client.0.smithi046.stdout:3/896: write d12/d13/d14/d1e/f3f [5177983,127399] 0 2024-09-10T04:05:34.234 INFO:tasks.workunit.client.1.smithi070.stdout:4/948: write de/d13/d5b/d7b/dd5/d85/f133 [1679162,77814] 0 2024-09-10T04:05:34.246 INFO:tasks.workunit.client.0.smithi046.stdout:2/934: write d2/d6/d23/d42/de6/f110 [3845955,1118] 0 2024-09-10T04:05:34.248 INFO:tasks.workunit.client.0.smithi046.stdout:3/897: rename d12/d13/d37/d51/db5/d4b/d91/c125 to d12/d13/d14/d1e/d2d/d6f/d11f/c136 0 2024-09-10T04:05:34.248 INFO:tasks.workunit.client.0.smithi046.stdout:8/920: dwrite d1/d29/d17/d1f/d46/d6a/f94 [0,4194304] 0 2024-09-10T04:05:34.249 INFO:tasks.workunit.client.0.smithi046.stdout:8/921: chown d1/d29/d17/l24 2647950 1 2024-09-10T04:05:34.253 INFO:tasks.workunit.client.0.smithi046.stdout:2/935: symlink d2/d6/d23/d55/l147 0 2024-09-10T04:05:34.257 INFO:tasks.workunit.client.0.smithi046.stdout:6/818: dwrite da/db/d11/dd7/f52 [0,4194304] 0 2024-09-10T04:05:34.258 INFO:tasks.workunit.client.0.smithi046.stdout:7/992: dwrite d5/d6/db/dd/f2f [4194304,4194304] 0 2024-09-10T04:05:34.260 INFO:tasks.workunit.client.0.smithi046.stdout:3/898: mkdir d12/d13/d37/d51/db5/d50/d137 0 2024-09-10T04:05:34.263 INFO:tasks.workunit.client.0.smithi046.stdout:2/936: chown d2/d6/d15/caa 3021 1 2024-09-10T04:05:34.265 INFO:tasks.workunit.client.0.smithi046.stdout:8/922: creat d1/d29/d17/d1f/d2a/df6/d102/d12e/f133 x:0 0 0 2024-09-10T04:05:34.270 INFO:tasks.workunit.client.0.smithi046.stdout:6/819: rmdir da/db/d11/d2b/d5a/dc2 39 2024-09-10T04:05:34.273 INFO:tasks.workunit.client.0.smithi046.stdout:3/899: unlink d12/d13/d37/d94/fa2 0 2024-09-10T04:05:34.275 INFO:tasks.workunit.client.0.smithi046.stdout:6/820: dread da/db/d63/fa1 [0,4194304] 0 2024-09-10T04:05:34.275 INFO:tasks.workunit.client.0.smithi046.stdout:2/937: mknod d2/d8/d37/c148 0 2024-09-10T04:05:34.278 INFO:tasks.workunit.client.0.smithi046.stdout:8/923: read d1/f79 [260777,23220] 0 2024-09-10T04:05:34.279 INFO:tasks.workunit.client.0.smithi046.stdout:7/993: mkdir d5/d7f/d159 0 2024-09-10T04:05:34.283 INFO:tasks.workunit.client.0.smithi046.stdout:2/938: dwrite d2/d6/d9d/db6/dbe/fde [0,4194304] 0 2024-09-10T04:05:34.294 INFO:tasks.workunit.client.0.smithi046.stdout:8/924: mknod d1/d29/d19/d36/d4c/dd3/c134 0 2024-09-10T04:05:34.295 INFO:tasks.workunit.client.0.smithi046.stdout:8/925: fdatasync d1/d29/d17/d1f/f54 0 2024-09-10T04:05:34.297 INFO:tasks.workunit.client.0.smithi046.stdout:7/994: fsync d5/d6/db/dd/d4b/f52 0 2024-09-10T04:05:34.300 INFO:tasks.workunit.client.0.smithi046.stdout:2/939: mknod d2/d6/d23/d42/c149 0 2024-09-10T04:05:34.302 INFO:tasks.workunit.client.0.smithi046.stdout:6/821: truncate da/db/d11/d13/d1e/f7e 490230 0 2024-09-10T04:05:34.315 INFO:tasks.workunit.client.0.smithi046.stdout:7/995: dwrite d5/d6/db/dd/d24/d76/fe6 [0,4194304] 0 2024-09-10T04:05:34.315 INFO:tasks.workunit.client.0.smithi046.stdout:8/926: symlink d1/d29/d17/d1f/d46/d67/dc1/d124/l135 0 2024-09-10T04:05:34.316 INFO:tasks.workunit.client.1.smithi070.stdout:1/888: sync 2024-09-10T04:05:34.316 INFO:tasks.workunit.client.0.smithi046.stdout:7/996: chown d5/d6/d32/f74 4231870 1 2024-09-10T04:05:34.317 INFO:tasks.workunit.client.0.smithi046.stdout:8/927: fsync d1/d29/d17/d1f/d46/d67/dc1/f60 0 2024-09-10T04:05:34.321 INFO:tasks.workunit.client.0.smithi046.stdout:3/900: getdents d12/d13/d37/d51/db5/d4b/d91 0 2024-09-10T04:05:34.322 INFO:tasks.workunit.client.0.smithi046.stdout:6/822: mkdir da/db/d10d 0 2024-09-10T04:05:34.323 INFO:tasks.workunit.client.0.smithi046.stdout:6/823: write f6 [7150317,121878] 0 2024-09-10T04:05:34.328 INFO:tasks.workunit.client.0.smithi046.stdout:7/997: creat d5/d9/d12/d72/d77/de0/d1f/d116/f15a x:0 0 0 2024-09-10T04:05:34.330 INFO:tasks.workunit.client.0.smithi046.stdout:7/998: read d5/d6/d32/f95 [2489542,65927] 0 2024-09-10T04:05:34.333 INFO:tasks.workunit.client.0.smithi046.stdout:3/901: symlink d12/d13/d37/d51/db5/d50/d7a/dc1/l138 0 2024-09-10T04:05:34.334 INFO:tasks.workunit.client.0.smithi046.stdout:6/824: rmdir da/d71 39 2024-09-10T04:05:34.336 INFO:tasks.workunit.client.0.smithi046.stdout:7/999: unlink d5/d6/d55/d7d/ff1 0 2024-09-10T04:05:34.337 INFO:tasks.workunit.client.0.smithi046.stdout:8/928: creat d1/d29/d19/d36/d4c/f136 x:0 0 0 2024-09-10T04:05:34.341 INFO:tasks.workunit.client.0.smithi046.stdout:3/902: dwrite d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fe4 [0,4194304] 0 2024-09-10T04:05:34.345 INFO:tasks.workunit.client.0.smithi046.stdout:6/825: getdents da/db/d11/d13/d1e/d23/d54/dad 0 2024-09-10T04:05:34.348 INFO:tasks.workunit.client.1.smithi070.stdout:3/970: creat db/d83/d92/d11b/f151 x:0 0 0 2024-09-10T04:05:34.349 INFO:tasks.workunit.client.1.smithi070.stdout:2/916: rename d15/d2b/d106 to d15/df6/d14b 0 2024-09-10T04:05:34.353 INFO:tasks.workunit.client.1.smithi070.stdout:1/889: write d3/d8/d12/d2f/f7e [4955343,85893] 0 2024-09-10T04:05:34.354 INFO:tasks.workunit.client.1.smithi070.stdout:3/971: truncate db/d83/fce 409409 0 2024-09-10T04:05:34.355 INFO:tasks.workunit.client.0.smithi046.stdout:3/903: dread d12/d13/d14/d1e/d2d/f88 [0,4194304] 0 2024-09-10T04:05:34.356 INFO:tasks.workunit.client.1.smithi070.stdout:6/992: rename d0/d3/d5c/cd5 to d0/d3/d18/d1b/d17/d54/dc4/dd2/dde/c146 0 2024-09-10T04:05:34.356 INFO:tasks.workunit.client.0.smithi046.stdout:3/904: fdatasync d12/d5d/d71/d49/d5c/fe9 0 2024-09-10T04:05:34.360 INFO:tasks.workunit.client.0.smithi046.stdout:3/905: mkdir d12/d13/d14/d1e/d2d/d6f/d11f/d139 0 2024-09-10T04:05:34.361 INFO:tasks.workunit.client.1.smithi070.stdout:4/949: mknod de/d13/d18/d23/d47/d59/d106/c142 0 2024-09-10T04:05:34.362 INFO:tasks.workunit.client.1.smithi070.stdout:2/917: mknod d15/d18/d56/d7c/d10c/d145/c14c 0 2024-09-10T04:05:34.363 INFO:tasks.workunit.client.1.smithi070.stdout:3/972: dread db/d1c/f8c [0,4194304] 0 2024-09-10T04:05:34.364 INFO:tasks.workunit.client.1.smithi070.stdout:1/890: mknod d3/d10/dfa/c122 0 2024-09-10T04:05:34.365 INFO:tasks.workunit.client.1.smithi070.stdout:1/891: write d3/d10/db2/dc7/d54/fd7 [1663455,64306] 0 2024-09-10T04:05:34.367 INFO:tasks.workunit.client.1.smithi070.stdout:4/950: creat de/d13/d5b/d7b/dea/f143 x:0 0 0 2024-09-10T04:05:34.367 INFO:tasks.workunit.client.0.smithi046.stdout:3/906: dwrite d12/d13/d14/f33 [4194304,4194304] 0 2024-09-10T04:05:34.370 INFO:tasks.workunit.client.1.smithi070.stdout:3/973: mkdir db/d1c/d25/d28/dd3/de5/d12c/d152 0 2024-09-10T04:05:34.373 INFO:tasks.workunit.client.1.smithi070.stdout:4/951: dwrite de/d13/d5b/da0/f12d [0,4194304] 0 2024-09-10T04:05:34.375 INFO:tasks.workunit.client.1.smithi070.stdout:1/892: rename d3/d8/d13/f14 to d3/d90/dcb/f123 0 2024-09-10T04:05:34.375 INFO:tasks.workunit.client.1.smithi070.stdout:4/952: chown de/d13/d5b/d7b/dea/l10a 481 1 2024-09-10T04:05:34.382 INFO:tasks.workunit.client.0.smithi046.stdout:3/907: getdents d12/d13/d37/d51/db5/d50 0 2024-09-10T04:05:34.384 INFO:tasks.workunit.client.1.smithi070.stdout:4/953: dwrite de/d13/d18/f3b [0,4194304] 0 2024-09-10T04:05:34.385 INFO:tasks.workunit.client.1.smithi070.stdout:4/954: chown de/d13/d18/d23/d47/d59/d79/f66 13526493 1 2024-09-10T04:05:34.388 INFO:tasks.workunit.client.1.smithi070.stdout:4/955: write de/d13/d5b/d7b/dd5/d85/d10e/d48/f119 [252555,105955] 0 2024-09-10T04:05:34.389 INFO:tasks.workunit.client.1.smithi070.stdout:1/893: link d3/d8/d13/f25 d3/d10/db2/dee/f124 0 2024-09-10T04:05:34.390 INFO:tasks.workunit.client.1.smithi070.stdout:1/894: chown d3/d8/d12/d2f/ld1 5498 1 2024-09-10T04:05:34.391 INFO:tasks.workunit.client.1.smithi070.stdout:1/895: write d3/d8/d12/d2f/f5b [2842820,92592] 0 2024-09-10T04:05:34.400 INFO:tasks.workunit.client.1.smithi070.stdout:4/956: write de/d13/d18/d23/d47/fa7 [606726,22556] 0 2024-09-10T04:05:34.401 INFO:tasks.workunit.client.1.smithi070.stdout:1/896: write d3/d8/d13/ff1 [983380,29183] 0 2024-09-10T04:05:34.401 INFO:tasks.workunit.client.0.smithi046.stdout:3/908: dread d12/d13/d14/f36 [0,4194304] 0 2024-09-10T04:05:34.403 INFO:tasks.workunit.client.0.smithi046.stdout:3/909: rmdir d12/d13/d14/d1e/d2d/d39/d67/da0/d114 39 2024-09-10T04:05:34.404 INFO:tasks.workunit.client.1.smithi070.stdout:2/918: dwrite d15/d18/d29/f7f [4194304,4194304] 0 2024-09-10T04:05:34.406 INFO:tasks.workunit.client.1.smithi070.stdout:1/897: creat d3/d8/d12/d61/f125 x:0 0 0 2024-09-10T04:05:34.406 INFO:tasks.workunit.client.1.smithi070.stdout:3/974: dwrite db/dcc/ff3 [0,4194304] 0 2024-09-10T04:05:34.411 INFO:tasks.workunit.client.1.smithi070.stdout:3/975: creat db/d1c/d25/d28/dd3/de5/d10f/db5/f153 x:0 0 0 2024-09-10T04:05:34.412 INFO:tasks.workunit.client.1.smithi070.stdout:4/957: dread de/d13/d18/d23/d47/d59/d79/f66 [0,4194304] 0 2024-09-10T04:05:34.413 INFO:tasks.workunit.client.1.smithi070.stdout:2/919: rename d15/d18/d29/d2a/d2e/d32/d36/f149 to d15/d18/d29/d2a/d2e/da6/f14d 0 2024-09-10T04:05:34.418 INFO:tasks.workunit.client.1.smithi070.stdout:4/958: truncate de/d13/d5b/d7b/dd5/d85/d10e/f67 12783 0 2024-09-10T04:05:34.418 INFO:tasks.workunit.client.0.smithi046.stdout:3/910: dread d12/d13/d14/f15 [0,4194304] 0 2024-09-10T04:05:34.420 INFO:tasks.workunit.client.1.smithi070.stdout:3/976: link db/d1c/d25/d28/dd3/de5/d10f/fa9 db/d1c/d25/d28/dd3/de5/d10f/d94/d101/f154 0 2024-09-10T04:05:34.420 INFO:tasks.workunit.client.1.smithi070.stdout:4/959: fdatasync de/d13/d18/d23/d47/d59/d79/fb3 0 2024-09-10T04:05:34.422 INFO:tasks.workunit.client.0.smithi046.stdout:3/911: creat d12/d13/d37/d51/db5/d50/d52/f13a x:0 0 0 2024-09-10T04:05:34.423 INFO:tasks.workunit.client.0.smithi046.stdout:3/912: chown d12/d13/d14/d1e/d2d/c111 2360 1 2024-09-10T04:05:34.425 INFO:tasks.workunit.client.1.smithi070.stdout:4/960: mknod de/d13/d43/d88/c144 0 2024-09-10T04:05:34.426 INFO:tasks.workunit.client.0.smithi046.stdout:3/913: creat d12/d13/d37/d51/da7/f13b x:0 0 0 2024-09-10T04:05:34.428 INFO:tasks.workunit.client.0.smithi046.stdout:3/914: creat d12/d13/d37/dab/f13c x:0 0 0 2024-09-10T04:05:34.428 INFO:tasks.workunit.client.1.smithi070.stdout:2/920: dread d15/d18/d56/d7c/d55/f141 [0,4194304] 0 2024-09-10T04:05:34.431 INFO:tasks.workunit.client.1.smithi070.stdout:2/921: mkdir d15/d59/d14e 0 2024-09-10T04:05:34.432 INFO:tasks.workunit.client.0.smithi046.stdout:3/915: mkdir d12/d13/d37/d51/db5/d4b/db7/d13d 0 2024-09-10T04:05:34.434 INFO:tasks.workunit.client.1.smithi070.stdout:1/898: dwrite d3/d8/d27/f68 [0,4194304] 0 2024-09-10T04:05:34.435 INFO:tasks.workunit.client.1.smithi070.stdout:1/899: readlink d3/d10/d1d/l87 0 2024-09-10T04:05:34.436 INFO:tasks.workunit.client.0.smithi046.stdout:3/916: mkdir d12/d13/d14/d1e/d2d/d6f/d11f/d13e 0 2024-09-10T04:05:34.437 INFO:tasks.workunit.client.1.smithi070.stdout:3/977: write db/d1c/d25/f75 [5754360,61882] 0 2024-09-10T04:05:34.438 INFO:tasks.workunit.client.1.smithi070.stdout:4/961: dread de/d13/d5b/d7b/dd5/d85/d10e/d48/fab [0,4194304] 0 2024-09-10T04:05:34.439 INFO:tasks.workunit.client.1.smithi070.stdout:1/900: fsync d3/d8/d13/faf 0 2024-09-10T04:05:34.441 INFO:tasks.workunit.client.1.smithi070.stdout:3/978: mknod db/d1c/d3e/c155 0 2024-09-10T04:05:34.442 INFO:tasks.workunit.client.0.smithi046.stdout:3/917: creat d12/d13/d14/d1e/d2d/d6f/d11f/d13e/f13f x:0 0 0 2024-09-10T04:05:34.444 INFO:tasks.workunit.client.1.smithi070.stdout:2/922: write d15/d18/d56/d7c/fe7 [1396149,111372] 0 2024-09-10T04:05:34.446 INFO:tasks.workunit.client.1.smithi070.stdout:4/962: creat de/d13/d18/d23/d47/d59/d79/d6a/dce/f145 x:0 0 0 2024-09-10T04:05:34.449 INFO:tasks.workunit.client.1.smithi070.stdout:1/901: creat d3/d10/d115/f126 x:0 0 0 2024-09-10T04:05:34.452 INFO:tasks.workunit.client.1.smithi070.stdout:4/963: unlink de/d13/d5b/d7b/dd5/d85/d10e/c54 0 2024-09-10T04:05:34.454 INFO:tasks.workunit.client.0.smithi046.stdout:3/918: write d12/d13/fc7 [691941,63468] 0 2024-09-10T04:05:34.455 INFO:tasks.workunit.client.1.smithi070.stdout:1/902: creat d3/d8/d117/d94/da5/f127 x:0 0 0 2024-09-10T04:05:34.456 INFO:tasks.workunit.client.1.smithi070.stdout:4/964: dread - de/d13/d18/d26/d8c/f90 zero size 2024-09-10T04:05:34.458 INFO:tasks.workunit.client.1.smithi070.stdout:3/979: dwrite db/fea [0,4194304] 0 2024-09-10T04:05:34.459 INFO:tasks.workunit.client.1.smithi070.stdout:2/923: truncate f10 180759 0 2024-09-10T04:05:34.461 INFO:tasks.workunit.client.1.smithi070.stdout:1/903: rename d3/d8/d12/d61/fc0 to d3/d8/d13/d3f/d6d/f128 0 2024-09-10T04:05:34.462 INFO:tasks.workunit.client.1.smithi070.stdout:2/924: dread d15/d18/d29/d2a/d2e/d32/fc9 [0,4194304] 0 2024-09-10T04:05:34.463 INFO:tasks.workunit.client.1.smithi070.stdout:3/980: creat db/d1c/d25/d28/dd3/de5/d10f/d110/f156 x:0 0 0 2024-09-10T04:05:34.463 INFO:tasks.workunit.client.1.smithi070.stdout:3/981: stat db/d1c/d25/c36 0 2024-09-10T04:05:34.464 INFO:tasks.workunit.client.1.smithi070.stdout:4/965: symlink de/d13/d18/d23/l146 0 2024-09-10T04:05:34.464 INFO:tasks.workunit.client.0.smithi046.stdout:3/919: write d12/d13/d37/d51/db5/d4b/fb6 [573163,92620] 0 2024-09-10T04:05:34.466 INFO:tasks.workunit.client.0.smithi046.stdout:3/920: stat d12/d13/d37/d51/db5/d50/d7a/dc1/f97 0 2024-09-10T04:05:34.467 INFO:tasks.workunit.client.1.smithi070.stdout:1/904: rmdir d3/d8/d27/d9a 39 2024-09-10T04:05:34.468 INFO:tasks.workunit.client.0.smithi046.stdout:3/921: dread - d12/d13/d37/d51/db5/d50/d7a/dc1/d96/ff0 zero size 2024-09-10T04:05:34.469 INFO:tasks.workunit.client.0.smithi046.stdout:3/922: dread - d12/d13/d37/d51/dbd/ffe zero size 2024-09-10T04:05:34.470 INFO:tasks.workunit.client.0.smithi046.stdout:3/923: chown d12/d13/d14/d1e 124013693 1 2024-09-10T04:05:34.472 INFO:tasks.workunit.client.0.smithi046.stdout:3/924: write d12/d13/d37/d51/db5/d50/d52/f65 [9212132,78881] 0 2024-09-10T04:05:34.472 INFO:tasks.workunit.client.1.smithi070.stdout:4/966: truncate de/d13/d18/d23/d47/f51 4005372 0 2024-09-10T04:05:34.473 INFO:tasks.workunit.client.0.smithi046.stdout:3/925: fdatasync d12/d13/d14/d1e/dd2/fe6 0 2024-09-10T04:05:34.474 INFO:tasks.workunit.client.1.smithi070.stdout:1/905: mknod d3/d8/d92/dec/c129 0 2024-09-10T04:05:34.480 INFO:tasks.workunit.client.1.smithi070.stdout:4/967: unlink de/d13/d43/d88/ff9 0 2024-09-10T04:05:34.480 INFO:tasks.workunit.client.1.smithi070.stdout:6/993: sync 2024-09-10T04:05:34.483 INFO:tasks.workunit.client.1.smithi070.stdout:3/982: dwrite db/d1c/d25/d28/dd3/de5/d10f/dd4/fd7 [0,4194304] 0 2024-09-10T04:05:34.483 INFO:tasks.workunit.client.1.smithi070.stdout:6/994: dread - d0/d3/d18/d1b/d17/d54/dc4/d125/f139 zero size 2024-09-10T04:05:34.487 INFO:tasks.workunit.client.1.smithi070.stdout:2/925: truncate d15/d18/d29/d2a/d2e/d32/f82 900496 0 2024-09-10T04:05:34.491 INFO:tasks.workunit.client.0.smithi046.stdout:3/926: link d12/d13/l3e d12/d13/d14/d1e/d2d/d6f/d11f/d139/l140 0 2024-09-10T04:05:34.491 INFO:tasks.workunit.client.1.smithi070.stdout:1/906: write d3/d8/d117/fa7 [1527342,6501] 0 2024-09-10T04:05:34.492 INFO:tasks.workunit.client.1.smithi070.stdout:3/983: mknod db/d83/d11a/c157 0 2024-09-10T04:05:34.493 INFO:tasks.workunit.client.1.smithi070.stdout:1/907: chown d3/d90/dcb/cff 184 1 2024-09-10T04:05:34.493 INFO:tasks.workunit.client.1.smithi070.stdout:3/984: chown db/l5b 59955 1 2024-09-10T04:05:34.494 INFO:tasks.workunit.client.1.smithi070.stdout:2/926: symlink d15/d18/db2/l14f 0 2024-09-10T04:05:34.496 INFO:tasks.workunit.client.0.smithi046.stdout:3/927: creat d12/d13/d37/d51/db5/d4b/db7/d13d/f141 x:0 0 0 2024-09-10T04:05:34.498 INFO:tasks.workunit.client.1.smithi070.stdout:4/968: write de/d13/d5b/da0/f84 [210827,30527] 0 2024-09-10T04:05:34.500 INFO:tasks.workunit.client.0.smithi046.stdout:9/882: sync 2024-09-10T04:05:34.505 INFO:tasks.workunit.client.1.smithi070.stdout:2/927: fdatasync f3 0 2024-09-10T04:05:34.506 INFO:tasks.workunit.client.0.smithi046.stdout:9/883: fdatasync d4/d3c/d4d/d5c/f82 0 2024-09-10T04:05:34.507 INFO:tasks.workunit.client.1.smithi070.stdout:4/969: symlink de/d13/d5b/d7b/dea/l147 0 2024-09-10T04:05:34.509 INFO:tasks.workunit.client.1.smithi070.stdout:6/995: link d0/d3/d18/d1b/d17/d53/fb4 d0/d3/d18/d1b/d17/d54/dc4/d125/d132/f147 0 2024-09-10T04:05:34.509 INFO:tasks.workunit.client.0.smithi046.stdout:3/928: write d12/d13/fd0 [603726,61154] 0 2024-09-10T04:05:34.511 INFO:tasks.workunit.client.1.smithi070.stdout:1/908: mknod d3/d20/c12a 0 2024-09-10T04:05:34.514 INFO:tasks.workunit.client.0.smithi046.stdout:3/929: creat d12/d5d/d71/db1/f142 x:0 0 0 2024-09-10T04:05:34.516 INFO:tasks.workunit.client.1.smithi070.stdout:2/928: rmdir d15/d18/d29/d2a/d2e/d32/d9b 39 2024-09-10T04:05:34.518 INFO:tasks.workunit.client.1.smithi070.stdout:4/970: creat de/d13/d18/d26/d111/f148 x:0 0 0 2024-09-10T04:05:34.518 INFO:tasks.workunit.client.1.smithi070.stdout:3/985: write db/d1c/f4b [2557569,69225] 0 2024-09-10T04:05:34.520 INFO:tasks.workunit.client.0.smithi046.stdout:3/930: creat d12/d5d/d71/d49/d5c/f143 x:0 0 0 2024-09-10T04:05:34.521 INFO:tasks.workunit.client.1.smithi070.stdout:6/996: symlink d0/d3/d18/d1b/d17/d54/l148 0 2024-09-10T04:05:34.522 INFO:tasks.workunit.client.0.smithi046.stdout:9/884: rename d4/d7/d1f/c47 to d4/c116 0 2024-09-10T04:05:34.522 INFO:tasks.workunit.client.0.smithi046.stdout:9/885: stat d4/d66 0 2024-09-10T04:05:34.522 INFO:tasks.workunit.client.1.smithi070.stdout:1/909: mkdir d3/dd3/d12b 0 2024-09-10T04:05:34.523 INFO:tasks.workunit.client.0.smithi046.stdout:9/886: dread - d4/d7/d61/fd7 zero size 2024-09-10T04:05:34.523 INFO:tasks.workunit.client.0.smithi046.stdout:3/931: mkdir d12/d13/d37/d94/d144 0 2024-09-10T04:05:34.527 INFO:tasks.workunit.client.0.smithi046.stdout:9/887: unlink d4/d66/c88 0 2024-09-10T04:05:34.529 INFO:tasks.workunit.client.1.smithi070.stdout:3/986: readlink db/d1c/l98 0 2024-09-10T04:05:34.533 INFO:tasks.workunit.client.1.smithi070.stdout:1/910: creat d3/d10/d1d/dbd/da2/f12c x:0 0 0 2024-09-10T04:05:34.535 INFO:tasks.workunit.client.1.smithi070.stdout:3/987: fdatasync db/dcc/f11d 0 2024-09-10T04:05:34.537 INFO:tasks.workunit.client.1.smithi070.stdout:2/929: write d15/d18/d56/d7c/d55/f11a [1645451,18213] 0 2024-09-10T04:05:34.537 INFO:tasks.workunit.client.1.smithi070.stdout:1/911: symlink d3/d90/l12d 0 2024-09-10T04:05:34.538 INFO:tasks.workunit.client.1.smithi070.stdout:4/971: creat de/d13/d18/f149 x:0 0 0 2024-09-10T04:05:34.538 INFO:tasks.workunit.client.0.smithi046.stdout:3/932: write d12/d13/d37/d51/db5/d50/d7a/dc1/fbb [793972,118503] 0 2024-09-10T04:05:34.542 INFO:tasks.workunit.client.1.smithi070.stdout:6/997: creat d0/d3/d18/d1b/d17/f149 x:0 0 0 2024-09-10T04:05:34.543 INFO:tasks.workunit.client.1.smithi070.stdout:2/930: readlink d15/d2b/d6e/ld3 0 2024-09-10T04:05:34.543 INFO:tasks.workunit.client.0.smithi046.stdout:3/933: symlink d12/d13/d37/d51/db5/d4b/db7/l145 0 2024-09-10T04:05:34.544 INFO:tasks.workunit.client.0.smithi046.stdout:9/888: dwrite d4/d18/f4f [0,4194304] 0 2024-09-10T04:05:34.546 INFO:tasks.workunit.client.1.smithi070.stdout:3/988: truncate db/f4e 866755 0 2024-09-10T04:05:34.547 INFO:tasks.workunit.client.0.smithi046.stdout:3/934: symlink d12/d13/d37/d51/l146 0 2024-09-10T04:05:34.549 INFO:tasks.workunit.client.1.smithi070.stdout:2/931: truncate d15/d18/d56/fa4 983160 0 2024-09-10T04:05:34.550 INFO:tasks.workunit.client.0.smithi046.stdout:9/889: read d4/f99 [337340,68986] 0 2024-09-10T04:05:34.550 INFO:tasks.workunit.client.0.smithi046.stdout:3/935: read - d12/d13/d14/f120 zero size 2024-09-10T04:05:34.552 INFO:tasks.workunit.client.0.smithi046.stdout:3/936: dread d12/d5d/d71/db1/ff1 [0,4194304] 0 2024-09-10T04:05:34.552 INFO:tasks.workunit.client.1.smithi070.stdout:2/932: fdatasync d15/d18/d29/fcd 0 2024-09-10T04:05:34.554 INFO:tasks.workunit.client.0.smithi046.stdout:9/890: symlink d4/d66/da6/l117 0 2024-09-10T04:05:34.555 INFO:tasks.workunit.client.0.smithi046.stdout:9/891: dread - d4/d3c/d4d/d5c/fc6 zero size 2024-09-10T04:05:34.556 INFO:tasks.workunit.client.1.smithi070.stdout:1/912: write d3/d8/f49 [2657561,34415] 0 2024-09-10T04:05:34.557 INFO:tasks.workunit.client.0.smithi046.stdout:3/937: readlink d12/d13/l38 0 2024-09-10T04:05:34.562 INFO:tasks.workunit.client.1.smithi070.stdout:3/989: rename db/d83/d92/f96 to db/d1c/d25/d28/f158 0 2024-09-10T04:05:34.562 INFO:tasks.workunit.client.1.smithi070.stdout:6/998: write d0/d3/d18/d1b/d17/d3f/d40/f48 [2558274,127532] 0 2024-09-10T04:05:34.562 INFO:tasks.workunit.client.0.smithi046.stdout:9/892: read d4/d18/f25 [1852952,59380] 0 2024-09-10T04:05:34.565 INFO:tasks.workunit.client.1.smithi070.stdout:4/972: dwrite de/d13/f3e [0,4194304] 0 2024-09-10T04:05:34.565 INFO:tasks.workunit.client.1.smithi070.stdout:3/990: stat db/d1c/d25/d28/d43/c65 0 2024-09-10T04:05:34.567 INFO:tasks.workunit.client.1.smithi070.stdout:4/973: chown de/d13/d5b/d7b/dd5/d85/d10e/d77/fb1 0 1 2024-09-10T04:05:34.568 INFO:tasks.workunit.client.1.smithi070.stdout:6/999: creat d0/d3/d18/d62/d70/f14a x:0 0 0 2024-09-10T04:05:34.573 INFO:tasks.workunit.client.1.smithi070.stdout:4/974: dread de/d13/d5b/d7b/dd5/d85/d10e/f57 [0,4194304] 0 2024-09-10T04:05:34.574 INFO:tasks.workunit.client.1.smithi070.stdout:3/991: symlink db/l159 0 2024-09-10T04:05:34.576 INFO:tasks.workunit.client.1.smithi070.stdout:2/933: dread d15/d18/d29/d2a/d2e/d32/fb5 [0,4194304] 0 2024-09-10T04:05:34.577 INFO:tasks.workunit.client.1.smithi070.stdout:4/975: mkdir de/d13/d5b/d7b/dd5/dff/d14a 0 2024-09-10T04:05:34.577 INFO:tasks.workunit.client.1.smithi070.stdout:1/913: write d3/d8/d13/d3f/f69 [1411907,101356] 0 2024-09-10T04:05:34.580 INFO:tasks.workunit.client.0.smithi046.stdout:9/893: truncate d4/d7/d1f/d54/ff9 470803 0 2024-09-10T04:05:34.580 INFO:tasks.workunit.client.0.smithi046.stdout:3/938: dwrite d12/d13/d14/d1e/d2d/d39/f87 [0,4194304] 0 2024-09-10T04:05:34.581 INFO:tasks.workunit.client.1.smithi070.stdout:3/992: mkdir db/d15a 0 2024-09-10T04:05:34.581 INFO:tasks.workunit.client.1.smithi070.stdout:2/934: dread d15/d18/d56/d7c/dfa/f10f [0,4194304] 0 2024-09-10T04:05:34.585 INFO:tasks.workunit.client.0.smithi046.stdout:3/939: dread d12/d13/d14/d1e/d2d/d39/f87 [0,4194304] 0 2024-09-10T04:05:34.586 INFO:tasks.workunit.client.0.smithi046.stdout:3/940: write d12/d13/d14/d1e/d2d/d39/d67/f6b [2714406,122429] 0 2024-09-10T04:05:34.587 INFO:tasks.workunit.client.1.smithi070.stdout:4/976: mkdir de/d13/d43/d88/db5/d139/d14b 0 2024-09-10T04:05:34.588 INFO:tasks.workunit.client.0.smithi046.stdout:9/894: creat d4/d18/d89/ddb/dee/f118 x:0 0 0 2024-09-10T04:05:34.590 INFO:tasks.workunit.client.1.smithi070.stdout:2/935: rmdir d15/df6/d14b 39 2024-09-10T04:05:34.594 INFO:tasks.workunit.client.0.smithi046.stdout:3/941: creat d12/d13/d37/d94/f147 x:0 0 0 2024-09-10T04:05:34.598 INFO:tasks.workunit.client.1.smithi070.stdout:4/977: creat de/d13/d5b/d7b/dd5/d85/f14c x:0 0 0 2024-09-10T04:05:34.600 INFO:tasks.workunit.client.1.smithi070.stdout:2/936: unlink d15/d18/d29/d64/f81 0 2024-09-10T04:05:34.601 INFO:tasks.workunit.client.0.smithi046.stdout:3/942: mknod d12/d13/d37/d51/c148 0 2024-09-10T04:05:34.603 INFO:tasks.workunit.client.0.smithi046.stdout:3/943: chown d12/d13/d37/d51/db5/d50/lf6 206 1 2024-09-10T04:05:34.603 INFO:tasks.workunit.client.1.smithi070.stdout:1/914: creat d3/d20/f12e x:0 0 0 2024-09-10T04:05:34.607 INFO:tasks.workunit.client.1.smithi070.stdout:2/937: creat d15/d18/d56/d7c/d10c/f150 x:0 0 0 2024-09-10T04:05:34.607 INFO:tasks.workunit.client.0.smithi046.stdout:9/895: write d4/f99 [1134122,68876] 0 2024-09-10T04:05:34.608 INFO:tasks.workunit.client.1.smithi070.stdout:1/915: mkdir d3/d90/dcb/d12f 0 2024-09-10T04:05:34.609 INFO:tasks.workunit.client.1.smithi070.stdout:3/993: dwrite db/d1c/d25/d28/dd3/de5/d10f/db6/dc3/ff0 [0,4194304] 0 2024-09-10T04:05:34.611 INFO:tasks.workunit.client.1.smithi070.stdout:4/978: link de/d13/d18/d23/d47/d59/d79/cbd de/d13/d43/c14d 0 2024-09-10T04:05:34.611 INFO:tasks.workunit.client.0.smithi046.stdout:3/944: dwrite d12/d13/d14/f108 [0,4194304] 0 2024-09-10T04:05:34.612 INFO:tasks.workunit.client.0.smithi046.stdout:9/896: mknod d4/d44/d108/c119 0 2024-09-10T04:05:34.612 INFO:tasks.workunit.client.1.smithi070.stdout:4/979: chown de/d13/d5b/da0/f12d 905 1 2024-09-10T04:05:34.616 INFO:tasks.workunit.client.0.smithi046.stdout:9/897: symlink d4/d44/l11a 0 2024-09-10T04:05:34.619 INFO:tasks.workunit.client.1.smithi070.stdout:1/916: creat d3/d8/d27/d9a/f130 x:0 0 0 2024-09-10T04:05:34.620 INFO:tasks.workunit.client.0.smithi046.stdout:3/945: unlink d12/d13/d14/d1e/d2d/d39/ld5 0 2024-09-10T04:05:34.622 INFO:tasks.workunit.client.0.smithi046.stdout:3/946: chown d12/d13/d14/d1e/d2d/d39/d67/f11c 22133809 1 2024-09-10T04:05:34.622 INFO:tasks.workunit.client.1.smithi070.stdout:2/938: rename d15/d18/d56/d7c/d10c to d15/d18/d56/d7c/d55/d151 0 2024-09-10T04:05:34.628 INFO:tasks.workunit.client.0.smithi046.stdout:1/717: sync 2024-09-10T04:05:34.628 INFO:tasks.workunit.client.1.smithi070.stdout:1/917: dread d3/d8/f5c [0,4194304] 0 2024-09-10T04:05:34.629 INFO:tasks.workunit.client.1.smithi070.stdout:1/918: chown d3/d8/d117/df4/ffe 0 1 2024-09-10T04:05:34.633 INFO:tasks.workunit.client.0.smithi046.stdout:3/947: truncate d12/d5d/d71/f5b 1312041 0 2024-09-10T04:05:34.636 INFO:tasks.workunit.client.0.smithi046.stdout:9/898: creat d4/d3c/d4d/d5c/d87/f11b x:0 0 0 2024-09-10T04:05:34.640 INFO:tasks.workunit.client.1.smithi070.stdout:1/919: rename d3/d8/d27/fac to d3/d10/db2/dc7/f131 0 2024-09-10T04:05:34.641 INFO:tasks.workunit.client.0.smithi046.stdout:3/948: dwrite d12/d13/d37/dab/f13c [0,4194304] 0 2024-09-10T04:05:34.641 INFO:tasks.workunit.client.1.smithi070.stdout:3/994: dwrite db/d1c/d53/fd0 [0,4194304] 0 2024-09-10T04:05:34.644 INFO:tasks.workunit.client.0.smithi046.stdout:1/718: mkdir d3/da1/d1f/df4 0 2024-09-10T04:05:34.647 INFO:tasks.workunit.client.1.smithi070.stdout:1/920: dread - d3/d8/d13/d3f/d6d/f128 zero size 2024-09-10T04:05:34.648 INFO:tasks.workunit.client.1.smithi070.stdout:3/995: dwrite db/d83/d92/d11b/f128 [0,4194304] 0 2024-09-10T04:05:34.653 INFO:tasks.workunit.client.0.smithi046.stdout:1/719: dwrite d3/dc/d34/d58/fd5 [0,4194304] 0 2024-09-10T04:05:34.655 INFO:tasks.workunit.client.0.smithi046.stdout:1/720: fdatasync d3/d27/d5d/fad 0 2024-09-10T04:05:34.655 INFO:tasks.workunit.client.1.smithi070.stdout:4/980: write de/d13/d18/d5c/f96 [1266432,58136] 0 2024-09-10T04:05:34.658 INFO:tasks.workunit.client.1.smithi070.stdout:2/939: creat d15/d18/d29/d2a/d2e/da6/dc0/f152 x:0 0 0 2024-09-10T04:05:34.658 INFO:tasks.workunit.client.1.smithi070.stdout:4/981: write de/d13/d5b/d7b/dd5/d85/d10e/d48/f119 [1125685,45020] 0 2024-09-10T04:05:34.659 INFO:tasks.workunit.client.0.smithi046.stdout:1/721: write d3/d9b/fbc [519246,123557] 0 2024-09-10T04:05:34.660 INFO:tasks.workunit.client.1.smithi070.stdout:4/982: readlink de/d13/d18/d26/l11e 0 2024-09-10T04:05:34.662 INFO:tasks.workunit.client.1.smithi070.stdout:4/983: rmdir de/d13/d18 39 2024-09-10T04:05:34.665 INFO:tasks.workunit.client.1.smithi070.stdout:3/996: getdents db/d1c/d25/d28/dd3/de5/d10f/db5 0 2024-09-10T04:05:34.666 INFO:tasks.workunit.client.1.smithi070.stdout:4/984: dread - de/d13/d18/d26/f134 zero size 2024-09-10T04:05:34.668 INFO:tasks.workunit.client.0.smithi046.stdout:9/899: write d4/d7/d1f/f15 [508561,17444] 0 2024-09-10T04:05:34.669 INFO:tasks.workunit.client.0.smithi046.stdout:1/722: dwrite d3/dc/d21/d3a/fc7 [0,4194304] 0 2024-09-10T04:05:34.677 INFO:tasks.workunit.client.0.smithi046.stdout:1/723: rename d3/d27/d2a/fd0 to d3/d27/d5d/d60/dd7/ff5 0 2024-09-10T04:05:34.677 INFO:tasks.workunit.client.0.smithi046.stdout:9/900: read d4/d29/d65/f75 [3855237,96636] 0 2024-09-10T04:05:34.678 INFO:tasks.workunit.client.1.smithi070.stdout:1/921: write d3/d8/d12/d2f/d32/f58 [4347098,39051] 0 2024-09-10T04:05:34.678 INFO:tasks.workunit.client.0.smithi046.stdout:3/949: dwrite d12/d5d/fcd [0,4194304] 0 2024-09-10T04:05:34.680 INFO:tasks.workunit.client.0.smithi046.stdout:9/901: read - d4/d18/d89/f102 zero size 2024-09-10T04:05:34.680 INFO:tasks.workunit.client.0.smithi046.stdout:1/724: dread - d3/dc/fec zero size 2024-09-10T04:05:34.683 INFO:tasks.workunit.client.0.smithi046.stdout:9/902: chown d4/d7/d1f/d54/dc7/dc8 473774 1 2024-09-10T04:05:34.686 INFO:tasks.workunit.client.0.smithi046.stdout:1/725: dread d3/da1/d1f/f55 [0,4194304] 0 2024-09-10T04:05:34.686 INFO:tasks.workunit.client.1.smithi070.stdout:4/985: write de/d13/d18/d5c/db6/fd4 [860666,110830] 0 2024-09-10T04:05:34.689 INFO:tasks.workunit.client.0.smithi046.stdout:9/903: rmdir d4/d3c/d4d/d5c 39 2024-09-10T04:05:34.689 INFO:tasks.workunit.client.1.smithi070.stdout:2/940: dwrite d15/d18/d29/d2a/d2e/f87 [0,4194304] 0 2024-09-10T04:05:34.690 INFO:tasks.workunit.client.0.smithi046.stdout:3/950: dwrite d12/d13/d37/d51/db5/d4b/db7/ff5 [0,4194304] 0 2024-09-10T04:05:34.692 INFO:tasks.workunit.client.0.smithi046.stdout:1/726: creat d3/d27/d5d/d60/dd7/ff6 x:0 0 0 2024-09-10T04:05:34.693 INFO:tasks.workunit.client.1.smithi070.stdout:3/997: dwrite db/f74 [4194304,4194304] 0 2024-09-10T04:05:34.694 INFO:tasks.workunit.client.0.smithi046.stdout:9/904: creat d4/d29/d65/f11c x:0 0 0 2024-09-10T04:05:34.695 INFO:tasks.workunit.client.0.smithi046.stdout:3/951: creat d12/d13/d14/d1e/d2d/d6f/d11f/d13e/f149 x:0 0 0 2024-09-10T04:05:34.699 INFO:tasks.workunit.client.0.smithi046.stdout:9/905: read d4/d3c/de0/f43 [2524330,61805] 0 2024-09-10T04:05:34.699 INFO:tasks.workunit.client.1.smithi070.stdout:3/998: creat db/d1c/d25/d28/dd3/f15b x:0 0 0 2024-09-10T04:05:34.705 INFO:tasks.workunit.client.1.smithi070.stdout:3/999: dwrite db/d1c/d25/f144 [0,4194304] 0 2024-09-10T04:05:34.789 INFO:tasks.workunit.client.0.smithi046.stdout:3/952: chown f7 2374064 1 2024-09-10T04:05:34.790 INFO:tasks.workunit.client.1.smithi070.stdout:4/986: symlink de/d13/d18/d23/d47/d59/d79/d6a/d137/l14e 0 2024-09-10T04:05:34.795 INFO:tasks.workunit.client.0.smithi046.stdout:9/906: creat d4/d3c/d4d/d80/f11d x:0 0 0 2024-09-10T04:05:34.798 INFO:tasks.workunit.client.0.smithi046.stdout:3/953: mknod d12/d13/d14/d1e/d2d/d6f/d11f/c14a 0 2024-09-10T04:05:34.807 INFO:tasks.workunit.client.0.smithi046.stdout:3/954: mkdir d12/d13/d37/d51/db5/d4b/d14b 0 2024-09-10T04:05:34.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:34 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:34.903 INFO:tasks.workunit.client.0.smithi046.stdout:1/727: link d3/dc/d21/d3a/d90/dbf/fe5 d3/dc/d21/d3a/ff7 0 2024-09-10T04:05:34.906 INFO:tasks.workunit.client.0.smithi046.stdout:1/728: mkdir d3/d62/df8 0 2024-09-10T04:05:34.908 INFO:tasks.workunit.client.0.smithi046.stdout:1/729: mkdir d3/d27/d5d/d60/dd7/df9 0 2024-09-10T04:05:34.911 INFO:tasks.workunit.client.1.smithi070.stdout:4/987: write de/d13/d5b/d7b/dd5/d85/d10e/d48/fab [2448905,9159] 0 2024-09-10T04:05:34.911 INFO:tasks.workunit.client.1.smithi070.stdout:4/988: chown de/d13/d18/d26/f2f 447282 1 2024-09-10T04:05:34.943 INFO:tasks.workunit.client.0.smithi046.stdout:9/907: rmdir d4/d7/d81 39 2024-09-10T04:05:34.943 INFO:tasks.workunit.client.0.smithi046.stdout:9/908: chown d4/d7/d1f/d54/dc7/dc8/cfe 298935 1 2024-09-10T04:05:34.951 INFO:tasks.workunit.client.1.smithi070.stdout:2/941: sync 2024-09-10T04:05:34.951 INFO:tasks.workunit.client.1.smithi070.stdout:1/922: sync 2024-09-10T04:05:34.952 INFO:tasks.workunit.client.1.smithi070.stdout:1/923: stat d3/d8/d117/d94/f118 0 2024-09-10T04:05:34.953 INFO:tasks.workunit.client.1.smithi070.stdout:1/924: chown d3/d8/d12/d2f/d32/f6b 5 1 2024-09-10T04:05:34.956 INFO:tasks.workunit.client.0.smithi046.stdout:9/909: creat d4/d3c/d4d/f11e x:0 0 0 2024-09-10T04:05:34.958 INFO:tasks.workunit.client.0.smithi046.stdout:9/910: creat d4/d7/d1f/dce/dd4/f11f x:0 0 0 2024-09-10T04:05:34.963 INFO:tasks.workunit.client.1.smithi070.stdout:2/942: dread d15/d2b/d6e/fd4 [0,4194304] 0 2024-09-10T04:05:34.982 INFO:tasks.workunit.client.0.smithi046.stdout:0/749: sync 2024-09-10T04:05:34.987 INFO:tasks.workunit.client.0.smithi046.stdout:0/750: chown d1/dc/d25/d33/f1b 494 1 2024-09-10T04:05:34.995 INFO:tasks.workunit.client.0.smithi046.stdout:0/751: mkdir d1/d13/d37/d75/d94/d9a/da4/dea 0 2024-09-10T04:05:35.022 INFO:tasks.workunit.client.1.smithi070.stdout:2/943: dwrite d15/d59/ff4 [0,4194304] 0 2024-09-10T04:05:35.024 INFO:tasks.workunit.client.1.smithi070.stdout:2/944: symlink d15/d2b/l153 0 2024-09-10T04:05:35.029 INFO:tasks.workunit.client.1.smithi070.stdout:2/945: getdents d15/d18 0 2024-09-10T04:05:35.032 INFO:tasks.workunit.client.1.smithi070.stdout:2/946: truncate d15/d18/d56/d7c/d55/f141 2784796 0 2024-09-10T04:05:35.034 INFO:tasks.workunit.client.1.smithi070.stdout:2/947: creat d15/d18/d56/d7c/d55/d151/f154 x:0 0 0 2024-09-10T04:05:35.038 INFO:tasks.workunit.client.1.smithi070.stdout:2/948: dwrite d15/d18/f26 [0,4194304] 0 2024-09-10T04:05:35.040 INFO:tasks.workunit.client.1.smithi070.stdout:2/949: fsync d15/d18/d29/d2a/d2e/d32/d9b/f12d 0 2024-09-10T04:05:35.042 INFO:tasks.workunit.client.1.smithi070.stdout:2/950: rmdir d15/d18/d56/d7c/dfa 39 2024-09-10T04:05:35.049 INFO:tasks.workunit.client.1.smithi070.stdout:2/951: getdents d15/d18/d56/d7c/d55/d6d 0 2024-09-10T04:05:35.050 INFO:tasks.workunit.client.1.smithi070.stdout:2/952: creat d15/d18/d29/d64/f155 x:0 0 0 2024-09-10T04:05:35.063 INFO:tasks.workunit.client.1.smithi070.stdout:2/953: dwrite d15/d18/d29/fcd [0,4194304] 0 2024-09-10T04:05:35.074 INFO:tasks.workunit.client.1.smithi070.stdout:2/954: dwrite d15/d18/d29/d2a/d2e/d32/fd7 [0,4194304] 0 2024-09-10T04:05:35.076 INFO:tasks.workunit.client.1.smithi070.stdout:2/955: rmdir d15/d18/d29/d2a/d2e 39 2024-09-10T04:05:35.077 INFO:tasks.workunit.client.1.smithi070.stdout:2/956: creat d15/d59/f156 x:0 0 0 2024-09-10T04:05:35.079 INFO:tasks.workunit.client.1.smithi070.stdout:2/957: stat d15/d18/d29/d2a/d2e/d32/f73 0 2024-09-10T04:05:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:34 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:35.162 INFO:tasks.workunit.client.1.smithi070.stdout:2/958: write d15/d18/d56/f86 [39144,17088] 0 2024-09-10T04:05:35.192 INFO:tasks.workunit.client.0.smithi046.stdout:2/940: sync 2024-09-10T04:05:35.192 INFO:tasks.workunit.client.0.smithi046.stdout:8/929: sync 2024-09-10T04:05:35.193 INFO:tasks.workunit.client.0.smithi046.stdout:8/930: write d1/d29/d17/d1f/d2a/f58 [97787,41094] 0 2024-09-10T04:05:35.194 INFO:tasks.workunit.client.0.smithi046.stdout:2/941: mkdir d2/d6/d14a 0 2024-09-10T04:05:35.197 INFO:tasks.workunit.client.0.smithi046.stdout:2/942: stat d2/d6/d23/d55/fa3 0 2024-09-10T04:05:35.205 INFO:tasks.workunit.client.0.smithi046.stdout:8/931: dread d1/d29/d19/d36/d4c/d5e/fa1 [0,4194304] 0 2024-09-10T04:05:35.206 INFO:tasks.workunit.client.0.smithi046.stdout:8/932: dread - d1/d29/d19/d36/d6e/f115 zero size 2024-09-10T04:05:35.207 INFO:tasks.workunit.client.0.smithi046.stdout:8/933: fdatasync d1/d29/d19/f56 0 2024-09-10T04:05:35.209 INFO:tasks.workunit.client.0.smithi046.stdout:8/934: mknod d1/c137 0 2024-09-10T04:05:35.211 INFO:tasks.workunit.client.0.smithi046.stdout:2/943: dread d2/d6/d23/d42/de6/fcf [0,4194304] 0 2024-09-10T04:05:35.212 INFO:tasks.workunit.client.0.smithi046.stdout:2/944: fsync d2/d6/d23/dd5/df8/f128 0 2024-09-10T04:05:35.213 INFO:tasks.workunit.client.0.smithi046.stdout:8/935: creat d1/d29/d19/f138 x:0 0 0 2024-09-10T04:05:35.215 INFO:tasks.workunit.client.0.smithi046.stdout:8/936: write d1/d29/d19/d36/d8d/f12c [1038445,75006] 0 2024-09-10T04:05:35.215 INFO:tasks.workunit.client.0.smithi046.stdout:2/945: chown d2/d6/d23/d5d/d77/l6e 875995 1 2024-09-10T04:05:35.215 INFO:tasks.workunit.client.0.smithi046.stdout:2/946: chown d2/f25 2068 1 2024-09-10T04:05:35.217 INFO:tasks.workunit.client.0.smithi046.stdout:2/947: dread d2/d6/d9d/f106 [0,4194304] 0 2024-09-10T04:05:35.218 INFO:tasks.workunit.client.0.smithi046.stdout:2/948: chown d2/d6/d9d/f106 965943 1 2024-09-10T04:05:35.219 INFO:tasks.workunit.client.0.smithi046.stdout:2/949: fsync d2/d6/d23/d5d/d77/d7b/d85/d8d/fc5 0 2024-09-10T04:05:35.220 INFO:tasks.workunit.client.0.smithi046.stdout:8/937: link d1/d29/d19/db7/c10a d1/d29/d17/d1f/d2a/df6/d117/c139 0 2024-09-10T04:05:35.222 INFO:tasks.workunit.client.0.smithi046.stdout:2/950: symlink d2/d8/d109/l14b 0 2024-09-10T04:05:35.223 INFO:tasks.workunit.client.0.smithi046.stdout:8/938: mkdir d1/d29/d19/d36/d4c/d5e/dc0/d13a 0 2024-09-10T04:05:35.225 INFO:tasks.workunit.client.0.smithi046.stdout:2/951: mkdir d2/d6/d23/d5d/d77/dac/d14c 0 2024-09-10T04:05:35.226 INFO:tasks.workunit.client.0.smithi046.stdout:2/952: mkdir d2/d86/d14d 0 2024-09-10T04:05:35.228 INFO:tasks.workunit.client.0.smithi046.stdout:2/953: read d2/d6/d23/d5d/fae [1407745,42149] 0 2024-09-10T04:05:35.229 INFO:tasks.workunit.client.0.smithi046.stdout:8/939: link d1/d29/d17/d1f/d46/d67/l118 d1/d29/d17/l13b 0 2024-09-10T04:05:35.232 INFO:tasks.workunit.client.0.smithi046.stdout:8/940: creat d1/d29/d19/d36/d4c/d5e/dc7/f13c x:0 0 0 2024-09-10T04:05:35.233 INFO:tasks.workunit.client.0.smithi046.stdout:8/941: dread - d1/d29/d19/d36/f128 zero size 2024-09-10T04:05:35.234 INFO:tasks.workunit.client.0.smithi046.stdout:2/954: rename d2/d6/d9d to d2/d6/d15/d14e 0 2024-09-10T04:05:35.241 INFO:tasks.workunit.client.1.smithi070.stdout:4/989: sync 2024-09-10T04:05:35.241 INFO:tasks.workunit.client.0.smithi046.stdout:8/942: creat d1/d29/d19/f13d x:0 0 0 2024-09-10T04:05:35.244 INFO:tasks.workunit.client.0.smithi046.stdout:2/955: getdents d2/d8 0 2024-09-10T04:05:35.245 INFO:tasks.workunit.client.0.smithi046.stdout:2/956: fdatasync d2/d8/d37/dc2/f137 0 2024-09-10T04:05:35.250 INFO:tasks.workunit.client.0.smithi046.stdout:2/957: rename d2/d49/c4c to d2/d8/d109/c14f 0 2024-09-10T04:05:35.251 INFO:tasks.workunit.client.0.smithi046.stdout:2/958: fsync d2/d8/d37/dc2/f116 0 2024-09-10T04:05:35.255 INFO:tasks.workunit.client.0.smithi046.stdout:2/959: creat d2/d6/d23/d42/dc8/f150 x:0 0 0 2024-09-10T04:05:35.256 INFO:tasks.workunit.client.0.smithi046.stdout:3/955: sync 2024-09-10T04:05:35.256 INFO:tasks.workunit.client.0.smithi046.stdout:6/826: sync 2024-09-10T04:05:35.256 INFO:tasks.workunit.client.0.smithi046.stdout:2/960: chown d2/d6/d23/d5d/d77/dac 10988 1 2024-09-10T04:05:35.258 INFO:tasks.workunit.client.0.smithi046.stdout:3/956: chown d12/d13/d14/d1e/d2d/d6f/d11f/c121 3535797 1 2024-09-10T04:05:35.259 INFO:tasks.workunit.client.0.smithi046.stdout:3/957: fdatasync d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/f10a 0 2024-09-10T04:05:35.260 INFO:tasks.workunit.client.0.smithi046.stdout:6/827: creat da/db/d10/d27/f10e x:0 0 0 2024-09-10T04:05:35.261 INFO:tasks.workunit.client.0.smithi046.stdout:6/828: fsync f6 0 2024-09-10T04:05:35.265 INFO:tasks.workunit.client.0.smithi046.stdout:2/961: dread d2/d6/f1a [0,4194304] 0 2024-09-10T04:05:35.269 INFO:tasks.workunit.client.0.smithi046.stdout:3/958: dread d12/d13/d37/d51/db5/d50/d7a/dc1/f93 [0,4194304] 0 2024-09-10T04:05:35.271 INFO:tasks.workunit.client.0.smithi046.stdout:2/962: dread d2/d6/d23/d5d/d77/d7b/d85/d8d/f108 [0,4194304] 0 2024-09-10T04:05:35.273 INFO:tasks.workunit.client.0.smithi046.stdout:2/963: stat d2/d8/f6f 0 2024-09-10T04:05:35.279 INFO:tasks.workunit.client.0.smithi046.stdout:3/959: creat d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/f14c x:0 0 0 2024-09-10T04:05:35.282 INFO:tasks.workunit.client.0.smithi046.stdout:2/964: unlink d2/d8/cd 0 2024-09-10T04:05:35.286 INFO:tasks.workunit.client.0.smithi046.stdout:6/829: rename da/db/d11/d13/d1e/lf2 to da/db/d18/l10f 0 2024-09-10T04:05:35.289 INFO:tasks.workunit.client.0.smithi046.stdout:2/965: creat d2/d86/d9c/dbc/ded/f151 x:0 0 0 2024-09-10T04:05:35.297 INFO:tasks.workunit.client.0.smithi046.stdout:6/830: dwrite da/d45/f107 [0,4194304] 0 2024-09-10T04:05:35.310 INFO:tasks.workunit.client.1.smithi070.stdout:1/925: sync 2024-09-10T04:05:35.310 INFO:tasks.workunit.client.1.smithi070.stdout:2/959: sync 2024-09-10T04:05:35.312 INFO:tasks.workunit.client.1.smithi070.stdout:4/990: symlink de/d13/d18/d26/d111/d12b/l14f 0 2024-09-10T04:05:35.313 INFO:tasks.workunit.client.1.smithi070.stdout:4/991: fdatasync de/d13/d5b/d7b/dd5/d85/d10e/d48/f119 0 2024-09-10T04:05:35.314 INFO:tasks.workunit.client.1.smithi070.stdout:4/992: chown de/d13/d18/l13d 1023130728 1 2024-09-10T04:05:35.314 INFO:tasks.workunit.client.1.smithi070.stdout:2/960: rename d15/d18/d29/d2a/d2e/da6 to d15/d18/d29/d2a/d2e/d32/d36/d157 0 2024-09-10T04:05:35.315 INFO:tasks.workunit.client.1.smithi070.stdout:1/926: mkdir d3/d8/d117/d94/dfc/d132 0 2024-09-10T04:05:35.316 INFO:tasks.workunit.client.1.smithi070.stdout:4/993: truncate de/f34 2320978 0 2024-09-10T04:05:35.317 INFO:tasks.workunit.client.1.smithi070.stdout:1/927: unlink d3/d8/d13/d67/f10a 0 2024-09-10T04:05:35.319 INFO:tasks.workunit.client.1.smithi070.stdout:4/994: creat de/d13/d18/d26/d8c/f150 x:0 0 0 2024-09-10T04:05:35.319 INFO:tasks.workunit.client.1.smithi070.stdout:2/961: dwrite d15/d18/d29/ddb/f13c [0,4194304] 0 2024-09-10T04:05:35.321 INFO:tasks.workunit.client.1.smithi070.stdout:2/962: read d15/d18/f121 [2189,99712] 0 2024-09-10T04:05:35.322 INFO:tasks.workunit.client.1.smithi070.stdout:1/928: mknod d3/d8/d13/d3f/d6d/c133 0 2024-09-10T04:05:35.323 INFO:tasks.workunit.client.1.smithi070.stdout:4/995: creat de/d13/d43/d88/f151 x:0 0 0 2024-09-10T04:05:35.323 INFO:tasks.workunit.client.1.smithi070.stdout:2/963: read d15/d2b/d94/f9a [3062128,25681] 0 2024-09-10T04:05:35.325 INFO:tasks.workunit.client.1.smithi070.stdout:1/929: mkdir d3/d8/d92/dec/d134 0 2024-09-10T04:05:35.326 INFO:tasks.workunit.client.0.smithi046.stdout:2/966: mkdir d2/d8/d65/d152 0 2024-09-10T04:05:35.328 INFO:tasks.workunit.client.1.smithi070.stdout:2/964: rename d15/d18/d56/d7c/d55/d6d/c107 to d15/d2b/c158 0 2024-09-10T04:05:35.329 INFO:tasks.workunit.client.0.smithi046.stdout:2/967: fdatasync d2/d8/d37/dc2/f116 0 2024-09-10T04:05:35.330 INFO:tasks.workunit.client.1.smithi070.stdout:4/996: mknod de/d13/d125/c152 0 2024-09-10T04:05:35.331 INFO:tasks.workunit.client.1.smithi070.stdout:2/965: mkdir d15/d18/d29/d2a/d2e/d32/d36/d157/d159 0 2024-09-10T04:05:35.337 INFO:tasks.workunit.client.1.smithi070.stdout:4/997: creat de/d13/f153 x:0 0 0 2024-09-10T04:05:35.337 INFO:tasks.workunit.client.0.smithi046.stdout:3/960: dwrite d12/d5d/fae [0,4194304] 0 2024-09-10T04:05:35.337 INFO:tasks.workunit.client.0.smithi046.stdout:6/831: write da/db/d11/d13/f39 [3930568,62335] 0 2024-09-10T04:05:35.338 INFO:tasks.workunit.client.1.smithi070.stdout:2/966: creat d15/d18/d29/f15a x:0 0 0 2024-09-10T04:05:35.339 INFO:tasks.workunit.client.0.smithi046.stdout:3/961: chown d12/d5d/d71/d49/f10e 629819 1 2024-09-10T04:05:35.339 INFO:tasks.workunit.client.0.smithi046.stdout:6/832: readlink da/db/d11/d2b/l46 0 2024-09-10T04:05:35.342 INFO:tasks.workunit.client.1.smithi070.stdout:4/998: rmdir de/d13/d5b/d7b/dea 39 2024-09-10T04:05:35.347 INFO:tasks.workunit.client.1.smithi070.stdout:1/930: dread d3/d8/d12/d2f/d32/f41 [0,4194304] 0 2024-09-10T04:05:35.348 INFO:tasks.workunit.client.0.smithi046.stdout:3/962: dwrite d12/d13/d37/d51/db5/d4b/d91/d112/f12a [0,4194304] 0 2024-09-10T04:05:35.351 INFO:tasks.workunit.client.0.smithi046.stdout:3/963: readlink d12/d13/d14/d1e/d2d/d39/d67/l8b 0 2024-09-10T04:05:35.353 INFO:tasks.workunit.client.1.smithi070.stdout:1/931: dwrite d3/d8/d13/d3f/f52 [0,4194304] 0 2024-09-10T04:05:35.354 INFO:tasks.workunit.client.1.smithi070.stdout:1/932: chown d3/d8/d12/d2f/ld1 29 1 2024-09-10T04:05:35.358 INFO:tasks.workunit.client.0.smithi046.stdout:3/964: dread d12/d13/d37/d51/db5/d50/d7a/dc1/f97 [0,4194304] 0 2024-09-10T04:05:35.359 INFO:tasks.workunit.client.0.smithi046.stdout:3/965: readlink d12/d13/d14/d1e/d2d/d39/d67/l73 0 2024-09-10T04:05:35.361 INFO:tasks.workunit.client.1.smithi070.stdout:1/933: rename d3/d8/d13/f25 to d3/d8/d12/d2f/d32/da8/f135 0 2024-09-10T04:05:35.363 INFO:tasks.workunit.client.1.smithi070.stdout:1/934: rmdir d3/d8/d13/d3f/d6d/dbe 39 2024-09-10T04:05:35.365 INFO:tasks.workunit.client.1.smithi070.stdout:1/935: chown d3/d10/d1d/dbd/fd4 11575 1 2024-09-10T04:05:35.366 INFO:tasks.workunit.client.1.smithi070.stdout:1/936: read - d3/d8/d13/d3f/f11e zero size 2024-09-10T04:05:35.368 INFO:tasks.workunit.client.0.smithi046.stdout:3/966: dwrite d12/d5d/d71/d49/d5c/fd1 [0,4194304] 0 2024-09-10T04:05:35.369 INFO:tasks.workunit.client.0.smithi046.stdout:2/968: write d2/f35 [1453828,78944] 0 2024-09-10T04:05:35.370 INFO:tasks.workunit.client.0.smithi046.stdout:3/967: chown d12/d13/d37/d94/dff 27212 1 2024-09-10T04:05:35.373 INFO:tasks.workunit.client.1.smithi070.stdout:1/937: dwrite f0 [0,4194304] 0 2024-09-10T04:05:35.374 INFO:tasks.workunit.client.0.smithi046.stdout:2/969: mknod d2/d6/d23/d42/db4/dca/d13f/c153 0 2024-09-10T04:05:35.375 INFO:tasks.workunit.client.1.smithi070.stdout:4/999: write de/d13/d18/d5c/fd7 [654937,72248] 0 2024-09-10T04:05:35.376 INFO:tasks.workunit.client.0.smithi046.stdout:3/968: creat d12/d13/d37/d51/db5/d4b/db7/f14d x:0 0 0 2024-09-10T04:05:35.379 INFO:tasks.workunit.client.1.smithi070.stdout:2/967: dwrite d15/d59/feb [0,4194304] 0 2024-09-10T04:05:35.381 INFO:tasks.workunit.client.0.smithi046.stdout:3/969: truncate d12/d5d/d71/d49/d5c/fe9 232676 0 2024-09-10T04:05:35.384 INFO:tasks.workunit.client.1.smithi070.stdout:2/968: mkdir d15/d18/d56/d7c/d15b 0 2024-09-10T04:05:35.387 INFO:tasks.workunit.client.1.smithi070.stdout:2/969: mknod d15/d18/db2/ded/df8/c15c 0 2024-09-10T04:05:35.389 INFO:tasks.workunit.client.0.smithi046.stdout:2/970: write d2/d8/d65/f7d [980167,20577] 0 2024-09-10T04:05:35.393 INFO:tasks.workunit.client.0.smithi046.stdout:3/970: link d12/d5d/f6e d12/d13/d37/d51/db5/d50/d7a/dc1/d96/f14e 0 2024-09-10T04:05:35.393 INFO:tasks.workunit.client.1.smithi070.stdout:2/970: dread d15/d18/d29/d2a/d2e/d32/f82 [0,4194304] 0 2024-09-10T04:05:35.395 INFO:tasks.workunit.client.0.smithi046.stdout:3/971: creat d12/d13/d14/d1e/d2d/d39/d67/f14f x:0 0 0 2024-09-10T04:05:35.396 INFO:tasks.workunit.client.1.smithi070.stdout:2/971: chown d15/d18/d29/d2a/d2e/cac 595894689 1 2024-09-10T04:05:35.397 INFO:tasks.workunit.client.0.smithi046.stdout:3/972: truncate d12/d13/d14/f120 179911 0 2024-09-10T04:05:35.398 INFO:tasks.workunit.client.0.smithi046.stdout:2/971: write d2/d6/d23/d5d/d77/f54 [663944,64321] 0 2024-09-10T04:05:35.401 INFO:tasks.workunit.client.1.smithi070.stdout:2/972: dwrite d15/d18/d56/d7c/d55/d6d/fa8 [0,4194304] 0 2024-09-10T04:05:35.403 INFO:tasks.workunit.client.0.smithi046.stdout:2/972: unlink d2/d6/d15/d14e/fa5 0 2024-09-10T04:05:35.406 INFO:tasks.workunit.client.0.smithi046.stdout:3/973: creat d12/d13/d14/f150 x:0 0 0 2024-09-10T04:05:35.407 INFO:tasks.workunit.client.1.smithi070.stdout:2/973: creat d15/d18/db2/ded/f15d x:0 0 0 2024-09-10T04:05:35.409 INFO:tasks.workunit.client.1.smithi070.stdout:2/974: write d15/d18/d29/d2a/d2e/d32/d36/d157/dc0/f152 [966049,44616] 0 2024-09-10T04:05:35.409 INFO:tasks.workunit.client.0.smithi046.stdout:2/973: write d2/d6/d23/d55/f7e [937090,123996] 0 2024-09-10T04:05:35.410 INFO:tasks.workunit.client.0.smithi046.stdout:3/974: dread - d12/d5d/f6e zero size 2024-09-10T04:05:35.412 INFO:tasks.workunit.client.1.smithi070.stdout:2/975: fsync d15/d18/d56/d7c/d55/d6d/fba 0 2024-09-10T04:05:35.414 INFO:tasks.workunit.client.0.smithi046.stdout:2/974: rename d2/d8/d37/d63/f75 to d2/d8/d65/f154 0 2024-09-10T04:05:35.415 INFO:tasks.workunit.client.0.smithi046.stdout:3/975: creat d12/d5d/d71/df7/d124/f151 x:0 0 0 2024-09-10T04:05:35.421 INFO:tasks.workunit.client.1.smithi070.stdout:2/976: rename d15/d18/d56/c11d to d15/d18/d29/d2a/d2e/c15e 0 2024-09-10T04:05:35.421 INFO:tasks.workunit.client.0.smithi046.stdout:3/976: dread d12/d13/d37/f45 [0,4194304] 0 2024-09-10T04:05:35.423 INFO:tasks.workunit.client.0.smithi046.stdout:3/977: mknod d12/d5d/d71/d49/d5c/c152 0 2024-09-10T04:05:35.425 INFO:tasks.workunit.client.1.smithi070.stdout:2/977: rename d15/d59/da1/ce6 to d15/d18/d29/d2a/d2e/d32/d36/d157/d159/c15f 0 2024-09-10T04:05:35.426 INFO:tasks.workunit.client.0.smithi046.stdout:2/975: dread d2/d6/f80 [0,4194304] 0 2024-09-10T04:05:35.427 INFO:tasks.workunit.client.0.smithi046.stdout:3/978: mkdir d12/d13/d37/d51/db5/d4b/db7/dc0/ded/d153 0 2024-09-10T04:05:35.429 INFO:tasks.workunit.client.1.smithi070.stdout:2/978: dread d15/d18/f21 [0,4194304] 0 2024-09-10T04:05:35.433 INFO:tasks.workunit.client.1.smithi070.stdout:2/979: dread d15/fa7 [0,4194304] 0 2024-09-10T04:05:35.433 INFO:tasks.workunit.client.0.smithi046.stdout:3/979: link d12/d13/d14/f108 d12/d5d/f154 0 2024-09-10T04:05:35.433 INFO:tasks.workunit.client.0.smithi046.stdout:2/976: write d2/d8/d37/f41 [4345032,1784] 0 2024-09-10T04:05:35.435 INFO:tasks.workunit.client.1.smithi070.stdout:2/980: symlink d15/d18/d29/d2a/d2e/d32/d36/d10b/l160 0 2024-09-10T04:05:35.437 INFO:tasks.workunit.client.1.smithi070.stdout:2/981: rmdir d15/d18/db2/dc3 39 2024-09-10T04:05:35.440 INFO:tasks.workunit.client.1.smithi070.stdout:2/982: truncate d15/d18/d56/d7c/f76 1028118 0 2024-09-10T04:05:35.441 INFO:tasks.workunit.client.0.smithi046.stdout:2/977: link d2/d6/d23/d5d/d77/d7b/d85/ca6 d2/d8/d65/d90/c155 0 2024-09-10T04:05:35.441 INFO:tasks.workunit.client.1.smithi070.stdout:2/983: mknod d15/d18/d56/c161 0 2024-09-10T04:05:35.442 INFO:tasks.workunit.client.1.smithi070.stdout:2/984: fdatasync d15/d59/f115 0 2024-09-10T04:05:35.445 INFO:tasks.workunit.client.0.smithi046.stdout:3/980: dwrite d12/d13/d14/f120 [0,4194304] 0 2024-09-10T04:05:35.445 INFO:tasks.workunit.client.1.smithi070.stdout:2/985: creat d15/d18/d56/d7c/dfa/f162 x:0 0 0 2024-09-10T04:05:35.453 INFO:tasks.workunit.client.1.smithi070.stdout:2/986: creat d15/d18/d29/d2a/f163 x:0 0 0 2024-09-10T04:05:35.454 INFO:tasks.workunit.client.0.smithi046.stdout:2/978: truncate d2/d6/f18 410369 0 2024-09-10T04:05:35.455 INFO:tasks.workunit.client.0.smithi046.stdout:2/979: mkdir d2/d86/d9c/d156 0 2024-09-10T04:05:35.459 INFO:tasks.workunit.client.1.smithi070.stdout:2/987: dwrite d15/d18/d56/d7c/dfa/f162 [0,4194304] 0 2024-09-10T04:05:35.461 INFO:tasks.workunit.client.1.smithi070.stdout:2/988: chown d15/d18/d29/d2a/d2e/d32/d36/d157/dc0/f152 24402 1 2024-09-10T04:05:35.462 INFO:tasks.workunit.client.1.smithi070.stdout:2/989: stat d15/d2b/f7a 0 2024-09-10T04:05:35.467 INFO:tasks.workunit.client.0.smithi046.stdout:2/980: truncate d2/d6/d15/fbb 2849982 0 2024-09-10T04:05:35.467 INFO:tasks.workunit.client.1.smithi070.stdout:2/990: dread d15/d18/d56/d7c/dfa/f162 [0,4194304] 0 2024-09-10T04:05:35.468 INFO:tasks.workunit.client.0.smithi046.stdout:2/981: dread - d2/d8/d65/fc7 zero size 2024-09-10T04:05:35.470 INFO:tasks.workunit.client.0.smithi046.stdout:2/982: read d2/d8/d65/f7d [1156373,85717] 0 2024-09-10T04:05:35.471 INFO:tasks.workunit.client.1.smithi070.stdout:2/991: mkdir d15/d18/d29/d2a/d2e/d164 0 2024-09-10T04:05:35.473 INFO:tasks.workunit.client.0.smithi046.stdout:2/983: creat d2/d6/d15/d14e/db6/dbe/df2/f157 x:0 0 0 2024-09-10T04:05:35.475 INFO:tasks.workunit.client.0.smithi046.stdout:3/981: dread d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/f126 [0,4194304] 0 2024-09-10T04:05:35.475 INFO:tasks.workunit.client.0.smithi046.stdout:2/984: read d2/d6/d23/f87 [828583,63117] 0 2024-09-10T04:05:35.476 INFO:tasks.workunit.client.0.smithi046.stdout:3/982: write d12/d13/d37/d94/f147 [132581,106698] 0 2024-09-10T04:05:35.481 INFO:tasks.workunit.client.0.smithi046.stdout:3/983: rename d12/d5d/d71/d49/d5c to d12/d5d/d71/d155 0 2024-09-10T04:05:35.484 INFO:tasks.workunit.client.0.smithi046.stdout:2/985: write d2/d6/f43 [1661543,85989] 0 2024-09-10T04:05:35.484 INFO:tasks.workunit.client.1.smithi070.stdout:1/938: sync 2024-09-10T04:05:35.484 INFO:tasks.workunit.client.1.smithi070.stdout:1/939: chown d3/d8 334084057 1 2024-09-10T04:05:35.487 INFO:tasks.workunit.client.1.smithi070.stdout:2/992: dwrite d15/d18/d29/d2a/d2e/d32/d36/f5b [0,4194304] 0 2024-09-10T04:05:35.487 INFO:tasks.workunit.client.0.smithi046.stdout:2/986: mkdir d2/d6/d23/d55/d9f/d158 0 2024-09-10T04:05:35.487 INFO:tasks.workunit.client.0.smithi046.stdout:3/984: dread d12/d13/ff2 [0,4194304] 0 2024-09-10T04:05:35.488 INFO:tasks.workunit.client.0.smithi046.stdout:3/985: chown d12/d13/d37/d51/db5 7 1 2024-09-10T04:05:35.489 INFO:tasks.workunit.client.0.smithi046.stdout:2/987: fdatasync d2/d6/d15/d14e/db6/dbe/df2/f113 0 2024-09-10T04:05:35.489 INFO:tasks.workunit.client.1.smithi070.stdout:1/940: creat d3/d10/f136 x:0 0 0 2024-09-10T04:05:35.490 INFO:tasks.workunit.client.0.smithi046.stdout:3/986: fsync d12/f5f 0 2024-09-10T04:05:35.492 INFO:tasks.workunit.client.0.smithi046.stdout:1/730: sync 2024-09-10T04:05:35.492 INFO:tasks.workunit.client.0.smithi046.stdout:3/987: readlink d12/d13/d37/d51/db5/d50/l6a 0 2024-09-10T04:05:35.496 INFO:tasks.workunit.client.1.smithi070.stdout:1/941: creat d3/d8/d13/d67/d83/def/dfb/f137 x:0 0 0 2024-09-10T04:05:35.497 INFO:tasks.workunit.client.0.smithi046.stdout:2/988: dwrite d2/d49/f11c [0,4194304] 0 2024-09-10T04:05:35.498 INFO:tasks.workunit.client.1.smithi070.stdout:2/993: link d15/d18/d29/d2a/d2e/d32/d36/d157/dc0/dca/c12f d15/d18/d56/c165 0 2024-09-10T04:05:35.501 INFO:tasks.workunit.client.0.smithi046.stdout:3/988: creat d12/d13/d14/d1e/d2d/d6f/d11f/d13e/f156 x:0 0 0 2024-09-10T04:05:35.502 INFO:tasks.workunit.client.1.smithi070.stdout:1/942: rmdir d3/d10/db2/dc7 39 2024-09-10T04:05:35.505 INFO:tasks.workunit.client.0.smithi046.stdout:3/989: creat d12/d13/d37/d51/db5/d4b/db7/f157 x:0 0 0 2024-09-10T04:05:35.505 INFO:tasks.workunit.client.1.smithi070.stdout:2/994: dread d15/d18/d56/fa4 [0,4194304] 0 2024-09-10T04:05:35.511 INFO:tasks.workunit.client.0.smithi046.stdout:1/731: dread d3/dc/d34/f92 [0,4194304] 0 2024-09-10T04:05:35.513 INFO:tasks.workunit.client.1.smithi070.stdout:1/943: rmdir d3/d8/d117/d94/dfc/d132 0 2024-09-10T04:05:35.515 INFO:tasks.workunit.client.0.smithi046.stdout:2/989: write d2/d49/f9b [812057,22961] 0 2024-09-10T04:05:35.515 INFO:tasks.workunit.client.0.smithi046.stdout:1/732: rmdir d3/d27/d5d/d60/dd7 39 2024-09-10T04:05:35.516 INFO:tasks.workunit.client.0.smithi046.stdout:3/990: write d12/d5d/f154 [4700907,15052] 0 2024-09-10T04:05:35.517 INFO:tasks.workunit.client.1.smithi070.stdout:1/944: fdatasync d3/d10/db2/fb7 0 2024-09-10T04:05:35.518 INFO:tasks.workunit.client.1.smithi070.stdout:1/945: dread d3/d8/f5c [0,4194304] 0 2024-09-10T04:05:35.520 INFO:tasks.workunit.client.0.smithi046.stdout:2/990: rename d2/d6/d23/d5d/c132 to d2/d8/d37/c159 0 2024-09-10T04:05:35.522 INFO:tasks.workunit.client.0.smithi046.stdout:1/733: creat d3/d9b/dcb/ffa x:0 0 0 2024-09-10T04:05:35.524 INFO:tasks.workunit.client.0.smithi046.stdout:3/991: symlink d12/d13/d37/d51/db5/d4b/d91/d112/l158 0 2024-09-10T04:05:35.526 INFO:tasks.workunit.client.1.smithi070.stdout:2/995: dwrite d15/d18/f24 [0,4194304] 0 2024-09-10T04:05:35.529 INFO:tasks.workunit.client.0.smithi046.stdout:3/992: dread d12/d13/d37/d51/db5/d50/d7a/dc1/d96/da5/fe4 [0,4194304] 0 2024-09-10T04:05:35.530 INFO:tasks.workunit.client.0.smithi046.stdout:1/734: creat d3/d27/ffb x:0 0 0 2024-09-10T04:05:35.532 INFO:tasks.workunit.client.1.smithi070.stdout:2/996: dwrite d15/d59/feb [0,4194304] 0 2024-09-10T04:05:35.533 INFO:tasks.workunit.client.1.smithi070.stdout:2/997: chown d15/d18/d29/ddb/d148 127489033 1 2024-09-10T04:05:35.535 INFO:tasks.workunit.client.1.smithi070.stdout:1/946: truncate d3/d8/d27/f68 2859922 0 2024-09-10T04:05:35.535 INFO:tasks.workunit.client.0.smithi046.stdout:2/991: write d2/d6/f4f [464887,104115] 0 2024-09-10T04:05:35.537 INFO:tasks.workunit.client.1.smithi070.stdout:2/998: creat d15/d18/d56/f166 x:0 0 0 2024-09-10T04:05:35.538 INFO:tasks.workunit.client.0.smithi046.stdout:1/735: creat d3/ffc x:0 0 0 2024-09-10T04:05:35.538 INFO:tasks.workunit.client.1.smithi070.stdout:2/999: dread - d15/d18/d56/d7c/d55/d151/f154 zero size 2024-09-10T04:05:35.541 INFO:tasks.workunit.client.0.smithi046.stdout:2/992: rename d2/d8/d65/d152 to d2/d6/d15/d14e/db6/d15a 0 2024-09-10T04:05:35.546 INFO:tasks.workunit.client.0.smithi046.stdout:3/993: getdents d12/d13/d37/d51/db5/d50 0 2024-09-10T04:05:35.547 INFO:tasks.workunit.client.0.smithi046.stdout:2/993: rmdir d2/d8/d37 39 2024-09-10T04:05:35.548 INFO:tasks.workunit.client.0.smithi046.stdout:2/994: write d2/d6/d23/d5d/d77/f54 [4661962,111401] 0 2024-09-10T04:05:35.549 INFO:tasks.workunit.client.0.smithi046.stdout:1/736: mkdir d3/d27/d5d/d60/dd7/dfd 0 2024-09-10T04:05:35.553 INFO:tasks.workunit.client.1.smithi070.stdout:1/947: dwrite d3/d8/d12/d2f/d32/fad [0,4194304] 0 2024-09-10T04:05:35.556 INFO:tasks.workunit.client.1.smithi070.stdout:1/948: truncate d3/f60 2965291 0 2024-09-10T04:05:35.557 INFO:tasks.workunit.client.1.smithi070.stdout:1/949: chown d3/d8/d12/d2f/d32/f41 8683 1 2024-09-10T04:05:35.559 INFO:tasks.workunit.client.0.smithi046.stdout:3/994: dread d12/d13/f2c [0,4194304] 0 2024-09-10T04:05:35.559 INFO:tasks.workunit.client.0.smithi046.stdout:9/911: sync 2024-09-10T04:05:35.559 INFO:tasks.workunit.client.1.smithi070.stdout:1/950: fdatasync d3/d8/d13/d67/f8a 0 2024-09-10T04:05:35.562 INFO:tasks.workunit.client.0.smithi046.stdout:2/995: dwrite d2/d6/d23/d5d/d5e/f117 [0,4194304] 0 2024-09-10T04:05:35.562 INFO:tasks.workunit.client.0.smithi046.stdout:9/912: readlink d4/d3c/de0/l100 0 2024-09-10T04:05:35.563 INFO:tasks.workunit.client.0.smithi046.stdout:3/995: stat d12/d5d/d71/d155/ld8 0 2024-09-10T04:05:35.563 INFO:tasks.workunit.client.0.smithi046.stdout:1/737: dwrite d3/dc/d34/fc6 [0,4194304] 0 2024-09-10T04:05:35.565 INFO:tasks.workunit.client.0.smithi046.stdout:2/996: dread - d2/d6/d23/d42/db8/f10c zero size 2024-09-10T04:05:35.569 INFO:tasks.workunit.client.1.smithi070.stdout:1/951: dread d3/d8/d27/f93 [0,4194304] 0 2024-09-10T04:05:35.574 INFO:tasks.workunit.client.0.smithi046.stdout:2/997: mknod d2/d8/d37/c15b 0 2024-09-10T04:05:35.577 INFO:tasks.workunit.client.0.smithi046.stdout:3/996: write d12/d13/d14/d1e/dd2/f113 [959745,120987] 0 2024-09-10T04:05:35.578 INFO:tasks.workunit.client.0.smithi046.stdout:9/913: dwrite d4/d7/d1f/feb [0,4194304] 0 2024-09-10T04:05:35.582 INFO:tasks.workunit.client.0.smithi046.stdout:2/998: symlink d2/d6/d15/d14e/db6/l15c 0 2024-09-10T04:05:35.583 INFO:tasks.workunit.client.0.smithi046.stdout:2/999: chown d2/d6/d23/d42/de6 213547850 1 2024-09-10T04:05:35.584 INFO:tasks.workunit.client.0.smithi046.stdout:3/997: symlink d12/d5d/d71/df7/d11e/l159 0 2024-09-10T04:05:35.584 INFO:tasks.workunit.client.0.smithi046.stdout:1/738: dwrite d3/f26 [0,4194304] 0 2024-09-10T04:05:35.585 INFO:tasks.workunit.client.0.smithi046.stdout:1/739: fsync d3/da1/f43 0 2024-09-10T04:05:35.589 INFO:tasks.workunit.client.0.smithi046.stdout:1/740: fsync d3/d27/d5d/d60/fdd 0 2024-09-10T04:05:35.592 INFO:tasks.workunit.client.1.smithi070.stdout:1/952: chown d3/d10/db2/dc7/f131 110798690 1 2024-09-10T04:05:35.592 INFO:tasks.workunit.client.0.smithi046.stdout:0/752: sync 2024-09-10T04:05:35.593 INFO:tasks.workunit.client.0.smithi046.stdout:6/833: sync 2024-09-10T04:05:35.593 INFO:tasks.workunit.client.0.smithi046.stdout:8/943: sync 2024-09-10T04:05:35.593 INFO:tasks.workunit.client.1.smithi070.stdout:1/953: fdatasync d3/d8/d13/ff1 0 2024-09-10T04:05:35.593 INFO:tasks.workunit.client.0.smithi046.stdout:3/998: link d12/d13/d14/f15 d12/d13/d14/d1e/d2d/d6f/d11f/d13e/f15a 0 2024-09-10T04:05:35.595 INFO:tasks.workunit.client.0.smithi046.stdout:1/741: creat d3/da1/d1f/df4/ffe x:0 0 0 2024-09-10T04:05:35.596 INFO:tasks.workunit.client.0.smithi046.stdout:0/753: chown d1/dc/d25/d33/d4/laa 123 1 2024-09-10T04:05:35.597 INFO:tasks.workunit.client.1.smithi070.stdout:1/954: unlink d3/d8/d12/d2f/d32/f4c 0 2024-09-10T04:05:35.598 INFO:tasks.workunit.client.0.smithi046.stdout:8/944: write d1/d29/d17/d1f/d46/d76/f126 [591426,86357] 0 2024-09-10T04:05:35.599 INFO:tasks.workunit.client.0.smithi046.stdout:8/945: chown d1/c137 1 1 2024-09-10T04:05:35.601 INFO:tasks.workunit.client.1.smithi070.stdout:1/955: mkdir d3/d8/d117/d138 0 2024-09-10T04:05:35.603 INFO:tasks.workunit.client.1.smithi070.stdout:1/956: chown d3/d8/c2e 3021 1 2024-09-10T04:05:35.605 INFO:tasks.workunit.client.0.smithi046.stdout:0/754: link d1/dc/d25/d33/d3/dd/d55/d4d/f71 d1/d13/d37/d75/d94/dd4/feb 0 2024-09-10T04:05:35.607 INFO:tasks.workunit.client.1.smithi070.stdout:1/957: mknod d3/d8/d117/d94/da5/c139 0 2024-09-10T04:05:35.607 INFO:tasks.workunit.client.1.smithi070.stdout:1/958: chown d3/d8/d27/d9a/lca 57223 1 2024-09-10T04:05:35.609 INFO:tasks.workunit.client.0.smithi046.stdout:9/914: write d4/d3c/de0/fd9 [408337,13331] 0 2024-09-10T04:05:35.610 INFO:tasks.workunit.client.0.smithi046.stdout:8/946: dread d1/f39 [0,4194304] 0 2024-09-10T04:05:35.611 INFO:tasks.workunit.client.1.smithi070.stdout:1/959: unlink d3/d8/d13/d3f/d6d/f128 0 2024-09-10T04:05:35.612 INFO:tasks.workunit.client.0.smithi046.stdout:0/755: symlink d1/dc/d49/db2/dcb/lec 0 2024-09-10T04:05:35.612 INFO:tasks.workunit.client.0.smithi046.stdout:1/742: dread d3/d62/faa [0,4194304] 0 2024-09-10T04:05:35.614 INFO:tasks.workunit.client.0.smithi046.stdout:0/756: chown d1/dc/d25/d33/d1a/d41/f5a 17303386 1 2024-09-10T04:05:35.617 INFO:tasks.workunit.client.1.smithi070.stdout:1/960: dwrite d3/d90/dcb/f105 [0,4194304] 0 2024-09-10T04:05:35.623 INFO:tasks.workunit.client.0.smithi046.stdout:8/947: dread d1/d29/d17/d1f/fc8 [0,4194304] 0 2024-09-10T04:05:35.623 INFO:tasks.workunit.client.0.smithi046.stdout:3/999: write d12/d13/d37/d51/db5/d4b/fef [830541,76778] 0 2024-09-10T04:05:35.624 INFO:tasks.workunit.client.0.smithi046.stdout:6/834: write da/d96/ff0 [1832126,432] 0 2024-09-10T04:05:35.629 INFO:tasks.workunit.client.0.smithi046.stdout:8/948: unlink d1/d29/d17/d1f/d46/d67/dc1/d124/l135 0 2024-09-10T04:05:35.631 INFO:tasks.workunit.client.0.smithi046.stdout:9/915: write d4/d18/f22 [3704289,92296] 0 2024-09-10T04:05:35.632 INFO:tasks.workunit.client.0.smithi046.stdout:1/743: write d3/da1/f69 [1529038,96045] 0 2024-09-10T04:05:35.632 INFO:tasks.workunit.client.0.smithi046.stdout:0/757: write d1/dc/d25/d33/f7e [121934,3281] 0 2024-09-10T04:05:35.634 INFO:tasks.workunit.client.0.smithi046.stdout:6/835: dwrite da/db/d11/d9f/ffa [0,4194304] 0 2024-09-10T04:05:35.635 INFO:tasks.workunit.client.0.smithi046.stdout:9/916: chown d4/d29/d65/c83 10591811 1 2024-09-10T04:05:35.637 INFO:tasks.workunit.client.0.smithi046.stdout:9/917: chown d4/d66/cb4 368 1 2024-09-10T04:05:35.637 INFO:tasks.workunit.client.1.smithi070.stdout:1/961: dwrite d3/d10/d1d/f88 [0,4194304] 0 2024-09-10T04:05:35.641 INFO:tasks.workunit.client.0.smithi046.stdout:8/949: link d1/d29/d19/db7/fd5 d1/d29/d17/d1f/d46/d6a/d82/d99/f13e 0 2024-09-10T04:05:35.641 INFO:tasks.workunit.client.0.smithi046.stdout:0/758: symlink d1/dc/d25/d33/d3/dd/d55/d77/da6/led 0 2024-09-10T04:05:35.643 INFO:tasks.workunit.client.1.smithi070.stdout:1/962: truncate d3/d8/d12/d2f/fd5 990007 0 2024-09-10T04:05:35.643 INFO:tasks.workunit.client.0.smithi046.stdout:9/918: creat d4/d44/d108/f120 x:0 0 0 2024-09-10T04:05:35.644 INFO:tasks.workunit.client.0.smithi046.stdout:8/950: readlink d1/d29/d17/d1f/d46/d67/d116/l12d 0 2024-09-10T04:05:35.644 INFO:tasks.workunit.client.0.smithi046.stdout:1/744: dread d3/dc/f23 [0,4194304] 0 2024-09-10T04:05:35.646 INFO:tasks.workunit.client.0.smithi046.stdout:0/759: creat d1/d13/d37/d75/d94/d9a/da4/fee x:0 0 0 2024-09-10T04:05:35.646 INFO:tasks.workunit.client.0.smithi046.stdout:1/745: fsync d3/d27/ffb 0 2024-09-10T04:05:35.647 INFO:tasks.workunit.client.0.smithi046.stdout:6/836: link da/d45/fe6 da/db/d18/f110 0 2024-09-10T04:05:35.647 INFO:tasks.workunit.client.0.smithi046.stdout:1/746: stat d3/d27/d5d/d60/dd7/deb 0 2024-09-10T04:05:35.648 INFO:tasks.workunit.client.0.smithi046.stdout:9/919: mkdir d4/d7/d1f/d54/dc7/dc8/d121 0 2024-09-10T04:05:35.649 INFO:tasks.workunit.client.0.smithi046.stdout:8/951: symlink d1/d29/d17/d1f/d2a/df6/d102/l13f 0 2024-09-10T04:05:35.649 INFO:tasks.workunit.client.0.smithi046.stdout:1/747: read - d3/da1/d1f/df4/ffe zero size 2024-09-10T04:05:35.650 INFO:tasks.workunit.client.0.smithi046.stdout:6/837: dread - da/db/d11/d2b/fbe zero size 2024-09-10T04:05:35.651 INFO:tasks.workunit.client.0.smithi046.stdout:1/748: readlink d3/d9b/l9d 0 2024-09-10T04:05:35.655 INFO:tasks.workunit.client.0.smithi046.stdout:9/920: symlink d4/d3c/d4d/d5c/d59/l122 0 2024-09-10T04:05:35.656 INFO:tasks.workunit.client.0.smithi046.stdout:0/760: dwrite d1/fdc [0,4194304] 0 2024-09-10T04:05:35.662 INFO:tasks.workunit.client.0.smithi046.stdout:8/952: dwrite d1/d29/d17/d1f/d2a/df6/f121 [0,4194304] 0 2024-09-10T04:05:35.668 INFO:tasks.workunit.client.0.smithi046.stdout:9/921: symlink d4/d3c/d4d/d5c/d87/l123 0 2024-09-10T04:05:35.671 INFO:tasks.workunit.client.0.smithi046.stdout:0/761: dread d1/dc/d25/f9e [0,4194304] 0 2024-09-10T04:05:35.671 INFO:tasks.workunit.client.0.smithi046.stdout:1/749: write d3/f4d [1192993,27635] 0 2024-09-10T04:05:35.673 INFO:tasks.workunit.client.0.smithi046.stdout:6/838: write da/db/d11/f8c [2010092,26879] 0 2024-09-10T04:05:35.675 INFO:tasks.workunit.client.0.smithi046.stdout:1/750: write d3/dc/d34/f76 [1851674,81811] 0 2024-09-10T04:05:35.676 INFO:tasks.workunit.client.0.smithi046.stdout:8/953: rename d1/d29/d17/f9c to d1/d29/d19/d36/d4c/f140 0 2024-09-10T04:05:35.678 INFO:tasks.workunit.client.0.smithi046.stdout:0/762: dread d1/d13/d37/d75/d94/d9a/f90 [0,4194304] 0 2024-09-10T04:05:35.678 INFO:tasks.workunit.client.0.smithi046.stdout:1/751: write d3/f4a [2581770,41455] 0 2024-09-10T04:05:35.680 INFO:tasks.workunit.client.0.smithi046.stdout:1/752: write d3/d9b/fbc [448267,106098] 0 2024-09-10T04:05:35.682 INFO:tasks.workunit.client.0.smithi046.stdout:0/763: symlink d1/dc/d25/lef 0 2024-09-10T04:05:35.685 INFO:tasks.workunit.client.0.smithi046.stdout:9/922: dread d4/f6 [0,4194304] 0 2024-09-10T04:05:35.685 INFO:tasks.workunit.client.0.smithi046.stdout:6/839: symlink da/db/d11/d13/d1e/d23/d54/dad/dfe/l111 0 2024-09-10T04:05:35.688 INFO:tasks.workunit.client.0.smithi046.stdout:6/840: unlink da/db/c99 0 2024-09-10T04:05:35.690 INFO:tasks.workunit.client.0.smithi046.stdout:9/923: fdatasync d4/d3c/d4d/fe1 0 2024-09-10T04:05:35.691 INFO:tasks.workunit.client.0.smithi046.stdout:8/954: link d1/d29/d17/d1f/d46/d6a/ffc d1/d29/d17/d1f/d46/d6a/d82/d99/dda/f141 0 2024-09-10T04:05:35.692 INFO:tasks.workunit.client.0.smithi046.stdout:6/841: fdatasync da/d45/fe6 0 2024-09-10T04:05:35.694 INFO:tasks.workunit.client.0.smithi046.stdout:0/764: dread d1/dc/d25/d33/d1a/f31 [0,4194304] 0 2024-09-10T04:05:35.697 INFO:tasks.workunit.client.0.smithi046.stdout:1/753: truncate d3/da1/f2b 1242370 0 2024-09-10T04:05:35.697 INFO:tasks.workunit.client.0.smithi046.stdout:1/754: readlink d3/d27/d5d/d60/lcc 0 2024-09-10T04:05:35.698 INFO:tasks.workunit.client.0.smithi046.stdout:1/755: dread - d3/dc/d21/d3a/d90/fe7 zero size 2024-09-10T04:05:35.701 INFO:tasks.workunit.client.0.smithi046.stdout:9/924: link d4/d3c/d4d/d5c/d59/l70 d4/d66/l124 0 2024-09-10T04:05:35.702 INFO:tasks.workunit.client.0.smithi046.stdout:0/765: dread d1/dc/d25/d33/d3/dd/d55/d77/da6/fb3 [0,4194304] 0 2024-09-10T04:05:35.703 INFO:tasks.workunit.client.0.smithi046.stdout:0/766: chown d1/dc/d25/d33/d4/d19/f58 5102 1 2024-09-10T04:05:35.703 INFO:tasks.workunit.client.0.smithi046.stdout:6/842: link da/db/d11/d2b/d2f/f74 da/db/d11/d9f/dd6/f112 0 2024-09-10T04:05:35.705 INFO:tasks.workunit.client.0.smithi046.stdout:8/955: getdents d1/d29/d17/d1f/d2a 0 2024-09-10T04:05:35.705 INFO:tasks.workunit.client.0.smithi046.stdout:1/756: mknod d3/d27/d5d/d60/dd7/df9/cff 0 2024-09-10T04:05:35.706 INFO:tasks.workunit.client.0.smithi046.stdout:6/843: symlink da/db/d63/l113 0 2024-09-10T04:05:35.707 INFO:tasks.workunit.client.0.smithi046.stdout:1/757: fsync d3/d27/d2a/fa3 0 2024-09-10T04:05:35.708 INFO:tasks.workunit.client.0.smithi046.stdout:9/925: creat d4/d29/f125 x:0 0 0 2024-09-10T04:05:35.709 INFO:tasks.workunit.client.0.smithi046.stdout:6/844: mknod da/db/d10/d51/c114 0 2024-09-10T04:05:35.740 INFO:tasks.workunit.client.0.smithi046.stdout:1/758: creat d3/d9b/f100 x:0 0 0 2024-09-10T04:05:35.741 INFO:tasks.workunit.client.0.smithi046.stdout:9/926: fsync d4/d18/d89/fed 0 2024-09-10T04:05:35.741 INFO:tasks.workunit.client.0.smithi046.stdout:1/759: fsync d3/f4d 0 2024-09-10T04:05:35.744 INFO:tasks.workunit.client.0.smithi046.stdout:1/760: rmdir d3/dc/d34/d58 39 2024-09-10T04:05:35.747 INFO:tasks.workunit.client.0.smithi046.stdout:9/927: creat d4/d18/d89/ddb/dee/d109/f126 x:0 0 0 2024-09-10T04:05:35.750 INFO:tasks.workunit.client.0.smithi046.stdout:8/956: dread d1/f2e [4194304,4194304] 0 2024-09-10T04:05:35.751 INFO:tasks.workunit.client.0.smithi046.stdout:9/928: truncate d4/d3c/f6b 1553482 0 2024-09-10T04:05:35.753 INFO:tasks.workunit.client.0.smithi046.stdout:9/929: dread - d4/d7/d1f/dce/dd4/f11f zero size 2024-09-10T04:05:35.754 INFO:tasks.workunit.client.0.smithi046.stdout:9/930: fdatasync d4/d3c/de0/d95/fab 0 2024-09-10T04:05:35.756 INFO:tasks.workunit.client.0.smithi046.stdout:9/931: fsync d4/d18/f25 0 2024-09-10T04:05:35.757 INFO:tasks.workunit.client.0.smithi046.stdout:6/845: dread da/db/d11/d13/d1e/d23/f104 [0,4194304] 0 2024-09-10T04:05:35.759 INFO:tasks.workunit.client.0.smithi046.stdout:1/761: dread d3/da1/d1f/fb0 [0,4194304] 0 2024-09-10T04:05:35.760 INFO:tasks.workunit.client.0.smithi046.stdout:8/957: write d1/d29/d19/d36/d4c/f5b [3879577,79614] 0 2024-09-10T04:05:35.760 INFO:tasks.workunit.client.0.smithi046.stdout:6/846: stat da/db/d10/d51/c114 0 2024-09-10T04:05:35.761 INFO:tasks.workunit.client.0.smithi046.stdout:6/847: fdatasync f9 0 2024-09-10T04:05:35.762 INFO:tasks.workunit.client.0.smithi046.stdout:8/958: chown d1/d29/d19/l3d 1767402 1 2024-09-10T04:05:35.763 INFO:tasks.workunit.client.0.smithi046.stdout:9/932: rename d4/d7/d61/fd7 to d4/d7/d1f/d54/dc7/dc8/f127 0 2024-09-10T04:05:35.764 INFO:tasks.workunit.client.0.smithi046.stdout:1/762: symlink d3/d27/d5d/d60/dd7/df9/l101 0 2024-09-10T04:05:35.766 INFO:tasks.workunit.client.0.smithi046.stdout:8/959: mknod d1/d29/d19/d36/d8d/c142 0 2024-09-10T04:05:35.768 INFO:tasks.workunit.client.0.smithi046.stdout:6/848: rmdir da/db/d11/d2b/d5a/dc2 39 2024-09-10T04:05:35.769 INFO:tasks.workunit.client.0.smithi046.stdout:6/849: chown da/db/d11/d2b/d2f/d3b/c10c 11 1 2024-09-10T04:05:35.769 INFO:tasks.workunit.client.0.smithi046.stdout:8/960: creat d1/d29/d19/d36/d4c/d5e/dc7/dea/f143 x:0 0 0 2024-09-10T04:05:35.770 INFO:tasks.workunit.client.0.smithi046.stdout:1/763: rename d3/dc/d34/d58/da6 to d3/d27/d5d/d102 0 2024-09-10T04:05:35.771 INFO:tasks.workunit.client.0.smithi046.stdout:9/933: creat d4/d3c/d4d/d5c/f128 x:0 0 0 2024-09-10T04:05:35.779 INFO:tasks.workunit.client.0.smithi046.stdout:1/764: link d3/d27/f41 d3/dc/d21/d3a/d90/dbf/f103 0 2024-09-10T04:05:35.782 INFO:tasks.workunit.client.0.smithi046.stdout:6/850: dread da/db/d11/ffc [0,4194304] 0 2024-09-10T04:05:35.782 INFO:tasks.workunit.client.0.smithi046.stdout:6/851: stat f1 0 2024-09-10T04:05:35.786 INFO:tasks.workunit.client.0.smithi046.stdout:9/934: truncate d4/d18/da8/fcf 523959 0 2024-09-10T04:05:35.787 INFO:tasks.workunit.client.0.smithi046.stdout:1/765: write d3/d27/f41 [1895300,107603] 0 2024-09-10T04:05:35.791 INFO:tasks.workunit.client.0.smithi046.stdout:8/961: dwrite d1/d29/d17/d1f/d46/d6a/d82/d99/dda/ffe [0,4194304] 0 2024-09-10T04:05:35.794 INFO:tasks.workunit.client.0.smithi046.stdout:9/935: write d4/d29/d65/f67 [984666,21132] 0 2024-09-10T04:05:35.798 INFO:tasks.workunit.client.0.smithi046.stdout:6/852: dread da/db/d18/f35 [0,4194304] 0 2024-09-10T04:05:35.799 INFO:tasks.workunit.client.0.smithi046.stdout:8/962: link d1/d29/d17/d1f/d46/d67/de0/fe5 d1/d29/d17/d1f/d2a/df6/d117/f144 0 2024-09-10T04:05:35.801 INFO:tasks.workunit.client.0.smithi046.stdout:8/963: rename d1/d29 to d1/d29/d17/d1f/d46/d67/dc1/d124/d145 22 2024-09-10T04:05:35.801 INFO:tasks.workunit.client.0.smithi046.stdout:9/936: mkdir d4/d7/d129 0 2024-09-10T04:05:35.806 INFO:tasks.workunit.client.0.smithi046.stdout:8/964: creat d1/d29/d19/d36/f146 x:0 0 0 2024-09-10T04:05:35.807 INFO:tasks.workunit.client.0.smithi046.stdout:6/853: getdents da/db/d11/d13/d1e/d23/d86/dbf 0 2024-09-10T04:05:35.809 INFO:tasks.workunit.client.0.smithi046.stdout:9/937: dwrite d4/d18/d92/fe7 [0,4194304] 0 2024-09-10T04:05:35.809 INFO:tasks.workunit.client.0.smithi046.stdout:6/854: write da/db/d11/d13/f57 [597156,53817] 0 2024-09-10T04:05:35.812 INFO:tasks.workunit.client.0.smithi046.stdout:9/938: mkdir d4/d29/d10e/d12a 0 2024-09-10T04:05:35.813 INFO:tasks.workunit.client.0.smithi046.stdout:8/965: dread d1/d29/d19/d36/f3f [0,4194304] 0 2024-09-10T04:05:35.816 INFO:tasks.workunit.client.0.smithi046.stdout:8/966: chown d1/d29/d17/d1f/d46/d6a/d82/d99/fb5 0 1 2024-09-10T04:05:35.817 INFO:tasks.workunit.client.0.smithi046.stdout:9/939: creat d4/d29/d65/f12b x:0 0 0 2024-09-10T04:05:35.820 INFO:tasks.workunit.client.0.smithi046.stdout:9/940: dread - d4/d7/d81/fea zero size 2024-09-10T04:05:35.820 INFO:tasks.workunit.client.0.smithi046.stdout:9/941: chown d4/d66 2659 1 2024-09-10T04:05:35.822 INFO:tasks.workunit.client.0.smithi046.stdout:6/855: dread da/db/d10/d27/f8a [0,4194304] 0 2024-09-10T04:05:35.823 INFO:tasks.workunit.client.0.smithi046.stdout:6/856: read da/fe [163018,46846] 0 2024-09-10T04:05:35.826 INFO:tasks.workunit.client.0.smithi046.stdout:6/857: mknod da/db/d18/dd1/c115 0 2024-09-10T04:05:35.831 INFO:tasks.workunit.client.0.smithi046.stdout:8/967: rename d1/d29/d19/d36 to d1/d29/d17/d1f/d2a/df6/d102/d12e/d147 0 2024-09-10T04:05:35.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:35 smithi070 ceph-mon[35495]: pgmap v271: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 227 MiB/s wr, 490 op/s 2024-09-10T04:05:35.836 INFO:tasks.workunit.client.0.smithi046.stdout:8/968: write d1/d29/d17/d1f/d46/d6a/d82/d99/f13e [2581086,96853] 0 2024-09-10T04:05:35.838 INFO:tasks.workunit.client.0.smithi046.stdout:8/969: write d1/d29/f131 [145864,80499] 0 2024-09-10T04:05:35.840 INFO:tasks.workunit.client.0.smithi046.stdout:9/942: dwrite d4/fc9 [0,4194304] 0 2024-09-10T04:05:35.841 INFO:tasks.workunit.client.0.smithi046.stdout:6/858: dread da/db/f78 [0,4194304] 0 2024-09-10T04:05:35.842 INFO:tasks.workunit.client.0.smithi046.stdout:6/859: readlink da/d45/ld0 0 2024-09-10T04:05:35.843 INFO:tasks.workunit.client.0.smithi046.stdout:9/943: fdatasync d4/d7/d61/ff3 0 2024-09-10T04:05:35.846 INFO:tasks.workunit.client.0.smithi046.stdout:6/860: fdatasync da/db/d10/d27/f36 0 2024-09-10T04:05:35.847 INFO:tasks.workunit.client.0.smithi046.stdout:8/970: write d1/d29/d17/d1f/d46/d6a/fe8 [277380,74714] 0 2024-09-10T04:05:35.849 INFO:tasks.workunit.client.0.smithi046.stdout:6/861: write da/db/d11/d13/f7c [5123932,124977] 0 2024-09-10T04:05:35.852 INFO:tasks.workunit.client.0.smithi046.stdout:8/971: mkdir d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/d4c/dd3/d10c/d148 0 2024-09-10T04:05:35.854 INFO:tasks.workunit.client.0.smithi046.stdout:6/862: read da/db/d11/d2b/d55/f81 [3397310,22515] 0 2024-09-10T04:05:35.857 INFO:tasks.workunit.client.0.smithi046.stdout:8/972: fsync d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/f3c 0 2024-09-10T04:05:35.858 INFO:tasks.workunit.client.0.smithi046.stdout:9/944: dwrite d4/d18/d89/fca [0,4194304] 0 2024-09-10T04:05:35.860 INFO:tasks.workunit.client.0.smithi046.stdout:9/945: dread - d4/d3c/de0/d95/f111 zero size 2024-09-10T04:05:35.864 INFO:tasks.workunit.client.0.smithi046.stdout:9/946: creat d4/d44/d108/f12c x:0 0 0 2024-09-10T04:05:35.867 INFO:tasks.workunit.client.0.smithi046.stdout:6/863: dwrite da/db/d63/fa0 [0,4194304] 0 2024-09-10T04:05:35.874 INFO:tasks.workunit.client.0.smithi046.stdout:6/864: write da/db/d63/fa2 [1087193,55454] 0 2024-09-10T04:05:35.876 INFO:tasks.workunit.client.1.smithi070.stdout:1/963: sync 2024-09-10T04:05:35.876 INFO:tasks.workunit.client.0.smithi046.stdout:8/973: dread d1/d29/d19/f56 [0,4194304] 0 2024-09-10T04:05:35.877 INFO:tasks.workunit.client.0.smithi046.stdout:6/865: rmdir da/db/d11/d2b/d55 39 2024-09-10T04:05:35.878 INFO:tasks.workunit.client.0.smithi046.stdout:9/947: dread d4/d3c/f6b [0,4194304] 0 2024-09-10T04:05:35.879 INFO:tasks.workunit.client.1.smithi070.stdout:1/964: unlink d3/d10/f73 0 2024-09-10T04:05:35.880 INFO:tasks.workunit.client.0.smithi046.stdout:8/974: rmdir d1/d29/d17/d1f/d46/d67/d116 39 2024-09-10T04:05:35.880 INFO:tasks.workunit.client.0.smithi046.stdout:6/866: dread da/db/f16 [0,4194304] 0 2024-09-10T04:05:35.882 INFO:tasks.workunit.client.1.smithi070.stdout:1/965: creat d3/d8/d92/dec/d134/f13a x:0 0 0 2024-09-10T04:05:35.883 INFO:tasks.workunit.client.1.smithi070.stdout:1/966: truncate d3/d10/d1d/dbd/da2/f12c 428207 0 2024-09-10T04:05:35.884 INFO:tasks.workunit.client.0.smithi046.stdout:6/867: mknod da/db/c116 0 2024-09-10T04:05:35.884 INFO:tasks.workunit.client.1.smithi070.stdout:1/967: write d3/dd3/f10b [888166,120842] 0 2024-09-10T04:05:35.884 INFO:tasks.workunit.client.0.smithi046.stdout:9/948: creat d4/d3c/d4d/f12d x:0 0 0 2024-09-10T04:05:35.886 INFO:tasks.workunit.client.0.smithi046.stdout:8/975: rename d1/d29/d19/d73/dd2 to d1/d29/d17/d1f/d46/d67/d116/d149 0 2024-09-10T04:05:35.887 INFO:tasks.workunit.client.1.smithi070.stdout:1/968: read d3/d8/d117/d94/f101 [1852654,72058] 0 2024-09-10T04:05:35.888 INFO:tasks.workunit.client.0.smithi046.stdout:8/976: fdatasync d1/d29/d17/d1f/d46/d6a/d82/dc2/f119 0 2024-09-10T04:05:35.888 INFO:tasks.workunit.client.0.smithi046.stdout:6/868: creat da/db/d11/dd7/f117 x:0 0 0 2024-09-10T04:05:35.889 INFO:tasks.workunit.client.0.smithi046.stdout:6/869: read - da/db/d10/d51/fec zero size 2024-09-10T04:05:35.890 INFO:tasks.workunit.client.0.smithi046.stdout:9/949: unlink d4/d18/d89/ddb/dee/d109/f126 0 2024-09-10T04:05:35.898 INFO:tasks.workunit.client.0.smithi046.stdout:8/977: rename d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/d4c/fa9 to d1/d29/d19/d73/f14a 0 2024-09-10T04:05:35.906 INFO:tasks.workunit.client.1.smithi070.stdout:1/969: dwrite d3/d8/d12/d2f/fa9 [0,4194304] 0 2024-09-10T04:05:35.906 INFO:tasks.workunit.client.0.smithi046.stdout:6/870: link da/db/d11/d13/d1e/d69/f87 da/db/d11/d2b/f118 0 2024-09-10T04:05:35.907 INFO:tasks.workunit.client.1.smithi070.stdout:1/970: readlink d3/d8/d13/d3f/d6d/d81/lde 0 2024-09-10T04:05:35.909 INFO:tasks.workunit.client.1.smithi070.stdout:1/971: creat d3/d10/d1d/d79/f13b x:0 0 0 2024-09-10T04:05:35.909 INFO:tasks.workunit.client.0.smithi046.stdout:9/950: dwrite d4/d7/d61/f6f [0,4194304] 0 2024-09-10T04:05:35.910 INFO:tasks.workunit.client.0.smithi046.stdout:8/978: dwrite d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/d6e/d75/f11b [0,4194304] 0 2024-09-10T04:05:35.910 INFO:tasks.workunit.client.1.smithi070.stdout:1/972: fsync d3/d8/fbc 0 2024-09-10T04:05:35.911 INFO:tasks.workunit.client.0.smithi046.stdout:6/871: chown da/db/d11/d13/d1e/d23/d54/c8e 43385 1 2024-09-10T04:05:35.917 INFO:tasks.workunit.client.0.smithi046.stdout:9/951: symlink d4/d3c/de0/l12e 0 2024-09-10T04:05:35.921 INFO:tasks.workunit.client.0.smithi046.stdout:8/979: mknod d1/d29/d17/d1f/d46/d67/c14b 0 2024-09-10T04:05:35.927 INFO:tasks.workunit.client.0.smithi046.stdout:9/952: dwrite d4/d3c/d4d/f11e [0,4194304] 0 2024-09-10T04:05:35.930 INFO:tasks.workunit.client.1.smithi070.stdout:1/973: dwrite d3/d8/d117/d94/fa0 [0,4194304] 0 2024-09-10T04:05:35.933 INFO:tasks.workunit.client.1.smithi070.stdout:1/974: read - d3/d8/d117/d94/da5/f127 zero size 2024-09-10T04:05:35.934 INFO:tasks.workunit.client.0.smithi046.stdout:8/980: dread d1/f79 [0,4194304] 0 2024-09-10T04:05:35.934 INFO:tasks.workunit.client.0.smithi046.stdout:6/872: truncate da/db/d11/dd7/f34 1602179 0 2024-09-10T04:05:35.935 INFO:tasks.workunit.client.1.smithi070.stdout:1/975: rmdir d3/d8 39 2024-09-10T04:05:35.940 INFO:tasks.workunit.client.0.smithi046.stdout:8/981: dread - d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/fb8 zero size 2024-09-10T04:05:35.945 INFO:tasks.workunit.client.0.smithi046.stdout:8/982: symlink d1/d29/d17/d1f/d46/d67/d116/l14c 0 2024-09-10T04:05:35.946 INFO:tasks.workunit.client.0.smithi046.stdout:9/953: dwrite d4/d29/f6d [0,4194304] 0 2024-09-10T04:05:35.946 INFO:tasks.workunit.client.0.smithi046.stdout:6/873: dwrite da/d71/fc7 [0,4194304] 0 2024-09-10T04:05:35.948 INFO:tasks.workunit.client.1.smithi070.stdout:1/976: dwrite d3/d8/d12/d2f/f5b [4194304,4194304] 0 2024-09-10T04:05:35.951 INFO:tasks.workunit.client.0.smithi046.stdout:9/954: truncate d4/d3c/d4d/d5c/d87/f11b 873893 0 2024-09-10T04:05:35.955 INFO:tasks.workunit.client.0.smithi046.stdout:8/983: rename d1/d29/d17/d1f/d2a/l9b to d1/d29/d17/d1f/d46/d67/d116/d149/l14d 0 2024-09-10T04:05:35.957 INFO:tasks.workunit.client.1.smithi070.stdout:1/977: chown d3/d8/d12/d2f/d32/da8/lb6 6018708 1 2024-09-10T04:05:35.959 INFO:tasks.workunit.client.0.smithi046.stdout:9/955: mknod d4/d29/d10e/d12a/c12f 0 2024-09-10T04:05:35.962 INFO:tasks.workunit.client.1.smithi070.stdout:1/978: rename d3/d8/d12/d2f/d32/f6b to d3/d8/d92/dec/f13c 0 2024-09-10T04:05:35.963 INFO:tasks.workunit.client.1.smithi070.stdout:1/979: write d3/d8/d117/fa7 [885468,73787] 0 2024-09-10T04:05:35.964 INFO:tasks.workunit.client.0.smithi046.stdout:8/984: dwrite d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/d4c/d5e/dc7/dea/f143 [0,4194304] 0 2024-09-10T04:05:35.967 INFO:tasks.workunit.client.0.smithi046.stdout:9/956: getdents d4/d44/d108 0 2024-09-10T04:05:35.969 INFO:tasks.workunit.client.0.smithi046.stdout:9/957: unlink d4/d18/d89/ddb/dee/f118 0 2024-09-10T04:05:35.970 INFO:tasks.workunit.client.0.smithi046.stdout:9/958: chown d4/d3c/d4d 36 1 2024-09-10T04:05:35.971 INFO:tasks.workunit.client.1.smithi070.stdout:1/980: link d3/d8/d13/d3f/d5d/l70 d3/d8/d27/d9a/d11b/l13d 0 2024-09-10T04:05:35.971 INFO:tasks.workunit.client.1.smithi070.stdout:1/981: readlink d3/d90/l99 0 2024-09-10T04:05:35.974 INFO:tasks.workunit.client.0.smithi046.stdout:9/959: creat d4/d29/d65/f130 x:0 0 0 2024-09-10T04:05:35.974 INFO:tasks.workunit.client.0.smithi046.stdout:8/985: getdents d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/d4c/d5e/dc7 0 2024-09-10T04:05:35.975 INFO:tasks.workunit.client.0.smithi046.stdout:6/874: write da/db/d11/d13/d1e/d23/d86/dbf/f95 [4439060,4935] 0 2024-09-10T04:05:35.976 INFO:tasks.workunit.client.1.smithi070.stdout:1/982: creat d3/d10/d115/f13e x:0 0 0 2024-09-10T04:05:35.977 INFO:tasks.workunit.client.1.smithi070.stdout:1/983: chown d3/d90/cdf 1 1 2024-09-10T04:05:35.978 INFO:tasks.workunit.client.1.smithi070.stdout:1/984: readlink d3/d8/d27/d9a/lca 0 2024-09-10T04:05:35.978 INFO:tasks.workunit.client.0.smithi046.stdout:9/960: chown d4/le3 3 1 2024-09-10T04:05:35.979 INFO:tasks.workunit.client.0.smithi046.stdout:8/986: truncate d1/fd 4710270 0 2024-09-10T04:05:35.984 INFO:tasks.workunit.client.1.smithi070.stdout:1/985: dwrite d3/d10/db2/ff2 [0,4194304] 0 2024-09-10T04:05:35.985 INFO:tasks.workunit.client.0.smithi046.stdout:6/875: getdents da/db/d11/d13/d1e/d23 0 2024-09-10T04:05:35.987 INFO:tasks.workunit.client.1.smithi070.stdout:1/986: mknod d3/d8/d117/d138/c13f 0 2024-09-10T04:05:35.993 INFO:tasks.workunit.client.1.smithi070.stdout:1/987: mknod d3/d90/dcb/c140 0 2024-09-10T04:05:35.994 INFO:tasks.workunit.client.1.smithi070.stdout:1/988: write d3/dd3/f10b [1602784,901] 0 2024-09-10T04:05:35.999 INFO:tasks.workunit.client.0.smithi046.stdout:8/987: dread d1/f86 [0,4194304] 0 2024-09-10T04:05:36.001 INFO:tasks.workunit.client.0.smithi046.stdout:8/988: fdatasync d1/d29/d17/d1f/d2a/f58 0 2024-09-10T04:05:36.003 INFO:tasks.workunit.client.0.smithi046.stdout:9/961: dwrite d4/d7/d81/fc1 [0,4194304] 0 2024-09-10T04:05:36.004 INFO:tasks.workunit.client.0.smithi046.stdout:9/962: chown d4/d18/d92/l10b 515298576 1 2024-09-10T04:05:36.006 INFO:tasks.workunit.client.1.smithi070.stdout:1/989: link d3/d8/d13/d3f/d6d/dbe/lf8 d3/d116/l141 0 2024-09-10T04:05:36.013 INFO:tasks.workunit.client.1.smithi070.stdout:1/990: read d3/d8/d13/d3f/d6d/f8e [2790258,70633] 0 2024-09-10T04:05:36.016 INFO:tasks.workunit.client.1.smithi070.stdout:1/991: creat d3/d10/db2/dc7/d54/f142 x:0 0 0 2024-09-10T04:05:36.017 INFO:tasks.workunit.client.1.smithi070.stdout:1/992: dread d3/d8/d27/f93 [0,4194304] 0 2024-09-10T04:05:36.021 INFO:tasks.workunit.client.1.smithi070.stdout:1/993: link d3/d10/f2c d3/d10/d115/f143 0 2024-09-10T04:05:36.023 INFO:tasks.workunit.client.0.smithi046.stdout:6/876: truncate da/db/d11/d13/d1e/d9d/f102 4522277 0 2024-09-10T04:05:36.023 INFO:tasks.workunit.client.0.smithi046.stdout:8/989: read - d1/d29/d17/d1f/d46/d6a/d82/fd6 zero size 2024-09-10T04:05:36.024 INFO:tasks.workunit.client.0.smithi046.stdout:6/877: chown da/db/d63/fa1 32304984 1 2024-09-10T04:05:36.025 INFO:tasks.workunit.client.0.smithi046.stdout:6/878: chown da/db/f16 32 1 2024-09-10T04:05:36.026 INFO:tasks.workunit.client.1.smithi070.stdout:1/994: creat d3/f144 x:0 0 0 2024-09-10T04:05:36.026 INFO:tasks.workunit.client.0.smithi046.stdout:0/767: sync 2024-09-10T04:05:36.030 INFO:tasks.workunit.client.0.smithi046.stdout:6/879: rename da/db/d11/d13/d1e/d23/d54/dad/c105 to da/db/d11/d9f/dd6/c119 0 2024-09-10T04:05:36.032 INFO:tasks.workunit.client.0.smithi046.stdout:0/768: rmdir d1/dc/d25/d33/d3/d29 39 2024-09-10T04:05:36.032 INFO:tasks.workunit.client.0.smithi046.stdout:0/769: readlink d1/dc/d25/d33/d1a/l9f 0 2024-09-10T04:05:36.035 INFO:tasks.workunit.client.0.smithi046.stdout:0/770: fsync d1/dc/f76 0 2024-09-10T04:05:36.036 INFO:tasks.workunit.client.0.smithi046.stdout:0/771: chown d1/dc/d25/d33/d3/dd/d55/d77/da6/ccc 45655 1 2024-09-10T04:05:36.037 INFO:tasks.workunit.client.0.smithi046.stdout:8/990: getdents d1/d29/d17/d1f/d46/d67/d116/d149 0 2024-09-10T04:05:36.038 INFO:tasks.workunit.client.0.smithi046.stdout:6/880: link da/db/d10/fac da/db/d18/dd1/f11a 0 2024-09-10T04:05:36.043 INFO:tasks.workunit.client.0.smithi046.stdout:6/881: rename da/db/d18/f49 to da/d9b/f11b 0 2024-09-10T04:05:36.045 INFO:tasks.workunit.client.0.smithi046.stdout:8/991: dwrite d1/d29/d17/d1f/d46/d6a/f94 [0,4194304] 0 2024-09-10T04:05:36.048 INFO:tasks.workunit.client.0.smithi046.stdout:0/772: dwrite d1/dc/d25/d33/d3/d29/d2d/d5e/f89 [0,4194304] 0 2024-09-10T04:05:36.053 INFO:tasks.workunit.client.0.smithi046.stdout:6/882: dwrite f6 [4194304,4194304] 0 2024-09-10T04:05:36.060 INFO:tasks.workunit.client.0.smithi046.stdout:6/883: rmdir da/db/d10/d27 39 2024-09-10T04:05:36.063 INFO:tasks.workunit.client.0.smithi046.stdout:9/963: truncate d4/d7/d1f/f15 1148485 0 2024-09-10T04:05:36.065 INFO:tasks.workunit.client.0.smithi046.stdout:9/964: unlink d4/f99 0 2024-09-10T04:05:36.067 INFO:tasks.workunit.client.0.smithi046.stdout:6/884: fdatasync da/db/d11/d2b/d5a/f5c 0 2024-09-10T04:05:36.068 INFO:tasks.workunit.client.0.smithi046.stdout:1/766: sync 2024-09-10T04:05:36.068 INFO:tasks.workunit.client.1.smithi070.stdout:1/995: dwrite d3/d8/d13/faf [0,4194304] 0 2024-09-10T04:05:36.072 INFO:tasks.workunit.client.0.smithi046.stdout:6/885: fdatasync da/db/d11/d13/d1e/d23/f104 0 2024-09-10T04:05:36.074 INFO:tasks.workunit.client.0.smithi046.stdout:6/886: chown da/db/d11/d2b/d2f/f74 18 1 2024-09-10T04:05:36.074 INFO:tasks.workunit.client.0.smithi046.stdout:6/887: stat da/de3/lee 0 2024-09-10T04:05:36.074 INFO:tasks.workunit.client.1.smithi070.stdout:1/996: creat d3/d10/db2/dc7/d54/f145 x:0 0 0 2024-09-10T04:05:36.076 INFO:tasks.workunit.client.1.smithi070.stdout:1/997: write d3/d8/d27/d9a/f130 [950880,8613] 0 2024-09-10T04:05:36.077 INFO:tasks.workunit.client.0.smithi046.stdout:9/965: mkdir d4/d29/d131 0 2024-09-10T04:05:36.082 INFO:tasks.workunit.client.0.smithi046.stdout:8/992: dwrite d1/d29/f27 [0,4194304] 0 2024-09-10T04:05:36.082 INFO:tasks.workunit.client.1.smithi070.stdout:1/998: mkdir d3/d8/d117/d138/d146 0 2024-09-10T04:05:36.083 INFO:tasks.workunit.client.1.smithi070.stdout:1/999: readlink d3/d8/d13/d67/d83/lc9 0 2024-09-10T04:05:36.083 INFO:tasks.workunit.client.0.smithi046.stdout:0/773: dwrite d1/dc/d25/d33/d1a/d80/f8a [0,4194304] 0 2024-09-10T04:05:36.084 INFO:tasks.workunit.client.0.smithi046.stdout:8/993: chown d1/f39 0 1 2024-09-10T04:05:36.086 INFO:tasks.workunit.client.0.smithi046.stdout:1/767: link d3/d27/c5e d3/d9b/dcb/c104 0 2024-09-10T04:05:36.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:35 smithi046 ceph-mon[28328]: pgmap v271: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 227 MiB/s wr, 490 op/s 2024-09-10T04:05:36.087 INFO:tasks.workunit.client.0.smithi046.stdout:6/888: truncate da/db/d11/f93 564033 0 2024-09-10T04:05:36.088 INFO:tasks.workunit.client.1.smithi070.stderr:+ rm -rf -- ./tmp.2QsmQqCNVi 2024-09-10T04:05:36.091 INFO:tasks.workunit.client.0.smithi046.stdout:8/994: rmdir d1/d29/d19/db7 39 2024-09-10T04:05:36.092 INFO:tasks.workunit.client.0.smithi046.stdout:0/774: stat d1/dc/d25/d33/d1a/d41/f84 0 2024-09-10T04:05:36.094 INFO:tasks.workunit.client.0.smithi046.stdout:1/768: rename d3/d62/faa to d3/d27/d5d/d60/dd7/df9/f105 0 2024-09-10T04:05:36.095 INFO:tasks.workunit.client.0.smithi046.stdout:1/769: truncate d3/f4a 5126475 0 2024-09-10T04:05:36.096 INFO:tasks.workunit.client.0.smithi046.stdout:9/966: creat d4/d7/f132 x:0 0 0 2024-09-10T04:05:36.096 INFO:tasks.workunit.client.0.smithi046.stdout:1/770: rename d3/d27 to d3/d27/df1/d106 22 2024-09-10T04:05:36.096 INFO:tasks.workunit.client.0.smithi046.stdout:9/967: readlink d4/l1d 0 2024-09-10T04:05:36.097 INFO:tasks.workunit.client.0.smithi046.stdout:1/771: readlink d3/d9b/l9d 0 2024-09-10T04:05:36.098 INFO:tasks.workunit.client.0.smithi046.stdout:6/889: mknod da/db/d11/d13/d1e/d9d/c11c 0 2024-09-10T04:05:36.098 INFO:tasks.workunit.client.0.smithi046.stdout:9/968: write d4/d3c/d4d/d80/f11d [939739,86707] 0 2024-09-10T04:05:36.098 INFO:tasks.workunit.client.0.smithi046.stdout:1/772: write d3/d9b/fbc [3483555,75442] 0 2024-09-10T04:05:36.099 INFO:tasks.workunit.client.0.smithi046.stdout:6/890: fsync da/db/d63/fa2 0 2024-09-10T04:05:36.100 INFO:tasks.workunit.client.0.smithi046.stdout:9/969: stat d4/d3c/de0/l100 0 2024-09-10T04:05:36.101 INFO:tasks.workunit.client.0.smithi046.stdout:0/775: fdatasync d1/dc/d25/d33/d3/dd/d55/d4d/f71 0 2024-09-10T04:05:36.101 INFO:tasks.workunit.client.0.smithi046.stdout:6/891: dread - da/db/d18/f106 zero size 2024-09-10T04:05:36.105 INFO:tasks.workunit.client.0.smithi046.stdout:1/773: dread d3/da1/d1f/fb0 [0,4194304] 0 2024-09-10T04:05:36.106 INFO:tasks.workunit.client.0.smithi046.stdout:1/774: chown d3/d27/ffb 60845585 1 2024-09-10T04:05:36.107 INFO:tasks.workunit.client.0.smithi046.stdout:1/775: write d3/dc/f84 [351216,63528] 0 2024-09-10T04:05:36.110 INFO:tasks.workunit.client.0.smithi046.stdout:0/776: symlink d1/dc/d25/lf0 0 2024-09-10T04:05:36.111 INFO:tasks.workunit.client.0.smithi046.stdout:6/892: creat da/dc/da5/f11d x:0 0 0 2024-09-10T04:05:36.111 INFO:tasks.workunit.client.0.smithi046.stdout:9/970: truncate d4/d29/fd2 434259 0 2024-09-10T04:05:36.112 INFO:tasks.workunit.client.0.smithi046.stdout:9/971: write d4/d18/f4f [5050015,36803] 0 2024-09-10T04:05:36.114 INFO:tasks.workunit.client.0.smithi046.stdout:9/972: fsync d4/d7/d1f/feb 0 2024-09-10T04:05:36.114 INFO:tasks.workunit.client.0.smithi046.stdout:1/776: creat d3/d9b/f107 x:0 0 0 2024-09-10T04:05:36.114 INFO:tasks.workunit.client.0.smithi046.stdout:8/995: link d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/d4c/dd3/d10c/l10f d1/d29/d19/d73/ddc/l14e 0 2024-09-10T04:05:36.116 INFO:tasks.workunit.client.0.smithi046.stdout:6/893: creat da/db/d10/d51/f11e x:0 0 0 2024-09-10T04:05:36.119 INFO:tasks.workunit.client.0.smithi046.stdout:1/777: unlink d3/dc/d34/d58/lbd 0 2024-09-10T04:05:36.122 INFO:tasks.workunit.client.0.smithi046.stdout:8/996: symlink d1/d29/d17/d1f/d2a/df6/d102/d12e/d147/d4c/dd3/d10c/l14f 0 2024-09-10T04:05:36.127 INFO:tasks.workunit.client.0.smithi046.stdout:8/997: unlink d1/d29/d17/d1f/d46/d67/d116/d149/cf7 0 2024-09-10T04:05:36.128 INFO:tasks.workunit.client.0.smithi046.stdout:0/777: getdents d1/dc/d25/d33/d4/d19 0 2024-09-10T04:05:36.128 INFO:tasks.workunit.client.0.smithi046.stdout:9/973: dread d4/d7/f1e [0,4194304] 0 2024-09-10T04:05:36.129 INFO:tasks.workunit.client.0.smithi046.stdout:0/778: chown d1/dc/d25/d33/d3/dd/l20 487200 1 2024-09-10T04:05:36.143 INFO:tasks.workunit.client.0.smithi046.stdout:9/974: mkdir d4/d29/d10e/d133 0 2024-09-10T04:05:36.144 INFO:tasks.workunit.client.0.smithi046.stdout:6/894: write da/db/f16 [1140194,37897] 0 2024-09-10T04:05:36.144 INFO:tasks.workunit.client.0.smithi046.stdout:8/998: write d1/f39 [1236870,124124] 0 2024-09-10T04:05:36.146 INFO:tasks.workunit.client.0.smithi046.stdout:8/999: readlink d1/d29/d19/l53 0 2024-09-10T04:05:36.148 INFO:tasks.workunit.client.0.smithi046.stdout:0/779: dread d1/dc/d25/d33/d1a/d80/f8a [0,4194304] 0 2024-09-10T04:05:36.151 INFO:tasks.workunit.client.0.smithi046.stdout:0/780: creat d1/dc/d25/d33/d4/ff1 x:0 0 0 2024-09-10T04:05:36.152 INFO:tasks.workunit.client.0.smithi046.stdout:6/895: mknod da/db/d11/d2b/d55/c11f 0 2024-09-10T04:05:36.154 INFO:tasks.workunit.client.0.smithi046.stdout:6/896: truncate da/db/d11/d2b/d2f/f37 1420578 0 2024-09-10T04:05:36.154 INFO:tasks.workunit.client.0.smithi046.stdout:0/781: rmdir d1/dc/d25/d33/d3/d29 39 2024-09-10T04:05:36.155 INFO:tasks.workunit.client.0.smithi046.stdout:9/975: link d4/d7/d1f/d54/f6e d4/d18/d89/ddb/f134 0 2024-09-10T04:05:36.155 INFO:tasks.workunit.client.0.smithi046.stdout:0/782: dread - d1/dc/d25/d33/d4/fd8 zero size 2024-09-10T04:05:36.156 INFO:tasks.workunit.client.0.smithi046.stdout:6/897: mkdir da/d9b/d120 0 2024-09-10T04:05:36.158 INFO:tasks.workunit.client.0.smithi046.stdout:0/783: creat d1/d13/d37/ff2 x:0 0 0 2024-09-10T04:05:36.160 INFO:tasks.workunit.client.0.smithi046.stdout:9/976: rename d4/d7/d1f/d54/fef to d4/d3c/d4d/d80/f135 0 2024-09-10T04:05:36.160 INFO:tasks.workunit.client.0.smithi046.stdout:0/784: symlink d1/d13/d37/d75/d94/d9a/dcf/lf3 0 2024-09-10T04:05:36.161 INFO:tasks.workunit.client.0.smithi046.stdout:9/977: read - d4/d29/d65/ff6 zero size 2024-09-10T04:05:36.161 INFO:tasks.workunit.client.0.smithi046.stdout:6/898: rmdir da/d59 39 2024-09-10T04:05:36.163 INFO:tasks.workunit.client.0.smithi046.stdout:0/785: rename d1/f35 to d1/dc/d25/d72/ff4 0 2024-09-10T04:05:36.166 INFO:tasks.workunit.client.0.smithi046.stdout:6/899: creat da/db/d10/d27/f121 x:0 0 0 2024-09-10T04:05:36.167 INFO:tasks.workunit.client.0.smithi046.stdout:0/786: mkdir d1/d13/d37/d75/d94/d9a/dd7/df5 0 2024-09-10T04:05:36.167 INFO:tasks.workunit.client.0.smithi046.stdout:9/978: truncate d4/d7/d1f/d54/dc7/dc8/f127 711441 0 2024-09-10T04:05:36.171 INFO:tasks.workunit.client.0.smithi046.stdout:6/900: truncate da/db/d11/d13/fda 366979 0 2024-09-10T04:05:36.173 INFO:tasks.workunit.client.0.smithi046.stdout:9/979: creat d4/d7/d61/d10a/f136 x:0 0 0 2024-09-10T04:05:36.173 INFO:tasks.workunit.client.0.smithi046.stdout:0/787: dwrite d1/dc/f76 [0,4194304] 0 2024-09-10T04:05:36.175 INFO:tasks.workunit.client.0.smithi046.stdout:6/901: unlink da/d59/f85 0 2024-09-10T04:05:36.179 INFO:tasks.workunit.client.0.smithi046.stdout:0/788: fsync d1/dc/f95 0 2024-09-10T04:05:36.180 INFO:tasks.workunit.client.0.smithi046.stdout:6/902: rename da/db/d63 to da/db/d11/d2b/d2f/dde/d122 0 2024-09-10T04:05:36.194 INFO:tasks.workunit.client.0.smithi046.stdout:6/903: creat da/db/d11/d13/d1e/d23/d86/f123 x:0 0 0 2024-09-10T04:05:36.195 INFO:tasks.workunit.client.0.smithi046.stdout:6/904: chown da/db/d11/d2b/l46 49139 1 2024-09-10T04:05:36.195 INFO:tasks.workunit.client.0.smithi046.stdout:6/905: stat da/cf5 0 2024-09-10T04:05:36.200 INFO:tasks.workunit.client.0.smithi046.stdout:9/980: getdents d4/d18/d89/ddb 0 2024-09-10T04:05:36.201 INFO:tasks.workunit.client.0.smithi046.stdout:9/981: chown d4/d3c/de0/d95 840 1 2024-09-10T04:05:36.202 INFO:tasks.workunit.client.0.smithi046.stdout:6/906: dwrite da/db/d11/d13/d1e/d23/d86/dbf/ff3 [0,4194304] 0 2024-09-10T04:05:36.206 INFO:tasks.workunit.client.0.smithi046.stdout:9/982: stat d4/d3c/f84 0 2024-09-10T04:05:36.213 INFO:tasks.workunit.client.0.smithi046.stdout:6/907: symlink da/db/d18/l124 0 2024-09-10T04:05:36.218 INFO:tasks.workunit.client.0.smithi046.stdout:0/789: write d1/dc/d25/d33/f11 [2023815,76774] 0 2024-09-10T04:05:36.225 INFO:tasks.workunit.client.0.smithi046.stdout:0/790: creat d1/dc/d25/d33/ff6 x:0 0 0 2024-09-10T04:05:36.227 INFO:tasks.workunit.client.0.smithi046.stdout:6/908: dread da/db/d11/d2b/d5a/ff4 [4194304,4194304] 0 2024-09-10T04:05:36.228 INFO:tasks.workunit.client.0.smithi046.stdout:9/983: dwrite d4/d3c/d4d/d5c/d59/f7f [0,4194304] 0 2024-09-10T04:05:36.234 INFO:tasks.workunit.client.0.smithi046.stdout:6/909: symlink da/db/d11/d13/d1e/d23/d54/l125 0 2024-09-10T04:05:36.236 INFO:tasks.workunit.client.0.smithi046.stdout:9/984: creat d4/d7/d1f/dce/f137 x:0 0 0 2024-09-10T04:05:36.237 INFO:tasks.workunit.client.0.smithi046.stdout:0/791: creat d1/d13/d37/d75/d94/d9a/dd7/df5/ff7 x:0 0 0 2024-09-10T04:05:36.238 INFO:tasks.workunit.client.0.smithi046.stdout:9/985: write d4/d3c/d4d/d5c/d59/f7f [4120977,57889] 0 2024-09-10T04:05:36.241 INFO:tasks.workunit.client.0.smithi046.stdout:0/792: symlink d1/d13/d37/d75/lf8 0 2024-09-10T04:05:36.243 INFO:tasks.workunit.client.0.smithi046.stdout:9/986: creat d4/d7/d1f/dce/f138 x:0 0 0 2024-09-10T04:05:36.247 INFO:tasks.workunit.client.0.smithi046.stdout:0/793: symlink d1/d13/d37/d75/d94/d9a/dd7/df5/lf9 0 2024-09-10T04:05:36.250 INFO:tasks.workunit.client.0.smithi046.stdout:6/910: truncate da/db/d11/d2b/d2f/dde/d122/fa0 2599889 0 2024-09-10T04:05:36.250 INFO:tasks.workunit.client.0.smithi046.stdout:9/987: read d4/d18/d89/fed [3829987,124260] 0 2024-09-10T04:05:36.251 INFO:tasks.workunit.client.0.smithi046.stdout:6/911: fdatasync da/db/d11/dd7/f52 0 2024-09-10T04:05:36.252 INFO:tasks.workunit.client.0.smithi046.stdout:0/794: creat d1/dc/d49/db2/dc3/ffa x:0 0 0 2024-09-10T04:05:36.255 INFO:tasks.workunit.client.0.smithi046.stdout:9/988: mknod d4/d7/d1f/d54/c139 0 2024-09-10T04:05:36.268 INFO:tasks.workunit.client.0.smithi046.stdout:9/989: mknod d4/d18/d92/c13a 0 2024-09-10T04:05:36.271 INFO:tasks.workunit.client.0.smithi046.stdout:9/990: unlink d4/d7/d61/l71 0 2024-09-10T04:05:36.277 INFO:tasks.workunit.client.0.smithi046.stdout:9/991: mkdir d4/d29/d131/d13b 0 2024-09-10T04:05:36.279 INFO:tasks.workunit.client.0.smithi046.stdout:6/912: dwrite da/db/d10/f1a [0,4194304] 0 2024-09-10T04:05:36.282 INFO:tasks.workunit.client.0.smithi046.stdout:0/795: dwrite d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:36.295 INFO:tasks.workunit.client.0.smithi046.stdout:6/913: unlink da/db/d11/d13/d1e/f4d 0 2024-09-10T04:05:36.297 INFO:tasks.workunit.client.0.smithi046.stdout:9/992: unlink d4/d18/d92/c13a 0 2024-09-10T04:05:36.301 INFO:tasks.workunit.client.0.smithi046.stdout:9/993: rename d4/d3c/d4d/d5c/d87 to d4/d18/d89/ddb/dee/d109/d13c 0 2024-09-10T04:05:36.303 INFO:tasks.workunit.client.0.smithi046.stdout:6/914: creat da/db/d10/dc9/f126 x:0 0 0 2024-09-10T04:05:36.304 INFO:tasks.workunit.client.0.smithi046.stdout:9/994: mkdir d4/d7/d1f/dce/dd4/d13d 0 2024-09-10T04:05:36.306 INFO:tasks.workunit.client.0.smithi046.stdout:9/995: chown d4/d18/c39 92 1 2024-09-10T04:05:36.308 INFO:tasks.workunit.client.0.smithi046.stdout:0/796: dwrite d1/d13/d37/d75/f8b [0,4194304] 0 2024-09-10T04:05:36.310 INFO:tasks.workunit.client.0.smithi046.stdout:0/797: stat d1/dc/d25/d33/d3/dd/d55/f43 0 2024-09-10T04:05:36.310 INFO:tasks.workunit.client.0.smithi046.stdout:9/996: creat d4/d7/d1f/dce/dd4/f13e x:0 0 0 2024-09-10T04:05:36.311 INFO:tasks.workunit.client.0.smithi046.stdout:6/915: truncate da/db/d11/d2b/f118 1572764 0 2024-09-10T04:05:36.311 INFO:tasks.workunit.client.0.smithi046.stdout:0/798: stat d1/dc/d25/d33/d3/dd 0 2024-09-10T04:05:36.313 INFO:tasks.workunit.client.0.smithi046.stdout:0/799: fdatasync d1/dc/d25/d33/f3f 0 2024-09-10T04:05:36.313 INFO:tasks.workunit.client.0.smithi046.stdout:6/916: rmdir da/db/d11/d13/d1e/d23/d54 39 2024-09-10T04:05:36.318 INFO:tasks.workunit.client.0.smithi046.stdout:1/778: sync 2024-09-10T04:05:36.319 INFO:tasks.workunit.client.0.smithi046.stdout:6/917: dread da/db/d11/f93 [0,4194304] 0 2024-09-10T04:05:36.321 INFO:tasks.workunit.client.0.smithi046.stdout:0/800: rmdir d1/dc/d25/d33/d3/d29 39 2024-09-10T04:05:36.322 INFO:tasks.workunit.client.0.smithi046.stdout:9/997: link d4/d18/d89/ddb/dee/d109/d13c/fbe d4/d7/d1f/d54/dc7/f13f 0 2024-09-10T04:05:36.326 INFO:tasks.workunit.client.0.smithi046.stdout:9/998: mknod d4/d7/d1f/dce/c140 0 2024-09-10T04:05:36.330 INFO:tasks.workunit.client.0.smithi046.stdout:1/779: truncate d3/dc/f17 2726138 0 2024-09-10T04:05:36.335 INFO:tasks.workunit.client.0.smithi046.stdout:0/801: dread d1/dc/d25/d33/f3f [0,4194304] 0 2024-09-10T04:05:36.336 INFO:tasks.workunit.client.0.smithi046.stdout:6/918: dread da/d9b/f11b [0,4194304] 0 2024-09-10T04:05:36.339 INFO:tasks.workunit.client.0.smithi046.stdout:0/802: creat d1/dc/d25/d33/d1a/d80/ffb x:0 0 0 2024-09-10T04:05:36.341 INFO:tasks.workunit.client.0.smithi046.stdout:6/919: creat da/db/d11/d13/d1e/d69/ddf/f127 x:0 0 0 2024-09-10T04:05:36.347 INFO:tasks.workunit.client.0.smithi046.stdout:0/803: rename d1/dc/d25/d33/d3/d29/l5d to d1/dc/d25/d33/d3/dd/d55/d77/lfc 0 2024-09-10T04:05:36.348 INFO:tasks.workunit.client.0.smithi046.stdout:6/920: dwrite da/db/d11/d13/d1e/f82 [0,4194304] 0 2024-09-10T04:05:36.351 INFO:tasks.workunit.client.0.smithi046.stdout:0/804: fdatasync d1/d13/d37/f6a 0 2024-09-10T04:05:36.354 INFO:tasks.workunit.client.0.smithi046.stdout:6/921: rename c4 to da/db/d10d/c128 0 2024-09-10T04:05:36.378 INFO:tasks.workunit.client.0.smithi046.stdout:0/805: write d1/d13/d37/d75/d94/d9a/fbd [1999908,67799] 0 2024-09-10T04:05:36.382 INFO:tasks.workunit.client.0.smithi046.stdout:0/806: truncate d1/dc/d25/d33/d3/d29/d2d/d5e/f89 4658047 0 2024-09-10T04:05:36.383 INFO:tasks.workunit.client.0.smithi046.stdout:6/922: dwrite da/db/d10/d27/f29 [0,4194304] 0 2024-09-10T04:05:36.385 INFO:tasks.workunit.client.0.smithi046.stdout:6/923: chown da/db/d11 2 1 2024-09-10T04:05:36.391 INFO:tasks.workunit.client.0.smithi046.stdout:0/807: dwrite d1/d13/d37/d75/f7f [0,4194304] 0 2024-09-10T04:05:36.395 INFO:tasks.workunit.client.0.smithi046.stdout:0/808: symlink d1/dc/d25/d33/d3/d29/d2d/lfd 0 2024-09-10T04:05:36.402 INFO:tasks.workunit.client.0.smithi046.stdout:6/924: write da/db/d11/dd7/f34 [1993344,96773] 0 2024-09-10T04:05:36.409 INFO:tasks.workunit.client.0.smithi046.stdout:0/809: dread d1/dc/d25/d33/d3/dd/d55/d4d/f71 [4194304,4194304] 0 2024-09-10T04:05:36.414 INFO:tasks.workunit.client.0.smithi046.stdout:0/810: dread d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi046.stdout:6/925: dread da/db/d11/d13/d1e/f3e [0,4194304] 0 2024-09-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi046.stdout:0/811: chown d1/dc/d25/lf0 1 1 2024-09-10T04:05:36.415 INFO:tasks.workunit.client.0.smithi046.stdout:0/812: chown d1/dc/d49/fbe 3 1 2024-09-10T04:05:36.417 INFO:tasks.workunit.client.0.smithi046.stdout:0/813: write d1/dc/d25/d33/f11 [2347085,123507] 0 2024-09-10T04:05:36.419 INFO:tasks.workunit.client.0.smithi046.stdout:6/926: dread da/f60 [0,4194304] 0 2024-09-10T04:05:36.419 INFO:tasks.workunit.client.0.smithi046.stdout:6/927: chown da/de3/lee 52 1 2024-09-10T04:05:36.421 INFO:tasks.workunit.client.0.smithi046.stdout:0/814: creat d1/dc/d49/db2/dc3/ffe x:0 0 0 2024-09-10T04:05:36.426 INFO:tasks.workunit.client.0.smithi046.stdout:6/928: truncate da/db/d11/dd7/f34 1192135 0 2024-09-10T04:05:36.436 INFO:tasks.workunit.client.0.smithi046.stdout:6/929: dread da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:36.560 INFO:tasks.workunit.client.0.smithi046.stdout:6/930: rename da/db/d11/d13/d1e/d23/d86/dbf/c103 to da/db/d11/d13/d1e/c129 0 2024-09-10T04:05:36.563 INFO:tasks.workunit.client.0.smithi046.stdout:6/931: readlink da/db/d18/dd1/le0 0 2024-09-10T04:05:36.565 INFO:tasks.workunit.client.0.smithi046.stdout:6/932: mknod da/db/d10/d27/c12a 0 2024-09-10T04:05:36.567 INFO:tasks.workunit.client.0.smithi046.stdout:6/933: symlink da/db/d11/d9f/dd6/de9/l12b 0 2024-09-10T04:05:36.574 INFO:tasks.workunit.client.0.smithi046.stdout:6/934: write f1 [561486,117829] 0 2024-09-10T04:05:36.578 INFO:tasks.workunit.client.0.smithi046.stdout:6/935: rename da/db/d10/d27/l2d to da/d59/l12c 0 2024-09-10T04:05:36.580 INFO:tasks.workunit.client.0.smithi046.stdout:6/936: rmdir da/db/d11/d9f/dd6/de9 39 2024-09-10T04:05:36.583 INFO:tasks.workunit.client.0.smithi046.stdout:6/937: creat da/db/d11/d2b/d5a/f12d x:0 0 0 2024-09-10T04:05:36.584 INFO:tasks.workunit.client.0.smithi046.stdout:6/938: chown da/db/d11/d9f/dd6/df8 359616017 1 2024-09-10T04:05:36.585 INFO:tasks.workunit.client.0.smithi046.stdout:9/999: sync 2024-09-10T04:05:36.586 INFO:tasks.workunit.client.0.smithi046.stdout:6/939: symlink da/d9b/d120/l12e 0 2024-09-10T04:05:36.597 INFO:tasks.workunit.client.0.smithi046.stdout:6/940: dwrite da/db/d11/d2b/d55/f81 [0,4194304] 0 2024-09-10T04:05:36.601 INFO:tasks.workunit.client.0.smithi046.stdout:6/941: symlink da/db/d11/d13/d1e/l12f 0 2024-09-10T04:05:36.605 INFO:tasks.workunit.client.0.smithi046.stdout:6/942: rename da/db/d10/d27/f10e to da/f130 0 2024-09-10T04:05:36.606 INFO:tasks.workunit.client.0.smithi046.stdout:6/943: chown da/d9b/d120/l12e 71 1 2024-09-10T04:05:36.610 INFO:tasks.workunit.client.0.smithi046.stdout:6/944: creat da/db/d11/d2b/f131 x:0 0 0 2024-09-10T04:05:36.613 INFO:tasks.workunit.client.0.smithi046.stdout:6/945: creat da/db/d18/dd1/f132 x:0 0 0 2024-09-10T04:05:36.615 INFO:tasks.workunit.client.0.smithi046.stdout:6/946: mknod da/db/d11/d13/d1e/d23/c133 0 2024-09-10T04:05:36.616 INFO:tasks.workunit.client.0.smithi046.stdout:6/947: write da/db/d11/d2b/f131 [640243,53051] 0 2024-09-10T04:05:36.621 INFO:tasks.workunit.client.0.smithi046.stdout:6/948: rename da/db/d11/d13/d1e/d23/l8d to da/dc/da5/l134 0 2024-09-10T04:05:36.626 INFO:tasks.workunit.client.0.smithi046.stdout:6/949: dwrite da/db/d10/f1a [0,4194304] 0 2024-09-10T04:05:36.627 INFO:tasks.workunit.client.0.smithi046.stdout:1/780: sync 2024-09-10T04:05:36.629 INFO:tasks.workunit.client.0.smithi046.stdout:6/950: dread da/db/d11/d2b/d2f/d3b/fc3 [0,4194304] 0 2024-09-10T04:05:36.635 INFO:tasks.workunit.client.0.smithi046.stdout:6/951: getdents da/db/d11/dd7 0 2024-09-10T04:05:36.636 INFO:tasks.workunit.client.0.smithi046.stdout:6/952: dread - da/db/d10/d27/f121 zero size 2024-09-10T04:05:36.640 INFO:tasks.workunit.client.0.smithi046.stdout:6/953: link da/db/d11/d9f/fed da/db/d10/d51/f135 0 2024-09-10T04:05:36.645 INFO:tasks.workunit.client.0.smithi046.stdout:1/781: write d3/dc/d34/d51/f78 [1152568,44580] 0 2024-09-10T04:05:36.647 INFO:tasks.workunit.client.0.smithi046.stdout:6/954: dwrite da/db/d11/d13/f39 [0,4194304] 0 2024-09-10T04:05:36.654 INFO:tasks.workunit.client.0.smithi046.stdout:1/782: creat d3/d27/f108 x:0 0 0 2024-09-10T04:05:36.656 INFO:tasks.workunit.client.0.smithi046.stdout:1/783: mkdir d3/dc/d109 0 2024-09-10T04:05:36.657 INFO:tasks.workunit.client.0.smithi046.stdout:1/784: dread - d3/d9b/f100 zero size 2024-09-10T04:05:36.659 INFO:tasks.workunit.client.0.smithi046.stdout:1/785: chown d3/da1/da7/fcf 15041612 1 2024-09-10T04:05:36.661 INFO:tasks.workunit.client.0.smithi046.stdout:1/786: mkdir d3/d27/d10a 0 2024-09-10T04:05:36.663 INFO:tasks.workunit.client.0.smithi046.stdout:6/955: dread da/db/d10/d27/f92 [0,4194304] 0 2024-09-10T04:05:36.664 INFO:tasks.workunit.client.0.smithi046.stdout:1/787: unlink d3/d27/c5e 0 2024-09-10T04:05:36.665 INFO:tasks.workunit.client.0.smithi046.stdout:6/956: rmdir da/db/d11/d13/d1e/d9d 39 2024-09-10T04:05:36.668 INFO:tasks.workunit.client.0.smithi046.stdout:0/815: sync 2024-09-10T04:05:36.669 INFO:tasks.workunit.client.0.smithi046.stdout:1/788: rmdir d3/d27/d10a 0 2024-09-10T04:05:36.672 INFO:tasks.workunit.client.0.smithi046.stdout:6/957: dwrite da/db/d11/d13/f39 [0,4194304] 0 2024-09-10T04:05:36.674 INFO:tasks.workunit.client.0.smithi046.stdout:1/789: mknod d3/d27/d2a/c10b 0 2024-09-10T04:05:36.674 INFO:tasks.workunit.client.0.smithi046.stdout:0/816: dread d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:36.680 INFO:tasks.workunit.client.0.smithi046.stdout:0/817: symlink d1/dc/d25/d33/d3/d29/d2d/d5e/lff 0 2024-09-10T04:05:36.692 INFO:tasks.workunit.client.0.smithi046.stdout:1/790: dread d3/d27/d5d/d60/fc0 [0,4194304] 0 2024-09-10T04:05:36.693 INFO:tasks.workunit.client.0.smithi046.stdout:1/791: write d3/da1/f69 [1482029,57400] 0 2024-09-10T04:05:36.697 INFO:tasks.workunit.client.0.smithi046.stdout:0/818: dread d1/d13/f53 [0,4194304] 0 2024-09-10T04:05:36.698 INFO:tasks.workunit.client.0.smithi046.stdout:6/958: write da/db/d10/d51/f80 [2418078,42553] 0 2024-09-10T04:05:36.701 INFO:tasks.workunit.client.0.smithi046.stdout:1/792: truncate d3/d27/d5d/fad 3580715 0 2024-09-10T04:05:36.703 INFO:tasks.workunit.client.0.smithi046.stdout:0/819: write d1/dc/d25/f9e [783163,38048] 0 2024-09-10T04:05:36.703 INFO:tasks.workunit.client.0.smithi046.stdout:6/959: chown da/dc/c75 16472544 1 2024-09-10T04:05:36.704 INFO:tasks.workunit.client.0.smithi046.stdout:0/820: chown d1/dc/d49/f4a 26 1 2024-09-10T04:05:36.707 INFO:tasks.workunit.client.0.smithi046.stdout:6/960: creat da/db/d11/d2b/d2f/dde/f136 x:0 0 0 2024-09-10T04:05:36.714 INFO:tasks.workunit.client.0.smithi046.stdout:6/961: mkdir da/db/d137 0 2024-09-10T04:05:36.715 INFO:tasks.workunit.client.0.smithi046.stdout:1/793: dwrite d3/da1/d1f/f3c [0,4194304] 0 2024-09-10T04:05:36.717 INFO:tasks.workunit.client.0.smithi046.stdout:6/962: stat da/db/d11/d2b/d5a/dc2/fb9 0 2024-09-10T04:05:36.720 INFO:tasks.workunit.client.0.smithi046.stdout:0/821: dread d1/dc/d25/d72/fb9 [0,4194304] 0 2024-09-10T04:05:36.721 INFO:tasks.workunit.client.0.smithi046.stdout:0/822: chown d1/dc/d25/d33/d3/d29 123 1 2024-09-10T04:05:36.730 INFO:tasks.workunit.client.0.smithi046.stdout:0/823: dread d1/dc/d25/d33/d3/dd/d55/d4d/f4e [0,4194304] 0 2024-09-10T04:05:36.730 INFO:tasks.workunit.client.0.smithi046.stdout:1/794: dwrite d3/dc/f17 [0,4194304] 0 2024-09-10T04:05:36.731 INFO:tasks.workunit.client.0.smithi046.stdout:6/963: rename da/db/d18/lbb to da/l138 0 2024-09-10T04:05:36.733 INFO:tasks.workunit.client.0.smithi046.stdout:0/824: symlink d1/dc/de9/l100 0 2024-09-10T04:05:36.734 INFO:tasks.workunit.client.0.smithi046.stdout:0/825: write d1/dc/d25/d33/f7e [164349,119542] 0 2024-09-10T04:05:36.736 INFO:tasks.workunit.client.0.smithi046.stdout:6/964: symlink da/db/d11/d9f/dd6/l139 0 2024-09-10T04:05:36.738 INFO:tasks.workunit.client.0.smithi046.stdout:0/826: mknod d1/dc/d49/db2/dcb/c101 0 2024-09-10T04:05:36.744 INFO:tasks.workunit.client.0.smithi046.stdout:0/827: mkdir d1/dc/d25/d33/d3/dd/d55/d4d/d102 0 2024-09-10T04:05:36.746 INFO:tasks.workunit.client.0.smithi046.stdout:0/828: read d1/dc/d25/d33/d3/dd/d55/f43 [2919943,60532] 0 2024-09-10T04:05:36.748 INFO:tasks.workunit.client.0.smithi046.stdout:1/795: dread d3/dc/d34/f72 [0,4194304] 0 2024-09-10T04:05:36.750 INFO:tasks.workunit.client.0.smithi046.stdout:0/829: readlink d1/dc/lf 0 2024-09-10T04:05:36.753 INFO:tasks.workunit.client.0.smithi046.stdout:1/796: dread d3/da1/d1f/f3c [0,4194304] 0 2024-09-10T04:05:36.759 INFO:tasks.workunit.client.0.smithi046.stdout:6/965: dwrite da/db/d11/d2b/d2f/f66 [0,4194304] 0 2024-09-10T04:05:36.759 INFO:tasks.workunit.client.0.smithi046.stdout:6/966: chown f9 1749648 1 2024-09-10T04:05:36.761 INFO:tasks.workunit.client.0.smithi046.stdout:1/797: write d3/d9b/dcb/ffa [387982,27982] 0 2024-09-10T04:05:36.763 INFO:tasks.workunit.client.0.smithi046.stdout:1/798: chown d3/f26 2 1 2024-09-10T04:05:36.763 INFO:tasks.workunit.client.0.smithi046.stdout:0/830: rename d1/dc/d25/d72/le2 to d1/dc/d25/d33/l103 0 2024-09-10T04:05:36.766 INFO:tasks.workunit.client.0.smithi046.stdout:1/799: creat d3/d62/f10c x:0 0 0 2024-09-10T04:05:36.767 INFO:tasks.workunit.client.0.smithi046.stdout:6/967: dread da/db/d11/d13/d1e/d23/d86/dbf/f95 [0,4194304] 0 2024-09-10T04:05:36.770 INFO:tasks.workunit.client.0.smithi046.stdout:0/831: rename d1/dc/d25/d33/d3/dd/d55/d4d/c5c to d1/dc/d25/d33/d1a/c104 0 2024-09-10T04:05:36.772 INFO:tasks.workunit.client.0.smithi046.stdout:6/968: write da/d9b/f11b [1309080,70805] 0 2024-09-10T04:05:36.774 INFO:tasks.workunit.client.0.smithi046.stdout:1/800: unlink d3/dc/d34/c82 0 2024-09-10T04:05:36.777 INFO:tasks.workunit.client.0.smithi046.stdout:1/801: unlink d3/dc/d34/ff0 0 2024-09-10T04:05:36.783 INFO:tasks.workunit.client.0.smithi046.stdout:0/832: dread d1/dc/d25/d33/d3/dd/d55/f4c [0,4194304] 0 2024-09-10T04:05:36.785 INFO:tasks.workunit.client.0.smithi046.stdout:6/969: write da/db/d11/d2b/d5a/fd8 [17240,9325] 0 2024-09-10T04:05:36.790 INFO:tasks.workunit.client.0.smithi046.stdout:6/970: dread - da/dc/fcd zero size 2024-09-10T04:05:36.790 INFO:tasks.workunit.client.0.smithi046.stdout:1/802: link d3/da1/d1f/ca5 d3/dc/d21/d3a/d90/dbf/c10d 0 2024-09-10T04:05:36.790 INFO:tasks.workunit.client.0.smithi046.stdout:0/833: dwrite d1/d13/d37/d75/fd0 [4194304,4194304] 0 2024-09-10T04:05:36.793 INFO:tasks.workunit.client.0.smithi046.stdout:1/803: symlink d3/da1/d1f/df4/l10e 0 2024-09-10T04:05:36.794 INFO:tasks.workunit.client.0.smithi046.stdout:0/834: fdatasync d1/dc/d25/d33/d3/d29/d2d/d5e/f89 0 2024-09-10T04:05:36.795 INFO:tasks.workunit.client.0.smithi046.stdout:6/971: creat da/db/d11/d2b/d2f/dde/f13a x:0 0 0 2024-09-10T04:05:36.796 INFO:tasks.workunit.client.0.smithi046.stdout:1/804: chown d3/d27/d5d/d60/dd7/df9/cff 1229 1 2024-09-10T04:05:36.797 INFO:tasks.workunit.client.0.smithi046.stdout:6/972: read - da/db/d11/d2b/d2f/dde/f136 zero size 2024-09-10T04:05:36.798 INFO:tasks.workunit.client.0.smithi046.stdout:6/973: chown da/db/d11/d13 7 1 2024-09-10T04:05:36.801 INFO:tasks.workunit.client.0.smithi046.stdout:1/805: dread d3/d27/fe9 [0,4194304] 0 2024-09-10T04:05:36.801 INFO:tasks.workunit.client.0.smithi046.stdout:6/974: readlink da/d71/la9 0 2024-09-10T04:05:36.802 INFO:tasks.workunit.client.0.smithi046.stdout:0/835: symlink d1/l105 0 2024-09-10T04:05:36.803 INFO:tasks.workunit.client.0.smithi046.stdout:6/975: fdatasync da/d45/fe6 0 2024-09-10T04:05:36.807 INFO:tasks.workunit.client.0.smithi046.stdout:0/836: mkdir d1/d13/d37/d75/d106 0 2024-09-10T04:05:36.808 INFO:tasks.workunit.client.0.smithi046.stdout:1/806: creat d3/d62/df8/f10f x:0 0 0 2024-09-10T04:05:36.820 INFO:tasks.workunit.client.0.smithi046.stdout:1/807: truncate d3/d9b/f9e 928725 0 2024-09-10T04:05:36.821 INFO:tasks.workunit.client.0.smithi046.stdout:0/837: write d1/dc/d25/d33/d3/dd/d55/d77/da6/fb3 [983039,102567] 0 2024-09-10T04:05:36.825 INFO:tasks.workunit.client.0.smithi046.stdout:6/976: dwrite da/db/d11/d9f/dd6/f112 [0,4194304] 0 2024-09-10T04:05:36.830 INFO:tasks.workunit.client.0.smithi046.stdout:1/808: dwrite d3/d9b/f100 [0,4194304] 0 2024-09-10T04:05:36.833 INFO:tasks.workunit.client.0.smithi046.stdout:6/977: symlink da/db/d11/d2b/d2f/dde/d122/l13b 0 2024-09-10T04:05:36.837 INFO:tasks.workunit.client.0.smithi046.stdout:0/838: dwrite d1/dc/d25/d33/d3/d29/fc2 [0,4194304] 0 2024-09-10T04:05:36.837 INFO:tasks.workunit.client.0.smithi046.stdout:6/978: write da/db/d11/d13/f57 [1195763,123714] 0 2024-09-10T04:05:36.840 INFO:tasks.workunit.client.0.smithi046.stdout:0/839: truncate d1/dc/d25/d33/f3f 4899615 0 2024-09-10T04:05:36.846 INFO:tasks.workunit.client.0.smithi046.stdout:0/840: rename d1/dc/d25/d33/d3/d29/l59 to d1/d13/d37/d75/d94/dd4/l107 0 2024-09-10T04:05:36.849 INFO:tasks.workunit.client.0.smithi046.stdout:1/809: dread d3/d9b/f8d [0,4194304] 0 2024-09-10T04:05:36.849 INFO:tasks.workunit.client.0.smithi046.stdout:0/841: write d1/d13/d37/d75/d94/d9a/dcf/fe4 [642974,90289] 0 2024-09-10T04:05:36.853 INFO:tasks.workunit.client.0.smithi046.stdout:0/842: mkdir d1/dc/d25/d33/d3/d29/d108 0 2024-09-10T04:05:36.855 INFO:tasks.workunit.client.0.smithi046.stdout:1/810: rename d3/d9b/ce2 to d3/d27/d5d/d60/dd7/c110 0 2024-09-10T04:05:36.855 INFO:tasks.workunit.client.0.smithi046.stdout:6/979: dwrite da/f60 [0,4194304] 0 2024-09-10T04:05:36.856 INFO:tasks.workunit.client.0.smithi046.stdout:6/980: chown da/dc 327512077 1 2024-09-10T04:05:36.861 INFO:tasks.workunit.client.0.smithi046.stdout:0/843: creat d1/dc/d25/d33/d3/d29/d2d/f109 x:0 0 0 2024-09-10T04:05:36.865 INFO:tasks.workunit.client.0.smithi046.stdout:0/844: creat d1/dc/d49/db2/f10a x:0 0 0 2024-09-10T04:05:36.866 INFO:tasks.workunit.client.0.smithi046.stdout:1/811: link d3/dc/c11 d3/d27/d5d/d60/c111 0 2024-09-10T04:05:36.867 INFO:tasks.workunit.client.0.smithi046.stdout:0/845: read d1/d13/f53 [2290288,129834] 0 2024-09-10T04:05:36.867 INFO:tasks.workunit.client.0.smithi046.stdout:6/981: link da/db/d11/d2b/d2f/d3b/l3d da/d9b/l13c 0 2024-09-10T04:05:36.869 INFO:tasks.workunit.client.0.smithi046.stdout:6/982: stat da/db/d11/d2b/d2f/f65 0 2024-09-10T04:05:36.871 INFO:tasks.workunit.client.0.smithi046.stdout:0/846: creat d1/d13/d37/d75/d94/f10b x:0 0 0 2024-09-10T04:05:36.875 INFO:tasks.workunit.client.0.smithi046.stdout:0/847: creat d1/dc/d25/d33/d3/dcd/f10c x:0 0 0 2024-09-10T04:05:36.875 INFO:tasks.workunit.client.0.smithi046.stdout:1/812: link d3/c1b d3/dc/d21/d3a/d90/dbf/c112 0 2024-09-10T04:05:36.880 INFO:tasks.workunit.client.0.smithi046.stdout:0/848: dread d1/dc/d25/d33/d3/dd/d55/d4d/f71 [4194304,4194304] 0 2024-09-10T04:05:36.881 INFO:tasks.workunit.client.0.smithi046.stdout:0/849: readlink d1/dc/d25/d33/d3/dd/d55/l54 0 2024-09-10T04:05:36.886 INFO:tasks.workunit.client.0.smithi046.stdout:0/850: creat d1/dc/d25/d33/d3/dd/f10d x:0 0 0 2024-09-10T04:05:36.886 INFO:tasks.workunit.client.0.smithi046.stdout:6/983: write da/db/d11/d2b/f6d [713580,16000] 0 2024-09-10T04:05:36.887 INFO:tasks.workunit.client.0.smithi046.stdout:0/851: read - d1/dc/d49/db2/dcb/fe5 zero size 2024-09-10T04:05:36.887 INFO:tasks.workunit.client.0.smithi046.stdout:6/984: chown da/db/d11/dd7/f117 12 1 2024-09-10T04:05:36.888 INFO:tasks.workunit.client.0.smithi046.stdout:6/985: stat da/db/d11/d13/f7c 0 2024-09-10T04:05:36.889 INFO:tasks.workunit.client.0.smithi046.stdout:0/852: write d1/d13/d37/d75/f8b [5118346,30214] 0 2024-09-10T04:05:36.891 INFO:tasks.workunit.client.0.smithi046.stdout:0/853: fdatasync d1/dc/d25/d33/d1a/f97 0 2024-09-10T04:05:36.892 INFO:tasks.workunit.client.0.smithi046.stdout:0/854: chown d1/dc/d49/fa1 20 1 2024-09-10T04:05:36.893 INFO:tasks.workunit.client.0.smithi046.stdout:6/986: fsync da/db/d10/fac 0 2024-09-10T04:05:36.896 INFO:tasks.workunit.client.0.smithi046.stdout:6/987: rmdir da/db/d11/d13/d4f 39 2024-09-10T04:05:36.900 INFO:tasks.workunit.client.0.smithi046.stdout:6/988: dread da/db/d11/d13/f57 [0,4194304] 0 2024-09-10T04:05:36.902 INFO:tasks.workunit.client.0.smithi046.stdout:0/855: link d1/dc/d25/d33/d3/d29/d2d/lfd d1/dc/d25/d33/d1a/d41/l10e 0 2024-09-10T04:05:36.904 INFO:tasks.workunit.client.0.smithi046.stdout:0/856: rmdir d1/dc/d25/d33/d1a 39 2024-09-10T04:05:36.905 INFO:tasks.workunit.client.0.smithi046.stdout:0/857: read - d1/dc/d49/db2/fd2 zero size 2024-09-10T04:05:36.905 INFO:tasks.workunit.client.0.smithi046.stdout:0/858: fsync d1/d13/fa9 0 2024-09-10T04:05:36.908 INFO:tasks.workunit.client.0.smithi046.stdout:0/859: creat d1/dc/d25/d33/d3/dd/d55/d4d/f10f x:0 0 0 2024-09-10T04:05:36.912 INFO:tasks.workunit.client.0.smithi046.stdout:0/860: symlink d1/dc/d25/d33/d1a/l110 0 2024-09-10T04:05:36.917 INFO:tasks.workunit.client.0.smithi046.stdout:0/861: dwrite d1/d13/d37/d75/f8b [0,4194304] 0 2024-09-10T04:05:36.919 INFO:tasks.workunit.client.0.smithi046.stdout:0/862: read d1/d13/d37/f6a [159015,40715] 0 2024-09-10T04:05:36.932 INFO:tasks.workunit.client.0.smithi046.stdout:0/863: dread d1/dc/fe7 [0,4194304] 0 2024-09-10T04:05:36.936 INFO:tasks.workunit.client.0.smithi046.stdout:0/864: getdents d1/d13/d37/d75/d94/d9a 0 2024-09-10T04:05:36.949 INFO:tasks.workunit.client.0.smithi046.stdout:0/865: dread d1/dc/d25/d33/d4/f6f [0,4194304] 0 2024-09-10T04:05:36.953 INFO:tasks.workunit.client.0.smithi046.stdout:0/866: mkdir d1/dc/d25/d111 0 2024-09-10T04:05:36.960 INFO:tasks.workunit.client.0.smithi046.stdout:0/867: unlink d1/dc/d25/d33/d3/dd/d55/cc7 0 2024-09-10T04:05:36.963 INFO:tasks.workunit.client.0.smithi046.stdout:0/868: symlink d1/d13/d37/d75/d94/d9a/da4/l112 0 2024-09-10T04:05:36.969 INFO:tasks.workunit.client.0.smithi046.stdout:0/869: getdents d1/dc 0 2024-09-10T04:05:36.972 INFO:tasks.workunit.client.0.smithi046.stdout:0/870: unlink d1/dc/d25/c2f 0 2024-09-10T04:05:36.973 INFO:tasks.workunit.client.0.smithi046.stdout:0/871: chown d1/dc/d25/d33/d3/d29/d2d/d5e/fde 21585 1 2024-09-10T04:05:36.977 INFO:tasks.workunit.client.0.smithi046.stdout:0/872: rename d1/d13/d37/d75/d94/d9a/c86 to d1/dc/d25/d33/d3/dcd/c113 0 2024-09-10T04:05:36.979 INFO:tasks.workunit.client.0.smithi046.stdout:0/873: chown d1/dc/d25/d33/d3/dd/d55/d4d/l9c 23145 1 2024-09-10T04:05:36.980 INFO:tasks.workunit.client.0.smithi046.stdout:0/874: mknod d1/dc/de9/c114 0 2024-09-10T04:05:36.987 INFO:tasks.workunit.client.0.smithi046.stdout:0/875: dwrite d1/d13/d37/d75/d94/d9a/dd7/df5/ff7 [0,4194304] 0 2024-09-10T04:05:36.989 INFO:tasks.workunit.client.0.smithi046.stdout:0/876: write d1/dc/d49/db2/fd2 [884525,3832] 0 2024-09-10T04:05:37.000 INFO:tasks.workunit.client.0.smithi046.stdout:0/877: write d1/dc/d25/d72/fb9 [572556,119708] 0 2024-09-10T04:05:37.007 INFO:tasks.workunit.client.0.smithi046.stdout:0/878: dwrite d1/dc/d25/d33/f11 [0,4194304] 0 2024-09-10T04:05:37.041 INFO:tasks.workunit.client.0.smithi046.stdout:0/879: truncate d1/dc/d25/d33/d1a/f8e 779747 0 2024-09-10T04:05:37.066 INFO:tasks.workunit.client.0.smithi046.stdout:0/880: truncate d1/d13/d37/f61 674770 0 2024-09-10T04:05:37.067 INFO:tasks.workunit.client.0.smithi046.stdout:0/881: write d1/dc/d25/d33/d3/fdf [745528,90842] 0 2024-09-10T04:05:37.068 INFO:tasks.workunit.client.0.smithi046.stdout:0/882: readlink d1/dc/d25/d33/d4/laa 0 2024-09-10T04:05:37.069 INFO:tasks.workunit.client.0.smithi046.stdout:0/883: fsync d1/d13/d37/d75/f8b 0 2024-09-10T04:05:37.073 INFO:tasks.workunit.client.0.smithi046.stdout:0/884: mkdir d1/dc/d25/d33/d3/d29/d2d/d115 0 2024-09-10T04:05:37.074 INFO:tasks.workunit.client.0.smithi046.stdout:0/885: truncate d1/dc/d25/f9e 1429070 0 2024-09-10T04:05:37.075 INFO:tasks.workunit.client.0.smithi046.stdout:0/886: chown d1/d13/d37/d75/d94/d9a/da4/l112 509067370 1 2024-09-10T04:05:37.077 INFO:tasks.workunit.client.0.smithi046.stdout:0/887: creat d1/dc/d25/d33/d3/dd/d55/d77/da6/f116 x:0 0 0 2024-09-10T04:05:37.077 INFO:tasks.workunit.client.0.smithi046.stdout:0/888: stat d1 0 2024-09-10T04:05:37.078 INFO:tasks.workunit.client.0.smithi046.stdout:0/889: write d1/dc/f76 [4220828,45305] 0 2024-09-10T04:05:37.084 INFO:tasks.workunit.client.0.smithi046.stdout:0/890: dwrite d1/d13/d37/fb7 [0,4194304] 0 2024-09-10T04:05:37.088 INFO:tasks.workunit.client.0.smithi046.stdout:0/891: creat d1/dc/d25/d33/d1a/dc9/f117 x:0 0 0 2024-09-10T04:05:37.093 INFO:tasks.workunit.client.0.smithi046.stdout:0/892: dread d1/dc/d25/d33/d3/fdf [0,4194304] 0 2024-09-10T04:05:37.097 INFO:tasks.workunit.client.0.smithi046.stdout:0/893: rename d1/dc/d25/d33/d3/dd/f23 to d1/dc/d25/d33/d1a/d80/f118 0 2024-09-10T04:05:37.103 INFO:tasks.workunit.client.0.smithi046.stdout:0/894: dwrite d1/dc/d25/d33/d4/ff1 [0,4194304] 0 2024-09-10T04:05:37.105 INFO:tasks.workunit.client.0.smithi046.stdout:0/895: chown d1/dc/d25/d33/d3/d29/d2d/d5e/lff 10 1 2024-09-10T04:05:37.109 INFO:tasks.workunit.client.0.smithi046.stdout:0/896: unlink d1/d13/d37/fb7 0 2024-09-10T04:05:37.112 INFO:tasks.workunit.client.0.smithi046.stdout:0/897: unlink d1/dc/de9/c114 0 2024-09-10T04:05:37.118 INFO:tasks.workunit.client.0.smithi046.stdout:0/898: write d1/d13/f8c [4313992,129783] 0 2024-09-10T04:05:37.119 INFO:tasks.workunit.client.0.smithi046.stdout:0/899: read - d1/dc/d49/db2/dcb/fe5 zero size 2024-09-10T04:05:37.121 INFO:tasks.workunit.client.0.smithi046.stdout:0/900: write d1/dc/d25/d33/d3/dd/d55/d77/da6/fb3 [1014738,126049] 0 2024-09-10T04:05:37.124 INFO:tasks.workunit.client.0.smithi046.stdout:0/901: dread d1/d13/d37/d75/d94/d9a/da4/faf [0,4194304] 0 2024-09-10T04:05:37.125 INFO:tasks.workunit.client.0.smithi046.stdout:0/902: write d1/d13/d37/d75/f7f [1502818,80788] 0 2024-09-10T04:05:37.126 INFO:tasks.workunit.client.0.smithi046.stdout:0/903: chown d1/dc/d25/d33/d3/d29/d2d/f4f 1464 1 2024-09-10T04:05:37.132 INFO:tasks.workunit.client.0.smithi046.stdout:0/904: dwrite d1/dc/d25/d33/d3/d29/d2d/d5e/f89 [4194304,4194304] 0 2024-09-10T04:05:37.134 INFO:tasks.workunit.client.0.smithi046.stdout:0/905: stat d1/dc/d25/d33/d1a/d80/ffb 0 2024-09-10T04:05:37.136 INFO:tasks.workunit.client.0.smithi046.stdout:0/906: creat d1/d13/d37/d75/d94/dca/f119 x:0 0 0 2024-09-10T04:05:37.139 INFO:tasks.workunit.client.0.smithi046.stdout:0/907: creat d1/d13/d37/d75/f11a x:0 0 0 2024-09-10T04:05:37.139 INFO:tasks.workunit.client.0.smithi046.stdout:0/908: fsync d1/dc/d25/d33/f3f 0 2024-09-10T04:05:37.144 INFO:tasks.workunit.client.0.smithi046.stdout:0/909: dread d1/d13/d37/f61 [0,4194304] 0 2024-09-10T04:05:37.146 INFO:tasks.workunit.client.0.smithi046.stdout:0/910: mkdir d1/dc/d49/db2/dcb/d11b 0 2024-09-10T04:05:37.148 INFO:tasks.workunit.client.0.smithi046.stdout:0/911: creat d1/d13/d37/f11c x:0 0 0 2024-09-10T04:05:37.149 INFO:tasks.workunit.client.0.smithi046.stdout:0/912: symlink d1/dc/d25/d72/l11d 0 2024-09-10T04:05:37.151 INFO:tasks.workunit.client.0.smithi046.stdout:0/913: creat d1/d13/d37/d75/d94/f11e x:0 0 0 2024-09-10T04:05:37.153 INFO:tasks.workunit.client.0.smithi046.stdout:0/914: mkdir d1/d13/d37/d75/d11f 0 2024-09-10T04:05:37.154 INFO:tasks.workunit.client.0.smithi046.stdout:0/915: stat d1/dc/d25/d33/f3f 0 2024-09-10T04:05:37.184 INFO:tasks.workunit.client.0.smithi046.stdout:0/916: dwrite d1/dc/d25/d33/d1a/d41/fc6 [0,4194304] 0 2024-09-10T04:05:37.186 INFO:tasks.workunit.client.0.smithi046.stdout:0/917: write d1/dc/f76 [1228556,94369] 0 2024-09-10T04:05:37.192 INFO:tasks.workunit.client.0.smithi046.stdout:0/918: rename d1/dc/d25/d33/d4/fd8 to d1/dc/d25/d33/d3/d29/d108/f120 0 2024-09-10T04:05:37.195 INFO:tasks.workunit.client.0.smithi046.stdout:0/919: mknod d1/dc/d49/db2/dcb/c121 0 2024-09-10T04:05:37.196 INFO:tasks.workunit.client.0.smithi046.stdout:1/813: sync 2024-09-10T04:05:37.196 INFO:tasks.workunit.client.0.smithi046.stdout:6/989: sync 2024-09-10T04:05:37.200 INFO:tasks.workunit.client.0.smithi046.stdout:0/920: dread d1/d13/d37/d75/f7f [0,4194304] 0 2024-09-10T04:05:37.205 INFO:tasks.workunit.client.0.smithi046.stdout:1/814: dwrite d3/d27/fe9 [0,4194304] 0 2024-09-10T04:05:37.205 INFO:tasks.workunit.client.0.smithi046.stdout:0/921: creat d1/d13/d37/d75/d94/d9a/da4/f122 x:0 0 0 2024-09-10T04:05:37.211 INFO:tasks.workunit.client.0.smithi046.stdout:1/815: dwrite d3/da1/d1f/f35 [0,4194304] 0 2024-09-10T04:05:37.216 INFO:tasks.workunit.client.0.smithi046.stdout:1/816: chown d3/dc/d21/d3a/d90/dbf/c112 115 1 2024-09-10T04:05:37.217 INFO:tasks.workunit.client.0.smithi046.stdout:6/990: dread da/db/d11/d13/d1e/d69/f87 [0,4194304] 0 2024-09-10T04:05:37.218 INFO:tasks.workunit.client.0.smithi046.stdout:1/817: write d3/d27/d5d/d60/fc0 [6143367,54209] 0 2024-09-10T04:05:37.219 INFO:tasks.workunit.client.0.smithi046.stdout:6/991: chown da/db/d11/f8c 298267 1 2024-09-10T04:05:37.220 INFO:tasks.workunit.client.0.smithi046.stdout:1/818: write d3/dc/d34/fe0 [1987029,74500] 0 2024-09-10T04:05:37.224 INFO:tasks.workunit.client.0.smithi046.stdout:1/819: rename d3/d9b to d3/dc/d21/dde/d113 0 2024-09-10T04:05:37.229 INFO:tasks.workunit.client.0.smithi046.stdout:1/820: creat d3/da1/d1f/f114 x:0 0 0 2024-09-10T04:05:37.230 INFO:tasks.workunit.client.0.smithi046.stdout:1/821: write d3/d27/d5d/fe1 [395012,93143] 0 2024-09-10T04:05:37.236 INFO:tasks.workunit.client.0.smithi046.stdout:6/992: dread da/db/d11/d2b/d2f/dde/d122/fa1 [0,4194304] 0 2024-09-10T04:05:37.240 INFO:tasks.workunit.client.0.smithi046.stdout:1/822: mknod d3/dc/d21/dde/d113/dcb/c115 0 2024-09-10T04:05:37.243 INFO:tasks.workunit.client.0.smithi046.stdout:6/993: creat da/db/d11/d2b/d55/f13d x:0 0 0 2024-09-10T04:05:37.248 INFO:tasks.workunit.client.0.smithi046.stdout:1/823: dread d3/d27/d5d/fad [0,4194304] 0 2024-09-10T04:05:37.251 INFO:tasks.workunit.client.0.smithi046.stdout:6/994: mknod da/db/d11/d9f/dd6/c13e 0 2024-09-10T04:05:37.254 INFO:tasks.workunit.client.0.smithi046.stdout:6/995: readlink da/db/d11/d13/d1e/d23/d54/dad/dfe/l108 0 2024-09-10T04:05:37.259 INFO:tasks.workunit.client.0.smithi046.stdout:1/824: link d3/dc/d21/lb4 d3/da1/l116 0 2024-09-10T04:05:37.261 INFO:tasks.workunit.client.0.smithi046.stdout:1/825: creat d3/dc/d21/d3a/f117 x:0 0 0 2024-09-10T04:05:37.263 INFO:tasks.workunit.client.0.smithi046.stdout:6/996: dread da/db/d11/dd7/f34 [0,4194304] 0 2024-09-10T04:05:37.265 INFO:tasks.workunit.client.0.smithi046.stdout:6/997: write da/db/d11/d2b/d2f/dde/f136 [449380,124695] 0 2024-09-10T04:05:37.265 INFO:tasks.workunit.client.0.smithi046.stdout:6/998: dread - da/db/d18/f110 zero size 2024-09-10T04:05:37.268 INFO:tasks.workunit.client.0.smithi046.stdout:1/826: link d3/d27/d5d/d60/dd7/ff6 d3/f118 0 2024-09-10T04:05:37.269 INFO:tasks.workunit.client.0.smithi046.stdout:6/999: rename da/db/d11/d13/d1e/d23/d54/dad/dfe to da/db/d10/d27/d13f 0 2024-09-10T04:05:37.270 INFO:tasks.workunit.client.0.smithi046.stdout:1/827: rmdir d3/dc/d21/dde/d113 39 2024-09-10T04:05:37.305 INFO:tasks.workunit.client.0.smithi046.stdout:1/828: dwrite d3/dc/d21/dde/d113/f9e [0,4194304] 0 2024-09-10T04:05:37.311 INFO:tasks.workunit.client.0.smithi046.stdout:1/829: creat d3/dc/d21/dde/d113/dcb/f119 x:0 0 0 2024-09-10T04:05:37.314 INFO:tasks.workunit.client.0.smithi046.stdout:1/830: fsync d3/d27/d5d/d60/fdd 0 2024-09-10T04:05:37.318 INFO:tasks.workunit.client.0.smithi046.stdout:1/831: creat d3/da1/f11a x:0 0 0 2024-09-10T04:05:37.461 INFO:tasks.workunit.client.0.smithi046.stdout:0/922: sync 2024-09-10T04:05:37.467 INFO:tasks.workunit.client.0.smithi046.stdout:0/923: link d1/dc/f27 d1/dc/d25/d33/d1a/d80/f123 0 2024-09-10T04:05:37.468 INFO:tasks.workunit.client.0.smithi046.stdout:0/924: write d1/dc/d25/d33/d3/d29/d2d/f4f [2996612,98977] 0 2024-09-10T04:05:37.471 INFO:tasks.workunit.client.0.smithi046.stdout:0/925: symlink d1/dc/l124 0 2024-09-10T04:05:37.475 INFO:tasks.workunit.client.0.smithi046.stdout:0/926: mknod d1/d13/d37/d75/d94/d9a/da4/c125 0 2024-09-10T04:05:37.481 INFO:tasks.workunit.client.0.smithi046.stdout:0/927: dread - d1/dc/d25/d33/d3/d29/d108/f120 zero size 2024-09-10T04:05:37.483 INFO:tasks.workunit.client.0.smithi046.stdout:0/928: creat d1/d13/d37/d75/d94/d9a/dd7/f126 x:0 0 0 2024-09-10T04:05:37.486 INFO:tasks.workunit.client.0.smithi046.stdout:0/929: creat d1/d13/d37/d75/d94/d9a/dcf/f127 x:0 0 0 2024-09-10T04:05:37.487 INFO:tasks.workunit.client.0.smithi046.stdout:0/930: chown d1/dc/d25/d33/d1a/dc9/fe6 35 1 2024-09-10T04:05:37.489 INFO:tasks.workunit.client.0.smithi046.stdout:0/931: mkdir d1/dc/d128 0 2024-09-10T04:05:37.492 INFO:tasks.workunit.client.0.smithi046.stdout:0/932: symlink d1/dc/d25/d33/l129 0 2024-09-10T04:05:37.494 INFO:tasks.workunit.client.0.smithi046.stdout:0/933: getdents d1/d13/d37/d75/d11f 0 2024-09-10T04:05:37.495 INFO:tasks.workunit.client.0.smithi046.stdout:1/832: sync 2024-09-10T04:05:37.503 INFO:tasks.workunit.client.0.smithi046.stdout:0/934: getdents d1/dc/d49/db2 0 2024-09-10T04:05:37.506 INFO:tasks.workunit.client.0.smithi046.stdout:0/935: symlink d1/dc/d25/d33/d1a/l12a 0 2024-09-10T04:05:37.507 INFO:tasks.workunit.client.0.smithi046.stdout:0/936: fsync d1/d13/d37/d75/d94/dca/f119 0 2024-09-10T04:05:37.510 INFO:tasks.workunit.client.0.smithi046.stdout:1/833: write d3/d27/f20 [874613,74152] 0 2024-09-10T04:05:37.511 INFO:tasks.workunit.client.0.smithi046.stdout:0/937: symlink d1/dc/de9/l12b 0 2024-09-10T04:05:37.511 INFO:tasks.workunit.client.0.smithi046.stdout:1/834: readlink d3/dc/d21/d3a/l4c 0 2024-09-10T04:05:37.514 INFO:tasks.workunit.client.0.smithi046.stdout:1/835: rename d3/d27/d5d/d60/lcc to d3/d27/d5d/l11b 0 2024-09-10T04:05:37.520 INFO:tasks.workunit.client.0.smithi046.stdout:1/836: rename d3/d27/d5d/d102/ca9 to d3/d27/d5d/d60/c11c 0 2024-09-10T04:05:37.522 INFO:tasks.workunit.client.0.smithi046.stdout:1/837: truncate d3/dc/d21/dde/d113/f107 974915 0 2024-09-10T04:05:37.523 INFO:tasks.workunit.client.0.smithi046.stdout:0/938: dread d1/dc/d25/d33/d4/d19/f58 [0,4194304] 0 2024-09-10T04:05:37.525 INFO:tasks.workunit.client.0.smithi046.stdout:0/939: dread - d1/dc/d25/d33/d1a/dc9/f117 zero size 2024-09-10T04:05:37.528 INFO:tasks.workunit.client.0.smithi046.stdout:0/940: creat d1/dc/d25/d33/d1a/d41/f12c x:0 0 0 2024-09-10T04:05:37.530 INFO:tasks.workunit.client.0.smithi046.stdout:0/941: mkdir d1/d13/d37/d75/d11f/d12d 0 2024-09-10T04:05:37.533 INFO:tasks.workunit.client.0.smithi046.stdout:0/942: creat d1/dc/de9/f12e x:0 0 0 2024-09-10T04:05:37.536 INFO:tasks.workunit.client.0.smithi046.stdout:1/838: dread d3/dc/d21/fc3 [0,4194304] 0 2024-09-10T04:05:37.539 INFO:tasks.workunit.client.0.smithi046.stdout:0/943: getdents d1/dc/d25/d33/d3 0 2024-09-10T04:05:37.540 INFO:tasks.workunit.client.0.smithi046.stdout:1/839: creat d3/dc/d21/dde/d113/dcb/f11d x:0 0 0 2024-09-10T04:05:37.544 INFO:tasks.workunit.client.0.smithi046.stdout:1/840: unlink d3/dc/d34/d51/c89 0 2024-09-10T04:05:37.546 INFO:tasks.workunit.client.0.smithi046.stdout:0/944: dwrite d1/dc/d25/d33/d1a/f97 [0,4194304] 0 2024-09-10T04:05:37.551 INFO:tasks.workunit.client.0.smithi046.stdout:1/841: mkdir d3/dc/d34/d11e 0 2024-09-10T04:05:37.557 INFO:tasks.workunit.client.0.smithi046.stdout:0/945: dwrite d1/dc/d25/d33/d3/dd/d55/f43 [0,4194304] 0 2024-09-10T04:05:37.558 INFO:tasks.workunit.client.0.smithi046.stdout:1/842: dwrite d3/da1/f11a [0,4194304] 0 2024-09-10T04:05:37.560 INFO:tasks.workunit.client.0.smithi046.stdout:1/843: read - d3/dc/d21/d3a/f117 zero size 2024-09-10T04:05:37.562 INFO:tasks.workunit.client.0.smithi046.stdout:0/946: chown d1/dc/d25/d33/d3/dd/c15 124 1 2024-09-10T04:05:37.562 INFO:tasks.workunit.client.0.smithi046.stdout:1/844: dread - d3/da1/d1f/df4/ffe zero size 2024-09-10T04:05:37.566 INFO:tasks.workunit.client.0.smithi046.stdout:1/845: creat d3/dc/d21/f11f x:0 0 0 2024-09-10T04:05:37.570 INFO:tasks.workunit.client.0.smithi046.stdout:1/846: mkdir d3/dc/d109/d120 0 2024-09-10T04:05:37.571 INFO:tasks.workunit.client.0.smithi046.stdout:1/847: chown d3/da1/f2e 208338 1 2024-09-10T04:05:37.572 INFO:tasks.workunit.client.0.smithi046.stdout:1/848: rmdir d3/d62 39 2024-09-10T04:05:37.575 INFO:tasks.workunit.client.0.smithi046.stdout:1/849: creat d3/dc/d21/d3a/d90/dea/f121 x:0 0 0 2024-09-10T04:05:37.582 INFO:tasks.workunit.client.0.smithi046.stdout:0/947: dwrite d1/dc/d25/d33/d1a/f31 [0,4194304] 0 2024-09-10T04:05:37.584 INFO:tasks.workunit.client.0.smithi046.stdout:1/850: dwrite d3/dc/d34/d58/fd5 [0,4194304] 0 2024-09-10T04:05:37.591 INFO:tasks.workunit.client.0.smithi046.stdout:1/851: dwrite d3/dc/f88 [0,4194304] 0 2024-09-10T04:05:37.591 INFO:tasks.workunit.client.0.smithi046.stdout:0/948: rmdir d1/dc/d49/db2/dcb/d11b 0 2024-09-10T04:05:37.592 INFO:tasks.workunit.client.0.smithi046.stdout:1/852: chown d3/dc/d21/d3a/f117 33376358 1 2024-09-10T04:05:37.597 INFO:tasks.workunit.client.0.smithi046.stdout:0/949: fsync d1/dc/d25/d33/d4/d19/f58 0 2024-09-10T04:05:37.608 INFO:tasks.workunit.client.0.smithi046.stdout:0/950: dwrite d1/dc/d25/d33/d3/d29/d108/f120 [0,4194304] 0 2024-09-10T04:05:37.612 INFO:tasks.workunit.client.0.smithi046.stdout:0/951: read d1/dc/f76 [1103495,37028] 0 2024-09-10T04:05:37.615 INFO:tasks.workunit.client.0.smithi046.stdout:0/952: write d1/d13/d37/d75/d94/d9a/dd7/f126 [179404,130122] 0 2024-09-10T04:05:37.617 INFO:tasks.workunit.client.0.smithi046.stdout:0/953: truncate d1/dc/d49/fa1 1647500 0 2024-09-10T04:05:37.619 INFO:tasks.workunit.client.0.smithi046.stdout:1/853: dread d3/d27/fb7 [0,4194304] 0 2024-09-10T04:05:37.620 INFO:tasks.workunit.client.0.smithi046.stdout:0/954: creat d1/d13/d37/d75/d11f/f12f x:0 0 0 2024-09-10T04:05:37.628 INFO:tasks.workunit.client.0.smithi046.stdout:0/955: link d1/dc/d25/d33/cdd d1/dc/d25/d33/d4/d19/c130 0 2024-09-10T04:05:37.630 INFO:tasks.workunit.client.0.smithi046.stdout:0/956: write d1/d13/d37/d75/d94/dca/f119 [190262,24180] 0 2024-09-10T04:05:37.630 INFO:tasks.workunit.client.0.smithi046.stdout:1/854: truncate d3/d27/d5d/fe1 317203 0 2024-09-10T04:05:37.633 INFO:tasks.workunit.client.0.smithi046.stdout:1/855: mknod d3/da1/c122 0 2024-09-10T04:05:37.635 INFO:tasks.workunit.client.0.smithi046.stdout:0/957: rename d1/d13/d37/ff2 to d1/dc/d25/d33/d3/dd/f131 0 2024-09-10T04:05:37.638 INFO:tasks.workunit.client.0.smithi046.stdout:1/856: rename d3/dc/d21/d3a/c8c to d3/dc/c123 0 2024-09-10T04:05:37.643 INFO:tasks.workunit.client.0.smithi046.stdout:1/857: dwrite d3/f4a [4194304,4194304] 0 2024-09-10T04:05:37.650 INFO:tasks.workunit.client.0.smithi046.stdout:1/858: mkdir d3/d124 0 2024-09-10T04:05:37.655 INFO:tasks.workunit.client.0.smithi046.stdout:0/958: truncate d1/dc/d25/d33/d3/dd/d55/f4c 561202 0 2024-09-10T04:05:37.656 INFO:tasks.workunit.client.0.smithi046.stdout:1/859: symlink d3/dc/d34/d51/l125 0 2024-09-10T04:05:37.660 INFO:tasks.workunit.client.0.smithi046.stdout:1/860: mkdir d3/d27/d5d/d126 0 2024-09-10T04:05:37.673 INFO:tasks.workunit.client.0.smithi046.stdout:1/861: dread d3/f4d [0,4194304] 0 2024-09-10T04:05:37.679 INFO:tasks.workunit.client.0.smithi046.stdout:1/862: creat d3/da1/d1f/f127 x:0 0 0 2024-09-10T04:05:37.680 INFO:tasks.workunit.client.0.smithi046.stdout:1/863: dread - d3/dc/fec zero size 2024-09-10T04:05:37.684 INFO:tasks.workunit.client.0.smithi046.stdout:1/864: mkdir d3/d27/d128 0 2024-09-10T04:05:37.685 INFO:tasks.workunit.client.0.smithi046.stdout:1/865: read - d3/da1/d1f/f127 zero size 2024-09-10T04:05:37.689 INFO:tasks.workunit.client.0.smithi046.stdout:1/866: rename d3/dc/d21/d3a/f91 to d3/d27/d128/f129 0 2024-09-10T04:05:37.696 INFO:tasks.workunit.client.0.smithi046.stdout:1/867: dwrite d3/dc/d34/d51/f78 [0,4194304] 0 2024-09-10T04:05:37.697 INFO:tasks.workunit.client.0.smithi046.stdout:1/868: chown d3/d27/d2a/c67 85074 1 2024-09-10T04:05:37.704 INFO:tasks.workunit.client.0.smithi046.stdout:1/869: mknod d3/d62/df8/c12a 0 2024-09-10T04:05:37.706 INFO:tasks.workunit.client.0.smithi046.stdout:1/870: truncate d3/da1/d1f/f35 4214234 0 2024-09-10T04:05:37.706 INFO:tasks.workunit.client.0.smithi046.stdout:1/871: dread - d3/d62/f10c zero size 2024-09-10T04:05:37.715 INFO:tasks.workunit.client.0.smithi046.stdout:1/872: dread d3/dc/d34/d51/fdf [0,4194304] 0 2024-09-10T04:05:37.720 INFO:tasks.workunit.client.0.smithi046.stdout:1/873: dread d3/dc/d21/dde/d113/f9e [0,4194304] 0 2024-09-10T04:05:37.722 INFO:tasks.workunit.client.0.smithi046.stdout:1/874: symlink d3/dc/d109/l12b 0 2024-09-10T04:05:37.734 INFO:tasks.workunit.client.0.smithi046.stdout:1/875: write d3/dc/d21/fc5 [6504,44816] 0 2024-09-10T04:05:37.741 INFO:tasks.workunit.client.0.smithi046.stdout:1/876: dwrite d3/dc/d34/d58/fd5 [0,4194304] 0 2024-09-10T04:05:37.742 INFO:tasks.workunit.client.0.smithi046.stdout:1/877: dread - d3/da1/d1f/f127 zero size 2024-09-10T04:05:37.749 INFO:tasks.workunit.client.0.smithi046.stdout:1/878: dwrite d3/da1/d1f/f29 [0,4194304] 0 2024-09-10T04:05:37.752 INFO:tasks.workunit.client.0.smithi046.stdout:1/879: chown d3/da1/d1f/f55 0 1 2024-09-10T04:05:37.756 INFO:tasks.workunit.client.0.smithi046.stdout:1/880: symlink d3/d27/d5d/d60/dd7/df9/l12c 0 2024-09-10T04:05:37.759 INFO:tasks.workunit.client.0.smithi046.stdout:1/881: truncate d3/da1/d1f/df4/ffe 509893 0 2024-09-10T04:05:37.762 INFO:tasks.workunit.client.0.smithi046.stdout:1/882: mknod d3/dc/d109/d120/c12d 0 2024-09-10T04:05:37.764 INFO:tasks.workunit.client.0.smithi046.stdout:1/883: truncate d3/dc/d21/d3a/d90/fe7 779396 0 2024-09-10T04:05:37.764 INFO:tasks.workunit.client.0.smithi046.stdout:1/884: readlink d3/dc/l95 0 2024-09-10T04:05:37.765 INFO:tasks.workunit.client.0.smithi046.stdout:1/885: chown d3/dc/d21/d3a 14 1 2024-09-10T04:05:37.822 INFO:tasks.workunit.client.0.smithi046.stdout:1/886: unlink d3/d27/fd8 0 2024-09-10T04:05:37.822 INFO:tasks.workunit.client.0.smithi046.stdout:1/887: readlink d3/dc/l7f 0 2024-09-10T04:05:37.823 INFO:tasks.workunit.client.0.smithi046.stdout:1/888: readlink d3/dc/d21/l64 0 2024-09-10T04:05:37.824 INFO:tasks.workunit.client.0.smithi046.stdout:1/889: fsync d3/d27/f20 0 2024-09-10T04:05:37.826 INFO:tasks.workunit.client.0.smithi046.stdout:1/890: dread - d3/dc/d34/d51/f99 zero size 2024-09-10T04:05:37.829 INFO:tasks.workunit.client.0.smithi046.stdout:1/891: write d3/dc/d21/dde/d113/f9e [22664,76431] 0 2024-09-10T04:05:37.831 INFO:tasks.workunit.client.0.smithi046.stdout:1/892: truncate d3/da1/d1f/f127 481144 0 2024-09-10T04:05:37.835 INFO:tasks.workunit.client.0.smithi046.stdout:1/893: mkdir d3/d27/d5d/d60/dd7/df9/d12e 0 2024-09-10T04:05:37.838 INFO:tasks.workunit.client.0.smithi046.stdout:1/894: rename d3/dc/l7f to d3/d62/l12f 0 2024-09-10T04:05:37.839 INFO:tasks.workunit.client.0.smithi046.stdout:1/895: fsync d3/dc/d21/dde/d113/f8d 0 2024-09-10T04:05:37.841 INFO:tasks.workunit.client.0.smithi046.stdout:1/896: write d3/d27/d5d/d60/fc0 [1367256,18323] 0 2024-09-10T04:05:37.845 INFO:tasks.workunit.client.0.smithi046.stdout:1/897: getdents d3/d27/df1 0 2024-09-10T04:05:37.847 INFO:tasks.workunit.client.0.smithi046.stdout:1/898: write d3/dc/d21/dde/d113/dcb/f119 [370120,52189] 0 2024-09-10T04:05:37.850 INFO:tasks.workunit.client.0.smithi046.stdout:1/899: creat d3/d124/f130 x:0 0 0 2024-09-10T04:05:37.857 INFO:tasks.workunit.client.0.smithi046.stdout:1/900: dwrite d3/dc/d34/fc6 [4194304,4194304] 0 2024-09-10T04:05:37.866 INFO:tasks.workunit.client.0.smithi046.stdout:1/901: dwrite d3/dc/f17 [0,4194304] 0 2024-09-10T04:05:37.869 INFO:tasks.workunit.client.0.smithi046.stdout:1/902: chown d3/d27/f66 202321737 1 2024-09-10T04:05:37.871 INFO:tasks.workunit.client.0.smithi046.stdout:1/903: mkdir d3/d62/d131 0 2024-09-10T04:05:37.875 INFO:tasks.workunit.client.0.smithi046.stdout:1/904: unlink d3/da1/d1f/f127 0 2024-09-10T04:05:37.883 INFO:tasks.workunit.client.0.smithi046.stdout:1/905: dwrite d3/da1/d1f/f3c [0,4194304] 0 2024-09-10T04:05:37.900 INFO:tasks.workunit.client.0.smithi046.stdout:1/906: dread d3/dc/d34/d58/fce [0,4194304] 0 2024-09-10T04:05:37.901 INFO:tasks.workunit.client.0.smithi046.stdout:1/907: rmdir d3/da1/d1f 39 2024-09-10T04:05:37.902 INFO:tasks.workunit.client.0.smithi046.stdout:1/908: fsync d3/dc/d21/dde/d113/fbc 0 2024-09-10T04:05:37.907 INFO:tasks.workunit.client.0.smithi046.stdout:1/909: getdents d3/dc/d109/d120 0 2024-09-10T04:05:37.908 INFO:tasks.workunit.client.0.smithi046.stdout:1/910: chown d3/d27/d2a/c67 36837501 1 2024-09-10T04:05:37.912 INFO:tasks.workunit.client.0.smithi046.stdout:1/911: mknod d3/da1/d1f/c132 0 2024-09-10T04:05:37.915 INFO:tasks.workunit.client.0.smithi046.stdout:1/912: mknod d3/d27/d5d/d60/dd7/dfd/c133 0 2024-09-10T04:05:37.917 INFO:tasks.workunit.client.0.smithi046.stdout:1/913: rmdir d3/d27/d5d/d60/dd7/dfd 39 2024-09-10T04:05:37.923 INFO:tasks.workunit.client.0.smithi046.stdout:1/914: dwrite d3/da1/f11a [0,4194304] 0 2024-09-10T04:05:37.931 INFO:tasks.workunit.client.0.smithi046.stdout:1/915: creat d3/dc/d34/d58/f134 x:0 0 0 2024-09-10T04:05:37.935 INFO:tasks.workunit.client.0.smithi046.stdout:1/916: symlink d3/d27/d5d/d60/dd7/deb/l135 0 2024-09-10T04:05:37.938 INFO:tasks.workunit.client.0.smithi046.stdout:1/917: rmdir d3/da1 39 2024-09-10T04:05:37.953 INFO:tasks.workunit.client.0.smithi046.stdout:1/918: dread d3/da1/f2e [0,4194304] 0 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: pgmap v272: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 112 MiB/s rd, 246 MiB/s wr, 539 op/s 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr fail", "who": "smithi046.zvckjg"}]: dispatch 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: Activating manager daemon smithi070.tezbzn 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "mgr fail", "who": "smithi046.zvckjg"}]': finished 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:05:37.955 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: osdmap e44: 6 total, 6 up, 6 in 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: mgrmap e21: smithi070.tezbzn(active, starting, since 0.0141462s) 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr metadata", "who": "smithi070.tezbzn", "id": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:05:37.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:05:37.957 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T04:05:37.957 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T04:05:37.957 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:37 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T04:05:37.957 INFO:tasks.workunit.client.0.smithi046.stdout:1/919: symlink d3/d62/l136 0 2024-09-10T04:05:37.958 INFO:tasks.workunit.client.0.smithi046.stdout:1/920: mknod d3/da1/d1f/c137 0 2024-09-10T04:05:37.959 INFO:tasks.workunit.client.0.smithi046.stdout:1/921: write d3/da1/d1f/df4/ffe [1241603,57227] 0 2024-09-10T04:05:37.971 INFO:tasks.workunit.client.0.smithi046.stdout:0/959: sync 2024-09-10T04:05:37.977 INFO:tasks.workunit.client.0.smithi046.stdout:0/960: dwrite d1/d13/d37/d75/d94/d9a/dd7/df5/ff7 [0,4194304] 0 2024-09-10T04:05:37.980 INFO:tasks.workunit.client.0.smithi046.stdout:0/961: write d1/dc/d25/d33/f11 [5238668,46660] 0 2024-09-10T04:05:38.005 INFO:tasks.workunit.client.0.smithi046.stdout:0/962: write d1/dc/d25/d33/d4/f6f [5403439,103025] 0 2024-09-10T04:05:38.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: pgmap v272: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 112 MiB/s rd, 246 MiB/s wr, 539 op/s 2024-09-10T04:05:38.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:38.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:38.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:38.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:05:38.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr fail", "who": "smithi046.zvckjg"}]: dispatch 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: Activating manager daemon smithi070.tezbzn 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.14217 172.21.15.46:0/2901901742' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "mgr fail", "who": "smithi046.zvckjg"}]': finished 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"}]: dispatch 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"}]: dispatch 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"}]: dispatch 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: osdmap e44: 6 total, 6 up, 6 in 2024-09-10T04:05:38.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: mgrmap e21: smithi070.tezbzn(active, starting, since 0.0141462s) 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr metadata", "who": "smithi070.tezbzn", "id": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T04:05:38.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:37 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T04:05:38.194 INFO:tasks.workunit.client.0.smithi046.stdout:1/922: sync 2024-09-10T04:05:38.222 INFO:tasks.workunit.client.0.smithi046.stdout:1/923: rmdir d3 39 2024-09-10T04:05:38.338 INFO:tasks.workunit.client.0.smithi046.stdout:0/963: sync 2024-09-10T04:05:38.339 INFO:tasks.workunit.client.0.smithi046.stdout:0/964: read d1/d13/d37/d75/f8b [2559753,56467] 0 2024-09-10T04:05:38.340 INFO:tasks.workunit.client.0.smithi046.stdout:0/965: dread - d1/d13/d37/d75/d94/d9a/da4/fee zero size 2024-09-10T04:05:38.342 INFO:tasks.workunit.client.0.smithi046.stdout:0/966: write d1/dc/d25/d33/d3/fdf [1530317,96191] 0 2024-09-10T04:05:38.344 INFO:tasks.workunit.client.0.smithi046.stdout:0/967: read - d1/dc/d25/d33/d4/f62 zero size 2024-09-10T04:05:38.345 INFO:tasks.workunit.client.0.smithi046.stdout:0/968: write d1/d13/d37/d75/fd0 [1748803,125651] 0 2024-09-10T04:05:38.352 INFO:tasks.workunit.client.0.smithi046.stdout:0/969: dwrite d1/dc/d25/d33/d3/fdf [0,4194304] 0 2024-09-10T04:05:38.353 INFO:tasks.workunit.client.0.smithi046.stdout:0/970: readlink d1/dc/d25/d33/d3/le 0 2024-09-10T04:05:38.355 INFO:tasks.workunit.client.0.smithi046.stdout:0/971: mknod d1/dc/d25/d33/d3/d29/d2d/d5e/c132 0 2024-09-10T04:05:38.359 INFO:tasks.workunit.client.0.smithi046.stdout:0/972: symlink d1/dc/d25/d33/d3/dd/d55/d77/l133 0 2024-09-10T04:05:38.365 INFO:tasks.workunit.client.0.smithi046.stdout:0/973: dwrite d1/dc/d25/d72/fb9 [4194304,4194304] 0 2024-09-10T04:05:38.366 INFO:tasks.workunit.client.0.smithi046.stdout:1/924: sync 2024-09-10T04:05:38.370 INFO:tasks.workunit.client.0.smithi046.stdout:1/925: readlink d3/d27/d5d/d60/dd7/deb/l135 0 2024-09-10T04:05:38.375 INFO:tasks.workunit.client.0.smithi046.stdout:0/974: rename d1/fdc to d1/dc/d25/d33/d3/dd/d55/d4d/f134 0 2024-09-10T04:05:38.376 INFO:tasks.workunit.client.0.smithi046.stdout:1/926: fsync d3/dc/d34/d58/f59 0 2024-09-10T04:05:38.381 INFO:tasks.workunit.client.0.smithi046.stdout:0/975: symlink d1/d13/d37/d75/d94/d9a/da4/l135 0 2024-09-10T04:05:38.383 INFO:tasks.workunit.client.0.smithi046.stdout:1/927: truncate d3/dc/d21/dde/d113/f8d 604217 0 2024-09-10T04:05:38.383 INFO:tasks.workunit.client.0.smithi046.stdout:1/928: read - d3/dc/d21/f11f zero size 2024-09-10T04:05:38.384 INFO:tasks.workunit.client.0.smithi046.stdout:0/976: creat d1/dc/d49/f136 x:0 0 0 2024-09-10T04:05:38.386 INFO:tasks.workunit.client.0.smithi046.stdout:1/929: symlink d3/dc/d21/l138 0 2024-09-10T04:05:38.387 INFO:tasks.workunit.client.0.smithi046.stdout:1/930: fsync d3/fee 0 2024-09-10T04:05:38.392 INFO:tasks.workunit.client.0.smithi046.stdout:0/977: mkdir d1/dc/d25/d111/d137 0 2024-09-10T04:05:38.395 INFO:tasks.workunit.client.0.smithi046.stdout:0/978: rmdir d1/dc/d25/d33/d4/d19 39 2024-09-10T04:05:38.396 INFO:tasks.workunit.client.0.smithi046.stdout:0/979: fsync d1/dc/d25/d33/d1a/d80/ffb 0 2024-09-10T04:05:38.397 INFO:tasks.workunit.client.0.smithi046.stdout:0/980: write d1/dc/de9/f12e [769487,85219] 0 2024-09-10T04:05:38.401 INFO:tasks.workunit.client.0.smithi046.stdout:0/981: symlink d1/dc/d25/l138 0 2024-09-10T04:05:38.408 INFO:tasks.workunit.client.0.smithi046.stdout:0/982: dwrite d1/dc/de9/f12e [0,4194304] 0 2024-09-10T04:05:38.409 INFO:tasks.workunit.client.0.smithi046.stdout:0/983: rename d1/dc/d25 to d1/dc/d25/d33/d3/d29/d139 22 2024-09-10T04:05:38.412 INFO:tasks.workunit.client.0.smithi046.stdout:0/984: rmdir d1/d13/d37/d75/d94/d9a/dcf 39 2024-09-10T04:05:38.413 INFO:tasks.workunit.client.0.smithi046.stdout:1/931: write d3/f77 [3995675,90366] 0 2024-09-10T04:05:38.416 INFO:tasks.workunit.client.0.smithi046.stdout:0/985: fsync d1/dc/d25/d33/d3/d29/d2d/d5e/fde 0 2024-09-10T04:05:38.422 INFO:tasks.workunit.client.0.smithi046.stdout:0/986: symlink d1/d13/d37/d75/d94/d9a/dd7/df5/l13a 0 2024-09-10T04:05:38.422 INFO:tasks.workunit.client.0.smithi046.stdout:1/932: symlink d3/da1/l139 0 2024-09-10T04:05:38.423 INFO:tasks.workunit.client.0.smithi046.stdout:1/933: dread - d3/dc/d21/d3a/f117 zero size 2024-09-10T04:05:38.424 INFO:tasks.workunit.client.0.smithi046.stdout:0/987: write d1/dc/d25/d33/d3/dd/f10d [978713,825] 0 2024-09-10T04:05:38.427 INFO:tasks.workunit.client.0.smithi046.stdout:0/988: creat d1/d13/d37/d75/f13b x:0 0 0 2024-09-10T04:05:38.430 INFO:tasks.workunit.client.0.smithi046.stdout:0/989: unlink d1/dc/d49/db2/dc3/ffe 0 2024-09-10T04:05:38.430 INFO:tasks.workunit.client.0.smithi046.stdout:1/934: dread d3/da1/f2b [0,4194304] 0 2024-09-10T04:05:38.431 INFO:tasks.workunit.client.0.smithi046.stdout:1/935: dread - d3/d27/d2a/fa3 zero size 2024-09-10T04:05:38.432 INFO:tasks.workunit.client.0.smithi046.stdout:0/990: symlink d1/dc/d49/db2/dcb/l13c 0 2024-09-10T04:05:38.433 INFO:tasks.workunit.client.0.smithi046.stdout:1/936: write d3/dc/d21/d3a/d90/fe7 [301297,77554] 0 2024-09-10T04:05:38.433 INFO:tasks.workunit.client.0.smithi046.stdout:0/991: dread - d1/dc/d49/f136 zero size 2024-09-10T04:05:38.435 INFO:tasks.workunit.client.0.smithi046.stdout:1/937: fdatasync d3/dc/d21/d3a/faf 0 2024-09-10T04:05:38.436 INFO:tasks.workunit.client.0.smithi046.stdout:0/992: creat d1/dc/d49/f13d x:0 0 0 2024-09-10T04:05:38.439 INFO:tasks.workunit.client.0.smithi046.stdout:0/993: dread d1/dc/d25/d33/d1a/d41/f84 [0,4194304] 0 2024-09-10T04:05:38.440 INFO:tasks.workunit.client.0.smithi046.stdout:1/938: unlink d3/da1/l116 0 2024-09-10T04:05:38.444 INFO:tasks.workunit.client.0.smithi046.stdout:0/994: unlink d1/d13/d37/d75/d94/d9a/dcf/lf3 0 2024-09-10T04:05:38.444 INFO:tasks.workunit.client.0.smithi046.stdout:0/995: stat d1/dc/d25/d33/d4/fae 0 2024-09-10T04:05:38.444 INFO:tasks.workunit.client.0.smithi046.stdout:1/939: creat d3/d27/f13a x:0 0 0 2024-09-10T04:05:38.445 INFO:tasks.workunit.client.0.smithi046.stdout:0/996: chown d1/dc/d25/d33/f1b 3564 1 2024-09-10T04:05:38.446 INFO:tasks.workunit.client.0.smithi046.stdout:0/997: dread - d1/d13/d37/f11c zero size 2024-09-10T04:05:38.446 INFO:tasks.workunit.client.0.smithi046.stdout:1/940: creat d3/d62/f13b x:0 0 0 2024-09-10T04:05:38.453 INFO:tasks.workunit.client.0.smithi046.stdout:0/998: getdents d1/dc/d25/d33/d3/d29 0 2024-09-10T04:05:38.455 INFO:tasks.workunit.client.0.smithi046.stdout:1/941: getdents d3/d27/d5d/d60/dd7/deb 0 2024-09-10T04:05:38.456 INFO:tasks.workunit.client.0.smithi046.stdout:1/942: read d3/f4d [2212533,96620] 0 2024-09-10T04:05:38.460 INFO:tasks.workunit.client.0.smithi046.stdout:1/943: mknod d3/d27/d5d/d60/c13c 0 2024-09-10T04:05:38.461 INFO:tasks.workunit.client.0.smithi046.stdout:1/944: chown d3/dc/d21/dde/d113/fa0 4597 1 2024-09-10T04:05:38.463 INFO:tasks.workunit.client.0.smithi046.stdout:0/999: getdents d1/d13/d37/d75/d94/d9a 0 2024-09-10T04:05:38.469 INFO:tasks.workunit.client.0.smithi046.stdout:1/945: creat d3/d27/d5d/d60/dd7/f13d x:0 0 0 2024-09-10T04:05:38.473 INFO:tasks.workunit.client.0.smithi046.stdout:1/946: dread d3/da1/d1f/f35 [0,4194304] 0 2024-09-10T04:05:38.474 INFO:tasks.workunit.client.0.smithi046.stdout:1/947: write d3/dc/d21/d3a/d90/fe7 [635954,38528] 0 2024-09-10T04:05:38.475 INFO:tasks.workunit.client.0.smithi046.stdout:1/948: fsync d3/dc/f88 0 2024-09-10T04:05:38.489 INFO:tasks.workunit.client.0.smithi046.stdout:1/949: dread d3/dc/d34/fc6 [0,4194304] 0 2024-09-10T04:05:38.494 INFO:tasks.workunit.client.0.smithi046.stdout:1/950: fsync d3/d27/d5d/d60/f9f 0 2024-09-10T04:05:38.497 INFO:tasks.workunit.client.0.smithi046.stdout:1/951: readlink d3/dc/d21/dde/d113/l75 0 2024-09-10T04:05:38.500 INFO:tasks.workunit.client.0.smithi046.stdout:1/952: truncate d3/d27/d2a/fa3 574151 0 2024-09-10T04:05:38.501 INFO:tasks.workunit.client.0.smithi046.stdout:1/953: stat d3/dc/d21/d3a/d90/dbf/fe5 0 2024-09-10T04:05:38.503 INFO:tasks.workunit.client.0.smithi046.stdout:1/954: symlink d3/dc/d109/d120/l13e 0 2024-09-10T04:05:38.509 INFO:tasks.workunit.client.0.smithi046.stdout:1/955: dwrite d3/f4d [0,4194304] 0 2024-09-10T04:05:38.511 INFO:tasks.workunit.client.0.smithi046.stdout:1/956: chown d3/ffc 1 1 2024-09-10T04:05:38.511 INFO:tasks.workunit.client.0.smithi046.stdout:1/957: dread - d3/d27/f13a zero size 2024-09-10T04:05:38.511 INFO:tasks.workunit.client.0.smithi046.stdout:1/958: stat d3/d62/f13b 0 2024-09-10T04:05:38.519 INFO:tasks.workunit.client.0.smithi046.stdout:1/959: link d3/d62/df8/f10f d3/d27/d5d/d60/f13f 0 2024-09-10T04:05:38.522 INFO:tasks.workunit.client.0.smithi046.stdout:1/960: write d3/dc/d34/d58/fce [1115374,67815] 0 2024-09-10T04:05:38.527 INFO:tasks.workunit.client.0.smithi046.stdout:1/961: dread d3/dc/d21/fc3 [0,4194304] 0 2024-09-10T04:05:38.532 INFO:tasks.workunit.client.0.smithi046.stdout:1/962: dread d3/d27/d5d/fad [0,4194304] 0 2024-09-10T04:05:38.535 INFO:tasks.workunit.client.0.smithi046.stdout:1/963: creat d3/dc/d34/f140 x:0 0 0 2024-09-10T04:05:38.538 INFO:tasks.workunit.client.0.smithi046.stdout:1/964: write d3/dc/d34/fc6 [8098614,8617] 0 2024-09-10T04:05:38.544 INFO:tasks.workunit.client.0.smithi046.stdout:1/965: dread d3/dc/f23 [0,4194304] 0 2024-09-10T04:05:38.546 INFO:tasks.workunit.client.0.smithi046.stdout:1/966: mknod d3/d27/d128/c141 0 2024-09-10T04:05:38.547 INFO:tasks.workunit.client.0.smithi046.stdout:1/967: fsync d3/dc/fec 0 2024-09-10T04:05:38.568 INFO:tasks.workunit.client.0.smithi046.stdout:1/968: dwrite d3/d27/d5d/d60/dd7/ff6 [0,4194304] 0 2024-09-10T04:05:38.575 INFO:tasks.workunit.client.0.smithi046.stdout:1/969: mkdir d3/dc/d34/d11e/d142 0 2024-09-10T04:05:38.577 INFO:tasks.workunit.client.0.smithi046.stdout:1/970: readlink d3/dc/d21/d3a/la2 0 2024-09-10T04:05:38.591 INFO:tasks.workunit.client.0.smithi046.stdout:1/971: dread d3/dc/d34/f76 [0,4194304] 0 2024-09-10T04:05:38.591 INFO:tasks.workunit.client.0.smithi046.stdout:1/972: read - d3/d124/f130 zero size 2024-09-10T04:05:38.594 INFO:tasks.workunit.client.0.smithi046.stdout:1/973: mkdir d3/dc/d109/d143 0 2024-09-10T04:05:38.598 INFO:tasks.workunit.client.0.smithi046.stdout:1/974: rename d3/dc/d21/d3a/d90/dbf/c10d to d3/dc/d34/d51/c144 0 2024-09-10T04:05:38.613 INFO:tasks.workunit.client.0.smithi046.stdout:1/975: dread d3/da1/f43 [0,4194304] 0 2024-09-10T04:05:38.613 INFO:tasks.workunit.client.0.smithi046.stdout:1/976: chown d3/dc/d21/l138 1072 1 2024-09-10T04:05:38.619 INFO:tasks.workunit.client.0.smithi046.stdout:1/977: dwrite d3/dc/d21/f11f [0,4194304] 0 2024-09-10T04:05:38.637 INFO:tasks.workunit.client.0.smithi046.stdout:1/978: dread d3/d27/d5d/d60/f8a [0,4194304] 0 2024-09-10T04:05:38.638 INFO:tasks.workunit.client.0.smithi046.stdout:1/979: read d3/da1/da7/fcf [2022223,123659] 0 2024-09-10T04:05:38.642 INFO:tasks.workunit.client.0.smithi046.stdout:1/980: dread d3/d27/d5d/d60/dd7/ff6 [0,4194304] 0 2024-09-10T04:05:38.645 INFO:tasks.workunit.client.0.smithi046.stdout:1/981: mkdir d3/dc/d21/d3a/d90/dbf/d145 0 2024-09-10T04:05:38.646 INFO:tasks.workunit.client.0.smithi046.stdout:1/982: chown d3/da1/d1f/f114 19951 1 2024-09-10T04:05:38.648 INFO:tasks.workunit.client.0.smithi046.stdout:1/983: rmdir d3/dc/d34/d11e 39 2024-09-10T04:05:38.649 INFO:tasks.workunit.client.0.smithi046.stdout:1/984: rename d3 to d3/dc/d21/d3a/d90/dbf/d146 22 2024-09-10T04:05:38.650 INFO:tasks.workunit.client.0.smithi046.stdout:1/985: write d3/d27/f108 [804807,56355] 0 2024-09-10T04:05:38.651 INFO:tasks.workunit.client.0.smithi046.stdout:1/986: write d3/dc/d21/d3a/fc7 [1713233,98024] 0 2024-09-10T04:05:38.997 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: Manager daemon smithi070.tezbzn is now available 2024-09-10T04:05:38.997 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:38.997 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:38.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:05:38.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:38.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:05:38.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:05:38.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:05:38.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/trash_purge_schedule"}]: dispatch 2024-09-10T04:05:38.998 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:38 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/trash_purge_schedule"}]: dispatch 2024-09-10T04:05:39.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: Manager daemon smithi070.tezbzn is now available 2024-09-10T04:05:39.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:39.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:39.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:05:39.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:39.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:05:39.008 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:05:39.009 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:05:39.009 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/trash_purge_schedule"}]: dispatch 2024-09-10T04:05:39.009 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:38 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi070.tezbzn/trash_purge_schedule"}]: dispatch 2024-09-10T04:05:39.058 INFO:tasks.workunit.client.0.smithi046.stdout:1/987: sync 2024-09-10T04:05:39.059 INFO:tasks.workunit.client.0.smithi046.stdout:1/988: chown d3/da1/f11a 3685 1 2024-09-10T04:05:39.061 INFO:tasks.workunit.client.0.smithi046.stdout:1/989: symlink d3/d27/d5d/l147 0 2024-09-10T04:05:39.080 INFO:tasks.workunit.client.0.smithi046.stdout:1/990: dwrite d3/dc/d34/d51/f99 [0,4194304] 0 2024-09-10T04:05:39.087 INFO:tasks.workunit.client.0.smithi046.stdout:1/991: rmdir d3/dc/d21/d3a/d90/dbf/d145 0 2024-09-10T04:05:39.093 INFO:tasks.workunit.client.0.smithi046.stdout:1/992: creat d3/dc/d34/d51/f148 x:0 0 0 2024-09-10T04:05:39.095 INFO:tasks.workunit.client.0.smithi046.stdout:1/993: unlink d3/d27/fb7 0 2024-09-10T04:05:39.096 INFO:tasks.workunit.client.0.smithi046.stdout:1/994: stat d3/dc/d21/dde/d113/l9d 0 2024-09-10T04:05:39.099 INFO:tasks.workunit.client.0.smithi046.stdout:1/995: write d3/da1/f43 [2109036,90125] 0 2024-09-10T04:05:39.104 INFO:tasks.workunit.client.0.smithi046.stdout:1/996: rename d3/dc/d21/l138 to d3/da1/l149 0 2024-09-10T04:05:39.105 INFO:tasks.workunit.client.0.smithi046.stdout:1/997: chown d3/dc/d34/d51/f78 4024 1 2024-09-10T04:05:39.105 INFO:tasks.workunit.client.0.smithi046.stdout:1/998: chown d3 56 1 2024-09-10T04:05:39.106 INFO:tasks.workunit.client.0.smithi046.stdout:1/999: fdatasync d3/da1/f2b 0 2024-09-10T04:05:39.110 INFO:tasks.workunit.client.0.smithi046.stderr:+ rm -rf -- ./tmp.WKIhNanA7D 2024-09-10T04:05:40.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:39 smithi070 ceph-mon[35495]: mgrmap e22: smithi070.tezbzn(active, since 1.11269s) 2024-09-10T04:05:40.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:39 smithi070 ceph-mon[35495]: Deploying cephadm binary to smithi070 2024-09-10T04:05:40.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:39 smithi070 ceph-mon[35495]: Deploying cephadm binary to smithi046 2024-09-10T04:05:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:39 smithi046 ceph-mon[28328]: mgrmap e22: smithi070.tezbzn(active, since 1.11269s) 2024-09-10T04:05:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:39 smithi046 ceph-mon[28328]: Deploying cephadm binary to smithi070 2024-09-10T04:05:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:39 smithi046 ceph-mon[28328]: Deploying cephadm binary to smithi046 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:40 smithi070 ceph-mon[35495]: pgmap v4: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:40 smithi070 ceph-mon[35495]: [10/Sep/2024:04:05:39] ENGINE Bus STARTING 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:40 smithi070 ceph-mon[35495]: [10/Sep/2024:04:05:39] ENGINE Serving on http://172.21.15.70:8765 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:40 smithi070 ceph-mon[35495]: [10/Sep/2024:04:05:39] ENGINE Serving on https://172.21.15.70:7150 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:40 smithi070 ceph-mon[35495]: [10/Sep/2024:04:05:39] ENGINE Bus STARTED 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:40 smithi070 ceph-mon[35495]: [10/Sep/2024:04:05:40] ENGINE Error in HTTPServer.serve 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Traceback (most recent call last): 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-10T04:05:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._run(expiration_interval) 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: s = self.context.wrap_socket( 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: return self.sslsocket_class._create( 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self.do_handshake() 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:05:41.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._sslobj.do_handshake() 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:40 smithi046 ceph-mon[28328]: pgmap v4: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:40 smithi046 ceph-mon[28328]: [10/Sep/2024:04:05:39] ENGINE Bus STARTING 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:40 smithi046 ceph-mon[28328]: [10/Sep/2024:04:05:39] ENGINE Serving on http://172.21.15.70:8765 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:40 smithi046 ceph-mon[28328]: [10/Sep/2024:04:05:39] ENGINE Serving on https://172.21.15.70:7150 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:40 smithi046 ceph-mon[28328]: [10/Sep/2024:04:05:39] ENGINE Bus STARTED 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:40 smithi046 ceph-mon[28328]: [10/Sep/2024:04:05:40] ENGINE Error in HTTPServer.serve 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Traceback (most recent call last): 2024-09-10T04:05:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._run(expiration_interval) 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s = self.context.wrap_socket( 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: return self.sslsocket_class._create( 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self.do_handshake() 2024-09-10T04:05:41.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:05:41.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._sslobj.do_handshake() 2024-09-10T04:05:41.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:05:41.990 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:41 smithi046 ceph-mon[28328]: mgrmap e23: smithi070.tezbzn(active, since 3s) 2024-09-10T04:05:42.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:41 smithi070 ceph-mon[35495]: mgrmap e23: smithi070.tezbzn(active, since 3s) 2024-09-10T04:05:42.949 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:42 smithi046 ceph-mon[28328]: pgmap v5: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail 2024-09-10T04:05:43.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:42 smithi070 ceph-mon[35495]: pgmap v5: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail 2024-09-10T04:05:44.266 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:44 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:44.266 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:44 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:44.266 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:44 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:44.266 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:44 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:44.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:44 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:44.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:44 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:44.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:44 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:44.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:44 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:45.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:45 smithi070 ceph-mon[35495]: pgmap v6: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail 2024-09-10T04:05:45.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:45 smithi070 ceph-mon[35495]: Standby manager daemon smithi046.zvckjg started 2024-09-10T04:05:45.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:45 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/crt"}]: dispatch 2024-09-10T04:05:45.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:45 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:05:45.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:45 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/key"}]: dispatch 2024-09-10T04:05:45.163 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:45 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:05:45.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:45 smithi046 ceph-mon[28328]: pgmap v6: 65 pgs: 65 active+clean; 4.2 GiB data, 12 GiB used, 524 GiB / 536 GiB avail 2024-09-10T04:05:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:45 smithi046 ceph-mon[28328]: Standby manager daemon smithi046.zvckjg started 2024-09-10T04:05:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:45 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/crt"}]: dispatch 2024-09-10T04:05:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:45 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:05:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:45 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/key"}]: dispatch 2024-09-10T04:05:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:45 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/953844865' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:05:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:46 smithi070 ceph-mon[35495]: mgrmap e24: smithi070.tezbzn(active, since 7s), standbys: smithi046.zvckjg 2024-09-10T04:05:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:46 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T04:05:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:46 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:46 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:46.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:46 smithi046 ceph-mon[28328]: mgrmap e24: smithi070.tezbzn(active, since 7s), standbys: smithi046.zvckjg 2024-09-10T04:05:46.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:46 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T04:05:46.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:46 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:46.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:46 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: pgmap v7: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 958 KiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.281 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:47 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:47.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: pgmap v7: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 958 KiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:47.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:47 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi046 to 1217M 2024-09-10T04:05:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:48 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi046 to 1217M 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:48.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:48 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Adjusting osd_memory_target on smithi070 to 2240M 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: pgmap v8: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 743 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:05:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Reconfiguring ceph-exporter.smithi046 (dependencies changed)... 2024-09-10T04:05:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:49 smithi070 ceph-mon[35495]: Deploying daemon ceph-exporter.smithi046 on smithi046 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Adjusting osd_memory_target on smithi070 to 2240M 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: pgmap v8: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 743 KiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Reconfiguring ceph-exporter.smithi046 (dependencies changed)... 2024-09-10T04:05:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:49 smithi046 ceph-mon[28328]: Deploying daemon ceph-exporter.smithi046 on smithi046 2024-09-10T04:05:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.430+0000 7fd85e4a5640 1 -- 172.21.15.46:0/51972754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 msgr2=0x7fd858073680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:49.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.430+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/51972754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd858073680 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd8400097e0 tx=0x7fd84002a3f0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.431+0000 7fd85e4a5640 1 -- 172.21.15.46:0/51972754 shutdown_connections 2024-09-10T04:05:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.431+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/51972754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8581091f0 0x7fd858109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.431+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/51972754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd858073680 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.431+0000 7fd85e4a5640 1 -- 172.21.15.46:0/51972754 >> 172.21.15.46:0/51972754 conn(0x7fd85806ea50 msgr2=0x7fd858070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:49.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.432+0000 7fd85e4a5640 1 -- 172.21.15.46:0/51972754 shutdown_connections 2024-09-10T04:05:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.433+0000 7fd85e4a5640 1 -- 172.21.15.46:0/51972754 wait complete. 2024-09-10T04:05:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.434+0000 7fd85e4a5640 1 Processor -- start 2024-09-10T04:05:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.434+0000 7fd85e4a5640 1 -- start start 2024-09-10T04:05:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd85e4a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd8581a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd85e4a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8581091f0 0x7fd8581a6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:49.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd85e4a5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8581195b0 con 0x7fd8580732a0 2024-09-10T04:05:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd85e4a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd858119430 con 0x7fd8581091f0 2024-09-10T04:05:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd857fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd8581a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd857fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd8581a6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49212/0 (socket says 172.21.15.46:49212) 2024-09-10T04:05:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd857fff640 1 -- 172.21.15.46:0/1618143089 learned_addr learned my addr 172.21.15.46:0/1618143089 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:49.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.435+0000 7fd8577fe640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8581091f0 0x7fd8581a6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.436+0000 7fd857fff640 1 -- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8581091f0 msgr2=0x7fd8581a6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.436+0000 7fd857fff640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8581091f0 0x7fd8581a6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.436+0000 7fd857fff640 1 -- 172.21.15.46:0/1618143089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd840030a30 con 0x7fd8580732a0 2024-09-10T04:05:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.436+0000 7fd857fff640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd8581a6210 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd840030750 tx=0x7fd8400042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.437+0000 7fd8557fa640 1 -- 172.21.15.46:0/1618143089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd840040070 con 0x7fd8580732a0 2024-09-10T04:05:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.437+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd858078440 con 0x7fd8580732a0 2024-09-10T04:05:49.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.437+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd858078970 con 0x7fd8580732a0 2024-09-10T04:05:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.438+0000 7fd8557fa640 1 -- 172.21.15.46:0/1618143089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8400044d0 con 0x7fd8580732a0 2024-09-10T04:05:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.438+0000 7fd8557fa640 1 -- 172.21.15.46:0/1618143089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd840045690 con 0x7fd8580732a0 2024-09-10T04:05:49.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.440+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8380052c0 con 0x7fd8580732a0 2024-09-10T04:05:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.441+0000 7fd8557fa640 1 -- 172.21.15.46:0/1618143089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7fd840039040 con 0x7fd8580732a0 2024-09-10T04:05:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.442+0000 7fd8557fa640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fd8440747f0 0x7fd844076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:49.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.442+0000 7fd8557fa640 1 -- 172.21.15.46:0/1618143089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7fd8400b7330 con 0x7fd8580732a0 2024-09-10T04:05:49.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.444+0000 7fd8577fe640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fd8440747f0 0x7fd844076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:49.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.450+0000 7fd8577fe640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fd8440747f0 0x7fd844076cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8581a7340 tx=0x7fd848009260 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:49.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.452+0000 7fd8557fa640 1 -- 172.21.15.46:0/1618143089 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fd840082ac0 con 0x7fd8580732a0 2024-09-10T04:05:49.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.808+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd838000bf0 con 0x7fd8440747f0 2024-09-10T04:05:49.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.810+0000 7fd8557fa640 1 -- 172.21.15.46:0/1618143089 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7fd838000bf0 con 0x7fd8440747f0 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.812+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fd8440747f0 msgr2=0x7fd844076cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.812+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fd8440747f0 0x7fd844076cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd8581a7340 tx=0x7fd848009260 comp rx=0 tx=0).stop 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 msgr2=0x7fd8581a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd8581a6210 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd840030750 tx=0x7fd8400042e0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 shutdown_connections 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fd8440747f0 0x7fd844076cb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8581091f0 0x7fd8581a6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 --2- 172.21.15.46:0/1618143089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8580732a0 0x7fd8581a6210 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 >> 172.21.15.46:0/1618143089 conn(0x7fd85806ea50 msgr2=0x7fd858071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 shutdown_connections 2024-09-10T04:05:49.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.813+0000 7fd85e4a5640 1 -- 172.21.15.46:0/1618143089 wait complete. 2024-09-10T04:05:49.825 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:05:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.929+0000 7f13bf3d1640 1 -- 172.21.15.46:0/2796086564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b81001e0 msgr2=0x7f13b81005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.929+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/2796086564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b81001e0 0x7f13b81005e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009880 tx=0x7f13b402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.930+0000 7f13bf3d1640 1 -- 172.21.15.46:0/2796086564 shutdown_connections 2024-09-10T04:05:49.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.930+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/2796086564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b8100b20 0x7f13b8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.930+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/2796086564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b81001e0 0x7f13b81005e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.930+0000 7f13bf3d1640 1 -- 172.21.15.46:0/2796086564 >> 172.21.15.46:0/2796086564 conn(0x7f13b80fc010 msgr2=0x7f13b80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.930+0000 7f13bf3d1640 1 -- 172.21.15.46:0/2796086564 shutdown_connections 2024-09-10T04:05:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.930+0000 7f13bf3d1640 1 -- 172.21.15.46:0/2796086564 wait complete. 2024-09-10T04:05:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.930+0000 7f13bf3d1640 1 Processor -- start 2024-09-10T04:05:49.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bf3d1640 1 -- start start 2024-09-10T04:05:49.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bf3d1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b81001e0 0x7f13b8073200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bf3d1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b8100b20 0x7f13b8073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bf3d1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13b810c970 con 0x7f13b8100b20 2024-09-10T04:05:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bf3d1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f13b810c7f0 con 0x7f13b81001e0 2024-09-10T04:05:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bc96c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b8100b20 0x7f13b8073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bd16d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b81001e0 0x7f13b8073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:49.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bc96c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b8100b20 0x7f13b8073740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49230/0 (socket says 172.21.15.46:49230) 2024-09-10T04:05:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.931+0000 7f13bc96c640 1 -- 172.21.15.46:0/4266487894 learned_addr learned my addr 172.21.15.46:0/4266487894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13bc96c640 1 -- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b81001e0 msgr2=0x7f13b8073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13bc96c640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b81001e0 0x7f13b8073200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13bc96c640 1 -- 172.21.15.46:0/4266487894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b8073de0 con 0x7f13b8100b20 2024-09-10T04:05:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13bc96c640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b8100b20 0x7f13b8073740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f13a800e9e0 tx=0x7f13a800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13ae7fc640 1 -- 172.21.15.46:0/4266487894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13a800cb50 con 0x7f13b8100b20 2024-09-10T04:05:49.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13ae7fc640 1 -- 172.21.15.46:0/4266487894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13a800ccf0 con 0x7f13b8100b20 2024-09-10T04:05:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13ae7fc640 1 -- 172.21.15.46:0/4266487894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f13a80049d0 con 0x7f13b8100b20 2024-09-10T04:05:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.932+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b81a74c0 con 0x7f13b8100b20 2024-09-10T04:05:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.933+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b81a7a00 con 0x7f13b8100b20 2024-09-10T04:05:49.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.934+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b81005e0 con 0x7f13b8100b20 2024-09-10T04:05:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.935+0000 7f13ae7fc640 1 -- 172.21.15.46:0/4266487894 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7f13a8004020 con 0x7f13b8100b20 2024-09-10T04:05:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.935+0000 7f13ae7fc640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f13a40746a0 0x7f13a4076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.936+0000 7f13ae7fc640 1 -- 172.21.15.46:0/4266487894 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7f13a8095e80 con 0x7f13b8100b20 2024-09-10T04:05:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.936+0000 7f13bd16d640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f13a40746a0 0x7f13a4076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:49.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.937+0000 7f13bd16d640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f13a40746a0 0x7f13a4076b60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13b4000c00 tx=0x7f13b4005a20 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:49.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:49.944+0000 7f13ae7fc640 1 -- 172.21.15.46:0/4266487894 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f13a8061630 con 0x7f13b8100b20 2024-09-10T04:05:50.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.242+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f13b8062630 con 0x7f13a40746a0 2024-09-10T04:05:50.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.243+0000 7f13ae7fc640 1 -- 172.21.15.46:0/4266487894 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7f13b8062630 con 0x7f13a40746a0 2024-09-10T04:05:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.245+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f13a40746a0 msgr2=0x7f13a4076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.245+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f13a40746a0 0x7f13a4076b60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13b4000c00 tx=0x7f13b4005a20 comp rx=0 tx=0).stop 2024-09-10T04:05:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.245+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b8100b20 msgr2=0x7f13b8073740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.245+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b8100b20 0x7f13b8073740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f13a800e9e0 tx=0x7f13a800eea0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.246+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 shutdown_connections 2024-09-10T04:05:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.246+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f13a40746a0 0x7f13a4076b60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.246+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b8100b20 0x7f13b8073740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.246+0000 7f13bf3d1640 1 --2- 172.21.15.46:0/4266487894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b81001e0 0x7f13b8073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.246+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 >> 172.21.15.46:0/4266487894 conn(0x7f13b80fc010 msgr2=0x7f13b8108a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.246+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 shutdown_connections 2024-09-10T04:05:50.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.246+0000 7f13bf3d1640 1 -- 172.21.15.46:0/4266487894 wait complete. 2024-09-10T04:05:50.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 -- 172.21.15.46:0/839759227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 msgr2=0x7fe600107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 --2- 172.21.15.46:0/839759227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600107130 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40098b0 tx=0x7fe5f402a4d0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 -- 172.21.15.46:0/839759227 shutdown_connections 2024-09-10T04:05:50.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 --2- 172.21.15.46:0/839759227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600107130 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 --2- 172.21.15.46:0/839759227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600102410 0x7fe600104800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 -- 172.21.15.46:0/839759227 >> 172.21.15.46:0/839759227 conn(0x7fe6000fc010 msgr2=0x7fe6000fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:50.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 -- 172.21.15.46:0/839759227 shutdown_connections 2024-09-10T04:05:50.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.356+0000 7fe605b5c640 1 -- 172.21.15.46:0/839759227 wait complete. 2024-09-10T04:05:50.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.357+0000 7fe605b5c640 1 Processor -- start 2024-09-10T04:05:50.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.357+0000 7fe605b5c640 1 -- start start 2024-09-10T04:05:50.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe605b5c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600102410 0x7fe600197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:50.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe605b5c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:50.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe605b5c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe60010a7b0 con 0x7fe600104d40 2024-09-10T04:05:50.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe605b5c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe60010a630 con 0x7fe600102410 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5feffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5feffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49250/0 (socket says 172.21.15.46:49250) 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5feffd640 1 -- 172.21.15.46:0/3758000004 learned_addr learned my addr 172.21.15.46:0/3758000004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5ff7fe640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600102410 0x7fe600197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5feffd640 1 -- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600102410 msgr2=0x7fe600197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5feffd640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600102410 0x7fe600197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5feffd640 1 -- 172.21.15.46:0/3758000004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6001980f0 con 0x7fe600104d40 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.358+0000 7fe5ff7fe640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600102410 0x7fe600197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:05:50.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.359+0000 7fe5feffd640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600197a70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009230 tx=0x7fe5f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:50.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.359+0000 7fe5fcff9640 1 -- 172.21.15.46:0/3758000004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5f403f070 con 0x7fe600104d40 2024-09-10T04:05:50.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.359+0000 7fe5fcff9640 1 -- 172.21.15.46:0/3758000004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5f40049d0 con 0x7fe600104d40 2024-09-10T04:05:50.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.359+0000 7fe5fcff9640 1 -- 172.21.15.46:0/3758000004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5f4035030 con 0x7fe600104d40 2024-09-10T04:05:50.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.359+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60019cad0 con 0x7fe600104d40 2024-09-10T04:05:50.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.359+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60019d060 con 0x7fe600104d40 2024-09-10T04:05:50.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.360+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6001027f0 con 0x7fe600104d40 2024-09-10T04:05:50.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.361+0000 7fe5fcff9640 1 -- 172.21.15.46:0/3758000004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7fe5f4042070 con 0x7fe600104d40 2024-09-10T04:05:50.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.361+0000 7fe5fcff9640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fe5e80746a0 0x7fe5e8076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:50.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.362+0000 7fe5fcff9640 1 -- 172.21.15.46:0/3758000004 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7fe5f40b6840 con 0x7fe600104d40 2024-09-10T04:05:50.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.362+0000 7fe5ff7fe640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fe5e80746a0 0x7fe5e8076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:50.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.362+0000 7fe5ff7fe640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fe5e80746a0 0x7fe5e8076b60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009c20 tx=0x7fe5f0005f90 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:50.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.369+0000 7fe5fcff9640 1 -- 172.21.15.46:0/3758000004 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fe5f4081ff0 con 0x7fe600104d40 2024-09-10T04:05:50.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.693+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe600105720 con 0x7fe5e80746a0 2024-09-10T04:05:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.703+0000 7fe5fcff9640 1 -- 172.21.15.46:0/3758000004 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fe600105720 con 0x7fe5e80746a0 2024-09-10T04:05:50.705 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:05:50.705 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (7m) 4s ago 8m 26.4M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:05:50.706 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (8m) 4s ago 8m 8896k - 19.1.1-298-gb53b6d3e 54467abc55f8 3888d245a02a 2024-09-10T04:05:50.706 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (8m) 7s ago 8m 5268k - 19.1.1-298-gb53b6d3e 54467abc55f8 54f37f2cb1f7 2024-09-10T04:05:50.706 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (8m) 4s ago 8m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:05:50.706 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (8m) 7s ago 8m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:05:50.706 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (7m) 4s ago 8m 83.0M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (3m) 4s ago 3m 265M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (3m) 4s ago 3m 244M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (3m) 7s ago 3m 22.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (3m) 7s ago 3m 22.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:9283,8765,8443 running (10m) 4s ago 10m 244M - 19.1.1-298-gb53b6d3e 54467abc55f8 a4fa65bf88c5 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (31s) 7s ago 7m 494M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9bcab2dbf2ab 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (10m) 4s ago 10m 52.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (7m) 7s ago 7m 40.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:05:50.707 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (8m) 4s ago 8m 15.6M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 7s ago 7m 16.7M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (6m) 4s ago 6m 216M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (6m) 4s ago 6m 214M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (5m) 4s ago 5m 172M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (5m) 7s ago 5m 234M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (4m) 7s ago 4m 194M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (4m) 7s ago 4m 227M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:05:50.708 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (6m) 4s ago 8m 64.2M - 2.43.0 a07b618ecd1d 15eb5a503403 2024-09-10T04:05:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fe5e80746a0 msgr2=0x7fe5e8076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fe5e80746a0 0x7fe5e8076b60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009c20 tx=0x7fe5f0005f90 comp rx=0 tx=0).stop 2024-09-10T04:05:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 msgr2=0x7fe600197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600197a70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009230 tx=0x7fe5f40045e0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 shutdown_connections 2024-09-10T04:05:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fe5e80746a0 0x7fe5e8076b60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe600104d40 0x7fe600197a70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 --2- 172.21.15.46:0/3758000004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe600102410 0x7fe600197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 >> 172.21.15.46:0/3758000004 conn(0x7fe6000fc010 msgr2=0x7fe6000fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 shutdown_connections 2024-09-10T04:05:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.707+0000 7fe605b5c640 1 -- 172.21.15.46:0/3758000004 wait complete. 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 -- 172.21.15.46:0/947315273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 msgr2=0x7fc974104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 --2- 172.21.15.46:0/947315273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc974104fc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc968009880 tx=0x7fc96802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 -- 172.21.15.46:0/947315273 shutdown_connections 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 --2- 172.21.15.46:0/947315273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc974104fc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 --2- 172.21.15.46:0/947315273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc974100200 0x7fc974102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 -- 172.21.15.46:0/947315273 >> 172.21.15.46:0/947315273 conn(0x7fc9740fc010 msgr2=0x7fc9740fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 -- 172.21.15.46:0/947315273 shutdown_connections 2024-09-10T04:05:50.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.816+0000 7fc97a966640 1 -- 172.21.15.46:0/947315273 wait complete. 2024-09-10T04:05:50.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc97a966640 1 Processor -- start 2024-09-10T04:05:50.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc97a966640 1 -- start start 2024-09-10T04:05:50.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc97a966640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc974100200 0x7fc97419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:50.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc97a966640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc97419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc9737fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc97419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc9737fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc97419dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49272/0 (socket says 172.21.15.46:49272) 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc9737fe640 1 -- 172.21.15.46:0/1211675165 learned_addr learned my addr 172.21.15.46:0/1211675165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc974110d90 con 0x7fc974102b50 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.817+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc974110c10 con 0x7fc974100200 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc973fff640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc974100200 0x7fc97419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc9737fe640 1 -- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc974100200 msgr2=0x7fc97419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc9737fe640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc974100200 0x7fc97419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc9737fe640 1 -- 172.21.15.46:0/1211675165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97419e640 con 0x7fc974102b50 2024-09-10T04:05:50.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc9737fe640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc97419dfc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc968006010 tx=0x7fc9680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:50.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc9717fa640 1 -- 172.21.15.46:0/1211675165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc96803f070 con 0x7fc974102b50 2024-09-10T04:05:50.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc9717fa640 1 -- 172.21.15.46:0/1211675165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc968004ac0 con 0x7fc974102b50 2024-09-10T04:05:50.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9741a3020 con 0x7fc974102b50 2024-09-10T04:05:50.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc9717fa640 1 -- 172.21.15.46:0/1211675165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc968035030 con 0x7fc974102b50 2024-09-10T04:05:50.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.818+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9741a35b0 con 0x7fc974102b50 2024-09-10T04:05:50.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.820+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc974100600 con 0x7fc974102b50 2024-09-10T04:05:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.820+0000 7fc9717fa640 1 -- 172.21.15.46:0/1211675165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7fc968042070 con 0x7fc974102b50 2024-09-10T04:05:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.821+0000 7fc9717fa640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fc95c0746a0 0x7fc95c076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.821+0000 7fc9717fa640 1 -- 172.21.15.46:0/1211675165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7fc9680b6860 con 0x7fc974102b50 2024-09-10T04:05:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.821+0000 7fc973fff640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fc95c0746a0 0x7fc95c076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:50.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.822+0000 7fc973fff640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fc95c0746a0 0x7fc95c076b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc964005d00 tx=0x7fc96400b410 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:50.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:50.827+0000 7fc9717fa640 1 -- 172.21.15.46:0/1211675165 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fc968082010 con 0x7fc974102b50 2024-09-10T04:05:51.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.210+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc9741a3860 con 0x7fc974102b50 2024-09-10T04:05:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.210+0000 7fc9717fa640 1 -- 172.21.15.46:0/1211675165 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+841 (secure 0 0 0) 0x7fc968084530 con 0x7fc974102b50 2024-09-10T04:05:51.211 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:05:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:05:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:05:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:05:51.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 13, 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:05:51.212 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:05:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.212+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fc95c0746a0 msgr2=0x7fc95c076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.212+0000 7fc97a966640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fc95c0746a0 0x7fc95c076b60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc964005d00 tx=0x7fc96400b410 comp rx=0 tx=0).stop 2024-09-10T04:05:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.212+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 msgr2=0x7fc97419dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.212+0000 7fc97a966640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc97419dfc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc968006010 tx=0x7fc9680046c0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.213+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 shutdown_connections 2024-09-10T04:05:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.213+0000 7fc97a966640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fc95c0746a0 0x7fc95c076b60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.213+0000 7fc97a966640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc974102b50 0x7fc97419dfc0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.213+0000 7fc97a966640 1 --2- 172.21.15.46:0/1211675165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc974100200 0x7fc97419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.213+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 >> 172.21.15.46:0/1211675165 conn(0x7fc9740fc010 msgr2=0x7fc9740fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.213+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 shutdown_connections 2024-09-10T04:05:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.213+0000 7fc97a966640 1 -- 172.21.15.46:0/1211675165 wait complete. 2024-09-10T04:05:51.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:51 smithi070 ceph-mon[35495]: pgmap v9: 65 pgs: 65 active+clean; 2.3 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-09-10T04:05:51.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:51 smithi070 ceph-mon[35495]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:51.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:51 smithi046 ceph-mon[28328]: pgmap v9: 65 pgs: 65 active+clean; 2.3 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 177 op/s 2024-09-10T04:05:51.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:51 smithi046 ceph-mon[28328]: from='client.14714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/2618606871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 msgr2=0x7f6b68101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/2618606871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68101d70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b580097e0 tx=0x7f6b5802a3f0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/2618606871 shutdown_connections 2024-09-10T04:05:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/2618606871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b68102b70 0x7f6b68102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/2618606871 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68101d70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/2618606871 >> 172.21.15.46:0/2618606871 conn(0x7f6b680fd140 msgr2=0x7f6b680ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/2618606871 shutdown_connections 2024-09-10T04:05:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.344+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/2618606871 wait complete. 2024-09-10T04:05:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.345+0000 7f6b6c8c9640 1 Processor -- start 2024-09-10T04:05:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.345+0000 7f6b6c8c9640 1 -- start start 2024-09-10T04:05:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.345+0000 7f6b6c8c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.345+0000 7f6b6c8c9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b68102b70 0x7f6b680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.345+0000 7f6b6c8c9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6810a790 con 0x7f6b68102b70 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.345+0000 7f6b6c8c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6810a610 con 0x7f6b68101970 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b65d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b68102b70 0x7f6b680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b6659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b6659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:48118/0 (socket says 172.21.15.46:48118) 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b65d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b68102b70 0x7f6b680745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49288/0 (socket says 172.21.15.46:49288) 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b6659c640 1 -- 172.21.15.46:0/718529041 learned_addr learned my addr 172.21.15.46:0/718529041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b6659c640 1 -- 172.21.15.46:0/718529041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b68102b70 msgr2=0x7f6b680745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b6659c640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b68102b70 0x7f6b680745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b6659c640 1 -- 172.21.15.46:0/718529041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b68074b20 con 0x7f6b68101970 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.346+0000 7f6b6659c640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68075f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b58030750 tx=0x7f6b580043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.347+0000 7f6b577fe640 1 -- 172.21.15.46:0/718529041 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b58040070 con 0x7f6b68101970 2024-09-10T04:05:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.347+0000 7f6b577fe640 1 -- 172.21.15.46:0/718529041 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b58030c00 con 0x7f6b68101970 2024-09-10T04:05:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.347+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b68074db0 con 0x7f6b68101970 2024-09-10T04:05:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.347+0000 7f6b577fe640 1 -- 172.21.15.46:0/718529041 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b58003cd0 con 0x7f6b68101970 2024-09-10T04:05:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.347+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b680752e0 con 0x7f6b68101970 2024-09-10T04:05:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.348+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b68101d70 con 0x7f6b68101970 2024-09-10T04:05:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.350+0000 7f6b577fe640 1 -- 172.21.15.46:0/718529041 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7f6b58039040 con 0x7f6b68101970 2024-09-10T04:05:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.351+0000 7f6b577fe640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f6b500746a0 0x7f6b50076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:05:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.351+0000 7f6b577fe640 1 -- 172.21.15.46:0/718529041 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7f6b580b7470 con 0x7f6b68101970 2024-09-10T04:05:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.351+0000 7f6b65d9b640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f6b500746a0 0x7f6b50076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:05:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.352+0000 7f6b65d9b640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f6b500746a0 0x7f6b50076b60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b68075ca0 tx=0x7f6b5c009260 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:05:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.354+0000 7f6b577fe640 1 -- 172.21.15.46:0/718529041 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f6b58082cc0 con 0x7f6b68101970 2024-09-10T04:05:51.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.735+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6b6819f540 con 0x7f6b500746a0 2024-09-10T04:05:51.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.736+0000 7f6b577fe640 1 -- 172.21.15.46:0/718529041 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+310 (secure 0 0 0) 0x7f6b6819f540 con 0x7f6b500746a0 2024-09-10T04:05:51.738 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:05:51.738 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:05:51.738 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:05:51.739 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:05:51.739 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [], 2024-09-10T04:05:51.739 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "1/2 daemons upgraded", 2024-09-10T04:05:51.739 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-09-10T04:05:51.739 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:05:51.739 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:05:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f6b500746a0 msgr2=0x7f6b50076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f6b500746a0 0x7f6b50076b60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b68075ca0 tx=0x7f6b5c009260 comp rx=0 tx=0).stop 2024-09-10T04:05:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 msgr2=0x7f6b68075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:05:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68075f20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6b58030750 tx=0x7f6b580043b0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 shutdown_connections 2024-09-10T04:05:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f6b500746a0 0x7f6b50076b60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b68102b70 0x7f6b680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 --2- 172.21.15.46:0/718529041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b68101970 0x7f6b68075f20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:05:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 >> 172.21.15.46:0/718529041 conn(0x7f6b680fd140 msgr2=0x7f6b680ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:05:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 shutdown_connections 2024-09-10T04:05:51.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:05:51.739+0000 7f6b6c8c9640 1 -- 172.21.15.46:0/718529041 wait complete. 2024-09-10T04:05:52.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:52 smithi070 ceph-mon[35495]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:52.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:52 smithi070 ceph-mon[35495]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:52.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:52 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1211675165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:05:52.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:52 smithi046 ceph-mon[28328]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:52.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:52 smithi046 ceph-mon[28328]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:52.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:52 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1211675165' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:05:53.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:53 smithi070 ceph-mon[35495]: pgmap v10: 65 pgs: 65 active+clean; 2.3 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-09-10T04:05:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:53 smithi070 ceph-mon[35495]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:53 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:53 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:53.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:53 smithi046 ceph-mon[28328]: pgmap v10: 65 pgs: 65 active+clean; 2.3 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-09-10T04:05:53.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:53 smithi046 ceph-mon[28328]: from='client.24481 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:05:53.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:53 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:53.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:53 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:54.097 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T04:05:54.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-10T04:05:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:54 smithi070 ceph-mon[35495]: Reconfiguring prometheus.smithi046 (dependencies changed)... 2024-09-10T04:05:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:54 smithi070 ceph-mon[35495]: Reconfiguring daemon prometheus.smithi046 on smithi046 2024-09-10T04:05:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:54 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:54.447 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:54 smithi046 ceph-mon[28328]: Reconfiguring prometheus.smithi046 (dependencies changed)... 2024-09-10T04:05:54.447 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:54 smithi046 ceph-mon[28328]: Reconfiguring daemon prometheus.smithi046 on smithi046 2024-09-10T04:05:54.447 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:54 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:05:55.426 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:55 smithi046 ceph-mon[28328]: pgmap v11: 65 pgs: 65 active+clean; 2.3 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-09-10T04:05:55.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:55 smithi070 ceph-mon[35495]: pgmap v11: 65 pgs: 65 active+clean; 2.3 GiB data, 7.3 GiB used, 529 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-09-10T04:05:57.097 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T04:05:57.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T04:05:57.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:57 smithi046 ceph-mon[28328]: pgmap v12: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-09-10T04:05:57.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:57 smithi070 ceph-mon[35495]: pgmap v12: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-09-10T04:05:59.256 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:59 smithi070 ceph-mon[35495]: pgmap v13: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-10T04:05:59.256 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:59 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:59.257 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:59 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:59.257 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:59 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:59.257 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:59 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:59.257 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:05:59 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:05:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:59 smithi046 ceph-mon[28328]: pgmap v13: 65 pgs: 65 active+clean; 1.2 GiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-09-10T04:05:59.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:59 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:59.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:59 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:05:59.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:59 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:59.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:59 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T04:05:59.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:05:59 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:06:00.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:00 smithi070 ceph-mon[35495]: Reconfiguring ceph-exporter.smithi070 (dependencies changed)... 2024-09-10T04:06:00.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:00 smithi070 ceph-mon[35495]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-09-10T04:06:00.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:00 smithi046 ceph-mon[28328]: Reconfiguring ceph-exporter.smithi070 (dependencies changed)... 2024-09-10T04:06:00.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:00 smithi046 ceph-mon[28328]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-09-10T04:06:01.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:01 smithi070 ceph-mon[35495]: pgmap v14: 65 pgs: 65 active+clean; 289 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-10T04:06:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:01 smithi046 ceph-mon[28328]: pgmap v14: 65 pgs: 65 active+clean; 289 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-09-10T04:06:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:03 smithi070 ceph-mon[35495]: pgmap v15: 65 pgs: 65 active+clean; 289 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-10T04:06:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:03 smithi046 ceph-mon[28328]: pgmap v15: 65 pgs: 65 active+clean; 289 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-10T04:06:05.158 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:05 smithi070 ceph-mon[35495]: pgmap v16: 65 pgs: 65 active+clean; 289 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-10T04:06:05.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:05 smithi046 ceph-mon[28328]: pgmap v16: 65 pgs: 65 active+clean; 289 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-09-10T04:06:06.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:06:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:06:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: Upgrade: Need to upgrade myself (mgr.smithi070.tezbzn) 2024-09-10T04:06:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: Upgrade: Need to upgrade myself (mgr.smithi070.tezbzn) 2024-09-10T04:06:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:06 smithi070 ceph-mon[35495]: pgmap v17: 65 pgs: 65 active+clean; 285 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-10T04:06:06.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:06.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:06.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:06:06.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:06:06.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:06.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: Upgrade: Need to upgrade myself (mgr.smithi070.tezbzn) 2024-09-10T04:06:06.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: Upgrade: Need to upgrade myself (mgr.smithi070.tezbzn) 2024-09-10T04:06:06.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:06 smithi046 ceph-mon[28328]: pgmap v17: 65 pgs: 65 active+clean; 285 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-09-10T04:06:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:08 smithi046 ceph-mon[28328]: pgmap v18: 65 pgs: 65 active+clean; 285 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-10T04:06:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:08 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:08 smithi070 ceph-mon[35495]: pgmap v18: 65 pgs: 65 active+clean; 285 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-09-10T04:06:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:08 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:10.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:10 smithi046 ceph-mon[28328]: pgmap v19: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-10T04:06:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:10 smithi070 ceph-mon[35495]: pgmap v19: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-10T04:06:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:12 smithi070 ceph-mon[35495]: pgmap v20: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-09-10T04:06:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:12 smithi046 ceph-mon[28328]: pgmap v20: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-09-10T04:06:15.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:14 smithi070 ceph-mon[35495]: pgmap v21: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-09-10T04:06:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:14 smithi046 ceph-mon[28328]: pgmap v21: 65 pgs: 65 active+clean; 292 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-09-10T04:06:17.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:16 smithi070 ceph-mon[35495]: pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2024-09-10T04:06:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:16 smithi046 ceph-mon[28328]: pgmap v22: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: Upgrade: Updating mgr.smithi046.zvckjg 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi046.zvckjg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi046.zvckjg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: Deploying daemon mgr.smithi046.zvckjg on smithi046 2024-09-10T04:06:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:18 smithi070 ceph-mon[35495]: pgmap v23: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 73 op/s 2024-09-10T04:06:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: Upgrade: Updating mgr.smithi046.zvckjg 2024-09-10T04:06:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi046.zvckjg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:06:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi046.zvckjg", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:06:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T04:06:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:06:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: Deploying daemon mgr.smithi046.zvckjg on smithi046 2024-09-10T04:06:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:18 smithi046 ceph-mon[28328]: pgmap v23: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 73 op/s 2024-09-10T04:06:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:20 smithi070 ceph-mon[35495]: pgmap v24: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-10T04:06:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:20 smithi046 ceph-mon[28328]: pgmap v24: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-10T04:06:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 -- 172.21.15.46:0/1152753595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c104d20 msgr2=0x7f095c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 --2- 172.21.15.46:0/1152753595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c104d20 0x7f095c107110 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097e0 tx=0x7f094c02a3f0 comp rx=0 tx=0).stop 2024-09-10T04:06:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 -- 172.21.15.46:0/1152753595 shutdown_connections 2024-09-10T04:06:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 --2- 172.21.15.46:0/1152753595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c104d20 0x7f095c107110 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 --2- 172.21.15.46:0/1152753595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c1023f0 0x7f095c1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 -- 172.21.15.46:0/1152753595 >> 172.21.15.46:0/1152753595 conn(0x7f095c0fc010 msgr2=0x7f095c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 -- 172.21.15.46:0/1152753595 shutdown_connections 2024-09-10T04:06:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.839+0000 7f09641f5640 1 -- 172.21.15.46:0/1152753595 wait complete. 2024-09-10T04:06:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.840+0000 7f09641f5640 1 Processor -- start 2024-09-10T04:06:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.840+0000 7f09641f5640 1 -- start start 2024-09-10T04:06:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.840+0000 7f09641f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c1023f0 0x7f095c1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:21.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.840+0000 7f09641f5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c104d20 0x7f095c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.840+0000 7f09641f5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10a790 con 0x7f095c104d20 2024-09-10T04:06:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.840+0000 7f09641f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f095c10a610 con 0x7f095c1023f0 2024-09-10T04:06:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c104d20 0x7f095c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c104d20 0x7f095c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50990/0 (socket says 172.21.15.46:50990) 2024-09-10T04:06:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961790640 1 -- 172.21.15.46:0/2789844291 learned_addr learned my addr 172.21.15.46:0/2789844291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961f91640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c1023f0 0x7f095c1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961790640 1 -- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c1023f0 msgr2=0x7f095c1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961790640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c1023f0 0x7f095c1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961790640 1 -- 172.21.15.46:0/2789844291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095c1980b0 con 0x7f095c104d20 2024-09-10T04:06:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961f91640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c1023f0 0x7f095c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:06:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.841+0000 7f0961790640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c104d20 0x7f095c197a30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f094c004650 tx=0x7f094c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.842+0000 7f0952ffd640 1 -- 172.21.15.46:0/2789844291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f094c040070 con 0x7f095c104d20 2024-09-10T04:06:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.842+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095c19ca90 con 0x7f095c104d20 2024-09-10T04:06:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.842+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f095c19d020 con 0x7f095c104d20 2024-09-10T04:06:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.842+0000 7f0952ffd640 1 -- 172.21.15.46:0/2789844291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f094c0318c0 con 0x7f095c104d20 2024-09-10T04:06:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.842+0000 7f0952ffd640 1 -- 172.21.15.46:0/2789844291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f094c0039c0 con 0x7f095c104d20 2024-09-10T04:06:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.843+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09400052c0 con 0x7f095c104d20 2024-09-10T04:06:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.844+0000 7f0952ffd640 1 -- 172.21.15.46:0/2789844291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7f094c039040 con 0x7f095c104d20 2024-09-10T04:06:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.844+0000 7f0952ffd640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f09480746a0 0x7f0948076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.844+0000 7f0952ffd640 1 -- 172.21.15.46:0/2789844291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7f094c0b6ff0 con 0x7f095c104d20 2024-09-10T04:06:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.844+0000 7f0961f91640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f09480746a0 0x7f0948076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.845+0000 7f0961f91640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f09480746a0 0x7f0948076b60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f09580096c0 tx=0x7f0958009260 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:21.853+0000 7f0952ffd640 1 -- 172.21.15.46:0/2789844291 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f094c0828c0 con 0x7f095c104d20 2024-09-10T04:06:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.211+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0940000bf0 con 0x7f09480746a0 2024-09-10T04:06:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.213+0000 7f0952ffd640 1 -- 172.21.15.46:0/2789844291 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f0940000bf0 con 0x7f09480746a0 2024-09-10T04:06:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f09480746a0 msgr2=0x7f0948076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f09480746a0 0x7f0948076b60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f09580096c0 tx=0x7f0958009260 comp rx=0 tx=0).stop 2024-09-10T04:06:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c104d20 msgr2=0x7f095c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c104d20 0x7f095c197a30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f094c004650 tx=0x7f094c0305d0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 shutdown_connections 2024-09-10T04:06:22.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f09480746a0 0x7f0948076b60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f095c104d20 0x7f095c197a30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 --2- 172.21.15.46:0/2789844291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f095c1023f0 0x7f095c1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 >> 172.21.15.46:0/2789844291 conn(0x7f095c0fc010 msgr2=0x7f095c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:22.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 shutdown_connections 2024-09-10T04:06:22.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.216+0000 7f09641f5640 1 -- 172.21.15.46:0/2789844291 wait complete. 2024-09-10T04:06:22.228 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.314+0000 7f259e122640 1 -- 172.21.15.46:0/3295005197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 msgr2=0x7f2598107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.314+0000 7f259e122640 1 --2- 172.21.15.46:0/3295005197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 0x7f2598107110 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f258c0097e0 tx=0x7f258c02a3f0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.315+0000 7f259e122640 1 -- 172.21.15.46:0/3295005197 shutdown_connections 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.315+0000 7f259e122640 1 --2- 172.21.15.46:0/3295005197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 0x7f2598107110 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.315+0000 7f259e122640 1 --2- 172.21.15.46:0/3295005197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 0x7f25981047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.315+0000 7f259e122640 1 -- 172.21.15.46:0/3295005197 >> 172.21.15.46:0/3295005197 conn(0x7f25980fc010 msgr2=0x7f25980fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.315+0000 7f259e122640 1 -- 172.21.15.46:0/3295005197 shutdown_connections 2024-09-10T04:06:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.315+0000 7f259e122640 1 -- 172.21.15.46:0/3295005197 wait complete. 2024-09-10T04:06:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.315+0000 7f259e122640 1 Processor -- start 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f259e122640 1 -- start start 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f259e122640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 0x7f25981996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f259e122640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 0x7f2598199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f259e122640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f259810a790 con 0x7f25981023f0 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f259e122640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f259810a610 con 0x7f2598104d20 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f25977fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 0x7f25981996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f25977fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 0x7f25981996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51000/0 (socket says 172.21.15.46:51000) 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f25977fe640 1 -- 172.21.15.46:0/2104237619 learned_addr learned my addr 172.21.15.46:0/2104237619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:22.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f2596ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 0x7f2598199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f25977fe640 1 -- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 msgr2=0x7f2598199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f25977fe640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 0x7f2598199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f25977fe640 1 -- 172.21.15.46:0/2104237619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259819a2a0 con 0x7f25981023f0 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.316+0000 7f2596ffd640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 0x7f2598199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.317+0000 7f25977fe640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 0x7f25981996e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f258800e940 tx=0x7f258800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.317+0000 7f2594ff9640 1 -- 172.21.15.46:0/2104237619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f258800cac0 con 0x7f25981023f0 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.317+0000 7f2594ff9640 1 -- 172.21.15.46:0/2104237619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25880054e0 con 0x7f25981023f0 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.317+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259819ece0 con 0x7f25981023f0 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.317+0000 7f2594ff9640 1 -- 172.21.15.46:0/2104237619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2588004df0 con 0x7f25981023f0 2024-09-10T04:06:22.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.317+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259819f190 con 0x7f25981023f0 2024-09-10T04:06:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.318+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25981027d0 con 0x7f25981023f0 2024-09-10T04:06:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.319+0000 7f2594ff9640 1 -- 172.21.15.46:0/2104237619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7f2588004f90 con 0x7f25981023f0 2024-09-10T04:06:22.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.320+0000 7f2594ff9640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f2580074630 0x7f2580076af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.320+0000 7f2594ff9640 1 -- 172.21.15.46:0/2104237619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7f25880964c0 con 0x7f25981023f0 2024-09-10T04:06:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.320+0000 7f2596ffd640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f2580074630 0x7f2580076af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:22.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.321+0000 7f2596ffd640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f2580074630 0x7f2580076af0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c0094f0 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:22.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.325+0000 7f2594ff9640 1 -- 172.21.15.46:0/2104237619 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f2588061bf0 con 0x7f25981023f0 2024-09-10T04:06:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.637+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2598062630 con 0x7f2580074630 2024-09-10T04:06:22.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.639+0000 7f2594ff9640 1 -- 172.21.15.46:0/2104237619 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f2598062630 con 0x7f2580074630 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f2580074630 msgr2=0x7f2580076af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f2580074630 0x7f2580076af0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f258c000c00 tx=0x7f258c0094f0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 msgr2=0x7f25981996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 0x7f25981996e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f258800e940 tx=0x7f258800ee00 comp rx=0 tx=0).stop 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 shutdown_connections 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f2580074630 0x7f2580076af0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2598104d20 0x7f2598199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 --2- 172.21.15.46:0/2104237619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25981023f0 0x7f25981996e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 >> 172.21.15.46:0/2104237619 conn(0x7f25980fc010 msgr2=0x7f2598101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 shutdown_connections 2024-09-10T04:06:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.641+0000 7f259e122640 1 -- 172.21.15.46:0/2104237619 wait complete. 2024-09-10T04:06:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 -- 172.21.15.46:0/2915282155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38101960 msgr2=0x7fab38101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 --2- 172.21.15.46:0/2915282155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38101960 0x7fab38101d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fab28009880 tx=0x7fab2802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 -- 172.21.15.46:0/2915282155 shutdown_connections 2024-09-10T04:06:22.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 --2- 172.21.15.46:0/2915282155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab38102b60 0x7fab38102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 --2- 172.21.15.46:0/2915282155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38101960 0x7fab38101d60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 -- 172.21.15.46:0/2915282155 >> 172.21.15.46:0/2915282155 conn(0x7fab380fd0f0 msgr2=0x7fab380ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:22.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 -- 172.21.15.46:0/2915282155 shutdown_connections 2024-09-10T04:06:22.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.742+0000 7fab3e11a640 1 -- 172.21.15.46:0/2915282155 wait complete. 2024-09-10T04:06:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.743+0000 7fab3e11a640 1 Processor -- start 2024-09-10T04:06:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.743+0000 7fab3e11a640 1 -- start start 2024-09-10T04:06:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.743+0000 7fab3e11a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab38101960 0x7fab38075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.743+0000 7fab3e11a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38102b60 0x7fab380745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.743+0000 7fab3e11a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab3810a780 con 0x7fab38102b60 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.743+0000 7fab3e11a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fab3810a600 con 0x7fab38101960 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab2ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38102b60 0x7fab380745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab38101960 0x7fab38075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab2ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38102b60 0x7fab380745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51022/0 (socket says 172.21.15.46:51022) 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab2ffff640 1 -- 172.21.15.46:0/1088763813 learned_addr learned my addr 172.21.15.46:0/1088763813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab377fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab38101960 0x7fab38075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:51140/0 (socket says 172.21.15.46:51140) 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab2ffff640 1 -- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab38101960 msgr2=0x7fab38075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab2ffff640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab38101960 0x7fab38075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab2ffff640 1 -- 172.21.15.46:0/1088763813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab38074b20 con 0x7fab38102b60 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab2ffff640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38102b60 0x7fab380745e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fab2000e970 tx=0x7fab2000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab357fa640 1 -- 172.21.15.46:0/1088763813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fab2000cac0 con 0x7fab38102b60 2024-09-10T04:06:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab357fa640 1 -- 172.21.15.46:0/1088763813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fab2000cc60 con 0x7fab38102b60 2024-09-10T04:06:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.744+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab38074e10 con 0x7fab38102b60 2024-09-10T04:06:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.745+0000 7fab357fa640 1 -- 172.21.15.46:0/1088763813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fab20018720 con 0x7fab38102b60 2024-09-10T04:06:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.745+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab380753a0 con 0x7fab38102b60 2024-09-10T04:06:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.746+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab38101d60 con 0x7fab38102b60 2024-09-10T04:06:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.747+0000 7fab357fa640 1 -- 172.21.15.46:0/1088763813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7fab20010430 con 0x7fab38102b60 2024-09-10T04:06:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.747+0000 7fab357fa640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fab24074650 0x7fab24076b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.747+0000 7fab357fa640 1 -- 172.21.15.46:0/1088763813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7fab200963b0 con 0x7fab38102b60 2024-09-10T04:06:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.748+0000 7fab377fe640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fab24074650 0x7fab24076b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.748+0000 7fab377fe640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fab24074650 0x7fab24076b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fab28009230 tx=0x7fab280058d0 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:22.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:22.754+0000 7fab357fa640 1 -- 172.21.15.46:0/1088763813 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fab20061ae0 con 0x7fab38102b60 2024-09-10T04:06:22.954 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:22 smithi046 ceph-mon[28328]: pgmap v25: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 76 op/s 2024-09-10T04:06:23.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.082+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fab38107130 con 0x7fab24074650 2024-09-10T04:06:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:22 smithi070 ceph-mon[35495]: pgmap v25: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 76 op/s 2024-09-10T04:06:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.092+0000 7fab357fa640 1 -- 172.21.15.46:0/1088763813 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fab38107130 con 0x7fab24074650 2024-09-10T04:06:23.094 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (7m) 37s ago 9m 26.4M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 starting - - - - 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 starting - - - - 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (9m) 37s ago 9m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (8m) 40s ago 8m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (7m) 37s ago 9m 83.0M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (3m) 37s ago 3m 265M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (3m) 37s ago 3m 244M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (3m) 40s ago 3m 22.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (3m) 40s ago 3m 22.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:9283,8765,8443 running (10m) 37s ago 10m 244M - 19.1.1-298-gb53b6d3e 54467abc55f8 a4fa65bf88c5 2024-09-10T04:06:23.095 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (63s) 40s ago 8m 494M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9bcab2dbf2ab 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (10m) 37s ago 10m 52.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (8m) 40s ago 8m 40.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (9m) 37s ago 9m 15.6M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 40s ago 8m 16.7M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (7m) 37s ago 7m 216M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (6m) 37s ago 6m 214M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (6m) 37s ago 6m 172M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (5m) 40s ago 5m 234M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (5m) 40s ago 5m 194M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (4m) 40s ago 4m 227M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:06:23.096 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 starting - - - - 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fab24074650 msgr2=0x7fab24076b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fab24074650 0x7fab24076b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fab28009230 tx=0x7fab280058d0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38102b60 msgr2=0x7fab380745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38102b60 0x7fab380745e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fab2000e970 tx=0x7fab2000ee30 comp rx=0 tx=0).stop 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 shutdown_connections 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fab24074650 0x7fab24076b10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab38102b60 0x7fab380745e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 --2- 172.21.15.46:0/1088763813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab38101960 0x7fab38075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 >> 172.21.15.46:0/1088763813 conn(0x7fab380fd0f0 msgr2=0x7fab380ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 shutdown_connections 2024-09-10T04:06:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.096+0000 7fab3e11a640 1 -- 172.21.15.46:0/1088763813 wait complete. 2024-09-10T04:06:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.220+0000 7fbb88e3c640 1 -- 172.21.15.46:0/242060867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 msgr2=0x7fbb84101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.220+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/242060867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 0x7fbb84101ee0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009880 tx=0x7fbb7802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.221+0000 7fbb88e3c640 1 -- 172.21.15.46:0/242060867 shutdown_connections 2024-09-10T04:06:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.221+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/242060867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 0x7fbb84101ee0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.221+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/242060867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb84100860 0x7fbb84100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.221+0000 7fbb88e3c640 1 -- 172.21.15.46:0/242060867 >> 172.21.15.46:0/242060867 conn(0x7fbb840fc010 msgr2=0x7fbb840fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.221+0000 7fbb88e3c640 1 -- 172.21.15.46:0/242060867 shutdown_connections 2024-09-10T04:06:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.221+0000 7fbb88e3c640 1 -- 172.21.15.46:0/242060867 wait complete. 2024-09-10T04:06:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.223+0000 7fbb88e3c640 1 Processor -- start 2024-09-10T04:06:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.223+0000 7fbb88e3c640 1 -- start start 2024-09-10T04:06:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.223+0000 7fbb88e3c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb84100860 0x7fbb841974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.223+0000 7fbb88e3c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 0x7fbb84197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.223+0000 7fbb88e3c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8410a780 con 0x7fbb84101a60 2024-09-10T04:06:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb88e3c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8410a600 con 0x7fbb84100860 2024-09-10T04:06:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb81d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 0x7fbb84197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb81d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 0x7fbb84197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51050/0 (socket says 172.21.15.46:51050) 2024-09-10T04:06:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb81d9b640 1 -- 172.21.15.46:0/430744631 learned_addr learned my addr 172.21.15.46:0/430744631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb8259c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb84100860 0x7fbb841974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb8259c640 1 -- 172.21.15.46:0/430744631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 msgr2=0x7fbb84197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb8259c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 0x7fbb84197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.224+0000 7fbb8259c640 1 -- 172.21.15.46:0/430744631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb84198080 con 0x7fbb84100860 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.225+0000 7fbb8259c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb84100860 0x7fbb841974c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400e940 tx=0x7fbb7400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.225+0000 7fbb737fe640 1 -- 172.21.15.46:0/430744631 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb7400cac0 con 0x7fbb84100860 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.225+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8419cac0 con 0x7fbb84100860 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.225+0000 7fbb737fe640 1 -- 172.21.15.46:0/430744631 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb7400cc60 con 0x7fbb84100860 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.225+0000 7fbb737fe640 1 -- 172.21.15.46:0/430744631 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb740186c0 con 0x7fbb84100860 2024-09-10T04:06:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.225+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8419d050 con 0x7fbb84100860 2024-09-10T04:06:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.226+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb84100c60 con 0x7fbb84100860 2024-09-10T04:06:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.227+0000 7fbb737fe640 1 -- 172.21.15.46:0/430744631 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7fbb740188f0 con 0x7fbb84100860 2024-09-10T04:06:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.227+0000 7fbb737fe640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fbb6c0746a0 0x7fbb6c076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.227+0000 7fbb737fe640 1 -- 172.21.15.46:0/430744631 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7fbb74096220 con 0x7fbb84100860 2024-09-10T04:06:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.227+0000 7fbb81d9b640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fbb6c0746a0 0x7fbb6c076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.228+0000 7fbb81d9b640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fbb6c0746a0 0x7fbb6c076b60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb84198970 tx=0x7fbb78005a20 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.233+0000 7fbb737fe640 1 -- 172.21.15.46:0/430744631 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fbb74055db0 con 0x7fbb84100860 2024-09-10T04:06:23.588 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:06:23.589 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 13, 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.586+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fbb84062df0 con 0x7fbb84100860 2024-09-10T04:06:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.586+0000 7fbb737fe640 1 -- 172.21.15.46:0/430744631 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+841 (secure 0 0 0) 0x7fbb7400ce10 con 0x7fbb84100860 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fbb6c0746a0 msgr2=0x7fbb6c076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fbb6c0746a0 0x7fbb6c076b60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbb84198970 tx=0x7fbb78005a20 comp rx=0 tx=0).stop 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb84100860 msgr2=0x7fbb841974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb84100860 0x7fbb841974c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400e940 tx=0x7fbb7400ee00 comp rx=0 tx=0).stop 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 shutdown_connections 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7fbb6c0746a0 0x7fbb6c076b60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb84101a60 0x7fbb84197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 --2- 172.21.15.46:0/430744631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb84100860 0x7fbb841974c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.589+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 >> 172.21.15.46:0/430744631 conn(0x7fbb840fc010 msgr2=0x7fbb84104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.590+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 shutdown_connections 2024-09-10T04:06:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.590+0000 7fbb88e3c640 1 -- 172.21.15.46:0/430744631 wait complete. 2024-09-10T04:06:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.687+0000 7f43941c4640 1 -- 172.21.15.46:0/1303677882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c100860 msgr2=0x7f438c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.687+0000 7f43941c4640 1 --2- 172.21.15.46:0/1303677882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c100860 0x7f438c100c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43880097e0 tx=0x7f438802a3f0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.687+0000 7f43941c4640 1 -- 172.21.15.46:0/1303677882 shutdown_connections 2024-09-10T04:06:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.687+0000 7f43941c4640 1 --2- 172.21.15.46:0/1303677882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f438c101a60 0x7f438c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.687+0000 7f43941c4640 1 --2- 172.21.15.46:0/1303677882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c100860 0x7f438c100c60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.687+0000 7f43941c4640 1 -- 172.21.15.46:0/1303677882 >> 172.21.15.46:0/1303677882 conn(0x7f438c0fc010 msgr2=0x7f438c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.687+0000 7f43941c4640 1 -- 172.21.15.46:0/1303677882 shutdown_connections 2024-09-10T04:06:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.688+0000 7f43941c4640 1 -- 172.21.15.46:0/1303677882 wait complete. 2024-09-10T04:06:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.688+0000 7f43941c4640 1 Processor -- start 2024-09-10T04:06:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.688+0000 7f43941c4640 1 -- start start 2024-09-10T04:06:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.688+0000 7f43941c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f438c100860 0x7f438c104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:23.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.688+0000 7f43941c4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c101a60 0x7f438c102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.688+0000 7f43941c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f438c109680 con 0x7f438c100860 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.688+0000 7f43941c4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f438c109500 con 0x7f438c101a60 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f439175f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c101a60 0x7f438c102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f4391f60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f438c100860 0x7f438c104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f4391f60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f438c100860 0x7f438c104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51072/0 (socket says 172.21.15.46:51072) 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f4391f60640 1 -- 172.21.15.46:0/2768368204 learned_addr learned my addr 172.21.15.46:0/2768368204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f439175f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c101a60 0x7f438c102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:51178/0 (socket says 172.21.15.46:51178) 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f439175f640 1 -- 172.21.15.46:0/2768368204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f438c100860 msgr2=0x7f438c104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f439175f640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f438c100860 0x7f438c104780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f439175f640 1 -- 172.21.15.46:0/2768368204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438c103380 con 0x7f438c101a60 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.689+0000 7f439175f640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c101a60 0x7f438c102e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f437c005fd0 tx=0x7f437c00edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.690+0000 7f4382ffd640 1 -- 172.21.15.46:0/2768368204 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f437c00cac0 con 0x7f438c101a60 2024-09-10T04:06:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.690+0000 7f4382ffd640 1 -- 172.21.15.46:0/2768368204 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f437c00cc60 con 0x7f438c101a60 2024-09-10T04:06:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.690+0000 7f4382ffd640 1 -- 172.21.15.46:0/2768368204 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f437c003e00 con 0x7f438c101a60 2024-09-10T04:06:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.690+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438c103670 con 0x7f438c101a60 2024-09-10T04:06:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.690+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438c103c00 con 0x7f438c101a60 2024-09-10T04:06:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.691+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f438c100c60 con 0x7f438c101a60 2024-09-10T04:06:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.691+0000 7f4382ffd640 1 -- 172.21.15.46:0/2768368204 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 24) ==== 97783+0+0 (secure 0 0 0) 0x7f437c004020 con 0x7f438c101a60 2024-09-10T04:06:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.692+0000 7f4382ffd640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f43780746a0 0x7f4378076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.692+0000 7f4382ffd640 1 -- 172.21.15.46:0/2768368204 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5726+0+0 (secure 0 0 0) 0x7f437c096260 con 0x7f438c101a60 2024-09-10T04:06:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.692+0000 7f4391f60640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f43780746a0 0x7f4378076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.693+0000 7f4391f60640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f43780746a0 0x7f4378076b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43880097b0 tx=0x7f4388030890 comp rx=0 tx=0).ready entity=mgr.24439 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.697+0000 7f4382ffd640 1 -- 172.21.15.46:0/2768368204 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f437c009510 con 0x7f438c101a60 2024-09-10T04:06:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.990+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 --> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f438c19f5b0 con 0x7f43780746a0 2024-09-10T04:06:23.992 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:23 smithi046 ceph-mon[28328]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:23.992 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:23 smithi046 ceph-mon[28328]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:23.992 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:23 smithi046 ceph-mon[28328]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:23.992 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:23 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/430744631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:06:23.992 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:23 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.991+0000 7f4382ffd640 1 -- 172.21.15.46:0/2768368204 <== mgr.24439 v2:172.21.15.70:6828/3363961098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+341 (secure 0 0 0) 0x7f438c19f5b0 con 0x7f43780746a0 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [], 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "1/2 daemons upgraded", 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mgr daemons", 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:06:23.993 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f43780746a0 msgr2=0x7f4378076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f43780746a0 0x7f4378076b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43880097b0 tx=0x7f4388030890 comp rx=0 tx=0).stop 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c101a60 msgr2=0x7f438c102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c101a60 0x7f438c102e40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f437c005fd0 tx=0x7f437c00edd0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 shutdown_connections 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:6828/3363961098,v1:172.21.15.70:6829/3363961098] conn(0x7f43780746a0 0x7f4378076b60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c101a60 0x7f438c102e40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 --2- 172.21.15.46:0/2768368204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f438c100860 0x7f438c104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 >> 172.21.15.46:0/2768368204 conn(0x7f438c0fc010 msgr2=0x7f438c069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 shutdown_connections 2024-09-10T04:06:23.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:23.994+0000 7f43941c4640 1 -- 172.21.15.46:0/2768368204 wait complete. 2024-09-10T04:06:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:23 smithi070 ceph-mon[35495]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:23 smithi070 ceph-mon[35495]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:23 smithi070 ceph-mon[35495]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:23 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/430744631' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:06:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:23 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:24.939 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:24 smithi046 ceph-mon[28328]: pgmap v26: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 76 op/s 2024-09-10T04:06:24.939 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:24 smithi046 ceph-mon[28328]: from='client.24505 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:25.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:24 smithi070 ceph-mon[35495]: pgmap v26: 65 pgs: 65 active+clean; 306 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 76 op/s 2024-09-10T04:06:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:24 smithi070 ceph-mon[35495]: from='client.24505 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:27.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:26 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:27.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:26 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:27.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:26 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:27.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:26 smithi046 ceph-mon[28328]: pgmap v27: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-09-10T04:06:27.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:26 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:27.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:26 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:26 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:26 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:26 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:26 smithi070 ceph-mon[35495]: pgmap v27: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 124 op/s 2024-09-10T04:06:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:26 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:26 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:28 smithi070 ceph-mon[35495]: pgmap v28: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-09-10T04:06:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:28 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:28 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:28 smithi046 ceph-mon[28328]: pgmap v28: 65 pgs: 65 active+clean; 302 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-09-10T04:06:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:28 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:28 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:30 smithi070 ceph-mon[35495]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 120 op/s 2024-09-10T04:06:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:30 smithi046 ceph-mon[28328]: pgmap v29: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 120 op/s 2024-09-10T04:06:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:32 smithi070 ceph-mon[35495]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-10T04:06:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:32 smithi070 ceph-mon[35495]: Standby manager daemon smithi046.zvckjg restarted 2024-09-10T04:06:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:32 smithi070 ceph-mon[35495]: Standby manager daemon smithi046.zvckjg started 2024-09-10T04:06:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:32 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/crt"}]: dispatch 2024-09-10T04:06:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:32 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:06:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:32 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/key"}]: dispatch 2024-09-10T04:06:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:32 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:06:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:32 smithi046 ceph-mon[28328]: pgmap v30: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-10T04:06:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:32 smithi046 ceph-mon[28328]: Standby manager daemon smithi046.zvckjg restarted 2024-09-10T04:06:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:32 smithi046 ceph-mon[28328]: Standby manager daemon smithi046.zvckjg started 2024-09-10T04:06:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:32 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/crt"}]: dispatch 2024-09-10T04:06:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:32 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:06:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:32 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi046.zvckjg/key"}]: dispatch 2024-09-10T04:06:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:32 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:06:34.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:33 smithi070 ceph-mon[35495]: mgrmap e25: smithi070.tezbzn(active, since 55s), standbys: smithi046.zvckjg 2024-09-10T04:06:34.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:33 smithi046 ceph-mon[28328]: mgrmap e25: smithi070.tezbzn(active, since 55s), standbys: smithi046.zvckjg 2024-09-10T04:06:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:34 smithi070 ceph-mon[35495]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-10T04:06:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:34 smithi046 ceph-mon[28328]: pgmap v31: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-10T04:06:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:36 smithi070 ceph-mon[35495]: pgmap v32: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 364 op/s 2024-09-10T04:06:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:36 smithi046 ceph-mon[28328]: pgmap v32: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 364 op/s 2024-09-10T04:06:39.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:38 smithi070 ceph-mon[35495]: pgmap v33: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 317 op/s 2024-09-10T04:06:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:38 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:38 smithi046 ceph-mon[28328]: pgmap v33: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 317 op/s 2024-09-10T04:06:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:38 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:40 smithi070 ceph-mon[35495]: pgmap v34: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 403 op/s 2024-09-10T04:06:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:40 smithi046 ceph-mon[28328]: pgmap v34: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 403 op/s 2024-09-10T04:06:42.886 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:42 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:42.886 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:42 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:42.887 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:42 smithi046 ceph-mon[28328]: pgmap v35: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 365 op/s 2024-09-10T04:06:42.887 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:42 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:42.887 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:42 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:43.084 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:42 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:42 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:42 smithi070 ceph-mon[35495]: pgmap v35: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 365 op/s 2024-09-10T04:06:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:42 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:42 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:44.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:44 smithi046 ceph-mon[28328]: pgmap v36: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 365 op/s 2024-09-10T04:06:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:44 smithi070 ceph-mon[35495]: pgmap v36: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 365 op/s 2024-09-10T04:06:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:46 smithi070 ceph-mon[35495]: pgmap v37: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 526 op/s 2024-09-10T04:06:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:46 smithi046 ceph-mon[28328]: pgmap v37: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 526 op/s 2024-09-10T04:06:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:48 smithi070 ceph-mon[35495]: pgmap v38: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-09-10T04:06:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:48 smithi046 ceph-mon[28328]: pgmap v38: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 247 op/s 2024-09-10T04:06:51.042 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:50 smithi070 ceph-mon[35495]: pgmap v39: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-09-10T04:06:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:50 smithi046 ceph-mon[28328]: pgmap v39: 65 pgs: 65 active+clean; 287 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 330 op/s 2024-09-10T04:06:53.161 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:53.161 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:53.161 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr fail", "who": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr fail", "who": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: Activating manager daemon smithi046.zvckjg 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd='[{"prefix": "mgr fail", "who": "smithi070.tezbzn"}]': finished 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: mgrmap e26: smithi046.zvckjg(active, starting, since 0.00902805s) 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:06:53.162 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:06:53.163 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:06:53.164 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:06:53.164 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T04:06:53.164 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T04:06:53.164 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T04:06:53.164 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:52 smithi046 ceph-mon[28328]: Manager daemon smithi046.zvckjg is now available 2024-09-10T04:06:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:06:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 172.21.15.70:0/4286182091' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr fail", "who": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "mgr fail", "who": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: Activating manager daemon smithi046.zvckjg 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.24439 ' entity='mgr.smithi070.tezbzn' cmd='[{"prefix": "mgr fail", "who": "smithi070.tezbzn"}]': finished 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: mgrmap e26: smithi046.zvckjg(active, starting, since 0.00902805s) 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi046"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata", "id": "smithi070"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"}]: dispatch 2024-09-10T04:06:53.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi046.zvckjg", "id": "smithi046.zvckjg"}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T04:06:53.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:52 smithi070 ceph-mon[35495]: Manager daemon smithi046.zvckjg is now available 2024-09-10T04:06:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 -- 172.21.15.46:0/2204280308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 msgr2=0x7f3e00073100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 --2- 172.21.15.46:0/2204280308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 0x7f3e00073100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009880 tx=0x7f3df402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 -- 172.21.15.46:0/2204280308 shutdown_connections 2024-09-10T04:06:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 --2- 172.21.15.46:0/2204280308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 0x7f3e00073100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 --2- 172.21.15.46:0/2204280308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e00109450 0x7f3e00109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 -- 172.21.15.46:0/2204280308 >> 172.21.15.46:0/2204280308 conn(0x7f3e0006ed60 msgr2=0x7f3e000711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:54.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 -- 172.21.15.46:0/2204280308 shutdown_connections 2024-09-10T04:06:54.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.124+0000 7f3e05633640 1 -- 172.21.15.46:0/2204280308 wait complete. 2024-09-10T04:06:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.126+0000 7f3e05633640 1 Processor -- start 2024-09-10T04:06:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.126+0000 7f3e05633640 1 -- start start 2024-09-10T04:06:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.126+0000 7f3e05633640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e00109450 0x7f3e001a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3e05633640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 0x7f3e001a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3e05633640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e00115190 con 0x7f3e00109e20 2024-09-10T04:06:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3e05633640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e00115010 con 0x7f3e00109450 2024-09-10T04:06:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3dfeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e00109450 0x7f3e001a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3dfeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e00109450 0x7f3e001a1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:46616/0 (socket says 172.21.15.46:46616) 2024-09-10T04:06:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3dfeffd640 1 -- 172.21.15.46:0/2148401156 learned_addr learned my addr 172.21.15.46:0/2148401156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:54.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3dfe7fc640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 0x7f3e001a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3dfe7fc640 1 -- 172.21.15.46:0/2148401156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e00109450 msgr2=0x7f3e001a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3dfe7fc640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e00109450 0x7f3e001a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.127+0000 7f3dfe7fc640 1 -- 172.21.15.46:0/2148401156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e001a2a80 con 0x7f3e00109e20 2024-09-10T04:06:54.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.128+0000 7f3dfe7fc640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 0x7f3e001a2400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009230 tx=0x7f3df40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.128+0000 7f3de7fff640 1 -- 172.21.15.46:0/2148401156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3df403f070 con 0x7f3e00109e20 2024-09-10T04:06:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.128+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e001a7460 con 0x7f3e00109e20 2024-09-10T04:06:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.128+0000 7f3de7fff640 1 -- 172.21.15.46:0/2148401156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3df40049d0 con 0x7f3e00109e20 2024-09-10T04:06:54.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.128+0000 7f3de7fff640 1 -- 172.21.15.46:0/2148401156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3df4035030 con 0x7f3e00109e20 2024-09-10T04:06:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.128+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e001a79f0 con 0x7f3e00109e20 2024-09-10T04:06:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.130+0000 7f3de7fff640 1 -- 172.21.15.46:0/2148401156 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 51701+0+0 (secure 0 0 0) 0x7f3df4042070 con 0x7f3e00109e20 2024-09-10T04:06:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.130+0000 7f3de7fff640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3de803f690 0x7f3de8041b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.130+0000 7f3de7fff640 1 -- 172.21.15.46:0/2148401156 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f3df4075c70 con 0x7f3e00109e20 2024-09-10T04:06:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.130+0000 7f3dfeffd640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3de803f690 0x7f3de8041b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.131+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e00109850 con 0x7f3e00109e20 2024-09-10T04:06:54.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.131+0000 7f3dfeffd640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3de803f690 0x7f3de8041b50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009c90 tx=0x7f3df0005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:54.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.141+0000 7f3de7fff640 1 -- 172.21.15.46:0/2148401156 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f3df402f300 con 0x7f3e00109e20 2024-09-10T04:06:54.143 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:54.143 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:54.143 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:06:54.143 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:06:54.143 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: mgrmap e27: smithi046.zvckjg(active, since 1.01257s) 2024-09-10T04:06:54.143 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:06:54.143 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: pgmap v3: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:06:54.144 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:06:54.144 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T04:06:54.144 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:53 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T04:06:54.266 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:06:54.266 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: mgrmap e27: smithi046.zvckjg(active, since 1.01257s) 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/mirror_snapshot_schedule"}]: dispatch 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: pgmap v3: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T04:06:54.267 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:53 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi046.zvckjg/trash_purge_schedule"}]: dispatch 2024-09-10T04:06:54.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.464+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3e0010bcf0 con 0x7f3de803f690 2024-09-10T04:06:54.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.467+0000 7f3de7fff640 1 -- 172.21.15.46:0/2148401156 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+329 (secure 0 0 0) 0x7f3e0010bcf0 con 0x7f3de803f690 2024-09-10T04:06:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.471+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3de803f690 msgr2=0x7f3de8041b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.471+0000 7f3e05633640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3de803f690 0x7f3de8041b50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3df0009c90 tx=0x7f3df0005f90 comp rx=0 tx=0).stop 2024-09-10T04:06:54.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.471+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 msgr2=0x7f3e001a2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.471+0000 7f3e05633640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 0x7f3e001a2400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009230 tx=0x7f3df40045e0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.472+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 shutdown_connections 2024-09-10T04:06:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.472+0000 7f3e05633640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3de803f690 0x7f3de8041b50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.472+0000 7f3e05633640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e00109e20 0x7f3e001a2400 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.472+0000 7f3e05633640 1 --2- 172.21.15.46:0/2148401156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e00109450 0x7f3e001a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.472+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 >> 172.21.15.46:0/2148401156 conn(0x7f3e0006ed60 msgr2=0x7f3e000711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.472+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 shutdown_connections 2024-09-10T04:06:54.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.472+0000 7f3e05633640 1 -- 172.21.15.46:0/2148401156 wait complete. 2024-09-10T04:06:54.488 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.586+0000 7f0f68bc3640 1 -- 172.21.15.46:0/1596327578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64100860 msgr2=0x7f0f64100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.586+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/1596327578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64100860 0x7f0f64100c60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009880 tx=0x7f0f5402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.586+0000 7f0f68bc3640 1 -- 172.21.15.46:0/1596327578 shutdown_connections 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.586+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/1596327578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f64101a60 0x7f0f64101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.586+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/1596327578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64100860 0x7f0f64100c60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.586+0000 7f0f68bc3640 1 -- 172.21.15.46:0/1596327578 >> 172.21.15.46:0/1596327578 conn(0x7f0f640fc010 msgr2=0x7f0f640fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.587+0000 7f0f68bc3640 1 -- 172.21.15.46:0/1596327578 shutdown_connections 2024-09-10T04:06:54.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.587+0000 7f0f68bc3640 1 -- 172.21.15.46:0/1596327578 wait complete. 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.587+0000 7f0f68bc3640 1 Processor -- start 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.587+0000 7f0f68bc3640 1 -- start start 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f68bc3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f64100860 0x7f0f64104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f68bc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64101a60 0x7f0f64102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f68bc3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64109680 con 0x7f0f64101a60 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f68bc3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64109500 con 0x7f0f64100860 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f59d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64101a60 0x7f0f64102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f59d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64101a60 0x7f0f64102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35012/0 (socket says 172.21.15.46:35012) 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f59d9b640 1 -- 172.21.15.46:0/842450099 learned_addr learned my addr 172.21.15.46:0/842450099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:54.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.588+0000 7f0f6259c640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f64100860 0x7f0f64104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f59d9b640 1 -- 172.21.15.46:0/842450099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f64100860 msgr2=0x7f0f64104780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f59d9b640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f64100860 0x7f0f64104780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f59d9b640 1 -- 172.21.15.46:0/842450099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f64103380 con 0x7f0f64101a60 2024-09-10T04:06:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f6259c640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f64100860 0x7f0f64104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:06:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f59d9b640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64101a60 0x7f0f64102e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d8d0 tx=0x7f0f4c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f5bfff640 1 -- 172.21.15.46:0/842450099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f4c017030 con 0x7f0f64101a60 2024-09-10T04:06:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f5bfff640 1 -- 172.21.15.46:0/842450099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f4c004020 con 0x7f0f64101a60 2024-09-10T04:06:54.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f64103670 con 0x7f0f64101a60 2024-09-10T04:06:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f5bfff640 1 -- 172.21.15.46:0/842450099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f4c0047a0 con 0x7f0f64101a60 2024-09-10T04:06:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.589+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f64103c00 con 0x7f0f64101a60 2024-09-10T04:06:54.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.591+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f64100c60 con 0x7f0f64101a60 2024-09-10T04:06:54.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.591+0000 7f0f5bfff640 1 -- 172.21.15.46:0/842450099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 51701+0+0 (secure 0 0 0) 0x7f0f4c0041d0 con 0x7f0f64101a60 2024-09-10T04:06:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.592+0000 7f0f5bfff640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f5003f640 0x7f0f50041b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.592+0000 7f0f5bfff640 1 -- 172.21.15.46:0/842450099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f0f4c018070 con 0x7f0f64101a60 2024-09-10T04:06:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.592+0000 7f0f6259c640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f5003f640 0x7f0f50041b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.592+0000 7f0f6259c640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f5003f640 0x7f0f50041b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009230 tx=0x7f0f540058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:54.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.597+0000 7f0f5bfff640 1 -- 172.21.15.46:0/842450099 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f0f4c0171d0 con 0x7f0f64101a60 2024-09-10T04:06:54.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.875+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0f6406a400 con 0x7f0f5003f640 2024-09-10T04:06:54.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.876+0000 7f0f5bfff640 1 -- 172.21.15.46:0/842450099 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+329 (secure 0 0 0) 0x7f0f6406a400 con 0x7f0f5003f640 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f5003f640 msgr2=0x7f0f50041b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f5003f640 0x7f0f50041b00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009230 tx=0x7f0f540058d0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64101a60 msgr2=0x7f0f64102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64101a60 0x7f0f64102e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d8d0 tx=0x7f0f4c00dd90 comp rx=0 tx=0).stop 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 shutdown_connections 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f5003f640 0x7f0f50041b00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f64101a60 0x7f0f64102e40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 --2- 172.21.15.46:0/842450099 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f64100860 0x7f0f64104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 >> 172.21.15.46:0/842450099 conn(0x7f0f640fc010 msgr2=0x7f0f64069cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 shutdown_connections 2024-09-10T04:06:54.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.880+0000 7f0f68bc3640 1 -- 172.21.15.46:0/842450099 wait complete. 2024-09-10T04:06:54.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.982+0000 7fa0bcf92640 1 -- 172.21.15.46:0/2723796260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101a60 msgr2=0x7fa0b8101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.982+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/2723796260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101a60 0x7fa0b8101ee0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009880 tx=0x7fa0ac02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.983+0000 7fa0bcf92640 1 -- 172.21.15.46:0/2723796260 shutdown_connections 2024-09-10T04:06:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.983+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/2723796260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101a60 0x7fa0b8101ee0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.983+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/2723796260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8100860 0x7fa0b8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.983+0000 7fa0bcf92640 1 -- 172.21.15.46:0/2723796260 >> 172.21.15.46:0/2723796260 conn(0x7fa0b80fc010 msgr2=0x7fa0b80fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.983+0000 7fa0bcf92640 1 -- 172.21.15.46:0/2723796260 shutdown_connections 2024-09-10T04:06:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.983+0000 7fa0bcf92640 1 -- 172.21.15.46:0/2723796260 wait complete. 2024-09-10T04:06:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.984+0000 7fa0bcf92640 1 Processor -- start 2024-09-10T04:06:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.984+0000 7fa0bcf92640 1 -- start start 2024-09-10T04:06:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.984+0000 7fa0bcf92640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8100860 0x7fa0b81974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.985+0000 7fa0bcf92640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101a60 0x7fa0b8197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.985+0000 7fa0bcf92640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8109680 con 0x7fa0b8100860 2024-09-10T04:06:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.985+0000 7fa0bcf92640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8109500 con 0x7fa0b8101a60 2024-09-10T04:06:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.985+0000 7fa0b5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101a60 0x7fa0b8197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.985+0000 7fa0b5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101a60 0x7fa0b8197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:46650/0 (socket says 172.21.15.46:46650) 2024-09-10T04:06:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.985+0000 7fa0b5d9b640 1 -- 172.21.15.46:0/406997135 learned_addr learned my addr 172.21.15.46:0/406997135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.985+0000 7fa0b659c640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8100860 0x7fa0b81974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0b659c640 1 -- 172.21.15.46:0/406997135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101a60 msgr2=0x7fa0b8197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:54.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0b659c640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101a60 0x7fa0b8197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0b659c640 1 -- 172.21.15.46:0/406997135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b8198090 con 0x7fa0b8100860 2024-09-10T04:06:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0b5d9b640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101a60 0x7fa0b8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:06:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0b659c640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8100860 0x7fa0b81974d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e940 tx=0x7fa0a800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0a800cac0 con 0x7fa0b8100860 2024-09-10T04:06:54.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa0a800cc60 con 0x7fa0b8100860 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b819cad0 con 0x7fa0b8100860 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0a800cac0 con 0x7fa0b8100860 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.986+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b819d060 con 0x7fa0b8100860 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.988+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0980052c0 con 0x7fa0b8100860 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.988+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 27) ==== 51701+0+0 (secure 0 0 0) 0x7fa0a801f020 con 0x7fa0b8100860 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.988+0000 7fa0a77fe640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a003f7e0 0x7fa0a0041ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.988+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fa0a8010a80 con 0x7fa0b8100860 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.989+0000 7fa0b5d9b640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a003f7e0 0x7fa0a0041ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.989+0000 7fa0b5d9b640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a003f7e0 0x7fa0a0041ca0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0096c0 tx=0x7fa0ac005a20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:54.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:54.995+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fa0a800ce10 con 0x7fa0b8100860 2024-09-10T04:06:55.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.015+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 28) ==== 51995+0+0 (secure 0 0 0) 0x7fa0a801f020 con 0x7fa0b8100860 2024-09-10T04:06:55.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.259+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa098000bf0 con 0x7fa0a003f7e0 2024-09-10T04:06:55.261 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:55 smithi046 ceph-mon[28328]: pgmap v4: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:06:55.261 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:55 smithi046 ceph-mon[28328]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:55.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.272+0000 7fa0a77fe640 1 -- 172.21.15.46:0/406997135 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fa098000bf0 con 0x7fa0a003f7e0 2024-09-10T04:06:55.274 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:06:55.274 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (8m) 13s ago 9m 26.4M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:06:55.274 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (63s) 13s ago 10m 8392k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (50s) 26s ago 9m 8232k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (10m) 13s ago 10m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (9m) 26s ago 9m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (8m) 13s ago 9m 80.7M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (4m) 13s ago 4m 248M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (4m) 13s ago 4m 238M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (4m) 26s ago 4m 25.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (4m) 26s ago 4m 25.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (30s) 13s ago 11m 90.3M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (96s) 26s ago 9m 510M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9bcab2dbf2ab 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (11m) 13s ago 11m 54.1M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:06:55.275 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (8m) 26s ago 8m 42.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (9m) 13s ago 9m 15.4M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (9m) 26s ago 9m 16.5M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (7m) 13s ago 7m 186M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (7m) 13s ago 7m 202M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (6m) 13s ago 6m 158M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (6m) 26s ago 6m 210M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (5m) 26s ago 5m 187M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (5m) 26s ago 5m 173M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:06:55.276 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (57s) 13s ago 9m 44.7M - 2.43.0 a07b618ecd1d 43b5e97f60e3 2024-09-10T04:06:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a003f7e0 msgr2=0x7fa0a0041ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a003f7e0 0x7fa0a0041ca0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac0096c0 tx=0x7fa0ac005a20 comp rx=0 tx=0).stop 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8100860 msgr2=0x7fa0b81974d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8100860 0x7fa0b81974d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800e940 tx=0x7fa0a800ee00 comp rx=0 tx=0).stop 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 shutdown_connections 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a003f7e0 0x7fa0a0041ca0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101a60 0x7fa0b8197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 --2- 172.21.15.46:0/406997135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8100860 0x7fa0b81974d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 >> 172.21.15.46:0/406997135 conn(0x7fa0b80fc010 msgr2=0x7fa0b80fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 shutdown_connections 2024-09-10T04:06:55.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.276+0000 7fa0bcf92640 1 -- 172.21.15.46:0/406997135 wait complete. 2024-09-10T04:06:55.329 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:55 smithi070 ceph-mon[35495]: pgmap v4: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:06:55.329 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:55 smithi070 ceph-mon[35495]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 -- 172.21.15.46:0/4086216673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 msgr2=0x7f90fc107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 --2- 172.21.15.46:0/4086216673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc107110 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f90f00098b0 tx=0x7f90f002a4d0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 -- 172.21.15.46:0/4086216673 shutdown_connections 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 --2- 172.21.15.46:0/4086216673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc107110 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 --2- 172.21.15.46:0/4086216673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc1023f0 0x7f90fc1047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 -- 172.21.15.46:0/4086216673 >> 172.21.15.46:0/4086216673 conn(0x7f90fc0fc010 msgr2=0x7f90fc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 -- 172.21.15.46:0/4086216673 shutdown_connections 2024-09-10T04:06:55.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.375+0000 7f910263e640 1 -- 172.21.15.46:0/4086216673 wait complete. 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f910263e640 1 Processor -- start 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f910263e640 1 -- start start 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f910263e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc1023f0 0x7f90fc197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f910263e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f910263e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc10a7b0 con 0x7f90fc104d20 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f910263e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc10a630 con 0x7f90fc1023f0 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f90fb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35070/0 (socket says 172.21.15.46:35070) 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f90fb7fe640 1 -- 172.21.15.46:0/3733315386 learned_addr learned my addr 172.21.15.46:0/3733315386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:55.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f90fbfff640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc1023f0 0x7f90fc197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f90fb7fe640 1 -- 172.21.15.46:0/3733315386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc1023f0 msgr2=0x7f90fc197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f90fb7fe640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc1023f0 0x7f90fc197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.376+0000 7f90fb7fe640 1 -- 172.21.15.46:0/3733315386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc1980f0 con 0x7f90fc104d20 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.377+0000 7f90fb7fe640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc197a70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009880 tx=0x7f90f00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.377+0000 7f90f97fa640 1 -- 172.21.15.46:0/3733315386 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f90f003f070 con 0x7f90fc104d20 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.377+0000 7f90f97fa640 1 -- 172.21.15.46:0/3733315386 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90f0004a50 con 0x7f90fc104d20 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.377+0000 7f90f97fa640 1 -- 172.21.15.46:0/3733315386 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f90f0035030 con 0x7f90fc104d20 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.377+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc19cad0 con 0x7f90fc104d20 2024-09-10T04:06:55.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.377+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc19d060 con 0x7f90fc104d20 2024-09-10T04:06:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.378+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc1027d0 con 0x7f90fc104d20 2024-09-10T04:06:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.379+0000 7f90f97fa640 1 -- 172.21.15.46:0/3733315386 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 28) ==== 51995+0+0 (secure 0 0 0) 0x7f90f0042070 con 0x7f90fc104d20 2024-09-10T04:06:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.379+0000 7f90f97fa640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f90e403f970 0x7f90e4041e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.379+0000 7f90f97fa640 1 -- 172.21.15.46:0/3733315386 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f90f0076220 con 0x7f90fc104d20 2024-09-10T04:06:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.379+0000 7f90fbfff640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f90e403f970 0x7f90e4041e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:55.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.380+0000 7f90fbfff640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f90e403f970 0x7f90e4041e30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005fd0 tx=0x7f90ec00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:55.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.384+0000 7f90f97fa640 1 -- 172.21.15.46:0/3733315386 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f90f0004c00 con 0x7f90fc104d20 2024-09-10T04:06:55.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.778+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f90fc19d4c0 con 0x7f90fc104d20 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.779+0000 7f90f97fa640 1 -- 172.21.15.46:0/3733315386 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7f90f00372d0 con 0x7f90fc104d20 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:06:55.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 12, 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:06:55.782 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:06:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.782+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f90e403f970 msgr2=0x7f90e4041e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.782+0000 7f910263e640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f90e403f970 0x7f90e4041e30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f90ec005fd0 tx=0x7f90ec00b390 comp rx=0 tx=0).stop 2024-09-10T04:06:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.782+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 msgr2=0x7f90fc197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.782+0000 7f910263e640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc197a70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009880 tx=0x7f90f00045e0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.783+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 shutdown_connections 2024-09-10T04:06:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.783+0000 7f910263e640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f90e403f970 0x7f90e4041e30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.783+0000 7f910263e640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90fc104d20 0x7f90fc197a70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.783+0000 7f910263e640 1 --2- 172.21.15.46:0/3733315386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90fc1023f0 0x7f90fc197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.783+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 >> 172.21.15.46:0/3733315386 conn(0x7f90fc0fc010 msgr2=0x7f90fc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.783+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 shutdown_connections 2024-09-10T04:06:55.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.783+0000 7f910263e640 1 -- 172.21.15.46:0/3733315386 wait complete. 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.890+0000 7fa726d44640 1 -- 172.21.15.46:0/572593926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 msgr2=0x7fa720100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.890+0000 7fa726d44640 1 --2- 172.21.15.46:0/572593926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720100c60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa714009880 tx=0x7fa71402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.890+0000 7fa726d44640 1 -- 172.21.15.46:0/572593926 shutdown_connections 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.890+0000 7fa726d44640 1 --2- 172.21.15.46:0/572593926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa720101a60 0x7fa720101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.890+0000 7fa726d44640 1 --2- 172.21.15.46:0/572593926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720100c60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.890+0000 7fa726d44640 1 -- 172.21.15.46:0/572593926 >> 172.21.15.46:0/572593926 conn(0x7fa7200fc010 msgr2=0x7fa7200fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.890+0000 7fa726d44640 1 -- 172.21.15.46:0/572593926 shutdown_connections 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.891+0000 7fa726d44640 1 -- 172.21.15.46:0/572593926 wait complete. 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.891+0000 7fa726d44640 1 Processor -- start 2024-09-10T04:06:55.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.891+0000 7fa726d44640 1 -- start start 2024-09-10T04:06:55.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.891+0000 7fa726d44640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:55.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.891+0000 7fa726d44640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa720101a60 0x7fa720197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.891+0000 7fa726d44640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201096a0 con 0x7fa720100860 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.891+0000 7fa726d44640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa720109520 con 0x7fa720101a60 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa724ae0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa724ae0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35080/0 (socket says 172.21.15.46:35080) 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa71ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa720101a60 0x7fa720197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa724ae0640 1 -- 172.21.15.46:0/256151479 learned_addr learned my addr 172.21.15.46:0/256151479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa71ffff640 1 -- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 msgr2=0x7fa720197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa71ffff640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa71ffff640 1 -- 172.21.15.46:0/256151479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7201980e0 con 0x7fa720101a60 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa724ae0640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa71ffff640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa720101a60 0x7fa720197a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e970 tx=0x7fa70c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:55.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa71dffb640 1 -- 172.21.15.46:0/256151479 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa70c00cac0 con 0x7fa720101a60 2024-09-10T04:06:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.892+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72019cb20 con 0x7fa720101a60 2024-09-10T04:06:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.893+0000 7fa71dffb640 1 -- 172.21.15.46:0/256151479 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa70c00cc60 con 0x7fa720101a60 2024-09-10T04:06:55.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.893+0000 7fa71dffb640 1 -- 172.21.15.46:0/256151479 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa70c0047a0 con 0x7fa720101a60 2024-09-10T04:06:55.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.894+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72019d0b0 con 0x7fa720101a60 2024-09-10T04:06:55.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.894+0000 7fa71dffb640 1 -- 172.21.15.46:0/256151479 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 28) ==== 51995+0+0 (secure 0 0 0) 0x7fa70c004a60 con 0x7fa720101a60 2024-09-10T04:06:55.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.894+0000 7fa71dffb640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71003f9c0 0x7fa710041e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:06:55.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.894+0000 7fa71dffb640 1 -- 172.21.15.46:0/256151479 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fa70c055200 con 0x7fa720101a60 2024-09-10T04:06:55.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.895+0000 7fa724ae0640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71003f9c0 0x7fa710041e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:06:55.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.895+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa720100c60 con 0x7fa720101a60 2024-09-10T04:06:55.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.895+0000 7fa724ae0640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71003f9c0 0x7fa710041e80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa7140096c0 tx=0x7fa7140058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:06:55.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:55.900+0000 7fa71dffb640 1 -- 172.21.15.46:0/256151479 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fa70c00ce10 con 0x7fa720101a60 2024-09-10T04:06:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:56 smithi046 ceph-mon[28328]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:56 smithi046 ceph-mon[28328]: mgrmap e28: smithi046.zvckjg(active, since 3s) 2024-09-10T04:06:56.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:56 smithi046 ceph-mon[28328]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:56.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:56 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/3733315386' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:06:56.136 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:56 smithi070 ceph-mon[35495]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:56.136 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:56 smithi070 ceph-mon[35495]: mgrmap e28: smithi046.zvckjg(active, since 3s) 2024-09-10T04:06:56.136 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:56 smithi070 ceph-mon[35495]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:56.136 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:56 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/3733315386' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:06:56.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.203+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa72006a460 con 0x7fa71003f9c0 2024-09-10T04:06:56.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.205+0000 7fa71dffb640 1 -- 172.21.15.46:0/256151479 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+329 (secure 0 0 0) 0x7fa72006a460 con 0x7fa71003f9c0 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:06:56.207 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:06:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71003f9c0 msgr2=0x7fa710041e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71003f9c0 0x7fa710041e80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa7140096c0 tx=0x7fa7140058d0 comp rx=0 tx=0).stop 2024-09-10T04:06:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa720101a60 msgr2=0x7fa720197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:06:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa720101a60 0x7fa720197a60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e970 tx=0x7fa70c00ee30 comp rx=0 tx=0).stop 2024-09-10T04:06:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 shutdown_connections 2024-09-10T04:06:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71003f9c0 0x7fa710041e80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa720101a60 0x7fa720197a60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 --2- 172.21.15.46:0/256151479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa720100860 0x7fa720197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:06:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 >> 172.21.15.46:0/256151479 conn(0x7fa7200fc010 msgr2=0x7fa720069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:06:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 shutdown_connections 2024-09-10T04:06:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:06:56.207+0000 7fa726d44640 1 -- 172.21.15.46:0/256151479 wait complete. 2024-09-10T04:06:57.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:57 smithi070 ceph-mon[35495]: pgmap v5: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:06:57.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:57 smithi070 ceph-mon[35495]: from='client.24549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:57.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:57 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:57.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:57 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:57 smithi046 ceph-mon[28328]: pgmap v5: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:06:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:57 smithi046 ceph-mon[28328]: from='client.24549 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:06:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:57 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:57 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: [10/Sep/2024:04:06:56] ENGINE Bus STARTING 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: [10/Sep/2024:04:06:56] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: [10/Sep/2024:04:06:56] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: [10/Sep/2024:04:06:56] ENGINE Bus STARTED 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: [10/Sep/2024:04:06:56] ENGINE Error in HTTPServer.serve 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Traceback (most recent call last): 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:06:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._run(expiration_interval) 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: s = self.context.wrap_socket( 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: return self.sslsocket_class._create( 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self.do_handshake() 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:06:58.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: self._sslobj.do_handshake() 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: mgrmap e29: smithi046.zvckjg(active, since 5s) 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: Standby manager daemon smithi070.tezbzn started 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T04:06:58.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:58 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: [10/Sep/2024:04:06:56] ENGINE Bus STARTING 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: [10/Sep/2024:04:06:56] ENGINE Serving on http://172.21.15.46:8765 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: [10/Sep/2024:04:06:56] ENGINE Serving on https://172.21.15.46:7150 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: [10/Sep/2024:04:06:56] ENGINE Bus STARTED 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: [10/Sep/2024:04:06:56] ENGINE Error in HTTPServer.serve 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Traceback (most recent call last): 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._run(expiration_interval) 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:06:58.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: s = self.context.wrap_socket( 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: return self.sslsocket_class._create( 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self.do_handshake() 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: self._sslobj.do_handshake() 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: mgrmap e29: smithi046.zvckjg(active, since 5s) 2024-09-10T04:06:58.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: Standby manager daemon smithi070.tezbzn started 2024-09-10T04:06:58.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T04:06:58.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:06:58.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T04:06:58.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:58 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/1658763974' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:06:59.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:59 smithi046 ceph-mon[28328]: pgmap v6: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 509 KiB/s rd, 547 KiB/s wr, 93 op/s 2024-09-10T04:06:59.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:59 smithi046 ceph-mon[28328]: mgrmap e30: smithi046.zvckjg(active, since 6s), standbys: smithi070.tezbzn 2024-09-10T04:06:59.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:06:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi070.tezbzn", "id": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:06:59.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:59 smithi070 ceph-mon[35495]: pgmap v6: 65 pgs: 65 active+clean; 286 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 509 KiB/s rd, 547 KiB/s wr, 93 op/s 2024-09-10T04:06:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:59 smithi070 ceph-mon[35495]: mgrmap e30: smithi046.zvckjg(active, since 6s), standbys: smithi070.tezbzn 2024-09-10T04:06:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:06:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr metadata", "who": "smithi070.tezbzn", "id": "smithi070.tezbzn"}]: dispatch 2024-09-10T04:07:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:01 smithi070 ceph-mon[35495]: pgmap v7: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 944 KiB/s rd, 794 KiB/s wr, 120 op/s 2024-09-10T04:07:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:01 smithi046 ceph-mon[28328]: pgmap v7: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 944 KiB/s rd, 794 KiB/s wr, 120 op/s 2024-09-10T04:07:03.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:03 smithi046 ceph-mon[28328]: pgmap v8: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 734 KiB/s rd, 618 KiB/s wr, 93 op/s 2024-09-10T04:07:03.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:03 smithi070 ceph-mon[35495]: pgmap v8: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 734 KiB/s rd, 618 KiB/s wr, 93 op/s 2024-09-10T04:07:04.999 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:04 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:05.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:04 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:05.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:04 smithi046 ceph-mon[28328]: pgmap v9: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-10T04:07:05.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:04 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:05.000 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:04 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:04 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:04 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:04 smithi070 ceph-mon[35495]: pgmap v9: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-10T04:07:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:04 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:04 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:07 smithi070 ceph-mon[35495]: pgmap v10: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-09-10T04:07:07.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:07 smithi046 ceph-mon[28328]: pgmap v10: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-09-10T04:07:07.379 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-09-10T04:07:07.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-10T04:07:07.722 DEBUG:teuthology.parallel:result is None 2024-09-10T04:07:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:08 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:08.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:08 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:09.214 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:09 smithi046 ceph-mon[28328]: pgmap v11: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T04:07:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:09 smithi070 ceph-mon[35495]: pgmap v11: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T04:07:09.793 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-10T04:07:09.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T04:07:10.165 DEBUG:teuthology.parallel:result is None 2024-09-10T04:07:10.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T04:07:10.204 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T04:07:10.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-10T04:07:10.238 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-10T04:07:10.238 DEBUG:teuthology.parallel:result is None 2024-09-10T04:07:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:11 smithi070 ceph-mon[35495]: pgmap v12: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-10T04:07:11.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:11 smithi046 ceph-mon[28328]: pgmap v12: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-10T04:07:12.393 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.393 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.393 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:12.393 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:07:12.393 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.394 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:12 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:12.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: Updating smithi046:/etc/ceph/ceph.conf 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:12.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:12 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:13 smithi046 ceph-mon[28328]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:07:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:13 smithi046 ceph-mon[28328]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:07:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:13 smithi046 ceph-mon[28328]: Reconfiguring prometheus.smithi046 (dependencies changed)... 2024-09-10T04:07:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:13 smithi046 ceph-mon[28328]: pgmap v13: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-09-10T04:07:13.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:13 smithi046 ceph-mon[28328]: Reconfiguring daemon prometheus.smithi046 on smithi046 2024-09-10T04:07:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:13 smithi070 ceph-mon[35495]: Updating smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:07:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:13 smithi070 ceph-mon[35495]: Updating smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:07:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:13 smithi070 ceph-mon[35495]: Reconfiguring prometheus.smithi046 (dependencies changed)... 2024-09-10T04:07:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:13 smithi070 ceph-mon[35495]: pgmap v13: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 180 op/s 2024-09-10T04:07:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:13 smithi070 ceph-mon[35495]: Reconfiguring daemon prometheus.smithi046 on smithi046 2024-09-10T04:07:15.387 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:15 smithi070 ceph-mon[35495]: pgmap v14: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 234 op/s 2024-09-10T04:07:15.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:15 smithi046 ceph-mon[28328]: pgmap v14: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 234 op/s 2024-09-10T04:07:17.466 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:17 smithi046 ceph-mon[28328]: pgmap v15: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 856 KiB/s rd, 775 KiB/s wr, 141 op/s 2024-09-10T04:07:17.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:17 smithi070 ceph-mon[35495]: pgmap v15: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 856 KiB/s rd, 775 KiB/s wr, 141 op/s 2024-09-10T04:07:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:18 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:18 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:18 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:07:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:18 smithi070 ceph-mon[35495]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:07:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:18 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:18 smithi070 ceph-mon[35495]: pgmap v16: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 879 KiB/s rd, 797 KiB/s wr, 143 op/s 2024-09-10T04:07:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:18 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:19.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:18 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:19.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:18 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:07:19.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:18 smithi046 ceph-mon[28328]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:07:19.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:18 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:19.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:18 smithi046 ceph-mon[28328]: pgmap v16: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 879 KiB/s rd, 797 KiB/s wr, 143 op/s 2024-09-10T04:07:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:19 smithi070 ceph-mon[35495]: Upgrade: Updating mgr.smithi070.tezbzn 2024-09-10T04:07:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:19 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:07:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:19 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:07:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:19 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T04:07:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:19 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:19 smithi070 ceph-mon[35495]: Deploying daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T04:07:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:19 smithi046 ceph-mon[28328]: Upgrade: Updating mgr.smithi070.tezbzn 2024-09-10T04:07:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:19 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:07:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:19 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi070.tezbzn", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T04:07:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:19 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T04:07:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:19 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:19 smithi046 ceph-mon[28328]: Deploying daemon mgr.smithi070.tezbzn on smithi070 2024-09-10T04:07:21.063 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:20 smithi070 ceph-mon[35495]: pgmap v17: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 612 KiB/s rd, 498 KiB/s wr, 91 op/s 2024-09-10T04:07:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:20 smithi046 ceph-mon[28328]: pgmap v17: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 612 KiB/s rd, 498 KiB/s wr, 91 op/s 2024-09-10T04:07:23.331 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:23 smithi046 ceph-mon[28328]: pgmap v18: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 270 KiB/s rd, 242 KiB/s wr, 56 op/s 2024-09-10T04:07:23.399 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:23 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:23.399 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:23 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:23.399 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:23 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:23.399 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:23 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:23.400 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:23 smithi070 ceph-mon[35495]: pgmap v18: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 270 KiB/s rd, 242 KiB/s wr, 56 op/s 2024-09-10T04:07:23.400 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:23 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:23.400 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:23 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:23.400 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:23 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:23.400 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:23 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:25.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:25 smithi070 ceph-mon[35495]: pgmap v19: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 271 KiB/s rd, 242 KiB/s wr, 57 op/s 2024-09-10T04:07:25.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:25 smithi046 ceph-mon[28328]: pgmap v19: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 271 KiB/s rd, 242 KiB/s wr, 57 op/s 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 -- 172.21.15.46:0/1905138657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0102b60 msgr2=0x7fe5f0102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 --2- 172.21.15.46:0/1905138657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0102b60 0x7fe5f0102fe0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4009880 tx=0x7fe5e402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 -- 172.21.15.46:0/1905138657 shutdown_connections 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 --2- 172.21.15.46:0/1905138657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0102b60 0x7fe5f0102fe0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 --2- 172.21.15.46:0/1905138657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f0101960 0x7fe5f0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 -- 172.21.15.46:0/1905138657 >> 172.21.15.46:0/1905138657 conn(0x7fe5f00fd0f0 msgr2=0x7fe5f00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 -- 172.21.15.46:0/1905138657 shutdown_connections 2024-09-10T04:07:26.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.297+0000 7fe5f5042640 1 -- 172.21.15.46:0/1905138657 wait complete. 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5f5042640 1 Processor -- start 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5f5042640 1 -- start start 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5f5042640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0101960 0x7fe5f0197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5f5042640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f0102b60 0x7fe5f01979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5f5042640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f010a7a0 con 0x7fe5f0101960 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5f5042640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f010a620 con 0x7fe5f0102b60 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5eed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0101960 0x7fe5f0197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5eed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0101960 0x7fe5f0197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59056/0 (socket says 172.21.15.46:59056) 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5eed9d640 1 -- 172.21.15.46:0/2386897477 learned_addr learned my addr 172.21.15.46:0/2386897477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.298+0000 7fe5ee59c640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f0102b60 0x7fe5f01979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5eed9d640 1 -- 172.21.15.46:0/2386897477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f0102b60 msgr2=0x7fe5f01979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5eed9d640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f0102b60 0x7fe5f01979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5eed9d640 1 -- 172.21.15.46:0/2386897477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f0198050 con 0x7fe5f0101960 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5eed9d640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0101960 0x7fe5f0197490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000eaf0 tx=0x7fe5e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5d7fff640 1 -- 172.21.15.46:0/2386897477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5e000cd10 con 0x7fe5f0101960 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f019ca90 con 0x7fe5f0101960 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5d7fff640 1 -- 172.21.15.46:0/2386897477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5e0003e30 con 0x7fe5f0101960 2024-09-10T04:07:26.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5d7fff640 1 -- 172.21.15.46:0/2386897477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5e0010660 con 0x7fe5f0101960 2024-09-10T04:07:26.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.299+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f019cff0 con 0x7fe5f0101960 2024-09-10T04:07:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.300+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f010af80 con 0x7fe5f0101960 2024-09-10T04:07:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.301+0000 7fe5d7fff640 1 -- 172.21.15.46:0/2386897477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 30) ==== 103268+0+0 (secure 0 0 0) 0x7fe5e0003970 con 0x7fe5f0101960 2024-09-10T04:07:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.301+0000 7fe5d7fff640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5d807ad10 0x7fe5d807d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.301+0000 7fe5d7fff640 1 -- 172.21.15.46:0/2386897477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fe5e009e720 con 0x7fe5f0101960 2024-09-10T04:07:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.302+0000 7fe5ee59c640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5d807ad10 0x7fe5d807d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.302+0000 7fe5ee59c640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5d807ad10 0x7fe5d807d1d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e40058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.306+0000 7fe5d7fff640 1 -- 172.21.15.46:0/2386897477 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fe5e0069cf0 con 0x7fe5f0101960 2024-09-10T04:07:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.517+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe5f002c640 con 0x7fe5d807ad10 2024-09-10T04:07:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.518+0000 7fe5d7fff640 1 -- 172.21.15.46:0/2386897477 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7fe5f002c640 con 0x7fe5d807ad10 2024-09-10T04:07:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.520+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5d807ad10 msgr2=0x7fe5d807d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.520+0000 7fe5f5042640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5d807ad10 0x7fe5d807d1d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e40058d0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.520+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0101960 msgr2=0x7fe5f0197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.520+0000 7fe5f5042640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0101960 0x7fe5f0197490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000eaf0 tx=0x7fe5e000c2d0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.521+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 shutdown_connections 2024-09-10T04:07:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.521+0000 7fe5f5042640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5d807ad10 0x7fe5d807d1d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.521+0000 7fe5f5042640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f0102b60 0x7fe5f01979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.521+0000 7fe5f5042640 1 --2- 172.21.15.46:0/2386897477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5f0101960 0x7fe5f0197490 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.521+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 >> 172.21.15.46:0/2386897477 conn(0x7fe5f00fd0f0 msgr2=0x7fe5f0105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.521+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 shutdown_connections 2024-09-10T04:07:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.521+0000 7fe5f5042640 1 -- 172.21.15.46:0/2386897477 wait complete. 2024-09-10T04:07:26.530 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.622+0000 7f8f3faf7640 1 -- 172.21.15.46:0/718229157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 msgr2=0x7f8f38100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.622+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/718229157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f38100c60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009880 tx=0x7f8f3402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.622+0000 7f8f3faf7640 1 -- 172.21.15.46:0/718229157 shutdown_connections 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.622+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/718229157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f38101a60 0x7f8f38101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.622+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/718229157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f38100c60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.622+0000 7f8f3faf7640 1 -- 172.21.15.46:0/718229157 >> 172.21.15.46:0/718229157 conn(0x7f8f380fc010 msgr2=0x7f8f380fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 -- 172.21.15.46:0/718229157 shutdown_connections 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 -- 172.21.15.46:0/718229157 wait complete. 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 Processor -- start 2024-09-10T04:07:26.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 -- start start 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f381087a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f38101a60 0x7f8f38106e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810a7a0 con 0x7f8f38100860 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.623+0000 7f8f3faf7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3810a620 con 0x7f8f38101a60 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d893640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f381087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d893640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f381087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59076/0 (socket says 172.21.15.46:59076) 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d893640 1 -- 172.21.15.46:0/61845135 learned_addr learned my addr 172.21.15.46:0/61845135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:26.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d092640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f38101a60 0x7f8f38106e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d893640 1 -- 172.21.15.46:0/61845135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f38101a60 msgr2=0x7f8f38106e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d893640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f38101a60 0x7f8f38106e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d893640 1 -- 172.21.15.46:0/61845135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f381073a0 con 0x7f8f38100860 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3d893640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f381087a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f34006010 tx=0x7f8f340045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f2effd640 1 -- 172.21.15.46:0/61845135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f3403f070 con 0x7f8f38100860 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f2effd640 1 -- 172.21.15.46:0/61845135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f340049d0 con 0x7f8f38100860 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f38107630 con 0x7f8f38100860 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f2effd640 1 -- 172.21.15.46:0/61845135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f34035030 con 0x7f8f38100860 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.624+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f38107b60 con 0x7f8f38100860 2024-09-10T04:07:26.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.625+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f38100c60 con 0x7f8f38100860 2024-09-10T04:07:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.626+0000 7f8f2effd640 1 -- 172.21.15.46:0/61845135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 30) ==== 103268+0+0 (secure 0 0 0) 0x7f8f34042070 con 0x7f8f38100860 2024-09-10T04:07:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.627+0000 7f8f2effd640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2407ad70 0x7f8f2407d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.627+0000 7f8f2effd640 1 -- 172.21.15.46:0/61845135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f8f340be8c0 con 0x7f8f38100860 2024-09-10T04:07:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.627+0000 7f8f3d092640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2407ad70 0x7f8f2407d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.627+0000 7f8f3d092640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2407ad70 0x7f8f2407d230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8f38108520 tx=0x7f8f28005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:26.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.631+0000 7f8f2effd640 1 -- 172.21.15.46:0/61845135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f8f34089e70 con 0x7f8f38100860 2024-09-10T04:07:26.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.833+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8f38062630 con 0x7f8f2407ad70 2024-09-10T04:07:26.834 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:26 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.834 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:26 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.834 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:26 smithi046 ceph-mon[28328]: pgmap v20: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 22 KiB/s wr, 3 op/s 2024-09-10T04:07:26.834 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:26 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.834 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:26 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.834+0000 7f8f2effd640 1 -- 172.21.15.46:0/61845135 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f8f38062630 con 0x7f8f2407ad70 2024-09-10T04:07:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:26 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:26 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:26 smithi070 ceph-mon[35495]: pgmap v20: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 22 KiB/s wr, 3 op/s 2024-09-10T04:07:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:26 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:26 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2407ad70 msgr2=0x7f8f2407d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2407ad70 0x7f8f2407d230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8f38108520 tx=0x7f8f28005f90 comp rx=0 tx=0).stop 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 msgr2=0x7f8f381087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f381087a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8f34006010 tx=0x7f8f340045e0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 shutdown_connections 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2407ad70 0x7f8f2407d230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f38101a60 0x7f8f38106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 --2- 172.21.15.46:0/61845135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f38100860 0x7f8f381087a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 >> 172.21.15.46:0/61845135 conn(0x7f8f380fc010 msgr2=0x7f8f38104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 shutdown_connections 2024-09-10T04:07:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.836+0000 7f8f3faf7640 1 -- 172.21.15.46:0/61845135 wait complete. 2024-09-10T04:07:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.922+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/3454371564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e4100b20 msgr2=0x7fe7e4100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.922+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/3454371564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e4100b20 0x7fe7e4100f80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009880 tx=0x7fe7d402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.922+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/3454371564 shutdown_connections 2024-09-10T04:07:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.922+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/3454371564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e4100b20 0x7fe7e4100f80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.922+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/3454371564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7e41001e0 0x7fe7e41005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.922+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/3454371564 >> 172.21.15.46:0/3454371564 conn(0x7fe7e40fbff0 msgr2=0x7fe7e40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.923+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/3454371564 shutdown_connections 2024-09-10T04:07:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.923+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/3454371564 wait complete. 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.923+0000 7fe7ebdf2640 1 Processor -- start 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.923+0000 7fe7ebdf2640 1 -- start start 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7ebdf2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e41001e0 0x7fe7e41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7ebdf2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7e4100b20 0x7fe7e4199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7ebdf2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e410c990 con 0x7fe7e41001e0 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7ebdf2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e410c810 con 0x7fe7e4100b20 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e9b8e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e41001e0 0x7fe7e41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e9b8e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e41001e0 0x7fe7e41996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59092/0 (socket says 172.21.15.46:59092) 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e9b8e640 1 -- 172.21.15.46:0/2188514195 learned_addr learned my addr 172.21.15.46:0/2188514195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e938d640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7e4100b20 0x7fe7e4199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e9b8e640 1 -- 172.21.15.46:0/2188514195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7e4100b20 msgr2=0x7fe7e4199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e9b8e640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7e4100b20 0x7fe7e4199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e9b8e640 1 -- 172.21.15.46:0/2188514195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e419a270 con 0x7fe7e41001e0 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e938d640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7e4100b20 0x7fe7e4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7e9b8e640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e41001e0 0x7fe7e41996b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e970 tx=0x7fe7e000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.924+0000 7fe7daffd640 1 -- 172.21.15.46:0/2188514195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe7e000cac0 con 0x7fe7e41001e0 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.925+0000 7fe7daffd640 1 -- 172.21.15.46:0/2188514195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7e00054e0 con 0x7fe7e41001e0 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.925+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e419ecb0 con 0x7fe7e41001e0 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.925+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e419f1c0 con 0x7fe7e41001e0 2024-09-10T04:07:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.925+0000 7fe7daffd640 1 -- 172.21.15.46:0/2188514195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe7e0010430 con 0x7fe7e41001e0 2024-09-10T04:07:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.926+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e410d170 con 0x7fe7e41001e0 2024-09-10T04:07:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.926+0000 7fe7daffd640 1 -- 172.21.15.46:0/2188514195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 30) ==== 103268+0+0 (secure 0 0 0) 0x7fe7e0004f00 con 0x7fe7e41001e0 2024-09-10T04:07:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.927+0000 7fe7daffd640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe7d007ad10 0x7fe7d007d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.927+0000 7fe7daffd640 1 -- 172.21.15.46:0/2188514195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fe7e009e540 con 0x7fe7e41001e0 2024-09-10T04:07:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.927+0000 7fe7e938d640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe7d007ad10 0x7fe7d007d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.928+0000 7fe7e938d640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe7d007ad10 0x7fe7d007d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4000c00 tx=0x7fe7d40058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:26.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:26.936+0000 7fe7daffd640 1 -- 172.21.15.46:0/2188514195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fe7e0069b10 con 0x7fe7e41001e0 2024-09-10T04:07:27.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.131+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe7e4109b90 con 0x7fe7d007ad10 2024-09-10T04:07:27.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.141+0000 7fe7daffd640 1 -- 172.21.15.46:0/2188514195 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fe7e4109b90 con 0x7fe7d007ad10 2024-09-10T04:07:27.141 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (8m) 23s ago 10m 26.4M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (95s) 23s ago 10m 8619k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (82s) 1s ago 9m 8308k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (10m) 23s ago 10m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (9m) 1s ago 9m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (8m) 23s ago 10m 81.5M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (4m) 23s ago 4m 179M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (5m) 23s ago 5m 199M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (5m) 1s ago 4m 26.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (4m) 1s ago 4m 25.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (62s) 23s ago 11m 494M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (4s) 1s ago 9m 65.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:07:27.142 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (11m) 23s ago 11m 57.4M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (9m) 1s ago 9m 46.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (10m) 23s ago 10m 15.7M - 1.5.0 0da6a335fe13 511ca49aaa2b 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (9m) 1s ago 9m 16.5M - 1.5.0 0da6a335fe13 f891c416025f 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 23s ago 8m 203M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (7m) 23s ago 7m 217M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (7m) 23s ago 7m 172M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (6m) 1s ago 6m 234M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (6m) 1s ago 6m 209M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (6m) 1s ago 6m 193M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:07:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 starting - - - - 2024-09-10T04:07:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe7d007ad10 msgr2=0x7fe7d007d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe7d007ad10 0x7fe7d007d1d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4000c00 tx=0x7fe7d40058d0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e41001e0 msgr2=0x7fe7e41996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e41001e0 0x7fe7e41996b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000e970 tx=0x7fe7e000ed70 comp rx=0 tx=0).stop 2024-09-10T04:07:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 shutdown_connections 2024-09-10T04:07:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe7d007ad10 0x7fe7d007d1d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7e4100b20 0x7fe7e4199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 --2- 172.21.15.46:0/2188514195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e41001e0 0x7fe7e41996b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 >> 172.21.15.46:0/2188514195 conn(0x7fe7e40fbff0 msgr2=0x7fe7e40fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.143+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 shutdown_connections 2024-09-10T04:07:27.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.144+0000 7fe7ebdf2640 1 -- 172.21.15.46:0/2188514195 wait complete. 2024-09-10T04:07:27.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.230+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2944931976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 msgr2=0x7fd184101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.230+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2944931976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184101ee0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd174009880 tx=0x7fd17402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2944931976 shutdown_connections 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2944931976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184101ee0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2944931976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd184100860 0x7fd184100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2944931976 >> 172.21.15.46:0/2944931976 conn(0x7fd1840fc010 msgr2=0x7fd1840fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2944931976 shutdown_connections 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2944931976 wait complete. 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 Processor -- start 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.231+0000 7fd18c7b8640 1 -- start start 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18c7b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd184100860 0x7fd184104780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18c7b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18c7b8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd184109680 con 0x7fd184101a60 2024-09-10T04:07:27.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18c7b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd184109500 con 0x7fd184100860 2024-09-10T04:07:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd189d53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd189d53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59114/0 (socket says 172.21.15.46:59114) 2024-09-10T04:07:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd189d53640 1 -- 172.21.15.46:0/2221150773 learned_addr learned my addr 172.21.15.46:0/2221150773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:27.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18a554640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd184100860 0x7fd184104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18a554640 1 -- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 msgr2=0x7fd184102e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18a554640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184102e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18a554640 1 -- 172.21.15.46:0/2221150773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd184103380 con 0x7fd184100860 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd189d53640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd18a554640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd184100860 0x7fd184104780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd18000e940 tx=0x7fd18000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd17b7fe640 1 -- 172.21.15.46:0/2221150773 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd18000cac0 con 0x7fd184100860 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.232+0000 7fd17b7fe640 1 -- 172.21.15.46:0/2221150773 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd18000cc60 con 0x7fd184100860 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.233+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd184103670 con 0x7fd184100860 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.233+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd184103c00 con 0x7fd184100860 2024-09-10T04:07:27.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.233+0000 7fd17b7fe640 1 -- 172.21.15.46:0/2221150773 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd180003e10 con 0x7fd184100860 2024-09-10T04:07:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.234+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd184100c60 con 0x7fd184100860 2024-09-10T04:07:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.235+0000 7fd17b7fe640 1 -- 172.21.15.46:0/2221150773 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 30) ==== 103268+0+0 (secure 0 0 0) 0x7fd180004020 con 0x7fd184100860 2024-09-10T04:07:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.235+0000 7fd17b7fe640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17007ad10 0x7fd17007d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.236+0000 7fd17b7fe640 1 -- 172.21.15.46:0/2221150773 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fd18009e3a0 con 0x7fd184100860 2024-09-10T04:07:27.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.236+0000 7fd189d53640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17007ad10 0x7fd17007d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:27.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.236+0000 7fd189d53640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17007ad10 0x7fd17007d1d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1740096c0 tx=0x7fd174005a20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:27.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.240+0000 7fd17b7fe640 1 -- 172.21.15.46:0/2221150773 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fd180069970 con 0x7fd184100860 2024-09-10T04:07:27.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.498+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fd18419f7a0 con 0x7fd184100860 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.499+0000 7fd17b7fe640 1 -- 172.21.15.46:0/2221150773 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7fd18006e820 con 0x7fd184100860 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:07:27.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 12, 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:07:27.501 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:07:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17007ad10 msgr2=0x7fd17007d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17007ad10 0x7fd17007d1d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd1740096c0 tx=0x7fd174005a20 comp rx=0 tx=0).stop 2024-09-10T04:07:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd184100860 msgr2=0x7fd184104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd184100860 0x7fd184104780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd18000e940 tx=0x7fd18000ee00 comp rx=0 tx=0).stop 2024-09-10T04:07:27.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 shutdown_connections 2024-09-10T04:07:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17007ad10 0x7fd17007d1d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd184101a60 0x7fd184102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 --2- 172.21.15.46:0/2221150773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd184100860 0x7fd184104780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 >> 172.21.15.46:0/2221150773 conn(0x7fd1840fc010 msgr2=0x7fd184069d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 shutdown_connections 2024-09-10T04:07:27.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.501+0000 7fd18c7b8640 1 -- 172.21.15.46:0/2221150773 wait complete. 2024-09-10T04:07:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 -- 172.21.15.46:0/3624603215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10101960 msgr2=0x7f9f10101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 --2- 172.21.15.46:0/3624603215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10101960 0x7f9f10101d60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009880 tx=0x7f9f0002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 -- 172.21.15.46:0/3624603215 shutdown_connections 2024-09-10T04:07:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 --2- 172.21.15.46:0/3624603215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f10102b60 0x7f9f10102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 --2- 172.21.15.46:0/3624603215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10101960 0x7f9f10101d60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 -- 172.21.15.46:0/3624603215 >> 172.21.15.46:0/3624603215 conn(0x7f9f100fd0f0 msgr2=0x7f9f100ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:27.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 -- 172.21.15.46:0/3624603215 shutdown_connections 2024-09-10T04:07:27.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.587+0000 7f9f1524d640 1 -- 172.21.15.46:0/3624603215 wait complete. 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f1524d640 1 Processor -- start 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f1524d640 1 -- start start 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f1524d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f10101960 0x7f9f10199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f1524d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10102b60 0x7f9f10199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f1524d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010a7a0 con 0x7f9f10102b60 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f1524d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1010a620 con 0x7f9f10101960 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f0659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10102b60 0x7f9f10199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f0659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10102b60 0x7f9f10199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59142/0 (socket says 172.21.15.46:59142) 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f0659c640 1 -- 172.21.15.46:0/2216126736 learned_addr learned my addr 172.21.15.46:0/2216126736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f0ed9d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f10101960 0x7f9f10199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f0659c640 1 -- 172.21.15.46:0/2216126736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f10101960 msgr2=0x7f9f10199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f0659c640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f10101960 0x7f9f10199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.588+0000 7f9f0659c640 1 -- 172.21.15.46:0/2216126736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1019a2e0 con 0x7f9f10102b60 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.589+0000 7f9f0659c640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10102b60 0x7f9f10199c60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d6f0 tx=0x7f9ef800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.589+0000 7f9f0cd99640 1 -- 172.21.15.46:0/2216126736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9ef8017070 con 0x7f9f10102b60 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.589+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1019ed20 con 0x7f9f10102b60 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.589+0000 7f9f0cd99640 1 -- 172.21.15.46:0/2216126736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9ef800de00 con 0x7f9f10102b60 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.589+0000 7f9f0cd99640 1 -- 172.21.15.46:0/2216126736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9ef8004bb0 con 0x7f9f10102b60 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.589+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f1019f250 con 0x7f9f10102b60 2024-09-10T04:07:27.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.590+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f10101d60 con 0x7f9f10102b60 2024-09-10T04:07:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.591+0000 7f9f0cd99640 1 -- 172.21.15.46:0/2216126736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 30) ==== 103268+0+0 (secure 0 0 0) 0x7f9ef8004e60 con 0x7f9f10102b60 2024-09-10T04:07:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.591+0000 7f9f0cd99640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9efc07ad10 0x7f9efc07d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.591+0000 7f9f0cd99640 1 -- 172.21.15.46:0/2216126736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f9ef809daa0 con 0x7f9f10102b60 2024-09-10T04:07:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.591+0000 7f9f0ed9d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9efc07ad10 0x7f9efc07d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:27.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.592+0000 7f9f0ed9d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9efc07ad10 0x7f9efc07d1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009230 tx=0x7f9f000058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:27.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.596+0000 7f9f0cd99640 1 -- 172.21.15.46:0/2216126736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f9ef8068fd0 con 0x7f9f10102b60 2024-09-10T04:07:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:27 smithi070 ceph-mon[35495]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:27 smithi070 ceph-mon[35495]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:27 smithi070 ceph-mon[35495]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:27 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2221150773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:27 smithi046 ceph-mon[28328]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:27 smithi046 ceph-mon[28328]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:27 smithi046 ceph-mon[28328]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:27 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2221150773' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:27.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.845+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9f10075dd0 con 0x7f9efc07ad10 2024-09-10T04:07:27.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.846+0000 7f9f0cd99640 1 -- 172.21.15.46:0/2216126736 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+360 (secure 0 0 0) 0x7f9f10075dd0 con 0x7f9efc07ad10 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mgr daemons", 2024-09-10T04:07:27.847 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:07:27.848 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:07:27.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.848+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9efc07ad10 msgr2=0x7f9efc07d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.848+0000 7f9f1524d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9efc07ad10 0x7f9efc07d1d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009230 tx=0x7f9f000058d0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.848+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10102b60 msgr2=0x7f9f10199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:27.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.848+0000 7f9f1524d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10102b60 0x7f9f10199c60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800d6f0 tx=0x7f9ef800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.848+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 shutdown_connections 2024-09-10T04:07:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.848+0000 7f9f1524d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9efc07ad10 0x7f9efc07d1d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.848+0000 7f9f1524d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f10102b60 0x7f9f10199c60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.849+0000 7f9f1524d640 1 --2- 172.21.15.46:0/2216126736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f10101960 0x7f9f10199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.849+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 >> 172.21.15.46:0/2216126736 conn(0x7f9f100fd0f0 msgr2=0x7f9f100746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.849+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 shutdown_connections 2024-09-10T04:07:27.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:27.849+0000 7f9f1524d640 1 -- 172.21.15.46:0/2216126736 wait complete. 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: Standby manager daemon smithi070.tezbzn restarted 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: Standby manager daemon smithi070.tezbzn started 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:28.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:28.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:28 smithi070 ceph-mon[35495]: pgmap v21: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 22 KiB/s wr, 5 op/s 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: Standby manager daemon smithi070.tezbzn restarted 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: Standby manager daemon smithi070.tezbzn started 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/crt"}]: dispatch 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi070.tezbzn/key"}]: dispatch 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: from='mgr.? 172.21.15.70:0/4029886645' entity='mgr.smithi070.tezbzn' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: from='client.14828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:28.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:28 smithi046 ceph-mon[28328]: pgmap v21: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 22 KiB/s wr, 5 op/s 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: mgrmap e31: smithi046.zvckjg(active, since 36s), standbys: smithi070.tezbzn 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all mgr 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi046.zvckjg"}]: dispatch 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi046.zvckjg"}]: dispatch 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi046.zvckjg"}]': finished 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.tezbzn"}]: dispatch 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.tezbzn"}]: dispatch 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.tezbzn"}]': finished 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all rgw 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all iscsi 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all nfs 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all nvmeof 2024-09-10T04:07:29.833 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:29 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: mgrmap e31: smithi046.zvckjg(active, since 36s), standbys: smithi070.tezbzn 2024-09-10T04:07:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:07:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all mgr 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi046.zvckjg"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi046.zvckjg"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi046.zvckjg"}]': finished 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.tezbzn"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.tezbzn"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi070.tezbzn"}]': finished 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all rgw 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all iscsi 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all nfs 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all nvmeof 2024-09-10T04:07:29.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:29 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:30 smithi070 ceph-mon[35495]: pgmap v22: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-10T04:07:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:30 smithi046 ceph-mon[28328]: pgmap v22: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-10T04:07:31.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:31 smithi046 ceph-mon[28328]: Upgrade: Updating node-exporter.smithi046 (1/2) 2024-09-10T04:07:31.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:31 smithi046 ceph-mon[28328]: Deploying daemon node-exporter.smithi046 on smithi046 2024-09-10T04:07:31.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:31 smithi070 ceph-mon[35495]: Upgrade: Updating node-exporter.smithi046 (1/2) 2024-09-10T04:07:31.865 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:31 smithi070 ceph-mon[35495]: Deploying daemon node-exporter.smithi046 on smithi046 2024-09-10T04:07:32.780 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:32 smithi046 ceph-mon[28328]: pgmap v23: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-10T04:07:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:32 smithi070 ceph-mon[35495]: pgmap v23: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 3 op/s 2024-09-10T04:07:35.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:35 smithi070 ceph-mon[35495]: pgmap v24: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T04:07:35.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:35 smithi046 ceph-mon[28328]: pgmap v24: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T04:07:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:37 smithi070 ceph-mon[35495]: pgmap v25: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:37 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:37 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:37 smithi046 ceph-mon[28328]: pgmap v25: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:37 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:37 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:38.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:38 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:38.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:38 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:39.820 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:39 smithi070 ceph-mon[35495]: Upgrade: Updating node-exporter.smithi070 (2/2) 2024-09-10T04:07:39.821 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:39 smithi070 ceph-mon[35495]: Deploying daemon node-exporter.smithi070 on smithi070 2024-09-10T04:07:39.821 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:39 smithi070 ceph-mon[35495]: pgmap v26: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:39 smithi046 ceph-mon[28328]: Upgrade: Updating node-exporter.smithi070 (2/2) 2024-09-10T04:07:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:39 smithi046 ceph-mon[28328]: Deploying daemon node-exporter.smithi070 on smithi070 2024-09-10T04:07:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:39 smithi046 ceph-mon[28328]: pgmap v26: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:41.768 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:41 smithi070 ceph-mon[35495]: pgmap v27: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:41.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:41 smithi046 ceph-mon[28328]: pgmap v27: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:43 smithi070 ceph-mon[35495]: pgmap v28: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:43 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:43 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:43 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:43 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:43 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:43 smithi046 ceph-mon[28328]: pgmap v28: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:43 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:43 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:43 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:43 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:43 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:45.701 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:45 smithi046 ceph-mon[28328]: pgmap v29: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:45.788 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:45 smithi070 ceph-mon[35495]: pgmap v29: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:46.776 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:46 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:46.776 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:46 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:46.776 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:46 smithi070 ceph-mon[35495]: pgmap v30: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:46.777 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:46 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:46.777 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:46 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:46 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:46 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:46 smithi046 ceph-mon[28328]: pgmap v30: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:46 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:46 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:48.972 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:48 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:48.972 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:48 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:48.972 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:48 smithi046 ceph-mon[28328]: pgmap v31: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:48.972 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:48 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:48.972 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:48 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:49.081 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:48 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:49.081 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:48 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:49.082 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:48 smithi070 ceph-mon[35495]: pgmap v31: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:49.082 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:48 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:49.082 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:48 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:50.472 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:50 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:50.472 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:50 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:50 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:50 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:51.501 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:51 smithi046 ceph-mon[28328]: pgmap v32: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:51 smithi070 ceph-mon[35495]: pgmap v32: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:52.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:52.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:52.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:52.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:07:52.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:52.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:52.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:52 smithi046 ceph-mon[28328]: pgmap v33: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:07:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:07:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:07:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:07:52.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:52 smithi070 ceph-mon[35495]: pgmap v33: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:53 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:53 smithi070 ceph-mon[35495]: Upgrade: Updating prometheus.smithi046 2024-09-10T04:07:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:53 smithi070 ceph-mon[35495]: Deploying daemon prometheus.smithi046 on smithi046 2024-09-10T04:07:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:53 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:07:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:53 smithi046 ceph-mon[28328]: Upgrade: Updating prometheus.smithi046 2024-09-10T04:07:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:53 smithi046 ceph-mon[28328]: Deploying daemon prometheus.smithi046 on smithi046 2024-09-10T04:07:54.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:54 smithi046 ceph-mon[28328]: pgmap v34: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:54 smithi070 ceph-mon[35495]: pgmap v34: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:57.328 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:57 smithi070 ceph-mon[35495]: pgmap v35: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:57 smithi046 ceph-mon[28328]: pgmap v35: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:07:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.956+0000 7f551e1d1640 1 -- 172.21.15.46:0/863601653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 msgr2=0x7f5518102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.956+0000 7f551e1d1640 1 --2- 172.21.15.46:0/863601653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 0x7f5518102ff0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f550c009880 tx=0x7f550c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.956+0000 7f551e1d1640 1 -- 172.21.15.46:0/863601653 shutdown_connections 2024-09-10T04:07:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.956+0000 7f551e1d1640 1 --2- 172.21.15.46:0/863601653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 0x7f5518102ff0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.956+0000 7f551e1d1640 1 --2- 172.21.15.46:0/863601653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5518101970 0x7f5518101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:57.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.956+0000 7f551e1d1640 1 -- 172.21.15.46:0/863601653 >> 172.21.15.46:0/863601653 conn(0x7f55180fd140 msgr2=0x7f55180ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.957+0000 7f551e1d1640 1 -- 172.21.15.46:0/863601653 shutdown_connections 2024-09-10T04:07:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.957+0000 7f551e1d1640 1 -- 172.21.15.46:0/863601653 wait complete. 2024-09-10T04:07:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.958+0000 7f551e1d1640 1 Processor -- start 2024-09-10T04:07:57.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.958+0000 7f551e1d1640 1 -- start start 2024-09-10T04:07:57.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.958+0000 7f551e1d1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5518101970 0x7f55181996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:57.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.958+0000 7f551e1d1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 0x7f5518199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:57.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.958+0000 7f551e1d1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5518075f40 con 0x7f5518102b70 2024-09-10T04:07:57.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.958+0000 7f551e1d1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5518075dc0 con 0x7f5518101970 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f5516ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 0x7f5518199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f5516ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 0x7f5518199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34464/0 (socket says 172.21.15.46:34464) 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f55177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5518101970 0x7f55181996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f5516ffd640 1 -- 172.21.15.46:0/3584056142 learned_addr learned my addr 172.21.15.46:0/3584056142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f55177fe640 1 -- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 msgr2=0x7f5518199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f55177fe640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 0x7f5518199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f55177fe640 1 -- 172.21.15.46:0/3584056142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551819a260 con 0x7f5518101970 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.959+0000 7f55177fe640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5518101970 0x7f55181996a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f550800e940 tx=0x7f550800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.960+0000 7f5514ff9640 1 -- 172.21.15.46:0/3584056142 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f550800cac0 con 0x7f5518101970 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.960+0000 7f5514ff9640 1 -- 172.21.15.46:0/3584056142 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f550800cc60 con 0x7f5518101970 2024-09-10T04:07:57.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.960+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551819eca0 con 0x7f5518101970 2024-09-10T04:07:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.960+0000 7f5514ff9640 1 -- 172.21.15.46:0/3584056142 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5508003e10 con 0x7f5518101970 2024-09-10T04:07:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.960+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551819f180 con 0x7f5518101970 2024-09-10T04:07:57.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.961+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5518101d70 con 0x7f5518101970 2024-09-10T04:07:57.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.962+0000 7f5514ff9640 1 -- 172.21.15.46:0/3584056142 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 31) ==== 103268+0+0 (secure 0 0 0) 0x7f5508004000 con 0x7f5518101970 2024-09-10T04:07:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.963+0000 7f5514ff9640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f550007ad10 0x7f550007d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.963+0000 7f5514ff9640 1 -- 172.21.15.46:0/3584056142 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f550809e410 con 0x7f5518101970 2024-09-10T04:07:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.963+0000 7f5516ffd640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f550007ad10 0x7f550007d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:57.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.964+0000 7f5516ffd640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f550007ad10 0x7f550007d1d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f550c0096c0 tx=0x7f550c005a20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:57.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:57.968+0000 7f5514ff9640 1 -- 172.21.15.46:0/3584056142 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f55080699e0 con 0x7f5518101970 2024-09-10T04:07:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.241+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5518062630 con 0x7f550007ad10 2024-09-10T04:07:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.243+0000 7f5514ff9640 1 -- 172.21.15.46:0/3584056142 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+367 (secure 0 0 0) 0x7f5518062630 con 0x7f550007ad10 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f550007ad10 msgr2=0x7f550007d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f550007ad10 0x7f550007d1d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f550c0096c0 tx=0x7f550c005a20 comp rx=0 tx=0).stop 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5518101970 msgr2=0x7f55181996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5518101970 0x7f55181996a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f550800e940 tx=0x7f550800ee00 comp rx=0 tx=0).stop 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 shutdown_connections 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f550007ad10 0x7f550007d1d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5518102b70 0x7f5518199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 --2- 172.21.15.46:0/3584056142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5518101970 0x7f55181996a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 >> 172.21.15.46:0/3584056142 conn(0x7f55180fd140 msgr2=0x7f5518106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 shutdown_connections 2024-09-10T04:07:58.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.245+0000 7f551e1d1640 1 -- 172.21.15.46:0/3584056142 wait complete. 2024-09-10T04:07:58.255 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:07:58.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 -- 172.21.15.46:0/2103896842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 msgr2=0x7f2e28104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 --2- 172.21.15.46:0/2103896842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e28104880 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009c20 tx=0x7f2e2402a970 comp rx=0 tx=0).stop 2024-09-10T04:07:58.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 -- 172.21.15.46:0/2103896842 shutdown_connections 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 --2- 172.21.15.46:0/2103896842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e28104dc0 0x7f2e281071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 --2- 172.21.15.46:0/2103896842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e28104880 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 -- 172.21.15.46:0/2103896842 >> 172.21.15.46:0/2103896842 conn(0x7f2e280fc010 msgr2=0x7f2e280fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 -- 172.21.15.46:0/2103896842 shutdown_connections 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 -- 172.21.15.46:0/2103896842 wait complete. 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 Processor -- start 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.330+0000 7f2e2fada640 1 -- start start 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2fada640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e281975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2fada640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e28104dc0 0x7f2e28197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2fada640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810a820 con 0x7f2e28102490 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2fada640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810a6a0 con 0x7f2e28104dc0 2024-09-10T04:07:58.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d876640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e281975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d876640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e281975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34480/0 (socket says 172.21.15.46:34480) 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d876640 1 -- 172.21.15.46:0/1927810997 learned_addr learned my addr 172.21.15.46:0/1927810997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d075640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e28104dc0 0x7f2e28197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d876640 1 -- 172.21.15.46:0/1927810997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e28104dc0 msgr2=0x7f2e28197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d876640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e28104dc0 0x7f2e28197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d876640 1 -- 172.21.15.46:0/1927810997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e28198180 con 0x7f2e28102490 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d075640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e28104dc0 0x7f2e28197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2d876640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e281975c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e2402f900 tx=0x7f2e2402f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e1effd640 1 -- 172.21.15.46:0/1927810997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e24047070 con 0x7f2e28102490 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e1effd640 1 -- 172.21.15.46:0/1927810997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e24003e30 con 0x7f2e28102490 2024-09-10T04:07:58.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e1effd640 1 -- 172.21.15.46:0/1927810997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e24034070 con 0x7f2e28102490 2024-09-10T04:07:58.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.331+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2819cb60 con 0x7f2e28102490 2024-09-10T04:07:58.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.332+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2819d0f0 con 0x7f2e28102490 2024-09-10T04:07:58.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.333+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e281028f0 con 0x7f2e28102490 2024-09-10T04:07:58.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.333+0000 7f2e1effd640 1 -- 172.21.15.46:0/1927810997 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 103268+0+0 (secure 0 0 0) 0x7f2e2404a070 con 0x7f2e28102490 2024-09-10T04:07:58.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.334+0000 7f2e1effd640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e1407ad10 0x7f2e1407d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.334+0000 7f2e1effd640 1 -- 172.21.15.46:0/1927810997 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f2e240bed90 con 0x7f2e28102490 2024-09-10T04:07:58.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.334+0000 7f2e2d075640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e1407ad10 0x7f2e1407d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.334+0000 7f2e2d075640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e1407ad10 0x7f2e1407d1d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:58.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.339+0000 7f2e1effd640 1 -- 172.21.15.46:0/1927810997 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f2e2403f070 con 0x7f2e28102490 2024-09-10T04:07:58.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.545+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2e28105800 con 0x7f2e1407ad10 2024-09-10T04:07:58.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.547+0000 7f2e1effd640 1 -- 172.21.15.46:0/1927810997 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+367 (secure 0 0 0) 0x7f2e28105800 con 0x7f2e1407ad10 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e1407ad10 msgr2=0x7f2e1407d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e1407ad10 0x7f2e1407d1d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18008040 comp rx=0 tx=0).stop 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 msgr2=0x7f2e281975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e281975c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e2402f900 tx=0x7f2e2402f730 comp rx=0 tx=0).stop 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 shutdown_connections 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e1407ad10 0x7f2e1407d1d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e28104dc0 0x7f2e28197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 --2- 172.21.15.46:0/1927810997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e28102490 0x7f2e281975c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 >> 172.21.15.46:0/1927810997 conn(0x7f2e280fc010 msgr2=0x7f2e280fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 shutdown_connections 2024-09-10T04:07:58.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.549+0000 7f2e2fada640 1 -- 172.21.15.46:0/1927810997 wait complete. 2024-09-10T04:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 -- 172.21.15.46:0/1760179798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c100860 msgr2=0x7f454c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 --2- 172.21.15.46:0/1760179798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c100860 0x7f454c100c60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4548009880 tx=0x7f454802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 -- 172.21.15.46:0/1760179798 shutdown_connections 2024-09-10T04:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 --2- 172.21.15.46:0/1760179798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c101a60 0x7f454c101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 --2- 172.21.15.46:0/1760179798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c100860 0x7f454c100c60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 -- 172.21.15.46:0/1760179798 >> 172.21.15.46:0/1760179798 conn(0x7f454c0fc010 msgr2=0x7f454c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 -- 172.21.15.46:0/1760179798 shutdown_connections 2024-09-10T04:07:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.633+0000 7f455427c640 1 -- 172.21.15.46:0/1760179798 wait complete. 2024-09-10T04:07:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f455427c640 1 Processor -- start 2024-09-10T04:07:58.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f455427c640 1 -- start start 2024-09-10T04:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f455427c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c100860 0x7f454c06aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f455427c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c101a60 0x7f454c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f455427c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1096a0 con 0x7f454c101a60 2024-09-10T04:07:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f455427c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f454c109520 con 0x7f454c100860 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4551817640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c101a60 0x7f454c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4551817640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c101a60 0x7f454c19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34488/0 (socket says 172.21.15.46:34488) 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4551817640 1 -- 172.21.15.46:0/117894001 learned_addr learned my addr 172.21.15.46:0/117894001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4552018640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c100860 0x7f454c06aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4551817640 1 -- 172.21.15.46:0/117894001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c100860 msgr2=0x7f454c06aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4551817640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c100860 0x7f454c06aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4551817640 1 -- 172.21.15.46:0/117894001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c19eb70 con 0x7f454c101a60 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.634+0000 7f4551817640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c101a60 0x7f454c19e630 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f453c00d6f0 tx=0x7f453c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.635+0000 7f45437fe640 1 -- 172.21.15.46:0/117894001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f453c017070 con 0x7f454c101a60 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.635+0000 7f45437fe640 1 -- 172.21.15.46:0/117894001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f453c00de00 con 0x7f454c101a60 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.635+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454c19ed40 con 0x7f454c101a60 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.635+0000 7f45437fe640 1 -- 172.21.15.46:0/117894001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f453c004bb0 con 0x7f454c101a60 2024-09-10T04:07:58.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.635+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454c19f270 con 0x7f454c101a60 2024-09-10T04:07:58.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.636+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f454c100c60 con 0x7f454c101a60 2024-09-10T04:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.636+0000 7f45437fe640 1 -- 172.21.15.46:0/117894001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 103268+0+0 (secure 0 0 0) 0x7f453c009a30 con 0x7f454c101a60 2024-09-10T04:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.637+0000 7f45437fe640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f453807ad70 0x7f453807d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.637+0000 7f4552018640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f453807ad70 0x7f453807d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.637+0000 7f45437fe640 1 -- 172.21.15.46:0/117894001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f453c09d9b0 con 0x7f454c101a60 2024-09-10T04:07:58.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.637+0000 7f4552018640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f453807ad70 0x7f453807d230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4548009230 tx=0x7f45480058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:58.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.642+0000 7f45437fe640 1 -- 172.21.15.46:0/117894001 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f453c068f70 con 0x7f454c101a60 2024-09-10T04:07:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.841+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f454c0626f0 con 0x7f453807ad70 2024-09-10T04:07:58.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.850+0000 7f45437fe640 1 -- 172.21.15.46:0/117894001 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f454c0626f0 con 0x7f453807ad70 2024-09-10T04:07:58.852 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:07:58.852 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (9m) 11s ago 10m 26.4M - 0.25.0 c8568f914cd2 36debec9a859 2024-09-10T04:07:58.852 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (2m) 11s ago 11m 8682k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:07:58.852 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (114s) 13s ago 10m 8350k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (11m) 11s ago 11m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (10m) 13s ago 10m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (9m) 11s ago 10m 77.2M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (5m) 11s ago 5m 90.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (5m) 11s ago 5m 148M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (5m) 13s ago 5m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (5m) 13s ago 5m 25.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (93s) 11s ago 12m 512M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (36s) 13s ago 10m 453M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (12m) 11s ago 12m 57.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (10m) 13s ago 10m 43.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:07:58.853 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (22s) 11s ago 11m 7894k - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (16s) 13s ago 10m 3213k - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 11s ago 8m 210M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (8m) 11s ago 8m 224M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (7m) 11s ago 7m 178M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (7m) 13s ago 7m 234M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (7m) 13s ago 7m 209M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (6m) 13s ago 6m 194M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:07:58.854 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (41s) 11s ago 10m 49.5M - 2.43.0 a07b618ecd1d c424c7e13c52 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f453807ad70 msgr2=0x7f453807d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f453807ad70 0x7f453807d230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4548009230 tx=0x7f45480058d0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c101a60 msgr2=0x7f454c19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c101a60 0x7f454c19e630 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f453c00d6f0 tx=0x7f453c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 shutdown_connections 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f453807ad70 0x7f453807d230 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f454c101a60 0x7f454c19e630 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.852+0000 7f455427c640 1 --2- 172.21.15.46:0/117894001 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c100860 0x7f454c06aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.853+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 >> 172.21.15.46:0/117894001 conn(0x7f454c0fc010 msgr2=0x7f454c104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.853+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 shutdown_connections 2024-09-10T04:07:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.853+0000 7f455427c640 1 -- 172.21.15.46:0/117894001 wait complete. 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 -- 172.21.15.46:0/1541524258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 msgr2=0x7f3e34101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/1541524258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e34101d60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009880 tx=0x7f3e2402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 -- 172.21.15.46:0/1541524258 shutdown_connections 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/1541524258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e34102b60 0x7f3e34102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/1541524258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e34101d60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 -- 172.21.15.46:0/1541524258 >> 172.21.15.46:0/1541524258 conn(0x7f3e340fd0f0 msgr2=0x7f3e340ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 -- 172.21.15.46:0/1541524258 shutdown_connections 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 -- 172.21.15.46:0/1541524258 wait complete. 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 Processor -- start 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.936+0000 7f3e3abf5640 1 -- start start 2024-09-10T04:07:58.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e3abf5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e341996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e3abf5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e34102b60 0x7f3e34199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e3abf5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3410a780 con 0x7f3e34101960 2024-09-10T04:07:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e3abf5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3410a600 con 0x7f3e34102b60 2024-09-10T04:07:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e38991640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e341996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e38991640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e341996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39980/0 (socket says 172.21.15.46:39980) 2024-09-10T04:07:58.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e38991640 1 -- 172.21.15.46:0/126862645 learned_addr learned my addr 172.21.15.46:0/126862645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e33fff640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e34102b60 0x7f3e34199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e38991640 1 -- 172.21.15.46:0/126862645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e34102b60 msgr2=0x7f3e34199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e38991640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e34102b60 0x7f3e34199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e38991640 1 -- 172.21.15.46:0/126862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3419a220 con 0x7f3e34101960 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e38991640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e341996f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e24006010 tx=0x7f3e240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e31ffb640 1 -- 172.21.15.46:0/126862645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3e2403f070 con 0x7f3e34101960 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.937+0000 7f3e31ffb640 1 -- 172.21.15.46:0/126862645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e240044d0 con 0x7f3e34101960 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.938+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3419ec30 con 0x7f3e34101960 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.938+0000 7f3e31ffb640 1 -- 172.21.15.46:0/126862645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3e24035030 con 0x7f3e34101960 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.938+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3419f130 con 0x7f3e34101960 2024-09-10T04:07:58.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.939+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e34101d60 con 0x7f3e34101960 2024-09-10T04:07:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.940+0000 7f3e31ffb640 1 -- 172.21.15.46:0/126862645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 103268+0+0 (secure 0 0 0) 0x7f3e24042070 con 0x7f3e34101960 2024-09-10T04:07:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.940+0000 7f3e31ffb640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e2007ad10 0x7f3e2007d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.940+0000 7f3e31ffb640 1 -- 172.21.15.46:0/126862645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f3e240beb30 con 0x7f3e34101960 2024-09-10T04:07:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.940+0000 7f3e33fff640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e2007ad10 0x7f3e2007d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:58.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.941+0000 7f3e33fff640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e2007ad10 0x7f3e2007d1d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c005e10 tx=0x7f3e1c005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:58.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:58.945+0000 7f3e31ffb640 1 -- 172.21.15.46:0/126862645 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f3e2408a100 con 0x7f3e34101960 2024-09-10T04:07:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.207+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f3e3419f580 con 0x7f3e34101960 2024-09-10T04:07:59.209 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:59 smithi046 ceph-mon[28328]: pgmap v36: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:59.209 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:07:59 smithi046 ceph-mon[28328]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:07:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.207+0000 7f3e31ffb640 1 -- 172.21.15.46:0/126862645 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7f3e2408efb0 con 0x7f3e34101960 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:59.210 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 12, 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:07:59.211 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:07:59.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.210+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e2007ad10 msgr2=0x7f3e2007d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.210+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e2007ad10 0x7f3e2007d1d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c005e10 tx=0x7f3e1c005da0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.210+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 msgr2=0x7f3e341996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.210+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e341996f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e24006010 tx=0x7f3e240048f0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.211+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 shutdown_connections 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.211+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e2007ad10 0x7f3e2007d1d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.211+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e34102b60 0x7f3e34199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.211+0000 7f3e3abf5640 1 --2- 172.21.15.46:0/126862645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e34101960 0x7f3e341996f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.211+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 >> 172.21.15.46:0/126862645 conn(0x7f3e340fd0f0 msgr2=0x7f3e34074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.211+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 shutdown_connections 2024-09-10T04:07:59.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.211+0000 7f3e3abf5640 1 -- 172.21.15.46:0/126862645 wait complete. 2024-09-10T04:07:59.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 -- 172.21.15.46:0/1623001539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200101a60 msgr2=0x7f6200101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:59.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 --2- 172.21.15.46:0/1623001539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200101a60 0x7f6200101ee0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009880 tx=0x7f61f402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 -- 172.21.15.46:0/1623001539 shutdown_connections 2024-09-10T04:07:59.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 --2- 172.21.15.46:0/1623001539 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200101a60 0x7f6200101ee0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 --2- 172.21.15.46:0/1623001539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6200100860 0x7f6200100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 -- 172.21.15.46:0/1623001539 >> 172.21.15.46:0/1623001539 conn(0x7f62000fc010 msgr2=0x7f62000fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:59.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 -- 172.21.15.46:0/1623001539 shutdown_connections 2024-09-10T04:07:59.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.299+0000 7f6205159640 1 -- 172.21.15.46:0/1623001539 wait complete. 2024-09-10T04:07:59.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.300+0000 7f6205159640 1 Processor -- start 2024-09-10T04:07:59.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.300+0000 7f6205159640 1 -- start start 2024-09-10T04:07:59.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.300+0000 7f6205159640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200100860 0x7f620006aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.300+0000 7f6205159640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6200101a60 0x7f620019c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f6205159640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62001096a0 con 0x7f6200100860 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f6205159640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6200109520 con 0x7f6200101a60 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200100860 0x7f620006aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200100860 0x7f620006aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40008/0 (socket says 172.21.15.46:40008) 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fed9d640 1 -- 172.21.15.46:0/393407296 learned_addr learned my addr 172.21.15.46:0/393407296 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fe59c640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6200101a60 0x7f620019c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fed9d640 1 -- 172.21.15.46:0/393407296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6200101a60 msgr2=0x7f620019c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:59.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fed9d640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6200101a60 0x7f620019c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fed9d640 1 -- 172.21.15.46:0/393407296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620019c7f0 con 0x7f6200100860 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61fed9d640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200100860 0x7f620006aca0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c8f0 tx=0x7f61f000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61e7fff640 1 -- 172.21.15.46:0/393407296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61f0004020 con 0x7f6200100860 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61e7fff640 1 -- 172.21.15.46:0/393407296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61f00041c0 con 0x7f6200100860 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620019cae0 con 0x7f6200100860 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f61e7fff640 1 -- 172.21.15.46:0/393407296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f61f00047a0 con 0x7f6200100860 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.301+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f620019d040 con 0x7f6200100860 2024-09-10T04:07:59.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.302+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6200100c60 con 0x7f6200100860 2024-09-10T04:07:59.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.303+0000 7f61e7fff640 1 -- 172.21.15.46:0/393407296 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 31) ==== 103268+0+0 (secure 0 0 0) 0x7f61f0004370 con 0x7f6200100860 2024-09-10T04:07:59.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.304+0000 7f61e7fff640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f61e807ad70 0x7f61e807d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:07:59.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.304+0000 7f61e7fff640 1 -- 172.21.15.46:0/393407296 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f61f009dbf0 con 0x7f6200100860 2024-09-10T04:07:59.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.304+0000 7f61fe59c640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f61e807ad70 0x7f61e807d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:07:59.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.304+0000 7f61fe59c640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f61e807ad70 0x7f61e807d230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61f4000c00 tx=0x7f61f40058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:07:59.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.308+0000 7f61e7fff640 1 -- 172.21.15.46:0/393407296 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f61f0014030 con 0x7f6200100860 2024-09-10T04:07:59.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.512+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6200062630 con 0x7f61e807ad70 2024-09-10T04:07:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.513+0000 7f61e7fff640 1 -- 172.21.15.46:0/393407296 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+367 (secure 0 0 0) 0x7f6200062630 con 0x7f61e807ad70 2024-09-10T04:07:59.515 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:07:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:07:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:07:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:07:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:07:59.515 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:07:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:07:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T04:07:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading prometheus daemons", 2024-09-10T04:07:59.516 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:07:59.516 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:07:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f61e807ad70 msgr2=0x7f61e807d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f61e807ad70 0x7f61e807d230 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f61f4000c00 tx=0x7f61f40058d0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200100860 msgr2=0x7f620006aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:07:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200100860 0x7f620006aca0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61f000c8f0 tx=0x7f61f000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 shutdown_connections 2024-09-10T04:07:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f61e807ad70 0x7f61e807d230 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6200101a60 0x7f620019c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 --2- 172.21.15.46:0/393407296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6200100860 0x7f620006aca0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:07:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.515+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 >> 172.21.15.46:0/393407296 conn(0x7f62000fc010 msgr2=0x7f6200104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:07:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.516+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 shutdown_connections 2024-09-10T04:07:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:07:59.516+0000 7f6205159640 1 -- 172.21.15.46:0/393407296 wait complete. 2024-09-10T04:07:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:59 smithi070 ceph-mon[35495]: pgmap v36: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:07:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:07:59 smithi070 ceph-mon[35495]: from='client.24577 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:00.477 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:00 smithi046 ceph-mon[28328]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:00.477 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:00 smithi046 ceph-mon[28328]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:00.477 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:00 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/126862645' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:00.477 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:00 smithi046 ceph-mon[28328]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:00.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:00 smithi070 ceph-mon[35495]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:00.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:00 smithi070 ceph-mon[35495]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:00.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:00 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/126862645' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:00.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:00 smithi070 ceph-mon[35495]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:01.500 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:01 smithi046 ceph-mon[28328]: pgmap v37: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:01 smithi070 ceph-mon[35495]: pgmap v37: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:03.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:03 smithi046 ceph-mon[28328]: pgmap v38: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:03 smithi070 ceph-mon[35495]: pgmap v38: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:05.242 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:05 smithi046 ceph-mon[28328]: pgmap v39: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:05.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:05 smithi070 ceph-mon[35495]: pgmap v39: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:06 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:06 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:06 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:06 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:06 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:06 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:07.756 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:07 smithi046 ceph-mon[28328]: pgmap v40: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:07 smithi070 ceph-mon[35495]: pgmap v40: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:08.702 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:08 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:08 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:09.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:09 smithi046 ceph-mon[28328]: pgmap v41: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:09 smithi070 ceph-mon[35495]: pgmap v41: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:11.654 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:11 smithi046 ceph-mon[28328]: pgmap v42: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:11.655 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:11 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:11.655 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:11 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:11.655 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:11 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:11.655 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:11 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:11 smithi070 ceph-mon[35495]: pgmap v42: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:11 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:11 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:11 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:11 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:13.759 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:13 smithi046 ceph-mon[28328]: pgmap v43: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:13 smithi070 ceph-mon[35495]: pgmap v43: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: pgmap v44: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:15 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: pgmap v44: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.121 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.122 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:15 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:16 smithi070 ceph-mon[35495]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:08:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:16 smithi046 ceph-mon[28328]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T04:08:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:17 smithi070 ceph-mon[35495]: Upgrade: Updating alertmanager.smithi046 2024-09-10T04:08:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:17 smithi070 ceph-mon[35495]: Deploying daemon alertmanager.smithi046 on smithi046 2024-09-10T04:08:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:17 smithi070 ceph-mon[35495]: pgmap v45: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:17 smithi046 ceph-mon[28328]: Upgrade: Updating alertmanager.smithi046 2024-09-10T04:08:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:17 smithi046 ceph-mon[28328]: Deploying daemon alertmanager.smithi046 on smithi046 2024-09-10T04:08:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:17 smithi046 ceph-mon[28328]: pgmap v45: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:19.724 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:19 smithi046 ceph-mon[28328]: pgmap v46: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:19 smithi070 ceph-mon[35495]: pgmap v46: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:21 smithi070 ceph-mon[35495]: pgmap v47: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:21 smithi046 ceph-mon[28328]: pgmap v47: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:23 smithi046 ceph-mon[28328]: pgmap v48: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:23 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:23 smithi070 ceph-mon[35495]: pgmap v48: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:23 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:25.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:25 smithi046 ceph-mon[28328]: pgmap v49: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:25.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:25 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:25.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:25 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:25.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:25 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:25 smithi070 ceph-mon[35495]: pgmap v49: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:25 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:25 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:25 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:26.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:26 smithi046 ceph-mon[28328]: mgrmap e32: smithi046.zvckjg(active, since 93s), standbys: smithi070.tezbzn 2024-09-10T04:08:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:26 smithi070 ceph-mon[35495]: mgrmap e32: smithi046.zvckjg(active, since 93s), standbys: smithi070.tezbzn 2024-09-10T04:08:27.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:27 smithi046 ceph-mon[28328]: pgmap v50: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:27 smithi070 ceph-mon[35495]: pgmap v50: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 -- 172.21.15.46:0/4257564843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 msgr2=0x7f2eb4104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/4257564843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4104860 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009880 tx=0x7f2eb002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 -- 172.21.15.46:0/4257564843 shutdown_connections 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/4257564843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb4104da0 0x7f2eb4107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/4257564843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4104860 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 -- 172.21.15.46:0/4257564843 >> 172.21.15.46:0/4257564843 conn(0x7f2eb40fc010 msgr2=0x7f2eb40fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 -- 172.21.15.46:0/4257564843 shutdown_connections 2024-09-10T04:08:29.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 -- 172.21.15.46:0/4257564843 wait complete. 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.604+0000 7f2ebadeb640 1 Processor -- start 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2ebadeb640 1 -- start start 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2ebadeb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2ebadeb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb4104da0 0x7f2eb4100420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2ebadeb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb410a7d0 con 0x7f2eb4102470 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2ebadeb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb410a650 con 0x7f2eb4104da0 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2eb8b87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2eb8b87640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50522/0 (socket says 172.21.15.46:50522) 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2eb8b87640 1 -- 172.21.15.46:0/1871695007 learned_addr learned my addr 172.21.15.46:0/1871695007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2eabfff640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb4104da0 0x7f2eb4100420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:29.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2eb8b87640 1 -- 172.21.15.46:0/1871695007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb4104da0 msgr2=0x7f2eb4100420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2eb8b87640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb4104da0 0x7f2eb4100420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.605+0000 7f2eb8b87640 1 -- 172.21.15.46:0/1871695007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb4100960 con 0x7f2eb4102470 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.606+0000 7f2eb8b87640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4101d60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006010 tx=0x7f2eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.606+0000 7f2ea9ffb640 1 -- 172.21.15.46:0/1871695007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2eb003f070 con 0x7f2eb4102470 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.606+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb4100bf0 con 0x7f2eb4102470 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.606+0000 7f2ea9ffb640 1 -- 172.21.15.46:0/1871695007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2eb0004ac0 con 0x7f2eb4102470 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.606+0000 7f2ea9ffb640 1 -- 172.21.15.46:0/1871695007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2eb0035030 con 0x7f2eb4102470 2024-09-10T04:08:29.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.606+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb4101120 con 0x7f2eb4102470 2024-09-10T04:08:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.607+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb4102850 con 0x7f2eb4102470 2024-09-10T04:08:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.608+0000 7f2ea9ffb640 1 -- 172.21.15.46:0/1871695007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2eb0042070 con 0x7f2eb4102470 2024-09-10T04:08:29.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.609+0000 7f2ea9ffb640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e9c07abf0 0x7f2e9c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.609+0000 7f2ea9ffb640 1 -- 172.21.15.46:0/1871695007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f2eb00be810 con 0x7f2eb4102470 2024-09-10T04:08:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.609+0000 7f2eabfff640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e9c07abf0 0x7f2e9c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:29.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.610+0000 7f2eabfff640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e9c07abf0 0x7f2e9c07d0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4101ae0 tx=0x7f2ea400b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:29.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.615+0000 7f2ea9ffb640 1 -- 172.21.15.46:0/1871695007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f2eb008a000 con 0x7f2eb4102470 2024-09-10T04:08:29.768 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:29 smithi046 ceph-mon[28328]: pgmap v51: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T04:08:29.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.825+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2eb4105720 con 0x7f2e9c07abf0 2024-09-10T04:08:29.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.826+0000 7f2ea9ffb640 1 -- 172.21.15.46:0/1871695007 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+369 (secure 0 0 0) 0x7f2eb4105720 con 0x7f2e9c07abf0 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e9c07abf0 msgr2=0x7f2e9c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e9c07abf0 0x7f2e9c07d0b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4101ae0 tx=0x7f2ea400b390 comp rx=0 tx=0).stop 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 msgr2=0x7f2eb4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4101d60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0006010 tx=0x7f2eb00046c0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 shutdown_connections 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e9c07abf0 0x7f2e9c07d0b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2eb4104da0 0x7f2eb4100420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 --2- 172.21.15.46:0/1871695007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb4102470 0x7f2eb4101d60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 >> 172.21.15.46:0/1871695007 conn(0x7f2eb40fc010 msgr2=0x7f2eb40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 shutdown_connections 2024-09-10T04:08:29.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.829+0000 7f2ebadeb640 1 -- 172.21.15.46:0/1871695007 wait complete. 2024-09-10T04:08:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:29 smithi070 ceph-mon[35495]: pgmap v51: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T04:08:29.839 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:08:29.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.914+0000 7f3079c9a640 1 -- 172.21.15.46:0/3355818757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 msgr2=0x7f30741047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.914+0000 7f3079c9a640 1 --2- 172.21.15.46:0/3355818757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f30741047e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f30640098b0 tx=0x7f306402a4d0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.914+0000 7f3079c9a640 1 -- 172.21.15.46:0/3355818757 shutdown_connections 2024-09-10T04:08:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.915+0000 7f3079c9a640 1 --2- 172.21.15.46:0/3355818757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3074104d20 0x7f3074107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.915+0000 7f3079c9a640 1 --2- 172.21.15.46:0/3355818757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f30741047e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.915+0000 7f3079c9a640 1 -- 172.21.15.46:0/3355818757 >> 172.21.15.46:0/3355818757 conn(0x7f30740fc010 msgr2=0x7f30740fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.915+0000 7f3079c9a640 1 -- 172.21.15.46:0/3355818757 shutdown_connections 2024-09-10T04:08:29.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.915+0000 7f3079c9a640 1 -- 172.21.15.46:0/3355818757 wait complete. 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.915+0000 7f3079c9a640 1 Processor -- start 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.915+0000 7f3079c9a640 1 -- start start 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3079c9a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f3074101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3079c9a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3074104d20 0x7f30741003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3079c9a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f307410a7c0 con 0x7f30741023f0 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3079c9a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f307410a640 con 0x7f3074104d20 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f30737fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f3074101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f30737fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f3074101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50550/0 (socket says 172.21.15.46:50550) 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f30737fe640 1 -- 172.21.15.46:0/687754003 learned_addr learned my addr 172.21.15.46:0/687754003 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3072ffd640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3074104d20 0x7f30741003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f30737fe640 1 -- 172.21.15.46:0/687754003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3074104d20 msgr2=0x7f30741003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f30737fe640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3074104d20 0x7f30741003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f30737fe640 1 -- 172.21.15.46:0/687754003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30741008e0 con 0x7f30741023f0 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f30737fe640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f3074101ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30640099d0 tx=0x7f30640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3070ff9640 1 -- 172.21.15.46:0/687754003 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f306403f070 con 0x7f30741023f0 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3074100b70 con 0x7f30741023f0 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.916+0000 7f3070ff9640 1 -- 172.21.15.46:0/687754003 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3064004a40 con 0x7f30741023f0 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.917+0000 7f3070ff9640 1 -- 172.21.15.46:0/687754003 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3064035030 con 0x7f30741023f0 2024-09-10T04:08:29.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.917+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30741010a0 con 0x7f30741023f0 2024-09-10T04:08:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.918+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30741027d0 con 0x7f30741023f0 2024-09-10T04:08:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.918+0000 7f3070ff9640 1 -- 172.21.15.46:0/687754003 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3064042070 con 0x7f30741023f0 2024-09-10T04:08:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.919+0000 7f3070ff9640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f305c07ac40 0x7f305c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.919+0000 7f3070ff9640 1 -- 172.21.15.46:0/687754003 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f30640be690 con 0x7f30741023f0 2024-09-10T04:08:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.919+0000 7f3072ffd640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f305c07ac40 0x7f305c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.920+0000 7f3072ffd640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f305c07ac40 0x7f305c07d100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3074101a60 tx=0x7f306800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:29.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:29.924+0000 7f3070ff9640 1 -- 172.21.15.46:0/687754003 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f3064089f00 con 0x7f30741023f0 2024-09-10T04:08:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.139+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3074105700 con 0x7f305c07ac40 2024-09-10T04:08:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.141+0000 7f3070ff9640 1 -- 172.21.15.46:0/687754003 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+369 (secure 0 0 0) 0x7f3074105700 con 0x7f305c07ac40 2024-09-10T04:08:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f305c07ac40 msgr2=0x7f305c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f305c07ac40 0x7f305c07d100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3074101a60 tx=0x7f306800b390 comp rx=0 tx=0).stop 2024-09-10T04:08:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 msgr2=0x7f3074101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f3074101ce0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30640099d0 tx=0x7f30640046c0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 shutdown_connections 2024-09-10T04:08:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f305c07ac40 0x7f305c07d100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3074104d20 0x7f30741003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 --2- 172.21.15.46:0/687754003 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30741023f0 0x7f3074101ce0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 >> 172.21.15.46:0/687754003 conn(0x7f30740fc010 msgr2=0x7f30740fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 shutdown_connections 2024-09-10T04:08:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.143+0000 7f3079c9a640 1 -- 172.21.15.46:0/687754003 wait complete. 2024-09-10T04:08:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.235+0000 7f5e05d7d640 1 -- 172.21.15.46:0/310319530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 msgr2=0x7f5e00107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.235+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/310319530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e00107110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5df40098b0 tx=0x7f5df402a4d0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 -- 172.21.15.46:0/310319530 shutdown_connections 2024-09-10T04:08:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/310319530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e00107110 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/310319530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e001023f0 0x7f5e001047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 -- 172.21.15.46:0/310319530 >> 172.21.15.46:0/310319530 conn(0x7f5e000fc010 msgr2=0x7f5e000fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 -- 172.21.15.46:0/310319530 shutdown_connections 2024-09-10T04:08:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 -- 172.21.15.46:0/310319530 wait complete. 2024-09-10T04:08:30.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 Processor -- start 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.236+0000 7f5e05d7d640 1 -- start start 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5e05d7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e001023f0 0x7f5e00101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5e05d7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e001003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5e05d7d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010a7b0 con 0x7f5e00104d20 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5e05d7d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0010a630 con 0x7f5e001023f0 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5dfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e001003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5dfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e001003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50564/0 (socket says 172.21.15.46:50564) 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5dfeffd640 1 -- 172.21.15.46:0/1344737288 learned_addr learned my addr 172.21.15.46:0/1344737288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5dff7fe640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e001023f0 0x7f5e00101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5dfeffd640 1 -- 172.21.15.46:0/1344737288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e001023f0 msgr2=0x7f5e00101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5dfeffd640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e001023f0 0x7f5e00101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.237+0000 7f5dfeffd640 1 -- 172.21.15.46:0/1344737288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e001008e0 con 0x7f5e00104d20 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.238+0000 7f5dfeffd640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e001003a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009230 tx=0x7f5df40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.238+0000 7f5dfcff9640 1 -- 172.21.15.46:0/1344737288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5df403f070 con 0x7f5e00104d20 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.238+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e00100b70 con 0x7f5e00104d20 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.238+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001010a0 con 0x7f5e00104d20 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.238+0000 7f5dfcff9640 1 -- 172.21.15.46:0/1344737288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5df40049d0 con 0x7f5e00104d20 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.238+0000 7f5dfcff9640 1 -- 172.21.15.46:0/1344737288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5df4035030 con 0x7f5e00104d20 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.239+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec0052c0 con 0x7f5e00104d20 2024-09-10T04:08:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.240+0000 7f5dfcff9640 1 -- 172.21.15.46:0/1344737288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5df4042070 con 0x7f5e00104d20 2024-09-10T04:08:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.240+0000 7f5dfcff9640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5de807ac40 0x7f5de807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.240+0000 7f5dfcff9640 1 -- 172.21.15.46:0/1344737288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f5df40be890 con 0x7f5e00104d20 2024-09-10T04:08:30.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.241+0000 7f5dff7fe640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5de807ac40 0x7f5de807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.241+0000 7f5dff7fe640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5de807ac40 0x7f5de807d100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009bd0 tx=0x7f5df0005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:30.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.248+0000 7f5dfcff9640 1 -- 172.21.15.46:0/1344737288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f5df408a1a0 con 0x7f5e00104d20 2024-09-10T04:08:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.446+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5dec000bf0 con 0x7f5de807ac40 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.464+0000 7f5dfcff9640 1 -- 172.21.15.46:0/1344737288 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f5dec000bf0 con 0x7f5de807ac40 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (6s) 0s ago 11m 14.8M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (2m) 0s ago 11m 8736k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (2m) 44s ago 10m 8350k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (11m) 0s ago 11m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (10m) 44s ago 10m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (9m) 0s ago 11m 80.1M - 9.4.12 2bacad6d85d8 b1da8ca83de6 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (6m) 0s ago 6m 90.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:08:30.466 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (6m) 0s ago 6m 148M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (6m) 44s ago 6m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (5m) 44s ago 5m 25.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (2m) 0s ago 12m 517M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (68s) 44s ago 10m 453M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (12m) 0s ago 12m 57.8M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (10m) 44s ago 10m 43.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (54s) 0s ago 11m 10.4M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (48s) 44s ago 10m 3213k - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (9m) 0s ago 9m 211M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (8m) 0s ago 8m 225M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (8m) 0s ago 8m 179M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (7m) 44s ago 7m 234M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (7m) 44s ago 7m 209M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:08:30.467 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (7m) 44s ago 7m 194M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:08:30.468 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (25s) 0s ago 11m 43.9M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:08:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5de807ac40 msgr2=0x7f5de807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5de807ac40 0x7f5de807d100 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009bd0 tx=0x7f5df0005f90 comp rx=0 tx=0).stop 2024-09-10T04:08:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 msgr2=0x7f5e001003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e001003a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009230 tx=0x7f5df40045e0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 shutdown_connections 2024-09-10T04:08:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5de807ac40 0x7f5de807d100 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e00104d20 0x7f5e001003a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 --2- 172.21.15.46:0/1344737288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e001023f0 0x7f5e00101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 >> 172.21.15.46:0/1344737288 conn(0x7f5e000fc010 msgr2=0x7f5e000fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 shutdown_connections 2024-09-10T04:08:30.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.466+0000 7f5e05d7d640 1 -- 172.21.15.46:0/1344737288 wait complete. 2024-09-10T04:08:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 -- 172.21.15.46:0/2236443566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 msgr2=0x7fc3441047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2236443566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc3441047e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc3340098b0 tx=0x7fc33402a4d0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 -- 172.21.15.46:0/2236443566 shutdown_connections 2024-09-10T04:08:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2236443566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc344104d20 0x7fc344107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2236443566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc3441047e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 -- 172.21.15.46:0/2236443566 >> 172.21.15.46:0/2236443566 conn(0x7fc3440fc010 msgr2=0x7fc3440fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 -- 172.21.15.46:0/2236443566 shutdown_connections 2024-09-10T04:08:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.553+0000 7fc3490f0640 1 -- 172.21.15.46:0/2236443566 wait complete. 2024-09-10T04:08:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.554+0000 7fc3490f0640 1 Processor -- start 2024-09-10T04:08:30.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.554+0000 7fc3490f0640 1 -- start start 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.554+0000 7fc3490f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc344101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.554+0000 7fc3490f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc344104d20 0x7fc3441003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.554+0000 7fc3490f0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410a830 con 0x7fc3441023f0 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.554+0000 7fc3490f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc34410a6b0 con 0x7fc344104d20 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.554+0000 7fc342d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc344101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc342d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc344101ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50586/0 (socket says 172.21.15.46:50586) 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc342d9d640 1 -- 172.21.15.46:0/2081345024 learned_addr learned my addr 172.21.15.46:0/2081345024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc34259c640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc344104d20 0x7fc3441003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc342d9d640 1 -- 172.21.15.46:0/2081345024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc344104d20 msgr2=0x7fc3441003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc342d9d640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc344104d20 0x7fc3441003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc342d9d640 1 -- 172.21.15.46:0/2081345024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3441008e0 con 0x7fc3441023f0 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc342d9d640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc344101ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc3340099d0 tx=0x7fc3340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:30.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc32bfff640 1 -- 172.21.15.46:0/2081345024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc33403f070 con 0x7fc3441023f0 2024-09-10T04:08:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc344100b70 con 0x7fc3441023f0 2024-09-10T04:08:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.555+0000 7fc32bfff640 1 -- 172.21.15.46:0/2081345024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc334004c70 con 0x7fc3441023f0 2024-09-10T04:08:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.556+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc344101120 con 0x7fc3441023f0 2024-09-10T04:08:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.556+0000 7fc32bfff640 1 -- 172.21.15.46:0/2081345024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc334035030 con 0x7fc3441023f0 2024-09-10T04:08:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.557+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3441027d0 con 0x7fc3441023f0 2024-09-10T04:08:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.558+0000 7fc32bfff640 1 -- 172.21.15.46:0/2081345024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc334042070 con 0x7fc3441023f0 2024-09-10T04:08:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.558+0000 7fc32bfff640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc32c07abf0 0x7fc32c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.559+0000 7fc32bfff640 1 -- 172.21.15.46:0/2081345024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fc3340be820 con 0x7fc3441023f0 2024-09-10T04:08:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.559+0000 7fc34259c640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc32c07abf0 0x7fc32c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.559+0000 7fc34259c640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc32c07abf0 0x7fc32c07d0b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc344101a60 tx=0x7fc338009110 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.563+0000 7fc32bfff640 1 -- 172.21.15.46:0/2081345024 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fc33408a010 con 0x7fc3441023f0 2024-09-10T04:08:30.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.822+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc34419f7d0 con 0x7fc3441023f0 2024-09-10T04:08:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.823+0000 7fc32bfff640 1 -- 172.21.15.46:0/2081345024 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7fc33408eec0 con 0x7fc3441023f0 2024-09-10T04:08:30.825 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:08:30.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:08:30.825 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:08:30.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:08:30.825 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 12, 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:08:30.826 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:08:30.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc32c07abf0 msgr2=0x7fc32c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc32c07abf0 0x7fc32c07d0b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc344101a60 tx=0x7fc338009110 comp rx=0 tx=0).stop 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 msgr2=0x7fc344101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc344101ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc3340099d0 tx=0x7fc3340048f0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 shutdown_connections 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc32c07abf0 0x7fc32c07d0b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc344104d20 0x7fc3441003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 --2- 172.21.15.46:0/2081345024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3441023f0 0x7fc344101ce0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 >> 172.21.15.46:0/2081345024 conn(0x7fc3440fc010 msgr2=0x7fc3440fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 shutdown_connections 2024-09-10T04:08:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.826+0000 7fc3490f0640 1 -- 172.21.15.46:0/2081345024 wait complete. 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 -- 172.21.15.46:0/4263857721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8102470 msgr2=0x7fe4f8104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/4263857721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8102470 0x7fe4f8104860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009880 tx=0x7fe4ec02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 -- 172.21.15.46:0/4263857721 shutdown_connections 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/4263857721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f8104da0 0x7fe4f8107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/4263857721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8102470 0x7fe4f8104860 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 -- 172.21.15.46:0/4263857721 >> 172.21.15.46:0/4263857721 conn(0x7fe4f80fc010 msgr2=0x7fe4f80fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 -- 172.21.15.46:0/4263857721 shutdown_connections 2024-09-10T04:08:30.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.913+0000 7fe4fedc1640 1 -- 172.21.15.46:0/4263857721 wait complete. 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4fedc1640 1 Processor -- start 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4fedc1640 1 -- start start 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4fedc1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f8102470 0x7fe4f81952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4fedc1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8104da0 0x7fe4f8195820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4fedc1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810a770 con 0x7fe4f8104da0 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4fedc1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f810a5f0 con 0x7fe4f8102470 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4f7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8104da0 0x7fe4f8195820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4f7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8104da0 0x7fe4f8195820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50604/0 (socket says 172.21.15.46:50604) 2024-09-10T04:08:30.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4f7fff640 1 -- 172.21.15.46:0/2153327452 learned_addr learned my addr 172.21.15.46:0/2153327452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.914+0000 7fe4fcb5d640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f8102470 0x7fe4f81952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4f7fff640 1 -- 172.21.15.46:0/2153327452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f8102470 msgr2=0x7fe4f81952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4f7fff640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f8102470 0x7fe4f81952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4f7fff640 1 -- 172.21.15.46:0/2153327452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f8195ea0 con 0x7fe4f8104da0 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4f7fff640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8104da0 0x7fe4f8195820 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400e940 tx=0x7fe4e400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4f5ffb640 1 -- 172.21.15.46:0/2153327452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe4e400cac0 con 0x7fe4f8104da0 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4f5ffb640 1 -- 172.21.15.46:0/2153327452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe4e40054e0 con 0x7fe4f8104da0 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f819a8e0 con 0x7fe4f8104da0 2024-09-10T04:08:30.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.915+0000 7fe4f5ffb640 1 -- 172.21.15.46:0/2153327452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe4e4004e40 con 0x7fe4f8104da0 2024-09-10T04:08:30.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.916+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f819ae70 con 0x7fe4f8104da0 2024-09-10T04:08:30.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.917+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f8102850 con 0x7fe4f8104da0 2024-09-10T04:08:30.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.917+0000 7fe4f5ffb640 1 -- 172.21.15.46:0/2153327452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe4e4005020 con 0x7fe4f8104da0 2024-09-10T04:08:30.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.918+0000 7fe4f5ffb640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe4e807ac90 0x7fe4e807d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:08:30.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.918+0000 7fe4f5ffb640 1 -- 172.21.15.46:0/2153327452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fe4e409e110 con 0x7fe4f8104da0 2024-09-10T04:08:30.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.918+0000 7fe4fcb5d640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe4e807ac90 0x7fe4e807d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:08:30.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.918+0000 7fe4fcb5d640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe4e807ac90 0x7fe4e807d150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec0058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:08:30.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:30.923+0000 7fe4f5ffb640 1 -- 172.21.15.46:0/2153327452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fe4e4069960 con 0x7fe4f8104da0 2024-09-10T04:08:31.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.131+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe4f8062630 con 0x7fe4e807ac90 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.132+0000 7fe4f5ffb640 1 -- 172.21.15.46:0/2153327452 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+369 (secure 0 0 0) 0x7fe4f8062630 con 0x7fe4e807ac90 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T04:08:31.134 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading alertmanager daemons", 2024-09-10T04:08:31.135 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:08:31.135 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:08:31.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.134+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe4e807ac90 msgr2=0x7fe4e807d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.134+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe4e807ac90 0x7fe4e807d150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec0058d0 comp rx=0 tx=0).stop 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.134+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8104da0 msgr2=0x7fe4f8195820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.134+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8104da0 0x7fe4f8195820 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400e940 tx=0x7fe4e400ee00 comp rx=0 tx=0).stop 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.135+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 shutdown_connections 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.135+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe4e807ac90 0x7fe4e807d150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.135+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4f8104da0 0x7fe4f8195820 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.135+0000 7fe4fedc1640 1 --2- 172.21.15.46:0/2153327452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4f8102470 0x7fe4f81952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.135+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 >> 172.21.15.46:0/2153327452 conn(0x7fe4f80fc010 msgr2=0x7fe4f8101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.135+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 shutdown_connections 2024-09-10T04:08:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:08:31.135+0000 7fe4fedc1640 1 -- 172.21.15.46:0/2153327452 wait complete. 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: pgmap v52: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/2081345024' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:31.363 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:31 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='client.14852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: pgmap v52: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T04:08:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:31.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/2081345024' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:31.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:31.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:31 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:32.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:32 smithi046 ceph-mon[28328]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:32 smithi070 ceph-mon[35495]: from='client.14868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:08:33.640 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:33 smithi046 ceph-mon[28328]: pgmap v53: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T04:08:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:33 smithi070 ceph-mon[35495]: pgmap v53: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T04:08:35.560 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:35 smithi046 ceph-mon[28328]: pgmap v54: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T04:08:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:35 smithi070 ceph-mon[35495]: pgmap v54: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T04:08:36.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:36.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:36.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:36.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:36.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:36.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:36 smithi046 ceph-mon[28328]: pgmap v55: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T04:08:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T04:08:37.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T04:08:37.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:37.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:37.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:36 smithi070 ceph-mon[35495]: pgmap v55: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T04:08:38.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:37 smithi070 ceph-mon[35495]: Upgrade: Updating grafana.smithi046 2024-09-10T04:08:38.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:37 smithi070 ceph-mon[35495]: Deploying daemon grafana.smithi046 on smithi046 2024-09-10T04:08:38.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:37 smithi046 ceph-mon[28328]: Upgrade: Updating grafana.smithi046 2024-09-10T04:08:38.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:37 smithi046 ceph-mon[28328]: Deploying daemon grafana.smithi046 on smithi046 2024-09-10T04:08:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:38 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:38 smithi070 ceph-mon[35495]: pgmap v56: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T04:08:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:38 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:38 smithi046 ceph-mon[28328]: pgmap v56: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-10T04:08:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:41 smithi070 ceph-mon[35495]: pgmap v57: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:08:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:41 smithi046 ceph-mon[28328]: pgmap v57: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:08:43.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:43 smithi046 ceph-mon[28328]: pgmap v58: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:08:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:43 smithi070 ceph-mon[35495]: pgmap v58: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:08:45.387 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:45 smithi070 ceph-mon[35495]: pgmap v59: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:08:45.578 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:45 smithi046 ceph-mon[28328]: pgmap v59: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:08:47.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:47 smithi070 ceph-mon[35495]: pgmap v60: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:08:47.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:47 smithi046 ceph-mon[28328]: pgmap v60: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:08:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:49 smithi046 ceph-mon[28328]: pgmap v61: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:08:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:49 smithi070 ceph-mon[35495]: pgmap v61: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:08:51.192 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:50 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:51.192 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:50 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:51.192 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:50 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:51.192 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:50 smithi046 ceph-mon[28328]: pgmap v62: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:51.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:50 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:51.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:50 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:51.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:50 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:51.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:50 smithi070 ceph-mon[35495]: pgmap v62: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:53.362 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:53 smithi046 ceph-mon[28328]: pgmap v63: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:53.362 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:53 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:53.378 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:53 smithi070 ceph-mon[35495]: pgmap v63: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:53.378 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:53 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:08:55.397 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:55 smithi046 ceph-mon[28328]: pgmap v64: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:55.398 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:55 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:55.398 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:55 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:55.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:55 smithi070 ceph-mon[35495]: pgmap v64: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:55.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:55 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:55.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:55 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:56 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:56 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:56 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:56 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:57 smithi070 ceph-mon[35495]: pgmap v65: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:57 smithi046 ceph-mon[28328]: pgmap v65: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:08:59.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: pgmap v66: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:59.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T04:08:59.590 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T04:08:59.591 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T04:08:59.592 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.593 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:59.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:08:59 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: pgmap v66: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T04:08:59.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T04:08:59.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T04:08:59.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:08:59.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:08:59.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:08:59.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:08:59 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:00 smithi070 ceph-mon[35495]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T04:09:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:00 smithi070 ceph-mon[35495]: Upgrade: Finalizing container_image settings 2024-09-10T04:09:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:00 smithi070 ceph-mon[35495]: Upgrade: Complete! 2024-09-10T04:09:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:00 smithi046 ceph-mon[28328]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T04:09:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:00 smithi046 ceph-mon[28328]: Upgrade: Finalizing container_image settings 2024-09-10T04:09:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:00 smithi046 ceph-mon[28328]: Upgrade: Complete! 2024-09-10T04:09:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 -- 172.21.15.46:0/3304747384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810103b60 msgr2=0x7f1810103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 --2- 172.21.15.46:0/3304747384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810103b60 0x7f1810103f60 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1800009880 tx=0x7f180002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 -- 172.21.15.46:0/3304747384 shutdown_connections 2024-09-10T04:09:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 --2- 172.21.15.46:0/3304747384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1810104d60 0x7f18101051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 --2- 172.21.15.46:0/3304747384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810103b60 0x7f1810103f60 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 -- 172.21.15.46:0/3304747384 >> 172.21.15.46:0/3304747384 conn(0x7f18100ff330 msgr2=0x7f1810101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 -- 172.21.15.46:0/3304747384 shutdown_connections 2024-09-10T04:09:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.235+0000 7f1814856640 1 -- 172.21.15.46:0/3304747384 wait complete. 2024-09-10T04:09:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f1814856640 1 Processor -- start 2024-09-10T04:09:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f1814856640 1 -- start start 2024-09-10T04:09:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f1814856640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1810103b60 0x7f18101996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:01.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f1814856640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810104d60 0x7f1810199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:01.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f1814856640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f181010c9a0 con 0x7f1810104d60 2024-09-10T04:09:01.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f1814856640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f181010c820 con 0x7f1810103b60 2024-09-10T04:09:01.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f180dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810104d60 0x7f1810199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:01.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f180dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810104d60 0x7f1810199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46646/0 (socket says 172.21.15.46:46646) 2024-09-10T04:09:01.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f180dd9b640 1 -- 172.21.15.46:0/3969873137 learned_addr learned my addr 172.21.15.46:0/3969873137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.236+0000 7f180e59c640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1810103b60 0x7f18101996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f180dd9b640 1 -- 172.21.15.46:0/3969873137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1810103b60 msgr2=0x7f18101996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f180dd9b640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1810103b60 0x7f18101996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f180dd9b640 1 -- 172.21.15.46:0/3969873137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181019a2a0 con 0x7f1810104d60 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f180e59c640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1810103b60 0x7f18101996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f180dd9b640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810104d60 0x7f1810199c20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f17f800c8f0 tx=0x7f17f800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f18077fe640 1 -- 172.21.15.46:0/3969873137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f17f80079b0 con 0x7f1810104d60 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181019ece0 con 0x7f1810104d60 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f18077fe640 1 -- 172.21.15.46:0/3969873137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17f80054e0 con 0x7f1810104d60 2024-09-10T04:09:01.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181019f210 con 0x7f1810104d60 2024-09-10T04:09:01.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.237+0000 7f18077fe640 1 -- 172.21.15.46:0/3969873137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f17f8004dd0 con 0x7f1810104d60 2024-09-10T04:09:01.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.238+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1810103f60 con 0x7f1810104d60 2024-09-10T04:09:01.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.239+0000 7f18077fe640 1 -- 172.21.15.46:0/3969873137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f17f8005020 con 0x7f1810104d60 2024-09-10T04:09:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.240+0000 7f18077fe640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17fc07ab90 0x7f17fc07d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.240+0000 7f18077fe640 1 -- 172.21.15.46:0/3969873137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f17f809dbe0 con 0x7f1810104d60 2024-09-10T04:09:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.240+0000 7f180e59c640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17fc07ab90 0x7f17fc07d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.240+0000 7f180e59c640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17fc07ab90 0x7f17fc07d050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1800000c00 tx=0x7f1800005a20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:01.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.245+0000 7f18077fe640 1 -- 172.21.15.46:0/3969873137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f17f8014030 con 0x7f1810104d60 2024-09-10T04:09:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.445+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f18100626f0 con 0x7f17fc07ab90 2024-09-10T04:09:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.446+0000 7f18077fe640 1 -- 172.21.15.46:0/3969873137 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+175 (secure 0 0 0) 0x7f18100626f0 con 0x7f17fc07ab90 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17fc07ab90 msgr2=0x7f17fc07d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17fc07ab90 0x7f17fc07d050 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1800000c00 tx=0x7f1800005a20 comp rx=0 tx=0).stop 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810104d60 msgr2=0x7f1810199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810104d60 0x7f1810199c20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f17f800c8f0 tx=0x7f17f800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 shutdown_connections 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17fc07ab90 0x7f17fc07d050 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1810104d60 0x7f1810199c20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 --2- 172.21.15.46:0/3969873137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1810103b60 0x7f18101996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 >> 172.21.15.46:0/3969873137 conn(0x7f18100ff330 msgr2=0x7f1810108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 shutdown_connections 2024-09-10T04:09:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:01.448+0000 7f1814856640 1 -- 172.21.15.46:0/3969873137 wait complete. 2024-09-10T04:09:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:01 smithi070 ceph-mon[35495]: pgmap v67: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:01 smithi046 ceph-mon[28328]: pgmap v67: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:02.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-09-10T04:09:02.596 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:02 smithi070 ceph-mon[35495]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:02.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:02 smithi046 ceph-mon[28328]: from='client.14872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:03.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 -- 172.21.15.46:0/3741345345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c102ac0 msgr2=0x7f6f6c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:03.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 --2- 172.21.15.46:0/3741345345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c102ac0 0x7f6f6c102ea0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009880 tx=0x7f6f5c02a530 comp rx=0 tx=0).stop 2024-09-10T04:09:03.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 -- 172.21.15.46:0/3741345345 shutdown_connections 2024-09-10T04:09:03.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 --2- 172.21.15.46:0/3741345345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c1033e0 0x7f6f6c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 --2- 172.21.15.46:0/3741345345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c102ac0 0x7f6f6c102ea0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 -- 172.21.15.46:0/3741345345 >> 172.21.15.46:0/3741345345 conn(0x7f6f6c0fc520 msgr2=0x7f6f6c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 -- 172.21.15.46:0/3741345345 shutdown_connections 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.085+0000 7f6f7218f640 1 -- 172.21.15.46:0/3741345345 wait complete. 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f7218f640 1 Processor -- start 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f7218f640 1 -- start start 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f7218f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c102ac0 0x7f6f6c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f7218f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c1033e0 0x7f6f6c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f7218f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10ee50 con 0x7f6f6c1033e0 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f7218f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10ecd0 con 0x7f6f6c102ac0 2024-09-10T04:09:03.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f6affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c1033e0 0x7f6f6c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f6affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c1033e0 0x7f6f6c100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46660/0 (socket says 172.21.15.46:46660) 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.086+0000 7f6f6affd640 1 -- 172.21.15.46:0/1159542829 learned_addr learned my addr 172.21.15.46:0/1159542829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f6b7fe640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c102ac0 0x7f6f6c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f6affd640 1 -- 172.21.15.46:0/1159542829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c102ac0 msgr2=0x7f6f6c100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f6affd640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c102ac0 0x7f6f6c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f6affd640 1 -- 172.21.15.46:0/1159542829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c1011a0 con 0x7f6f6c1033e0 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f6affd640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c1033e0 0x7f6f6c100c60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b370 tx=0x7f6f6000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f68ff9640 1 -- 172.21.15.46:0/1159542829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6f60015020 con 0x7f6f6c1033e0 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f68ff9640 1 -- 172.21.15.46:0/1159542829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f60011af0 con 0x7f6f6c1033e0 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c101490 con 0x7f6f6c1033e0 2024-09-10T04:09:03.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f68ff9640 1 -- 172.21.15.46:0/1159542829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6f60010d60 con 0x7f6f6c1033e0 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.087+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c1a3ee0 con 0x7f6f6c1033e0 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.088+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6c06aa90 con 0x7f6f6c1033e0 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.089+0000 7f6f68ff9640 1 -- 172.21.15.46:0/1159542829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6f60018020 con 0x7f6f6c1033e0 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.089+0000 7f6f68ff9640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6f5407ac40 0x7f6f5407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.089+0000 7f6f68ff9640 1 -- 172.21.15.46:0/1159542829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f6f6009d4b0 con 0x7f6f6c1033e0 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.089+0000 7f6f6b7fe640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6f5407ac40 0x7f6f5407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.090+0000 7f6f6b7fe640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6f5407ac40 0x7f6f5407d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c031460 tx=0x7f6f5c009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:03.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.094+0000 7f6f68ff9640 1 -- 172.21.15.46:0/1159542829 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f6f6001f020 con 0x7f6f6c1033e0 2024-09-10T04:09:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.354+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f6f6c1a41f0 con 0x7f6f6c1033e0 2024-09-10T04:09:03.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.354+0000 7f6f68ff9640 1 -- 172.21.15.46:0/1159542829 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7f6f60068d20 con 0x7f6f6c1033e0 2024-09-10T04:09:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6f5407ac40 msgr2=0x7f6f5407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6f5407ac40 0x7f6f5407d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c031460 tx=0x7f6f5c009560 comp rx=0 tx=0).stop 2024-09-10T04:09:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c1033e0 msgr2=0x7f6f6c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:03.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c1033e0 0x7f6f6c100c60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000b370 tx=0x7f6f6000b830 comp rx=0 tx=0).stop 2024-09-10T04:09:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 shutdown_connections 2024-09-10T04:09:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6f5407ac40 0x7f6f5407d100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f6c1033e0 0x7f6f6c100c60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 --2- 172.21.15.46:0/1159542829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f6c102ac0 0x7f6f6c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.356+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 >> 172.21.15.46:0/1159542829 conn(0x7f6f6c0fc520 msgr2=0x7f6f6c10af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.357+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 shutdown_connections 2024-09-10T04:09:03.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:03.357+0000 7f6f7218f640 1 -- 172.21.15.46:0/1159542829 wait complete. 2024-09-10T04:09:03.366 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:09:03.741 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:03 smithi046 ceph-mon[28328]: pgmap v68: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:03.741 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:03 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:03.741 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:03 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1159542829' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:03 smithi070 ceph-mon[35495]: pgmap v68: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:03 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:03 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1159542829' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:04.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-09-10T04:09:04.338 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.799+0000 7fc848d70640 1 -- 172.21.15.46:0/346189606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 msgr2=0x7fc84410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.799+0000 7fc848d70640 1 --2- 172.21.15.46:0/346189606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84410b8d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc838009880 tx=0x7fc83802a530 comp rx=0 tx=0).stop 2024-09-10T04:09:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 -- 172.21.15.46:0/346189606 shutdown_connections 2024-09-10T04:09:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 --2- 172.21.15.46:0/346189606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84410b8d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 --2- 172.21.15.46:0/346189606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc844102ac0 0x7fc844102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 -- 172.21.15.46:0/346189606 >> 172.21.15.46:0/346189606 conn(0x7fc8440fc520 msgr2=0x7fc8440fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 -- 172.21.15.46:0/346189606 shutdown_connections 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 -- 172.21.15.46:0/346189606 wait complete. 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 Processor -- start 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.800+0000 7fc848d70640 1 -- start start 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc848d70640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc844102ac0 0x7fc84419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc848d70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc848d70640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc84410edf0 con 0x7fc8441033e0 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc848d70640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc84410ec70 con 0x7fc844102ac0 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc841d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc841d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46676/0 (socket says 172.21.15.46:46676) 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc841d9b640 1 -- 172.21.15.46:0/57115095 learned_addr learned my addr 172.21.15.46:0/57115095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc84259c640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc844102ac0 0x7fc84419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:04.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc841d9b640 1 -- 172.21.15.46:0/57115095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc844102ac0 msgr2=0x7fc84419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc841d9b640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc844102ac0 0x7fc84419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc841d9b640 1 -- 172.21.15.46:0/57115095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc844197800 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc841d9b640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84419dbc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc838031460 tx=0x7fc8380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc8337fe640 1 -- 172.21.15.46:0/57115095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc83803f070 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc8337fe640 1 -- 172.21.15.46:0/57115095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc838004ac0 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.801+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc844197a90 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.802+0000 7fc8337fe640 1 -- 172.21.15.46:0/57115095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc838035030 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.802+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc844197fc0 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.802+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84406aa90 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.803+0000 7fc8337fe640 1 -- 172.21.15.46:0/57115095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc838042070 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.804+0000 7fc8337fe640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc82c07abf0 0x7fc82c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.804+0000 7fc8337fe640 1 -- 172.21.15.46:0/57115095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fc8380be630 con 0x7fc8441033e0 2024-09-10T04:09:04.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.804+0000 7fc84259c640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc82c07abf0 0x7fc82c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.805+0000 7fc84259c640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc82c07abf0 0x7fc82c07d0b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc8340058c0 tx=0x7fc834005c60 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:04.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:04.809+0000 7fc8337fe640 1 -- 172.21.15.46:0/57115095 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fc838089e20 con 0x7fc8441033e0 2024-09-10T04:09:05.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.068+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc844105f60 con 0x7fc8441033e0 2024-09-10T04:09:05.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.068+0000 7fc8337fe640 1 -- 172.21.15.46:0/57115095 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7fc83808ecd0 con 0x7fc8441033e0 2024-09-10T04:09:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc82c07abf0 msgr2=0x7fc82c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc82c07abf0 0x7fc82c07d0b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc8340058c0 tx=0x7fc834005c60 comp rx=0 tx=0).stop 2024-09-10T04:09:05.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 msgr2=0x7fc84419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84419dbc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc838031460 tx=0x7fc8380046c0 comp rx=0 tx=0).stop 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 shutdown_connections 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc82c07abf0 0x7fc82c07d0b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8441033e0 0x7fc84419dbc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 --2- 172.21.15.46:0/57115095 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc844102ac0 0x7fc84419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.070+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 >> 172.21.15.46:0/57115095 conn(0x7fc8440fc520 msgr2=0x7fc844108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.071+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 shutdown_connections 2024-09-10T04:09:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:05.071+0000 7fc848d70640 1 -- 172.21.15.46:0/57115095 wait complete. 2024-09-10T04:09:05.081 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)" 2024-09-10T04:09:05.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-09-10T04:09:05.746 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:05 smithi046 ceph-mon[28328]: pgmap v69: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:05.746 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:05 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/57115095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:05 smithi070 ceph-mon[35495]: pgmap v69: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:05 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/57115095' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:05.984 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 -- 172.21.15.46:0/1783826195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0107900 msgr2=0x7f66e0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/1783826195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0107900 0x7f66e0107ce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009880 tx=0x7f66d402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 -- 172.21.15.46:0/1783826195 shutdown_connections 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/1783826195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e0101900 0x7f66e0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/1783826195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0107900 0x7f66e0107ce0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 -- 172.21.15.46:0/1783826195 >> 172.21.15.46:0/1783826195 conn(0x7f66e00fd600 msgr2=0x7f66e00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 -- 172.21.15.46:0/1783826195 shutdown_connections 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 -- 172.21.15.46:0/1783826195 wait complete. 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 Processor -- start 2024-09-10T04:09:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.466+0000 7f66e6e8d640 1 -- start start 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e6e8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0101900 0x7f66e019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e6e8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e0107900 0x7f66e019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e6e8d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010ee30 con 0x7f66e0101900 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e6e8d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010ecb0 con 0x7f66e0107900 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e4c29640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0101900 0x7f66e019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e4c29640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0101900 0x7f66e019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46698/0 (socket says 172.21.15.46:46698) 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e4c29640 1 -- 172.21.15.46:0/647905535 learned_addr learned my addr 172.21.15.46:0/647905535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66dffff640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e0107900 0x7f66e019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e4c29640 1 -- 172.21.15.46:0/647905535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e0107900 msgr2=0x7f66e019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e4c29640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e0107900 0x7f66e019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e4c29640 1 -- 172.21.15.46:0/647905535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e0197860 con 0x7f66e0101900 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66dffff640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e0107900 0x7f66e019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:09:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66e4c29640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0101900 0x7f66e019d6a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66d4031460 tx=0x7f66d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.467+0000 7f66ddffb640 1 -- 172.21.15.46:0/647905535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66d403f070 con 0x7f66e0101900 2024-09-10T04:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.468+0000 7f66ddffb640 1 -- 172.21.15.46:0/647905535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66d4004cf0 con 0x7f66e0101900 2024-09-10T04:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.468+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e0197af0 con 0x7f66e0101900 2024-09-10T04:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.468+0000 7f66ddffb640 1 -- 172.21.15.46:0/647905535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66d4035030 con 0x7f66e0101900 2024-09-10T04:09:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.468+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e0198020 con 0x7f66e0101900 2024-09-10T04:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.469+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e0103040 con 0x7f66e0101900 2024-09-10T04:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.469+0000 7f66ddffb640 1 -- 172.21.15.46:0/647905535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f66d4042070 con 0x7f66e0101900 2024-09-10T04:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.470+0000 7f66ddffb640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66d007ac40 0x7f66d007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.470+0000 7f66ddffb640 1 -- 172.21.15.46:0/647905535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f66d40be620 con 0x7f66e0101900 2024-09-10T04:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.470+0000 7f66dffff640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66d007ac40 0x7f66d007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.470+0000 7f66dffff640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66d007ac40 0x7f66d007d100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f66cc006fd0 tx=0x7f66cc008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:06.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.474+0000 7f66ddffb640 1 -- 172.21.15.46:0/647905535 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f66d4089e90 con 0x7f66e0101900 2024-09-10T04:09:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.731+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f66e0198d90 con 0x7f66e0101900 2024-09-10T04:09:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.731+0000 7f66ddffb640 1 -- 172.21.15.46:0/647905535 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7f66d408ed40 con 0x7f66e0101900 2024-09-10T04:09:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66d007ac40 msgr2=0x7f66d007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66d007ac40 0x7f66d007d100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f66cc006fd0 tx=0x7f66cc008040 comp rx=0 tx=0).stop 2024-09-10T04:09:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0101900 msgr2=0x7f66e019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0101900 0x7f66e019d6a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f66d4031460 tx=0x7f66d40048f0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 shutdown_connections 2024-09-10T04:09:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66d007ac40 0x7f66d007d100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e0107900 0x7f66e019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 --2- 172.21.15.46:0/647905535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66e0101900 0x7f66e019d6a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 >> 172.21.15.46:0/647905535 conn(0x7f66e00fd600 msgr2=0x7f66e00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 shutdown_connections 2024-09-10T04:09:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:06.734+0000 7f66e6e8d640 1 -- 172.21.15.46:0/647905535 wait complete. 2024-09-10T04:09:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:09:07.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-09-10T04:09:07.786 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:07 smithi046 ceph-mon[28328]: pgmap v70: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:07.786 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:07 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/647905535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:07 smithi070 ceph-mon[35495]: pgmap v70: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:07 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/647905535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:08.027 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:08 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:09:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:08 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:09:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:09 smithi070 ceph-mon[35495]: pgmap v71: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:09 smithi046 ceph-mon[28328]: pgmap v71: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:10 smithi070 ceph-mon[35495]: pgmap v72: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:10.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:10 smithi046 ceph-mon[28328]: pgmap v72: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 -- 172.21.15.46:0/4164663213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288102ac0 msgr2=0x7f5288102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 --2- 172.21.15.46:0/4164663213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288102ac0 0x7f5288102ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5278009880 tx=0x7f527802a530 comp rx=0 tx=0).stop 2024-09-10T04:09:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 -- 172.21.15.46:0/4164663213 shutdown_connections 2024-09-10T04:09:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 --2- 172.21.15.46:0/4164663213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52881033e0 0x7f528810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 --2- 172.21.15.46:0/4164663213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5288102ac0 0x7f5288102ea0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:10.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 -- 172.21.15.46:0/4164663213 >> 172.21.15.46:0/4164663213 conn(0x7f52880fc520 msgr2=0x7f52880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 -- 172.21.15.46:0/4164663213 shutdown_connections 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.851+0000 7f528e531640 1 -- 172.21.15.46:0/4164663213 wait complete. 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f528e531640 1 Processor -- start 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f528e531640 1 -- start start 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f528e531640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5288102ac0 0x7f5288100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f528e531640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52881033e0 0x7f5288100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f528e531640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f528810edd0 con 0x7f52881033e0 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f528e531640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f528810ec50 con 0x7f5288102ac0 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f52877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52881033e0 0x7f5288100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f52877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52881033e0 0x7f5288100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34098/0 (socket says 172.21.15.46:34098) 2024-09-10T04:09:10.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.852+0000 7f52877fe640 1 -- 172.21.15.46:0/1772540374 learned_addr learned my addr 172.21.15.46:0/1772540374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f5287fff640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5288102ac0 0x7f5288100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f52877fe640 1 -- 172.21.15.46:0/1772540374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5288102ac0 msgr2=0x7f5288100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f52877fe640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5288102ac0 0x7f5288100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f52877fe640 1 -- 172.21.15.46:0/1772540374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52881011a0 con 0x7f52881033e0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f52877fe640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52881033e0 0x7f5288100c60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c8c0 tx=0x7f527c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f52857fa640 1 -- 172.21.15.46:0/1772540374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f527c004020 con 0x7f52881033e0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f52857fa640 1 -- 172.21.15.46:0/1772540374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f527c0041c0 con 0x7f52881033e0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f52857fa640 1 -- 172.21.15.46:0/1772540374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f527c0047a0 con 0x7f52881033e0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5288101490 con 0x7f52881033e0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.853+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52881a3e40 con 0x7f52881033e0 2024-09-10T04:09:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.854+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528806aa90 con 0x7f52881033e0 2024-09-10T04:09:10.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.855+0000 7f52857fa640 1 -- 172.21.15.46:0/1772540374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f527c0079b0 con 0x7f52881033e0 2024-09-10T04:09:10.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.855+0000 7f52857fa640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f527007ac40 0x7f527007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:10.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.856+0000 7f52857fa640 1 -- 172.21.15.46:0/1772540374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f527c09d900 con 0x7f52881033e0 2024-09-10T04:09:10.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.856+0000 7f5287fff640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f527007ac40 0x7f527007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:10.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.856+0000 7f5287fff640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f527007ac40 0x7f527007d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5278000c00 tx=0x7f5278031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:10.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:10.860+0000 7f52857fa640 1 -- 172.21.15.46:0/1772540374 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f527c014030 con 0x7f52881033e0 2024-09-10T04:09:11.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:11.064+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "target": ["mon-mgr", ""]}) -- 0x7f5288062630 con 0x7f527007ac40 2024-09-10T04:09:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:11 smithi070 ceph-mon[35495]: from='client.14888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:11 smithi046 ceph-mon[28328]: from='client.14888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:12 smithi070 ceph-mon[35495]: pgmap v73: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:12 smithi046 ceph-mon[28328]: pgmap v73: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:13.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.122+0000 7f52857fa640 1 -- 172.21.15.46:0/1772540374 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+5741 (secure 0 0 0) 0x7f5288062630 con 0x7f527007ac40 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f527007ac40 msgr2=0x7f527007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f527007ac40 0x7f527007d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5278000c00 tx=0x7f5278031400 comp rx=0 tx=0).stop 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52881033e0 msgr2=0x7f5288100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52881033e0 0x7f5288100c60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c8c0 tx=0x7f527c00cd80 comp rx=0 tx=0).stop 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 shutdown_connections 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f527007ac40 0x7f527007d100 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52881033e0 0x7f5288100c60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:13.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 --2- 172.21.15.46:0/1772540374 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5288102ac0 0x7f5288100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:13.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 >> 172.21.15.46:0/1772540374 conn(0x7f52880fc520 msgr2=0x7f5288108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:13.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 shutdown_connections 2024-09-10T04:09:13.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:13.124+0000 7f528e531640 1 -- 172.21.15.46:0/1772540374 wait complete. 2024-09-10T04:09:13.136 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:09:13.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph orch ps' 2024-09-10T04:09:14.107 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 -- 172.21.15.46:0/2414727965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 msgr2=0x7f473810dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 --2- 172.21.15.46:0/2414727965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473810dcb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4728009880 tx=0x7f472802a490 comp rx=0 tx=0).stop 2024-09-10T04:09:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 -- 172.21.15.46:0/2414727965 shutdown_connections 2024-09-10T04:09:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 --2- 172.21.15.46:0/2414727965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473810dcb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 --2- 172.21.15.46:0/2414727965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4738100640 0x7f4738100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 -- 172.21.15.46:0/2414727965 >> 172.21.15.46:0/2414727965 conn(0x7f47380fc550 msgr2=0x7f47380fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 -- 172.21.15.46:0/2414727965 shutdown_connections 2024-09-10T04:09:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 -- 172.21.15.46:0/2414727965 wait complete. 2024-09-10T04:09:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 Processor -- start 2024-09-10T04:09:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.540+0000 7f473f6ad640 1 -- start start 2024-09-10T04:09:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473f6ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4738100640 0x7f473819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473f6ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473f6ad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4738110ff0 con 0x7f4738100ff0 2024-09-10T04:09:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473f6ad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4738110e70 con 0x7f4738100640 2024-09-10T04:09:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473cc48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473cc48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34118/0 (socket says 172.21.15.46:34118) 2024-09-10T04:09:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473cc48640 1 -- 172.21.15.46:0/3737407427 learned_addr learned my addr 172.21.15.46:0/3737407427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473d449640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4738100640 0x7f473819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473cc48640 1 -- 172.21.15.46:0/3737407427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4738100640 msgr2=0x7f473819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473cc48640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4738100640 0x7f473819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473cc48640 1 -- 172.21.15.46:0/3737407427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4738199a60 con 0x7f4738100ff0 2024-09-10T04:09:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473cc48640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473819fde0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4728031460 tx=0x7f47280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f472e7fc640 1 -- 172.21.15.46:0/3737407427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f472803f070 con 0x7f4738100ff0 2024-09-10T04:09:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f472e7fc640 1 -- 172.21.15.46:0/3737407427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4728004cf0 con 0x7f4738100ff0 2024-09-10T04:09:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4738199cf0 con 0x7f4738100ff0 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.541+0000 7f472e7fc640 1 -- 172.21.15.46:0/3737407427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4728035030 con 0x7f4738100ff0 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.542+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473819a280 con 0x7f4738100ff0 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.543+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473806a580 con 0x7f4738100ff0 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.543+0000 7f472e7fc640 1 -- 172.21.15.46:0/3737407427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4728042070 con 0x7f4738100ff0 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.544+0000 7f472e7fc640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f472407abf0 0x7f472407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.544+0000 7f472e7fc640 1 -- 172.21.15.46:0/3737407427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f47280be740 con 0x7f4738100ff0 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.544+0000 7f473d449640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f472407abf0 0x7f472407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.545+0000 7f473d449640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f472407abf0 0x7f472407d0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f473819aba0 tx=0x7f4734009110 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.549+0000 7f472e7fc640 1 -- 172.21.15.46:0/3737407427 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f4728089e90 con 0x7f4738100ff0 2024-09-10T04:09:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.743+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f473819af90 con 0x7f472407abf0 2024-09-10T04:09:14.755 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:09:14.755 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (50s) 20s ago 12m 15.6M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:09:14.755 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (3m) 20s ago 12m 8770k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:09:14.755 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (3m) 89s ago 11m 8350k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:09:14.755 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (12m) 20s ago 12m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:09:14.755 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (11m) 89s ago 11m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (25s) 20s ago 11m 44.5M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (6m) 20s ago 6m 90.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (6m) 20s ago 6m 147M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (6m) 89s ago 6m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (6m) 89s ago 6m 25.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (2m) 20s ago 13m 526M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (112s) 89s ago 11m 453M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (13m) 20s ago 13m 59.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (11m) 89s ago 11m 43.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (98s) 20s ago 12m 10.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (92s) 89s ago 11m 3213k - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (10m) 20s ago 10m 212M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (9m) 20s ago 9m 225M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:09:14.756 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (9m) 20s ago 9m 179M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:09:14.757 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (8m) 89s ago 8m 234M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:09:14.757 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (8m) 89s ago 8m 209M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:09:14.757 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (7m) 89s ago 7m 194M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:09:14.757 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (69s) 20s ago 11m 44.9M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.753+0000 7f472e7fc640 1 -- 172.21.15.46:0/3737407427 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f473819af90 con 0x7f472407abf0 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f472407abf0 msgr2=0x7f472407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f472407abf0 0x7f472407d0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f473819aba0 tx=0x7f4734009110 comp rx=0 tx=0).stop 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 msgr2=0x7f473819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473819fde0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4728031460 tx=0x7f47280048f0 comp rx=0 tx=0).stop 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 shutdown_connections 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f472407abf0 0x7f472407d0b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4738100ff0 0x7f473819fde0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 --2- 172.21.15.46:0/3737407427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4738100640 0x7f473819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 >> 172.21.15.46:0/3737407427 conn(0x7f47380fc550 msgr2=0x7f47380fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 shutdown_connections 2024-09-10T04:09:14.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:14.755+0000 7f473f6ad640 1 -- 172.21.15.46:0/3737407427 wait complete. 2024-09-10T04:09:15.361 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:15 smithi046 ceph-mon[28328]: pgmap v74: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:15.389 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T04:09:15.390 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:09:15.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-09-10T04:09:15.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:15 smithi070 ceph-mon[35495]: pgmap v74: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 -- 172.21.15.46:0/1094626363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb841006a0 msgr2=0x7fcb84100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 --2- 172.21.15.46:0/1094626363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb841006a0 0x7fcb84100a80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcb74009c20 tx=0x7fcb7402a960 comp rx=0 tx=0).stop 2024-09-10T04:09:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 -- 172.21.15.46:0/1094626363 shutdown_connections 2024-09-10T04:09:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 --2- 172.21.15.46:0/1094626363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb84101050 0x7fcb8410dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:16.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 --2- 172.21.15.46:0/1094626363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb841006a0 0x7fcb84100a80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 -- 172.21.15.46:0/1094626363 >> 172.21.15.46:0/1094626363 conn(0x7fcb840fc550 msgr2=0x7fcb840fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 -- 172.21.15.46:0/1094626363 shutdown_connections 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.155+0000 7fcb88b28640 1 -- 172.21.15.46:0/1094626363 wait complete. 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb88b28640 1 Processor -- start 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb88b28640 1 -- start start 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb88b28640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb841006a0 0x7fcb8419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb88b28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb84101050 0x7fcb8419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb88b28640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84110f10 con 0x7fcb84101050 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb88b28640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcb84110d90 con 0x7fcb841006a0 2024-09-10T04:09:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb81d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb84101050 0x7fcb8419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb81d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb84101050 0x7fcb8419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34134/0 (socket says 172.21.15.46:34134) 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb81d9b640 1 -- 172.21.15.46:0/400681279 learned_addr learned my addr 172.21.15.46:0/400681279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb81d9b640 1 -- 172.21.15.46:0/400681279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb841006a0 msgr2=0x7fcb8419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb81d9b640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb841006a0 0x7fcb8419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.156+0000 7fcb81d9b640 1 -- 172.21.15.46:0/400681279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb841999a0 con 0x7fcb84101050 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.157+0000 7fcb81d9b640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb84101050 0x7fcb8419fd20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c810 tx=0x7fcb7800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.157+0000 7fcb737fe640 1 -- 172.21.15.46:0/400681279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcb78004020 con 0x7fcb84101050 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.157+0000 7fcb737fe640 1 -- 172.21.15.46:0/400681279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcb780041c0 con 0x7fcb84101050 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.157+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb84199c90 con 0x7fcb84101050 2024-09-10T04:09:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.157+0000 7fcb737fe640 1 -- 172.21.15.46:0/400681279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcb780047a0 con 0x7fcb84101050 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.157+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8419a220 con 0x7fcb84101050 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.158+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8406a520 con 0x7fcb84101050 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.158+0000 7fcb737fe640 1 -- 172.21.15.46:0/400681279 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcb780079b0 con 0x7fcb84101050 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.159+0000 7fcb737fe640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcb6c07ac50 0x7fcb6c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.159+0000 7fcb8259c640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcb6c07ac50 0x7fcb6c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.159+0000 7fcb737fe640 1 -- 172.21.15.46:0/400681279 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fcb7809da50 con 0x7fcb84101050 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.159+0000 7fcb8259c640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcb6c07ac50 0x7fcb6c07d110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb74006010 tx=0x7fcb7402f800 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.163+0000 7fcb737fe640 1 -- 172.21.15.46:0/400681279 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fcb78014030 con 0x7fcb84101050 2024-09-10T04:09:16.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:16 smithi046 ceph-mon[28328]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:16.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.346+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) -- 0x7fcb8419aed0 con 0x7fcb84101050 2024-09-10T04:09:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.350+0000 7fcb737fe640 1 -- 172.21.15.46:0/400681279 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v64)=0 v64) ==== 125+0+0 (secure 0 0 0) 0x7fcb780692c0 con 0x7fcb84101050 2024-09-10T04:09:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcb6c07ac50 msgr2=0x7fcb6c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcb6c07ac50 0x7fcb6c07d110 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcb74006010 tx=0x7fcb7402f800 comp rx=0 tx=0).stop 2024-09-10T04:09:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb84101050 msgr2=0x7fcb8419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb84101050 0x7fcb8419fd20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c810 tx=0x7fcb7800ccd0 comp rx=0 tx=0).stop 2024-09-10T04:09:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 shutdown_connections 2024-09-10T04:09:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcb6c07ac50 0x7fcb6c07d110 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:16.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb84101050 0x7fcb8419fd20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 --2- 172.21.15.46:0/400681279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcb841006a0 0x7fcb8419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 >> 172.21.15.46:0/400681279 conn(0x7fcb840fc550 msgr2=0x7fcb840fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 shutdown_connections 2024-09-10T04:09:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:16.352+0000 7fcb88b28640 1 -- 172.21.15.46:0/400681279 wait complete. 2024-09-10T04:09:16.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:16 smithi070 ceph-mon[35495]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:17.040 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T04:09:17.041 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:09:17.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-10T04:09:17.298 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: pgmap v75: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/400681279' entity='client.admin' 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:09:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:17 smithi046 ceph-mon[28328]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: pgmap v75: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/400681279' entity='client.admin' 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:09:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:17 smithi070 ceph-mon[35495]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2914589091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f0102ac0 msgr2=0x7f80f0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2914589091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f0102ac0 0x7f80f0102ea0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009c20 tx=0x7f80e002aa00 comp rx=0 tx=0).stop 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2914589091 shutdown_connections 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2914589091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80f01033e0 0x7f80f010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2914589091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f0102ac0 0x7f80f0102ea0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2914589091 >> 172.21.15.46:0/2914589091 conn(0x7f80f00fc520 msgr2=0x7f80f00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2914589091 shutdown_connections 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.842+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2914589091 wait complete. 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f7ad8640 1 Processor -- start 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f7ad8640 1 -- start start 2024-09-10T04:09:17.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f7ad8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80f0102ac0 0x7f80f019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f7ad8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f01033e0 0x7f80f019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f7ad8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f80f010edf0 con 0x7f80f01033e0 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f7ad8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f80f010ec70 con 0x7f80f0102ac0 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f5073640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f01033e0 0x7f80f019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f5073640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f01033e0 0x7f80f019dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34154/0 (socket says 172.21.15.46:34154) 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f5073640 1 -- 172.21.15.46:0/2022022469 learned_addr learned my addr 172.21.15.46:0/2022022469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.843+0000 7f80f5874640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80f0102ac0 0x7f80f019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80f5073640 1 -- 172.21.15.46:0/2022022469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80f0102ac0 msgr2=0x7f80f019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80f5073640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80f0102ac0 0x7f80f019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80f5073640 1 -- 172.21.15.46:0/2022022469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f0197820 con 0x7f80f01033e0 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80f5073640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f01033e0 0x7f80f019dbe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f80e400d6d0 tx=0x7f80e400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80eeffd640 1 -- 172.21.15.46:0/2022022469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f80e4017070 con 0x7f80f01033e0 2024-09-10T04:09:17.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80eeffd640 1 -- 172.21.15.46:0/2022022469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80e40054e0 con 0x7f80f01033e0 2024-09-10T04:09:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f0197b10 con 0x7f80f01033e0 2024-09-10T04:09:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80eeffd640 1 -- 172.21.15.46:0/2022022469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f80e4004ae0 con 0x7f80f01033e0 2024-09-10T04:09:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.844+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f01980a0 con 0x7f80f01033e0 2024-09-10T04:09:17.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.845+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f006aa90 con 0x7f80f01033e0 2024-09-10T04:09:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.846+0000 7f80eeffd640 1 -- 172.21.15.46:0/2022022469 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f80e4004de0 con 0x7f80f01033e0 2024-09-10T04:09:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.846+0000 7f80eeffd640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f80d807ac00 0x7f80d807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.846+0000 7f80eeffd640 1 -- 172.21.15.46:0/2022022469 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f80e409d710 con 0x7f80f01033e0 2024-09-10T04:09:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.846+0000 7f80f5874640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f80d807ac00 0x7f80d807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.847+0000 7f80f5874640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f80d807ac00 0x7f80d807d0c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009430 tx=0x7f80e003a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:17.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:17.851+0000 7f80eeffd640 1 -- 172.21.15.46:0/2022022469 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f80e4068f00 con 0x7f80f01033e0 2024-09-10T04:09:18.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.035+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) -- 0x7f80f0105f90 con 0x7f80f01033e0 2024-09-10T04:09:18.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.036+0000 7f80eeffd640 1 -- 172.21.15.46:0/2022022469 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v64)=0 v64) ==== 155+0+0 (secure 0 0 0) 0x7f80e406ddb0 con 0x7f80f01033e0 2024-09-10T04:09:18.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f80d807ac00 msgr2=0x7f80d807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:18.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f80d807ac00 0x7f80d807d0c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009430 tx=0x7f80e003a040 comp rx=0 tx=0).stop 2024-09-10T04:09:18.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f01033e0 msgr2=0x7f80f019dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f01033e0 0x7f80f019dbe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f80e400d6d0 tx=0x7f80e400db90 comp rx=0 tx=0).stop 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 shutdown_connections 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f80d807ac00 0x7f80d807d0c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80f01033e0 0x7f80f019dbe0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 --2- 172.21.15.46:0/2022022469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80f0102ac0 0x7f80f019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 >> 172.21.15.46:0/2022022469 conn(0x7f80f00fc520 msgr2=0x7f80f0108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 shutdown_connections 2024-09-10T04:09:18.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:18.038+0000 7f80f7ad8640 1 -- 172.21.15.46:0/2022022469 wait complete. 2024-09-10T04:09:18.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-10T04:09:18.933 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/405950133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4106800 msgr2=0x7fcdd4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/405950133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4106800 0x7fcdd4106be0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009880 tx=0x7fcdc402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/405950133 shutdown_connections 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/405950133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdd4100800 0x7fcdd4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/405950133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4106800 0x7fcdd4106be0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/405950133 >> 172.21.15.46:0/405950133 conn(0x7fcdd40fc520 msgr2=0x7fcdd40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/405950133 shutdown_connections 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.399+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/405950133 wait complete. 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.400+0000 7fcdd8dbf640 1 Processor -- start 2024-09-10T04:09:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.400+0000 7fcdd8dbf640 1 -- start start 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.400+0000 7fcdd8dbf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4100800 0x7fcdd419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.400+0000 7fcdd8dbf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdd4106800 0x7fcdd419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.400+0000 7fcdd8dbf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410dcd0 con 0x7fcdd4100800 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.400+0000 7fcdd8dbf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd410db50 con 0x7fcdd4106800 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.400+0000 7fcdd259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4100800 0x7fcdd419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4100800 0x7fcdd419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41580/0 (socket says 172.21.15.46:41580) 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd259c640 1 -- 172.21.15.46:0/3760983332 learned_addr learned my addr 172.21.15.46:0/3760983332 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd1d9b640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdd4106800 0x7fcdd419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd259c640 1 -- 172.21.15.46:0/3760983332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdd4106800 msgr2=0x7fcdd419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd259c640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdd4106800 0x7fcdd419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd259c640 1 -- 172.21.15.46:0/3760983332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd4196720 con 0x7fcdd4100800 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd259c640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4100800 0x7fcdd419c650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4031460 tx=0x7fcdc4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdc37fe640 1 -- 172.21.15.46:0/3760983332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcdc403f070 con 0x7fcdd4100800 2024-09-10T04:09:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd41969b0 con 0x7fcdd4100800 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdc37fe640 1 -- 172.21.15.46:0/3760983332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcdc40035f0 con 0x7fcdd4100800 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdc37fe640 1 -- 172.21.15.46:0/3760983332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcdc4035030 con 0x7fcdd4100800 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.401+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd4196eb0 con 0x7fcdd4100800 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.402+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd4101f40 con 0x7fcdd4100800 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.403+0000 7fcdc37fe640 1 -- 172.21.15.46:0/3760983332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcdc4042070 con 0x7fcdd4100800 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.404+0000 7fcdc37fe640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdbc07abf0 0x7fcdbc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.404+0000 7fcdc37fe640 1 -- 172.21.15.46:0/3760983332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fcdc40be5a0 con 0x7fcdd4100800 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.404+0000 7fcdd1d9b640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdbc07abf0 0x7fcdbc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.404+0000 7fcdd1d9b640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdbc07abf0 0x7fcdbc07d0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4197f70 tx=0x7fcdc8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:19.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.408+0000 7fcdc37fe640 1 -- 172.21.15.46:0/3760983332 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fcdc4089cf0 con 0x7fcdd4100800 2024-09-10T04:09:19.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:19 smithi046 ceph-mon[28328]: pgmap v76: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.593+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) -- 0x7fcdd4108d70 con 0x7fcdd4100800 2024-09-10T04:09:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.593+0000 7fcdc37fe640 1 -- 172.21.15.46:0/3760983332 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v64)=0 v64) ==== 163+0+0 (secure 0 0 0) 0x7fcdc408eba0 con 0x7fcdd4100800 2024-09-10T04:09:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdbc07abf0 msgr2=0x7fcdbc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdbc07abf0 0x7fcdbc07d0b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4197f70 tx=0x7fcdc8005f90 comp rx=0 tx=0).stop 2024-09-10T04:09:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4100800 msgr2=0x7fcdd419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4100800 0x7fcdd419c650 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4031460 tx=0x7fcdc4004530 comp rx=0 tx=0).stop 2024-09-10T04:09:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 shutdown_connections 2024-09-10T04:09:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdbc07abf0 0x7fcdbc07d0b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdd4106800 0x7fcdd419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 --2- 172.21.15.46:0/3760983332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdd4100800 0x7fcdd419c650 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 >> 172.21.15.46:0/3760983332 conn(0x7fcdd40fc520 msgr2=0x7fcdd40fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 shutdown_connections 2024-09-10T04:09:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:19.595+0000 7fcdd8dbf640 1 -- 172.21.15.46:0/3760983332 wait complete. 2024-09-10T04:09:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:19 smithi070 ceph-mon[35495]: pgmap v76: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:20.329 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-10T04:09:20.590 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 -- 172.21.15.46:0/1571799512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08103430 msgr2=0x7f2c0810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 --2- 172.21.15.46:0/1571799512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08103430 0x7f2c0810b920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0098b0 tx=0x7f2bfc02a560 comp rx=0 tx=0).stop 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 -- 172.21.15.46:0/1571799512 shutdown_connections 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 --2- 172.21.15.46:0/1571799512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08103430 0x7f2c0810b920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 --2- 172.21.15.46:0/1571799512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c08102b10 0x7f2c08102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 -- 172.21.15.46:0/1571799512 >> 172.21.15.46:0/1571799512 conn(0x7f2c080fc550 msgr2=0x7f2c080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 -- 172.21.15.46:0/1571799512 shutdown_connections 2024-09-10T04:09:21.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 -- 172.21.15.46:0/1571799512 wait complete. 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.066+0000 7f2c0db86640 1 Processor -- start 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c0db86640 1 -- start start 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c0db86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08102b10 0x7f2c0819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c0db86640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c08103430 0x7f2c0819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c0db86640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0810ee40 con 0x7f2c08102b10 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c0db86640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0810ecc0 con 0x7f2c08103430 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08102b10 0x7f2c0819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c077fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08102b10 0x7f2c0819f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41594/0 (socket says 172.21.15.46:41594) 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c077fe640 1 -- 172.21.15.46:0/3057887938 learned_addr learned my addr 172.21.15.46:0/3057887938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c06ffd640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c08103430 0x7f2c0819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c077fe640 1 -- 172.21.15.46:0/3057887938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c08103430 msgr2=0x7f2c0819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c077fe640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c08103430 0x7f2c0819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c077fe640 1 -- 172.21.15.46:0/3057887938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c08199aa0 con 0x7f2c08102b10 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c06ffd640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c08103430 0x7f2c0819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.067+0000 7f2c077fe640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08102b10 0x7f2c0819f8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b420 tx=0x7f2bf800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.068+0000 7f2c04ff9640 1 -- 172.21.15.46:0/3057887938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bf8011630 con 0x7f2c08102b10 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.068+0000 7f2c04ff9640 1 -- 172.21.15.46:0/3057887938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bf8016070 con 0x7f2c08102b10 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.068+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c08199d90 con 0x7f2c08102b10 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.068+0000 7f2c04ff9640 1 -- 172.21.15.46:0/3057887938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bf8010d40 con 0x7f2c08102b10 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.068+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0819a320 con 0x7f2c08102b10 2024-09-10T04:09:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.069+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0806a520 con 0x7f2c08102b10 2024-09-10T04:09:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.070+0000 7f2c04ff9640 1 -- 172.21.15.46:0/3057887938 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2bf801b460 con 0x7f2c08102b10 2024-09-10T04:09:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.070+0000 7f2c04ff9640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bf007ac00 0x7f2bf007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.070+0000 7f2c04ff9640 1 -- 172.21.15.46:0/3057887938 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f2bf809d310 con 0x7f2c08102b10 2024-09-10T04:09:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.070+0000 7f2c06ffd640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bf007ac00 0x7f2bf007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:21.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.071+0000 7f2c06ffd640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bf007ac00 0x7f2bf007d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000c00 tx=0x7f2bfc004260 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:21.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.075+0000 7f2c04ff9640 1 -- 172.21.15.46:0/3057887938 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f2bf8021250 con 0x7f2c08102b10 2024-09-10T04:09:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.259+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) -- 0x7f2c0819afd0 con 0x7f2c08102b10 2024-09-10T04:09:21.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.259+0000 7f2c04ff9640 1 -- 172.21.15.46:0/3057887938 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v64)=0 v64) ==== 135+0+0 (secure 0 0 0) 0x7f2bf8068b00 con 0x7f2c08102b10 2024-09-10T04:09:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bf007ac00 msgr2=0x7f2bf007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bf007ac00 0x7f2bf007d0c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc000c00 tx=0x7f2bfc004260 comp rx=0 tx=0).stop 2024-09-10T04:09:21.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08102b10 msgr2=0x7f2c0819f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08102b10 0x7f2c0819f8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2bf800b420 tx=0x7f2bf800b8e0 comp rx=0 tx=0).stop 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 shutdown_connections 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bf007ac00 0x7f2bf007d0c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c08103430 0x7f2c0819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 --2- 172.21.15.46:0/3057887938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c08102b10 0x7f2c0819f8e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.261+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 >> 172.21.15.46:0/3057887938 conn(0x7f2c080fc550 msgr2=0x7f2c08100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.262+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 shutdown_connections 2024-09-10T04:09:21.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:21.262+0000 7f2c0db86640 1 -- 172.21.15.46:0/3057887938 wait complete. 2024-09-10T04:09:21.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:21 smithi046 ceph-mon[28328]: pgmap v77: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:21 smithi070 ceph-mon[35495]: pgmap v77: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:21.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-10T04:09:22.155 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 -- 172.21.15.46:0/2183677132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1007f0 msgr2=0x7f5e5c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2183677132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1007f0 0x7f5e5c100c50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009880 tx=0x7f5e4402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 -- 172.21.15.46:0/2183677132 shutdown_connections 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2183677132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1007f0 0x7f5e5c100c50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2183677132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e5c1067f0 0x7f5e5c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 -- 172.21.15.46:0/2183677132 >> 172.21.15.46:0/2183677132 conn(0x7f5e5c0fc550 msgr2=0x7f5e5c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 -- 172.21.15.46:0/2183677132 shutdown_connections 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 -- 172.21.15.46:0/2183677132 wait complete. 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.613+0000 7f5e60e91640 1 Processor -- start 2024-09-10T04:09:22.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e60e91640 1 -- start start 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e60e91640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e5c1007f0 0x7f5e5c0fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e60e91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1067f0 0x7f5e5c0fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e60e91640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c10dcc0 con 0x7f5e5c1067f0 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e60e91640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c10db40 con 0x7f5e5c1007f0 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e53fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1067f0 0x7f5e5c0fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e53fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1067f0 0x7f5e5c0fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41614/0 (socket says 172.21.15.46:41614) 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e53fff640 1 -- 172.21.15.46:0/2874906972 learned_addr learned my addr 172.21.15.46:0/2874906972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e5a59c640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e5c1007f0 0x7f5e5c0fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e53fff640 1 -- 172.21.15.46:0/2874906972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e5c1007f0 msgr2=0x7f5e5c0fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e53fff640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e5c1007f0 0x7f5e5c0fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e53fff640 1 -- 172.21.15.46:0/2874906972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c0fd750 con 0x7f5e5c1067f0 2024-09-10T04:09:22.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.614+0000 7f5e53fff640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1067f0 0x7f5e5c0fd210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e44031460 tx=0x7f5e440045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.615+0000 7f5e537fe640 1 -- 172.21.15.46:0/2874906972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e4403f070 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.615+0000 7f5e537fe640 1 -- 172.21.15.46:0/2874906972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e440035f0 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.615+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5c0fd9e0 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.615+0000 7f5e537fe640 1 -- 172.21.15.46:0/2874906972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e44035030 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.615+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5c1a3da0 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.616+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5c101f30 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.616+0000 7f5e537fe640 1 -- 172.21.15.46:0/2874906972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5e44042070 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.617+0000 7f5e537fe640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e4807abf0 0x7f5e4807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.617+0000 7f5e537fe640 1 -- 172.21.15.46:0/2874906972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f5e440be520 con 0x7f5e5c1067f0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.617+0000 7f5e5a59c640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e4807abf0 0x7f5e4807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:22.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.618+0000 7f5e5a59c640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e4807abf0 0x7f5e4807d0b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005fd0 tx=0x7f5e4c00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:22.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.622+0000 7f5e537fe640 1 -- 172.21.15.46:0/2874906972 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f5e44089d10 con 0x7f5e5c1067f0 2024-09-10T04:09:22.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.826+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "target": ["mon-mgr", ""]}) -- 0x7f5e5c062630 con 0x7f5e4807abf0 2024-09-10T04:09:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.832+0000 7f5e537fe640 1 -- 172.21.15.46:0/2874906972 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+89 (secure 0 0 0) 0x7f5e5c062630 con 0x7f5e4807abf0 2024-09-10T04:09:22.834 INFO:teuthology.orchestra.run.smithi046.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e4807abf0 msgr2=0x7f5e4807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e4807abf0 0x7f5e4807d0b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005fd0 tx=0x7f5e4c00b390 comp rx=0 tx=0).stop 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1067f0 msgr2=0x7f5e5c0fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1067f0 0x7f5e5c0fd210 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e44031460 tx=0x7f5e440045e0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 shutdown_connections 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e4807abf0 0x7f5e4807d0b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e5c1067f0 0x7f5e5c0fd210 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 --2- 172.21.15.46:0/2874906972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e5c1007f0 0x7f5e5c0fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 >> 172.21.15.46:0/2874906972 conn(0x7f5e5c0fc550 msgr2=0x7f5e5c069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 shutdown_connections 2024-09-10T04:09:22.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:22.835+0000 7f5e60e91640 1 -- 172.21.15.46:0/2874906972 wait complete. 2024-09-10T04:09:23.562 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T04:09:23.562 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:09:23.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-10T04:09:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:23 smithi070 ceph-mon[35495]: pgmap v78: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:23 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:23 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:23 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:23.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:23 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:09:23.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:23 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:23.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:23 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:09:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:23 smithi046 ceph-mon[28328]: pgmap v78: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:23 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:23.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:23 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:23.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:23 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:23.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:23 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:09:23.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:23 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:23.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:23 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:09:23.844 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 -- 172.21.15.46:0/1304990156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41007f0 msgr2=0x7f98b4100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 --2- 172.21.15.46:0/1304990156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41007f0 0x7f98b4100c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009880 tx=0x7f98a402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 -- 172.21.15.46:0/1304990156 shutdown_connections 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 --2- 172.21.15.46:0/1304990156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41007f0 0x7f98b4100c50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 --2- 172.21.15.46:0/1304990156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b41067f0 0x7f98b4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 -- 172.21.15.46:0/1304990156 >> 172.21.15.46:0/1304990156 conn(0x7f98b40fc550 msgr2=0x7f98b40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 -- 172.21.15.46:0/1304990156 shutdown_connections 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.506+0000 7f98bbd01640 1 -- 172.21.15.46:0/1304990156 wait complete. 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98bbd01640 1 Processor -- start 2024-09-10T04:09:24.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98bbd01640 1 -- start start 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98bbd01640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b41007f0 0x7f98b419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98bbd01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41067f0 0x7f98b419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98bbd01640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410dcc0 con 0x7f98b41067f0 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98bbd01640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f98b410db40 con 0x7f98b41007f0 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98b929c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41067f0 0x7f98b419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98b929c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41067f0 0x7f98b419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41630/0 (socket says 172.21.15.46:41630) 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98b929c640 1 -- 172.21.15.46:0/3666296112 learned_addr learned my addr 172.21.15.46:0/3666296112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98b9a9d640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b41007f0 0x7f98b419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98b929c640 1 -- 172.21.15.46:0/3666296112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b41007f0 msgr2=0x7f98b419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98b929c640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b41007f0 0x7f98b419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.507+0000 7f98b929c640 1 -- 172.21.15.46:0/3666296112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b41999b0 con 0x7f98b41067f0 2024-09-10T04:09:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.508+0000 7f98b929c640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41067f0 0x7f98b419fd30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f98a4031460 tx=0x7f98a40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.508+0000 7f98aaffd640 1 -- 172.21.15.46:0/3666296112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98a403f070 con 0x7f98b41067f0 2024-09-10T04:09:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.508+0000 7f98aaffd640 1 -- 172.21.15.46:0/3666296112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98a40035f0 con 0x7f98b41067f0 2024-09-10T04:09:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.508+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b4199c40 con 0x7f98b41067f0 2024-09-10T04:09:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.508+0000 7f98aaffd640 1 -- 172.21.15.46:0/3666296112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f98a4035030 con 0x7f98b41067f0 2024-09-10T04:09:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.508+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b419a170 con 0x7f98b41067f0 2024-09-10T04:09:24.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.509+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b4101f30 con 0x7f98b41067f0 2024-09-10T04:09:24.513 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:24 smithi046 ceph-mon[28328]: from='client.14912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:24.513 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:24 smithi046 ceph-mon[28328]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:09:24.513 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:24 smithi046 ceph-mon[28328]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:09:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.510+0000 7f98aaffd640 1 -- 172.21.15.46:0/3666296112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f98a4042070 con 0x7f98b41067f0 2024-09-10T04:09:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.510+0000 7f98aaffd640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f98a007abf0 0x7f98a007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.511+0000 7f98aaffd640 1 -- 172.21.15.46:0/3666296112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f98a40be5d0 con 0x7f98b41067f0 2024-09-10T04:09:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.511+0000 7f98b9a9d640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f98a007abf0 0x7f98a007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.511+0000 7f98b9a9d640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f98a007abf0 0x7f98a007d0b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f98b0005fd0 tx=0x7f98b0005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:24.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.516+0000 7f98aaffd640 1 -- 172.21.15.46:0/3666296112 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f98a4089dc0 con 0x7f98b41067f0 2024-09-10T04:09:24.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.722+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f98b419a420 con 0x7f98a007abf0 2024-09-10T04:09:24.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.724+0000 7f98aaffd640 1 -- 172.21.15.46:0/3666296112 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+358 (secure 0 0 0) 0x7f98b419a420 con 0x7f98a007abf0 2024-09-10T04:09:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.727+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f98a007abf0 msgr2=0x7f98a007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.727+0000 7f98bbd01640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f98a007abf0 0x7f98a007d0b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f98b0005fd0 tx=0x7f98b0005850 comp rx=0 tx=0).stop 2024-09-10T04:09:24.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.727+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41067f0 msgr2=0x7f98b419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.727+0000 7f98bbd01640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41067f0 0x7f98b419fd30 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f98a4031460 tx=0x7f98a40045e0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.728+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 shutdown_connections 2024-09-10T04:09:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.728+0000 7f98bbd01640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f98a007abf0 0x7f98a007d0b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.728+0000 7f98bbd01640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98b41067f0 0x7f98b419fd30 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.728+0000 7f98bbd01640 1 --2- 172.21.15.46:0/3666296112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98b41007f0 0x7f98b419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.728+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 >> 172.21.15.46:0/3666296112 conn(0x7f98b40fc550 msgr2=0x7f98b4069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:24.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.728+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 shutdown_connections 2024-09-10T04:09:24.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.728+0000 7f98bbd01640 1 -- 172.21.15.46:0/3666296112 wait complete. 2024-09-10T04:09:24.741 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:09:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.820+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1783140790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4106800 msgr2=0x7fd0e4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.820+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1783140790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4106800 0x7fd0e4106be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009880 tx=0x7fd0d402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1783140790 shutdown_connections 2024-09-10T04:09:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1783140790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4100800 0x7fd0e4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1783140790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4106800 0x7fd0e4106be0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1783140790 >> 172.21.15.46:0/1783140790 conn(0x7fd0e40fc520 msgr2=0x7fd0e40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:24.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1783140790 shutdown_connections 2024-09-10T04:09:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1783140790 wait complete. 2024-09-10T04:09:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 Processor -- start 2024-09-10T04:09:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.821+0000 7fd0eab3f640 1 -- start start 2024-09-10T04:09:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0eab3f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4100800 0x7fd0e419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0eab3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4106800 0x7fd0e419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0eab3f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e410dcb0 con 0x7fd0e4100800 2024-09-10T04:09:24.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0eab3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e410db30 con 0x7fd0e4106800 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e88db640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4100800 0x7fd0e419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e88db640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4100800 0x7fd0e419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41650/0 (socket says 172.21.15.46:41650) 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e88db640 1 -- 172.21.15.46:0/1221019641 learned_addr learned my addr 172.21.15.46:0/1221019641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e3fff640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4106800 0x7fd0e419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e88db640 1 -- 172.21.15.46:0/1221019641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4106800 msgr2=0x7fd0e419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e88db640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4106800 0x7fd0e419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e88db640 1 -- 172.21.15.46:0/1221019641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e41977b0 con 0x7fd0e4100800 2024-09-10T04:09:24.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e88db640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4100800 0x7fd0e419d630 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4031460 tx=0x7fd0d40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.822+0000 7fd0e1ffb640 1 -- 172.21.15.46:0/1221019641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd0d403f070 con 0x7fd0e4100800 2024-09-10T04:09:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.823+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e4197a40 con 0x7fd0e4100800 2024-09-10T04:09:24.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.823+0000 7fd0e1ffb640 1 -- 172.21.15.46:0/1221019641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0d40035f0 con 0x7fd0e4100800 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.823+0000 7fd0e1ffb640 1 -- 172.21.15.46:0/1221019641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd0d4035030 con 0x7fd0e4100800 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.823+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e4197f70 con 0x7fd0e4100800 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.824+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e4101f40 con 0x7fd0e4100800 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.825+0000 7fd0e1ffb640 1 -- 172.21.15.46:0/1221019641 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd0d4042070 con 0x7fd0e4100800 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.825+0000 7fd0e1ffb640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd0cc07abf0 0x7fd0cc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.825+0000 7fd0e1ffb640 1 -- 172.21.15.46:0/1221019641 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fd0d40be520 con 0x7fd0e4100800 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.826+0000 7fd0e3fff640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd0cc07abf0 0x7fd0cc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:24.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.826+0000 7fd0e3fff640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd0cc07abf0 0x7fd0cc07d0b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0e406b200 tx=0x7fd0d8009760 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:24.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:24.830+0000 7fd0e1ffb640 1 -- 172.21.15.46:0/1221019641 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fd0d4089d10 con 0x7fd0e4100800 2024-09-10T04:09:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:24 smithi070 ceph-mon[35495]: from='client.14912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:24 smithi070 ceph-mon[35495]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:09:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:24 smithi070 ceph-mon[35495]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T04:09:25.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.039+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd0e41982e0 con 0x7fd0cc07abf0 2024-09-10T04:09:25.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.041+0000 7fd0e1ffb640 1 -- 172.21.15.46:0/1221019641 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+358 (secure 0 0 0) 0x7fd0e41982e0 con 0x7fd0cc07abf0 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.043+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd0cc07abf0 msgr2=0x7fd0cc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.043+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd0cc07abf0 0x7fd0cc07d0b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0e406b200 tx=0x7fd0d8009760 comp rx=0 tx=0).stop 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.043+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4100800 msgr2=0x7fd0e419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.043+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4100800 0x7fd0e419d630 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4031460 tx=0x7fd0d40045e0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.044+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 shutdown_connections 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.044+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd0cc07abf0 0x7fd0cc07d0b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.044+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0e4106800 0x7fd0e419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.044+0000 7fd0eab3f640 1 --2- 172.21.15.46:0/1221019641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0e4100800 0x7fd0e419d630 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.044+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 >> 172.21.15.46:0/1221019641 conn(0x7fd0e40fc520 msgr2=0x7fd0e4109e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.044+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 shutdown_connections 2024-09-10T04:09:25.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.044+0000 7fd0eab3f640 1 -- 172.21.15.46:0/1221019641 wait complete. 2024-09-10T04:09:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.129+0000 7f1765b37640 1 -- 172.21.15.46:0/1902394709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760109b90 msgr2=0x7f1760109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.129+0000 7f1765b37640 1 --2- 172.21.15.46:0/1902394709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760109b90 0x7f1760109f70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1750009880 tx=0x7f175002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 -- 172.21.15.46:0/1902394709 shutdown_connections 2024-09-10T04:09:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 --2- 172.21.15.46:0/1902394709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1760103b00 0x7f1760103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 --2- 172.21.15.46:0/1902394709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760109b90 0x7f1760109f70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 -- 172.21.15.46:0/1902394709 >> 172.21.15.46:0/1902394709 conn(0x7f17600ff7a0 msgr2=0x7f1760101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 -- 172.21.15.46:0/1902394709 shutdown_connections 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 -- 172.21.15.46:0/1902394709 wait complete. 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 Processor -- start 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.130+0000 7f1765b37640 1 -- start start 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f1765b37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760103b00 0x7f176019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f1765b37640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1760109b90 0x7f176019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f1765b37640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1760111030 con 0x7f1760103b00 2024-09-10T04:09:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f1765b37640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1760110eb0 con 0x7f1760109b90 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760103b00 0x7f176019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760103b00 0x7f176019f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41672/0 (socket says 172.21.15.46:41672) 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175f7fe640 1 -- 172.21.15.46:0/2111311150 learned_addr learned my addr 172.21.15.46:0/2111311150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175f7fe640 1 -- 172.21.15.46:0/2111311150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1760109b90 msgr2=0x7f176019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f1756ffd640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1760109b90 0x7f176019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175f7fe640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1760109b90 0x7f176019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175f7fe640 1 -- 172.21.15.46:0/2111311150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1760199a00 con 0x7f1760103b00 2024-09-10T04:09:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175f7fe640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760103b00 0x7f176019f840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1750031460 tx=0x7f17500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.131+0000 7f175d7fa640 1 -- 172.21.15.46:0/2111311150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f175003f070 con 0x7f1760103b00 2024-09-10T04:09:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.132+0000 7f175d7fa640 1 -- 172.21.15.46:0/2111311150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17500044d0 con 0x7f1760103b00 2024-09-10T04:09:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.132+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1760199c90 con 0x7f1760103b00 2024-09-10T04:09:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.132+0000 7f175d7fa640 1 -- 172.21.15.46:0/2111311150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1750035030 con 0x7f1760103b00 2024-09-10T04:09:25.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.132+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176019a1c0 con 0x7f1760103b00 2024-09-10T04:09:25.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.133+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1760105200 con 0x7f1760103b00 2024-09-10T04:09:25.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.133+0000 7f175d7fa640 1 -- 172.21.15.46:0/2111311150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1750042070 con 0x7f1760103b00 2024-09-10T04:09:25.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.134+0000 7f175d7fa640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f174c07ac50 0x7f174c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.134+0000 7f175d7fa640 1 -- 172.21.15.46:0/2111311150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f17500be6a0 con 0x7f1760103b00 2024-09-10T04:09:25.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.134+0000 7f1756ffd640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f174c07ac50 0x7f174c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.135+0000 7f1756ffd640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f174c07ac50 0x7f174c07d110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1760101d60 tx=0x7f1748008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:25.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.138+0000 7f175d7fa640 1 -- 172.21.15.46:0/2111311150 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f1750004cf0 con 0x7f1760103b00 2024-09-10T04:09:25.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.334+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f176019af30 con 0x7f174c07ac50 2024-09-10T04:09:25.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.352+0000 7f175d7fa640 1 -- 172.21.15.46:0/2111311150 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f176019af30 con 0x7f174c07ac50 2024-09-10T04:09:25.354 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:09:25.354 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (60s) 30s ago 12m 15.6M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:09:25.354 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (3m) 30s ago 12m 8770k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (3m) 99s ago 11m 8350k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (12m) 30s ago 12m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (11m) 99s ago 11m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (35s) 30s ago 12m 44.5M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (6m) 30s ago 6m 90.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (7m) 30s ago 7m 147M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (6m) 99s ago 6m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (6m) 99s ago 6m 25.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (3m) 30s ago 13m 526M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (2m) 99s ago 11m 453M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (13m) 30s ago 13m 59.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9933ea41a29e 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (11m) 99s ago 11m 43.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 b5145a46761a 2024-09-10T04:09:25.355 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (109s) 30s ago 12m 10.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (103s) 99s ago 11m 3213k - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (10m) 30s ago 10m 212M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (9m) 30s ago 9m 225M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (9m) 30s ago 9m 179M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (8m) 99s ago 8m 234M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (8m) 99s ago 8m 209M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (8m) 99s ago 8m 194M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:09:25.356 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (80s) 30s ago 11m 44.9M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.354+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f174c07ac50 msgr2=0x7f174c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.354+0000 7f1765b37640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f174c07ac50 0x7f174c07d110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1760101d60 tx=0x7f1748008040 comp rx=0 tx=0).stop 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.354+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760103b00 msgr2=0x7f176019f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.354+0000 7f1765b37640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760103b00 0x7f176019f840 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1750031460 tx=0x7f17500048f0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.355+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 shutdown_connections 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.355+0000 7f1765b37640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f174c07ac50 0x7f174c07d110 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.355+0000 7f1765b37640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1760109b90 0x7f176019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.355+0000 7f1765b37640 1 --2- 172.21.15.46:0/2111311150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1760103b00 0x7f176019f840 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.355+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 >> 172.21.15.46:0/2111311150 conn(0x7f17600ff7a0 msgr2=0x7f1760100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:25.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.355+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 shutdown_connections 2024-09-10T04:09:25.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.355+0000 7f1765b37640 1 -- 172.21.15.46:0/2111311150 wait complete. 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 -- 172.21.15.46:0/2197083461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570103490 msgr2=0x7fa57010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 --2- 172.21.15.46:0/2197083461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570103490 0x7fa57010b980 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa564009880 tx=0x7fa56402a530 comp rx=0 tx=0).stop 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 -- 172.21.15.46:0/2197083461 shutdown_connections 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 --2- 172.21.15.46:0/2197083461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570103490 0x7fa57010b980 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 --2- 172.21.15.46:0/2197083461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa570102b70 0x7fa570102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 -- 172.21.15.46:0/2197083461 >> 172.21.15.46:0/2197083461 conn(0x7fa5700fc550 msgr2=0x7fa5700fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 -- 172.21.15.46:0/2197083461 shutdown_connections 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.439+0000 7fa575f81640 1 -- 172.21.15.46:0/2197083461 wait complete. 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.440+0000 7fa575f81640 1 Processor -- start 2024-09-10T04:09:25.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.440+0000 7fa575f81640 1 -- start start 2024-09-10T04:09:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.440+0000 7fa575f81640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570102b70 0x7fa57019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.440+0000 7fa575f81640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa570103490 0x7fa57019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.440+0000 7fa575f81640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010edc0 con 0x7fa570102b70 2024-09-10T04:09:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.440+0000 7fa575f81640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010ec40 con 0x7fa570103490 2024-09-10T04:09:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.440+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570102b70 0x7fa57019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570102b70 0x7fa57019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41692/0 (socket says 172.21.15.46:41692) 2024-09-10T04:09:25.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56f7fe640 1 -- 172.21.15.46:0/271925504 learned_addr learned my addr 172.21.15.46:0/271925504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56effd640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa570103490 0x7fa57019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56f7fe640 1 -- 172.21.15.46:0/271925504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa570103490 msgr2=0x7fa57019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56f7fe640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa570103490 0x7fa57019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56f7fe640 1 -- 172.21.15.46:0/271925504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570197790 con 0x7fa570102b70 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56f7fe640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570102b70 0x7fa57019d610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c8a0 tx=0x7fa56000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56cff9640 1 -- 172.21.15.46:0/271925504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa5600079b0 con 0x7fa570102b70 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa570197a80 con 0x7fa570102b70 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56cff9640 1 -- 172.21.15.46:0/271925504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa560007b50 con 0x7fa570102b70 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa56cff9640 1 -- 172.21.15.46:0/271925504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa560018700 con 0x7fa570102b70 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.441+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa570197fe0 con 0x7fa570102b70 2024-09-10T04:09:25.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.442+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57006a4c0 con 0x7fa570102b70 2024-09-10T04:09:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.443+0000 7fa56cff9640 1 -- 172.21.15.46:0/271925504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa560007d00 con 0x7fa570102b70 2024-09-10T04:09:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.444+0000 7fa56cff9640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.444+0000 7fa56cff9640 1 -- 172.21.15.46:0/271925504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fa56009de50 con 0x7fa570102b70 2024-09-10T04:09:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.444+0000 7fa56effd640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.444+0000 7fa56effd640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa570198ae0 tx=0x7fa564004f50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:25.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.448+0000 7fa56cff9640 1 -- 172.21.15.46:0/271925504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fa560014030 con 0x7fa570102b70 2024-09-10T04:09:25.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:25 smithi046 ceph-mon[28328]: pgmap v79: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:25.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.709+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fa570198cf0 con 0x7fa570102b70 2024-09-10T04:09:25.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.712+0000 7fa56cff9640 1 -- 172.21.15.46:0/271925504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+735 (secure 0 0 0) 0x7fa570198cf0 con 0x7fa570102b70 2024-09-10T04:09:25.713 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:09:25.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 12, 2024-09-10T04:09:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:09:25.715 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:09:25.715 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:09:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 msgr2=0x7fa55807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa570198ae0 tx=0x7fa564004f50 comp rx=0 tx=0).stop 2024-09-10T04:09:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570102b70 msgr2=0x7fa57019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570102b70 0x7fa57019d610 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa56000c8a0 tx=0x7fa56000cd60 comp rx=0 tx=0).stop 2024-09-10T04:09:25.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 shutdown_connections 2024-09-10T04:09:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa570103490 0x7fa57019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 --2- 172.21.15.46:0/271925504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa570102b70 0x7fa57019d610 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 >> 172.21.15.46:0/271925504 conn(0x7fa5700fc550 msgr2=0x7fa570103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 shutdown_connections 2024-09-10T04:09:25.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.714+0000 7fa575f81640 1 -- 172.21.15.46:0/271925504 wait complete. 2024-09-10T04:09:25.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 -- 172.21.15.46:0/21108905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 msgr2=0x7fda3010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 --2- 172.21.15.46:0/21108905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda3010b8c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fda24009880 tx=0x7fda2402a530 comp rx=0 tx=0).stop 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 -- 172.21.15.46:0/21108905 shutdown_connections 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 --2- 172.21.15.46:0/21108905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda3010b8c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 --2- 172.21.15.46:0/21108905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda30102ab0 0x7fda30102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 -- 172.21.15.46:0/21108905 >> 172.21.15.46:0/21108905 conn(0x7fda300fc550 msgr2=0x7fda300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 -- 172.21.15.46:0/21108905 shutdown_connections 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.800+0000 7fda35f6c640 1 -- 172.21.15.46:0/21108905 wait complete. 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda35f6c640 1 Processor -- start 2024-09-10T04:09:25.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda35f6c640 1 -- start start 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda35f6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda30102ab0 0x7fda30100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda35f6c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda30100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda35f6c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda3010ee40 con 0x7fda301033d0 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda35f6c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fda3010ecc0 con 0x7fda30102ab0 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda2effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda30100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.801+0000 7fda2effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda30100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41708/0 (socket says 172.21.15.46:41708) 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2effd640 1 -- 172.21.15.46:0/446899994 learned_addr learned my addr 172.21.15.46:0/446899994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:25.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2f7fe640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda30102ab0 0x7fda30100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2effd640 1 -- 172.21.15.46:0/446899994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda30102ab0 msgr2=0x7fda30100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2effd640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda30102ab0 0x7fda30100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2effd640 1 -- 172.21.15.46:0/446899994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda30101190 con 0x7fda301033d0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2effd640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda30100c50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fda24031460 tx=0x7fda240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2cff9640 1 -- 172.21.15.46:0/446899994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda2403f070 con 0x7fda301033d0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda30101420 con 0x7fda301033d0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2cff9640 1 -- 172.21.15.46:0/446899994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fda24004ac0 con 0x7fda301033d0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda2cff9640 1 -- 172.21.15.46:0/446899994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda24035030 con 0x7fda301033d0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.802+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda301a3df0 con 0x7fda301033d0 2024-09-10T04:09:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.803+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3006a580 con 0x7fda301033d0 2024-09-10T04:09:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.804+0000 7fda2cff9640 1 -- 172.21.15.46:0/446899994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fda24042070 con 0x7fda301033d0 2024-09-10T04:09:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.805+0000 7fda2cff9640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda1807abf0 0x7fda1807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.805+0000 7fda2cff9640 1 -- 172.21.15.46:0/446899994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fda240be570 con 0x7fda301033d0 2024-09-10T04:09:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.805+0000 7fda2f7fe640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda1807abf0 0x7fda1807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.805+0000 7fda2f7fe640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda1807abf0 0x7fda1807d0b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda20009990 tx=0x7fda20009110 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:25.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:25.809+0000 7fda2cff9640 1 -- 172.21.15.46:0/446899994 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7fda24089d60 con 0x7fda301033d0 2024-09-10T04:09:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:25 smithi070 ceph-mon[35495]: pgmap v79: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:26.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.046+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fda30105f50 con 0x7fda301033d0 2024-09-10T04:09:26.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.046+0000 7fda2cff9640 1 -- 172.21.15.46:0/446899994 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7fda2408ec10 con 0x7fda301033d0 2024-09-10T04:09:26.048 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:09:26.048 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:09:26.048 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:09:26.048 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:09:26.049 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:09:26.050 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:09:26.051 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda1807abf0 msgr2=0x7fda1807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda1807abf0 0x7fda1807d0b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fda20009990 tx=0x7fda20009110 comp rx=0 tx=0).stop 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 msgr2=0x7fda30100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda30100c50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fda24031460 tx=0x7fda240046c0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 shutdown_connections 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda1807abf0 0x7fda1807d0b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda301033d0 0x7fda30100c50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 --2- 172.21.15.46:0/446899994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda30102ab0 0x7fda30100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 >> 172.21.15.46:0/446899994 conn(0x7fda300fc550 msgr2=0x7fda30108e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 shutdown_connections 2024-09-10T04:09:26.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.048+0000 7fda35f6c640 1 -- 172.21.15.46:0/446899994 wait complete. 2024-09-10T04:09:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 -- 172.21.15.46:0/278553766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0101900 msgr2=0x7f36e0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 --2- 172.21.15.46:0/278553766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0101900 0x7f36e0101d60 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009880 tx=0x7f36cc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 -- 172.21.15.46:0/278553766 shutdown_connections 2024-09-10T04:09:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 --2- 172.21.15.46:0/278553766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0101900 0x7f36e0101d60 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 --2- 172.21.15.46:0/278553766 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36e0107900 0x7f36e0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 -- 172.21.15.46:0/278553766 >> 172.21.15.46:0/278553766 conn(0x7f36e00fd600 msgr2=0x7f36e00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 -- 172.21.15.46:0/278553766 shutdown_connections 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.135+0000 7f36e7003640 1 -- 172.21.15.46:0/278553766 wait complete. 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36e7003640 1 Processor -- start 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36e7003640 1 -- start start 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36e7003640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36e0101900 0x7f36e019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36e7003640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0107900 0x7f36e019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36e7003640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36e010ee30 con 0x7f36e0107900 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36e7003640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f36e010ecb0 con 0x7f36e0101900 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36dffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0107900 0x7f36e019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36dffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0107900 0x7f36e019dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41722/0 (socket says 172.21.15.46:41722) 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36dffff640 1 -- 172.21.15.46:0/3409496858 learned_addr learned my addr 172.21.15.46:0/3409496858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36dffff640 1 -- 172.21.15.46:0/3409496858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36e0101900 msgr2=0x7f36e019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36dffff640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36e0101900 0x7f36e019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36dffff640 1 -- 172.21.15.46:0/3409496858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e0197860 con 0x7f36e0107900 2024-09-10T04:09:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.136+0000 7f36dffff640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0107900 0x7f36e019dbe0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f36cc031460 tx=0x7f36cc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.137+0000 7f36ddffb640 1 -- 172.21.15.46:0/3409496858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f36cc03f070 con 0x7f36e0107900 2024-09-10T04:09:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.137+0000 7f36ddffb640 1 -- 172.21.15.46:0/3409496858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f36cc004cf0 con 0x7f36e0107900 2024-09-10T04:09:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.137+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e0197af0 con 0x7f36e0107900 2024-09-10T04:09:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.137+0000 7f36ddffb640 1 -- 172.21.15.46:0/3409496858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f36cc035030 con 0x7f36e0107900 2024-09-10T04:09:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.137+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e0198020 con 0x7f36e0107900 2024-09-10T04:09:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.138+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e0103040 con 0x7f36e0107900 2024-09-10T04:09:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.139+0000 7f36ddffb640 1 -- 172.21.15.46:0/3409496858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f36cc042070 con 0x7f36e0107900 2024-09-10T04:09:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.139+0000 7f36ddffb640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36d007abf0 0x7f36d007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.139+0000 7f36ddffb640 1 -- 172.21.15.46:0/3409496858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f36cc0be700 con 0x7f36e0107900 2024-09-10T04:09:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.139+0000 7f36e4d9f640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36d007abf0 0x7f36d007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.140+0000 7f36e4d9f640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36d007abf0 0x7f36d007d0b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f36e01989a0 tx=0x7f36d4006c90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.144+0000 7f36ddffb640 1 -- 172.21.15.46:0/3409496858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f36cc089ef0 con 0x7f36e0107900 2024-09-10T04:09:26.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.359+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f36e0198d90 con 0x7f36d007abf0 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.360+0000 7f36ddffb640 1 -- 172.21.15.46:0/3409496858 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+368 (secure 0 0 0) 0x7f36e0198d90 con 0x7f36d007abf0 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "2/23 daemons upgraded", 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mon daemons", 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:09:26.362 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:09:26.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36d007abf0 msgr2=0x7f36d007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36d007abf0 0x7f36d007d0b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f36e01989a0 tx=0x7f36d4006c90 comp rx=0 tx=0).stop 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0107900 msgr2=0x7f36e019dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0107900 0x7f36e019dbe0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f36cc031460 tx=0x7f36cc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 shutdown_connections 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36d007abf0 0x7f36d007d0b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e0107900 0x7f36e019dbe0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 --2- 172.21.15.46:0/3409496858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36e0101900 0x7f36e019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.362+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 >> 172.21.15.46:0/3409496858 conn(0x7f36e00fd600 msgr2=0x7f36e00fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.363+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 shutdown_connections 2024-09-10T04:09:26.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.363+0000 7f36e7003640 1 -- 172.21.15.46:0/3409496858 wait complete. 2024-09-10T04:09:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.449+0000 7f9b5a492640 1 -- 172.21.15.46:0/2805355265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54103470 msgr2=0x7f9b5410b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.449+0000 7f9b5a492640 1 --2- 172.21.15.46:0/2805355265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54103470 0x7f9b5410b960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009880 tx=0x7f9b4802a530 comp rx=0 tx=0).stop 2024-09-10T04:09:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.449+0000 7f9b5a492640 1 -- 172.21.15.46:0/2805355265 shutdown_connections 2024-09-10T04:09:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.449+0000 7f9b5a492640 1 --2- 172.21.15.46:0/2805355265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54103470 0x7f9b5410b960 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.449+0000 7f9b5a492640 1 --2- 172.21.15.46:0/2805355265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b54102b50 0x7f9b54102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.449+0000 7f9b5a492640 1 -- 172.21.15.46:0/2805355265 >> 172.21.15.46:0/2805355265 conn(0x7f9b540fc550 msgr2=0x7f9b540fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.450+0000 7f9b5a492640 1 -- 172.21.15.46:0/2805355265 shutdown_connections 2024-09-10T04:09:26.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.450+0000 7f9b5a492640 1 -- 172.21.15.46:0/2805355265 wait complete. 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.450+0000 7f9b5a492640 1 Processor -- start 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.450+0000 7f9b5a492640 1 -- start start 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b5a492640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54102b50 0x7f9b541007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b5a492640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b54103470 0x7f9b54100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b5a492640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5410edc0 con 0x7f9b54102b50 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b5a492640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5410ec40 con 0x7f9b54103470 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b53fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54102b50 0x7f9b541007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b53fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54102b50 0x7f9b541007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41740/0 (socket says 172.21.15.46:41740) 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b53fff640 1 -- 172.21.15.46:0/1272642986 learned_addr learned my addr 172.21.15.46:0/1272642986 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b537fe640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b54103470 0x7f9b54100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b53fff640 1 -- 172.21.15.46:0/1272642986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b54103470 msgr2=0x7f9b54100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b53fff640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b54103470 0x7f9b54100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b53fff640 1 -- 172.21.15.46:0/1272642986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b54101230 con 0x7f9b54102b50 2024-09-10T04:09:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.451+0000 7f9b53fff640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54102b50 0x7f9b541007b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400d8d0 tx=0x7f9b4400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.452+0000 7f9b517fa640 1 -- 172.21.15.46:0/1272642986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b44017070 con 0x7f9b54102b50 2024-09-10T04:09:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.452+0000 7f9b517fa640 1 -- 172.21.15.46:0/1272642986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b44010460 con 0x7f9b54102b50 2024-09-10T04:09:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.452+0000 7f9b517fa640 1 -- 172.21.15.46:0/1272642986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b4400f5f0 con 0x7f9b54102b50 2024-09-10T04:09:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.452+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b54101520 con 0x7f9b54102b50 2024-09-10T04:09:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.452+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b541a3e10 con 0x7f9b54102b50 2024-09-10T04:09:26.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.453+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b5406a4e0 con 0x7f9b54102b50 2024-09-10T04:09:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.454+0000 7f9b517fa640 1 -- 172.21.15.46:0/1272642986 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9b44009a30 con 0x7f9b54102b50 2024-09-10T04:09:26.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.455+0000 7f9b517fa640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b3c07ac40 0x7f9b3c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:26.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.455+0000 7f9b517fa640 1 -- 172.21.15.46:0/1272642986 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f9b4409da10 con 0x7f9b54102b50 2024-09-10T04:09:26.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.455+0000 7f9b537fe640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b3c07ac40 0x7f9b3c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:26.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.455+0000 7f9b537fe640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b3c07ac40 0x7f9b3c07d100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b54102240 tx=0x7f9b48004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:26.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.459+0000 7f9b517fa640 1 -- 172.21.15.46:0/1272642986 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195986 (secure 0 0 0) 0x7f9b44069320 con 0x7f9b54102b50 2024-09-10T04:09:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.719+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9b54105f70 con 0x7f9b54102b50 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='client.14920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='client.14924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:26.721 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/271925504' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:26.722 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T04:09:26.722 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:26 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/446899994' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T04:09:26.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.719+0000 7f9b517fa640 1 -- 172.21.15.46:0/1272642986 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7f9b4406e1d0 con 0x7f9b54102b50 2024-09-10T04:09:26.722 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:26.722 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:26.722 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:09:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b3c07ac40 msgr2=0x7f9b3c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b3c07ac40 0x7f9b3c07d100 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b54102240 tx=0x7f9b48004f30 comp rx=0 tx=0).stop 2024-09-10T04:09:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54102b50 msgr2=0x7f9b541007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54102b50 0x7f9b541007b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400d8d0 tx=0x7f9b4400dd90 comp rx=0 tx=0).stop 2024-09-10T04:09:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 shutdown_connections 2024-09-10T04:09:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b3c07ac40 0x7f9b3c07d100 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b54103470 0x7f9b54100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 --2- 172.21.15.46:0/1272642986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b54102b50 0x7f9b541007b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 >> 172.21.15.46:0/1272642986 conn(0x7f9b540fc550 msgr2=0x7f9b54108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 shutdown_connections 2024-09-10T04:09:26.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:26.721+0000 7f9b5a492640 1 -- 172.21.15.46:0/1272642986 wait complete. 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='client.14920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='client.14924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/271925504' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:26.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T04:09:26.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:26 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/446899994' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T04:09:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: Upgrade: Target is version 19.3.0-4885-g3a5e3f16 (squid) 2024-09-10T04:09:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca'] 2024-09-10T04:09:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: Upgrade: Setting container_image for all mgr 2024-09-10T04:09:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: pgmap v80: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:27.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: from='client.14936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:27.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: from='client.? 172.21.15.46:0/1272642986' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T04:09:27.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:27.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T04:09:27.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T04:09:27.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:27 smithi046 ceph-mon[28328]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: Upgrade: Target is version 19.3.0-4885-g3a5e3f16 (squid) 2024-09-10T04:09:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, digests ['quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca'] 2024-09-10T04:09:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: Upgrade: Setting container_image for all mgr 2024-09-10T04:09:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: pgmap v80: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:27.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: from='client.14936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:27.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: from='client.? 172.21.15.46:0/1272642986' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T04:09:27.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:27.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T04:09:27.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T04:09:27.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:27 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:28.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:28 smithi046 ceph-mon[28328]: Upgrade: Updating mon.smithi046 2024-09-10T04:09:28.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:28 smithi046 ceph-mon[28328]: Deploying daemon mon.smithi046 on smithi046 2024-09-10T04:09:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:28 smithi070 ceph-mon[35495]: Upgrade: Updating mon.smithi046 2024-09-10T04:09:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:28 smithi070 ceph-mon[35495]: Deploying daemon mon.smithi046 on smithi046 2024-09-10T04:09:29.142 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:28 smithi046 systemd[1]: Stopping Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:09:29.442 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[28324]: 2024-09-10T04:09:29.140+0000 7f2b42384640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi046 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:09:29.442 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[28324]: 2024-09-10T04:09:29.140+0000 7f2b42384640 -1 mon.smithi046@0(leader) e2 *** Got Signal Terminated *** 2024-09-10T04:09:29.442 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 podman[109343]: 2024-09-10 04:09:29.171517631 +0000 UTC m=+0.132537934 container died 9933ea41a29e06352837603dab77083048a972ce9995b598a832d6f5ee21015d (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-10T04:09:29.825 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 podman[109343]: 2024-09-10 04:09:29.540095118 +0000 UTC m=+0.501115420 container cleanup 9933ea41a29e06352837603dab77083048a972ce9995b598a832d6f5ee21015d (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-10T04:09:29.825 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 bash[109343]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046 2024-09-10T04:09:30.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 podman[109355]: 2024-09-10 04:09:29.824782734 +0000 UTC m=+0.649366801 container remove 9933ea41a29e06352837603dab77083048a972ce9995b598a832d6f5ee21015d (image=quay.ceph.io/ceph-ci/ceph:squid, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T04:09:30.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service: Deactivated successfully. 2024-09-10T04:09:30.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 systemd[1]: Stopped Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:09:30.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:29 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service: Consumed 14.450s CPU time. 2024-09-10T04:09:30.541 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 systemd[1]: Starting Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:09:30.541 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 podman[109488]: 2024-09-10 04:09:30.302018694 +0000 UTC m=+0.016689142 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:09:30.541 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 podman[109488]: 2024-09-10 04:09:30.434513074 +0000 UTC m=+0.149183569 container create da8d2333cd825834448ae8d2e4f822b655d9c5cc53b4cd09afa71510c3d7553e (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T04:09:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 podman[109488]: 2024-09-10 04:09:30.670870355 +0000 UTC m=+0.385540795 container init da8d2333cd825834448ae8d2e4f822b655d9c5cc53b4cd09afa71510c3d7553e (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T04:09:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 podman[109488]: 2024-09-10 04:09:30.674320714 +0000 UTC m=+0.388991161 container start da8d2333cd825834448ae8d2e4f822b655d9c5cc53b4cd09afa71510c3d7553e (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T04:09:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T04:09:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev), process ceph-mon, pid 2 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: pidfile_write: ignore empty --pid-file 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: load: jerasure load: lrc 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: RocksDB version: 7.9.2 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Git sha 0 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Compile date 2024-09-10 00:49:01 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: DB SUMMARY 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: DB Session ID: G69O40IH5UN54QA8EWLL 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: CURRENT file: CURRENT 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1496 Bytes 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi046/store.db dir, Total Num: 1, files: 000032.sst 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi046/store.db: 000030.log size: 2432473 ; 2024-09-10T04:09:30.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.error_if_exists: 0 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.create_if_missing: 0 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.paranoid_checks: 1 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.env: 0x55ed95e3e120 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.info_log: 0x55ed986918e0 2024-09-10T04:09:30.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.statistics: (nil) 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.use_fsync: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_log_file_size: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.allow_fallocate: 1 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.use_direct_reads: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T04:09:30.841 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.db_log_dir: 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.wal_dir: 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.write_buffer_manager: 0x55ed98695b80 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T04:09:30.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.unordered_write: 0 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.row_cache: None 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.wal_filter: None 2024-09-10T04:09:30.843 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.two_write_queues: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.wal_compression: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.atomic_flush: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.log_readahead_size: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T04:09:30.844 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_background_jobs: 2 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_background_compactions: -1 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_subcompactions: 1 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T04:09:30.845 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_open_files: -1 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_background_flushes: -1 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Compression algorithms supported: 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kZSTD supported: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kXpressCompression supported: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kBZip2Compression supported: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kLZ4Compression supported: 1 2024-09-10T04:09:30.846 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kZlibCompression supported: 1 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: kSnappyCompression supported: 1 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi046/store.db/MANIFEST-000015 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.merge_operator: 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_filter: None 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T04:09:30.847 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ed98691540) 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: index_type: 0 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: data_block_index_type: 0 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: index_shortening: 1 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: checksum: 4 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: no_block_cache: 0 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache: 0x55ed98685610 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-09-10T04:09:30.848 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache_options: 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: capacity : 536870912 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: num_shard_bits : 4 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: strict_capacity_limit : 0 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_cache_compressed: (nil) 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: persistent_cache: (nil) 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_size: 4096 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_size_deviation: 10 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_restart_interval: 16 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: index_block_restart_interval: 1 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: metadata_block_size: 4096 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: partition_filters: 0 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: use_delta_encoding: 1 2024-09-10T04:09:30.849 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: filter_policy: bloomfilter 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: whole_key_filtering: 1 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: verify_compression: 0 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: format_version: 5 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: enable_index_compression: 1 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: block_align: 0 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: max_auto_readahead_size: 262144 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: prepopulate_block_cache: 0 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: initial_auto_readahead_size: 8192 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression: NoCompression 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T04:09:30.850 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.num_levels: 7 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T04:09:30.851 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T04:09:30.852 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T04:09:30.853 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T04:09:30.854 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.inplace_update_support: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.bloom_locality: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.max_successive_merges: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.ttl: 2592000 2024-09-10T04:09:30.855 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.enable_blob_files: false 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.min_blob_size: 0 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T04:09:30.856 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi046/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12028, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 248d7314-bca8-471d-9f91-66e3162d0852 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725941370701970, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725941370710065, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 2112016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12029, "largest_seqno": 13686, "table_properties": {"data_size": 2104112, "index_size": 4939, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1925, "raw_key_size": 17419, "raw_average_key_size": 23, "raw_value_size": 2088167, "raw_average_value_size": 2799, "num_data_blocks": 226, "num_entries": 746, "num_filter_entries": 746, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725941370, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "248d7314-bca8-471d-9f91-66e3162d0852", "db_session_id": "G69O40IH5UN54QA8EWLL", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725941370710191, "job": 1, "event": "recovery_finished"} 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi046/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ed986b4e00 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: DB pointer 0x55ed987b6000 2024-09-10T04:09:30.857 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** DB Stats ** 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T04:09:30.858 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: L0 1/0 2.01 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 488.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: L6 1/0 7.18 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Sum 2/0 9.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 488.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 488.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** Compaction Stats [default] ** 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 488.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.859 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Cumulative compaction: 0.00 GB write, 188.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Interval compaction: 0.00 GB write, 188.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Block cache BinnedLRUCache@0x55ed98685610#2 capacity: 512.00 MB usage: 7.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T04:09:30.860 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: starting mon.smithi046 rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi046 fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???) e2 preinit fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???).mds e16 new map 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???).mds e16 print_map 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: e16 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: legacy client fscid: 1 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: Filesystem 'cephfs' (1) 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: fs_name cephfs 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: epoch 16 2024-09-10T04:09:30.861 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: tableserver 0 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: root 0 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: session_timeout 60 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: session_autoclose 300 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: max_file_size 1099511627776 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: max_xattr_size 65536 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: required_client_features {} 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: last_failure 0 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: last_failure_osd_epoch 0 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:30.862 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: max_mds 2 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: in 0,1 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: up {0=14558,1=24325} 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: failed 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: damaged 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: stopped 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: data_pools [3] 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: metadata_pool 2 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: inline_data enabled 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: balancer 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: bal_rank_mask -1 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: standby_count_wanted 1 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: [mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: [mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: [mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:30.863 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: [mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout: 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???).osd e45 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???).osd e45 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???).osd e45 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???).osd e45 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 ceph-mon[109506]: mon.smithi046@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 bash[109488]: da8d2333cd825834448ae8d2e4f822b655d9c5cc53b4cd09afa71510c3d7553e 2024-09-10T04:09:30.864 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:30 smithi046 systemd[1]: Started Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:09:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: pgmap v82: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: pgmap v83: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: mon.smithi046 calling monitor election 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: mon.smithi046 is new leader, mons smithi046,smithi070 in quorum (ranks 0,1) 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: monmap epoch 2 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: last_changed 2024-09-10T03:57:58.410804+0000 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: min_mon_release 19 (squid) 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: election_strategy: 1 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T04:09:34.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: 1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: mgrmap e32: smithi046.zvckjg(active, since 2m), standbys: smithi070.tezbzn 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='' 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:33 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:34.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: pgmap v82: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: pgmap v83: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: mon.smithi046 calling monitor election 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: mon.smithi046 is new leader, mons smithi046,smithi070 in quorum (ranks 0,1) 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: monmap epoch 2 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: last_changed 2024-09-10T03:57:58.410804+0000 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: min_mon_release 19 (squid) 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: election_strategy: 1 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: 1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-09-10T04:09:34.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: mgrmap e32: smithi046.zvckjg(active, since 2m), standbys: smithi070.tezbzn 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='' 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:34.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:35.069 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:34 smithi046 ceph-mon[109506]: pgmap v84: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:34 smithi070 ceph-mon[35495]: pgmap v84: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:37.385 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:37 smithi046 ceph-mon[109506]: pgmap v85: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:37 smithi070 ceph-mon[35495]: pgmap v85: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:38.216 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:09:38.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:38 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T04:09:39.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:39 smithi070 ceph-mon[35495]: pgmap v86: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:39.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:39 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:39.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:39 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:39 smithi046 ceph-mon[109506]: pgmap v86: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:39 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:39 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:40.558 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:40 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:40.558 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:40 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:40.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:40 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:40.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:40 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:41 smithi070 ceph-mon[35495]: pgmap v87: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:41 smithi046 ceph-mon[109506]: pgmap v87: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: pgmap v88: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:09:43.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:43.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:43.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:43.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:43 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T04:09:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: pgmap v88: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:43.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T04:09:43.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:43.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T04:09:43.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T04:09:43.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:43 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T04:09:45.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-mon[35495]: Upgrade: Updating mon.smithi070 2024-09-10T04:09:45.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-mon[35495]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-mon[35495]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-mon[35495]: Deploying daemon mon.smithi070 on smithi070 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-mon[35495]: pgmap v89: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:45 smithi046 ceph-mon[109506]: Upgrade: Updating mon.smithi070 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:45 smithi046 ceph-mon[109506]: from='mgr.14758 ' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:45.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T04:09:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T04:09:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T04:09:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:45 smithi046 ceph-mon[109506]: Deploying daemon mon.smithi070 on smithi070 2024-09-10T04:09:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:45 smithi046 ceph-mon[109506]: pgmap v89: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:09:45.937 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:09:46.294 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070[35491]: 2024-09-10T04:09:45.935+0000 7f6440d83640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:09:46.295 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070[35491]: 2024-09-10T04:09:45.935+0000 7f6440d83640 -1 mon.smithi070@1(peon) e2 *** Got Signal Terminated *** 2024-09-10T04:09:46.295 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:45 smithi070 podman[79605]: 2024-09-10 04:09:45.956239405 +0000 UTC m=+0.136613359 container died b5145a46761afe21e03716937f5c9e51d4f7409c8059b6b850f34cd7f853fde5 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T04:09:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:46 smithi070 podman[79605]: 2024-09-10 04:09:46.46040515 +0000 UTC m=+0.640779091 container cleanup b5145a46761afe21e03716937f5c9e51d4f7409c8059b6b850f34cd7f853fde5 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T04:09:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:46 smithi070 bash[79605]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070 2024-09-10T04:09:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:46 smithi070 podman[79617]: 2024-09-10 04:09:46.753554073 +0000 UTC m=+0.790234267 container remove b5145a46761afe21e03716937f5c9e51d4f7409c8059b6b850f34cd7f853fde5 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T04:09:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:46 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi070.service: Deactivated successfully. 2024-09-10T04:09:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:46 smithi070 systemd[1]: Stopped Ceph mon.smithi070 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:09:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:46 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi070.service: Consumed 7.303s CPU time. 2024-09-10T04:09:47.453 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 systemd[1]: Starting Ceph mon.smithi070 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:09:47.453 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 podman[79753]: 2024-09-10 04:09:47.216054491 +0000 UTC m=+0.016661488 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:09:47.454 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 podman[79753]: 2024-09-10 04:09:47.321675701 +0000 UTC m=+0.122282685 container create 9c38930714e4a15eb2c3bdd945213e59e04dc1a82f9441d3e030504125a70bfd (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 podman[79753]: 2024-09-10 04:09:47.557480063 +0000 UTC m=+0.358087047 container init 9c38930714e4a15eb2c3bdd945213e59e04dc1a82f9441d3e030504125a70bfd (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 podman[79753]: 2024-09-10 04:09:47.560869556 +0000 UTC m=+0.361476540 container start 9c38930714e4a15eb2c3bdd945213e59e04dc1a82f9441d3e030504125a70bfd (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev), process ceph-mon, pid 2 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: pidfile_write: ignore empty --pid-file 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: load: jerasure load: lrc 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: RocksDB version: 7.9.2 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Git sha 0 2024-09-10T04:09:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Compile date 2024-09-10 00:49:01 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: DB SUMMARY 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: DB Session ID: VDVNTTA3FO5I1AD1A117 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: CURRENT file: CURRENT 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1389 Bytes 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi070/store.db dir, Total Num: 1, files: 000027.sst 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi070/store.db: 000025.log size: 4222933 ; 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.error_if_exists: 0 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.create_if_missing: 0 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.paranoid_checks: 1 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T04:09:47.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.env: 0x55669d488120 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.info_log: 0x55669f75d8e0 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.statistics: (nil) 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.use_fsync: 0 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_log_file_size: 0 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.allow_fallocate: 1 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T04:09:47.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.use_direct_reads: 0 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.db_log_dir: 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.wal_dir: 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T04:09:47.839 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.write_buffer_manager: 0x55669f761b80 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.unordered_write: 0 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T04:09:47.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.row_cache: None 2024-09-10T04:09:47.841 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.wal_filter: None 2024-09-10T04:09:47.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T04:09:47.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T04:09:47.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.two_write_queues: 0 2024-09-10T04:09:47.842 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.wal_compression: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.atomic_flush: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.log_readahead_size: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T04:09:47.843 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_background_jobs: 2 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_background_compactions: -1 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_subcompactions: 1 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_open_files: -1 2024-09-10T04:09:47.844 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_background_flushes: -1 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Compression algorithms supported: 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kZSTD supported: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kXpressCompression supported: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kBZip2Compression supported: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kLZ4Compression supported: 1 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kZlibCompression supported: 1 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: kSnappyCompression supported: 1 2024-09-10T04:09:47.845 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000010 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.merge_operator: 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_filter: None 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55669f75d540) 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T04:09:47.846 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_type: 0 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_index_type: 0 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_shortening: 1 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: checksum: 4 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: no_block_cache: 0 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache: 0x55669f751610 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_options: 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: capacity : 536870912 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_shard_bits : 4 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: strict_capacity_limit : 0 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-09-10T04:09:47.847 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_cache_compressed: (nil) 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: persistent_cache: (nil) 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size: 4096 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_size_deviation: 10 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_restart_interval: 16 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: index_block_restart_interval: 1 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_block_size: 4096 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: partition_filters: 0 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: use_delta_encoding: 1 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: filter_policy: bloomfilter 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: whole_key_filtering: 1 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: verify_compression: 0 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-09-10T04:09:47.848 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: format_version: 5 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_index_compression: 1 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: block_align: 0 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_auto_readahead_size: 262144 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: prepopulate_block_cache: 0 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: initial_auto_readahead_size: 8192 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression: NoCompression 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.num_levels: 7 2024-09-10T04:09:47.849 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T04:09:47.850 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T04:09:47.851 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T04:09:47.852 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.inplace_update_support: 0 2024-09-10T04:09:47.853 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.bloom_locality: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.max_successive_merges: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.ttl: 2592000 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T04:09:47.854 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.enable_blob_files: false 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.min_blob_size: 0 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi070/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12254, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-09-10T04:09:47.855 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c3b06bec-76dd-47d1-b34d-7e10600ae36a 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725941387588412, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725941387600839, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2426843, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12259, "largest_seqno": 14378, "table_properties": {"data_size": 2417934, "index_size": 5687, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 2181, "raw_key_size": 19973, "raw_average_key_size": 23, "raw_value_size": 2399639, "raw_average_value_size": 2806, "num_data_blocks": 259, "num_entries": 855, "num_filter_entries": 855, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725941387, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c3b06bec-76dd-47d1-b34d-7e10600ae36a", "db_session_id": "VDVNTTA3FO5I1AD1A117", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725941387600951, "job": 1, "event": "recovery_finished"} 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi070/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55669f780e00 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: DB pointer 0x55669f882000 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** DB Stats ** 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T04:09:47.856 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L0 1/0 2.31 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 541.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: L6 1/0 7.18 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Sum 2/0 9.49 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 541.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 541.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:47.857 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 541.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T04:09:47.858 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Cumulative compaction: 0.00 GB write, 150.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Interval compaction: 0.00 GB write, 150.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache BinnedLRUCache@0x55669f751610#2 capacity: 512.00 MB usage: 8.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.28 KB,0.000435114%) IndexBlock(1,6.08 KB,0.00115931%) Misc(1,0.00 KB,0%) 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: starting mon.smithi070 rank 1 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi070 fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???) e2 preinit fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???).mds e16 new map 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???).mds e16 print_map 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: e16 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:09:47.859 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: legacy client fscid: 1 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: Filesystem 'cephfs' (1) 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: fs_name cephfs 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: epoch 16 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: tableserver 0 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: root 0 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: session_timeout 60 2024-09-10T04:09:47.860 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: session_autoclose 300 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_file_size 1099511627776 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_xattr_size 65536 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: required_client_features {} 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: last_failure 0 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: last_failure_osd_epoch 0 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: max_mds 2 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: in 0,1 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: up {0=14558,1=24325} 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: failed 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: damaged 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: stopped 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: data_pools [3] 2024-09-10T04:09:47.861 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: metadata_pool 2 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: inline_data enabled 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: balancer 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: bal_rank_mask -1 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: standby_count_wanted 1 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: [mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout: 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???).osd e45 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???).osd e45 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T04:09:47.862 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???).osd e45 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T04:09:47.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???).osd e45 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T04:09:47.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 ceph-mon[79774]: mon.smithi070@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-09-10T04:09:47.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 bash[79753]: 9c38930714e4a15eb2c3bdd945213e59e04dc1a82f9441d3e030504125a70bfd 2024-09-10T04:09:47.863 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:47 smithi070 systemd[1]: Started Ceph mon.smithi070 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:09:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mon metadata", "id": "smithi046"} : dispatch 2024-09-10T04:09:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mon metadata", "id": "smithi070"} : dispatch 2024-09-10T04:09:49.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:09:49.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mon metadata", "id": "smithi046"} : dispatch 2024-09-10T04:09:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mon metadata", "id": "smithi070"} : dispatch 2024-09-10T04:09:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:09:50.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:09:50.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: pgmap v90: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: pgmap v91: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: mon.smithi070 calling monitor election 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: mon.smithi046 calling monitor election 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: mon.smithi046 is new leader, mons smithi046,smithi070 in quorum (ranks 0,1) 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: monmap epoch 2 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: last_changed 2024-09-10T03:57:58.410804+0000 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: min_mon_release 19 (squid) 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: election_strategy: 1 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T04:09:50.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: 1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: mgrmap e32: smithi046.zvckjg(active, since 2m), standbys: smithi070.tezbzn 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:50 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:09:50.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: pgmap v90: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: pgmap v91: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: mon.smithi070 calling monitor election 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: mon.smithi046 calling monitor election 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: mon.smithi046 is new leader, mons smithi046,smithi070 in quorum (ranks 0,1) 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: monmap epoch 2 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: fsid 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: last_changed 2024-09-10T03:57:58.410804+0000 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: created 2024-09-10T03:55:29.528347+0000 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: min_mon_release 19 (squid) 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: election_strategy: 1 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.smithi046 2024-09-10T04:09:50.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: 1: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.smithi070 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2 up:standby-replay 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: mgrmap e32: smithi046.zvckjg(active, since 2m), standbys: smithi070.tezbzn 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:50.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:50 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:09:51.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:51 smithi070 ceph-mon[79774]: pgmap v92: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:09:51.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:51 smithi046 ceph-mon[109506]: pgmap v92: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:09:53.188 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:53.189 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:53.189 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:52 smithi070 ceph-mon[79774]: pgmap v93: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:09:53.189 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:53.189 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:53.276 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:53.276 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:53.276 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:52 smithi046 ceph-mon[109506]: pgmap v93: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:09:53.276 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:53.276 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:54.153 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:09:54.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:09:55.215 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:54 smithi070 ceph-mon[79774]: pgmap v94: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:09:55.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:54 smithi046 ceph-mon[109506]: pgmap v94: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:09:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:09:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:09:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:09:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all mon 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi046"} : dispatch 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi046"}]': finished 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"} : dispatch 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]': finished 2024-09-10T04:09:56.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:56 smithi046 ceph-mon[109506]: pgmap v95: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:09:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 -- 172.21.15.46:0/1399749116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c103430 msgr2=0x7fdb5c10b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/1399749116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c103430 0x7fdb5c10b920 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0098b0 tx=0x7fdb4c02a560 comp rx=0 tx=0).stop 2024-09-10T04:09:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 -- 172.21.15.46:0/1399749116 shutdown_connections 2024-09-10T04:09:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/1399749116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c103430 0x7fdb5c10b920 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:56.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/1399749116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb5c102b10 0x7fdb5c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 -- 172.21.15.46:0/1399749116 >> 172.21.15.46:0/1399749116 conn(0x7fdb5c0fc550 msgr2=0x7fdb5c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 -- 172.21.15.46:0/1399749116 shutdown_connections 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 -- 172.21.15.46:0/1399749116 wait complete. 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.810+0000 7fdb63cb8640 1 Processor -- start 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb63cb8640 1 -- start start 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb63cb8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c102b10 0x7fdb5c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb63cb8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb5c103430 0x7fdb5c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb63cb8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c10ede0 con 0x7fdb5c102b10 2024-09-10T04:09:56.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb63cb8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c10ec60 con 0x7fdb5c103430 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb61a54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c102b10 0x7fdb5c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb61a54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c102b10 0x7fdb5c19d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33074/0 (socket says 172.21.15.46:33074) 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb61a54640 1 -- 172.21.15.46:0/2740580620 learned_addr learned my addr 172.21.15.46:0/2740580620 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb61253640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb5c103430 0x7fdb5c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb61a54640 1 -- 172.21.15.46:0/2740580620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb5c103430 msgr2=0x7fdb5c19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb61a54640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb5c103430 0x7fdb5c19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.811+0000 7fdb61a54640 1 -- 172.21.15.46:0/2740580620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c1977f0 con 0x7fdb5c102b10 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.812+0000 7fdb61a54640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c102b10 0x7fdb5c19d670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800c8f0 tx=0x7fdb5800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.812+0000 7fdb52ffd640 1 -- 172.21.15.46:0/2740580620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdb580079b0 con 0x7fdb5c102b10 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.812+0000 7fdb52ffd640 1 -- 172.21.15.46:0/2740580620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdb580054e0 con 0x7fdb5c102b10 2024-09-10T04:09:56.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.812+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c197ae0 con 0x7fdb5c102b10 2024-09-10T04:09:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.812+0000 7fdb52ffd640 1 -- 172.21.15.46:0/2740580620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdb58004dd0 con 0x7fdb5c102b10 2024-09-10T04:09:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.812+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c198040 con 0x7fdb5c102b10 2024-09-10T04:09:56.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.813+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb5c06a520 con 0x7fdb5c102b10 2024-09-10T04:09:56.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.814+0000 7fdb52ffd640 1 -- 172.21.15.46:0/2740580620 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdb58005020 con 0x7fdb5c102b10 2024-09-10T04:09:56.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.814+0000 7fdb52ffd640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdb4807ab90 0x7fdb4807d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:56.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.814+0000 7fdb52ffd640 1 -- 172.21.15.46:0/2740580620 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fdb5809db20 con 0x7fdb5c102b10 2024-09-10T04:09:56.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.814+0000 7fdb61253640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdb4807ab90 0x7fdb4807d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:56.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.815+0000 7fdb61253640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdb4807ab90 0x7fdb4807d050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000c00 tx=0x7fdb4c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:56.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:56.819+0000 7fdb52ffd640 1 -- 172.21.15.46:0/2740580620 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdb58014030 con 0x7fdb5c102b10 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:09:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all mon 2024-09-10T04:09:56.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:56.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi046"} : dispatch 2024-09-10T04:09:56.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi046"}]': finished 2024-09-10T04:09:56.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"} : dispatch 2024-09-10T04:09:56.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi070"}]': finished 2024-09-10T04:09:56.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:56 smithi070 ceph-mon[79774]: pgmap v95: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:09:57.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.028+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdb5c198380 con 0x7fdb4807ab90 2024-09-10T04:09:57.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.029+0000 7fdb52ffd640 1 -- 172.21.15.46:0/2740580620 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+385 (secure 0 0 0) 0x7fdb5c198380 con 0x7fdb4807ab90 2024-09-10T04:09:57.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdb4807ab90 msgr2=0x7fdb4807d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdb4807ab90 0x7fdb4807d050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000c00 tx=0x7fdb4c031400 comp rx=0 tx=0).stop 2024-09-10T04:09:57.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c102b10 msgr2=0x7fdb5c19d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c102b10 0x7fdb5c19d670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800c8f0 tx=0x7fdb5800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 shutdown_connections 2024-09-10T04:09:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdb4807ab90 0x7fdb4807d050 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb5c103430 0x7fdb5c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 --2- 172.21.15.46:0/2740580620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb5c102b10 0x7fdb5c19d670 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.031+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 >> 172.21.15.46:0/2740580620 conn(0x7fdb5c0fc550 msgr2=0x7fdb5c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.032+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 shutdown_connections 2024-09-10T04:09:57.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.032+0000 7fdb63cb8640 1 -- 172.21.15.46:0/2740580620 wait complete. 2024-09-10T04:09:57.042 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:09:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 -- 172.21.15.46:0/1304215917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 msgr2=0x7fb7a8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 --2- 172.21.15.46:0/1304215917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a8102e90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009880 tx=0x7fb79c02a530 comp rx=0 tx=0).stop 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 -- 172.21.15.46:0/1304215917 shutdown_connections 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 --2- 172.21.15.46:0/1304215917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a81033d0 0x7fb7a810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 --2- 172.21.15.46:0/1304215917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a8102e90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 -- 172.21.15.46:0/1304215917 >> 172.21.15.46:0/1304215917 conn(0x7fb7a80fc550 msgr2=0x7fb7a80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 -- 172.21.15.46:0/1304215917 shutdown_connections 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.119+0000 7fb7b0055640 1 -- 172.21.15.46:0/1304215917 wait complete. 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7b0055640 1 Processor -- start 2024-09-10T04:09:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7b0055640 1 -- start start 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7b0055640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7b0055640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a81033d0 0x7fb7a819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7b0055640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a810edc0 con 0x7fb7a8102ab0 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7b0055640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a810ec40 con 0x7fb7a81033d0 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7addf1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7addf1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33094/0 (socket says 172.21.15.46:33094) 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7addf1640 1 -- 172.21.15.46:0/807943090 learned_addr learned my addr 172.21.15.46:0/807943090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7ad5f0640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a81033d0 0x7fb7a819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7addf1640 1 -- 172.21.15.46:0/807943090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a81033d0 msgr2=0x7fb7a819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7addf1640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a81033d0 0x7fb7a819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.120+0000 7fb7addf1640 1 -- 172.21.15.46:0/807943090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a81977b0 con 0x7fb7a8102ab0 2024-09-10T04:09:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.121+0000 7fb7addf1640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a819d630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb79c031460 tx=0x7fb79c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.121+0000 7fb79affd640 1 -- 172.21.15.46:0/807943090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb79c03f070 con 0x7fb7a8102ab0 2024-09-10T04:09:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.121+0000 7fb79affd640 1 -- 172.21.15.46:0/807943090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb79c004cf0 con 0x7fb7a8102ab0 2024-09-10T04:09:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.121+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a8197a40 con 0x7fb7a8102ab0 2024-09-10T04:09:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.121+0000 7fb79affd640 1 -- 172.21.15.46:0/807943090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb79c035030 con 0x7fb7a8102ab0 2024-09-10T04:09:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.121+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a8197f70 con 0x7fb7a8102ab0 2024-09-10T04:09:57.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.122+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a806a580 con 0x7fb7a8102ab0 2024-09-10T04:09:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.123+0000 7fb79affd640 1 -- 172.21.15.46:0/807943090 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb79c042070 con 0x7fb7a8102ab0 2024-09-10T04:09:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.123+0000 7fb79affd640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb79407ac40 0x7fb79407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.123+0000 7fb79affd640 1 -- 172.21.15.46:0/807943090 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fb79c0be600 con 0x7fb7a8102ab0 2024-09-10T04:09:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.123+0000 7fb7ad5f0640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb79407ac40 0x7fb79407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.124+0000 7fb7ad5f0640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb79407ac40 0x7fb79407d100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4005fd0 tx=0x7fb7a4005d60 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:57.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.128+0000 7fb79affd640 1 -- 172.21.15.46:0/807943090 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb79c089e70 con 0x7fb7a8102ab0 2024-09-10T04:09:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.348+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb7a8198220 con 0x7fb79407ac40 2024-09-10T04:09:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.349+0000 7fb79affd640 1 -- 172.21.15.46:0/807943090 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+385 (secure 0 0 0) 0x7fb7a8198220 con 0x7fb79407ac40 2024-09-10T04:09:57.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb79407ac40 msgr2=0x7fb79407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb79407ac40 0x7fb79407d100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4005fd0 tx=0x7fb7a4005d60 comp rx=0 tx=0).stop 2024-09-10T04:09:57.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 msgr2=0x7fb7a819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a819d630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb79c031460 tx=0x7fb79c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 shutdown_connections 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb79407ac40 0x7fb79407d100 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a81033d0 0x7fb7a819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 --2- 172.21.15.46:0/807943090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7a8102ab0 0x7fb7a819d630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.351+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 >> 172.21.15.46:0/807943090 conn(0x7fb7a80fc550 msgr2=0x7fb7a8108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.352+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 shutdown_connections 2024-09-10T04:09:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.352+0000 7fb7b0055640 1 -- 172.21.15.46:0/807943090 wait complete. 2024-09-10T04:09:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 -- 172.21.15.46:0/3437500802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 msgr2=0x7fdd0010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/3437500802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd0010b8c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4009880 tx=0x7fdcf402a530 comp rx=0 tx=0).stop 2024-09-10T04:09:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 -- 172.21.15.46:0/3437500802 shutdown_connections 2024-09-10T04:09:57.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/3437500802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd0010b8c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/3437500802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd00102ab0 0x7fdd00102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 -- 172.21.15.46:0/3437500802 >> 172.21.15.46:0/3437500802 conn(0x7fdd000fc550 msgr2=0x7fdd000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 -- 172.21.15.46:0/3437500802 shutdown_connections 2024-09-10T04:09:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.437+0000 7fdd05c3f640 1 -- 172.21.15.46:0/3437500802 wait complete. 2024-09-10T04:09:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdd05c3f640 1 Processor -- start 2024-09-10T04:09:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdd05c3f640 1 -- start start 2024-09-10T04:09:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdd05c3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd00102ab0 0x7fdd00100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdd05c3f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd00100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdd05c3f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0010edc0 con 0x7fdd001033d0 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdd05c3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0010ec40 con 0x7fdd00102ab0 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdcfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd00100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdcfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd00100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33104/0 (socket says 172.21.15.46:33104) 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdcfeffd640 1 -- 172.21.15.46:0/4190191504 learned_addr learned my addr 172.21.15.46:0/4190191504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdcff7fe640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd00102ab0 0x7fdd00100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdcfeffd640 1 -- 172.21.15.46:0/4190191504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd00102ab0 msgr2=0x7fdd00100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdcfeffd640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd00102ab0 0x7fdd00100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.438+0000 7fdcfeffd640 1 -- 172.21.15.46:0/4190191504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd00101190 con 0x7fdd001033d0 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.439+0000 7fdcfeffd640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd00100c50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4031460 tx=0x7fdcf4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.439+0000 7fdcfcff9640 1 -- 172.21.15.46:0/4190191504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdcf403f070 con 0x7fdd001033d0 2024-09-10T04:09:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.439+0000 7fdcfcff9640 1 -- 172.21.15.46:0/4190191504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdcf40035f0 con 0x7fdd001033d0 2024-09-10T04:09:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.439+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd00101420 con 0x7fdd001033d0 2024-09-10T04:09:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.439+0000 7fdcfcff9640 1 -- 172.21.15.46:0/4190191504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdcf4035030 con 0x7fdd001033d0 2024-09-10T04:09:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.439+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd001a3d50 con 0x7fdd001033d0 2024-09-10T04:09:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.440+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0006a580 con 0x7fdd001033d0 2024-09-10T04:09:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.441+0000 7fdcfcff9640 1 -- 172.21.15.46:0/4190191504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdcf4042070 con 0x7fdd001033d0 2024-09-10T04:09:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.441+0000 7fdcfcff9640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdce807ac40 0x7fdce807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.441+0000 7fdcfcff9640 1 -- 172.21.15.46:0/4190191504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fdcf40be4a0 con 0x7fdd001033d0 2024-09-10T04:09:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.441+0000 7fdcff7fe640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdce807ac40 0x7fdce807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.442+0000 7fdcff7fe640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdce807ac40 0x7fdce807d100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0005fd0 tx=0x7fdcf0009870 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.445+0000 7fdcfcff9640 1 -- 172.21.15.46:0/4190191504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdcf4089d10 con 0x7fdd001033d0 2024-09-10T04:09:57.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.644+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdd0010b5c0 con 0x7fdce807ac40 2024-09-10T04:09:57.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.654+0000 7fdcfcff9640 1 -- 172.21.15.46:0/4190191504 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7fdd0010b5c0 con 0x7fdce807ac40 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (93s) 18s ago 12m 20.8M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (4m) 18s ago 13m 8828k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (3m) 5s ago 12m 8510k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (13m) 18s ago 13m 6891k - 19.1.1-298-gb53b6d3e 54467abc55f8 6e8855af891d 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (12m) 5s ago 12m 6903k - 19.1.1-298-gb53b6d3e 54467abc55f8 c772daa738b3 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (68s) 18s ago 12m 75.3M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (7m) 18s ago 7m 90.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (7m) 18s ago 7m 147M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:09:57.656 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (7m) 5s ago 7m 25.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (7m) 5s ago 7m 25.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (3m) 18s ago 14m 526M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (2m) 5s ago 12m 455M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (27s) 18s ago 14m 24.5M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (10s) 5s ago 11m 20.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (2m) 18s ago 12m 10.0M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (2m) 5s ago 12m 11.6M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (10m) 18s ago 10m 211M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 ebeeeb3fce9b 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (10m) 18s ago 10m 226M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (9m) 18s ago 9m 179M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (9m) 5s ago 9m 236M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (8m) 5s ago 8m 209M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:09:57.657 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (8m) 5s ago 8m 196M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:09:57.658 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (112s) 18s ago 12m 48.5M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:09:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdce807ac40 msgr2=0x7fdce807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdce807ac40 0x7fdce807d100 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0005fd0 tx=0x7fdcf0009870 comp rx=0 tx=0).stop 2024-09-10T04:09:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 msgr2=0x7fdd00100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd00100c50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4031460 tx=0x7fdcf4004530 comp rx=0 tx=0).stop 2024-09-10T04:09:57.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 shutdown_connections 2024-09-10T04:09:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdce807ac40 0x7fdce807d100 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd001033d0 0x7fdd00100c50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 --2- 172.21.15.46:0/4190191504 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd00102ab0 0x7fdd00100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 >> 172.21.15.46:0/4190191504 conn(0x7fdd000fc550 msgr2=0x7fdd0010aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 shutdown_connections 2024-09-10T04:09:57.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.656+0000 7fdd05c3f640 1 -- 172.21.15.46:0/4190191504 wait complete. 2024-09-10T04:09:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 -- 172.21.15.46:0/1727239484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154101900 msgr2=0x7f5154101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 --2- 172.21.15.46:0/1727239484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154101900 0x7f5154101d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5148009880 tx=0x7f514802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 -- 172.21.15.46:0/1727239484 shutdown_connections 2024-09-10T04:09:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 --2- 172.21.15.46:0/1727239484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154101900 0x7f5154101d60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 --2- 172.21.15.46:0/1727239484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154107900 0x7f5154107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 -- 172.21.15.46:0/1727239484 >> 172.21.15.46:0/1727239484 conn(0x7f51540fd600 msgr2=0x7f51540ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 -- 172.21.15.46:0/1727239484 shutdown_connections 2024-09-10T04:09:57.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.743+0000 7f515c3b9640 1 -- 172.21.15.46:0/1727239484 wait complete. 2024-09-10T04:09:57.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f515c3b9640 1 Processor -- start 2024-09-10T04:09:57.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f515c3b9640 1 -- start start 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f515c3b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154101900 0x7f515419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f515c3b9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154107900 0x7f515419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f515c3b9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f515410ee30 con 0x7f5154107900 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f515c3b9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f515410ecb0 con 0x7f5154101900 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f5159954640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154107900 0x7f515419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f5159954640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154107900 0x7f515419dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33118/0 (socket says 172.21.15.46:33118) 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f5159954640 1 -- 172.21.15.46:0/557808278 learned_addr learned my addr 172.21.15.46:0/557808278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f515a155640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154101900 0x7f515419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f5159954640 1 -- 172.21.15.46:0/557808278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154101900 msgr2=0x7f515419d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f5159954640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154101900 0x7f515419d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.744+0000 7f5159954640 1 -- 172.21.15.46:0/557808278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5154197840 con 0x7f5154107900 2024-09-10T04:09:57.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.745+0000 7f515a155640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154101900 0x7f515419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:09:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.745+0000 7f5159954640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154107900 0x7f515419dc00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5148031460 tx=0x7f51480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.745+0000 7f51477fe640 1 -- 172.21.15.46:0/557808278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f514803f070 con 0x7f5154107900 2024-09-10T04:09:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.745+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5154197ad0 con 0x7f5154107900 2024-09-10T04:09:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.745+0000 7f51477fe640 1 -- 172.21.15.46:0/557808278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5148004cf0 con 0x7f5154107900 2024-09-10T04:09:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.745+0000 7f51477fe640 1 -- 172.21.15.46:0/557808278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5148035030 con 0x7f5154107900 2024-09-10T04:09:57.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.745+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5154198000 con 0x7f5154107900 2024-09-10T04:09:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.746+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5154103040 con 0x7f5154107900 2024-09-10T04:09:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.747+0000 7f51477fe640 1 -- 172.21.15.46:0/557808278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5148042070 con 0x7f5154107900 2024-09-10T04:09:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.747+0000 7f51477fe640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f514007abf0 0x7f514007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.748+0000 7f515a155640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f514007abf0 0x7f514007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.748+0000 7f51477fe640 1 -- 172.21.15.46:0/557808278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f51480be700 con 0x7f5154107900 2024-09-10T04:09:57.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.748+0000 7f515a155640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f514007abf0 0x7f514007d0b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5150009930 tx=0x7f5150008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:57.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:57.752+0000 7f51477fe640 1 -- 172.21.15.46:0/557808278 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5148089ef0 con 0x7f5154107900 2024-09-10T04:09:58.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.017+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f5154198d70 con 0x7f5154107900 2024-09-10T04:09:58.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.017+0000 7f51477fe640 1 -- 172.21.15.46:0/557808278 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+733 (secure 0 0 0) 0x7f514808eda0 con 0x7f5154107900 2024-09-10T04:09:58.019 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:09:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:09:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:09:58.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 10, 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:09:58.020 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:09:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f514007abf0 msgr2=0x7f514007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f514007abf0 0x7f514007d0b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5150009930 tx=0x7f5150008040 comp rx=0 tx=0).stop 2024-09-10T04:09:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154107900 msgr2=0x7f515419dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154107900 0x7f515419dc00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5148031460 tx=0x7f51480048f0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 shutdown_connections 2024-09-10T04:09:58.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f514007abf0 0x7f514007d0b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5154107900 0x7f515419dc00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 --2- 172.21.15.46:0/557808278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154101900 0x7f515419d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 >> 172.21.15.46:0/557808278 conn(0x7f51540fd600 msgr2=0x7f51540fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:58.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 shutdown_connections 2024-09-10T04:09:58.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.019+0000 7f515c3b9640 1 -- 172.21.15.46:0/557808278 wait complete. 2024-09-10T04:09:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 -- 172.21.15.46:0/1708648862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c1033e0 msgr2=0x7f580c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 --2- 172.21.15.46:0/1708648862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c1033e0 0x7f580c10b8d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5800009880 tx=0x7f580002a530 comp rx=0 tx=0).stop 2024-09-10T04:09:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 -- 172.21.15.46:0/1708648862 shutdown_connections 2024-09-10T04:09:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 --2- 172.21.15.46:0/1708648862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c1033e0 0x7f580c10b8d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 --2- 172.21.15.46:0/1708648862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f580c102ac0 0x7f580c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 -- 172.21.15.46:0/1708648862 >> 172.21.15.46:0/1708648862 conn(0x7f580c0fc520 msgr2=0x7f580c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 -- 172.21.15.46:0/1708648862 shutdown_connections 2024-09-10T04:09:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.106+0000 7f5810a8c640 1 -- 172.21.15.46:0/1708648862 wait complete. 2024-09-10T04:09:58.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f5810a8c640 1 Processor -- start 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f5810a8c640 1 -- start start 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f5810a8c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c102ac0 0x7f580c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f5810a8c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f580c1033e0 0x7f580c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f5810a8c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10edf0 con 0x7f580c102ac0 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f5810a8c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10ec70 con 0x7f580c1033e0 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f580a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c102ac0 0x7f580c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.107+0000 7f580a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c102ac0 0x7f580c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33134/0 (socket says 172.21.15.46:33134) 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f580a59c640 1 -- 172.21.15.46:0/668323452 learned_addr learned my addr 172.21.15.46:0/668323452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f5809d9b640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f580c1033e0 0x7f580c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f580a59c640 1 -- 172.21.15.46:0/668323452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f580c1033e0 msgr2=0x7f580c100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f580a59c640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f580c1033e0 0x7f580c100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f580a59c640 1 -- 172.21.15.46:0/668323452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c1011a0 con 0x7f580c102ac0 2024-09-10T04:09:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f580a59c640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c102ac0 0x7f580c100720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c8f0 tx=0x7f57fc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f57fb7fe640 1 -- 172.21.15.46:0/668323452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f57fc0079b0 con 0x7f580c102ac0 2024-09-10T04:09:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f57fb7fe640 1 -- 172.21.15.46:0/668323452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57fc0054e0 con 0x7f580c102ac0 2024-09-10T04:09:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c101490 con 0x7f580c102ac0 2024-09-10T04:09:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f57fb7fe640 1 -- 172.21.15.46:0/668323452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f57fc004dd0 con 0x7f580c102ac0 2024-09-10T04:09:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.108+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c1a3e80 con 0x7f580c102ac0 2024-09-10T04:09:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.109+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580c06aa90 con 0x7f580c102ac0 2024-09-10T04:09:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.110+0000 7f57fb7fe640 1 -- 172.21.15.46:0/668323452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f57fc005020 con 0x7f580c102ac0 2024-09-10T04:09:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.110+0000 7f57fb7fe640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f57f407abf0 0x7f57f407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.111+0000 7f5809d9b640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f57f407abf0 0x7f57f407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.111+0000 7f57fb7fe640 1 -- 172.21.15.46:0/668323452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f57fc09da50 con 0x7f580c102ac0 2024-09-10T04:09:58.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.111+0000 7f5809d9b640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f57f407abf0 0x7f57f407d0b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f580c1021b0 tx=0x7f5800031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:58.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.115+0000 7f57fb7fe640 1 -- 172.21.15.46:0/668323452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f57fc014030 con 0x7f580c102ac0 2024-09-10T04:09:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:58 smithi070 ceph-mon[79774]: Upgrade: Updating crash.smithi046 (1/2) 2024-09-10T04:09:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:58 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:58 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi046", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-10T04:09:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:58 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:09:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:58 smithi070 ceph-mon[79774]: Deploying daemon crash.smithi046 on smithi046 2024-09-10T04:09:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:58 smithi070 ceph-mon[79774]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:58.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:58 smithi070 ceph-mon[79774]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:58.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:58 smithi046 ceph-mon[109506]: Upgrade: Updating crash.smithi046 (1/2) 2024-09-10T04:09:58.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:58 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:09:58.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:58 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi046", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-10T04:09:58.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:58 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:09:58.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:58 smithi046 ceph-mon[109506]: Deploying daemon crash.smithi046 on smithi046 2024-09-10T04:09:58.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:58 smithi046 ceph-mon[109506]: from='client.34100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:58.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:58 smithi046 ceph-mon[109506]: from='client.34104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.342+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f580c105f60 con 0x7f580c102ac0 2024-09-10T04:09:58.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.343+0000 7f57fb7fe640 1 -- 172.21.15.46:0/668323452 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7f57fc0692c0 con 0x7f580c102ac0 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:09:58.345 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:09:58.346 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:09:58.347 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:09:58.348 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:09:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:09:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f57f407abf0 msgr2=0x7f57f407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f57f407abf0 0x7f57f407d0b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f580c1021b0 tx=0x7f5800031400 comp rx=0 tx=0).stop 2024-09-10T04:09:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c102ac0 msgr2=0x7f580c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c102ac0 0x7f580c100720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f57fc00c8f0 tx=0x7f57fc00cdb0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 shutdown_connections 2024-09-10T04:09:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f57f407abf0 0x7f57f407d0b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f580c1033e0 0x7f580c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 --2- 172.21.15.46:0/668323452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f580c102ac0 0x7f580c100720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 >> 172.21.15.46:0/668323452 conn(0x7f580c0fc520 msgr2=0x7f580c108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 shutdown_connections 2024-09-10T04:09:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.345+0000 7f5810a8c640 1 -- 172.21.15.46:0/668323452 wait complete. 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 -- 172.21.15.46:0/974446833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810cf10 msgr2=0x7fd018110f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 --2- 172.21.15.46:0/974446833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810cf10 0x7fd018110f60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd000009880 tx=0x7fd00002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 -- 172.21.15.46:0/974446833 shutdown_connections 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 --2- 172.21.15.46:0/974446833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810cf10 0x7fd018110f60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 --2- 172.21.15.46:0/974446833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd01810c560 0x7fd01810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 -- 172.21.15.46:0/974446833 >> 172.21.15.46:0/974446833 conn(0x7fd01806c5b0 msgr2=0x7fd01806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 -- 172.21.15.46:0/974446833 shutdown_connections 2024-09-10T04:09:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 -- 172.21.15.46:0/974446833 wait complete. 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 Processor -- start 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.431+0000 7fd01dc62640 1 -- start start 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd01dc62640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810c560 0x7fd0181a23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd01dc62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd01810cf10 0x7fd0181a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd01dc62640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0181197e0 con 0x7fd01810c560 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd01dc62640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd018119660 con 0x7fd01810cf10 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd0177fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810c560 0x7fd0181a23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd0177fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810c560 0x7fd0181a23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33154/0 (socket says 172.21.15.46:33154) 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd0177fe640 1 -- 172.21.15.46:0/881427876 learned_addr learned my addr 172.21.15.46:0/881427876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd00effd640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd01810cf10 0x7fd0181a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd0177fe640 1 -- 172.21.15.46:0/881427876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd01810cf10 msgr2=0x7fd0181a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd0177fe640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd01810cf10 0x7fd0181a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.432+0000 7fd0177fe640 1 -- 172.21.15.46:0/881427876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0181a6c80 con 0x7fd01810c560 2024-09-10T04:09:58.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.433+0000 7fd0177fe640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810c560 0x7fd0181a23b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d6f0 tx=0x7fd00800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.433+0000 7fd0157fa640 1 -- 172.21.15.46:0/881427876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd008017070 con 0x7fd01810c560 2024-09-10T04:09:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.433+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0181a6f70 con 0x7fd01810c560 2024-09-10T04:09:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.433+0000 7fd0157fa640 1 -- 172.21.15.46:0/881427876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd00800de00 con 0x7fd01810c560 2024-09-10T04:09:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.433+0000 7fd0157fa640 1 -- 172.21.15.46:0/881427876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd008004bb0 con 0x7fd01810c560 2024-09-10T04:09:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.433+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0181a7500 con 0x7fd01810c560 2024-09-10T04:09:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.434+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01810da40 con 0x7fd01810c560 2024-09-10T04:09:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.434+0000 7fd0157fa640 1 -- 172.21.15.46:0/881427876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd008004e60 con 0x7fd01810c560 2024-09-10T04:09:58.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.435+0000 7fd0157fa640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd00407abf0 0x7fd00407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.435+0000 7fd0157fa640 1 -- 172.21.15.46:0/881427876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7fd00809d770 con 0x7fd01810c560 2024-09-10T04:09:58.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.435+0000 7fd00effd640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd00407abf0 0x7fd00407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.436+0000 7fd00effd640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd00407abf0 0x7fd00407d0b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd01810be20 tx=0x7fd000031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.440+0000 7fd0157fa640 1 -- 172.21.15.46:0/881427876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd008068f60 con 0x7fd01810c560 2024-09-10T04:09:58.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.655+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd0181a77b0 con 0x7fd00407abf0 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.657+0000 7fd0157fa640 1 -- 172.21.15.46:0/881427876 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+385 (secure 0 0 0) 0x7fd0181a77b0 con 0x7fd00407abf0 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:09:58.659 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:09:58.660 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:09:58.660 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "4/23 daemons upgraded", 2024-09-10T04:09:58.660 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading crash daemons", 2024-09-10T04:09:58.660 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:09:58.660 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:09:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd00407abf0 msgr2=0x7fd00407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd00407abf0 0x7fd00407d0b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd01810be20 tx=0x7fd000031400 comp rx=0 tx=0).stop 2024-09-10T04:09:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810c560 msgr2=0x7fd0181a23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810c560 0x7fd0181a23b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd00800d6f0 tx=0x7fd00800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 shutdown_connections 2024-09-10T04:09:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd00407abf0 0x7fd00407d0b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd01810cf10 0x7fd0181a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 --2- 172.21.15.46:0/881427876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01810c560 0x7fd0181a23b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 >> 172.21.15.46:0/881427876 conn(0x7fd01806c5b0 msgr2=0x7fd01810a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.659+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 shutdown_connections 2024-09-10T04:09:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.660+0000 7fd01dc62640 1 -- 172.21.15.46:0/881427876 wait complete. 2024-09-10T04:09:58.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 -- 172.21.15.46:0/3907873357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 msgr2=0x7f879c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3907873357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c102f10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8798009880 tx=0x7f879802a530 comp rx=0 tx=0).stop 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 -- 172.21.15.46:0/3907873357 shutdown_connections 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3907873357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f879c103450 0x7f879c10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3907873357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c102f10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 -- 172.21.15.46:0/3907873357 >> 172.21.15.46:0/3907873357 conn(0x7f879c0fc550 msgr2=0x7f879c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 -- 172.21.15.46:0/3907873357 shutdown_connections 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.746+0000 7f87a35e5640 1 -- 172.21.15.46:0/3907873357 wait complete. 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a35e5640 1 Processor -- start 2024-09-10T04:09:58.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a35e5640 1 -- start start 2024-09-10T04:09:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a35e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a35e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f879c103450 0x7f879c19db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a35e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f879c10ede0 con 0x7f879c102b30 2024-09-10T04:09:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a35e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f879c10ec60 con 0x7f879c103450 2024-09-10T04:09:58.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a1381640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a1381640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33168/0 (socket says 172.21.15.46:33168) 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a1381640 1 -- 172.21.15.46:0/3750698604 learned_addr learned my addr 172.21.15.46:0/3750698604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.747+0000 7f87a0b80640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f879c103450 0x7f879c19db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87a1381640 1 -- 172.21.15.46:0/3750698604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f879c103450 msgr2=0x7f879c19db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87a1381640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f879c103450 0x7f879c19db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87a1381640 1 -- 172.21.15.46:0/3750698604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c1977d0 con 0x7f879c102b30 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87a1381640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c19d650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8798004f90 tx=0x7f87980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87927fc640 1 -- 172.21.15.46:0/3750698604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f879803f070 con 0x7f879c102b30 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c197a60 con 0x7f879c102b30 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87927fc640 1 -- 172.21.15.46:0/3750698604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8798004ac0 con 0x7f879c102b30 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87927fc640 1 -- 172.21.15.46:0/3750698604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8798035030 con 0x7f879c102b30 2024-09-10T04:09:58.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.748+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c197f90 con 0x7f879c102b30 2024-09-10T04:09:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.749+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c06a500 con 0x7f879c102b30 2024-09-10T04:09:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.750+0000 7f87927fc640 1 -- 172.21.15.46:0/3750698604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8798042070 con 0x7f879c102b30 2024-09-10T04:09:58.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.750+0000 7f87927fc640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f878807abf0 0x7f878807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:09:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.750+0000 7f87927fc640 1 -- 172.21.15.46:0/3750698604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6070+0+0 (secure 0 0 0) 0x7f87980be480 con 0x7f879c102b30 2024-09-10T04:09:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.750+0000 7f87a0b80640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f878807abf0 0x7f878807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:09:58.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.751+0000 7f87a0b80640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f878807abf0 0x7f878807d0b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f879c108900 tx=0x7f878c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:09:58.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:58.755+0000 7f87927fc640 1 -- 172.21.15.46:0/3750698604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8798089c70 con 0x7f879c102b30 2024-09-10T04:09:59.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.047+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f879c105f90 con 0x7f879c102b30 2024-09-10T04:09:59.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.048+0000 7f87927fc640 1 -- 172.21.15.46:0/3750698604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7f879808eb20 con 0x7f879c102b30 2024-09-10T04:09:59.050 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:59.050 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:09:59.050 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:09:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f878807abf0 msgr2=0x7f878807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f878807abf0 0x7f878807d0b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f879c108900 tx=0x7f878c005f90 comp rx=0 tx=0).stop 2024-09-10T04:09:59.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 msgr2=0x7f879c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c19d650 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8798004f90 tx=0x7f87980046c0 comp rx=0 tx=0).stop 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 shutdown_connections 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f878807abf0 0x7f878807d0b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f879c103450 0x7f879c19db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 --2- 172.21.15.46:0/3750698604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f879c102b30 0x7f879c19d650 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 >> 172.21.15.46:0/3750698604 conn(0x7f879c0fc550 msgr2=0x7f879c108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 shutdown_connections 2024-09-10T04:09:59.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:09:59.050+0000 7f87a35e5640 1 -- 172.21.15.46:0/3750698604 wait complete. 2024-09-10T04:09:59.274 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:59 smithi046 ceph-mon[109506]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:59.274 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:59 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/557808278' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:09:59.275 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:59 smithi046 ceph-mon[109506]: pgmap v96: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:09:59.275 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:09:59 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/668323452' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:09:59.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:59 smithi070 ceph-mon[79774]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:09:59.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:59 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/557808278' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:09:59.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:59 smithi070 ceph-mon[79774]: pgmap v96: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:09:59.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:09:59 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/668323452' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:10:00.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:00 smithi070 ceph-mon[79774]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:00.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:00 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3750698604' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:10:00.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:00 smithi070 ceph-mon[79774]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:10:00.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:00 smithi046 ceph-mon[109506]: from='client.34120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:00 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3750698604' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:10:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:00 smithi046 ceph-mon[109506]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:10:01.280 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:01 smithi070 ceph-mon[79774]: pgmap v97: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:01.281 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:01.281 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:01.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:01 smithi046 ceph-mon[109506]: pgmap v97: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:01.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:01.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:03.662 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:03 smithi070 ceph-mon[79774]: pgmap v98: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:03.662 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:03 smithi070 ceph-mon[79774]: Upgrade: Updating crash.smithi070 (2/2) 2024-09-10T04:10:03.662 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:03 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:03.662 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:03 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-10T04:10:03.662 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:03 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:03.662 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:03 smithi070 ceph-mon[79774]: Deploying daemon crash.smithi070 on smithi070 2024-09-10T04:10:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:03 smithi046 ceph-mon[109506]: pgmap v98: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:03 smithi046 ceph-mon[109506]: Upgrade: Updating crash.smithi070 (2/2) 2024-09-10T04:10:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:03 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:03 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi070", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-10T04:10:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:03 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:03 smithi046 ceph-mon[109506]: Deploying daemon crash.smithi070 on smithi070 2024-09-10T04:10:05.574 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:05 smithi070 ceph-mon[79774]: pgmap v99: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:05.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:05 smithi046 ceph-mon[109506]: pgmap v99: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:06 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:06 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:06 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:06 smithi070 ceph-mon[79774]: pgmap v100: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:06 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:06 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:06 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:06 smithi046 ceph-mon[109506]: pgmap v100: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:08.156 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:09 smithi070 ceph-mon[79774]: pgmap v101: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:09 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:09 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:09.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:09 smithi046 ceph-mon[109506]: pgmap v101: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:09.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:09 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:09.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:09 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:10 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:10 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:10.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:10 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:10.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:10 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.554 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:11 smithi046 ceph-mon[109506]: pgmap v102: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:11.554 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.554 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.554 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.554 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:11 smithi070 ceph-mon[79774]: pgmap v102: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:13 smithi046 ceph-mon[109506]: pgmap v103: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:13 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:13.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:13 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:13 smithi070 ceph-mon[79774]: pgmap v103: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:13 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:13 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: pgmap v104: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:15.594 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all crash 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi046"} : dispatch 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi046"}]': finished 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"} : dispatch 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]': finished 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-10T04:10:15.595 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:15 smithi070 ceph-mon[79774]: Upgrade: osd.0 is safe to restart 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: pgmap v104: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:15.760 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all crash 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi046"} : dispatch 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi046"}]': finished 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"} : dispatch 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi070"}]': finished 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-10T04:10:15.761 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:15 smithi046 ceph-mon[109506]: Upgrade: osd.0 is safe to restart 2024-09-10T04:10:16.800 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:16 smithi070 ceph-mon[79774]: Upgrade: Updating osd.0 2024-09-10T04:10:16.800 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:16.800 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T04:10:16.800 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:16.800 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:16 smithi070 ceph-mon[79774]: Deploying daemon osd.0 on smithi046 2024-09-10T04:10:16.800 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:16 smithi070 ceph-mon[79774]: pgmap v105: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:16.940 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:16 smithi046 ceph-mon[109506]: Upgrade: Updating osd.0 2024-09-10T04:10:16.940 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:16.940 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T04:10:16.940 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:16.940 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:16 smithi046 ceph-mon[109506]: Deploying daemon osd.0 on smithi046 2024-09-10T04:10:16.941 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:16 smithi046 ceph-mon[109506]: pgmap v105: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:19.194 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:19 smithi046 ceph-mon[109506]: pgmap v106: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:19.195 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:18 smithi046 systemd[1]: Stopping Ceph osd.0 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:10:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:19 smithi070 ceph-mon[79774]: pgmap v106: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:19.586 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:19 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[48974]: 2024-09-10T04:10:19.193+0000 7f68cccf7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:10:19.587 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:19 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[48974]: 2024-09-10T04:10:19.193+0000 7f68cccf7640 -1 osd.0 45 *** Got signal Terminated *** 2024-09-10T04:10:19.587 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:19 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[48974]: 2024-09-10T04:10:19.193+0000 7f68cccf7640 -1 osd.0 45 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:10:20.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:20 smithi070 ceph-mon[79774]: osd.0 marked itself down and dead 2024-09-10T04:10:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:20 smithi046 ceph-mon[109506]: osd.0 marked itself down and dead 2024-09-10T04:10:20.961 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:20 smithi046 podman[117799]: 2024-09-10 04:10:20.668037745 +0000 UTC m=+1.603425431 container died ebeeeb3fce9b1d700a63a71a781f06dee33a91958f84c536ba3561698a277b05 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-10T04:10:21.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:21 smithi046 ceph-mon[109506]: pgmap v107: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:21.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:21 smithi046 ceph-mon[109506]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:10:21.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:21 smithi046 ceph-mon[109506]: osdmap e46: 6 total, 5 up, 6 in 2024-09-10T04:10:21.337 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:21 smithi046 podman[117799]: 2024-09-10 04:10:21.109635975 +0000 UTC m=+2.045023645 container cleanup ebeeeb3fce9b1d700a63a71a781f06dee33a91958f84c536ba3561698a277b05 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-10T04:10:21.337 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:21 smithi046 bash[117799]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0 2024-09-10T04:10:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:21 smithi070 ceph-mon[79774]: pgmap v107: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:21 smithi070 ceph-mon[79774]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:10:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:21 smithi070 ceph-mon[79774]: osdmap e46: 6 total, 5 up, 6 in 2024-09-10T04:10:21.722 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:21 smithi046 podman[117814]: 2024-09-10 04:10:21.402731509 +0000 UTC m=+0.734588559 container remove ebeeeb3fce9b1d700a63a71a781f06dee33a91958f84c536ba3561698a277b05 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T04:10:22.048 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:21 smithi046 podman[117907]: 2024-09-10 04:10:21.625624431 +0000 UTC m=+0.016791948 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:10:22.049 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:21 smithi046 podman[117907]: 2024-09-10 04:10:21.778791318 +0000 UTC m=+0.169958827 container create cb6752998a240f2823e92f71dd30b7904da264df42664a211a667f61de83fffe (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T04:10:22.049 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 podman[117907]: 2024-09-10 04:10:22.048091961 +0000 UTC m=+0.439259469 container init cb6752998a240f2823e92f71dd30b7904da264df42664a211a667f61de83fffe (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T04:10:22.315 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:22 smithi046 ceph-mon[109506]: osdmap e47: 6 total, 5 up, 6 in 2024-09-10T04:10:22.316 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 podman[117907]: 2024-09-10 04:10:22.051540607 +0000 UTC m=+0.442708114 container start cb6752998a240f2823e92f71dd30b7904da264df42664a211a667f61de83fffe (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:10:22.316 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 podman[117907]: 2024-09-10 04:10:22.314780708 +0000 UTC m=+0.705948216 container attach cb6752998a240f2823e92f71dd30b7904da264df42664a211a667f61de83fffe (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T04:10:22.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:22 smithi070 ceph-mon[79774]: osdmap e47: 6 total, 5 up, 6 in 2024-09-10T04:10:22.586 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 podman[117907]: 2024-09-10 04:10:22.315048233 +0000 UTC m=+0.706215741 container died cb6752998a240f2823e92f71dd30b7904da264df42664a211a667f61de83fffe (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T04:10:23.188 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:23 smithi046 ceph-mon[109506]: pgmap v110: 65 pgs: 9 stale+active+clean, 56 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T04:10:23.188 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:23.188 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:23.188 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 podman[117929]: 2024-09-10 04:10:22.935084691 +0000 UTC m=+0.639434496 container remove cb6752998a240f2823e92f71dd30b7904da264df42664a211a667f61de83fffe (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:10:23.188 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-10T04:10:23.189 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 systemd[1]: Stopped Ceph osd.0 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:10:23.189 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:22 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.0.service: Consumed 55.093s CPU time. 2024-09-10T04:10:23.189 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 systemd[1]: Starting Ceph osd.0 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:10:23.483 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 podman[118027]: 2024-09-10 04:10:23.305798484 +0000 UTC m=+0.017436874 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:10:23.483 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 podman[118027]: 2024-09-10 04:10:23.436473833 +0000 UTC m=+0.148112207 container create ce723901f4645528c3ea995f30d8af9a4201f7130bae198ca1b7d3313d872b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:10:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:23 smithi070 ceph-mon[79774]: pgmap v110: 65 pgs: 9 stale+active+clean, 56 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T04:10:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:23.773 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 podman[118027]: 2024-09-10 04:10:23.672375394 +0000 UTC m=+0.384013784 container init ce723901f4645528c3ea995f30d8af9a4201f7130bae198ca1b7d3313d872b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T04:10:23.773 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 podman[118027]: 2024-09-10 04:10:23.675760831 +0000 UTC m=+0.387399214 container start ce723901f4645528c3ea995f30d8af9a4201f7130bae198ca1b7d3313d872b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T04:10:23.773 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 podman[118027]: 2024-09-10 04:10:23.772269317 +0000 UTC m=+0.483907708 container attach ce723901f4645528c3ea995f30d8af9a4201f7130bae198ca1b7d3313d872b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:10:24.087 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:10:24.087 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 bash[118027]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:10:24.087 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:10:24.087 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:23 smithi046 bash[118027]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:10:24.576 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-mon[109506]: Health check failed: Degraded data redundancy: 58/312 objects degraded (18.590%), 20 pgs degraded (PG_DEGRADED) 2024-09-10T04:10:24.622 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:24 smithi070 ceph-mon[79774]: Health check failed: Degraded data redundancy: 58/312 objects degraded (18.590%), 20 pgs degraded (PG_DEGRADED) 2024-09-10T04:10:24.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T04:10:24.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 bash[118027]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T04:10:24.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T04:10:24.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 bash[118027]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T04:10:25.182 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T04:10:25.182 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 bash[118027]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T04:10:25.182 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T04:10:25.182 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 bash[118027]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T04:10:25.182 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-10T04:10:25.182 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 bash[118027]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-10T04:10:25.182 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T04:10:25.183 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 bash[118027]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T04:10:25.183 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate[118042]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-10T04:10:25.183 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 bash[118027]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-10T04:10:25.183 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:24 smithi046 podman[118027]: 2024-09-10 04:10:24.891999151 +0000 UTC m=+1.603637529 container died ce723901f4645528c3ea995f30d8af9a4201f7130bae198ca1b7d3313d872b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:10:25.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:25 smithi070 ceph-mon[79774]: pgmap v111: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:25.518 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:25 smithi046 ceph-mon[109506]: pgmap v111: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:25.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:25 smithi046 podman[118192]: 2024-09-10 04:10:25.517552658 +0000 UTC m=+0.615635349 container remove ce723901f4645528c3ea995f30d8af9a4201f7130bae198ca1b7d3313d872b20 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-activate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731) 2024-09-10T04:10:25.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:25 smithi046 podman[118252]: 2024-09-10 04:10:25.633907277 +0000 UTC m=+0.016572914 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:10:25.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:25 smithi046 podman[118252]: 2024-09-10 04:10:25.75187248 +0000 UTC m=+0.134538147 container create 5359fceff12da4da4dd517379ba1ee50b1cd396b9f648f58ac6fc4f7b76c80f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T04:10:26.141 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:25 smithi046 podman[118252]: 2024-09-10 04:10:25.987961718 +0000 UTC m=+0.370627362 container init 5359fceff12da4da4dd517379ba1ee50b1cd396b9f648f58ac6fc4f7b76c80f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:10:26.141 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:25 smithi046 podman[118252]: 2024-09-10 04:10:25.992251267 +0000 UTC m=+0.374916911 container start 5359fceff12da4da4dd517379ba1ee50b1cd396b9f648f58ac6fc4f7b76c80f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T04:10:26.141 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:26 smithi046 bash[118252]: 5359fceff12da4da4dd517379ba1ee50b1cd396b9f648f58ac6fc4f7b76c80f1 2024-09-10T04:10:26.141 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:26 smithi046 systemd[1]: Started Ceph osd.0 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:10:27.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:27 smithi046 ceph-mon[109506]: pgmap v112: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:27.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:27.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:27.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:27 smithi070 ceph-mon[79774]: pgmap v112: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:27.870 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:27 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[118266]: 2024-09-10T04:10:27.651+0000 7fe9faafe640 -1 Falling back to public interface 2024-09-10T04:10:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/2852447353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8107900 msgr2=0x7fd7a8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/2852447353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8107900 0x7fd7a8107ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd798009880 tx=0x7fd79802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/2852447353 shutdown_connections 2024-09-10T04:10:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/2852447353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a8101900 0x7fd7a8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/2852447353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8107900 0x7fd7a8107ce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/2852447353 >> 172.21.15.46:0/2852447353 conn(0x7fd7a80fd600 msgr2=0x7fd7a80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:29.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/2852447353 shutdown_connections 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/2852447353 wait complete. 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.140+0000 7fd7ae0ee640 1 Processor -- start 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7ae0ee640 1 -- start start 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7ae0ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8101900 0x7fd7a819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7ae0ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a8107900 0x7fd7a819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7ae0ee640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a810edd0 con 0x7fd7a8101900 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7ae0ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a810ec50 con 0x7fd7a8107900 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8101900 0x7fd7a819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8101900 0x7fd7a819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57888/0 (socket says 172.21.15.46:57888) 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7a77fe640 1 -- 172.21.15.46:0/4120371347 learned_addr learned my addr 172.21.15.46:0/4120371347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd79ffff640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a8107900 0x7fd7a819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7a77fe640 1 -- 172.21.15.46:0/4120371347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a8107900 msgr2=0x7fd7a819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7a77fe640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a8107900 0x7fd7a819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7a77fe640 1 -- 172.21.15.46:0/4120371347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a81977a0 con 0x7fd7a8101900 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.141+0000 7fd7a77fe640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8101900 0x7fd7a819d620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd798031460 tx=0x7fd798004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.142+0000 7fd7a57fa640 1 -- 172.21.15.46:0/4120371347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd79803f070 con 0x7fd7a8101900 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.142+0000 7fd7a57fa640 1 -- 172.21.15.46:0/4120371347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd7980035f0 con 0x7fd7a8101900 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.142+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a8197a30 con 0x7fd7a8101900 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.142+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a8197f60 con 0x7fd7a8101900 2024-09-10T04:10:29.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.142+0000 7fd7a57fa640 1 -- 172.21.15.46:0/4120371347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd798035030 con 0x7fd7a8101900 2024-09-10T04:10:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.143+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a8103040 con 0x7fd7a8101900 2024-09-10T04:10:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.144+0000 7fd7a57fa640 1 -- 172.21.15.46:0/4120371347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd798042070 con 0x7fd7a8101900 2024-09-10T04:10:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.144+0000 7fd7a57fa640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd79407abf0 0x7fd79407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.144+0000 7fd7a57fa640 1 -- 172.21.15.46:0/4120371347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6070+0+0 (secure 0 0 0) 0x7fd7980be4c0 con 0x7fd7a8101900 2024-09-10T04:10:29.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.145+0000 7fd79ffff640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd79407abf0 0x7fd79407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.145+0000 7fd79ffff640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd79407abf0 0x7fd79407d0b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80ffbe0 tx=0x7fd790005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:29.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.149+0000 7fd7a57fa640 1 -- 172.21.15.46:0/4120371347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd798089cb0 con 0x7fd7a8101900 2024-09-10T04:10:29.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.368+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd7a81982d0 con 0x7fd79407abf0 2024-09-10T04:10:29.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.369+0000 7fd7a57fa640 1 -- 172.21.15.46:0/4120371347 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fd7a81982d0 con 0x7fd79407abf0 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd79407abf0 msgr2=0x7fd79407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd79407abf0 0x7fd79407d0b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80ffbe0 tx=0x7fd790005f90 comp rx=0 tx=0).stop 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8101900 msgr2=0x7fd7a819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8101900 0x7fd7a819d620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd798031460 tx=0x7fd798004530 comp rx=0 tx=0).stop 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 shutdown_connections 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd79407abf0 0x7fd79407d0b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7a8107900 0x7fd7a819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 --2- 172.21.15.46:0/4120371347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7a8101900 0x7fd7a819d620 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 >> 172.21.15.46:0/4120371347 conn(0x7fd7a80fd600 msgr2=0x7fd7a80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:29.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 shutdown_connections 2024-09-10T04:10:29.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.372+0000 7fd7ae0ee640 1 -- 172.21.15.46:0/4120371347 wait complete. 2024-09-10T04:10:29.382 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:10:29.453 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:29 smithi046 ceph-mon[109506]: pgmap v113: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:29.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.459+0000 7fea7e535640 1 -- 172.21.15.46:0/1827529887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 msgr2=0x7fea7810b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.459+0000 7fea7e535640 1 --2- 172.21.15.46:0/1827529887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea7810b930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009880 tx=0x7fea6c02a530 comp rx=0 tx=0).stop 2024-09-10T04:10:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.459+0000 7fea7e535640 1 -- 172.21.15.46:0/1827529887 shutdown_connections 2024-09-10T04:10:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.459+0000 7fea7e535640 1 --2- 172.21.15.46:0/1827529887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea7810b930 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.459+0000 7fea7e535640 1 --2- 172.21.15.46:0/1827529887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea78102b20 0x7fea78102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.459+0000 7fea7e535640 1 -- 172.21.15.46:0/1827529887 >> 172.21.15.46:0/1827529887 conn(0x7fea780fc580 msgr2=0x7fea780fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.460+0000 7fea7e535640 1 -- 172.21.15.46:0/1827529887 shutdown_connections 2024-09-10T04:10:29.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.460+0000 7fea7e535640 1 -- 172.21.15.46:0/1827529887 wait complete. 2024-09-10T04:10:29.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.460+0000 7fea7e535640 1 Processor -- start 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.460+0000 7fea7e535640 1 -- start start 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea7e535640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea78102b20 0x7fea78100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea7e535640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea78100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea7e535640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea7810ee50 con 0x7fea78103440 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea7e535640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fea7810ecd0 con 0x7fea78102b20 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea777fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea78100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea777fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea78100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57904/0 (socket says 172.21.15.46:57904) 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea777fe640 1 -- 172.21.15.46:0/3385458635 learned_addr learned my addr 172.21.15.46:0/3385458635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea77fff640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea78102b20 0x7fea78100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea777fe640 1 -- 172.21.15.46:0/3385458635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea78102b20 msgr2=0x7fea78100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea777fe640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea78102b20 0x7fea78100780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea777fe640 1 -- 172.21.15.46:0/3385458635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea78101200 con 0x7fea78103440 2024-09-10T04:10:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.461+0000 7fea777fe640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea78100cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea6c031460 tx=0x7fea6c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.462+0000 7fea757fa640 1 -- 172.21.15.46:0/3385458635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea6c03f070 con 0x7fea78103440 2024-09-10T04:10:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.462+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea78101490 con 0x7fea78103440 2024-09-10T04:10:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.462+0000 7fea757fa640 1 -- 172.21.15.46:0/3385458635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fea6c0035f0 con 0x7fea78103440 2024-09-10T04:10:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.462+0000 7fea757fa640 1 -- 172.21.15.46:0/3385458635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fea6c035030 con 0x7fea78103440 2024-09-10T04:10:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.462+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea781a3e20 con 0x7fea78103440 2024-09-10T04:10:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.463+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7806ada0 con 0x7fea78103440 2024-09-10T04:10:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.464+0000 7fea757fa640 1 -- 172.21.15.46:0/3385458635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fea6c042070 con 0x7fea78103440 2024-09-10T04:10:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.464+0000 7fea757fa640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fea6007abf0 0x7fea6007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.464+0000 7fea757fa640 1 -- 172.21.15.46:0/3385458635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6070+0+0 (secure 0 0 0) 0x7fea6c0be500 con 0x7fea78103440 2024-09-10T04:10:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.464+0000 7fea77fff640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fea6007abf0 0x7fea6007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.465+0000 7fea77fff640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fea6007abf0 0x7fea6007d0b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea68005fd0 tx=0x7fea6800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:29.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.469+0000 7fea757fa640 1 -- 172.21.15.46:0/3385458635 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fea6c089cf0 con 0x7fea78103440 2024-09-10T04:10:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:29 smithi070 ceph-mon[79774]: pgmap v113: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 3 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:29.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.678+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fea78101cb0 con 0x7fea6007abf0 2024-09-10T04:10:29.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.679+0000 7fea757fa640 1 -- 172.21.15.46:0/3385458635 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7fea78101cb0 con 0x7fea6007abf0 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fea6007abf0 msgr2=0x7fea6007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fea6007abf0 0x7fea6007d0b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fea68005fd0 tx=0x7fea6800b390 comp rx=0 tx=0).stop 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 msgr2=0x7fea78100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea78100cc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fea6c031460 tx=0x7fea6c004530 comp rx=0 tx=0).stop 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 shutdown_connections 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fea6007abf0 0x7fea6007d0b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea78103440 0x7fea78100cc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 --2- 172.21.15.46:0/3385458635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea78102b20 0x7fea78100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 >> 172.21.15.46:0/3385458635 conn(0x7fea780fc580 msgr2=0x7fea78108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 shutdown_connections 2024-09-10T04:10:29.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.682+0000 7fea7e535640 1 -- 172.21.15.46:0/3385458635 wait complete. 2024-09-10T04:10:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 -- 172.21.15.46:0/3419648366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 msgr2=0x7f3384101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 --2- 172.21.15.46:0/3419648366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f3384101d60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3378009880 tx=0x7f337802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 -- 172.21.15.46:0/3419648366 shutdown_connections 2024-09-10T04:10:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 --2- 172.21.15.46:0/3419648366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f3384101d60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 --2- 172.21.15.46:0/3419648366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3384107900 0x7f3384107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 -- 172.21.15.46:0/3419648366 >> 172.21.15.46:0/3419648366 conn(0x7f33840fd600 msgr2=0x7f33840ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 -- 172.21.15.46:0/3419648366 shutdown_connections 2024-09-10T04:10:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.771+0000 7f338a4d7640 1 -- 172.21.15.46:0/3419648366 wait complete. 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f338a4d7640 1 Processor -- start 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f338a4d7640 1 -- start start 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f338a4d7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f33840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f338a4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3384107900 0x7f3384074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f338a4d7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f338410edd0 con 0x7f3384101900 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f338a4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f338410ec50 con 0x7f3384107900 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f3383fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f33840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f3383fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f33840744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57922/0 (socket says 172.21.15.46:57922) 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f3383fff640 1 -- 172.21.15.46:0/109670019 learned_addr learned my addr 172.21.15.46:0/109670019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.772+0000 7f33837fe640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3384107900 0x7f3384074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f3383fff640 1 -- 172.21.15.46:0/109670019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3384107900 msgr2=0x7f3384074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f3383fff640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3384107900 0x7f3384074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f3383fff640 1 -- 172.21.15.46:0/109670019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3384074f70 con 0x7f3384101900 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f33837fe640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3384107900 0x7f3384074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f3383fff640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f33840744f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f337400e970 tx=0x7f337400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f33817fa640 1 -- 172.21.15.46:0/109670019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f337400cac0 con 0x7f3384101900 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f33817fa640 1 -- 172.21.15.46:0/109670019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33740054e0 con 0x7f3384101900 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3384075260 con 0x7f3384101900 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f33817fa640 1 -- 172.21.15.46:0/109670019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3374010430 con 0x7f3384101900 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.773+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33841a3e80 con 0x7f3384101900 2024-09-10T04:10:29.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.774+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3384103040 con 0x7f3384101900 2024-09-10T04:10:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.775+0000 7f33817fa640 1 -- 172.21.15.46:0/109670019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3374004e80 con 0x7f3384101900 2024-09-10T04:10:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.775+0000 7f33817fa640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f336c07abf0 0x7f336c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.776+0000 7f33817fa640 1 -- 172.21.15.46:0/109670019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6070+0+0 (secure 0 0 0) 0x7f337409e2c0 con 0x7f3384101900 2024-09-10T04:10:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.776+0000 7f33837fe640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f336c07abf0 0x7f336c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.776+0000 7f33837fe640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f336c07abf0 0x7f336c07d0b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3384075f80 tx=0x7f3378031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:29.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.780+0000 7f33817fa640 1 -- 172.21.15.46:0/109670019 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3374069ab0 con 0x7f3384101900 2024-09-10T04:10:29.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.979+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3384062630 con 0x7f336c07abf0 2024-09-10T04:10:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.996+0000 7f33817fa640 1 -- 172.21.15.46:0/109670019 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f3384062630 con 0x7f336c07abf0 2024-09-10T04:10:29.997 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:10:29.997 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (2m) 19s ago 13m 20.8M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:10:29.997 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (4m) 19s ago 13m 8870k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:10:29.997 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (4m) 21s ago 12m 8539k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (29s) 19s ago 13m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (24s) 21s ago 12m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (100s) 19s ago 13m 75.0M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (8m) 19s ago 7m 90.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (8m) 19s ago 8m 146M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (8m) 21s ago 8m 26.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (7m) 21s ago 7m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (4m) 19s ago 14m 526M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (3m) 21s ago 12m 455M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (59s) 19s ago 14m 38.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (42s) 21s ago 12m 25.7M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (2m) 19s ago 13m 10.1M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (2m) 21s ago 12m 12.0M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:10:29.998 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 starting - - - 1217M 2024-09-10T04:10:29.999 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (10m) 19s ago 10m 227M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:10:29.999 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (10m) 19s ago 10m 180M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:10:29.999 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (9m) 21s ago 9m 236M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:10:29.999 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (9m) 21s ago 9m 210M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:10:29.999 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (9m) 21s ago 9m 196M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:10:29.999 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (2m) 19s ago 13m 51.4M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f336c07abf0 msgr2=0x7f336c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f336c07abf0 0x7f336c07d0b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3384075f80 tx=0x7f3378031400 comp rx=0 tx=0).stop 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 msgr2=0x7f33840744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f33840744f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f337400e970 tx=0x7f337400ed70 comp rx=0 tx=0).stop 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 shutdown_connections 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f336c07abf0 0x7f336c07d0b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3384107900 0x7f3384074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 --2- 172.21.15.46:0/109670019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3384101900 0x7f33840744f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 >> 172.21.15.46:0/109670019 conn(0x7f33840fd600 msgr2=0x7f33840fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:30.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 shutdown_connections 2024-09-10T04:10:30.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:29.999+0000 7f338a4d7640 1 -- 172.21.15.46:0/109670019 wait complete. 2024-09-10T04:10:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.085+0000 7f016f38c640 1 -- 172.21.15.46:0/2596416398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 msgr2=0x7f0168102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.085+0000 7f016f38c640 1 --2- 172.21.15.46:0/2596416398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f0168102f50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0164009880 tx=0x7f016402a530 comp rx=0 tx=0).stop 2024-09-10T04:10:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.085+0000 7f016f38c640 1 -- 172.21.15.46:0/2596416398 shutdown_connections 2024-09-10T04:10:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.085+0000 7f016f38c640 1 --2- 172.21.15.46:0/2596416398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0168103490 0x7f016810b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.085+0000 7f016f38c640 1 --2- 172.21.15.46:0/2596416398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f0168102f50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.085+0000 7f016f38c640 1 -- 172.21.15.46:0/2596416398 >> 172.21.15.46:0/2596416398 conn(0x7f01680fc550 msgr2=0x7f01680fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.086+0000 7f016f38c640 1 -- 172.21.15.46:0/2596416398 shutdown_connections 2024-09-10T04:10:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.086+0000 7f016f38c640 1 -- 172.21.15.46:0/2596416398 wait complete. 2024-09-10T04:10:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.086+0000 7f016f38c640 1 Processor -- start 2024-09-10T04:10:30.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.086+0000 7f016f38c640 1 -- start start 2024-09-10T04:10:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016f38c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f016819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016f38c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0168103490 0x7f016819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016f38c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f016810ede0 con 0x7f0168102b70 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016f38c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f016810ec60 con 0x7f0168103490 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016d128640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f016819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016d128640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f016819dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57942/0 (socket says 172.21.15.46:57942) 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016d128640 1 -- 172.21.15.46:0/4167914309 learned_addr learned my addr 172.21.15.46:0/4167914309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016c927640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0168103490 0x7f016819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016d128640 1 -- 172.21.15.46:0/4167914309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0168103490 msgr2=0x7f016819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016d128640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0168103490 0x7f016819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016d128640 1 -- 172.21.15.46:0/4167914309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01681a2880 con 0x7f0168102b70 2024-09-10T04:10:30.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016c927640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0168103490 0x7f016819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:10:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.087+0000 7f016d128640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f016819dfb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0164004f90 tx=0x7f01640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.088+0000 7f015e7fc640 1 -- 172.21.15.46:0/4167914309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f016403f070 con 0x7f0168102b70 2024-09-10T04:10:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.088+0000 7f015e7fc640 1 -- 172.21.15.46:0/4167914309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0164004b80 con 0x7f0168102b70 2024-09-10T04:10:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.088+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01681a2b10 con 0x7f0168102b70 2024-09-10T04:10:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.088+0000 7f015e7fc640 1 -- 172.21.15.46:0/4167914309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0164035030 con 0x7f0168102b70 2024-09-10T04:10:30.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.088+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a3040 con 0x7f0168102b70 2024-09-10T04:10:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.090+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016806a4c0 con 0x7f0168102b70 2024-09-10T04:10:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.090+0000 7f015e7fc640 1 -- 172.21.15.46:0/4167914309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0164042070 con 0x7f0168102b70 2024-09-10T04:10:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.091+0000 7f015e7fc640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f015407ac00 0x7f015407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.091+0000 7f015e7fc640 1 -- 172.21.15.46:0/4167914309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6070+0+0 (secure 0 0 0) 0x7f01640be6b0 con 0x7f0168102b70 2024-09-10T04:10:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.091+0000 7f016c927640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f015407ac00 0x7f015407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.091+0000 7f016c927640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f015407ac00 0x7f015407d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01680fef10 tx=0x7f0158005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:30.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.097+0000 7f015e7fc640 1 -- 172.21.15.46:0/4167914309 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0164089ea0 con 0x7f0168102b70 2024-09-10T04:10:30.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.360+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f0168106fd0 con 0x7f0168102b70 2024-09-10T04:10:30.361 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:30 smithi046 ceph-mon[109506]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.360+0000 7f015e7fc640 1 -- 172.21.15.46:0/4167914309 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+732 (secure 0 0 0) 0x7f016408ed50 con 0x7f0168102b70 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:10:30.362 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 5 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 9, 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:10:30.363 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:10:30.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f015407ac00 msgr2=0x7f015407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f015407ac00 0x7f015407d0c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f01680fef10 tx=0x7f0158005f90 comp rx=0 tx=0).stop 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 msgr2=0x7f016819dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f016819dfb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0164004f90 tx=0x7f01640046c0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 shutdown_connections 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f015407ac00 0x7f015407d0c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0168103490 0x7f016819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 --2- 172.21.15.46:0/4167914309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0168102b70 0x7f016819dfb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.363+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 >> 172.21.15.46:0/4167914309 conn(0x7f01680fc550 msgr2=0x7f0168100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.364+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 shutdown_connections 2024-09-10T04:10:30.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.364+0000 7f016f38c640 1 -- 172.21.15.46:0/4167914309 wait complete. 2024-09-10T04:10:30.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 -- 172.21.15.46:0/4034919189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010c560 msgr2=0x7fe39010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 --2- 172.21.15.46:0/4034919189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010c560 0x7fe39010c940 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009880 tx=0x7fe38c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 -- 172.21.15.46:0/4034919189 shutdown_connections 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 --2- 172.21.15.46:0/4034919189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39010cf10 0x7fe390110f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 --2- 172.21.15.46:0/4034919189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010c560 0x7fe39010c940 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 -- 172.21.15.46:0/4034919189 >> 172.21.15.46:0/4034919189 conn(0x7fe39006c5b0 msgr2=0x7fe39006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 -- 172.21.15.46:0/4034919189 shutdown_connections 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.453+0000 7fe398062640 1 -- 172.21.15.46:0/4034919189 wait complete. 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.454+0000 7fe398062640 1 Processor -- start 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.454+0000 7fe398062640 1 -- start start 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.454+0000 7fe398062640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39010c560 0x7fe3901a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.454+0000 7fe398062640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010cf10 0x7fe3901a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.454+0000 7fe398062640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901197c0 con 0x7fe39010cf10 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.454+0000 7fe398062640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe390119640 con 0x7fe39010c560 2024-09-10T04:10:30.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe3955fd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010cf10 0x7fe3901a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe395dfe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39010c560 0x7fe3901a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe3955fd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010cf10 0x7fe3901a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57952/0 (socket says 172.21.15.46:57952) 2024-09-10T04:10:30.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe3955fd640 1 -- 172.21.15.46:0/3320073964 learned_addr learned my addr 172.21.15.46:0/3320073964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe395dfe640 1 -- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010cf10 msgr2=0x7fe3901a28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe395dfe640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010cf10 0x7fe3901a28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe395dfe640 1 -- 172.21.15.46:0/3320073964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3901a6c70 con 0x7fe39010c560 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe3955fd640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010cf10 0x7fe3901a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe395dfe640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39010c560 0x7fe3901a23a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02a9c0 tx=0x7fe38c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe386ffd640 1 -- 172.21.15.46:0/3320073964 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe38c03f070 con 0x7fe39010c560 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.455+0000 7fe386ffd640 1 -- 172.21.15.46:0/3320073964 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe38c0035f0 con 0x7fe39010c560 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.456+0000 7fe386ffd640 1 -- 172.21.15.46:0/3320073964 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe38c035030 con 0x7fe39010c560 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.456+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3901a6f00 con 0x7fe39010c560 2024-09-10T04:10:30.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.456+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3901a7430 con 0x7fe39010c560 2024-09-10T04:10:30.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.457+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39010da40 con 0x7fe39010c560 2024-09-10T04:10:30.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.458+0000 7fe386ffd640 1 -- 172.21.15.46:0/3320073964 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe38c042070 con 0x7fe39010c560 2024-09-10T04:10:30.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.459+0000 7fe386ffd640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe37c07ac50 0x7fe37c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.459+0000 7fe386ffd640 1 -- 172.21.15.46:0/3320073964 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6070+0+0 (secure 0 0 0) 0x7fe38c0be340 con 0x7fe39010c560 2024-09-10T04:10:30.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.459+0000 7fe3955fd640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe37c07ac50 0x7fe37c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.459+0000 7fe3955fd640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe37c07ac50 0x7fe37c07d110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe39010bdf0 tx=0x7fe380005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:30.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.463+0000 7fe386ffd640 1 -- 172.21.15.46:0/3320073964 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe38c089bb0 con 0x7fe39010c560 2024-09-10T04:10:30.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:30 smithi070 ceph-mon[79774]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.725+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe390114870 con 0x7fe39010c560 2024-09-10T04:10:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.726+0000 7fe386ffd640 1 -- 172.21.15.46:0/3320073964 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7fe38c08ea60 con 0x7fe39010c560 2024-09-10T04:10:30.727 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:10:30.727 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:10:30.727 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:10:30.727 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:10:30.728 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:10:30.729 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:10:30.730 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:10:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:10:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe37c07ac50 msgr2=0x7fe37c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe37c07ac50 0x7fe37c07d110 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe39010bdf0 tx=0x7fe380005f90 comp rx=0 tx=0).stop 2024-09-10T04:10:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39010c560 msgr2=0x7fe3901a23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39010c560 0x7fe3901a23a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe38c02a9c0 tx=0x7fe38c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 shutdown_connections 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe37c07ac50 0x7fe37c07d110 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe39010cf10 0x7fe3901a28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 --2- 172.21.15.46:0/3320073964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39010c560 0x7fe3901a23a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.728+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 >> 172.21.15.46:0/3320073964 conn(0x7fe39006c5b0 msgr2=0x7fe39010a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.729+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 shutdown_connections 2024-09-10T04:10:30.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.729+0000 7fe398062640 1 -- 172.21.15.46:0/3320073964 wait complete. 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.818+0000 7f0560c71640 1 -- 172.21.15.46:0/441213038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c1033d0 msgr2=0x7f055c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.818+0000 7f0560c71640 1 --2- 172.21.15.46:0/441213038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c1033d0 0x7f055c10b8c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0550009880 tx=0x7f055002a530 comp rx=0 tx=0).stop 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.818+0000 7f0560c71640 1 -- 172.21.15.46:0/441213038 shutdown_connections 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.818+0000 7f0560c71640 1 --2- 172.21.15.46:0/441213038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c1033d0 0x7f055c10b8c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.819+0000 7f0560c71640 1 --2- 172.21.15.46:0/441213038 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f055c102ab0 0x7f055c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.819+0000 7f0560c71640 1 -- 172.21.15.46:0/441213038 >> 172.21.15.46:0/441213038 conn(0x7f055c0fc550 msgr2=0x7f055c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.819+0000 7f0560c71640 1 -- 172.21.15.46:0/441213038 shutdown_connections 2024-09-10T04:10:30.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.819+0000 7f0560c71640 1 -- 172.21.15.46:0/441213038 wait complete. 2024-09-10T04:10:30.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.819+0000 7f0560c71640 1 Processor -- start 2024-09-10T04:10:30.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.819+0000 7f0560c71640 1 -- start start 2024-09-10T04:10:30.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f0560c71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c102ab0 0x7f055c100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f0560c71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f055c1033d0 0x7f055c100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f0560c71640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10edc0 con 0x7f055c102ab0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f0560c71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10ec40 con 0x7f055c1033d0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f055a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c102ab0 0x7f055c100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f055a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c102ab0 0x7f055c100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57976/0 (socket says 172.21.15.46:57976) 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f055a59c640 1 -- 172.21.15.46:0/3055828208 learned_addr learned my addr 172.21.15.46:0/3055828208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f0559d9b640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f055c1033d0 0x7f055c100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f055a59c640 1 -- 172.21.15.46:0/3055828208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f055c1033d0 msgr2=0x7f055c100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f055a59c640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f055c1033d0 0x7f055c100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f055a59c640 1 -- 172.21.15.46:0/3055828208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c101190 con 0x7f055c102ab0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f0559d9b640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f055c1033d0 0x7f055c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.820+0000 7f055a59c640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c102ab0 0x7f055c100710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f054c00d8d0 tx=0x7f054c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.821+0000 7f054b7fe640 1 -- 172.21.15.46:0/3055828208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f054c017030 con 0x7f055c102ab0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.821+0000 7f054b7fe640 1 -- 172.21.15.46:0/3055828208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f054c004020 con 0x7f055c102ab0 2024-09-10T04:10:30.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.821+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c101480 con 0x7f055c102ab0 2024-09-10T04:10:30.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.821+0000 7f054b7fe640 1 -- 172.21.15.46:0/3055828208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f054c0047a0 con 0x7f055c102ab0 2024-09-10T04:10:30.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.821+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a3e10 con 0x7f055c102ab0 2024-09-10T04:10:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.822+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c06a580 con 0x7f055c102ab0 2024-09-10T04:10:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.823+0000 7f054b7fe640 1 -- 172.21.15.46:0/3055828208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f054c0041d0 con 0x7f055c102ab0 2024-09-10T04:10:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.823+0000 7f054b7fe640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f054407ac00 0x7f054407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.823+0000 7f054b7fe640 1 -- 172.21.15.46:0/3055828208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6070+0+0 (secure 0 0 0) 0x7f054c018070 con 0x7f055c102ab0 2024-09-10T04:10:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.823+0000 7f0559d9b640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f054407ac00 0x7f054407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:30.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.824+0000 7f0559d9b640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f054407ac00 0x7f054407d0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f055c1021a0 tx=0x7f0550031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:30.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:30.828+0000 7f054b7fe640 1 -- 172.21.15.46:0/3055828208 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f054c0691f0 con 0x7f055c102ab0 2024-09-10T04:10:31.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.037+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f055c10b5c0 con 0x7f054407ac00 2024-09-10T04:10:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.038+0000 7f054b7fe640 1 -- 172.21.15.46:0/3055828208 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f055c10b5c0 con 0x7f054407ac00 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "6/23 daemons upgraded", 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:10:31.039 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f054407ac00 msgr2=0x7f054407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f054407ac00 0x7f054407d0c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f055c1021a0 tx=0x7f0550031400 comp rx=0 tx=0).stop 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c102ab0 msgr2=0x7f055c100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c102ab0 0x7f055c100710 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f054c00d8d0 tx=0x7f054c00dd90 comp rx=0 tx=0).stop 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 shutdown_connections 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f054407ac00 0x7f054407d0c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f055c1033d0 0x7f055c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 --2- 172.21.15.46:0/3055828208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f055c102ab0 0x7f055c100710 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 >> 172.21.15.46:0/3055828208 conn(0x7f055c0fc550 msgr2=0x7f055c10aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 shutdown_connections 2024-09-10T04:10:31.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.040+0000 7f0560c71640 1 -- 172.21.15.46:0/3055828208 wait complete. 2024-09-10T04:10:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 -- 172.21.15.46:0/2300600713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 msgr2=0x7f0498107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:31.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 --2- 172.21.15.46:0/2300600713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f0498107ce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0488009880 tx=0x7f048802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 -- 172.21.15.46:0/2300600713 shutdown_connections 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 --2- 172.21.15.46:0/2300600713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0498101900 0x7f0498101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 --2- 172.21.15.46:0/2300600713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f0498107ce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 -- 172.21.15.46:0/2300600713 >> 172.21.15.46:0/2300600713 conn(0x7f04980fd600 msgr2=0x7f04980ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 -- 172.21.15.46:0/2300600713 shutdown_connections 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.124+0000 7f049d6e3640 1 -- 172.21.15.46:0/2300600713 wait complete. 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f049d6e3640 1 Processor -- start 2024-09-10T04:10:31.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f049d6e3640 1 -- start start 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f049d6e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0498101900 0x7f049819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f049d6e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f049819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f049d6e3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f049810edd0 con 0x7f0498107900 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f049d6e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f049810ec50 con 0x7f0498101900 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f048ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f049819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f048ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f049819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57988/0 (socket says 172.21.15.46:57988) 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f048ffff640 1 -- 172.21.15.46:0/3649113554 learned_addr learned my addr 172.21.15.46:0/3649113554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:10:31.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f0496ffd640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0498101900 0x7f049819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f048ffff640 1 -- 172.21.15.46:0/3649113554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0498101900 msgr2=0x7f049819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f048ffff640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0498101900 0x7f049819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.125+0000 7f048ffff640 1 -- 172.21.15.46:0/3649113554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04981977a0 con 0x7f0498107900 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.126+0000 7f048ffff640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f049819db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f048000e970 tx=0x7f048000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.126+0000 7f0494ff9640 1 -- 172.21.15.46:0/3649113554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f048000cac0 con 0x7f0498107900 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.126+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0498197a90 con 0x7f0498107900 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.126+0000 7f0494ff9640 1 -- 172.21.15.46:0/3649113554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f04800054e0 con 0x7f0498107900 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.126+0000 7f0494ff9640 1 -- 172.21.15.46:0/3649113554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0480010430 con 0x7f0498107900 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.126+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0498198020 con 0x7f0498107900 2024-09-10T04:10:31.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.127+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0498103040 con 0x7f0498107900 2024-09-10T04:10:31.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.128+0000 7f0494ff9640 1 -- 172.21.15.46:0/3649113554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0480004e80 con 0x7f0498107900 2024-09-10T04:10:31.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.128+0000 7f0494ff9640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f048407abf0 0x7f048407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:10:31.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.128+0000 7f0494ff9640 1 -- 172.21.15.46:0/3649113554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6070+0+0 (secure 0 0 0) 0x7f048009e2c0 con 0x7f0498107900 2024-09-10T04:10:31.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.128+0000 7f0496ffd640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f048407abf0 0x7f048407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:10:31.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.129+0000 7f0496ffd640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f048407abf0 0x7f048407d0b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04880096c0 tx=0x7f0488031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:10:31.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.133+0000 7f0494ff9640 1 -- 172.21.15.46:0/3649113554 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0480069ab0 con 0x7f0498107900 2024-09-10T04:10:31.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:31 smithi046 ceph-mon[109506]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:31.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:31 smithi046 ceph-mon[109506]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:31.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:31 smithi046 ceph-mon[109506]: pgmap v114: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:31.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:31 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4167914309' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:31.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:31 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3320073964' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:10:31.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.447+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f0498109e70 con 0x7f0498107900 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.448+0000 7f0494ff9640 1 -- 172.21.15.46:0/3649113554 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1573 (secure 0 0 0) 0x7f048006e960 con 0x7f0498107900 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 58/312 objects degraded (18.590%), 20 pgs degraded 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: osd.0 (root=default,host=smithi046) is down 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 58/312 objects degraded (18.590%), 20 pgs degraded 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-09-10T04:10:31.449 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-09-10T04:10:31.450 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-09-10T04:10:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-09-10T04:10:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-09-10T04:10:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-09-10T04:10:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-09-10T04:10:31.451 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f048407abf0 msgr2=0x7f048407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f048407abf0 0x7f048407d0b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04880096c0 tx=0x7f0488031400 comp rx=0 tx=0).stop 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 msgr2=0x7f049819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f049819db60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f048000e970 tx=0x7f048000ed70 comp rx=0 tx=0).stop 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 shutdown_connections 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f048407abf0 0x7f048407d0b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0498107900 0x7f049819db60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 --2- 172.21.15.46:0/3649113554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0498101900 0x7f049819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.450+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 >> 172.21.15.46:0/3649113554 conn(0x7f04980fd600 msgr2=0x7f04980fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.451+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 shutdown_connections 2024-09-10T04:10:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:10:31.451+0000 7f049d6e3640 1 -- 172.21.15.46:0/3649113554 wait complete. 2024-09-10T04:10:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:31 smithi070 ceph-mon[79774]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:31 smithi070 ceph-mon[79774]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:31 smithi070 ceph-mon[79774]: pgmap v114: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:31 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4167914309' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:31 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3320073964' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:10:32.478 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:32 smithi046 ceph-mon[109506]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:32.478 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:32 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3649113554' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:10:32.478 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:32 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:32.478 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:32 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:32.478 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:32 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:32.478 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:32 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:32 smithi070 ceph-mon[79774]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:10:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:32 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3649113554' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:10:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:32 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:32 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:32 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:32 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:33.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:33 smithi046 ceph-mon[109506]: pgmap v115: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:33.424 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:33 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[118266]: 2024-09-10T04:10:33.306+0000 7fe9faafe640 -1 osd.0 45 log_to_monitors true 2024-09-10T04:10:33.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:33 smithi070 ceph-mon[79774]: pgmap v115: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:34.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:34 smithi070 ceph-mon[79774]: from='osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T04:10:34.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:34 smithi046 ceph-mon[109506]: from='osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T04:10:34.589 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:10:34 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[118266]: 2024-09-10T04:10:34.181+0000 7fe9ee8a0640 -1 osd.0 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:10:35.411 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:35 smithi046 ceph-mon[109506]: pgmap v116: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:35.411 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:35 smithi046 ceph-mon[109506]: from='osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T04:10:35.411 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:35 smithi046 ceph-mon[109506]: osdmap e48: 6 total, 5 up, 6 in 2024-09-10T04:10:35.411 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:35 smithi046 ceph-mon[109506]: from='osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]} : dispatch 2024-09-10T04:10:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:35 smithi070 ceph-mon[79774]: pgmap v116: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:35 smithi070 ceph-mon[79774]: from='osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T04:10:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:35 smithi070 ceph-mon[79774]: osdmap e48: 6 total, 5 up, 6 in 2024-09-10T04:10:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:35 smithi070 ceph-mon[79774]: from='osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]} : dispatch 2024-09-10T04:10:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:10:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010] boot 2024-09-10T04:10:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: osdmap e49: 6 total, 6 up, 6 in 2024-09-10T04:10:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T04:10:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T04:10:36.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:36 smithi070 ceph-mon[79774]: osdmap e50: 6 total, 6 up, 6 in 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: osd.0 [v2:172.21.15.46:6802/94627010,v1:172.21.15.46:6803/94627010] boot 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: osdmap e49: 6 total, 6 up, 6 in 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:10:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:36.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:36.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:10:36.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-10T04:10:36.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T04:10:36.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:36 smithi046 ceph-mon[109506]: osdmap e50: 6 total, 6 up, 6 in 2024-09-10T04:10:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:37 smithi070 ceph-mon[79774]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-10T04:10:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:37 smithi070 ceph-mon[79774]: pgmap v119: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:37.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:37 smithi046 ceph-mon[109506]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-10T04:10:37.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:37 smithi046 ceph-mon[109506]: pgmap v119: 65 pgs: 14 active+undersized, 20 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 58/312 objects degraded (18.590%) 2024-09-10T04:10:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:39 smithi070 ceph-mon[79774]: pgmap v121: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 6 op/s; 7/312 objects degraded (2.244%) 2024-09-10T04:10:39.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:39 smithi046 ceph-mon[109506]: pgmap v121: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 3.3 KiB/s rd, 6 op/s; 7/312 objects degraded (2.244%) 2024-09-10T04:10:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:40 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 7/312 objects degraded (2.244%), 3 pgs degraded (PG_DEGRADED) 2024-09-10T04:10:40.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:40 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 7/312 objects degraded (2.244%), 3 pgs degraded (PG_DEGRADED) 2024-09-10T04:10:41.280 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:41 smithi046 ceph-mon[109506]: pgmap v122: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:41 smithi070 ceph-mon[79774]: pgmap v122: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:42.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:42 smithi070 ceph-mon[79774]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/312 objects degraded (2.244%), 3 pgs degraded) 2024-09-10T04:10:42.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:42 smithi046 ceph-mon[109506]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/312 objects degraded (2.244%), 3 pgs degraded) 2024-09-10T04:10:43.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:43 smithi070 ceph-mon[79774]: pgmap v123: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-10T04:10:43.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:43 smithi046 ceph-mon[109506]: pgmap v123: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-10T04:10:45.444 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:45 smithi070 ceph-mon[79774]: pgmap v124: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T04:10:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:45 smithi046 ceph-mon[109506]: pgmap v124: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T04:10:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:47 smithi070 ceph-mon[79774]: pgmap v125: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:10:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:47 smithi046 ceph-mon[109506]: pgmap v125: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:10:49.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:49.337 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:49 smithi070 ceph-mon[79774]: pgmap v126: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:49 smithi046 ceph-mon[109506]: pgmap v126: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:51.168 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:51 smithi046 ceph-mon[109506]: pgmap v127: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:51.168 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:51 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-10T04:10:51.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:51 smithi070 ceph-mon[79774]: pgmap v127: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:51.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:51 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-10T04:10:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:52 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T04:10:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:52 smithi046 ceph-mon[109506]: Upgrade: osd.1 is safe to restart 2024-09-10T04:10:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T04:10:52.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:52 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T04:10:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:52 smithi070 ceph-mon[79774]: Upgrade: osd.1 is safe to restart 2024-09-10T04:10:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T04:10:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:10:53.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:53 smithi046 ceph-mon[109506]: Upgrade: Updating osd.1 2024-09-10T04:10:53.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:53 smithi046 ceph-mon[109506]: Deploying daemon osd.1 on smithi046 2024-09-10T04:10:53.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:53 smithi046 ceph-mon[109506]: pgmap v128: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:53.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:53.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:53 smithi070 ceph-mon[79774]: Upgrade: Updating osd.1 2024-09-10T04:10:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:53 smithi070 ceph-mon[79774]: Deploying daemon osd.1 on smithi046 2024-09-10T04:10:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:53 smithi070 ceph-mon[79774]: pgmap v128: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:10:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:10:55.454 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:55 smithi046 ceph-mon[109506]: pgmap v129: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:55.455 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:55 smithi046 systemd[1]: Stopping Ceph osd.1 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:10:55.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:55 smithi070 ceph-mon[79774]: pgmap v129: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:10:55.836 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:55 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[56628]: 2024-09-10T04:10:55.469+0000 7f81c09f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:10:55.837 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:55 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[56628]: 2024-09-10T04:10:55.469+0000 7f81c09f2640 -1 osd.1 50 *** Got signal Terminated *** 2024-09-10T04:10:55.837 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:55 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[56628]: 2024-09-10T04:10:55.469+0000 7f81c09f2640 -1 osd.1 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:10:56.554 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:56 smithi046 ceph-mon[109506]: osd.1 marked itself down and dead 2024-09-10T04:10:56.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:56 smithi070 ceph-mon[79774]: osd.1 marked itself down and dead 2024-09-10T04:10:56.837 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:56 smithi046 podman[123387]: 2024-09-10 04:10:56.553359688 +0000 UTC m=+1.193431507 container died a717950e54e36324c705d487d492c509b4d7a94b1b31c11ed67045ffee57d840 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T04:10:57.143 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:57 smithi046 podman[123387]: 2024-09-10 04:10:57.028703882 +0000 UTC m=+1.668775789 container cleanup a717950e54e36324c705d487d492c509b4d7a94b1b31c11ed67045ffee57d840 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-10T04:10:57.144 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:57 smithi046 bash[123387]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1 2024-09-10T04:10:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:57 smithi070 ceph-mon[79774]: pgmap v130: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:57 smithi070 ceph-mon[79774]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:10:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:57 smithi070 ceph-mon[79774]: osdmap e51: 6 total, 5 up, 6 in 2024-09-10T04:10:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:57 smithi046 ceph-mon[109506]: pgmap v130: 65 pgs: 65 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:10:57.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:57 smithi046 ceph-mon[109506]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:10:57.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:57 smithi046 ceph-mon[109506]: osdmap e51: 6 total, 5 up, 6 in 2024-09-10T04:10:57.587 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:57 smithi046 podman[123402]: 2024-09-10 04:10:57.355484009 +0000 UTC m=+0.792042496 container remove a717950e54e36324c705d487d492c509b4d7a94b1b31c11ed67045ffee57d840 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:10:57.934 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:57 smithi046 podman[123487]: 2024-09-10 04:10:57.574563585 +0000 UTC m=+0.016422985 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:10:57.934 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:57 smithi046 podman[123487]: 2024-09-10 04:10:57.697933537 +0000 UTC m=+0.139792967 container create 6071d2ff4d97bf669889f177b9035c5f96e35216a76239e116374fdf69280d94 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:10:57.934 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:57 smithi046 podman[123487]: 2024-09-10 04:10:57.933665642 +0000 UTC m=+0.375525032 container init 6071d2ff4d97bf669889f177b9035c5f96e35216a76239e116374fdf69280d94 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=) 2024-09-10T04:10:58.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:58 smithi046 ceph-mon[109506]: osdmap e52: 6 total, 5 up, 6 in 2024-09-10T04:10:58.337 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:57 smithi046 podman[123487]: 2024-09-10 04:10:57.937134618 +0000 UTC m=+0.378994009 container start 6071d2ff4d97bf669889f177b9035c5f96e35216a76239e116374fdf69280d94 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T04:10:58.337 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:58 smithi046 podman[123487]: 2024-09-10 04:10:58.025276273 +0000 UTC m=+0.467135663 container attach 6071d2ff4d97bf669889f177b9035c5f96e35216a76239e116374fdf69280d94 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T04:10:58.337 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:58 smithi046 podman[123487]: 2024-09-10 04:10:58.130730148 +0000 UTC m=+0.572589545 container died 6071d2ff4d97bf669889f177b9035c5f96e35216a76239e116374fdf69280d94 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T04:10:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:58 smithi070 ceph-mon[79774]: osdmap e52: 6 total, 5 up, 6 in 2024-09-10T04:10:59.147 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:58 smithi046 podman[123510]: 2024-09-10 04:10:58.887735767 +0000 UTC m=+0.747703339 container remove 6071d2ff4d97bf669889f177b9035c5f96e35216a76239e116374fdf69280d94 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:10:59.147 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:58 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-10T04:10:59.147 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:58 smithi046 systemd[1]: Stopped Ceph osd.1 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:10:59.147 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:58 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.1.service: Consumed 1min 3.036s CPU time. 2024-09-10T04:10:59.469 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:59 smithi046 ceph-mon[109506]: pgmap v133: 65 pgs: 16 active+undersized, 3 stale+active+clean, 13 active+undersized+degraded, 33 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 42/312 objects degraded (13.462%) 2024-09-10T04:10:59.469 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:10:59 smithi046 ceph-mon[109506]: Health check failed: Degraded data redundancy: 42/312 objects degraded (13.462%), 13 pgs degraded (PG_DEGRADED) 2024-09-10T04:10:59.469 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 systemd[1]: Starting Ceph osd.1 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:10:59.469 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 podman[123608]: 2024-09-10 04:10:59.264170696 +0000 UTC m=+0.016598326 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:10:59.469 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 podman[123608]: 2024-09-10 04:10:59.364108478 +0000 UTC m=+0.116536139 container create a19e1bf8915bae3b6b8690835e710deafedc6a1fe85a990eda030de0bc711728 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T04:10:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:59 smithi070 ceph-mon[79774]: pgmap v133: 65 pgs: 16 active+undersized, 3 stale+active+clean, 13 active+undersized+degraded, 33 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 42/312 objects degraded (13.462%) 2024-09-10T04:10:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:10:59 smithi070 ceph-mon[79774]: Health check failed: Degraded data redundancy: 42/312 objects degraded (13.462%), 13 pgs degraded (PG_DEGRADED) 2024-09-10T04:10:59.763 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 podman[123608]: 2024-09-10 04:10:59.624894922 +0000 UTC m=+0.377322551 container init a19e1bf8915bae3b6b8690835e710deafedc6a1fe85a990eda030de0bc711728 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T04:10:59.764 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 podman[123608]: 2024-09-10 04:10:59.628902076 +0000 UTC m=+0.381329700 container start a19e1bf8915bae3b6b8690835e710deafedc6a1fe85a990eda030de0bc711728 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True) 2024-09-10T04:10:59.764 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 podman[123608]: 2024-09-10 04:10:59.708250921 +0000 UTC m=+0.460678549 container attach a19e1bf8915bae3b6b8690835e710deafedc6a1fe85a990eda030de0bc711728 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:11:00.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:00.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 bash[123608]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:00.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:00.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:10:59 smithi046 bash[123608]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:00.811 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T04:11:00.812 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 bash[123608]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T04:11:00.812 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T04:11:00.812 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 bash[123608]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T04:11:01.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T04:11:01.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 bash[123608]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T04:11:01.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T04:11:01.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 bash[123608]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T04:11:01.088 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-10T04:11:01.088 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 bash[123608]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-10T04:11:01.088 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T04:11:01.088 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 bash[123608]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T04:11:01.088 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate[123622]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-10T04:11:01.088 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 bash[123608]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-10T04:11:01.088 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:00 smithi046 podman[123608]: 2024-09-10 04:11:00.841163165 +0000 UTC m=+1.593590788 container died a19e1bf8915bae3b6b8690835e710deafedc6a1fe85a990eda030de0bc711728 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:11:01.553 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:01 smithi046 ceph-mon[109506]: pgmap v134: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.550+0000 7f8e1f89a640 1 -- 172.21.15.46:0/465350904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 msgr2=0x7f8e18100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.550+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/465350904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e18100a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8e14009880 tx=0x7f8e1402a490 comp rx=0 tx=0).stop 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.550+0000 7f8e1f89a640 1 -- 172.21.15.46:0/465350904 shutdown_connections 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.550+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/465350904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e18101000 0x7f8e1810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.550+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/465350904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e18100a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.550+0000 7f8e1f89a640 1 -- 172.21.15.46:0/465350904 >> 172.21.15.46:0/465350904 conn(0x7f8e180fc520 msgr2=0x7f8e180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.551+0000 7f8e1f89a640 1 -- 172.21.15.46:0/465350904 shutdown_connections 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.551+0000 7f8e1f89a640 1 -- 172.21.15.46:0/465350904 wait complete. 2024-09-10T04:11:01.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.551+0000 7f8e1f89a640 1 Processor -- start 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1f89a640 1 -- start start 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1f89a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e1819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1f89a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e18101000 0x7f8e1819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1f89a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18110fb0 con 0x7f8e18100650 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1f89a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8e18110e30 con 0x7f8e18101000 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1d636640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e1819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1d636640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e1819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35806/0 (socket says 172.21.15.46:35806) 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1d636640 1 -- 172.21.15.46:0/514235015 learned_addr learned my addr 172.21.15.46:0/514235015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1ce35640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e18101000 0x7f8e1819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1d636640 1 -- 172.21.15.46:0/514235015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e18101000 msgr2=0x7f8e1819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1d636640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e18101000 0x7f8e1819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.552+0000 7f8e1d636640 1 -- 172.21.15.46:0/514235015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e181999a0 con 0x7f8e18100650 2024-09-10T04:11:01.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.553+0000 7f8e1d636640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e1819f7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e14031460 tx=0x7f8e140045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.553+0000 7f8e0e7fc640 1 -- 172.21.15.46:0/514235015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8e1403f070 con 0x7f8e18100650 2024-09-10T04:11:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.553+0000 7f8e0e7fc640 1 -- 172.21.15.46:0/514235015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e140035f0 con 0x7f8e18100650 2024-09-10T04:11:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.553+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e18199c30 con 0x7f8e18100650 2024-09-10T04:11:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.553+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1819a1c0 con 0x7f8e18100650 2024-09-10T04:11:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.553+0000 7f8e0e7fc640 1 -- 172.21.15.46:0/514235015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8e14035030 con 0x7f8e18100650 2024-09-10T04:11:01.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.554+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e1806aa90 con 0x7f8e18100650 2024-09-10T04:11:01.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.555+0000 7f8e0e7fc640 1 -- 172.21.15.46:0/514235015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8e14042070 con 0x7f8e18100650 2024-09-10T04:11:01.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.556+0000 7f8e0e7fc640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e0407ac40 0x7f8e0407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:01.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.556+0000 7f8e0e7fc640 1 -- 172.21.15.46:0/514235015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6070+0+0 (secure 0 0 0) 0x7f8e140bebc0 con 0x7f8e18100650 2024-09-10T04:11:01.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.556+0000 7f8e1ce35640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e0407ac40 0x7f8e0407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:01.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.556+0000 7f8e1ce35640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e0407ac40 0x7f8e0407d100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005fd0 tx=0x7f8e08005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:01.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.560+0000 7f8e0e7fc640 1 -- 172.21.15.46:0/514235015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8e1408a410 con 0x7f8e18100650 2024-09-10T04:11:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:01 smithi070 ceph-mon[79774]: pgmap v134: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.769+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8e1819aed0 con 0x7f8e0407ac40 2024-09-10T04:11:01.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.771+0000 7f8e0e7fc640 1 -- 172.21.15.46:0/514235015 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f8e1819aed0 con 0x7f8e0407ac40 2024-09-10T04:11:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e0407ac40 msgr2=0x7f8e0407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e0407ac40 0x7f8e0407d100 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8e08005fd0 tx=0x7f8e08005850 comp rx=0 tx=0).stop 2024-09-10T04:11:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 msgr2=0x7f8e1819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e1819f7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e14031460 tx=0x7f8e140045e0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 shutdown_connections 2024-09-10T04:11:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e0407ac40 0x7f8e0407d100 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e18101000 0x7f8e1819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 --2- 172.21.15.46:0/514235015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e18100650 0x7f8e1819f7e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 >> 172.21.15.46:0/514235015 conn(0x7f8e180fc520 msgr2=0x7f8e180fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:01.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 shutdown_connections 2024-09-10T04:11:01.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.773+0000 7f8e1f89a640 1 -- 172.21.15.46:0/514235015 wait complete. 2024-09-10T04:11:01.785 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:11:01.836 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:01 smithi046 podman[123772]: 2024-09-10 04:11:01.729184925 +0000 UTC m=+0.877641986 container remove a19e1bf8915bae3b6b8690835e710deafedc6a1fe85a990eda030de0bc711728 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:11:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 -- 172.21.15.46:0/3867944663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 msgr2=0x7f7a84100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 --2- 172.21.15.46:0/3867944663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a84100c50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009880 tx=0x7f7a6c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 -- 172.21.15.46:0/3867944663 shutdown_connections 2024-09-10T04:11:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 --2- 172.21.15.46:0/3867944663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a84100c50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 --2- 172.21.15.46:0/3867944663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a841067f0 0x7f7a84106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 -- 172.21.15.46:0/3867944663 >> 172.21.15.46:0/3867944663 conn(0x7f7a840fc550 msgr2=0x7f7a840fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 -- 172.21.15.46:0/3867944663 shutdown_connections 2024-09-10T04:11:01.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.865+0000 7f7a892ee640 1 -- 172.21.15.46:0/3867944663 wait complete. 2024-09-10T04:11:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a892ee640 1 Processor -- start 2024-09-10T04:11:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a892ee640 1 -- start start 2024-09-10T04:11:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a892ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a8419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a892ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a841067f0 0x7f7a8419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a892ee640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8410dca0 con 0x7f7a841007f0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a892ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8410db20 con 0x7f7a841067f0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a82ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a8419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a82ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a8419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35820/0 (socket says 172.21.15.46:35820) 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.866+0000 7f7a82ffd640 1 -- 172.21.15.46:0/15066047 learned_addr learned my addr 172.21.15.46:0/15066047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a7bfff640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a841067f0 0x7f7a8419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a82ffd640 1 -- 172.21.15.46:0/15066047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a841067f0 msgr2=0x7f7a8419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a82ffd640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a841067f0 0x7f7a8419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a82ffd640 1 -- 172.21.15.46:0/15066047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a84197780 con 0x7f7a841007f0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a82ffd640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a8419d600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400d8d0 tx=0x7f7a7400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a80ff9640 1 -- 172.21.15.46:0/15066047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a74017070 con 0x7f7a841007f0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a80ff9640 1 -- 172.21.15.46:0/15066047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a74010460 con 0x7f7a841007f0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a80ff9640 1 -- 172.21.15.46:0/15066047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a7400f5f0 con 0x7f7a841007f0 2024-09-10T04:11:01.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a84197a70 con 0x7f7a841007f0 2024-09-10T04:11:01.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.867+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a84197fd0 con 0x7f7a841007f0 2024-09-10T04:11:01.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.868+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a84101f30 con 0x7f7a841007f0 2024-09-10T04:11:01.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.869+0000 7f7a80ff9640 1 -- 172.21.15.46:0/15066047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7a74009a30 con 0x7f7a841007f0 2024-09-10T04:11:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.869+0000 7f7a80ff9640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7007ac00 0x7f7a7007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.869+0000 7f7a80ff9640 1 -- 172.21.15.46:0/15066047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6070+0+0 (secure 0 0 0) 0x7f7a7409dab0 con 0x7f7a841007f0 2024-09-10T04:11:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.869+0000 7f7a7bfff640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7007ac00 0x7f7a7007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:01.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.870+0000 7f7a7bfff640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7007ac00 0x7f7a7007d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a84198ad0 tx=0x7f7a6c0095c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:01.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:01.874+0000 7f7a80ff9640 1 -- 172.21.15.46:0/15066047 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7a740692a0 con 0x7f7a841007f0 2024-09-10T04:11:02.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.080+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7a841981f0 con 0x7f7a7007ac00 2024-09-10T04:11:02.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.081+0000 7f7a80ff9640 1 -- 172.21.15.46:0/15066047 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f7a841981f0 con 0x7f7a7007ac00 2024-09-10T04:11:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7007ac00 msgr2=0x7f7a7007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7007ac00 0x7f7a7007d0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7a84198ad0 tx=0x7f7a6c0095c0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 msgr2=0x7f7a8419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a8419d600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400d8d0 tx=0x7f7a7400dd90 comp rx=0 tx=0).stop 2024-09-10T04:11:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 shutdown_connections 2024-09-10T04:11:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7007ac00 0x7f7a7007d0c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a841067f0 0x7f7a8419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 --2- 172.21.15.46:0/15066047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a841007f0 0x7f7a8419d600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 >> 172.21.15.46:0/15066047 conn(0x7f7a840fc550 msgr2=0x7f7a84069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:02.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 shutdown_connections 2024-09-10T04:11:02.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.083+0000 7f7a892ee640 1 -- 172.21.15.46:0/15066047 wait complete. 2024-09-10T04:11:02.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 -- 172.21.15.46:0/3075050673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 msgr2=0x7f4064106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 --2- 172.21.15.46:0/3075050673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f4064106be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4054009880 tx=0x7f405402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 -- 172.21.15.46:0/3075050673 shutdown_connections 2024-09-10T04:11:02.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 --2- 172.21.15.46:0/3075050673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4064100800 0x7f4064100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 --2- 172.21.15.46:0/3075050673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f4064106be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 -- 172.21.15.46:0/3075050673 >> 172.21.15.46:0/3075050673 conn(0x7f40640fc520 msgr2=0x7f40640fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:02.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 -- 172.21.15.46:0/3075050673 shutdown_connections 2024-09-10T04:11:02.172 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:01 smithi046 podman[123859]: 2024-09-10 04:11:01.844994962 +0000 UTC m=+0.017489057 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:11:02.172 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:01 smithi046 podman[123859]: 2024-09-10 04:11:01.971851733 +0000 UTC m=+0.144345819 container create 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:11:02.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.169+0000 7f406983f640 1 -- 172.21.15.46:0/3075050673 wait complete. 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.170+0000 7f406983f640 1 Processor -- start 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.170+0000 7f406983f640 1 -- start start 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f406983f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4064100800 0x7f406419c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f406983f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f406419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f406983f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f406410dcb0 con 0x7f4064106800 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f406983f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f406410db30 con 0x7f4064100800 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f40627fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f406419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f40627fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f406419cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35840/0 (socket says 172.21.15.46:35840) 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f40627fc640 1 -- 172.21.15.46:0/4220091368 learned_addr learned my addr 172.21.15.46:0/4220091368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f4062ffd640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4064100800 0x7f406419c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f40627fc640 1 -- 172.21.15.46:0/4220091368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4064100800 msgr2=0x7f406419c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f40627fc640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4064100800 0x7f406419c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f40627fc640 1 -- 172.21.15.46:0/4220091368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40641966d0 con 0x7f4064106800 2024-09-10T04:11:02.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f40627fc640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f406419cb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f405800d8d0 tx=0x7f405800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.171+0000 7f406883d640 1 -- 172.21.15.46:0/4220091368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4058017030 con 0x7f4064106800 2024-09-10T04:11:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.172+0000 7f406883d640 1 -- 172.21.15.46:0/4220091368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4058004020 con 0x7f4064106800 2024-09-10T04:11:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.172+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40641969c0 con 0x7f4064106800 2024-09-10T04:11:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.172+0000 7f406883d640 1 -- 172.21.15.46:0/4220091368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f40580047a0 con 0x7f4064106800 2024-09-10T04:11:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.172+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4064196f20 con 0x7f4064106800 2024-09-10T04:11:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.173+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4064101f40 con 0x7f4064106800 2024-09-10T04:11:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.173+0000 7f406883d640 1 -- 172.21.15.46:0/4220091368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f40580041d0 con 0x7f4064106800 2024-09-10T04:11:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.174+0000 7f406883d640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f404c07ac40 0x7f404c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.174+0000 7f406883d640 1 -- 172.21.15.46:0/4220091368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6070+0+0 (secure 0 0 0) 0x7f4058018070 con 0x7f4064106800 2024-09-10T04:11:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.174+0000 7f4062ffd640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f404c07ac40 0x7f404c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.174+0000 7f4062ffd640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f404c07ac40 0x7f404c07d100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40540096c0 tx=0x7f4054031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:02.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.178+0000 7f406883d640 1 -- 172.21.15.46:0/4220091368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4058069130 con 0x7f4064106800 2024-09-10T04:11:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.385+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4064197230 con 0x7f404c07ac40 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.395+0000 7f406883d640 1 -- 172.21.15.46:0/4220091368 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f4064197230 con 0x7f404c07ac40 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (2m) 30s ago 13m 20.8M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (5m) 30s ago 14m 8896k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (4m) 53s ago 13m 8539k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (62s) 30s ago 14m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (57s) 53s ago 13m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:11:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (2m) 30s ago 13m 72.7M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (8m) 30s ago 8m 90.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (8m) 30s ago 8m 146M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (8m) 53s ago 8m 26.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (8m) 53s ago 8m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (4m) 30s ago 15m 526M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (3m) 53s ago 13m 455M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (92s) 30s ago 15m 39.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (75s) 53s ago 13m 25.7M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (3m) 30s ago 14m 11.7M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (3m) 53s ago 13m 12.0M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (36s) 30s ago 11m 11.6M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:11:02.398 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (11m) 30s ago 11m 226M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 a717950e54e3 2024-09-10T04:11:02.399 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (10m) 30s ago 10m 179M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:11:02.399 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (10m) 53s ago 10m 236M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:11:02.399 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (10m) 53s ago 10m 210M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:11:02.399 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (9m) 53s ago 9m 196M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:11:02.399 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (2m) 30s ago 13m 51.4M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f404c07ac40 msgr2=0x7f404c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f404c07ac40 0x7f404c07d100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40540096c0 tx=0x7f4054031400 comp rx=0 tx=0).stop 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 msgr2=0x7f406419cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f406419cb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f405800d8d0 tx=0x7f405800dd90 comp rx=0 tx=0).stop 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 shutdown_connections 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f404c07ac40 0x7f404c07d100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4064106800 0x7f406419cb40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 --2- 172.21.15.46:0/4220091368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4064100800 0x7f406419c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 >> 172.21.15.46:0/4220091368 conn(0x7f40640fc520 msgr2=0x7f40640fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 shutdown_connections 2024-09-10T04:11:02.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.399+0000 7f406983f640 1 -- 172.21.15.46:0/4220091368 wait complete. 2024-09-10T04:11:02.493 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:02 smithi046 podman[123859]: 2024-09-10 04:11:02.238166833 +0000 UTC m=+0.410660936 container init 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:11:02.493 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:02 smithi046 podman[123859]: 2024-09-10 04:11:02.242268466 +0000 UTC m=+0.414762551 container start 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:11:02.493 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:02 smithi046 ceph-osd[123918]: -- 172.21.15.46:0/2708300816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa8d3e01e0 con 0x55aa8c57fc00 2024-09-10T04:11:02.493 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:02 smithi046 bash[123859]: 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a 2024-09-10T04:11:02.493 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:02 smithi046 systemd[1]: Started Ceph osd.1 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:11:02.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.490+0000 7f697edf3640 1 -- 172.21.15.46:0/1055862361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781067f0 msgr2=0x7f6978106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.490+0000 7f697edf3640 1 --2- 172.21.15.46:0/1055862361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781067f0 0x7f6978106bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f696c009880 tx=0x7f696c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.490+0000 7f697edf3640 1 -- 172.21.15.46:0/1055862361 shutdown_connections 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.490+0000 7f697edf3640 1 --2- 172.21.15.46:0/1055862361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69781007f0 0x7f6978100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.490+0000 7f697edf3640 1 --2- 172.21.15.46:0/1055862361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781067f0 0x7f6978106bd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.490+0000 7f697edf3640 1 -- 172.21.15.46:0/1055862361 >> 172.21.15.46:0/1055862361 conn(0x7f69780fc550 msgr2=0x7f69780fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.491+0000 7f697edf3640 1 -- 172.21.15.46:0/1055862361 shutdown_connections 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.491+0000 7f697edf3640 1 -- 172.21.15.46:0/1055862361 wait complete. 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.491+0000 7f697edf3640 1 Processor -- start 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.491+0000 7f697edf3640 1 -- start start 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697edf3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781007f0 0x7f69781988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697edf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69781067f0 0x7f6978198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697edf3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f697810dca0 con 0x7f69781007f0 2024-09-10T04:11:02.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697edf3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f697810db20 con 0x7f69781067f0 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697cb8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781007f0 0x7f69781988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697cb8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781007f0 0x7f69781988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35872/0 (socket says 172.21.15.46:35872) 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697cb8f640 1 -- 172.21.15.46:0/3600397187 learned_addr learned my addr 172.21.15.46:0/3600397187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f6977fff640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69781067f0 0x7f6978198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697cb8f640 1 -- 172.21.15.46:0/3600397187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69781067f0 msgr2=0x7f6978198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697cb8f640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69781067f0 0x7f6978198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.492+0000 7f697cb8f640 1 -- 172.21.15.46:0/3600397187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6978199410 con 0x7f69781007f0 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.493+0000 7f697cb8f640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781007f0 0x7f69781988d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f696c031460 tx=0x7f696c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.493+0000 7f6975ffb640 1 -- 172.21.15.46:0/3600397187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f696c03f070 con 0x7f69781007f0 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.493+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69781996a0 con 0x7f69781007f0 2024-09-10T04:11:02.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.493+0000 7f6975ffb640 1 -- 172.21.15.46:0/3600397187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f696c0035f0 con 0x7f69781007f0 2024-09-10T04:11:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.493+0000 7f6975ffb640 1 -- 172.21.15.46:0/3600397187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f696c035030 con 0x7f69781007f0 2024-09-10T04:11:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.493+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69781a1c50 con 0x7f69781007f0 2024-09-10T04:11:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.494+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6978101f30 con 0x7f69781007f0 2024-09-10T04:11:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.495+0000 7f6975ffb640 1 -- 172.21.15.46:0/3600397187 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f696c042070 con 0x7f69781007f0 2024-09-10T04:11:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.496+0000 7f6975ffb640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696807ac40 0x7f696807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.496+0000 7f6975ffb640 1 -- 172.21.15.46:0/3600397187 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6070+0+0 (secure 0 0 0) 0x7f696c0be590 con 0x7f69781007f0 2024-09-10T04:11:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.496+0000 7f6977fff640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696807ac40 0x7f696807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.496+0000 7f6977fff640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696807ac40 0x7f696807d100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6964005fd0 tx=0x7f6964005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:02.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.500+0000 7f6975ffb640 1 -- 172.21.15.46:0/3600397187 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f696c089d60 con 0x7f69781007f0 2024-09-10T04:11:02.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.771+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f697806a2c0 con 0x7f69781007f0 2024-09-10T04:11:02.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.772+0000 7f6975ffb640 1 -- 172.21.15.46:0/3600397187 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f697806a2c0 con 0x7f69781007f0 2024-09-10T04:11:02.774 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:11:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:11:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:11:02.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:11:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:11:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:11:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4, 2024-09-10T04:11:02.775 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:11:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:11:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:11:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:11:02.776 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 8, 2024-09-10T04:11:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 5 2024-09-10T04:11:02.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:11:02.777 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:11:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696807ac40 msgr2=0x7f696807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696807ac40 0x7f696807d100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6964005fd0 tx=0x7f6964005850 comp rx=0 tx=0).stop 2024-09-10T04:11:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781007f0 msgr2=0x7f69781988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781007f0 0x7f69781988d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f696c031460 tx=0x7f696c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 shutdown_connections 2024-09-10T04:11:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696807ac40 0x7f696807d100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69781067f0 0x7f6978198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 --2- 172.21.15.46:0/3600397187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69781007f0 0x7f69781988d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 >> 172.21.15.46:0/3600397187 conn(0x7f69780fc550 msgr2=0x7f69780fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 shutdown_connections 2024-09-10T04:11:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.774+0000 7f697edf3640 1 -- 172.21.15.46:0/3600397187 wait complete. 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 -- 172.21.15.46:0/2381637593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc54100a90 msgr2=0x7fbc541065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/2381637593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc54100a90 0x7fbc541065d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009880 tx=0x7fbc4802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 -- 172.21.15.46:0/2381637593 shutdown_connections 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/2381637593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc54100a90 0x7fbc541065d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/2381637593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc541000e0 0x7fbc541004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 -- 172.21.15.46:0/2381637593 >> 172.21.15.46:0/2381637593 conn(0x7fbc540fbc90 msgr2=0x7fbc540fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 -- 172.21.15.46:0/2381637593 shutdown_connections 2024-09-10T04:11:02.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 -- 172.21.15.46:0/2381637593 wait complete. 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 Processor -- start 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.863+0000 7fbc5ad38640 1 -- start start 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc5ad38640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc541000e0 0x7fbc5419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc5ad38640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc54100a90 0x7fbc5419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc5ad38640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5410ee30 con 0x7fbc541000e0 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc5ad38640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5410ecb0 con 0x7fbc54100a90 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc58ad4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc541000e0 0x7fbc5419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc58ad4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc541000e0 0x7fbc5419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35886/0 (socket says 172.21.15.46:35886) 2024-09-10T04:11:02.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc58ad4640 1 -- 172.21.15.46:0/3218884445 learned_addr learned my addr 172.21.15.46:0/3218884445 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc53fff640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc54100a90 0x7fbc5419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc58ad4640 1 -- 172.21.15.46:0/3218884445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc54100a90 msgr2=0x7fbc5419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc58ad4640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc54100a90 0x7fbc5419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc58ad4640 1 -- 172.21.15.46:0/3218884445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc54197860 con 0x7fbc541000e0 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc53fff640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc54100a90 0x7fbc5419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc58ad4640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc541000e0 0x7fbc5419d6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d8d0 tx=0x7fbc4400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc51ffb640 1 -- 172.21.15.46:0/3218884445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbc44017030 con 0x7fbc541000e0 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc51ffb640 1 -- 172.21.15.46:0/3218884445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc44004020 con 0x7fbc541000e0 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc54197b50 con 0x7fbc541000e0 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.864+0000 7fbc51ffb640 1 -- 172.21.15.46:0/3218884445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbc440047a0 con 0x7fbc541000e0 2024-09-10T04:11:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.865+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc541980e0 con 0x7fbc541000e0 2024-09-10T04:11:02.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.866+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc541031a0 con 0x7fbc541000e0 2024-09-10T04:11:02.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.866+0000 7fbc51ffb640 1 -- 172.21.15.46:0/3218884445 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbc440041d0 con 0x7fbc541000e0 2024-09-10T04:11:02.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.867+0000 7fbc51ffb640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbc3c07abf0 0x7fbc3c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:02.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.867+0000 7fbc51ffb640 1 -- 172.21.15.46:0/3218884445 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6070+0+0 (secure 0 0 0) 0x7fbc44018070 con 0x7fbc541000e0 2024-09-10T04:11:02.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.867+0000 7fbc53fff640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbc3c07abf0 0x7fbc3c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:02.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.867+0000 7fbc53fff640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbc3c07abf0 0x7fbc3c07d0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc48031460 tx=0x7fbc480095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:02.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:02.871+0000 7fbc51ffb640 1 -- 172.21.15.46:0/3218884445 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbc440691e0 con 0x7fbc541000e0 2024-09-10T04:11:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.098+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fbc54198d90 con 0x7fbc541000e0 2024-09-10T04:11:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.099+0000 7fbc51ffb640 1 -- 172.21.15.46:0/3218884445 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7fbc4406e090 con 0x7fbc541000e0 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:11:03.102 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:11:03.103 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:11:03.104 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:11:03.105 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:11:03.105 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:11:03.105 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:11:03.105 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:11:03.105 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:03.105 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:03.105 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:03.106 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:03.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:11:03.106 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:11:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:11:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbc3c07abf0 msgr2=0x7fbc3c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbc3c07abf0 0x7fbc3c07d0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbc48031460 tx=0x7fbc480095f0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc541000e0 msgr2=0x7fbc5419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc541000e0 0x7fbc5419d6a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400d8d0 tx=0x7fbc4400dd90 comp rx=0 tx=0).stop 2024-09-10T04:11:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 shutdown_connections 2024-09-10T04:11:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbc3c07abf0 0x7fbc3c07d0b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc54100a90 0x7fbc5419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 --2- 172.21.15.46:0/3218884445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc541000e0 0x7fbc5419d6a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.101+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 >> 172.21.15.46:0/3218884445 conn(0x7fbc540fbc90 msgr2=0x7fbc540fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:03.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.102+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 shutdown_connections 2024-09-10T04:11:03.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.102+0000 7fbc5ad38640 1 -- 172.21.15.46:0/3218884445 wait complete. 2024-09-10T04:11:03.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 -- 172.21.15.46:0/4036336649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338102b70 msgr2=0x7f0338102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 --2- 172.21.15.46:0/4036336649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338102b70 0x7f0338102f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0334009900 tx=0x7f033402e970 comp rx=0 tx=0).stop 2024-09-10T04:11:03.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 -- 172.21.15.46:0/4036336649 shutdown_connections 2024-09-10T04:11:03.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 --2- 172.21.15.46:0/4036336649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338103490 0x7f033810b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 --2- 172.21.15.46:0/4036336649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338102b70 0x7f0338102f50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 -- 172.21.15.46:0/4036336649 >> 172.21.15.46:0/4036336649 conn(0x7f03380fc550 msgr2=0x7f03380fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 -- 172.21.15.46:0/4036336649 shutdown_connections 2024-09-10T04:11:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.189+0000 7f033f64c640 1 -- 172.21.15.46:0/4036336649 wait complete. 2024-09-10T04:11:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033f64c640 1 Processor -- start 2024-09-10T04:11:03.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033f64c640 1 -- start start 2024-09-10T04:11:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033f64c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338102b70 0x7f033819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033f64c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338103490 0x7f033819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:03.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033f64c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f033810cbd0 con 0x7f0338103490 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033f64c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f033810ca50 con 0x7f0338102b70 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033cbe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338103490 0x7f033819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033cbe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338103490 0x7f033819b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35902/0 (socket says 172.21.15.46:35902) 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033cbe7640 1 -- 172.21.15.46:0/2384615591 learned_addr learned my addr 172.21.15.46:0/2384615591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033d3e8640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338102b70 0x7f033819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033cbe7640 1 -- 172.21.15.46:0/2384615591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338102b70 msgr2=0x7f033819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033cbe7640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338102b70 0x7f033819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.190+0000 7f033cbe7640 1 -- 172.21.15.46:0/2384615591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03381955a0 con 0x7f0338103490 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.191+0000 7f033cbe7640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338103490 0x7f033819b920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f032800e970 tx=0x7f032800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.191+0000 7f032e7fc640 1 -- 172.21.15.46:0/2384615591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f032800cac0 con 0x7f0338103490 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.191+0000 7f032e7fc640 1 -- 172.21.15.46:0/2384615591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03280054e0 con 0x7f0338103490 2024-09-10T04:11:03.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.191+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0338195890 con 0x7f0338103490 2024-09-10T04:11:03.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.191+0000 7f032e7fc640 1 -- 172.21.15.46:0/2384615591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0328010430 con 0x7f0338103490 2024-09-10T04:11:03.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.191+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0338195e20 con 0x7f0338103490 2024-09-10T04:11:03.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.192+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033806a4c0 con 0x7f0338103490 2024-09-10T04:11:03.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.192+0000 7f032e7fc640 1 -- 172.21.15.46:0/2384615591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0328004f00 con 0x7f0338103490 2024-09-10T04:11:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.193+0000 7f032e7fc640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f032407ac40 0x7f032407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.193+0000 7f033d3e8640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f032407ac40 0x7f032407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.193+0000 7f032e7fc640 1 -- 172.21.15.46:0/2384615591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6070+0+0 (secure 0 0 0) 0x7f032809e220 con 0x7f0338103490 2024-09-10T04:11:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.194+0000 7f033d3e8640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f032407ac40 0x7f032407d100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0334000c00 tx=0x7f0334004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.198+0000 7f032e7fc640 1 -- 172.21.15.46:0/2384615591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0328069a90 con 0x7f0338103490 2024-09-10T04:11:03.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.404+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0338196130 con 0x7f032407ac40 2024-09-10T04:11:03.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.406+0000 7f032e7fc640 1 -- 172.21.15.46:0/2384615591 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f0338196130 con 0x7f032407ac40 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "7/23 daemons upgraded", 2024-09-10T04:11:03.408 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T04:11:03.409 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:11:03.409 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f032407ac40 msgr2=0x7f032407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f032407ac40 0x7f032407d100 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0334000c00 tx=0x7f0334004f30 comp rx=0 tx=0).stop 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338103490 msgr2=0x7f033819b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338103490 0x7f033819b920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f032800e970 tx=0x7f032800ed70 comp rx=0 tx=0).stop 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 shutdown_connections 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f032407ac40 0x7f032407d100 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0338103490 0x7f033819b920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 --2- 172.21.15.46:0/2384615591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0338102b70 0x7f033819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 >> 172.21.15.46:0/2384615591 conn(0x7f03380fc550 msgr2=0x7f0338108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 shutdown_connections 2024-09-10T04:11:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.408+0000 7f033f64c640 1 -- 172.21.15.46:0/2384615591 wait complete. 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 -- 172.21.15.46:0/2904899878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 msgr2=0x7f2448106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 --2- 172.21.15.46:0/2904899878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448106bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2444009880 tx=0x7f244402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 -- 172.21.15.46:0/2904899878 shutdown_connections 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 --2- 172.21.15.46:0/2904899878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24481007f0 0x7f2448100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 --2- 172.21.15.46:0/2904899878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448106bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 -- 172.21.15.46:0/2904899878 >> 172.21.15.46:0/2904899878 conn(0x7f24480fc550 msgr2=0x7f24480fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 -- 172.21.15.46:0/2904899878 shutdown_connections 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.497+0000 7f244ffb7640 1 -- 172.21.15.46:0/2904899878 wait complete. 2024-09-10T04:11:03.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244ffb7640 1 Processor -- start 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244ffb7640 1 -- start start 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244ffb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24481007f0 0x7f2448107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244ffb7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244ffb7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f244810dca0 con 0x7f24481067f0 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244ffb7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f244810db20 con 0x7f24481007f0 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244d552640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244d552640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35922/0 (socket says 172.21.15.46:35922) 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244d552640 1 -- 172.21.15.46:0/3507560270 learned_addr learned my addr 172.21.15.46:0/3507560270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244dd53640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24481007f0 0x7f2448107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244d552640 1 -- 172.21.15.46:0/3507560270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24481007f0 msgr2=0x7f2448107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244d552640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24481007f0 0x7f2448107720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244d552640 1 -- 172.21.15.46:0/3507560270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24481081a0 con 0x7f24481067f0 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.498+0000 7f244dd53640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24481007f0 0x7f2448107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.499+0000 7f244d552640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448107c60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f243800d8d0 tx=0x7f243800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.499+0000 7f243effd640 1 -- 172.21.15.46:0/3507560270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2438017070 con 0x7f24481067f0 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.499+0000 7f243effd640 1 -- 172.21.15.46:0/3507560270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2438010460 con 0x7f24481067f0 2024-09-10T04:11:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.499+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2448108490 con 0x7f24481067f0 2024-09-10T04:11:03.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.499+0000 7f243effd640 1 -- 172.21.15.46:0/3507560270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f243800f5f0 con 0x7f24481067f0 2024-09-10T04:11:03.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.499+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24481a3dd0 con 0x7f24481067f0 2024-09-10T04:11:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.500+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2448101f30 con 0x7f24481067f0 2024-09-10T04:11:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.501+0000 7f243effd640 1 -- 172.21.15.46:0/3507560270 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2438009a30 con 0x7f24481067f0 2024-09-10T04:11:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.501+0000 7f243effd640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f243407abf0 0x7f243407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.502+0000 7f243effd640 1 -- 172.21.15.46:0/3507560270 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6070+0+0 (secure 0 0 0) 0x7f243809dab0 con 0x7f24481067f0 2024-09-10T04:11:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.502+0000 7f244dd53640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f243407abf0 0x7f243407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.502+0000 7f244dd53640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f243407abf0 0x7f243407d0b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2444006010 tx=0x7f24440095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:03.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.507+0000 7f243effd640 1 -- 172.21.15.46:0/3507560270 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f24380692a0 con 0x7f24481067f0 2024-09-10T04:11:03.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.786+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f24481a40b0 con 0x7f24481067f0 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: pgmap v135: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3600397187' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:03.789 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:03 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3218884445' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:11:03.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.787+0000 7f243effd640 1 -- 172.21.15.46:0/3507560270 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+1294 (secure 0 0 0) 0x7f243806e150 con 0x7f24481067f0 2024-09-10T04:11:03.790 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 46/312 objects degraded (14.744%), 15 pgs degraded 2024-09-10T04:11:03.790 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:11:03.790 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:11:03.790 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: osd.1 (root=default,host=smithi046) is down 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 46/312 objects degraded (14.744%), 15 pgs degraded 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-09-10T04:11:03.791 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-09-10T04:11:03.792 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f243407abf0 msgr2=0x7f243407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f243407abf0 0x7f243407d0b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2444006010 tx=0x7f24440095f0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 msgr2=0x7f2448107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448107c60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f243800d8d0 tx=0x7f243800dd90 comp rx=0 tx=0).stop 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 shutdown_connections 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f243407abf0 0x7f243407d0b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24481067f0 0x7f2448107c60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 --2- 172.21.15.46:0/3507560270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24481007f0 0x7f2448107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:03.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 >> 172.21.15.46:0/3507560270 conn(0x7f24480fc550 msgr2=0x7f2448069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 shutdown_connections 2024-09-10T04:11:03.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:03.789+0000 7f244ffb7640 1 -- 172.21.15.46:0/3507560270 wait complete. 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: pgmap v135: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3600397187' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:03.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:03 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3218884445' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:11:04.412 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:04 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[123892]: 2024-09-10T04:11:04.141+0000 7f1a808bf640 -1 Falling back to public interface 2024-09-10T04:11:04.736 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:04 smithi046 ceph-mon[109506]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:04.736 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:04 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3507560270' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:11:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:04 smithi070 ceph-mon[79774]: from='client.34182 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:04 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3507560270' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:11:05.582 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:05 smithi046 ceph-mon[109506]: pgmap v136: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:05.822 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:05 smithi070 ceph-mon[79774]: pgmap v136: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:06.755 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:06 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 46/312 objects degraded (14.744%), 15 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:06 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 46/312 objects degraded (14.744%), 15 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:07 smithi070 ceph-mon[79774]: pgmap v137: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:07 smithi046 ceph-mon[109506]: pgmap v137: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:08.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:09.564 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:09 smithi046 ceph-mon[109506]: pgmap v138: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:09 smithi070 ceph-mon[79774]: pgmap v138: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:10.142 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:09 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[123892]: 2024-09-10T04:11:09.887+0000 7f1a808bf640 -1 osd.1 50 log_to_monitors true 2024-09-10T04:11:10.759 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:10 smithi046 ceph-mon[109506]: from='osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T04:11:10.759 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:11:10 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[123892]: 2024-09-10T04:11:10.516+0000 7f1a74661640 -1 osd.1 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:11:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:10 smithi070 ceph-mon[79774]: from='osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T04:11:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:11 smithi070 ceph-mon[79774]: pgmap v139: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:11 smithi070 ceph-mon[79774]: from='osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T04:11:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:11 smithi070 ceph-mon[79774]: osdmap e53: 6 total, 5 up, 6 in 2024-09-10T04:11:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:11 smithi070 ceph-mon[79774]: from='osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]} : dispatch 2024-09-10T04:11:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:11 smithi046 ceph-mon[109506]: pgmap v139: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 46/312 objects degraded (14.744%) 2024-09-10T04:11:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:11 smithi046 ceph-mon[109506]: from='osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T04:11:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:11 smithi046 ceph-mon[109506]: osdmap e53: 6 total, 5 up, 6 in 2024-09-10T04:11:11.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:11 smithi046 ceph-mon[109506]: from='osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]} : dispatch 2024-09-10T04:11:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:11:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559] boot 2024-09-10T04:11:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: osdmap e54: 6 total, 6 up, 6 in 2024-09-10T04:11:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T04:11:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:12 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-10T04:11:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: osd.1 [v2:172.21.15.46:6810/1304569559,v1:172.21.15.46:6811/1304569559] boot 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: osdmap e54: 6 total, 6 up, 6 in 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:12 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-10T04:11:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:13 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T04:11:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:13 smithi070 ceph-mon[79774]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-10T04:11:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:13 smithi070 ceph-mon[79774]: pgmap v142: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 46/312 objects degraded (14.744%) 2024-09-10T04:11:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:13 smithi070 ceph-mon[79774]: osdmap e55: 6 total, 6 up, 6 in 2024-09-10T04:11:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:13 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T04:11:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:13 smithi046 ceph-mon[109506]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-10T04:11:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:13 smithi046 ceph-mon[109506]: pgmap v142: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 46/312 objects degraded (14.744%) 2024-09-10T04:11:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:13 smithi046 ceph-mon[109506]: osdmap e55: 6 total, 6 up, 6 in 2024-09-10T04:11:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:14 smithi070 ceph-mon[79774]: pgmap v144: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 24/312 objects degraded (7.692%) 2024-09-10T04:11:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:14 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 24/312 objects degraded (7.692%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:14 smithi046 ceph-mon[109506]: pgmap v144: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s; 24/312 objects degraded (7.692%) 2024-09-10T04:11:14.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:14 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 24/312 objects degraded (7.692%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:16.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:16 smithi070 ceph-mon[79774]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/312 objects degraded (7.692%), 7 pgs degraded) 2024-09-10T04:11:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:16 smithi046 ceph-mon[109506]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 24/312 objects degraded (7.692%), 7 pgs degraded) 2024-09-10T04:11:17.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:17 smithi070 ceph-mon[79774]: pgmap v145: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-10T04:11:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:17 smithi046 ceph-mon[109506]: pgmap v145: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-10T04:11:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:19 smithi070 ceph-mon[79774]: pgmap v146: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:11:19.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:19 smithi046 ceph-mon[109506]: pgmap v146: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:11:21.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:21 smithi046 ceph-mon[109506]: pgmap v147: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T04:11:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:21 smithi070 ceph-mon[79774]: pgmap v147: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T04:11:23.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:23 smithi046 ceph-mon[109506]: pgmap v148: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T04:11:23.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:23.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:23.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:23 smithi070 ceph-mon[79774]: pgmap v148: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T04:11:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:25.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:25 smithi070 ceph-mon[79774]: pgmap v149: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:11:25.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:25 smithi046 ceph-mon[109506]: pgmap v149: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:11:27.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:27 smithi046 ceph-mon[109506]: pgmap v150: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:11:27.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-10T04:11:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:27 smithi070 ceph-mon[79774]: pgmap v150: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:11:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-10T04:11:28.480 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:28 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T04:11:28.480 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:28 smithi046 ceph-mon[109506]: Upgrade: osd.2 is safe to restart 2024-09-10T04:11:28.480 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:28 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:28.480 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:28 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T04:11:28.481 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:28 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:11:28.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:28 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T04:11:28.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:28 smithi070 ceph-mon[79774]: Upgrade: osd.2 is safe to restart 2024-09-10T04:11:28.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:28 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:28.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:28 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T04:11:28.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:28 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:11:29.458 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:29 smithi046 ceph-mon[109506]: pgmap v151: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:11:29.458 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:29 smithi046 ceph-mon[109506]: Upgrade: Updating osd.2 2024-09-10T04:11:29.458 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:29 smithi046 ceph-mon[109506]: Deploying daemon osd.2 on smithi046 2024-09-10T04:11:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:29 smithi070 ceph-mon[79774]: pgmap v151: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:11:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:29 smithi070 ceph-mon[79774]: Upgrade: Updating osd.2 2024-09-10T04:11:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:29 smithi070 ceph-mon[79774]: Deploying daemon osd.2 on smithi046 2024-09-10T04:11:31.424 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:31 smithi046 ceph-mon[109506]: pgmap v152: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:11:31.424 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:31 smithi046 systemd[1]: Stopping Ceph osd.2 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:11:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:31 smithi070 ceph-mon[79774]: pgmap v152: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:11:31.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:31 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[64430]: 2024-09-10T04:11:31.421+0000 7fd879175640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:11:31.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:31 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[64430]: 2024-09-10T04:11:31.421+0000 7fd879175640 -1 osd.2 55 *** Got signal Terminated *** 2024-09-10T04:11:31.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:31 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[64430]: 2024-09-10T04:11:31.421+0000 7fd879175640 -1 osd.2 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:11:32.560 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:32 smithi046 ceph-mon[109506]: osd.2 marked itself down and dead 2024-09-10T04:11:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:32 smithi070 ceph-mon[79774]: osd.2 marked itself down and dead 2024-09-10T04:11:32.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:32 smithi046 podman[128855]: 2024-09-10 04:11:32.559156974 +0000 UTC m=+1.251210003 container died c85ce2f99b9e33199ac755316113fe389bce2542cb97bdc5d7f0e116f27a2199 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T04:11:33.210 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:32 smithi046 podman[128855]: 2024-09-10 04:11:32.947377431 +0000 UTC m=+1.639430435 container cleanup c85ce2f99b9e33199ac755316113fe389bce2542cb97bdc5d7f0e116f27a2199 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:11:33.211 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:32 smithi046 bash[128855]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2 2024-09-10T04:11:33.483 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:33 smithi046 podman[128873]: 2024-09-10 04:11:33.232077454 +0000 UTC m=+0.665567955 container remove c85ce2f99b9e33199ac755316113fe389bce2542cb97bdc5d7f0e116f27a2199 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T04:11:33.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:33 smithi046 ceph-mon[109506]: pgmap v153: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:11:33.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:33 smithi046 ceph-mon[109506]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:11:33.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:33 smithi046 ceph-mon[109506]: osdmap e56: 6 total, 5 up, 6 in 2024-09-10T04:11:33.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:33 smithi070 ceph-mon[79774]: pgmap v153: 65 pgs: 65 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:11:33.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:33 smithi070 ceph-mon[79774]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:11:33.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:33 smithi070 ceph-mon[79774]: osdmap e56: 6 total, 5 up, 6 in 2024-09-10T04:11:33.743 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:33 smithi046 podman[128963]: 2024-09-10 04:11:33.48779613 +0000 UTC m=+0.016750743 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:11:33.743 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:33 smithi046 podman[128963]: 2024-09-10 04:11:33.633314299 +0000 UTC m=+0.162268895 container create bf5ea5e5247426fed78759cb51438e7618fa3279da270ef7bf3472e5a932feee (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731) 2024-09-10T04:11:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 -- 172.21.15.46:0/1877094244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 msgr2=0x7f662c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:33.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 --2- 172.21.15.46:0/1877094244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c107ce0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f661c009880 tx=0x7f661c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 -- 172.21.15.46:0/1877094244 shutdown_connections 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 --2- 172.21.15.46:0/1877094244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f662c101900 0x7f662c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 --2- 172.21.15.46:0/1877094244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c107ce0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 -- 172.21.15.46:0/1877094244 >> 172.21.15.46:0/1877094244 conn(0x7f662c0fd620 msgr2=0x7f662c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 -- 172.21.15.46:0/1877094244 shutdown_connections 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.879+0000 7f6630e3b640 1 -- 172.21.15.46:0/1877094244 wait complete. 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6630e3b640 1 Processor -- start 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6630e3b640 1 -- start start 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6630e3b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f662c101900 0x7f662c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6630e3b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:33.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6630e3b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f662c10edd0 con 0x7f662c107900 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6630e3b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f662c10ec50 con 0x7f662c101900 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6623fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6623fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58222/0 (socket says 172.21.15.46:58222) 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f6623fff640 1 -- 172.21.15.46:0/2447836276 learned_addr learned my addr 172.21.15.46:0/2447836276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.880+0000 7f662a59c640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f662c101900 0x7f662c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f6623fff640 1 -- 172.21.15.46:0/2447836276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f662c101900 msgr2=0x7f662c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f6623fff640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f662c101900 0x7f662c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f6623fff640 1 -- 172.21.15.46:0/2447836276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c1977c0 con 0x7f662c107900 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f662a59c640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f662c101900 0x7f662c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f6623fff640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c19db80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f661400d6f0 tx=0x7f661400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f66237fe640 1 -- 172.21.15.46:0/2447836276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6614017070 con 0x7f662c107900 2024-09-10T04:11:33.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f66237fe640 1 -- 172.21.15.46:0/2447836276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f661400de00 con 0x7f662c107900 2024-09-10T04:11:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662c197ab0 con 0x7f662c107900 2024-09-10T04:11:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f66237fe640 1 -- 172.21.15.46:0/2447836276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6614004bb0 con 0x7f662c107900 2024-09-10T04:11:33.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.881+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662c198040 con 0x7f662c107900 2024-09-10T04:11:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.882+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f662c103040 con 0x7f662c107900 2024-09-10T04:11:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.883+0000 7f66237fe640 1 -- 172.21.15.46:0/2447836276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6614004e60 con 0x7f662c107900 2024-09-10T04:11:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.884+0000 7f66237fe640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f661807abf0 0x7f661807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.884+0000 7f66237fe640 1 -- 172.21.15.46:0/2447836276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6070+0+0 (secure 0 0 0) 0x7f661409d820 con 0x7f662c107900 2024-09-10T04:11:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.884+0000 7f662a59c640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f661807abf0 0x7f661807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:33.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.884+0000 7f662a59c640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f661807abf0 0x7f661807d0b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f661c000c00 tx=0x7f661c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:33.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:33.888+0000 7f66237fe640 1 -- 172.21.15.46:0/2447836276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6614069010 con 0x7f662c107900 2024-09-10T04:11:34.087 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:33 smithi046 podman[128963]: 2024-09-10 04:11:33.835895779 +0000 UTC m=+0.364850384 container init bf5ea5e5247426fed78759cb51438e7618fa3279da270ef7bf3472e5a932feee (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T04:11:34.087 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:33 smithi046 podman[128963]: 2024-09-10 04:11:33.839591663 +0000 UTC m=+0.368546265 container start bf5ea5e5247426fed78759cb51438e7618fa3279da270ef7bf3472e5a932feee (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:11:34.087 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:33 smithi046 podman[128963]: 2024-09-10 04:11:33.952593704 +0000 UTC m=+0.481548307 container attach bf5ea5e5247426fed78759cb51438e7618fa3279da270ef7bf3472e5a932feee (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:11:34.087 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:34 smithi046 podman[128963]: 2024-09-10 04:11:34.04243462 +0000 UTC m=+0.571389215 container died bf5ea5e5247426fed78759cb51438e7618fa3279da270ef7bf3472e5a932feee (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:11:34.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.098+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f662c1983b0 con 0x7f661807abf0 2024-09-10T04:11:34.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.099+0000 7f66237fe640 1 -- 172.21.15.46:0/2447836276 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f662c1983b0 con 0x7f661807abf0 2024-09-10T04:11:34.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f661807abf0 msgr2=0x7f661807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f661807abf0 0x7f661807d0b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f661c000c00 tx=0x7f661c031400 comp rx=0 tx=0).stop 2024-09-10T04:11:34.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 msgr2=0x7f662c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c19db80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f661400d6f0 tx=0x7f661400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 shutdown_connections 2024-09-10T04:11:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f661807abf0 0x7f661807d0b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f662c107900 0x7f662c19db80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 --2- 172.21.15.46:0/2447836276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f662c101900 0x7f662c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 >> 172.21.15.46:0/2447836276 conn(0x7f662c0fd620 msgr2=0x7f662c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 shutdown_connections 2024-09-10T04:11:34.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.101+0000 7f6630e3b640 1 -- 172.21.15.46:0/2447836276 wait complete. 2024-09-10T04:11:34.112 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:11:34.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 -- 172.21.15.46:0/948793282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 msgr2=0x7f8cd4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/948793282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd4102e90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009880 tx=0x7f8cc402a530 comp rx=0 tx=0).stop 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 -- 172.21.15.46:0/948793282 shutdown_connections 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/948793282 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd41033d0 0x7f8cd410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/948793282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd4102e90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 -- 172.21.15.46:0/948793282 >> 172.21.15.46:0/948793282 conn(0x7f8cd40fc550 msgr2=0x7f8cd40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 -- 172.21.15.46:0/948793282 shutdown_connections 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.188+0000 7f8cd9a13640 1 -- 172.21.15.46:0/948793282 wait complete. 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.189+0000 7f8cd9a13640 1 Processor -- start 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.189+0000 7f8cd9a13640 1 -- start start 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.189+0000 7f8cd9a13640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.189+0000 7f8cd9a13640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd41033d0 0x7f8cd419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.189+0000 7f8cd9a13640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd410edc0 con 0x7f8cd4102ab0 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.189+0000 7f8cd9a13640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd410ec40 con 0x7f8cd41033d0 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58248/0 (socket says 172.21.15.46:58248) 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd2ffd640 1 -- 172.21.15.46:0/2213803046 learned_addr learned my addr 172.21.15.46:0/2213803046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd27fc640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd41033d0 0x7f8cd419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd2ffd640 1 -- 172.21.15.46:0/2213803046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd41033d0 msgr2=0x7f8cd419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd2ffd640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd41033d0 0x7f8cd419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd2ffd640 1 -- 172.21.15.46:0/2213803046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd41999a0 con 0x7f8cd4102ab0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd2ffd640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd419f7e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4031460 tx=0x7f8cc40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd8a11640 1 -- 172.21.15.46:0/2213803046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cc403f070 con 0x7f8cd4102ab0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd8a11640 1 -- 172.21.15.46:0/2213803046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cc40035f0 con 0x7f8cd4102ab0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd8a11640 1 -- 172.21.15.46:0/2213803046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cc4035030 con 0x7f8cd4102ab0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd4199c30 con 0x7f8cd4102ab0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.190+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd419a160 con 0x7f8cd4102ab0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.191+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd406a580 con 0x7f8cd4102ab0 2024-09-10T04:11:34.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.192+0000 7f8cd8a11640 1 -- 172.21.15.46:0/2213803046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8cc4042070 con 0x7f8cd4102ab0 2024-09-10T04:11:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.193+0000 7f8cd8a11640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8cbc07ac40 0x7f8cbc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.193+0000 7f8cd8a11640 1 -- 172.21.15.46:0/2213803046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6070+0+0 (secure 0 0 0) 0x7f8cc40be4f0 con 0x7f8cd4102ab0 2024-09-10T04:11:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.193+0000 7f8cd27fc640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8cbc07ac40 0x7f8cbc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.193+0000 7f8cd27fc640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8cbc07ac40 0x7f8cbc07d100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80058c0 tx=0x7f8cc8005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:34.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.197+0000 7f8cd8a11640 1 -- 172.21.15.46:0/2213803046 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8cc4089d60 con 0x7f8cd4102ab0 2024-09-10T04:11:34.391 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:34 smithi046 ceph-mon[109506]: osdmap e57: 6 total, 5 up, 6 in 2024-09-10T04:11:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.404+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8cd419aed0 con 0x7f8cbc07ac40 2024-09-10T04:11:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.406+0000 7f8cd8a11640 1 -- 172.21.15.46:0/2213803046 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f8cd419aed0 con 0x7f8cbc07ac40 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8cbc07ac40 msgr2=0x7f8cbc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8cbc07ac40 0x7f8cbc07d100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80058c0 tx=0x7f8cc8005850 comp rx=0 tx=0).stop 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 msgr2=0x7f8cd419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd419f7e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4031460 tx=0x7f8cc40045e0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 shutdown_connections 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8cbc07ac40 0x7f8cbc07d100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cd41033d0 0x7f8cd419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 --2- 172.21.15.46:0/2213803046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4102ab0 0x7f8cd419f7e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 >> 172.21.15.46:0/2213803046 conn(0x7f8cd40fc550 msgr2=0x7f8cd4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:34.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 shutdown_connections 2024-09-10T04:11:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.408+0000 7f8cd9a13640 1 -- 172.21.15.46:0/2213803046 wait complete. 2024-09-10T04:11:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/3046126281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e38102ac0 msgr2=0x7f5e38102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/3046126281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e38102ac0 0x7f5e38102ea0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009880 tx=0x7f5e2802a530 comp rx=0 tx=0).stop 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/3046126281 shutdown_connections 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/3046126281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e381033e0 0x7f5e3810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/3046126281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e38102ac0 0x7f5e38102ea0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/3046126281 >> 172.21.15.46:0/3046126281 conn(0x7f5e380fc520 msgr2=0x7f5e380fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/3046126281 shutdown_connections 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.492+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/3046126281 wait complete. 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e3dbf5640 1 Processor -- start 2024-09-10T04:11:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e3dbf5640 1 -- start start 2024-09-10T04:11:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e3dbf5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e38102ac0 0x7f5e3819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e3dbf5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e381033e0 0x7f5e3819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e3dbf5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810edf0 con 0x7f5e381033e0 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e3dbf5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5e3810ec70 con 0x7f5e38102ac0 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e2effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e381033e0 0x7f5e3819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e2effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e381033e0 0x7f5e3819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58268/0 (socket says 172.21.15.46:58268) 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e2effd640 1 -- 172.21.15.46:0/590208378 learned_addr learned my addr 172.21.15.46:0/590208378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.493+0000 7f5e377fe640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e38102ac0 0x7f5e3819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.494+0000 7f5e2effd640 1 -- 172.21.15.46:0/590208378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e38102ac0 msgr2=0x7f5e3819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.494+0000 7f5e2effd640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e38102ac0 0x7f5e3819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.494+0000 7f5e2effd640 1 -- 172.21.15.46:0/590208378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e381977e0 con 0x7f5e381033e0 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.494+0000 7f5e2effd640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e381033e0 0x7f5e3819dba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000d6d0 tx=0x7f5e2000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.494+0000 7f5e357fa640 1 -- 172.21.15.46:0/590208378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e20017070 con 0x7f5e381033e0 2024-09-10T04:11:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.494+0000 7f5e357fa640 1 -- 172.21.15.46:0/590208378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e200054e0 con 0x7f5e381033e0 2024-09-10T04:11:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.494+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e38197ad0 con 0x7f5e381033e0 2024-09-10T04:11:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.495+0000 7f5e357fa640 1 -- 172.21.15.46:0/590208378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e20004ae0 con 0x7f5e381033e0 2024-09-10T04:11:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.495+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e38198060 con 0x7f5e381033e0 2024-09-10T04:11:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.496+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e3806aa90 con 0x7f5e381033e0 2024-09-10T04:11:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.496+0000 7f5e357fa640 1 -- 172.21.15.46:0/590208378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5e20004d90 con 0x7f5e381033e0 2024-09-10T04:11:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.496+0000 7f5e357fa640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e2407ac40 0x7f5e2407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.497+0000 7f5e357fa640 1 -- 172.21.15.46:0/590208378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6070+0+0 (secure 0 0 0) 0x7f5e2009d720 con 0x7f5e381033e0 2024-09-10T04:11:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.497+0000 7f5e377fe640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e2407ac40 0x7f5e2407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.497+0000 7f5e377fe640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e2407ac40 0x7f5e2407d100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e28031460 tx=0x7f5e28009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.501+0000 7f5e357fa640 1 -- 172.21.15.46:0/590208378 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5e20068f90 con 0x7f5e381033e0 2024-09-10T04:11:34.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:34 smithi070 ceph-mon[79774]: osdmap e57: 6 total, 5 up, 6 in 2024-09-10T04:11:34.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.696+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5e38198310 con 0x7f5e2407ac40 2024-09-10T04:11:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.706+0000 7f5e357fa640 1 -- 172.21.15.46:0/590208378 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f5e38198310 con 0x7f5e2407ac40 2024-09-10T04:11:34.708 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:11:34.708 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (3m) 27s ago 14m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:11:34.708 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (5m) 27s ago 14m 8980k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (5m) 86s ago 13m 8539k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (94s) 27s ago 14m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (89s) 86s ago 13m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (2m) 27s ago 14m 77.3M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (9m) 27s ago 9m 90.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (9m) 27s ago 9m 145M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (9m) 86s ago 9m 26.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (9m) 86s ago 9m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (5m) 27s ago 15m 526M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (4m) 86s ago 13m 455M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (2m) 27s ago 16m 41.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (107s) 86s ago 13m 25.7M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (3m) 27s ago 14m 12.1M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:11:34.709 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (3m) 86s ago 13m 12.0M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:11:34.710 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (69s) 27s ago 12m 81.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:11:34.710 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (32s) 27s ago 11m 12.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:11:34.710 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (11m) 27s ago 11m 180M 1217M 19.1.1-298-gb53b6d3e 54467abc55f8 c85ce2f99b9e 2024-09-10T04:11:34.710 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (11m) 86s ago 11m 236M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:11:34.710 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (10m) 86s ago 10m 210M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:11:34.710 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (10m) 86s ago 10m 196M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:11:34.710 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (3m) 27s ago 14m 55.8M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e2407ac40 msgr2=0x7f5e2407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e2407ac40 0x7f5e2407d100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5e28031460 tx=0x7f5e28009560 comp rx=0 tx=0).stop 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e381033e0 msgr2=0x7f5e3819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e381033e0 0x7f5e3819dba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000d6d0 tx=0x7f5e2000db90 comp rx=0 tx=0).stop 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 shutdown_connections 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e2407ac40 0x7f5e2407d100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e381033e0 0x7f5e3819dba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 --2- 172.21.15.46:0/590208378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e38102ac0 0x7f5e3819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 >> 172.21.15.46:0/590208378 conn(0x7f5e380fc520 msgr2=0x7f5e38108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 shutdown_connections 2024-09-10T04:11:34.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.709+0000 7f5e3dbf5640 1 -- 172.21.15.46:0/590208378 wait complete. 2024-09-10T04:11:34.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.807+0000 7f7ad48e0640 1 -- 172.21.15.46:0/1491494114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01067f0 msgr2=0x7f7ad0106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.807+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/1491494114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01067f0 0x7f7ad0106bd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000dc90 tx=0x7f7ac0032a30 comp rx=0 tx=0).stop 2024-09-10T04:11:34.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.808+0000 7f7ad48e0640 1 -- 172.21.15.46:0/1491494114 shutdown_connections 2024-09-10T04:11:34.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.808+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/1491494114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ad01007f0 0x7f7ad0100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.808+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/1491494114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01067f0 0x7f7ad0106bd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.808+0000 7f7ad48e0640 1 -- 172.21.15.46:0/1491494114 >> 172.21.15.46:0/1491494114 conn(0x7f7ad00fc4d0 msgr2=0x7f7ad00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:34.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.809+0000 7f7ad48e0640 1 -- 172.21.15.46:0/1491494114 shutdown_connections 2024-09-10T04:11:34.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.809+0000 7f7ad48e0640 1 -- 172.21.15.46:0/1491494114 wait complete. 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.810+0000 7f7ad48e0640 1 Processor -- start 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.810+0000 7f7ad48e0640 1 -- start start 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.810+0000 7f7ad48e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01007f0 0x7f7ad0073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.810+0000 7f7ad48e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ad01067f0 0x7f7ad006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.810+0000 7f7ad48e0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad006c1f0 con 0x7f7ad01007f0 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.810+0000 7f7ad48e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad006c070 con 0x7f7ad01067f0 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7ace59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01007f0 0x7f7ad0073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7ace59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01007f0 0x7f7ad0073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58286/0 (socket says 172.21.15.46:58286) 2024-09-10T04:11:34.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7ace59c640 1 -- 172.21.15.46:0/2634717167 learned_addr learned my addr 172.21.15.46:0/2634717167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:34.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7acdd9b640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ad01067f0 0x7f7ad006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7ace59c640 1 -- 172.21.15.46:0/2634717167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ad01067f0 msgr2=0x7f7ad006ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:34.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7ace59c640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ad01067f0 0x7f7ad006ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:34.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7ace59c640 1 -- 172.21.15.46:0/2634717167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac00047a0 con 0x7f7ad01007f0 2024-09-10T04:11:34.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7acdd9b640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ad01067f0 0x7f7ad006ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:11:34.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.811+0000 7f7ace59c640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01007f0 0x7f7ad0073ef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0033020 tx=0x7f7ac0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:34.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.813+0000 7f7abf7fe640 1 -- 172.21.15.46:0/2634717167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ac0044070 con 0x7f7ad01007f0 2024-09-10T04:11:34.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.813+0000 7f7abf7fe640 1 -- 172.21.15.46:0/2634717167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ac0037480 con 0x7f7ad01007f0 2024-09-10T04:11:34.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.813+0000 7f7abf7fe640 1 -- 172.21.15.46:0/2634717167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ac003b040 con 0x7f7ad01007f0 2024-09-10T04:11:34.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.813+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad00706a0 con 0x7f7ad01007f0 2024-09-10T04:11:34.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.813+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad0070c30 con 0x7f7ad01007f0 2024-09-10T04:11:34.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.814+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad0101f30 con 0x7f7ad01007f0 2024-09-10T04:11:34.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.815+0000 7f7abf7fe640 1 -- 172.21.15.46:0/2634717167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7ac0035070 con 0x7f7ad01007f0 2024-09-10T04:11:34.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.816+0000 7f7abf7fe640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7ab807acf0 0x7f7ab807d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:34.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.816+0000 7f7abf7fe640 1 -- 172.21.15.46:0/2634717167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6070+0+0 (secure 0 0 0) 0x7f7ac00c6a30 con 0x7f7ad01007f0 2024-09-10T04:11:34.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.816+0000 7f7acdd9b640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7ab807acf0 0x7f7ab807d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:34.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.817+0000 7f7acdd9b640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7ab807acf0 0x7f7ab807d1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0071750 tx=0x7f7ac8009520 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:34.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:34.820+0000 7f7abf7fe640 1 -- 172.21.15.46:0/2634717167 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7ac0005670 con 0x7f7ad01007f0 2024-09-10T04:11:34.973 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:34 smithi046 podman[129008]: 2024-09-10 04:11:34.728743915 +0000 UTC m=+0.675270383 container remove bf5ea5e5247426fed78759cb51438e7618fa3279da270ef7bf3472e5a932feee (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T04:11:34.973 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:34 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-10T04:11:34.973 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:34 smithi046 systemd[1]: Stopped Ceph osd.2 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:11:34.973 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:34 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.2.service: Consumed 49.480s CPU time. 2024-09-10T04:11:34.973 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:34 smithi046 systemd[1]: Starting Ceph osd.2 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:11:35.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.092+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7ad0103a00 con 0x7f7ad01007f0 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.093+0000 7f7abf7fe640 1 -- 172.21.15.46:0/2634717167 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f7ac0092340 con 0x7f7ad01007f0 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:11:35.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 3, 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 7, 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:11:35.097 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7ab807acf0 msgr2=0x7f7ab807d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7ab807acf0 0x7f7ab807d1b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0071750 tx=0x7f7ac8009520 comp rx=0 tx=0).stop 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01007f0 msgr2=0x7f7ad0073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01007f0 0x7f7ad0073ef0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0033020 tx=0x7f7ac0004e20 comp rx=0 tx=0).stop 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 shutdown_connections 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7ab807acf0 0x7f7ab807d1b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ad01067f0 0x7f7ad006ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.095+0000 7f7ad48e0640 1 --2- 172.21.15.46:0/2634717167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ad01007f0 0x7f7ad0073ef0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.096+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 >> 172.21.15.46:0/2634717167 conn(0x7f7ad00fc4d0 msgr2=0x7f7ad00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.096+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 shutdown_connections 2024-09-10T04:11:35.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.096+0000 7f7ad48e0640 1 -- 172.21.15.46:0/2634717167 wait complete. 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 -- 172.21.15.46:0/2339406159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118101900 msgr2=0x7f6118101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 --2- 172.21.15.46:0/2339406159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118101900 0x7f6118101d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6108009c20 tx=0x7f610802a970 comp rx=0 tx=0).stop 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 -- 172.21.15.46:0/2339406159 shutdown_connections 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 --2- 172.21.15.46:0/2339406159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118101900 0x7f6118101d60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 --2- 172.21.15.46:0/2339406159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6118107900 0x7f6118107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 -- 172.21.15.46:0/2339406159 >> 172.21.15.46:0/2339406159 conn(0x7f61180fd600 msgr2=0x7f61180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 -- 172.21.15.46:0/2339406159 shutdown_connections 2024-09-10T04:11:35.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.180+0000 7f611edf2640 1 -- 172.21.15.46:0/2339406159 wait complete. 2024-09-10T04:11:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f611edf2640 1 Processor -- start 2024-09-10T04:11:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f611edf2640 1 -- start start 2024-09-10T04:11:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f611edf2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6118101900 0x7f611819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f611edf2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118107900 0x7f611819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f611edf2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f611810edd0 con 0x7f6118107900 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f611edf2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f611810ec50 con 0x7f6118101900 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f6117fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118107900 0x7f611819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f6117fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118107900 0x7f611819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58298/0 (socket says 172.21.15.46:58298) 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f6117fff640 1 -- 172.21.15.46:0/684704714 learned_addr learned my addr 172.21.15.46:0/684704714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f611cb8e640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6118101900 0x7f611819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f6117fff640 1 -- 172.21.15.46:0/684704714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6118101900 msgr2=0x7f611819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f6117fff640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6118101900 0x7f611819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.181+0000 7f6117fff640 1 -- 172.21.15.46:0/684704714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6118197800 con 0x7f6118107900 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.182+0000 7f6117fff640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118107900 0x7f611819dbc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6108006010 tx=0x7f610802fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.182+0000 7f6115ffb640 1 -- 172.21.15.46:0/684704714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6108047070 con 0x7f6118107900 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.182+0000 7f6115ffb640 1 -- 172.21.15.46:0/684704714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6108003e30 con 0x7f6118107900 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.182+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6118197a90 con 0x7f6118107900 2024-09-10T04:11:35.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.182+0000 7f6115ffb640 1 -- 172.21.15.46:0/684704714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6108034070 con 0x7f6118107900 2024-09-10T04:11:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.182+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6118197fc0 con 0x7f6118107900 2024-09-10T04:11:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.183+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6118103040 con 0x7f6118107900 2024-09-10T04:11:35.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.184+0000 7f6115ffb640 1 -- 172.21.15.46:0/684704714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f610804a070 con 0x7f6118107900 2024-09-10T04:11:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.184+0000 7f6115ffb640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f610407ac50 0x7f610407d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.184+0000 7f6115ffb640 1 -- 172.21.15.46:0/684704714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6070+0+0 (secure 0 0 0) 0x7f61080be800 con 0x7f6118107900 2024-09-10T04:11:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.184+0000 7f611cb8e640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f610407ac50 0x7f610407d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.185+0000 7f611cb8e640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f610407ac50 0x7f610407d110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6100005fd0 tx=0x7f610000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:35.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.189+0000 7f6115ffb640 1 -- 172.21.15.46:0/684704714 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f610803f070 con 0x7f6118107900 2024-09-10T04:11:35.279 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 podman[129174]: 2024-09-10 04:11:35.09264799 +0000 UTC m=+0.016599610 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:11:35.279 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 podman[129174]: 2024-09-10 04:11:35.217213103 +0000 UTC m=+0.141164714 container create 21eaaa32af7cb13fb24322f0ab4ad3246ee62116eb2d0acc6dc79bbedec0fce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T04:11:35.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:35 smithi046 ceph-mon[109506]: pgmap v156: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T04:11:35.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:35 smithi046 ceph-mon[109506]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:35.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:35 smithi046 ceph-mon[109506]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T04:11:35.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:35 smithi046 ceph-mon[109506]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:35.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:35 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2634717167' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.423+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6118109e70 con 0x7f6118107900 2024-09-10T04:11:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.423+0000 7f6115ffb640 1 -- 172.21.15.46:0/684704714 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7f610808a070 con 0x7f6118107900 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:11:35.426 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:11:35.427 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:35.428 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:35.429 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:11:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:11:35.429 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:11:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:11:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f610407ac50 msgr2=0x7f610407d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f610407ac50 0x7f610407d110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6100005fd0 tx=0x7f610000b390 comp rx=0 tx=0).stop 2024-09-10T04:11:35.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118107900 msgr2=0x7f611819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118107900 0x7f611819dbc0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6108006010 tx=0x7f610802fdd0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 shutdown_connections 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f610407ac50 0x7f610407d110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6118107900 0x7f611819dbc0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 --2- 172.21.15.46:0/684704714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6118101900 0x7f611819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 >> 172.21.15.46:0/684704714 conn(0x7f61180fd600 msgr2=0x7f61180fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 shutdown_connections 2024-09-10T04:11:35.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.427+0000 7f611edf2640 1 -- 172.21.15.46:0/684704714 wait complete. 2024-09-10T04:11:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.513+0000 7f4043de9640 1 -- 172.21.15.46:0/933668908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1067f0 msgr2=0x7f403c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.513+0000 7f4043de9640 1 --2- 172.21.15.46:0/933668908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1067f0 0x7f403c106bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4038009880 tx=0x7f403802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.513+0000 7f4043de9640 1 -- 172.21.15.46:0/933668908 shutdown_connections 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.513+0000 7f4043de9640 1 --2- 172.21.15.46:0/933668908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c1007f0 0x7f403c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.513+0000 7f4043de9640 1 --2- 172.21.15.46:0/933668908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1067f0 0x7f403c106bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.513+0000 7f4043de9640 1 -- 172.21.15.46:0/933668908 >> 172.21.15.46:0/933668908 conn(0x7f403c0fc4d0 msgr2=0x7f403c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.513+0000 7f4043de9640 1 -- 172.21.15.46:0/933668908 shutdown_connections 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.514+0000 7f4043de9640 1 -- 172.21.15.46:0/933668908 wait complete. 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.514+0000 7f4043de9640 1 Processor -- start 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.514+0000 7f4043de9640 1 -- start start 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.514+0000 7f4043de9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1007f0 0x7f403c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.514+0000 7f4043de9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c1067f0 0x7f403c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.514+0000 7f4043de9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f403c10dcc0 con 0x7f403c1007f0 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.514+0000 7f4043de9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f403c10db40 con 0x7f403c1067f0 2024-09-10T04:11:35.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041b85640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1007f0 0x7f403c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041b85640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1007f0 0x7f403c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58318/0 (socket says 172.21.15.46:58318) 2024-09-10T04:11:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041b85640 1 -- 172.21.15.46:0/1066808928 learned_addr learned my addr 172.21.15.46:0/1066808928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:35.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041384640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c1067f0 0x7f403c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041b85640 1 -- 172.21.15.46:0/1066808928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c1067f0 msgr2=0x7f403c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041b85640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c1067f0 0x7f403c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041b85640 1 -- 172.21.15.46:0/1066808928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403c199380 con 0x7f403c1007f0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4041b85640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1007f0 0x7f403c1987f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4038031460 tx=0x7f40380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4032ffd640 1 -- 172.21.15.46:0/1066808928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f403803f070 con 0x7f403c1007f0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403c199610 con 0x7f403c1007f0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.515+0000 7f4032ffd640 1 -- 172.21.15.46:0/1066808928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40380035f0 con 0x7f403c1007f0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.516+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403c1a1bc0 con 0x7f403c1007f0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.516+0000 7f4032ffd640 1 -- 172.21.15.46:0/1066808928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4038049060 con 0x7f403c1007f0 2024-09-10T04:11:35.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.517+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f403c101f30 con 0x7f403c1007f0 2024-09-10T04:11:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.517+0000 7f4032ffd640 1 -- 172.21.15.46:0/1066808928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4038042070 con 0x7f403c1007f0 2024-09-10T04:11:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.518+0000 7f4032ffd640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f402807abf0 0x7f402807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.518+0000 7f4032ffd640 1 -- 172.21.15.46:0/1066808928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6070+0+0 (secure 0 0 0) 0x7f40380be310 con 0x7f403c1007f0 2024-09-10T04:11:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.518+0000 7f4041384640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f402807abf0 0x7f402807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.518+0000 7f4041384640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f402807abf0 0x7f402807d0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f403c06b050 tx=0x7f402c00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:35.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.522+0000 7f4032ffd640 1 -- 172.21.15.46:0/1066808928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4038089b00 con 0x7f403c1007f0 2024-09-10T04:11:35.563 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 podman[129174]: 2024-09-10 04:11:35.478504815 +0000 UTC m=+0.402456425 container init 21eaaa32af7cb13fb24322f0ab4ad3246ee62116eb2d0acc6dc79bbedec0fce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:11:35.563 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 podman[129174]: 2024-09-10 04:11:35.483282863 +0000 UTC m=+0.407234477 container start 21eaaa32af7cb13fb24322f0ab4ad3246ee62116eb2d0acc6dc79bbedec0fce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T04:11:35.563 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 podman[129174]: 2024-09-10 04:11:35.561747229 +0000 UTC m=+0.485698861 container attach 21eaaa32af7cb13fb24322f0ab4ad3246ee62116eb2d0acc6dc79bbedec0fce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T04:11:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:35 smithi070 ceph-mon[79774]: pgmap v156: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T04:11:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:35 smithi070 ceph-mon[79774]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:35 smithi070 ceph-mon[79774]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T04:11:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:35 smithi070 ceph-mon[79774]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:35 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2634717167' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:35.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.730+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f403c062630 con 0x7f402807abf0 2024-09-10T04:11:35.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.731+0000 7f4032ffd640 1 -- 172.21.15.46:0/1066808928 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f403c062630 con 0x7f402807abf0 2024-09-10T04:11:35.733 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:11:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:11:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:11:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:11:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:11:35.733 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:11:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:11:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:11:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:11:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "8/23 daemons upgraded", 2024-09-10T04:11:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T04:11:35.734 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:11:35.734 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f402807abf0 msgr2=0x7f402807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f402807abf0 0x7f402807d0b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f403c06b050 tx=0x7f402c00b390 comp rx=0 tx=0).stop 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1007f0 msgr2=0x7f403c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1007f0 0x7f403c1987f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4038031460 tx=0x7f40380045e0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 shutdown_connections 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f402807abf0 0x7f402807d0b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f403c1067f0 0x7f403c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 --2- 172.21.15.46:0/1066808928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f403c1007f0 0x7f403c1987f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 >> 172.21.15.46:0/1066808928 conn(0x7f403c0fc4d0 msgr2=0x7f403c0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 shutdown_connections 2024-09-10T04:11:35.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.734+0000 7f4043de9640 1 -- 172.21.15.46:0/1066808928 wait complete. 2024-09-10T04:11:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 -- 172.21.15.46:0/4251209127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48103480 msgr2=0x7efe4810b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 --2- 172.21.15.46:0/4251209127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48103480 0x7efe4810b970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009880 tx=0x7efe3c02a530 comp rx=0 tx=0).stop 2024-09-10T04:11:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 -- 172.21.15.46:0/4251209127 shutdown_connections 2024-09-10T04:11:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 --2- 172.21.15.46:0/4251209127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48103480 0x7efe4810b970 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 --2- 172.21.15.46:0/4251209127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48102b60 0x7efe48102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 -- 172.21.15.46:0/4251209127 >> 172.21.15.46:0/4251209127 conn(0x7efe480fc520 msgr2=0x7efe480fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 -- 172.21.15.46:0/4251209127 shutdown_connections 2024-09-10T04:11:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.819+0000 7efe4d05f640 1 -- 172.21.15.46:0/4251209127 wait complete. 2024-09-10T04:11:35.822 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:35.822 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 bash[129174]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:35.822 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:35.823 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:35 smithi046 bash[129174]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.820+0000 7efe4d05f640 1 Processor -- start 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.820+0000 7efe4d05f640 1 -- start start 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.820+0000 7efe4d05f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48102b60 0x7efe4819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.820+0000 7efe4d05f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48103480 0x7efe4819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.820+0000 7efe4d05f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe4810edf0 con 0x7efe48102b60 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.820+0000 7efe4d05f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efe4810ec70 con 0x7efe48103480 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe46d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48102b60 0x7efe4819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe46d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48102b60 0x7efe4819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58346/0 (socket says 172.21.15.46:58346) 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe46d9d640 1 -- 172.21.15.46:0/1699134376 learned_addr learned my addr 172.21.15.46:0/1699134376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:11:35.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe4659c640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48103480 0x7efe4819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe46d9d640 1 -- 172.21.15.46:0/1699134376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48103480 msgr2=0x7efe4819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe46d9d640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48103480 0x7efe4819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe46d9d640 1 -- 172.21.15.46:0/1699134376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe481977e0 con 0x7efe48102b60 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe4659c640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48103480 0x7efe4819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe46d9d640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48102b60 0x7efe4819d660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe3800e970 tx=0x7efe3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.821+0000 7efe2ffff640 1 -- 172.21.15.46:0/1699134376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe3800cac0 con 0x7efe48102b60 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.822+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe48197ad0 con 0x7efe48102b60 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.822+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe48198030 con 0x7efe48102b60 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.822+0000 7efe2ffff640 1 -- 172.21.15.46:0/1699134376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe380054e0 con 0x7efe48102b60 2024-09-10T04:11:35.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.822+0000 7efe2ffff640 1 -- 172.21.15.46:0/1699134376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe38010430 con 0x7efe48102b60 2024-09-10T04:11:35.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.823+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe34005290 con 0x7efe48102b60 2024-09-10T04:11:35.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.823+0000 7efe2ffff640 1 -- 172.21.15.46:0/1699134376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efe380105d0 con 0x7efe48102b60 2024-09-10T04:11:35.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.824+0000 7efe2ffff640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe3007ac40 0x7efe3007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:11:35.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.824+0000 7efe2ffff640 1 -- 172.21.15.46:0/1699134376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6070+0+0 (secure 0 0 0) 0x7efe3809e1b0 con 0x7efe48102b60 2024-09-10T04:11:35.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.824+0000 7efe4659c640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe3007ac40 0x7efe3007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:11:35.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.825+0000 7efe4659c640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe3007ac40 0x7efe3007d100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efe48198b30 tx=0x7efe3c004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:11:35.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:35.828+0000 7efe2ffff640 1 -- 172.21.15.46:0/1699134376 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efe38069a20 con 0x7efe48102b60 2024-09-10T04:11:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.087+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7efe34005580 con 0x7efe48102b60 2024-09-10T04:11:36.090 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Reduced data availability: 2 pgs peering 2024-09-10T04:11:36.090 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:11:36.090 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:11:36.090 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T04:11:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: osd.2 (root=default,host=smithi046) is down 2024-09-10T04:11:36.091 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 2 pgs peering 2024-09-10T04:11:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: pg 2.c is stuck peering for 4m, current state peering, last acting [3,0] 2024-09-10T04:11:36.091 INFO:teuthology.orchestra.run.smithi046.stdout: pg 3.1e is stuck peering for 4m, current state peering, last acting [3,1] 2024-09-10T04:11:36.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.088+0000 7efe2ffff640 1 -- 172.21.15.46:0/1699134376 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+551 (secure 0 0 0) 0x7efe3806e8d0 con 0x7efe48102b60 2024-09-10T04:11:36.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe3007ac40 msgr2=0x7efe3007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:36.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe3007ac40 0x7efe3007d100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efe48198b30 tx=0x7efe3c004f30 comp rx=0 tx=0).stop 2024-09-10T04:11:36.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48102b60 msgr2=0x7efe4819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48102b60 0x7efe4819d660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe3800e970 tx=0x7efe3800ed70 comp rx=0 tx=0).stop 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 shutdown_connections 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe3007ac40 0x7efe3007d100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe48103480 0x7efe4819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 --2- 172.21.15.46:0/1699134376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe48102b60 0x7efe4819d660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.090+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 >> 172.21.15.46:0/1699134376 conn(0x7efe480fc520 msgr2=0x7efe48107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.091+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 shutdown_connections 2024-09-10T04:11:36.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:11:36.091+0000 7efe4d05f640 1 -- 172.21.15.46:0/1699134376 wait complete. 2024-09-10T04:11:36.377 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-mon[109506]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:36.378 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/684704714' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:11:36.378 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1699134376' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:11:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:36 smithi070 ceph-mon[79774]: from='client.34200 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:36 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/684704714' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:11:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:36 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1699134376' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:11:36.662 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T04:11:36.662 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 bash[129174]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T04:11:36.662 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T04:11:36.662 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 bash[129174]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 bash[129174]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 bash[129174]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 bash[129174]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 bash[129174]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate[129211]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-10T04:11:37.060 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 bash[129174]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-10T04:11:37.061 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:36 smithi046 podman[129174]: 2024-09-10 04:11:36.691811028 +0000 UTC m=+1.615762639 container died 21eaaa32af7cb13fb24322f0ab4ad3246ee62116eb2d0acc6dc79bbedec0fce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T04:11:37.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:37 smithi046 ceph-mon[109506]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:37.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:37 smithi046 ceph-mon[109506]: pgmap v157: 65 pgs: 8 active+undersized, 12 peering, 7 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 25/312 objects degraded (8.013%) 2024-09-10T04:11:37.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:37 smithi046 ceph-mon[109506]: Health check failed: Degraded data redundancy: 25/312 objects degraded (8.013%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:37 smithi070 ceph-mon[79774]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:11:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:37 smithi070 ceph-mon[79774]: pgmap v157: 65 pgs: 8 active+undersized, 12 peering, 7 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 25/312 objects degraded (8.013%) 2024-09-10T04:11:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:37 smithi070 ceph-mon[79774]: Health check failed: Degraded data redundancy: 25/312 objects degraded (8.013%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:37.653 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:37 smithi046 podman[129408]: 2024-09-10 04:11:37.382537706 +0000 UTC m=+0.684331559 container remove 21eaaa32af7cb13fb24322f0ab4ad3246ee62116eb2d0acc6dc79bbedec0fce4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T04:11:37.653 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:37 smithi046 podman[129468]: 2024-09-10 04:11:37.507190326 +0000 UTC m=+0.017243493 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:11:37.653 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:37 smithi046 podman[129468]: 2024-09-10 04:11:37.652315895 +0000 UTC m=+0.162369095 container create 66743aa5ec84046f37d3767154c5471cbd1f4f6db10244795d4e666ae86f15b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T04:11:38.283 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:37 smithi046 podman[129468]: 2024-09-10 04:11:37.904481026 +0000 UTC m=+0.414534187 container init 66743aa5ec84046f37d3767154c5471cbd1f4f6db10244795d4e666ae86f15b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T04:11:38.284 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:37 smithi046 podman[129468]: 2024-09-10 04:11:37.907975541 +0000 UTC m=+0.418028700 container start 66743aa5ec84046f37d3767154c5471cbd1f4f6db10244795d4e666ae86f15b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:11:38.284 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:37 smithi046 ceph-osd[129487]: -- 172.21.15.46:0/3859746598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556649fb61e0 con 0x556649f46000 2024-09-10T04:11:38.284 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:38 smithi046 bash[129468]: 66743aa5ec84046f37d3767154c5471cbd1f4f6db10244795d4e666ae86f15b0 2024-09-10T04:11:38.284 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:38 smithi046 systemd[1]: Started Ceph osd.2 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:11:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:39 smithi070 ceph-mon[79774]: pgmap v158: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:39.336 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:39 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[129483]: 2024-09-10T04:11:39.046+0000 7f7a41da1640 -1 Falling back to public interface 2024-09-10T04:11:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:39.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:39 smithi046 ceph-mon[109506]: pgmap v158: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:40.283 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:40 smithi046 ceph-mon[109506]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T04:11:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:40 smithi070 ceph-mon[79774]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T04:11:41.220 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:41 smithi046 ceph-mon[109506]: pgmap v159: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:41 smithi070 ceph-mon[79774]: pgmap v159: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:43.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:43 smithi046 ceph-mon[109506]: pgmap v160: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:43.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:43 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:43.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:43 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:43 smithi070 ceph-mon[79774]: pgmap v160: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:43 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:43 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:44.390 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:44 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[129483]: 2024-09-10T04:11:44.296+0000 7f7a41da1640 -1 osd.2 55 log_to_monitors true 2024-09-10T04:11:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:44 smithi070 ceph-mon[79774]: pgmap v161: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:44 smithi070 ceph-mon[79774]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T04:11:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:44 smithi046 ceph-mon[109506]: pgmap v161: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:44 smithi046 ceph-mon[109506]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T04:11:45.790 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:11:45 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[129483]: 2024-09-10T04:11:45.510+0000 7f7a35b43640 -1 osd.2 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:11:45.790 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:45 smithi046 ceph-mon[109506]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T04:11:45.790 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:45 smithi046 ceph-mon[109506]: osdmap e58: 6 total, 5 up, 6 in 2024-09-10T04:11:45.790 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:45 smithi046 ceph-mon[109506]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]} : dispatch 2024-09-10T04:11:45.790 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:45 smithi046 ceph-mon[109506]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' 2024-09-10T04:11:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:45 smithi070 ceph-mon[79774]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T04:11:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:45 smithi070 ceph-mon[79774]: osdmap e58: 6 total, 5 up, 6 in 2024-09-10T04:11:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:45 smithi070 ceph-mon[79774]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]} : dispatch 2024-09-10T04:11:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:45 smithi070 ceph-mon[79774]: from='osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402]' entity='osd.2' 2024-09-10T04:11:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:46 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 41/312 objects degraded (13.141%), 13 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:46 smithi070 ceph-mon[79774]: pgmap v163: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:46 smithi070 ceph-mon[79774]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:11:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:46 smithi070 ceph-mon[79774]: osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402] boot 2024-09-10T04:11:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:46 smithi070 ceph-mon[79774]: osdmap e59: 6 total, 6 up, 6 in 2024-09-10T04:11:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T04:11:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:46 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 41/312 objects degraded (13.141%), 13 pgs degraded (PG_DEGRADED) 2024-09-10T04:11:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:46 smithi046 ceph-mon[109506]: pgmap v163: 65 pgs: 14 active+undersized, 13 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 41/312 objects degraded (13.141%) 2024-09-10T04:11:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:46 smithi046 ceph-mon[109506]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:11:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:46 smithi046 ceph-mon[109506]: osd.2 [v2:172.21.15.46:6818/3090000402,v1:172.21.15.46:6819/3090000402] boot 2024-09-10T04:11:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:46 smithi046 ceph-mon[109506]: osdmap e59: 6 total, 6 up, 6 in 2024-09-10T04:11:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T04:11:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:11:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:11:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:48 smithi070 ceph-mon[79774]: osdmap e60: 6 total, 6 up, 6 in 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:48.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T04:11:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:48 smithi046 ceph-mon[109506]: osdmap e60: 6 total, 6 up, 6 in 2024-09-10T04:11:49.834 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:49 smithi046 ceph-mon[109506]: pgmap v166: 65 pgs: 9 active+undersized, 8 active+undersized+degraded, 48 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 5 op/s; 26/312 objects degraded (8.333%) 2024-09-10T04:11:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:49 smithi070 ceph-mon[79774]: pgmap v166: 65 pgs: 9 active+undersized, 8 active+undersized+degraded, 48 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.0 KiB/s rd, 5 op/s; 26/312 objects degraded (8.333%) 2024-09-10T04:11:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:51 smithi070 ceph-mon[79774]: pgmap v167: 65 pgs: 2 active+undersized, 1 active+undersized+degraded, 62 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 4/312 objects degraded (1.282%) 2024-09-10T04:11:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:51 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 4/312 objects degraded (1.282%), 1 pg degraded (PG_DEGRADED) 2024-09-10T04:11:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:51 smithi046 ceph-mon[109506]: pgmap v167: 65 pgs: 2 active+undersized, 1 active+undersized+degraded, 62 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 4/312 objects degraded (1.282%) 2024-09-10T04:11:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:51 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 4/312 objects degraded (1.282%), 1 pg degraded (PG_DEGRADED) 2024-09-10T04:11:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:52 smithi070 ceph-mon[79774]: pgmap v168: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-10T04:11:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:52 smithi070 ceph-mon[79774]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/312 objects degraded (1.282%), 1 pg degraded) 2024-09-10T04:11:52.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:52 smithi046 ceph-mon[109506]: pgmap v168: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-10T04:11:52.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:52 smithi046 ceph-mon[109506]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/312 objects degraded (1.282%), 1 pg degraded) 2024-09-10T04:11:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:54 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:54 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:54.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:54 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:54.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:54 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:11:55.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:55 smithi070 ceph-mon[79774]: pgmap v169: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T04:11:55.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:55 smithi046 ceph-mon[109506]: pgmap v169: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T04:11:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:57 smithi070 ceph-mon[79774]: pgmap v170: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T04:11:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:57 smithi046 ceph-mon[109506]: pgmap v170: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T04:11:59.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:59 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:59.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:11:59 smithi070 ceph-mon[79774]: pgmap v171: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-10T04:11:59.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:59 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:11:59.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:11:59 smithi046 ceph-mon[109506]: pgmap v171: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-09-10T04:12:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:01 smithi070 ceph-mon[79774]: pgmap v172: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:12:01.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:01 smithi046 ceph-mon[109506]: pgmap v172: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:12:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:03 smithi070 ceph-mon[79774]: pgmap v173: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:12:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:03 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-10T04:12:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:03 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T04:12:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:03 smithi070 ceph-mon[79774]: Upgrade: osd.3 is safe to restart 2024-09-10T04:12:03.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:03 smithi046 ceph-mon[109506]: pgmap v173: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:12:03.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:03 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-10T04:12:03.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:03 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T04:12:03.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:03 smithi046 ceph-mon[109506]: Upgrade: osd.3 is safe to restart 2024-09-10T04:12:04.975 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:04 smithi070 ceph-mon[79774]: Upgrade: Updating osd.3 2024-09-10T04:12:04.975 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:04 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:04.975 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:04 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T04:12:04.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:04 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:04.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:04 smithi070 ceph-mon[79774]: Deploying daemon osd.3 on smithi070 2024-09-10T04:12:04.976 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:04 smithi070 ceph-mon[79774]: pgmap v174: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:12:05.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:04 smithi046 ceph-mon[109506]: Upgrade: Updating osd.3 2024-09-10T04:12:05.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:04 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:05.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:04 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T04:12:05.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:04 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:05.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:04 smithi046 ceph-mon[109506]: Deploying daemon osd.3 on smithi070 2024-09-10T04:12:05.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:04 smithi046 ceph-mon[109506]: pgmap v174: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:12:06.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/2448483897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e481033e0 msgr2=0x7f3e4810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/2448483897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e481033e0 0x7f3e4810b8d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009880 tx=0x7f3e3002a530 comp rx=0 tx=0).stop 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/2448483897 shutdown_connections 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/2448483897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e481033e0 0x7f3e4810b8d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/2448483897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e48102ac0 0x7f3e48102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/2448483897 >> 172.21.15.46:0/2448483897 conn(0x7f3e480fc520 msgr2=0x7f3e480fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/2448483897 shutdown_connections 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.191+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/2448483897 wait complete. 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e4dcbf640 1 Processor -- start 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e4dcbf640 1 -- start start 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e4dcbf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e48102ac0 0x7f3e48100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e4dcbf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e481033e0 0x7f3e48100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e4dcbf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810edf0 con 0x7f3e48102ac0 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e4dcbf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4810ec70 con 0x7f3e481033e0 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e477fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e48102ac0 0x7f3e48100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e477fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e48102ac0 0x7f3e48100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50496/0 (socket says 172.21.15.46:50496) 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.192+0000 7f3e477fe640 1 -- 172.21.15.46:0/3849219868 learned_addr learned my addr 172.21.15.46:0/3849219868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e3effd640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e481033e0 0x7f3e48100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e477fe640 1 -- 172.21.15.46:0/3849219868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e481033e0 msgr2=0x7f3e48100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e477fe640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e481033e0 0x7f3e48100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e477fe640 1 -- 172.21.15.46:0/3849219868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e481011a0 con 0x7f3e48102ac0 2024-09-10T04:12:06.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e477fe640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e48102ac0 0x7f3e48100720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e970 tx=0x7f3e3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:06.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e457fa640 1 -- 172.21.15.46:0/3849219868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3e3800cac0 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e48101490 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e457fa640 1 -- 172.21.15.46:0/3849219868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e380054e0 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e457fa640 1 -- 172.21.15.46:0/3849219868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3e38010430 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.193+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481a3ea0 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.194+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4806aa90 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.195+0000 7f3e457fa640 1 -- 172.21.15.46:0/3849219868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3e38004e80 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.196+0000 7f3e457fa640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e3407abf0 0x7f3e3407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.196+0000 7f3e457fa640 1 -- 172.21.15.46:0/3849219868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6070+0+0 (secure 0 0 0) 0x7f3e3809e3b0 con 0x7f3e48102ac0 2024-09-10T04:12:06.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.196+0000 7f3e3effd640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e3407abf0 0x7f3e3407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.196+0000 7f3e3effd640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e3407abf0 0x7f3e3407d0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e481021b0 tx=0x7f3e30031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:06.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.200+0000 7f3e457fa640 1 -- 172.21.15.46:0/3849219868 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3e38069b00 con 0x7f3e48102ac0 2024-09-10T04:12:06.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.408+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3e48062630 con 0x7f3e3407abf0 2024-09-10T04:12:06.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.410+0000 7f3e457fa640 1 -- 172.21.15.46:0/3849219868 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f3e48062630 con 0x7f3e3407abf0 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e3407abf0 msgr2=0x7f3e3407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e3407abf0 0x7f3e3407d0b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e481021b0 tx=0x7f3e30031400 comp rx=0 tx=0).stop 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e48102ac0 msgr2=0x7f3e48100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e48102ac0 0x7f3e48100720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e970 tx=0x7f3e3800ed70 comp rx=0 tx=0).stop 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 shutdown_connections 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e3407abf0 0x7f3e3407d0b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e481033e0 0x7f3e48100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 --2- 172.21.15.46:0/3849219868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e48102ac0 0x7f3e48100720 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 >> 172.21.15.46:0/3849219868 conn(0x7f3e480fc520 msgr2=0x7f3e48108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 shutdown_connections 2024-09-10T04:12:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.412+0000 7f3e4dcbf640 1 -- 172.21.15.46:0/3849219868 wait complete. 2024-09-10T04:12:06.421 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 -- 172.21.15.46:0/1553668517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118102b10 msgr2=0x7f3118102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 --2- 172.21.15.46:0/1553668517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118102b10 0x7f3118102ef0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f31140098b0 tx=0x7f311402a560 comp rx=0 tx=0).stop 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 -- 172.21.15.46:0/1553668517 shutdown_connections 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 --2- 172.21.15.46:0/1553668517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3118103430 0x7f311810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 --2- 172.21.15.46:0/1553668517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118102b10 0x7f3118102ef0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 -- 172.21.15.46:0/1553668517 >> 172.21.15.46:0/1553668517 conn(0x7f31180fc550 msgr2=0x7f31180fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 -- 172.21.15.46:0/1553668517 shutdown_connections 2024-09-10T04:12:06.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 -- 172.21.15.46:0/1553668517 wait complete. 2024-09-10T04:12:06.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 Processor -- start 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.498+0000 7f31203b7640 1 -- start start 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f31203b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3118102b10 0x7f3118100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f31203b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118103430 0x7f3118100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f31203b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f311810ede0 con 0x7f3118103430 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f31203b7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f311810ec60 con 0x7f3118102b10 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311d952640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118103430 0x7f3118100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311e153640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3118102b10 0x7f3118100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311d952640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118103430 0x7f3118100cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50520/0 (socket says 172.21.15.46:50520) 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311d952640 1 -- 172.21.15.46:0/3851705086 learned_addr learned my addr 172.21.15.46:0/3851705086 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:06.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311d952640 1 -- 172.21.15.46:0/3851705086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3118102b10 msgr2=0x7f3118100770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311d952640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3118102b10 0x7f3118100770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311d952640 1 -- 172.21.15.46:0/3851705086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31181011f0 con 0x7f3118103430 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311e153640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3118102b10 0x7f3118100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.499+0000 7f311d952640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118103430 0x7f3118100cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f310800e970 tx=0x7f310800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.500+0000 7f310f7fe640 1 -- 172.21.15.46:0/3851705086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f310800cac0 con 0x7f3118103430 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.500+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31181014e0 con 0x7f3118103430 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.500+0000 7f310f7fe640 1 -- 172.21.15.46:0/3851705086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f31080054e0 con 0x7f3118103430 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.500+0000 7f310f7fe640 1 -- 172.21.15.46:0/3851705086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3108010430 con 0x7f3118103430 2024-09-10T04:12:06.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.500+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31181a3ef0 con 0x7f3118103430 2024-09-10T04:12:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.501+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f311806a520 con 0x7f3118103430 2024-09-10T04:12:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.502+0000 7f310f7fe640 1 -- 172.21.15.46:0/3851705086 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3108004e80 con 0x7f3118103430 2024-09-10T04:12:06.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.502+0000 7f310f7fe640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f310407abf0 0x7f310407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.502+0000 7f310f7fe640 1 -- 172.21.15.46:0/3851705086 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6070+0+0 (secure 0 0 0) 0x7f310809e220 con 0x7f3118103430 2024-09-10T04:12:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.502+0000 7f311e153640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f310407abf0 0x7f310407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.503+0000 7f311e153640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f310407abf0 0x7f310407d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3114000c00 tx=0x7f3114031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:06.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.507+0000 7f310f7fe640 1 -- 172.21.15.46:0/3851705086 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3108069a10 con 0x7f3118103430 2024-09-10T04:12:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.710+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3118062630 con 0x7f310407abf0 2024-09-10T04:12:06.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.711+0000 7f310f7fe640 1 -- 172.21.15.46:0/3851705086 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f3118062630 con 0x7f310407abf0 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f310407abf0 msgr2=0x7f310407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f310407abf0 0x7f310407d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3114000c00 tx=0x7f3114031400 comp rx=0 tx=0).stop 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118103430 msgr2=0x7f3118100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118103430 0x7f3118100cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f310800e970 tx=0x7f310800ed70 comp rx=0 tx=0).stop 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 shutdown_connections 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f310407abf0 0x7f310407d0b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3118103430 0x7f3118100cb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 --2- 172.21.15.46:0/3851705086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3118102b10 0x7f3118100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 >> 172.21.15.46:0/3851705086 conn(0x7f31180fc550 msgr2=0x7f3118107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 shutdown_connections 2024-09-10T04:12:06.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.713+0000 7f31203b7640 1 -- 172.21.15.46:0/3851705086 wait complete. 2024-09-10T04:12:06.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 -- 172.21.15.46:0/2141934329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 msgr2=0x7f0d50106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 --2- 172.21.15.46:0/2141934329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 0x7f0d50106bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009880 tx=0x7f0d4002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 -- 172.21.15.46:0/2141934329 shutdown_connections 2024-09-10T04:12:06.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 --2- 172.21.15.46:0/2141934329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 0x7f0d50100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 --2- 172.21.15.46:0/2141934329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 0x7f0d50106bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 -- 172.21.15.46:0/2141934329 >> 172.21.15.46:0/2141934329 conn(0x7f0d500fc550 msgr2=0x7f0d500fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:06.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 -- 172.21.15.46:0/2141934329 shutdown_connections 2024-09-10T04:12:06.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.798+0000 7f0d54891640 1 -- 172.21.15.46:0/2141934329 wait complete. 2024-09-10T04:12:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d54891640 1 Processor -- start 2024-09-10T04:12:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d54891640 1 -- start start 2024-09-10T04:12:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d54891640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 0x7f0d501988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d54891640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 0x7f0d50198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d54891640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010dca0 con 0x7f0d501067f0 2024-09-10T04:12:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d54891640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5010db20 con 0x7f0d501007f0 2024-09-10T04:12:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 0x7f0d50198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 0x7f0d501988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d4e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 0x7f0d501988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:52998/0 (socket says 172.21.15.46:52998) 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d4e59c640 1 -- 172.21.15.46:0/3115069932 learned_addr learned my addr 172.21.15.46:0/3115069932 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.799+0000 7f0d4dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 0x7f0d50198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50534/0 (socket says 172.21.15.46:50534) 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d4e59c640 1 -- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 msgr2=0x7f0d50198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d4e59c640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 0x7f0d50198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d4e59c640 1 -- 172.21.15.46:0/3115069932 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d50199410 con 0x7f0d501007f0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d4e59c640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 0x7f0d501988d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0d40031460 tx=0x7f0d400045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d3f7fe640 1 -- 172.21.15.46:0/3115069932 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d4003f070 con 0x7f0d501007f0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501996a0 con 0x7f0d501007f0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d3f7fe640 1 -- 172.21.15.46:0/3115069932 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d40004b10 con 0x7f0d501007f0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d3f7fe640 1 -- 172.21.15.46:0/3115069932 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d40035030 con 0x7f0d501007f0 2024-09-10T04:12:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.800+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a1c50 con 0x7f0d501007f0 2024-09-10T04:12:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.801+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d50101f30 con 0x7f0d501007f0 2024-09-10T04:12:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.802+0000 7f0d3f7fe640 1 -- 172.21.15.46:0/3115069932 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0d40042070 con 0x7f0d501007f0 2024-09-10T04:12:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.802+0000 7f0d3f7fe640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d3807ac00 0x7f0d3807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.803+0000 7f0d3f7fe640 1 -- 172.21.15.46:0/3115069932 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6070+0+0 (secure 0 0 0) 0x7f0d400be740 con 0x7f0d501007f0 2024-09-10T04:12:06.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.803+0000 7f0d4dd9b640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d3807ac00 0x7f0d3807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:06.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.803+0000 7f0d4dd9b640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d3807ac00 0x7f0d3807d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0d5006a7f0 tx=0x7f0d4400b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:06.812+0000 7f0d3f7fe640 1 -- 172.21.15.46:0/3115069932 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0d40089e90 con 0x7f0d501007f0 2024-09-10T04:12:06.921 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:06 smithi070 systemd[1]: Stopping Ceph osd.3 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:12:07.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.010+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0d5006a290 con 0x7f0d3807ac00 2024-09-10T04:12:07.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.020+0000 7f0d3f7fe640 1 -- 172.21.15.46:0/3115069932 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f0d5006a290 con 0x7f0d3807ac00 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (3m) 23s ago 14m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (6m) 23s ago 15m 9026k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (6m) 118s ago 14m 8539k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (2m) 23s ago 15m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (2m) 118s ago 14m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (3m) 23s ago 14m 77.6M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (9m) 23s ago 9m 90.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (9m) 23s ago 9m 145M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (9m) 118s ago 9m 26.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:12:07.021 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (9m) 118s ago 9m 25.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (5m) 23s ago 16m 528M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (4m) 118s ago 14m 455M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (2m) 23s ago 16m 43.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (2m) 118s ago 14m 25.7M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (4m) 23s ago 15m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (4m) 118s ago 14m 12.0M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (101s) 23s ago 13m 81.8M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (65s) 23s ago 12m 86.9M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (29s) 23s ago 12m 12.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (11m) 118s ago 11m 236M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 8555c17d5271 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (11m) 118s ago 11m 210M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (10m) 118s ago 10m 196M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:12:07.022 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (4m) 23s ago 14m 55.9M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:12:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d3807ac00 msgr2=0x7f0d3807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d3807ac00 0x7f0d3807d0c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0d5006a7f0 tx=0x7f0d4400b410 comp rx=0 tx=0).stop 2024-09-10T04:12:07.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 msgr2=0x7f0d501988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 0x7f0d501988d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0d40031460 tx=0x7f0d400045e0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 shutdown_connections 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d3807ac00 0x7f0d3807d0c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d501067f0 0x7f0d50198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 --2- 172.21.15.46:0/3115069932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d501007f0 0x7f0d501988d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 >> 172.21.15.46:0/3115069932 conn(0x7f0d500fc550 msgr2=0x7f0d500fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 shutdown_connections 2024-09-10T04:12:07.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.022+0000 7f0d54891640 1 -- 172.21.15.46:0/3115069932 wait complete. 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.106+0000 7ff32e0d9640 1 -- 172.21.15.46:0/914843604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 msgr2=0x7ff328106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.106+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/914843604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff328106be0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff318009880 tx=0x7ff31802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 -- 172.21.15.46:0/914843604 shutdown_connections 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/914843604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff328100800 0x7ff328100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/914843604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff328106be0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 -- 172.21.15.46:0/914843604 >> 172.21.15.46:0/914843604 conn(0x7ff3280fc520 msgr2=0x7ff3280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 -- 172.21.15.46:0/914843604 shutdown_connections 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 -- 172.21.15.46:0/914843604 wait complete. 2024-09-10T04:12:07.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 Processor -- start 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.107+0000 7ff32e0d9640 1 -- start start 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff32e0d9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff328100800 0x7ff3280fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff32e0d9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff3280fd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff32e0d9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810dcd0 con 0x7ff328106800 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff32e0d9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff32810db50 con 0x7ff328100800 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff326ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff3280fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff326ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff3280fd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50552/0 (socket says 172.21.15.46:50552) 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff326ffd640 1 -- 172.21.15.46:0/233068990 learned_addr learned my addr 172.21.15.46:0/233068990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:07.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff3277fe640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff328100800 0x7ff3280fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff326ffd640 1 -- 172.21.15.46:0/233068990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff328100800 msgr2=0x7ff3280fcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff326ffd640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff328100800 0x7ff3280fcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff326ffd640 1 -- 172.21.15.46:0/233068990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3280fd780 con 0x7ff328106800 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff326ffd640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff3280fd240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00e970 tx=0x7ff31c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.108+0000 7ff324ff9640 1 -- 172.21.15.46:0/233068990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff31c00cac0 con 0x7ff328106800 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.109+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3280fda70 con 0x7ff328106800 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.109+0000 7ff324ff9640 1 -- 172.21.15.46:0/233068990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff31c0054e0 con 0x7ff328106800 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.109+0000 7ff324ff9640 1 -- 172.21.15.46:0/233068990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff31c010430 con 0x7ff328106800 2024-09-10T04:12:07.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.109+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3281a3e40 con 0x7ff328106800 2024-09-10T04:12:07.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.110+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff328101f40 con 0x7ff328106800 2024-09-10T04:12:07.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.110+0000 7ff324ff9640 1 -- 172.21.15.46:0/233068990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff31c004ee0 con 0x7ff328106800 2024-09-10T04:12:07.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.111+0000 7ff324ff9640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff31007abf0 0x7ff31007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.111+0000 7ff324ff9640 1 -- 172.21.15.46:0/233068990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6070+0+0 (secure 0 0 0) 0x7ff31c09e280 con 0x7ff328106800 2024-09-10T04:12:07.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.111+0000 7ff3277fe640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff31007abf0 0x7ff31007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.111+0000 7ff3277fe640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff31007abf0 0x7ff31007d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3180096c0 tx=0x7ff318031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:07.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.115+0000 7ff324ff9640 1 -- 172.21.15.46:0/233068990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff31c069a70 con 0x7ff328106800 2024-09-10T04:12:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:07 smithi070 ceph-mon[79774]: pgmap v175: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:12:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:07 smithi070 ceph-mon[79774]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:07 smithi070 ceph-mon[79774]: osd.3 marked itself down and dead 2024-09-10T04:12:07.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:06 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[41459]: 2024-09-10T04:12:06.918+0000 7f5c4b5e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:12:07.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:06 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[41459]: 2024-09-10T04:12:06.918+0000 7f5c4b5e3640 -1 osd.3 60 *** Got signal Terminated *** 2024-09-10T04:12:07.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:06 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[41459]: 2024-09-10T04:12:06.918+0000 7f5c4b5e3640 -1 osd.3 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:12:07.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.382+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff328109d80 con 0x7ff328106800 2024-09-10T04:12:07.383 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:07 smithi046 ceph-mon[109506]: pgmap v175: 65 pgs: 65 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:12:07.383 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:07 smithi046 ceph-mon[109506]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:07.383 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:07 smithi046 ceph-mon[109506]: osd.3 marked itself down and dead 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.383+0000 7ff324ff9640 1 -- 172.21.15.46:0/233068990 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7ff31c06e920 con 0x7ff328106800 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:12:07.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2, 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6, 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 7 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:12:07.385 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:12:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff31007abf0 msgr2=0x7ff31007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff31007abf0 0x7ff31007d0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff3180096c0 tx=0x7ff318031400 comp rx=0 tx=0).stop 2024-09-10T04:12:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 msgr2=0x7ff3280fd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff3280fd240 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00e970 tx=0x7ff31c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:12:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 shutdown_connections 2024-09-10T04:12:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff31007abf0 0x7ff31007d0b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff328106800 0x7ff3280fd240 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 --2- 172.21.15.46:0/233068990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff328100800 0x7ff3280fcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 >> 172.21.15.46:0/233068990 conn(0x7ff3280fc520 msgr2=0x7ff32806a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:07.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 shutdown_connections 2024-09-10T04:12:07.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.385+0000 7ff32e0d9640 1 -- 172.21.15.46:0/233068990 wait complete. 2024-09-10T04:12:07.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 -- 172.21.15.46:0/3118287811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 msgr2=0x7fa210106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 --2- 172.21.15.46:0/3118287811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa210106be0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa200009880 tx=0x7fa20002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 -- 172.21.15.46:0/3118287811 shutdown_connections 2024-09-10T04:12:07.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 --2- 172.21.15.46:0/3118287811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa210100800 0x7fa210100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 --2- 172.21.15.46:0/3118287811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa210106be0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 -- 172.21.15.46:0/3118287811 >> 172.21.15.46:0/3118287811 conn(0x7fa2100fc520 msgr2=0x7fa2100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:07.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 -- 172.21.15.46:0/3118287811 shutdown_connections 2024-09-10T04:12:07.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 -- 172.21.15.46:0/3118287811 wait complete. 2024-09-10T04:12:07.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.469+0000 7fa215b55640 1 Processor -- start 2024-09-10T04:12:07.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa215b55640 1 -- start start 2024-09-10T04:12:07.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa215b55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa210100800 0x7fa2100fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa215b55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa2100fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa215b55640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa21010dcd0 con 0x7fa210106800 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa215b55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa21010db50 con 0x7fa210100800 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa20effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa2100fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa20effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa2100fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50566/0 (socket says 172.21.15.46:50566) 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa20effd640 1 -- 172.21.15.46:0/136943195 learned_addr learned my addr 172.21.15.46:0/136943195 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa20f7fe640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa210100800 0x7fa2100fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa20effd640 1 -- 172.21.15.46:0/136943195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa210100800 msgr2=0x7fa2100fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa20effd640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa210100800 0x7fa2100fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.470+0000 7fa20effd640 1 -- 172.21.15.46:0/136943195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2100fd650 con 0x7fa210106800 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.471+0000 7fa20effd640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa2100fd110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa20400e970 tx=0x7fa20400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.471+0000 7fa20cff9640 1 -- 172.21.15.46:0/136943195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa20400cac0 con 0x7fa210106800 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.471+0000 7fa20cff9640 1 -- 172.21.15.46:0/136943195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2040054e0 con 0x7fa210106800 2024-09-10T04:12:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.471+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2100fd940 con 0x7fa210106800 2024-09-10T04:12:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.471+0000 7fa20cff9640 1 -- 172.21.15.46:0/136943195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa204010430 con 0x7fa210106800 2024-09-10T04:12:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.471+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2101a3e40 con 0x7fa210106800 2024-09-10T04:12:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.472+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa210101f40 con 0x7fa210106800 2024-09-10T04:12:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.473+0000 7fa20cff9640 1 -- 172.21.15.46:0/136943195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa204004f00 con 0x7fa210106800 2024-09-10T04:12:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.473+0000 7fa20cff9640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f807ac40 0x7fa1f807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.473+0000 7fa20cff9640 1 -- 172.21.15.46:0/136943195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7fa20409e140 con 0x7fa210106800 2024-09-10T04:12:07.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.473+0000 7fa20f7fe640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f807ac40 0x7fa1f807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.474+0000 7fa20f7fe640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f807ac40 0x7fa1f807d100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2000096c0 tx=0x7fa200031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:07.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.478+0000 7fa20cff9640 1 -- 172.21.15.46:0/136943195 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa2040699b0 con 0x7fa210106800 2024-09-10T04:12:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.700+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa210109d80 con 0x7fa210106800 2024-09-10T04:12:07.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.700+0000 7fa20cff9640 1 -- 172.21.15.46:0/136943195 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7fa20406e860 con 0x7fa210106800 2024-09-10T04:12:07.701 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:12:07.701 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:12:07.701 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:12:07.701 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:12:07.702 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:12:07.703 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:12:07.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f807ac40 msgr2=0x7fa1f807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f807ac40 0x7fa1f807d100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa2000096c0 tx=0x7fa200031400 comp rx=0 tx=0).stop 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 msgr2=0x7fa2100fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa2100fd110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa20400e970 tx=0x7fa20400ed70 comp rx=0 tx=0).stop 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 shutdown_connections 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f807ac40 0x7fa1f807d100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa210106800 0x7fa2100fd110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 --2- 172.21.15.46:0/136943195 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa210100800 0x7fa2100fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.703+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 >> 172.21.15.46:0/136943195 conn(0x7fa2100fc520 msgr2=0x7fa21006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.704+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 shutdown_connections 2024-09-10T04:12:07.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.704+0000 7fa215b55640 1 -- 172.21.15.46:0/136943195 wait complete. 2024-09-10T04:12:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.789+0000 7f435d368640 1 -- 172.21.15.46:0/612639250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 msgr2=0x7f4358107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.789+0000 7f435d368640 1 --2- 172.21.15.46:0/612639250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f4358107ce0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4348009880 tx=0x7f434802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.789+0000 7f435d368640 1 -- 172.21.15.46:0/612639250 shutdown_connections 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.789+0000 7f435d368640 1 --2- 172.21.15.46:0/612639250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4358101900 0x7f4358101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.789+0000 7f435d368640 1 --2- 172.21.15.46:0/612639250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f4358107ce0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.789+0000 7f435d368640 1 -- 172.21.15.46:0/612639250 >> 172.21.15.46:0/612639250 conn(0x7f43580fd5e0 msgr2=0x7f43580ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.790+0000 7f435d368640 1 -- 172.21.15.46:0/612639250 shutdown_connections 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.790+0000 7f435d368640 1 -- 172.21.15.46:0/612639250 wait complete. 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.790+0000 7f435d368640 1 Processor -- start 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.790+0000 7f435d368640 1 -- start start 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f435d368640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4358101900 0x7f435819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f435d368640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f435819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f435d368640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f435810edb0 con 0x7f4358107900 2024-09-10T04:12:07.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f435d368640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f435810ec30 con 0x7f4358101900 2024-09-10T04:12:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f43567fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f435819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f4356ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4358101900 0x7f435819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f43567fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f435819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50578/0 (socket says 172.21.15.46:50578) 2024-09-10T04:12:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f4356ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4358101900 0x7f435819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:53060/0 (socket says 172.21.15.46:53060) 2024-09-10T04:12:07.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f43567fc640 1 -- 172.21.15.46:0/1475812650 learned_addr learned my addr 172.21.15.46:0/1475812650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f43567fc640 1 -- 172.21.15.46:0/1475812650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4358101900 msgr2=0x7f435819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f43567fc640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4358101900 0x7f435819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f43567fc640 1 -- 172.21.15.46:0/1475812650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43581977a0 con 0x7f4358107900 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f43567fc640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f435819db60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f434c00d8d0 tx=0x7f434c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f433ffff640 1 -- 172.21.15.46:0/1475812650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f434c017030 con 0x7f4358107900 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4358197a90 con 0x7f4358107900 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f433ffff640 1 -- 172.21.15.46:0/1475812650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f434c004020 con 0x7f4358107900 2024-09-10T04:12:07.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.791+0000 7f433ffff640 1 -- 172.21.15.46:0/1475812650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f434c0047a0 con 0x7f4358107900 2024-09-10T04:12:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.792+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4358198020 con 0x7f4358107900 2024-09-10T04:12:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.793+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4358103040 con 0x7f4358107900 2024-09-10T04:12:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.793+0000 7f433ffff640 1 -- 172.21.15.46:0/1475812650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f434c0041d0 con 0x7f4358107900 2024-09-10T04:12:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.794+0000 7f433ffff640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434007abf0 0x7f434007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.794+0000 7f433ffff640 1 -- 172.21.15.46:0/1475812650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7f434c018070 con 0x7f4358107900 2024-09-10T04:12:07.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.794+0000 7f4356ffd640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434007abf0 0x7f434007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:07.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.795+0000 7f4356ffd640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434007abf0 0x7f434007d0b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43480096c0 tx=0x7f4348031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:07.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:07.798+0000 7f433ffff640 1 -- 172.21.15.46:0/1475812650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f434c069110 con 0x7f4358107900 2024-09-10T04:12:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.002+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4358198330 con 0x7f434007abf0 2024-09-10T04:12:08.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.003+0000 7f433ffff640 1 -- 172.21.15.46:0/1475812650 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+400 (secure 0 0 0) 0x7f4358198330 con 0x7f434007abf0 2024-09-10T04:12:08.004 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "9/23 daemons upgraded", 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:12:08.005 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:12:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434007abf0 msgr2=0x7f434007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434007abf0 0x7f434007d0b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f43480096c0 tx=0x7f4348031400 comp rx=0 tx=0).stop 2024-09-10T04:12:08.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 msgr2=0x7f435819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f435819db60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f434c00d8d0 tx=0x7f434c00dd90 comp rx=0 tx=0).stop 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 shutdown_connections 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434007abf0 0x7f434007d0b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4358107900 0x7f435819db60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 --2- 172.21.15.46:0/1475812650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4358101900 0x7f435819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.006+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 >> 172.21.15.46:0/1475812650 conn(0x7f43580fd5e0 msgr2=0x7f43580fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.007+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 shutdown_connections 2024-09-10T04:12:08.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.007+0000 7f435d368640 1 -- 172.21.15.46:0/1475812650 wait complete. 2024-09-10T04:12:08.021 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:07 smithi070 podman[86710]: 2024-09-10 04:12:07.702043465 +0000 UTC m=+0.884770900 container died 8555c17d5271346b5c2046a97027d0b32dd4960836f21a6ed7a4f1d009fbf59a (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T04:12:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.091+0000 7fa72dd31640 1 -- 172.21.15.46:0/1524246458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728107960 msgr2=0x7fa728107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.091+0000 7fa72dd31640 1 --2- 172.21.15.46:0/1524246458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728107960 0x7fa728107d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa718009880 tx=0x7fa71802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 -- 172.21.15.46:0/1524246458 shutdown_connections 2024-09-10T04:12:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 --2- 172.21.15.46:0/1524246458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa728101960 0x7fa728101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 --2- 172.21.15.46:0/1524246458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728107960 0x7fa728107d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 -- 172.21.15.46:0/1524246458 >> 172.21.15.46:0/1524246458 conn(0x7fa7280fd680 msgr2=0x7fa7280ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:08.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 -- 172.21.15.46:0/1524246458 shutdown_connections 2024-09-10T04:12:08.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 -- 172.21.15.46:0/1524246458 wait complete. 2024-09-10T04:12:08.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 Processor -- start 2024-09-10T04:12:08.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.092+0000 7fa72dd31640 1 -- start start 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa72dd31640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728101960 0x7fa7280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa72dd31640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa728107960 0x7fa728074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa72dd31640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa72810ee90 con 0x7fa728101960 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa72dd31640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa72810ed10 con 0x7fa728107960 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa7277fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728101960 0x7fa7280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa7277fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728101960 0x7fa7280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50584/0 (socket says 172.21.15.46:50584) 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa7277fe640 1 -- 172.21.15.46:0/908930343 learned_addr learned my addr 172.21.15.46:0/908930343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa726ffd640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa728107960 0x7fa728074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa7277fe640 1 -- 172.21.15.46:0/908930343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa728107960 msgr2=0x7fa728074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa7277fe640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa728107960 0x7fa728074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa7277fe640 1 -- 172.21.15.46:0/908930343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa728074f70 con 0x7fa728101960 2024-09-10T04:12:08.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.093+0000 7fa7277fe640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728101960 0x7fa7280744f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa718031460 tx=0x7fa7180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.094+0000 7fa724ff9640 1 -- 172.21.15.46:0/908930343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa71803f070 con 0x7fa728101960 2024-09-10T04:12:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.094+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa728075200 con 0x7fa728101960 2024-09-10T04:12:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.094+0000 7fa724ff9640 1 -- 172.21.15.46:0/908930343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa718004cf0 con 0x7fa728101960 2024-09-10T04:12:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.094+0000 7fa724ff9640 1 -- 172.21.15.46:0/908930343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa718035030 con 0x7fa728101960 2024-09-10T04:12:08.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.094+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7281a3e40 con 0x7fa728101960 2024-09-10T04:12:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.095+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7281030a0 con 0x7fa728101960 2024-09-10T04:12:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.095+0000 7fa724ff9640 1 -- 172.21.15.46:0/908930343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa718042070 con 0x7fa728101960 2024-09-10T04:12:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.096+0000 7fa724ff9640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71007abf0 0x7fa71007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.096+0000 7fa724ff9640 1 -- 172.21.15.46:0/908930343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6070+0+0 (secure 0 0 0) 0x7fa7180be720 con 0x7fa728101960 2024-09-10T04:12:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.096+0000 7fa726ffd640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71007abf0 0x7fa71007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:08.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.096+0000 7fa726ffd640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71007abf0 0x7fa71007d0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa728075f80 tx=0x7fa71c009110 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.100+0000 7fa724ff9640 1 -- 172.21.15.46:0/908930343 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa718089e70 con 0x7fa728101960 2024-09-10T04:12:08.280 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:08.280 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: from='client.44213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:08.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:12:08.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: osdmap e61: 6 total, 5 up, 6 in 2024-09-10T04:12:08.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/233068990' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:08.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/136943195' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:12:08.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:08.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:08.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:08 smithi070 podman[86710]: 2024-09-10 04:12:08.119373946 +0000 UTC m=+1.302101372 container cleanup 8555c17d5271346b5c2046a97027d0b32dd4960836f21a6ed7a4f1d009fbf59a (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T04:12:08.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:08 smithi070 bash[86710]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3 2024-09-10T04:12:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: from='client.44213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:12:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: osdmap e61: 6 total, 5 up, 6 in 2024-09-10T04:12:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/233068990' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/136943195' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:12:08.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:08.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.378+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa728109ed0 con 0x7fa728101960 2024-09-10T04:12:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.379+0000 7fa724ff9640 1 -- 172.21.15.46:0/908930343 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+290 (secure 0 0 0) 0x7fa71808ed20 con 0x7fa728101960 2024-09-10T04:12:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-09-10T04:12:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:12:08.382 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:12:08.382 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T04:12:08.382 INFO:teuthology.orchestra.run.smithi046.stdout: osd.3 (root=default,host=smithi070) is down 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71007abf0 msgr2=0x7fa71007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71007abf0 0x7fa71007d0b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa728075f80 tx=0x7fa71c009110 comp rx=0 tx=0).stop 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728101960 msgr2=0x7fa7280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728101960 0x7fa7280744f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa718031460 tx=0x7fa7180048f0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 shutdown_connections 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa71007abf0 0x7fa71007d0b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa728107960 0x7fa728074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 --2- 172.21.15.46:0/908930343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa728101960 0x7fa7280744f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 >> 172.21.15.46:0/908930343 conn(0x7fa7280fd680 msgr2=0x7fa7280fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 shutdown_connections 2024-09-10T04:12:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:08.381+0000 7fa72dd31640 1 -- 172.21.15.46:0/908930343 wait complete. 2024-09-10T04:12:08.735 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:08 smithi070 podman[86735]: 2024-09-10 04:12:08.337358089 +0000 UTC m=+0.638094288 container remove 8555c17d5271346b5c2046a97027d0b32dd4960836f21a6ed7a4f1d009fbf59a (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:12:09.008 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:08 smithi070 podman[86826]: 2024-09-10 04:12:08.637178229 +0000 UTC m=+0.016016836 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:12:09.008 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:08 smithi070 podman[86826]: 2024-09-10 04:12:08.763291667 +0000 UTC m=+0.142130275 container create 01cd80ba6c88d8c33a11bcacd4dd879ab8cb96591d98bf3915ff834336417eac (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:12:09.008 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 podman[86826]: 2024-09-10 04:12:09.007537654 +0000 UTC m=+0.386376264 container init 01cd80ba6c88d8c33a11bcacd4dd879ab8cb96591d98bf3915ff834336417eac (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T04:12:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:09 smithi070 ceph-mon[79774]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:09 smithi070 ceph-mon[79774]: pgmap v177: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:12:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:09 smithi070 ceph-mon[79774]: osdmap e62: 6 total, 5 up, 6 in 2024-09-10T04:12:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:09 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/908930343' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:12:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:09 smithi070 ceph-mon[79774]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-09-10T04:12:09.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 podman[86826]: 2024-09-10 04:12:09.011789027 +0000 UTC m=+0.390627639 container start 01cd80ba6c88d8c33a11bcacd4dd879ab8cb96591d98bf3915ff834336417eac (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T04:12:09.336 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 podman[86826]: 2024-09-10 04:12:09.090654901 +0000 UTC m=+0.469493527 container attach 01cd80ba6c88d8c33a11bcacd4dd879ab8cb96591d98bf3915ff834336417eac (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T04:12:09.336 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 podman[86826]: 2024-09-10 04:12:09.198237972 +0000 UTC m=+0.577076585 container died 01cd80ba6c88d8c33a11bcacd4dd879ab8cb96591d98bf3915ff834336417eac (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T04:12:09.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:09 smithi046 ceph-mon[109506]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:09.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:09 smithi046 ceph-mon[109506]: pgmap v177: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T04:12:09.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:09 smithi046 ceph-mon[109506]: osdmap e62: 6 total, 5 up, 6 in 2024-09-10T04:12:09.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:09 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/908930343' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:12:09.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:09 smithi046 ceph-mon[109506]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-09-10T04:12:10.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 podman[86849]: 2024-09-10 04:12:09.778600627 +0000 UTC m=+0.574440369 container remove 01cd80ba6c88d8c33a11bcacd4dd879ab8cb96591d98bf3915ff834336417eac (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:12:10.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-10T04:12:10.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 systemd[1]: Stopped Ceph osd.3 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:12:10.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:09 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.3.service: Consumed 1min 1.925s CPU time. 2024-09-10T04:12:10.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 systemd[1]: Starting Ceph osd.3 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:12:10.485 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:10 smithi070 ceph-mon[79774]: Health check failed: Degraded data redundancy: 14/312 objects degraded (4.487%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:10.485 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 podman[86949]: 2024-09-10 04:12:10.126734828 +0000 UTC m=+0.015996428 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:12:10.485 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 podman[86949]: 2024-09-10 04:12:10.256402632 +0000 UTC m=+0.145664277 container create af28f6b5d435918b8c41a4cf7316aca194f8700f08f9bb938838952746c10d35 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T04:12:10.485 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 podman[86949]: 2024-09-10 04:12:10.484731608 +0000 UTC m=+0.373993206 container init af28f6b5d435918b8c41a4cf7316aca194f8700f08f9bb938838952746c10d35 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:12:10.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:10 smithi046 ceph-mon[109506]: Health check failed: Degraded data redundancy: 14/312 objects degraded (4.487%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:10.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 podman[86949]: 2024-09-10 04:12:10.488167869 +0000 UTC m=+0.377429469 container start af28f6b5d435918b8c41a4cf7316aca194f8700f08f9bb938838952746c10d35 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:12:10.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 podman[86949]: 2024-09-10 04:12:10.577910373 +0000 UTC m=+0.467171979 container attach af28f6b5d435918b8c41a4cf7316aca194f8700f08f9bb938838952746c10d35 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T04:12:10.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:10.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 bash[86949]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:10.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:10.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:10 smithi070 bash[86949]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-mon[79774]: pgmap v179: 65 pgs: 6 active+undersized, 18 peering, 2 stale+active+clean, 5 active+undersized+degraded, 34 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/312 objects degraded (4.487%) 2024-09-10T04:12:11.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T04:12:11.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 bash[86949]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T04:12:11.586 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T04:12:11.586 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 bash[86949]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T04:12:11.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:11 smithi046 ceph-mon[109506]: pgmap v179: 65 pgs: 6 active+undersized, 18 peering, 2 stale+active+clean, 5 active+undersized+degraded, 34 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/312 objects degraded (4.487%) 2024-09-10T04:12:11.949 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 bash[86949]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 bash[86949]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 bash[86949]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 bash[86949]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate[86964]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 bash[86949]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-10T04:12:11.950 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:11 smithi070 podman[86949]: 2024-09-10 04:12:11.669734116 +0000 UTC m=+1.558995725 container died af28f6b5d435918b8c41a4cf7316aca194f8700f08f9bb938838952746c10d35 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2) 2024-09-10T04:12:12.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 podman[87106]: 2024-09-10 04:12:12.3103983 +0000 UTC m=+0.634682033 container remove af28f6b5d435918b8c41a4cf7316aca194f8700f08f9bb938838952746c10d35 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T04:12:12.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 podman[87165]: 2024-09-10 04:12:12.418461477 +0000 UTC m=+0.016158861 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:12:12.839 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 podman[87165]: 2024-09-10 04:12:12.586494321 +0000 UTC m=+0.184191751 container create 0313330c72b68a051f416fc8f5252c102f5735c4f01895f720730cc11b9044cb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T04:12:12.840 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 podman[87165]: 2024-09-10 04:12:12.814331158 +0000 UTC m=+0.412028555 container init 0313330c72b68a051f416fc8f5252c102f5735c4f01895f720730cc11b9044cb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T04:12:12.840 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 podman[87165]: 2024-09-10 04:12:12.817557704 +0000 UTC m=+0.415255090 container start 0313330c72b68a051f416fc8f5252c102f5735c4f01895f720730cc11b9044cb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T04:12:13.212 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 ceph-osd[87185]: -- 172.21.15.70:0/3771238037 <== mon.1 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b1a2e21e0 con 0x556b19481c00 2024-09-10T04:12:13.212 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 bash[87165]: 0313330c72b68a051f416fc8f5252c102f5735c4f01895f720730cc11b9044cb 2024-09-10T04:12:13.212 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:12 smithi070 systemd[1]: Started Ceph osd.3 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:12:13.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:13 smithi046 ceph-mon[109506]: pgmap v180: 65 pgs: 9 active+undersized, 18 peering, 11 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 32/312 objects degraded (10.256%) 2024-09-10T04:12:13.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:13 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:13.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:13 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:13.483 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:13 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:13.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:13 smithi070 ceph-mon[79774]: pgmap v180: 65 pgs: 9 active+undersized, 18 peering, 11 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 32/312 objects degraded (10.256%) 2024-09-10T04:12:13.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:13 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:13.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:13 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:13.495 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:13 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:14.462 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:14 smithi070 ceph-mon[79774]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-09-10T04:12:14.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:14 smithi046 ceph-mon[109506]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-09-10T04:12:14.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:14 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:12:14.460+0000 7fc581f6f640 -1 Falling back to public interface 2024-09-10T04:12:15.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:15 smithi070 ceph-mon[79774]: pgmap v181: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 66/312 objects degraded (21.154%) 2024-09-10T04:12:15.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:15 smithi046 ceph-mon[109506]: pgmap v181: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 66/312 objects degraded (21.154%) 2024-09-10T04:12:16.832 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:16.832 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:16.832 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:16 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 66/312 objects degraded (21.154%), 19 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:16.832 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:16 smithi070 ceph-mon[79774]: pgmap v182: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 66/312 objects degraded (21.154%) 2024-09-10T04:12:16.832 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:16.832 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:16 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 66/312 objects degraded (21.154%), 19 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:16 smithi046 ceph-mon[109506]: pgmap v182: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 66/312 objects degraded (21.154%) 2024-09-10T04:12:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:19.406 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:19 smithi046 ceph-mon[109506]: pgmap v183: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 2 op/s; 66/312 objects degraded (21.154%) 2024-09-10T04:12:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:19 smithi070 ceph-mon[79774]: pgmap v183: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 818 B/s rd, 2 op/s; 66/312 objects degraded (21.154%) 2024-09-10T04:12:20.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:12:20.159+0000 7fc581f6f640 -1 osd.3 60 log_to_monitors true 2024-09-10T04:12:20.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:12:20.334+0000 7fc579d19640 -1 osd.3 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:12:20.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:20.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:20.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='osd.3 [v2:172.21.15.70:6800/2001163751,v1:172.21.15.70:6801/2001163751]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T04:12:20.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:20 smithi070 ceph-mon[79774]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-10T04:12:20.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T04:12:20.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-10T04:12:20.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='osd.3 [v2:172.21.15.70:6800/2001163751,v1:172.21.15.70:6801/2001163751]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T04:12:20.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:20 smithi046 ceph-mon[109506]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T04:12:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:21 smithi070 ceph-mon[79774]: pgmap v184: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 685 B/s rd, 2 op/s; 66/312 objects degraded (21.154%) 2024-09-10T04:12:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:21 smithi070 ceph-mon[79774]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T04:12:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:21 smithi070 ceph-mon[79774]: from='osd.3 [v2:172.21.15.70:6800/2001163751,v1:172.21.15.70:6801/2001163751]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:21 smithi070 ceph-mon[79774]: osdmap e63: 6 total, 5 up, 6 in 2024-09-10T04:12:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:21 smithi070 ceph-mon[79774]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:21.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:21 smithi046 ceph-mon[109506]: pgmap v184: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 685 B/s rd, 2 op/s; 66/312 objects degraded (21.154%) 2024-09-10T04:12:21.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:21 smithi046 ceph-mon[109506]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T04:12:21.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:21 smithi046 ceph-mon[109506]: from='osd.3 [v2:172.21.15.70:6800/2001163751,v1:172.21.15.70:6801/2001163751]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:21.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:21 smithi046 ceph-mon[109506]: osdmap e63: 6 total, 5 up, 6 in 2024-09-10T04:12:21.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:21 smithi046 ceph-mon[109506]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:22.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:22 smithi070 ceph-mon[79774]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:12:22.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:22 smithi070 ceph-mon[79774]: osd.3 [v2:172.21.15.70:6800/2001163751,v1:172.21.15.70:6801/2001163751] boot 2024-09-10T04:12:22.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:22 smithi070 ceph-mon[79774]: osdmap e64: 6 total, 6 up, 6 in 2024-09-10T04:12:22.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:22 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T04:12:22.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:22 smithi046 ceph-mon[109506]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:12:22.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:22 smithi046 ceph-mon[109506]: osd.3 [v2:172.21.15.70:6800/2001163751,v1:172.21.15.70:6801/2001163751] boot 2024-09-10T04:12:22.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:22 smithi046 ceph-mon[109506]: osdmap e64: 6 total, 6 up, 6 in 2024-09-10T04:12:22.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:22 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T04:12:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:23 smithi070 ceph-mon[79774]: pgmap v187: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 66/312 objects degraded (21.154%) 2024-09-10T04:12:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:23 smithi070 ceph-mon[79774]: osdmap e65: 6 total, 6 up, 6 in 2024-09-10T04:12:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:23 smithi046 ceph-mon[109506]: pgmap v187: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 66/312 objects degraded (21.154%) 2024-09-10T04:12:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:23 smithi046 ceph-mon[109506]: osdmap e65: 6 total, 6 up, 6 in 2024-09-10T04:12:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:24 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 57/312 objects degraded (18.269%), 17 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:24.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:24 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 57/312 objects degraded (18.269%), 17 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:25 smithi070 ceph-mon[79774]: pgmap v189: 65 pgs: 13 active+undersized, 17 active+undersized+degraded, 35 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 57/312 objects degraded (18.269%) 2024-09-10T04:12:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:25 smithi046 ceph-mon[109506]: pgmap v189: 65 pgs: 13 active+undersized, 17 active+undersized+degraded, 35 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 57/312 objects degraded (18.269%) 2024-09-10T04:12:26.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:26 smithi046 ceph-mon[109506]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 57/312 objects degraded (18.269%), 17 pgs degraded) 2024-09-10T04:12:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:26 smithi070 ceph-mon[79774]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 57/312 objects degraded (18.269%), 17 pgs degraded) 2024-09-10T04:12:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:27 smithi070 ceph-mon[79774]: pgmap v190: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-09-10T04:12:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:27 smithi046 ceph-mon[109506]: pgmap v190: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 4 op/s 2024-09-10T04:12:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:29 smithi046 ceph-mon[109506]: pgmap v191: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:12:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:29 smithi070 ceph-mon[79774]: pgmap v191: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:12:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:31 smithi070 ceph-mon[79774]: pgmap v192: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T04:12:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:31 smithi046 ceph-mon[109506]: pgmap v192: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T04:12:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:33 smithi070 ceph-mon[79774]: pgmap v193: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:12:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:33 smithi046 ceph-mon[109506]: pgmap v193: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:12:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:34.770 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:34 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-10T04:12:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:34 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-10T04:12:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:35 smithi046 ceph-mon[109506]: pgmap v194: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T04:12:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:35 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T04:12:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:35 smithi046 ceph-mon[109506]: Upgrade: osd.4 is safe to restart 2024-09-10T04:12:35.785 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:35 smithi070 ceph-mon[79774]: pgmap v194: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T04:12:35.785 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:35 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T04:12:35.785 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:35 smithi070 ceph-mon[79774]: Upgrade: osd.4 is safe to restart 2024-09-10T04:12:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:36 smithi070 ceph-mon[79774]: Upgrade: Updating osd.4 2024-09-10T04:12:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T04:12:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:36 smithi070 ceph-mon[79774]: Deploying daemon osd.4 on smithi070 2024-09-10T04:12:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:36 smithi070 ceph-mon[79774]: pgmap v195: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T04:12:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:36 smithi046 ceph-mon[109506]: Upgrade: Updating osd.4 2024-09-10T04:12:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T04:12:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:37.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:36 smithi046 ceph-mon[109506]: Deploying daemon osd.4 on smithi070 2024-09-10T04:12:37.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:36 smithi046 ceph-mon[109506]: pgmap v195: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T04:12:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 -- 172.21.15.46:0/1077303879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34106800 msgr2=0x7f4e34106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 --2- 172.21.15.46:0/1077303879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34106800 0x7f4e34106be0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e24009880 tx=0x7f4e2402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 -- 172.21.15.46:0/1077303879 shutdown_connections 2024-09-10T04:12:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 --2- 172.21.15.46:0/1077303879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e34100800 0x7f4e34100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 --2- 172.21.15.46:0/1077303879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34106800 0x7f4e34106be0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 -- 172.21.15.46:0/1077303879 >> 172.21.15.46:0/1077303879 conn(0x7f4e340fc520 msgr2=0x7f4e340fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 -- 172.21.15.46:0/1077303879 shutdown_connections 2024-09-10T04:12:38.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.476+0000 7f4e396a3640 1 -- 172.21.15.46:0/1077303879 wait complete. 2024-09-10T04:12:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.477+0000 7f4e396a3640 1 Processor -- start 2024-09-10T04:12:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.477+0000 7f4e396a3640 1 -- start start 2024-09-10T04:12:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e396a3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34100800 0x7f4e3419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:38.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e396a3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e34106800 0x7f4e3419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e396a3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3410dd30 con 0x7f4e34100800 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e396a3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3410dbb0 con 0x7f4e34106800 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e32ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34100800 0x7f4e3419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e32ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34100800 0x7f4e3419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39468/0 (socket says 172.21.15.46:39468) 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e32ffd640 1 -- 172.21.15.46:0/653944928 learned_addr learned my addr 172.21.15.46:0/653944928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e327fc640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e34106800 0x7f4e3419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e32ffd640 1 -- 172.21.15.46:0/653944928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e34106800 msgr2=0x7f4e3419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e32ffd640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e34106800 0x7f4e3419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e32ffd640 1 -- 172.21.15.46:0/653944928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e34197820 con 0x7f4e34100800 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e327fc640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e34106800 0x7f4e3419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.478+0000 7f4e32ffd640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34100800 0x7f4e3419d6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e24031460 tx=0x7f4e240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.479+0000 7f4e2bfff640 1 -- 172.21.15.46:0/653944928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e2403f070 con 0x7f4e34100800 2024-09-10T04:12:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.479+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e34197ab0 con 0x7f4e34100800 2024-09-10T04:12:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.479+0000 7f4e2bfff640 1 -- 172.21.15.46:0/653944928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e240044d0 con 0x7f4e34100800 2024-09-10T04:12:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.479+0000 7f4e2bfff640 1 -- 172.21.15.46:0/653944928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e24035030 con 0x7f4e34100800 2024-09-10T04:12:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.479+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e34197fe0 con 0x7f4e34100800 2024-09-10T04:12:38.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.480+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e34101f40 con 0x7f4e34100800 2024-09-10T04:12:38.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.480+0000 7f4e2bfff640 1 -- 172.21.15.46:0/653944928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4e24042070 con 0x7f4e34100800 2024-09-10T04:12:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.481+0000 7f4e2bfff640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2007ac00 0x7f4e2007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.481+0000 7f4e2bfff640 1 -- 172.21.15.46:0/653944928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6070+0+0 (secure 0 0 0) 0x7f4e240be7e0 con 0x7f4e34100800 2024-09-10T04:12:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.481+0000 7f4e327fc640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2007ac00 0x7f4e2007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.482+0000 7f4e327fc640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2007ac00 0x7f4e2007d0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e34104f40 tx=0x7f4e1c008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:38.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.486+0000 7f4e2bfff640 1 -- 172.21.15.46:0/653944928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4e24004cf0 con 0x7f4e34100800 2024-09-10T04:12:38.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.691+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4e34198350 con 0x7f4e2007ac00 2024-09-10T04:12:38.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.692+0000 7f4e2bfff640 1 -- 172.21.15.46:0/653944928 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f4e34198350 con 0x7f4e2007ac00 2024-09-10T04:12:38.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2007ac00 msgr2=0x7f4e2007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2007ac00 0x7f4e2007d0c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e34104f40 tx=0x7f4e1c008040 comp rx=0 tx=0).stop 2024-09-10T04:12:38.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34100800 msgr2=0x7f4e3419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34100800 0x7f4e3419d6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4e24031460 tx=0x7f4e240048f0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 shutdown_connections 2024-09-10T04:12:38.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2007ac00 0x7f4e2007d0c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e34106800 0x7f4e3419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 --2- 172.21.15.46:0/653944928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34100800 0x7f4e3419d6a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 >> 172.21.15.46:0/653944928 conn(0x7f4e340fc520 msgr2=0x7f4e3406a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:38.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 shutdown_connections 2024-09-10T04:12:38.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.694+0000 7f4e396a3640 1 -- 172.21.15.46:0/653944928 wait complete. 2024-09-10T04:12:38.705 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:12:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.779+0000 7f4e411a6640 1 -- 172.21.15.46:0/3408952606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 msgr2=0x7f4e3c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.779+0000 7f4e411a6640 1 --2- 172.21.15.46:0/3408952606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c100c50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009880 tx=0x7f4e3002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.779+0000 7f4e411a6640 1 -- 172.21.15.46:0/3408952606 shutdown_connections 2024-09-10T04:12:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.779+0000 7f4e411a6640 1 --2- 172.21.15.46:0/3408952606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c100c50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.779+0000 7f4e411a6640 1 --2- 172.21.15.46:0/3408952606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e3c1067f0 0x7f4e3c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.779+0000 7f4e411a6640 1 -- 172.21.15.46:0/3408952606 >> 172.21.15.46:0/3408952606 conn(0x7f4e3c0fc550 msgr2=0x7f4e3c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.780+0000 7f4e411a6640 1 -- 172.21.15.46:0/3408952606 shutdown_connections 2024-09-10T04:12:38.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.780+0000 7f4e411a6640 1 -- 172.21.15.46:0/3408952606 wait complete. 2024-09-10T04:12:38.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.780+0000 7f4e411a6640 1 Processor -- start 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e411a6640 1 -- start start 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e411a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e411a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e3c1067f0 0x7f4e3c196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e411a6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c10dcc0 con 0x7f4e3c1007f0 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e411a6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c10db40 con 0x7f4e3c1067f0 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c1965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39494/0 (socket says 172.21.15.46:39494) 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3ad9d640 1 -- 172.21.15.46:0/2441612240 learned_addr learned my addr 172.21.15.46:0/2441612240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3a59c640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e3c1067f0 0x7f4e3c196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3ad9d640 1 -- 172.21.15.46:0/2441612240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e3c1067f0 msgr2=0x7f4e3c196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3ad9d640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e3c1067f0 0x7f4e3c196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3ad9d640 1 -- 172.21.15.46:0/2441612240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c197060 con 0x7f4e3c1007f0 2024-09-10T04:12:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.781+0000 7f4e3ad9d640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c1965e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00d6f0 tx=0x7f4e2c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.782+0000 7f4e23fff640 1 -- 172.21.15.46:0/2441612240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e2c017070 con 0x7f4e3c1007f0 2024-09-10T04:12:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.782+0000 7f4e23fff640 1 -- 172.21.15.46:0/2441612240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e2c00de00 con 0x7f4e3c1007f0 2024-09-10T04:12:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.782+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c197350 con 0x7f4e3c1007f0 2024-09-10T04:12:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.782+0000 7f4e23fff640 1 -- 172.21.15.46:0/2441612240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4e2c004bb0 con 0x7f4e3c1007f0 2024-09-10T04:12:38.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.782+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c1a3e20 con 0x7f4e3c1007f0 2024-09-10T04:12:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.783+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c101f30 con 0x7f4e3c1007f0 2024-09-10T04:12:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.783+0000 7f4e23fff640 1 -- 172.21.15.46:0/2441612240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4e2c010460 con 0x7f4e3c1007f0 2024-09-10T04:12:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.784+0000 7f4e23fff640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2407abf0 0x7f4e2407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.784+0000 7f4e23fff640 1 -- 172.21.15.46:0/2441612240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6070+0+0 (secure 0 0 0) 0x7f4e2c09d760 con 0x7f4e3c1007f0 2024-09-10T04:12:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.784+0000 7f4e3a59c640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2407abf0 0x7f4e2407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:38.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.784+0000 7f4e3a59c640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2407abf0 0x7f4e2407d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c198740 tx=0x7f4e30031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:38.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.788+0000 7f4e23fff640 1 -- 172.21.15.46:0/2441612240 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4e2c068f50 con 0x7f4e3c1007f0 2024-09-10T04:12:38.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.991+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4e3c062630 con 0x7f4e2407abf0 2024-09-10T04:12:38.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.992+0000 7f4e23fff640 1 -- 172.21.15.46:0/2441612240 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f4e3c062630 con 0x7f4e2407abf0 2024-09-10T04:12:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2407abf0 msgr2=0x7f4e2407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2407abf0 0x7f4e2407d0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c198740 tx=0x7f4e30031400 comp rx=0 tx=0).stop 2024-09-10T04:12:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 msgr2=0x7f4e3c1965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c1965e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c00d6f0 tx=0x7f4e2c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 shutdown_connections 2024-09-10T04:12:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4e2407abf0 0x7f4e2407d0b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e3c1067f0 0x7f4e3c196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 --2- 172.21.15.46:0/2441612240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c1007f0 0x7f4e3c1965e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 >> 172.21.15.46:0/2441612240 conn(0x7f4e3c0fc550 msgr2=0x7f4e3c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 shutdown_connections 2024-09-10T04:12:38.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:38.994+0000 7f4e411a6640 1 -- 172.21.15.46:0/2441612240 wait complete. 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 -- 172.21.15.46:0/1245037242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601067f0 msgr2=0x7f2160106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 --2- 172.21.15.46:0/1245037242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601067f0 0x7f2160106bd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2150009880 tx=0x7f215002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 -- 172.21.15.46:0/1245037242 shutdown_connections 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 --2- 172.21.15.46:0/1245037242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21601007f0 0x7f2160100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 --2- 172.21.15.46:0/1245037242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601067f0 0x7f2160106bd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 -- 172.21.15.46:0/1245037242 >> 172.21.15.46:0/1245037242 conn(0x7f21600fc550 msgr2=0x7f21600fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 -- 172.21.15.46:0/1245037242 shutdown_connections 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 -- 172.21.15.46:0/1245037242 wait complete. 2024-09-10T04:12:39.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 Processor -- start 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.080+0000 7f2165ffb640 1 -- start start 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f2165ffb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601007f0 0x7f2160107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f2165ffb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21601067f0 0x7f2160107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f2165ffb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f216010dca0 con 0x7f21601007f0 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f2165ffb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f216010db20 con 0x7f21601067f0 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601007f0 0x7f2160107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601007f0 0x7f2160107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39530/0 (socket says 172.21.15.46:39530) 2024-09-10T04:12:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215f7fe640 1 -- 172.21.15.46:0/3930265173 learned_addr learned my addr 172.21.15.46:0/3930265173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215effd640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21601067f0 0x7f2160107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215f7fe640 1 -- 172.21.15.46:0/3930265173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21601067f0 msgr2=0x7f2160107c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215f7fe640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21601067f0 0x7f2160107c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215f7fe640 1 -- 172.21.15.46:0/3930265173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21601081a0 con 0x7f21601007f0 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.081+0000 7f215f7fe640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601007f0 0x7f2160107720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2150031460 tx=0x7f2150004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.082+0000 7f215cff9640 1 -- 172.21.15.46:0/3930265173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f215003f070 con 0x7f21601007f0 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.082+0000 7f215cff9640 1 -- 172.21.15.46:0/3930265173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f21500035f0 con 0x7f21601007f0 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.082+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2160108430 con 0x7f21601007f0 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.082+0000 7f215cff9640 1 -- 172.21.15.46:0/3930265173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2150035030 con 0x7f21601007f0 2024-09-10T04:12:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.082+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21601a3d10 con 0x7f21601007f0 2024-09-10T04:12:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.083+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2160101f30 con 0x7f21601007f0 2024-09-10T04:12:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.083+0000 7f215cff9640 1 -- 172.21.15.46:0/3930265173 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2150042070 con 0x7f21601007f0 2024-09-10T04:12:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.084+0000 7f215cff9640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f214807abf0 0x7f214807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.084+0000 7f215cff9640 1 -- 172.21.15.46:0/3930265173 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6070+0+0 (secure 0 0 0) 0x7f21500be5a0 con 0x7f21601007f0 2024-09-10T04:12:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.084+0000 7f215effd640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f214807abf0 0x7f214807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.085+0000 7f215effd640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f214807abf0 0x7f214807d0b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21601091b0 tx=0x7f2154005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.088+0000 7f215cff9640 1 -- 172.21.15.46:0/3930265173 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2150089cf0 con 0x7f21601007f0 2024-09-10T04:12:39.158 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:39.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:39.159 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:39 smithi070 ceph-mon[79774]: pgmap v196: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:12:39.159 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:38 smithi070 systemd[1]: Stopping Ceph osd.4 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:12:39.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.285+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f216006a520 con 0x7f214807abf0 2024-09-10T04:12:39.287 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:39.287 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:39.287 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:39 smithi046 ceph-mon[109506]: pgmap v196: 65 pgs: 65 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:12:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.294+0000 7f215cff9640 1 -- 172.21.15.46:0/3930265173 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f216006a520 con 0x7f214807abf0 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (4m) 56s ago 15m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (6m) 56s ago 15m 9026k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (6m) 23s ago 14m 8938k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (2m) 56s ago 15m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (2m) 23s ago 14m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (3m) 56s ago 15m 77.6M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (10m) 56s ago 10m 90.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (10m) 56s ago 10m 145M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (10m) 23s ago 10m 25.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:12:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (10m) 23s ago 10m 25.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (6m) 56s ago 16m 528M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (5m) 23s ago 14m 456M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (3m) 56s ago 17m 43.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (2m) 23s ago 14m 36.6M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (5m) 56s ago 15m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (4m) 23s ago 14m 10.9M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (2m) 56s ago 13m 81.8M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (97s) 56s ago 13m 86.9M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (61s) 56s ago 12m 12.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:12:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (26s) 23s ago 12m 11.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:12:39.299 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (11m) 23s ago 11m 210M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 e4a511ebee84 2024-09-10T04:12:39.299 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (11m) 23s ago 11m 196M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:12:39.299 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (4m) 56s ago 15m 55.9M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:12:39.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.296+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f214807abf0 msgr2=0x7f214807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.296+0000 7f2165ffb640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f214807abf0 0x7f214807d0b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f21601091b0 tx=0x7f2154005f90 comp rx=0 tx=0).stop 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.296+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601007f0 msgr2=0x7f2160107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601007f0 0x7f2160107720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2150031460 tx=0x7f2150004530 comp rx=0 tx=0).stop 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 shutdown_connections 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f214807abf0 0x7f214807d0b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f21601067f0 0x7f2160107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 --2- 172.21.15.46:0/3930265173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21601007f0 0x7f2160107720 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 >> 172.21.15.46:0/3930265173 conn(0x7f21600fc550 msgr2=0x7f2160069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 shutdown_connections 2024-09-10T04:12:39.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.297+0000 7f2165ffb640 1 -- 172.21.15.46:0/3930265173 wait complete. 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.379+0000 7f7e4f659640 1 -- 172.21.15.46:0/2904132260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 msgr2=0x7f7e48107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.379+0000 7f7e4f659640 1 --2- 172.21.15.46:0/2904132260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48107ce0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e44009880 tx=0x7f7e4402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 -- 172.21.15.46:0/2904132260 shutdown_connections 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 --2- 172.21.15.46:0/2904132260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48101900 0x7f7e48101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 --2- 172.21.15.46:0/2904132260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48107ce0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 -- 172.21.15.46:0/2904132260 >> 172.21.15.46:0/2904132260 conn(0x7f7e480fd600 msgr2=0x7f7e480ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 -- 172.21.15.46:0/2904132260 shutdown_connections 2024-09-10T04:12:39.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 -- 172.21.15.46:0/2904132260 wait complete. 2024-09-10T04:12:39.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 Processor -- start 2024-09-10T04:12:39.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.380+0000 7f7e4f659640 1 -- start start 2024-09-10T04:12:39.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4f659640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48101900 0x7f7e480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4f659640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4f659640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4810edb0 con 0x7f7e48107900 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4f659640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4810ec30 con 0x7f7e48101900 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4cbf4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4cbf4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39546/0 (socket says 172.21.15.46:39546) 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4cbf4640 1 -- 172.21.15.46:0/984056017 learned_addr learned my addr 172.21.15.46:0/984056017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4d3f5640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48101900 0x7f7e480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4cbf4640 1 -- 172.21.15.46:0/984056017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48101900 msgr2=0x7f7e480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4cbf4640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48101900 0x7f7e480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4cbf4640 1 -- 172.21.15.46:0/984056017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e48074f70 con 0x7f7e48107900 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4cbf4640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48074a30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d8d0 tx=0x7f7e3800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e3e7fc640 1 -- 172.21.15.46:0/984056017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e38017070 con 0x7f7e48107900 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e3e7fc640 1 -- 172.21.15.46:0/984056017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e38010460 con 0x7f7e48107900 2024-09-10T04:12:39.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.381+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e48075260 con 0x7f7e48107900 2024-09-10T04:12:39.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.382+0000 7f7e3e7fc640 1 -- 172.21.15.46:0/984056017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e3800f5f0 con 0x7f7e48107900 2024-09-10T04:12:39.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.382+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481a3e40 con 0x7f7e48107900 2024-09-10T04:12:39.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.383+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e48103040 con 0x7f7e48107900 2024-09-10T04:12:39.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.383+0000 7f7e3e7fc640 1 -- 172.21.15.46:0/984056017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7e38009a30 con 0x7f7e48107900 2024-09-10T04:12:39.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.384+0000 7f7e3e7fc640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e3407ac40 0x7f7e3407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.384+0000 7f7e3e7fc640 1 -- 172.21.15.46:0/984056017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6070+0+0 (secure 0 0 0) 0x7f7e3809d910 con 0x7f7e48107900 2024-09-10T04:12:39.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.384+0000 7f7e4d3f5640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e3407ac40 0x7f7e3407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.385+0000 7f7e4d3f5640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e3407ac40 0x7f7e3407d100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000c00 tx=0x7f7e44031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.389+0000 7f7e3e7fc640 1 -- 172.21.15.46:0/984056017 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7e38069180 con 0x7f7e48107900 2024-09-10T04:12:39.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:39 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[48052]: 2024-09-10T04:12:39.187+0000 7f74f9461640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:12:39.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:39 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[48052]: 2024-09-10T04:12:39.187+0000 7f74f9461640 -1 osd.4 65 *** Got signal Terminated *** 2024-09-10T04:12:39.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:39 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[48052]: 2024-09-10T04:12:39.187+0000 7f74f9461640 -1 osd.4 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:12:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.652+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7e48104b10 con 0x7f7e48107900 2024-09-10T04:12:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.653+0000 7f7e3e7fc640 1 -- 172.21.15.46:0/984056017 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f7e3806b6a0 con 0x7f7e48107900 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:12:39.655 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 2, 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 6, 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 8 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:12:39.656 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:12:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e3407ac40 msgr2=0x7f7e3407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e3407ac40 0x7f7e3407d100 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e44000c00 tx=0x7f7e44031400 comp rx=0 tx=0).stop 2024-09-10T04:12:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 msgr2=0x7f7e48074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48074a30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800d8d0 tx=0x7f7e3800dd90 comp rx=0 tx=0).stop 2024-09-10T04:12:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 shutdown_connections 2024-09-10T04:12:39.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e3407ac40 0x7f7e3407d100 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e48107900 0x7f7e48074a30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 --2- 172.21.15.46:0/984056017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e48101900 0x7f7e480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 >> 172.21.15.46:0/984056017 conn(0x7f7e480fd600 msgr2=0x7f7e480ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 shutdown_connections 2024-09-10T04:12:39.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.655+0000 7f7e4f659640 1 -- 172.21.15.46:0/984056017 wait complete. 2024-09-10T04:12:39.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.739+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/61843524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58106a00 msgr2=0x7f7a58106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.739+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/61843524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58106a00 0x7f7a58106de0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009880 tx=0x7f7a5402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.739+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/61843524 shutdown_connections 2024-09-10T04:12:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.739+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/61843524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a580ff9c0 0x7f7a580ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.739+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/61843524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a58106a00 0x7f7a58106de0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.739+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/61843524 >> 172.21.15.46:0/61843524 conn(0x7f7a580fb870 msgr2=0x7f7a580fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.740+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/61843524 shutdown_connections 2024-09-10T04:12:39.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.740+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/61843524 wait complete. 2024-09-10T04:12:39.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.740+0000 7f7a5fdc3640 1 Processor -- start 2024-09-10T04:12:39.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.740+0000 7f7a5fdc3640 1 -- start start 2024-09-10T04:12:39.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5fdc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a580ff9c0 0x7f7a5819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5fdc3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a58106a00 0x7f7a5819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5fdc3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5810eda0 con 0x7f7a580ff9c0 2024-09-10T04:12:39.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5fdc3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5810ec20 con 0x7f7a58106a00 2024-09-10T04:12:39.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5db5f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a580ff9c0 0x7f7a5819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5db5f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a580ff9c0 0x7f7a5819d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39556/0 (socket says 172.21.15.46:39556) 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5db5f640 1 -- 172.21.15.46:0/2790057496 learned_addr learned my addr 172.21.15.46:0/2790057496 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5d35e640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a58106a00 0x7f7a5819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5db5f640 1 -- 172.21.15.46:0/2790057496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a58106a00 msgr2=0x7f7a5819db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5db5f640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a58106a00 0x7f7a5819db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5db5f640 1 -- 172.21.15.46:0/2790057496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a58197790 con 0x7f7a580ff9c0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5db5f640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a580ff9c0 0x7f7a5819d610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a54031460 tx=0x7f7a540045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a4effd640 1 -- 172.21.15.46:0/2790057496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a5403f070 con 0x7f7a580ff9c0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a58197a20 con 0x7f7a580ff9c0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.741+0000 7f7a4effd640 1 -- 172.21.15.46:0/2790057496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a540035f0 con 0x7f7a580ff9c0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.742+0000 7f7a4effd640 1 -- 172.21.15.46:0/2790057496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a54035030 con 0x7f7a580ff9c0 2024-09-10T04:12:39.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.742+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a58197f50 con 0x7f7a580ff9c0 2024-09-10T04:12:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.742+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a58100e00 con 0x7f7a580ff9c0 2024-09-10T04:12:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.743+0000 7f7a4effd640 1 -- 172.21.15.46:0/2790057496 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7a54042070 con 0x7f7a580ff9c0 2024-09-10T04:12:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.744+0000 7f7a4effd640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a4407abf0 0x7f7a4407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:39.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.744+0000 7f7a4effd640 1 -- 172.21.15.46:0/2790057496 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6070+0+0 (secure 0 0 0) 0x7f7a540be520 con 0x7f7a580ff9c0 2024-09-10T04:12:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.744+0000 7f7a5d35e640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a4407abf0 0x7f7a4407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:39.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.744+0000 7f7a5d35e640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a4407abf0 0x7f7a4407d0b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a580fed60 tx=0x7f7a4800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:39.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.748+0000 7f7a4effd640 1 -- 172.21.15.46:0/2790057496 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7a54089d10 con 0x7f7a580ff9c0 2024-09-10T04:12:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.973+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7a581982c0 con 0x7f7a580ff9c0 2024-09-10T04:12:39.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.974+0000 7f7a4effd640 1 -- 172.21.15.46:0/2790057496 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7f7a5408ebc0 con 0x7f7a580ff9c0 2024-09-10T04:12:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:12:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:12:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:12:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:12:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:12:39.976 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:12:39.976 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:12:39.977 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:12:39.978 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:12:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:12:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a4407abf0 msgr2=0x7f7a4407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a4407abf0 0x7f7a4407d0b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a580fed60 tx=0x7f7a4800b390 comp rx=0 tx=0).stop 2024-09-10T04:12:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a580ff9c0 msgr2=0x7f7a5819d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a580ff9c0 0x7f7a5819d610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a54031460 tx=0x7f7a540045e0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 shutdown_connections 2024-09-10T04:12:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a4407abf0 0x7f7a4407d0b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a58106a00 0x7f7a5819db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 --2- 172.21.15.46:0/2790057496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a580ff9c0 0x7f7a5819d610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.976+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 >> 172.21.15.46:0/2790057496 conn(0x7f7a580fb870 msgr2=0x7f7a580fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.977+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 shutdown_connections 2024-09-10T04:12:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:39.977+0000 7f7a5fdc3640 1 -- 172.21.15.46:0/2790057496 wait complete. 2024-09-10T04:12:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 -- 172.21.15.46:0/1426324387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0600ffdc0 msgr2=0x7ff060100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 --2- 172.21.15.46:0/1426324387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0600ffdc0 0x7ff060100220 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff050009880 tx=0x7ff05002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 -- 172.21.15.46:0/1426324387 shutdown_connections 2024-09-10T04:12:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 --2- 172.21.15.46:0/1426324387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0600ffdc0 0x7ff060100220 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 --2- 172.21.15.46:0/1426324387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0601069a0 0x7ff060106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 -- 172.21.15.46:0/1426324387 >> 172.21.15.46:0/1426324387 conn(0x7ff0600fbc90 msgr2=0x7ff0600fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 -- 172.21.15.46:0/1426324387 shutdown_connections 2024-09-10T04:12:40.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.068+0000 7ff067486640 1 -- 172.21.15.46:0/1426324387 wait complete. 2024-09-10T04:12:40.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff067486640 1 Processor -- start 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff067486640 1 -- start start 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff067486640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ffdc0 0x7ff06019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff067486640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0601069a0 0x7ff06019dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff067486640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff06010ef40 con 0x7ff0601069a0 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff067486640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff06010edc0 con 0x7ff0600ffdc0 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff064a21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0601069a0 0x7ff06019dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff065222640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ffdc0 0x7ff06019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.069+0000 7ff064a21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0601069a0 0x7ff06019dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39578/0 (socket says 172.21.15.46:39578) 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff064a21640 1 -- 172.21.15.46:0/1036284606 learned_addr learned my addr 172.21.15.46:0/1036284606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff064a21640 1 -- 172.21.15.46:0/1036284606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ffdc0 msgr2=0x7ff06019d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff064a21640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ffdc0 0x7ff06019d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff064a21640 1 -- 172.21.15.46:0/1036284606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0601978f0 con 0x7ff0601069a0 2024-09-10T04:12:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff065222640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ffdc0 0x7ff06019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:12:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff064a21640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0601069a0 0x7ff06019dc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff050031460 tx=0x7ff050004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff0567fc640 1 -- 172.21.15.46:0/1036284606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff05003f070 con 0x7ff0601069a0 2024-09-10T04:12:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff0567fc640 1 -- 172.21.15.46:0/1036284606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff0500035f0 con 0x7ff0601069a0 2024-09-10T04:12:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff060197b80 con 0x7ff0601069a0 2024-09-10T04:12:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff0567fc640 1 -- 172.21.15.46:0/1036284606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff050035030 con 0x7ff0601069a0 2024-09-10T04:12:40.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.070+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601980b0 con 0x7ff0601069a0 2024-09-10T04:12:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.071+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff060100220 con 0x7ff0601069a0 2024-09-10T04:12:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.072+0000 7ff0567fc640 1 -- 172.21.15.46:0/1036284606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff050042070 con 0x7ff0601069a0 2024-09-10T04:12:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.072+0000 7ff0567fc640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff04c07abf0 0x7ff04c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.073+0000 7ff0567fc640 1 -- 172.21.15.46:0/1036284606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7ff0500be530 con 0x7ff0601069a0 2024-09-10T04:12:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.073+0000 7ff065222640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff04c07abf0 0x7ff04c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:40.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.073+0000 7ff065222640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff04c07abf0 0x7ff04c07d0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff060198a30 tx=0x7ff05c012040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.077+0000 7ff0567fc640 1 -- 172.21.15.46:0/1036284606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff050089d20 con 0x7ff0601069a0 2024-09-10T04:12:40.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.280+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff060198e20 con 0x7ff04c07abf0 2024-09-10T04:12:40.282 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:40 smithi046 ceph-mon[109506]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:40.282 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:40 smithi046 ceph-mon[109506]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:40.282 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:40 smithi046 ceph-mon[109506]: osd.4 marked itself down and dead 2024-09-10T04:12:40.282 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:40 smithi046 ceph-mon[109506]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:40.282 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:40 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/984056017' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:40.282 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:40 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2790057496' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.281+0000 7ff0567fc640 1 -- 172.21.15.46:0/1036284606 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7ff060198e20 con 0x7ff04c07abf0 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:12:40.283 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:12:40.284 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:12:40.284 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:12:40.284 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "10/23 daemons upgraded", 2024-09-10T04:12:40.284 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T04:12:40.284 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:12:40.284 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff04c07abf0 msgr2=0x7ff04c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff04c07abf0 0x7ff04c07d0b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff060198a30 tx=0x7ff05c012040 comp rx=0 tx=0).stop 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0601069a0 msgr2=0x7ff06019dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0601069a0 0x7ff06019dc70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff050031460 tx=0x7ff050004530 comp rx=0 tx=0).stop 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 shutdown_connections 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff04c07abf0 0x7ff04c07d0b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0601069a0 0x7ff06019dc70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 --2- 172.21.15.46:0/1036284606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0600ffdc0 0x7ff06019d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 >> 172.21.15.46:0/1036284606 conn(0x7ff0600fbc90 msgr2=0x7ff0600fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 shutdown_connections 2024-09-10T04:12:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.283+0000 7ff067486640 1 -- 172.21.15.46:0/1036284606 wait complete. 2024-09-10T04:12:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:40 smithi070 ceph-mon[79774]: from='client.34254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:40 smithi070 ceph-mon[79774]: from='client.34258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:40 smithi070 ceph-mon[79774]: osd.4 marked itself down and dead 2024-09-10T04:12:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:40 smithi070 ceph-mon[79774]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:40 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/984056017' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:40.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:40 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2790057496' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:12:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.368+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/3536191103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 msgr2=0x7f8f94106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.368+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/3536191103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f94106bd0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009880 tx=0x7f8f9002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/3536191103 shutdown_connections 2024-09-10T04:12:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/3536191103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f941007f0 0x7f8f94100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/3536191103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f94106bd0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/3536191103 >> 172.21.15.46:0/3536191103 conn(0x7f8f940fc550 msgr2=0x7f8f940fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/3536191103 shutdown_connections 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/3536191103 wait complete. 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 Processor -- start 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 -- start start 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f941007f0 0x7f8f9419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f9419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9410dcc0 con 0x7f8f941067f0 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.369+0000 7f8f9c1d6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9410db40 con 0x7f8f941007f0 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99771640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f9419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99771640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f9419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39598/0 (socket says 172.21.15.46:39598) 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99771640 1 -- 172.21.15.46:0/1365455530 learned_addr learned my addr 172.21.15.46:0/1365455530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:12:40.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99f72640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f941007f0 0x7f8f9419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99771640 1 -- 172.21.15.46:0/1365455530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f941007f0 msgr2=0x7f8f9419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99771640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f941007f0 0x7f8f9419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99771640 1 -- 172.21.15.46:0/1365455530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f941977b0 con 0x7f8f941067f0 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f99771640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f9419db70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e970 tx=0x7f8f8400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f8affd640 1 -- 172.21.15.46:0/1365455530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f8400cac0 con 0x7f8f941067f0 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f94197aa0 con 0x7f8f941067f0 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f8affd640 1 -- 172.21.15.46:0/1365455530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f840054e0 con 0x7f8f941067f0 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f8affd640 1 -- 172.21.15.46:0/1365455530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f84010430 con 0x7f8f941067f0 2024-09-10T04:12:40.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.370+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f94198030 con 0x7f8f941067f0 2024-09-10T04:12:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.371+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f94101f30 con 0x7f8f941067f0 2024-09-10T04:12:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.372+0000 7f8f8affd640 1 -- 172.21.15.46:0/1365455530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8f84004ec0 con 0x7f8f941067f0 2024-09-10T04:12:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.373+0000 7f8f8affd640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f8007ac00 0x7f8f8007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:12:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.373+0000 7f8f8affd640 1 -- 172.21.15.46:0/1365455530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6070+0+0 (secure 0 0 0) 0x7f8f8409e3c0 con 0x7f8f941067f0 2024-09-10T04:12:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.373+0000 7f8f99f72640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f8007ac00 0x7f8f8007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:12:40.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.373+0000 7f8f99f72640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f8007ac00 0x7f8f8007d0c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8f90031460 tx=0x7f8f900095c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:12:40.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.377+0000 7f8f8affd640 1 -- 172.21.15.46:0/1365455530 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8f84069b10 con 0x7f8f941067f0 2024-09-10T04:12:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.638+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f8f94109d70 con 0x7f8f941067f0 2024-09-10T04:12:40.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.638+0000 7f8f8affd640 1 -- 172.21.15.46:0/1365455530 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+290 (secure 0 0 0) 0x7f8f8406e9c0 con 0x7f8f941067f0 2024-09-10T04:12:40.640 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-09-10T04:12:40.640 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:12:40.640 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:12:40.640 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T04:12:40.640 INFO:teuthology.orchestra.run.smithi046.stdout: osd.4 (root=default,host=smithi070) is down 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f8007ac00 msgr2=0x7f8f8007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f8007ac00 0x7f8f8007d0c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8f90031460 tx=0x7f8f900095c0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 msgr2=0x7f8f9419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f9419db70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400e970 tx=0x7f8f8400ed70 comp rx=0 tx=0).stop 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 shutdown_connections 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f8007ac00 0x7f8f8007d0c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f941067f0 0x7f8f9419db70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 --2- 172.21.15.46:0/1365455530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f941007f0 0x7f8f9419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:12:40.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 >> 172.21.15.46:0/1365455530 conn(0x7f8f940fc550 msgr2=0x7f8f94069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:12:40.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 shutdown_connections 2024-09-10T04:12:40.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:12:40.640+0000 7f8f9c1d6640 1 -- 172.21.15.46:0/1365455530 wait complete. 2024-09-10T04:12:40.723 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:40 smithi070 podman[91667]: 2024-09-10 04:12:40.387894471 +0000 UTC m=+1.335379778 container died e4a511ebee84179708645fbe6efee8114e49d81ff41a47b7b355563a9a3b7a1e (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T04:12:41.009 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:40 smithi070 podman[91667]: 2024-09-10 04:12:40.820899021 +0000 UTC m=+1.768384328 container cleanup e4a511ebee84179708645fbe6efee8114e49d81ff41a47b7b355563a9a3b7a1e (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-10T04:12:41.009 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:40 smithi070 bash[91667]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4 2024-09-10T04:12:41.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:41 smithi046 ceph-mon[109506]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:12:41.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:41 smithi046 ceph-mon[109506]: osdmap e66: 6 total, 5 up, 6 in 2024-09-10T04:12:41.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:41 smithi046 ceph-mon[109506]: pgmap v198: 65 pgs: 6 stale+active+clean, 59 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:12:41.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:41 smithi046 ceph-mon[109506]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:41.279 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:41 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1365455530' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:12:41.335 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:41 smithi070 podman[91696]: 2024-09-10 04:12:41.039192835 +0000 UTC m=+0.648893906 container remove e4a511ebee84179708645fbe6efee8114e49d81ff41a47b7b355563a9a3b7a1e (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T04:12:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:41 smithi070 ceph-mon[79774]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:12:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:41 smithi070 ceph-mon[79774]: osdmap e66: 6 total, 5 up, 6 in 2024-09-10T04:12:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:41 smithi070 ceph-mon[79774]: pgmap v198: 65 pgs: 6 stale+active+clean, 59 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:12:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:41 smithi070 ceph-mon[79774]: from='client.34274 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:12:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:41 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1365455530' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:12:41.688 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:41 smithi070 podman[91786]: 2024-09-10 04:12:41.33899874 +0000 UTC m=+0.016031332 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:12:41.688 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:41 smithi070 podman[91786]: 2024-09-10 04:12:41.465018118 +0000 UTC m=+0.142050699 container create 17fde01f2b004ee5394d8c611e4a16b30ab1fe7199b225b34a93ccf41a4259f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:12:41.689 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:41 smithi070 podman[91786]: 2024-09-10 04:12:41.684511665 +0000 UTC m=+0.361544253 container init 17fde01f2b004ee5394d8c611e4a16b30ab1fe7199b225b34a93ccf41a4259f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T04:12:41.689 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:41 smithi070 podman[91786]: 2024-09-10 04:12:41.687901407 +0000 UTC m=+0.364933989 container start 17fde01f2b004ee5394d8c611e4a16b30ab1fe7199b225b34a93ccf41a4259f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T04:12:42.015 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:41 smithi070 podman[91786]: 2024-09-10 04:12:41.742785869 +0000 UTC m=+0.419818465 container attach 17fde01f2b004ee5394d8c611e4a16b30ab1fe7199b225b34a93ccf41a4259f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T04:12:42.016 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:41 smithi070 podman[91786]: 2024-09-10 04:12:41.882270199 +0000 UTC m=+0.559302782 container died 17fde01f2b004ee5394d8c611e4a16b30ab1fe7199b225b34a93ccf41a4259f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T04:12:42.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:42 smithi070 ceph-mon[79774]: osdmap e67: 6 total, 5 up, 6 in 2024-09-10T04:12:42.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:42 smithi046 ceph-mon[109506]: osdmap e67: 6 total, 5 up, 6 in 2024-09-10T04:12:42.680 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:42 smithi070 podman[91809]: 2024-09-10 04:12:42.42935082 +0000 UTC m=+0.540849754 container remove 17fde01f2b004ee5394d8c611e4a16b30ab1fe7199b225b34a93ccf41a4259f4 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T04:12:42.681 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:42 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.4.service: Deactivated successfully. 2024-09-10T04:12:42.681 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:42 smithi070 systemd[1]: Stopped Ceph osd.4 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:12:42.681 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:42 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.4.service: Consumed 51.768s CPU time. 2024-09-10T04:12:42.681 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:42 smithi070 systemd[1]: Starting Ceph osd.4 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:12:43.142 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:43 smithi070 ceph-mon[79774]: pgmap v200: 65 pgs: 19 peering, 4 stale+active+clean, 42 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:12:43.142 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:42 smithi070 podman[91907]: 2024-09-10 04:12:42.792125347 +0000 UTC m=+0.015629610 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:12:43.142 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:42 smithi070 podman[91907]: 2024-09-10 04:12:42.922560796 +0000 UTC m=+0.146065054 container create 8fcd6f2d8f57a00afcfedd378f358b06c912add7ff72cadbd91ae3c0dd5316a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:12:43.142 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:43 smithi070 podman[91907]: 2024-09-10 04:12:43.141477376 +0000 UTC m=+0.364981630 container init 8fcd6f2d8f57a00afcfedd378f358b06c912add7ff72cadbd91ae3c0dd5316a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T04:12:43.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:43 smithi046 ceph-mon[109506]: pgmap v200: 65 pgs: 19 peering, 4 stale+active+clean, 42 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:12:43.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:43 smithi070 podman[91907]: 2024-09-10 04:12:43.144946929 +0000 UTC m=+0.368451184 container start 8fcd6f2d8f57a00afcfedd378f358b06c912add7ff72cadbd91ae3c0dd5316a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T04:12:43.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:43 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:43.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:43 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:43.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:43 smithi070 bash[91907]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:43.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:43 smithi070 bash[91907]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:12:43.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:43 smithi070 podman[91907]: 2024-09-10 04:12:43.291790904 +0000 UTC m=+0.515295165 container attach 8fcd6f2d8f57a00afcfedd378f358b06c912add7ff72cadbd91ae3c0dd5316a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:12:44.274 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:44 smithi046 ceph-mon[109506]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T04:12:44.292 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T04:12:44.292 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 bash[91907]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T04:12:44.293 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T04:12:44.293 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 bash[91907]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T04:12:44.293 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-mon[79774]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T04:12:44.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T04:12:44.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 bash[91907]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T04:12:44.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 bash[91907]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 bash[91907]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 bash[91907]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate[91922]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 bash[91907]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-10T04:12:44.586 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 podman[91907]: 2024-09-10 04:12:44.319509263 +0000 UTC m=+1.543013518 container died 8fcd6f2d8f57a00afcfedd378f358b06c912add7ff72cadbd91ae3c0dd5316a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T04:12:45.262 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:45 smithi070 ceph-mon[79774]: pgmap v201: 65 pgs: 3 active+undersized, 19 peering, 3 stale+active+clean, 2 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s; 6/312 objects degraded (1.923%) 2024-09-10T04:12:45.262 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:44 smithi070 podman[92068]: 2024-09-10 04:12:44.920116525 +0000 UTC m=+0.594311401 container remove 8fcd6f2d8f57a00afcfedd378f358b06c912add7ff72cadbd91ae3c0dd5316a5 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731) 2024-09-10T04:12:45.263 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:45 smithi070 podman[92129]: 2024-09-10 04:12:45.029507941 +0000 UTC m=+0.016345360 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:12:45.263 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:45 smithi070 podman[92129]: 2024-09-10 04:12:45.14609036 +0000 UTC m=+0.132927821 container create 3bf61d5adfbe6a23955ba0a3bf6f086dba640c212485bb04bc0d0b892d04d6c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:12:45.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:45 smithi046 ceph-mon[109506]: pgmap v201: 65 pgs: 3 active+undersized, 19 peering, 3 stale+active+clean, 2 active+undersized+degraded, 38 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 2 op/s; 6/312 objects degraded (1.923%) 2024-09-10T04:12:45.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:45 smithi070 podman[92129]: 2024-09-10 04:12:45.38451979 +0000 UTC m=+0.371357219 container init 3bf61d5adfbe6a23955ba0a3bf6f086dba640c212485bb04bc0d0b892d04d6c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T04:12:45.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:45 smithi070 podman[92129]: 2024-09-10 04:12:45.387752988 +0000 UTC m=+0.374590407 container start 3bf61d5adfbe6a23955ba0a3bf6f086dba640c212485bb04bc0d0b892d04d6c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T04:12:45.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:45 smithi070 ceph-osd[92148]: -- 172.21.15.70:0/3892262292 <== mon.1 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560fd7d6e1e0 con 0x560fd7d54000 2024-09-10T04:12:45.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:45 smithi070 bash[92129]: 3bf61d5adfbe6a23955ba0a3bf6f086dba640c212485bb04bc0d0b892d04d6c0 2024-09-10T04:12:45.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:45 smithi070 systemd[1]: Started Ceph osd.4 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:12:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:46 smithi070 ceph-mon[79774]: Health check failed: Degraded data redundancy: 6/312 objects degraded (1.923%), 2 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:46.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:46.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:46 smithi046 ceph-mon[109506]: Health check failed: Degraded data redundancy: 6/312 objects degraded (1.923%), 2 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:47.051 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:47 smithi070 ceph-mon[79774]: pgmap v202: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:47.051 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:47 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:12:47.035+0000 7f050b8f9640 -1 Falling back to public interface 2024-09-10T04:12:47.286 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:47 smithi046 ceph-mon[109506]: pgmap v202: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:48 smithi070 ceph-mon[79774]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T04:12:48.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:48 smithi046 ceph-mon[109506]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T04:12:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:49 smithi070 ceph-mon[79774]: pgmap v203: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:49 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:49 smithi046 ceph-mon[109506]: pgmap v203: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:49.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:49.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:49.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:49 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:51.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:51 smithi046 ceph-mon[109506]: pgmap v204: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:51.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:51 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 52/312 objects degraded (16.667%), 15 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:51 smithi070 ceph-mon[79774]: pgmap v204: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:51 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 52/312 objects degraded (16.667%), 15 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:53.085 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:52 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:12:52.692+0000 7f050b8f9640 -1 osd.4 65 log_to_monitors true 2024-09-10T04:12:53.585 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:12:53.355+0000 7f05036a3640 -1 osd.4 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:12:53.626 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: pgmap v205: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:53.626 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.626 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:53.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:12:53.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:53.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:53.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:53.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:53.628 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-10T04:12:53.628 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T04:12:53.628 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T04:12:53.628 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='osd.4 [v2:172.21.15.70:6808/4085759346,v1:172.21.15.70:6809/4085759346]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T04:12:53.628 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T04:12:53.628 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.628 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:53.629 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: pgmap v205: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:53.629 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.629 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.629 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:12:53.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:12:53.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:12:53.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:53.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:53.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:12:53.630 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-10T04:12:53.631 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T04:12:53.631 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T04:12:53.631 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='osd.4 [v2:172.21.15.70:6808/4085759346,v1:172.21.15.70:6809/4085759346]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T04:12:53.631 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T04:12:53.631 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:12:53.631 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:12:54.603 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:54 smithi046 ceph-mon[109506]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T04:12:54.603 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:54 smithi046 ceph-mon[109506]: osdmap e68: 6 total, 5 up, 6 in 2024-09-10T04:12:54.603 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:54 smithi046 ceph-mon[109506]: from='osd.4 [v2:172.21.15.70:6808/4085759346,v1:172.21.15.70:6809/4085759346]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:54.603 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:54 smithi046 ceph-mon[109506]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:54 smithi070 ceph-mon[79774]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T04:12:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:54 smithi070 ceph-mon[79774]: osdmap e68: 6 total, 5 up, 6 in 2024-09-10T04:12:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:54 smithi070 ceph-mon[79774]: from='osd.4 [v2:172.21.15.70:6808/4085759346,v1:172.21.15.70:6809/4085759346]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:54 smithi070 ceph-mon[79774]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:12:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:55 smithi070 ceph-mon[79774]: pgmap v207: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:55 smithi070 ceph-mon[79774]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:12:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:55 smithi070 ceph-mon[79774]: osd.4 [v2:172.21.15.70:6808/4085759346,v1:172.21.15.70:6809/4085759346] boot 2024-09-10T04:12:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:55 smithi070 ceph-mon[79774]: osdmap e69: 6 total, 6 up, 6 in 2024-09-10T04:12:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:55 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T04:12:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:55 smithi046 ceph-mon[109506]: pgmap v207: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 52/312 objects degraded (16.667%) 2024-09-10T04:12:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:55 smithi046 ceph-mon[109506]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:12:55.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:55 smithi046 ceph-mon[109506]: osd.4 [v2:172.21.15.70:6808/4085759346,v1:172.21.15.70:6809/4085759346] boot 2024-09-10T04:12:55.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:55 smithi046 ceph-mon[109506]: osdmap e69: 6 total, 6 up, 6 in 2024-09-10T04:12:55.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:55 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T04:12:56.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:56 smithi046 ceph-mon[109506]: osdmap e70: 6 total, 6 up, 6 in 2024-09-10T04:12:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:56 smithi070 ceph-mon[79774]: osdmap e70: 6 total, 6 up, 6 in 2024-09-10T04:12:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:57 smithi070 ceph-mon[79774]: pgmap v210: 65 pgs: 11 peering, 10 active+undersized, 6 active+undersized+degraded, 38 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 5 op/s; 16/312 objects degraded (5.128%) 2024-09-10T04:12:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:57 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 16/312 objects degraded (5.128%), 6 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:57 smithi046 ceph-mon[109506]: pgmap v210: 65 pgs: 11 peering, 10 active+undersized, 6 active+undersized+degraded, 38 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 5 op/s; 16/312 objects degraded (5.128%) 2024-09-10T04:12:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:57 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 16/312 objects degraded (5.128%), 6 pgs degraded (PG_DEGRADED) 2024-09-10T04:12:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:12:59 smithi046 ceph-mon[109506]: pgmap v211: 65 pgs: 11 peering, 7 active+undersized, 5 active+undersized+degraded, 42 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 12/312 objects degraded (3.846%) 2024-09-10T04:12:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:12:59 smithi070 ceph-mon[79774]: pgmap v211: 65 pgs: 11 peering, 7 active+undersized, 5 active+undersized+degraded, 42 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 12/312 objects degraded (3.846%) 2024-09-10T04:13:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:00 smithi070 ceph-mon[79774]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/312 objects degraded (3.846%), 5 pgs degraded) 2024-09-10T04:13:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:00 smithi046 ceph-mon[109506]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/312 objects degraded (3.846%), 5 pgs degraded) 2024-09-10T04:13:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:01 smithi046 ceph-mon[109506]: pgmap v212: 65 pgs: 10 peering, 55 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T04:13:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:01 smithi070 ceph-mon[79774]: pgmap v212: 65 pgs: 10 peering, 55 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T04:13:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:03 smithi070 ceph-mon[79774]: pgmap v213: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:13:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:03 smithi046 ceph-mon[109506]: pgmap v213: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:13:05.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:05 smithi046 ceph-mon[109506]: pgmap v214: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T04:13:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:05 smithi070 ceph-mon[79774]: pgmap v214: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T04:13:07.718 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:07 smithi070 ceph-mon[79774]: pgmap v215: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-10T04:13:07.719 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:07 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-10T04:13:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:07 smithi046 ceph-mon[109506]: pgmap v215: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-10T04:13:07.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:07 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-10T04:13:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:08 smithi046 ceph-mon[109506]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T04:13:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:08 smithi046 ceph-mon[109506]: Upgrade: osd.5 is safe to restart 2024-09-10T04:13:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:08.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:08 smithi070 ceph-mon[79774]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T04:13:08.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:08 smithi070 ceph-mon[79774]: Upgrade: osd.5 is safe to restart 2024-09-10T04:13:08.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:08.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:08.710 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:09 smithi070 ceph-mon[79774]: pgmap v216: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:13:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:09 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:09 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T04:13:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:09 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:09 smithi046 ceph-mon[109506]: pgmap v216: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:13:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:09 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:09 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T04:13:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:09 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:10 smithi070 ceph-mon[79774]: Upgrade: Updating osd.5 2024-09-10T04:13:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:10 smithi070 ceph-mon[79774]: Deploying daemon osd.5 on smithi070 2024-09-10T04:13:10.742 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:10 smithi046 ceph-mon[109506]: Upgrade: Updating osd.5 2024-09-10T04:13:10.742 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:10 smithi046 ceph-mon[109506]: Deploying daemon osd.5 on smithi070 2024-09-10T04:13:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/2582920831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01007f0 msgr2=0x7f7fd0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/2582920831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01007f0 0x7f7fd0100c50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009c20 tx=0x7f7fc002a970 comp rx=0 tx=0).stop 2024-09-10T04:13:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/2582920831 shutdown_connections 2024-09-10T04:13:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/2582920831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01007f0 0x7f7fd0100c50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/2582920831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd01067f0 0x7f7fd0106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/2582920831 >> 172.21.15.46:0/2582920831 conn(0x7f7fd00fc550 msgr2=0x7f7fd00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/2582920831 shutdown_connections 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.739+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/2582920831 wait complete. 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.740+0000 7f7fd6ed0640 1 Processor -- start 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.740+0000 7f7fd6ed0640 1 -- start start 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.740+0000 7f7fd6ed0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd01007f0 0x7f7fd019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.740+0000 7f7fd6ed0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01067f0 0x7f7fd019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.740+0000 7f7fd6ed0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd010dcc0 con 0x7f7fd01067f0 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.740+0000 7f7fd6ed0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd010db40 con 0x7f7fd01007f0 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01067f0 0x7f7fd019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01067f0 0x7f7fd019cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44188/0 (socket says 172.21.15.46:44188) 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcffff640 1 -- 172.21.15.46:0/1336449543 learned_addr learned my addr 172.21.15.46:0/1336449543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fd4c6c640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd01007f0 0x7f7fd019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcffff640 1 -- 172.21.15.46:0/1336449543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd01007f0 msgr2=0x7f7fd019c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcffff640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd01007f0 0x7f7fd019c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcffff640 1 -- 172.21.15.46:0/1336449543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd01966b0 con 0x7f7fd01067f0 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcffff640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01067f0 0x7f7fd019cb20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7fc002ae90 tx=0x7f7fc002fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.741+0000 7f7fcdffb640 1 -- 172.21.15.46:0/1336449543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7fc0047070 con 0x7f7fd01067f0 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.742+0000 7f7fcdffb640 1 -- 172.21.15.46:0/1336449543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7fc0003e30 con 0x7f7fd01067f0 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.742+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd0196940 con 0x7f7fd01067f0 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.742+0000 7f7fcdffb640 1 -- 172.21.15.46:0/1336449543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7fc0034070 con 0x7f7fd01067f0 2024-09-10T04:13:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.742+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd0196e40 con 0x7f7fd01067f0 2024-09-10T04:13:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.743+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd0101f30 con 0x7f7fd01067f0 2024-09-10T04:13:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.744+0000 7f7fcdffb640 1 -- 172.21.15.46:0/1336449543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7fc004a070 con 0x7f7fd01067f0 2024-09-10T04:13:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.744+0000 7f7fcdffb640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fbc07abf0 0x7f7fbc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.744+0000 7f7fcdffb640 1 -- 172.21.15.46:0/1336449543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6070+0+0 (secure 0 0 0) 0x7f7fc00be7a0 con 0x7f7fd01067f0 2024-09-10T04:13:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.744+0000 7f7fd4c6c640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fbc07abf0 0x7f7fbc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.745+0000 7f7fd4c6c640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fbc07abf0 0x7f7fbc07d0b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009bd0 tx=0x7f7fc8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:10.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.749+0000 7f7fcdffb640 1 -- 172.21.15.46:0/1336449543 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7fc003f070 con 0x7f7fd01067f0 2024-09-10T04:13:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.958+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7fd01971b0 con 0x7f7fbc07abf0 2024-09-10T04:13:10.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.959+0000 7f7fcdffb640 1 -- 172.21.15.46:0/1336449543 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f7fd01971b0 con 0x7f7fbc07abf0 2024-09-10T04:13:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.961+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fbc07abf0 msgr2=0x7f7fbc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.961+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fbc07abf0 0x7f7fbc07d0b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009bd0 tx=0x7f7fc8005f90 comp rx=0 tx=0).stop 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.961+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01067f0 msgr2=0x7f7fd019cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01067f0 0x7f7fd019cb20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7fc002ae90 tx=0x7f7fc002fd60 comp rx=0 tx=0).stop 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 shutdown_connections 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fbc07abf0 0x7f7fbc07d0b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fd01067f0 0x7f7fd019cb20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 --2- 172.21.15.46:0/1336449543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7fd01007f0 0x7f7fd019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 >> 172.21.15.46:0/1336449543 conn(0x7f7fd00fc550 msgr2=0x7f7fd00fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 shutdown_connections 2024-09-10T04:13:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:10.962+0000 7f7fd6ed0640 1 -- 172.21.15.46:0/1336449543 wait complete. 2024-09-10T04:13:10.973 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 -- 172.21.15.46:0/575208110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1007f0 msgr2=0x7f470c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 --2- 172.21.15.46:0/575208110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1007f0 0x7f470c100c50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009880 tx=0x7f46fc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 -- 172.21.15.46:0/575208110 shutdown_connections 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 --2- 172.21.15.46:0/575208110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1007f0 0x7f470c100c50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 --2- 172.21.15.46:0/575208110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f470c1067f0 0x7f470c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 -- 172.21.15.46:0/575208110 >> 172.21.15.46:0/575208110 conn(0x7f470c0fc550 msgr2=0x7f470c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 -- 172.21.15.46:0/575208110 shutdown_connections 2024-09-10T04:13:11.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 -- 172.21.15.46:0/575208110 wait complete. 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 Processor -- start 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.048+0000 7f4714234640 1 -- start start 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f4714234640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f470c1007f0 0x7f470c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f4714234640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1067f0 0x7f470c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f4714234640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f470c10dca0 con 0x7f470c1067f0 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f4714234640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f470c10db20 con 0x7f470c1007f0 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f47117cf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1067f0 0x7f470c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f47117cf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1067f0 0x7f470c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44208/0 (socket says 172.21.15.46:44208) 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f47117cf640 1 -- 172.21.15.46:0/3513800093 learned_addr learned my addr 172.21.15.46:0/3513800093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f4711fd0640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f470c1007f0 0x7f470c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f47117cf640 1 -- 172.21.15.46:0/3513800093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f470c1007f0 msgr2=0x7f470c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f47117cf640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f470c1007f0 0x7f470c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f47117cf640 1 -- 172.21.15.46:0/3513800093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c0fd700 con 0x7f470c1067f0 2024-09-10T04:13:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.049+0000 7f47117cf640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1067f0 0x7f470c0fd1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f46fc031460 tx=0x7f46fc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.050+0000 7f4702ffd640 1 -- 172.21.15.46:0/3513800093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f46fc03f070 con 0x7f470c1067f0 2024-09-10T04:13:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.050+0000 7f4702ffd640 1 -- 172.21.15.46:0/3513800093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46fc0044d0 con 0x7f470c1067f0 2024-09-10T04:13:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.050+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c0fd990 con 0x7f470c1067f0 2024-09-10T04:13:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.050+0000 7f4702ffd640 1 -- 172.21.15.46:0/3513800093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f46fc035030 con 0x7f470c1067f0 2024-09-10T04:13:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.050+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c1a3cf0 con 0x7f470c1067f0 2024-09-10T04:13:11.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.051+0000 7f4702ffd640 1 -- 172.21.15.46:0/3513800093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f46fc042070 con 0x7f470c1067f0 2024-09-10T04:13:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.052+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c101f30 con 0x7f470c1067f0 2024-09-10T04:13:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.052+0000 7f4702ffd640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f46f807ac00 0x7f46f807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.052+0000 7f4702ffd640 1 -- 172.21.15.46:0/3513800093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6070+0+0 (secure 0 0 0) 0x7f46fc0be650 con 0x7f470c1067f0 2024-09-10T04:13:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.052+0000 7f4711fd0640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f46f807ac00 0x7f46f807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.053+0000 7f4711fd0640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f46f807ac00 0x7f46f807d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f470800abe0 tx=0x7f470800a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.057+0000 7f4702ffd640 1 -- 172.21.15.46:0/3513800093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f46fc004cf0 con 0x7f470c1067f0 2024-09-10T04:13:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.264+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f470c062630 con 0x7f46f807ac00 2024-09-10T04:13:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.265+0000 7f4702ffd640 1 -- 172.21.15.46:0/3513800093 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f470c062630 con 0x7f46f807ac00 2024-09-10T04:13:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.267+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f46f807ac00 msgr2=0x7f46f807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.267+0000 7f4714234640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f46f807ac00 0x7f46f807d0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f470800abe0 tx=0x7f470800a2d0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.267+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1067f0 msgr2=0x7f470c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.267+0000 7f4714234640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1067f0 0x7f470c0fd1c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f46fc031460 tx=0x7f46fc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.268+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 shutdown_connections 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.268+0000 7f4714234640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f46f807ac00 0x7f46f807d0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.268+0000 7f4714234640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f470c1067f0 0x7f470c0fd1c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.268+0000 7f4714234640 1 --2- 172.21.15.46:0/3513800093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f470c1007f0 0x7f470c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.268+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 >> 172.21.15.46:0/3513800093 conn(0x7f470c0fc550 msgr2=0x7f470c069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.268+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 shutdown_connections 2024-09-10T04:13:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.268+0000 7f4714234640 1 -- 172.21.15.46:0/3513800093 wait complete. 2024-09-10T04:13:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.352+0000 7f88849e5640 1 -- 172.21.15.46:0/3823902485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801007f0 msgr2=0x7f8880100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.352+0000 7f88849e5640 1 --2- 172.21.15.46:0/3823902485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801007f0 0x7f8880100c50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8874009880 tx=0x7f887402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 -- 172.21.15.46:0/3823902485 shutdown_connections 2024-09-10T04:13:11.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 --2- 172.21.15.46:0/3823902485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801007f0 0x7f8880100c50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 --2- 172.21.15.46:0/3823902485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88801067f0 0x7f8880106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 -- 172.21.15.46:0/3823902485 >> 172.21.15.46:0/3823902485 conn(0x7f88800fc550 msgr2=0x7f88800fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 -- 172.21.15.46:0/3823902485 shutdown_connections 2024-09-10T04:13:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 -- 172.21.15.46:0/3823902485 wait complete. 2024-09-10T04:13:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 Processor -- start 2024-09-10T04:13:11.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.353+0000 7f88849e5640 1 -- start start 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f88849e5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88801007f0 0x7f888019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f88849e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801067f0 0x7f888019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f88849e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f888010edc0 con 0x7f88801067f0 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f88849e5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f888010ec40 con 0x7f88801007f0 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801067f0 0x7f888019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801067f0 0x7f888019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44220/0 (socket says 172.21.15.46:44220) 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887dd9b640 1 -- 172.21.15.46:0/2573527061 learned_addr learned my addr 172.21.15.46:0/2573527061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887e59c640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88801007f0 0x7f888019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887dd9b640 1 -- 172.21.15.46:0/2573527061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88801007f0 msgr2=0x7f888019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887dd9b640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88801007f0 0x7f888019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887dd9b640 1 -- 172.21.15.46:0/2573527061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88801977b0 con 0x7f88801067f0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887e59c640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88801007f0 0x7f888019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.354+0000 7f887dd9b640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801067f0 0x7f888019db70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8874031460 tx=0x7f8874004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.355+0000 7f886f7fe640 1 -- 172.21.15.46:0/2573527061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f887403f070 con 0x7f88801067f0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.355+0000 7f886f7fe640 1 -- 172.21.15.46:0/2573527061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f88740035f0 con 0x7f88801067f0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.355+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8880197a40 con 0x7f88801067f0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.355+0000 7f886f7fe640 1 -- 172.21.15.46:0/2573527061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8874035030 con 0x7f88801067f0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.355+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8880197f70 con 0x7f88801067f0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.356+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8880101f30 con 0x7f88801067f0 2024-09-10T04:13:11.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.356+0000 7f886f7fe640 1 -- 172.21.15.46:0/2573527061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8874042070 con 0x7f88801067f0 2024-09-10T04:13:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.357+0000 7f886f7fe640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f886807ac40 0x7f886807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.357+0000 7f886f7fe640 1 -- 172.21.15.46:0/2573527061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6070+0+0 (secure 0 0 0) 0x7f88740be3f0 con 0x7f88801067f0 2024-09-10T04:13:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.357+0000 7f887e59c640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f886807ac40 0x7f886807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.357+0000 7f887e59c640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f886807ac40 0x7f886807d100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8870009fc0 tx=0x7f8870005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:11.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.362+0000 7f886f7fe640 1 -- 172.21.15.46:0/2573527061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8874089c60 con 0x7f88801067f0 2024-09-10T04:13:11.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.556+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f88801982e0 con 0x7f886807ac40 2024-09-10T04:13:11.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.566+0000 7f886f7fe640 1 -- 172.21.15.46:0/2573527061 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f88801982e0 con 0x7f886807ac40 2024-09-10T04:13:11.568 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:13:11.568 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (4m) 88s ago 16m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:13:11.568 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (7m) 88s ago 16m 9026k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:13:11.568 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (7m) 23s ago 15m 8975k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:13:11.568 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (3m) 88s ago 16m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:13:11.568 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (3m) 23s ago 15m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (4m) 88s ago 15m 77.6M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (10m) 88s ago 10m 90.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (10m) 88s ago 10m 145M - 19.1.1-298-gb53b6d3e 54467abc55f8 e9627c6fb553 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (10m) 23s ago 10m 26.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (10m) 23s ago 10m 25.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (6m) 88s ago 17m 528M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (5m) 23s ago 15m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (3m) 88s ago 17m 43.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (3m) 23s ago 15m 38.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (5m) 88s ago 16m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (5m) 23s ago 15m 11.6M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (2m) 88s ago 14m 81.8M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:13:11.569 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (2m) 88s ago 13m 86.9M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:13:11.570 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (94s) 88s ago 13m 12.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:13:11.570 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (59s) 23s ago 12m 88.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:13:11.570 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (26s) 23s ago 12m 12.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:13:11.570 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (11m) 23s ago 11m 197M 2240M 19.1.1-298-gb53b6d3e 54467abc55f8 5acf243605d5 2024-09-10T04:13:11.570 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (5m) 88s ago 15m 55.9M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f886807ac40 msgr2=0x7f886807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f886807ac40 0x7f886807d100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8870009fc0 tx=0x7f8870005f90 comp rx=0 tx=0).stop 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801067f0 msgr2=0x7f888019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801067f0 0x7f888019db70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8874031460 tx=0x7f8874004530 comp rx=0 tx=0).stop 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 shutdown_connections 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f886807ac40 0x7f886807d100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88801067f0 0x7f888019db70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 --2- 172.21.15.46:0/2573527061 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88801007f0 0x7f888019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 >> 172.21.15.46:0/2573527061 conn(0x7f88800fc550 msgr2=0x7f88800fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 shutdown_connections 2024-09-10T04:13:11.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.568+0000 7f88849e5640 1 -- 172.21.15.46:0/2573527061 wait complete. 2024-09-10T04:13:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 -- 172.21.15.46:0/4140337502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 msgr2=0x7f593c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 --2- 172.21.15.46:0/4140337502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c100c50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5924009880 tx=0x7f592402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.654 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:11 smithi046 ceph-mon[109506]: pgmap v217: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 -- 172.21.15.46:0/4140337502 shutdown_connections 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 --2- 172.21.15.46:0/4140337502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c100c50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 --2- 172.21.15.46:0/4140337502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593c1067f0 0x7f593c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 -- 172.21.15.46:0/4140337502 >> 172.21.15.46:0/4140337502 conn(0x7f593c0fc550 msgr2=0x7f593c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 -- 172.21.15.46:0/4140337502 shutdown_connections 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.652+0000 7f5941c67640 1 -- 172.21.15.46:0/4140337502 wait complete. 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f5941c67640 1 Processor -- start 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f5941c67640 1 -- start start 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f5941c67640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f5941c67640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593c1067f0 0x7f593c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f5941c67640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10dcc0 con 0x7f593c1007f0 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f5941c67640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10db40 con 0x7f593c1067f0 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f593b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44252/0 (socket says 172.21.15.46:44252) 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f593b7fe640 1 -- 172.21.15.46:0/3996057322 learned_addr learned my addr 172.21.15.46:0/3996057322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.653+0000 7f5933fff640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593c1067f0 0x7f593c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f593b7fe640 1 -- 172.21.15.46:0/3996057322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593c1067f0 msgr2=0x7f593c198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f593b7fe640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593c1067f0 0x7f593c198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f593b7fe640 1 -- 172.21.15.46:0/3996057322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c199340 con 0x7f593c1007f0 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f5933fff640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593c1067f0 0x7f593c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f593b7fe640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c1987b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f592c00e970 tx=0x7f592c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f59397fa640 1 -- 172.21.15.46:0/3996057322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f592c00cac0 con 0x7f593c1007f0 2024-09-10T04:13:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f59397fa640 1 -- 172.21.15.46:0/3996057322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f592c0054e0 con 0x7f593c1007f0 2024-09-10T04:13:11.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f59397fa640 1 -- 172.21.15.46:0/3996057322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f592c010430 con 0x7f593c1007f0 2024-09-10T04:13:11.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c1a1820 con 0x7f593c1007f0 2024-09-10T04:13:11.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.654+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1a1c20 con 0x7f593c1007f0 2024-09-10T04:13:11.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.656+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593c101f30 con 0x7f593c1007f0 2024-09-10T04:13:11.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.656+0000 7f59397fa640 1 -- 172.21.15.46:0/3996057322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f592c004f00 con 0x7f593c1007f0 2024-09-10T04:13:11.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.657+0000 7f59397fa640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f592807ac40 0x7f592807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:11.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.657+0000 7f5933fff640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f592807ac40 0x7f592807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.657+0000 7f59397fa640 1 -- 172.21.15.46:0/3996057322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6070+0+0 (secure 0 0 0) 0x7f592c09e170 con 0x7f593c1007f0 2024-09-10T04:13:11.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.657+0000 7f5933fff640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f592807ac40 0x7f592807d100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f593c06a830 tx=0x7f5924031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:11.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.661+0000 7f59397fa640 1 -- 172.21.15.46:0/3996057322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f592c0699e0 con 0x7f593c1007f0 2024-09-10T04:13:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:11 smithi070 ceph-mon[79774]: pgmap v217: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:13:11.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:11 smithi070 systemd[1]: Stopping Ceph osd.5 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:13:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.921+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f593c02d7c0 con 0x7f593c1007f0 2024-09-10T04:13:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.922+0000 7f59397fa640 1 -- 172.21.15.46:0/3996057322 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+836 (secure 0 0 0) 0x7f592c06e890 con 0x7f593c1007f0 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 5 2024-09-10T04:13:11.924 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 4 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 5, 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 9 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:13:11.925 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f592807ac40 msgr2=0x7f592807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f592807ac40 0x7f592807d100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f593c06a830 tx=0x7f5924031400 comp rx=0 tx=0).stop 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 msgr2=0x7f593c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c1987b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f592c00e970 tx=0x7f592c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 shutdown_connections 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f592807ac40 0x7f592807d100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f593c1067f0 0x7f593c198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 --2- 172.21.15.46:0/3996057322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f593c1007f0 0x7f593c1987b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 >> 172.21.15.46:0/3996057322 conn(0x7f593c0fc550 msgr2=0x7f593c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 shutdown_connections 2024-09-10T04:13:11.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:11.924+0000 7f5941c67640 1 -- 172.21.15.46:0/3996057322 wait complete. 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 -- 172.21.15.46:0/1834288089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130100800 msgr2=0x7f2130100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 --2- 172.21.15.46:0/1834288089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130100800 0x7f2130100c60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2124009c20 tx=0x7f212402a970 comp rx=0 tx=0).stop 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 -- 172.21.15.46:0/1834288089 shutdown_connections 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 --2- 172.21.15.46:0/1834288089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130100800 0x7f2130100c60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 --2- 172.21.15.46:0/1834288089 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2130106800 0x7f2130106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 -- 172.21.15.46:0/1834288089 >> 172.21.15.46:0/1834288089 conn(0x7f21300fc520 msgr2=0x7f21300fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 -- 172.21.15.46:0/1834288089 shutdown_connections 2024-09-10T04:13:12.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.008+0000 7f2135957640 1 -- 172.21.15.46:0/1834288089 wait complete. 2024-09-10T04:13:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f2135957640 1 Processor -- start 2024-09-10T04:13:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f2135957640 1 -- start start 2024-09-10T04:13:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f2135957640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2130100800 0x7f213019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f2135957640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130106800 0x7f213019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f2135957640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f213010dcb0 con 0x7f2130106800 2024-09-10T04:13:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f2135957640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f213010db30 con 0x7f2130100800 2024-09-10T04:13:12.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f212e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130106800 0x7f213019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f212e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130106800 0x7f213019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44278/0 (socket says 172.21.15.46:44278) 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f212e7fc640 1 -- 172.21.15.46:0/2621091451 learned_addr learned my addr 172.21.15.46:0/2621091451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.009+0000 7f212effd640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2130100800 0x7f213019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f212e7fc640 1 -- 172.21.15.46:0/2621091451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2130100800 msgr2=0x7f213019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f212e7fc640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2130100800 0x7f213019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f212e7fc640 1 -- 172.21.15.46:0/2621091451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21301977f0 con 0x7f2130106800 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f212e7fc640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130106800 0x7f213019dbb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f212402ae90 tx=0x7f212402f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f2134955640 1 -- 172.21.15.46:0/2621091451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2124047070 con 0x7f2130106800 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f2134955640 1 -- 172.21.15.46:0/2621091451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f212402f7e0 con 0x7f2130106800 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2130197a80 con 0x7f2130106800 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f2134955640 1 -- 172.21.15.46:0/2621091451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2124034070 con 0x7f2130106800 2024-09-10T04:13:12.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.010+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2130197fb0 con 0x7f2130106800 2024-09-10T04:13:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.011+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2130101f40 con 0x7f2130106800 2024-09-10T04:13:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.012+0000 7f2134955640 1 -- 172.21.15.46:0/2621091451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f212404a070 con 0x7f2130106800 2024-09-10T04:13:12.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.012+0000 7f2134955640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f211807ac40 0x7f211807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.012+0000 7f2134955640 1 -- 172.21.15.46:0/2621091451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6070+0+0 (secure 0 0 0) 0x7f21240be7b0 con 0x7f2130106800 2024-09-10T04:13:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.012+0000 7f212effd640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f211807ac40 0x7f211807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.013+0000 7f212effd640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f211807ac40 0x7f211807d100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2120005d00 tx=0x7f212000b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:12.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.017+0000 7f2134955640 1 -- 172.21.15.46:0/2621091451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f212403f070 con 0x7f2130106800 2024-09-10T04:13:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.239+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2130198d20 con 0x7f2130106800 2024-09-10T04:13:12.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.240+0000 7f2134955640 1 -- 172.21.15.46:0/2621091451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) ==== 76+0+2043 (secure 0 0 0) 0x7f212408a020 con 0x7f2130106800 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:e16 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:02:37:146557+0000 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:13:12.242 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 16 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:02:36.662727+0000 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-09-10T04:13:12.243 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=14558,1=24325} 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:13:12.244 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 14558 members: 14558,24325 2024-09-10T04:13:12.245 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:14558} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.46:6826/4038650990,v1:172.21.15.46:6827/4038650990] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:12.245 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{0:14574} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.46:6828/1498169047,v1:172.21.15.46:6829/1498169047] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:12.245 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.70:6824/1155442666,v1:172.21.15.70:6825/1155442666] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:12.245 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:24339} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.70:6826/3341360432,v1:172.21.15.70:6827/3341360432] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:12.245 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:13:12.245 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f211807ac40 msgr2=0x7f211807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f211807ac40 0x7f211807d100 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2120005d00 tx=0x7f212000b410 comp rx=0 tx=0).stop 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130106800 msgr2=0x7f213019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130106800 0x7f213019dbb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f212402ae90 tx=0x7f212402f5f0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 shutdown_connections 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f211807ac40 0x7f211807d100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2130106800 0x7f213019dbb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 --2- 172.21.15.46:0/2621091451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2130100800 0x7f213019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 >> 172.21.15.46:0/2621091451 conn(0x7f21300fc520 msgr2=0x7f213006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:12.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 shutdown_connections 2024-09-10T04:13:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.242+0000 7f2135957640 1 -- 172.21.15.46:0/2621091451 wait complete. 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 -- 172.21.15.46:0/1444190258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 msgr2=0x7f9e48100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1444190258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e48100c50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009880 tx=0x7f9e3c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 -- 172.21.15.46:0/1444190258 shutdown_connections 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1444190258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e48100c50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1444190258 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e481067f0 0x7f9e48106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 -- 172.21.15.46:0/1444190258 >> 172.21.15.46:0/1444190258 conn(0x7f9e480fc550 msgr2=0x7f9e480fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 -- 172.21.15.46:0/1444190258 shutdown_connections 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 -- 172.21.15.46:0/1444190258 wait complete. 2024-09-10T04:13:12.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 Processor -- start 2024-09-10T04:13:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.328+0000 7f9e4dace640 1 -- start start 2024-09-10T04:13:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e4dace640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e4819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e4dace640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e481067f0 0x7f9e4819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e4dace640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4810dcc0 con 0x7f9e481007f0 2024-09-10T04:13:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e4dace640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4810db40 con 0x7f9e481067f0 2024-09-10T04:13:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e477fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e4819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e477fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e4819f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44304/0 (socket says 172.21.15.46:44304) 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e46ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e481067f0 0x7f9e4819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e477fe640 1 -- 172.21.15.46:0/1349787219 learned_addr learned my addr 172.21.15.46:0/1349787219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e477fe640 1 -- 172.21.15.46:0/1349787219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e481067f0 msgr2=0x7f9e4819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e477fe640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e481067f0 0x7f9e4819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e477fe640 1 -- 172.21.15.46:0/1349787219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e481999b0 con 0x7f9e481007f0 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e46ffd640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e481067f0 0x7f9e4819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.329+0000 7f9e477fe640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e4819f7f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e970 tx=0x7f9e3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.330+0000 7f9e44ff9640 1 -- 172.21.15.46:0/1349787219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9e3800cac0 con 0x7f9e481007f0 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.330+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e48199ca0 con 0x7f9e481007f0 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.330+0000 7f9e44ff9640 1 -- 172.21.15.46:0/1349787219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9e380054e0 con 0x7f9e481007f0 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.330+0000 7f9e44ff9640 1 -- 172.21.15.46:0/1349787219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9e38010430 con 0x7f9e481007f0 2024-09-10T04:13:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.330+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4819a200 con 0x7f9e481007f0 2024-09-10T04:13:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.331+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e48101f30 con 0x7f9e481007f0 2024-09-10T04:13:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.331+0000 7f9e44ff9640 1 -- 172.21.15.46:0/1349787219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9e38004e80 con 0x7f9e481007f0 2024-09-10T04:13:12.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.332+0000 7f9e44ff9640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9e3007ac00 0x7f9e3007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.332+0000 7f9e44ff9640 1 -- 172.21.15.46:0/1349787219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6070+0+0 (secure 0 0 0) 0x7f9e3809e260 con 0x7f9e481007f0 2024-09-10T04:13:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.332+0000 7f9e46ffd640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9e3007ac00 0x7f9e3007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.332+0000 7f9e46ffd640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9e3007ac00 0x7f9e3007d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e4819b1a0 tx=0x7f9e3c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:12.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[55103]: 2024-09-10T04:13:11.910+0000 7f61bdbb0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:13:12.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[55103]: 2024-09-10T04:13:11.910+0000 7f61bdbb0640 -1 osd.5 70 *** Got signal Terminated *** 2024-09-10T04:13:12.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:11 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[55103]: 2024-09-10T04:13:11.910+0000 7f61bdbb0640 -1 osd.5 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:13:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.336+0000 7f9e44ff9640 1 -- 172.21.15.46:0/1349787219 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9e38069a50 con 0x7f9e481007f0 2024-09-10T04:13:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.545+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9e4819a510 con 0x7f9e3007ac00 2024-09-10T04:13:12.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:12 smithi046 ceph-mon[109506]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:12.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:12 smithi046 ceph-mon[109506]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:12.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:12 smithi046 ceph-mon[109506]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:12.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:12 smithi046 ceph-mon[109506]: osd.5 marked itself down and dead 2024-09-10T04:13:12.548 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:12 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3996057322' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:12.548 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:12 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2621091451' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:13:12.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.546+0000 7f9e44ff9640 1 -- 172.21.15.46:0/1349787219 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+401 (secure 0 0 0) 0x7f9e4819a510 con 0x7f9e3007ac00 2024-09-10T04:13:12.548 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:13:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:13:12.548 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "11/23 daemons upgraded", 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:13:12.549 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:13:12.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.548+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9e3007ac00 msgr2=0x7f9e3007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.548+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9e3007ac00 0x7f9e3007d0c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9e4819b1a0 tx=0x7f9e3c031400 comp rx=0 tx=0).stop 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.548+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 msgr2=0x7f9e4819f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.548+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e4819f7f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800e970 tx=0x7f9e3800ed70 comp rx=0 tx=0).stop 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.549+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 shutdown_connections 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.549+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9e3007ac00 0x7f9e3007d0c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.549+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9e481067f0 0x7f9e4819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.549+0000 7f9e4dace640 1 --2- 172.21.15.46:0/1349787219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e481007f0 0x7f9e4819f7f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.549+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 >> 172.21.15.46:0/1349787219 conn(0x7f9e480fc550 msgr2=0x7f9e48069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.549+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 shutdown_connections 2024-09-10T04:13:12.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.549+0000 7f9e4dace640 1 -- 172.21.15.46:0/1349787219 wait complete. 2024-09-10T04:13:12.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 -- 172.21.15.46:0/2105034370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a301000c0 msgr2=0x7f2a301004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/2105034370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a301000c0 0x7f2a301004a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a18009880 tx=0x7f2a1802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 -- 172.21.15.46:0/2105034370 shutdown_connections 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/2105034370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a30100a70 0x7f2a30106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/2105034370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a301000c0 0x7f2a301004a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 -- 172.21.15.46:0/2105034370 >> 172.21.15.46:0/2105034370 conn(0x7f2a300fbc90 msgr2=0x7f2a300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 -- 172.21.15.46:0/2105034370 shutdown_connections 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.633+0000 7f2a36f0d640 1 -- 172.21.15.46:0/2105034370 wait complete. 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a36f0d640 1 Processor -- start 2024-09-10T04:13:12.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a36f0d640 1 -- start start 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a36f0d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a301000c0 0x7f2a3019d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a36f0d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a30100a70 0x7f2a3019dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a36f0d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3010ee80 con 0x7f2a30100a70 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a36f0d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3010ed00 con 0x7f2a301000c0 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a2ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a30100a70 0x7f2a3019dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a34ca9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a301000c0 0x7f2a3019d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a2ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a30100a70 0x7f2a3019dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44332/0 (socket says 172.21.15.46:44332) 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a2ffff640 1 -- 172.21.15.46:0/137031457 learned_addr learned my addr 172.21.15.46:0/137031457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a2ffff640 1 -- 172.21.15.46:0/137031457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a301000c0 msgr2=0x7f2a3019d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a2ffff640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a301000c0 0x7f2a3019d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.634+0000 7f2a2ffff640 1 -- 172.21.15.46:0/137031457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a30197950 con 0x7f2a30100a70 2024-09-10T04:13:12.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.635+0000 7f2a2ffff640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a30100a70 0x7f2a3019dcd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000eb40 tx=0x7f2a2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.635+0000 7f2a2dffb640 1 -- 172.21.15.46:0/137031457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a2000ccb0 con 0x7f2a30100a70 2024-09-10T04:13:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.635+0000 7f2a2dffb640 1 -- 172.21.15.46:0/137031457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a2000ce50 con 0x7f2a30100a70 2024-09-10T04:13:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.635+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a30197c40 con 0x7f2a30100a70 2024-09-10T04:13:12.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.635+0000 7f2a2dffb640 1 -- 172.21.15.46:0/137031457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a20010660 con 0x7f2a30100a70 2024-09-10T04:13:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.635+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a301981d0 con 0x7f2a30100a70 2024-09-10T04:13:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.636+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a30103180 con 0x7f2a30100a70 2024-09-10T04:13:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.637+0000 7f2a2dffb640 1 -- 172.21.15.46:0/137031457 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2a20005020 con 0x7f2a30100a70 2024-09-10T04:13:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.637+0000 7f2a2dffb640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a1c07abf0 0x7f2a1c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.637+0000 7f2a2dffb640 1 -- 172.21.15.46:0/137031457 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6070+0+0 (secure 0 0 0) 0x7f2a2009e550 con 0x7f2a30100a70 2024-09-10T04:13:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.637+0000 7f2a34ca9640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a1c07abf0 0x7f2a1c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:12.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.638+0000 7f2a34ca9640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a1c07abf0 0x7f2a1c07d0b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a18000c00 tx=0x7f2a18031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:12.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.642+0000 7f2a2dffb640 1 -- 172.21.15.46:0/137031457 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2a20069d40 con 0x7f2a30100a70 2024-09-10T04:13:12.775 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:12 smithi070 ceph-mon[79774]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:12.775 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:12 smithi070 ceph-mon[79774]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:12.775 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:12 smithi070 ceph-mon[79774]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:12.775 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:12 smithi070 ceph-mon[79774]: osd.5 marked itself down and dead 2024-09-10T04:13:12.776 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:12 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3996057322' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:12.776 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:12 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2621091451' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:13:12.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.901+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f2a30109ef0 con 0x7f2a30100a70 2024-09-10T04:13:12.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.901+0000 7f2a2dffb640 1 -- 172.21.15.46:0/137031457 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+290 (secure 0 0 0) 0x7f2a2006ebf0 con 0x7f2a30100a70 2024-09-10T04:13:12.903 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down 2024-09-10T04:13:12.903 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:13:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:13:12.903 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T04:13:12.903 INFO:teuthology.orchestra.run.smithi046.stdout: osd.5 (root=default,host=smithi070) is down 2024-09-10T04:13:12.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a1c07abf0 msgr2=0x7f2a1c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a1c07abf0 0x7f2a1c07d0b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2a18000c00 tx=0x7f2a18031400 comp rx=0 tx=0).stop 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a30100a70 msgr2=0x7f2a3019dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a30100a70 0x7f2a3019dcd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000eb40 tx=0x7f2a2000c2d0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 shutdown_connections 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a1c07abf0 0x7f2a1c07d0b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a30100a70 0x7f2a3019dcd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 --2- 172.21.15.46:0/137031457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2a301000c0 0x7f2a3019d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 >> 172.21.15.46:0/137031457 conn(0x7f2a300fbc90 msgr2=0x7f2a300fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 shutdown_connections 2024-09-10T04:13:12.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:12.903+0000 7f2a36f0d640 1 -- 172.21.15.46:0/137031457 wait complete. 2024-09-10T04:13:13.054 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:12 smithi070 podman[96636]: 2024-09-10 04:13:12.774557067 +0000 UTC m=+0.984679095 container died 5acf243605d5b8836a4a14dab862ad2a33e5c18ce21bee5bd7ea9d68aaa1ab90 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T04:13:13.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:13 smithi070 podman[96636]: 2024-09-10 04:13:13.144083941 +0000 UTC m=+1.354205959 container cleanup 5acf243605d5b8836a4a14dab862ad2a33e5c18ce21bee5bd7ea9d68aaa1ab90 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:13:13.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:13 smithi070 bash[96636]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5 2024-09-10T04:13:13.804 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:13 smithi070 podman[96662]: 2024-09-10 04:13:13.457213722 +0000 UTC m=+0.676281845 container remove 5acf243605d5b8836a4a14dab862ad2a33e5c18ce21bee5bd7ea9d68aaa1ab90 (image=quay.ceph.io/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T04:13:13.804 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:13 smithi070 ceph-mon[79774]: pgmap v218: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:13:13.804 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:13 smithi070 ceph-mon[79774]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:13:13.804 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:13 smithi070 ceph-mon[79774]: osdmap e71: 6 total, 5 up, 6 in 2024-09-10T04:13:13.804 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:13 smithi070 ceph-mon[79774]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:13.804 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:13 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/137031457' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:13:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:13 smithi046 ceph-mon[109506]: pgmap v218: 65 pgs: 65 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:13:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:13 smithi046 ceph-mon[109506]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T04:13:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:13 smithi046 ceph-mon[109506]: osdmap e71: 6 total, 5 up, 6 in 2024-09-10T04:13:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:13 smithi046 ceph-mon[109506]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:13 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/137031457' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:13:14.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:13 smithi070 podman[96752]: 2024-09-10 04:13:13.705990913 +0000 UTC m=+0.015947033 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:13:14.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:13 smithi070 podman[96752]: 2024-09-10 04:13:13.829881702 +0000 UTC m=+0.139837868 container create 32c63b73ec688230d1541fb41961d3564268d589e4a0772e49cf2034b0d9acb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T04:13:14.447 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 podman[96752]: 2024-09-10 04:13:14.107585206 +0000 UTC m=+0.417541327 container init 32c63b73ec688230d1541fb41961d3564268d589e4a0772e49cf2034b0d9acb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-10T04:13:14.447 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 podman[96752]: 2024-09-10 04:13:14.11110839 +0000 UTC m=+0.421064511 container start 32c63b73ec688230d1541fb41961d3564268d589e4a0772e49cf2034b0d9acb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T04:13:14.447 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 podman[96752]: 2024-09-10 04:13:14.190738561 +0000 UTC m=+0.500694692 container attach 32c63b73ec688230d1541fb41961d3564268d589e4a0772e49cf2034b0d9acb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T04:13:14.447 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 podman[96752]: 2024-09-10 04:13:14.296344728 +0000 UTC m=+0.606300861 container died 32c63b73ec688230d1541fb41961d3564268d589e4a0772e49cf2034b0d9acb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T04:13:14.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:14 smithi046 ceph-mon[109506]: osdmap e72: 6 total, 5 up, 6 in 2024-09-10T04:13:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:14 smithi070 ceph-mon[79774]: osdmap e72: 6 total, 5 up, 6 in 2024-09-10T04:13:15.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 podman[96776]: 2024-09-10 04:13:14.928090779 +0000 UTC m=+0.625739643 container remove 32c63b73ec688230d1541fb41961d3564268d589e4a0772e49cf2034b0d9acb1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True) 2024-09-10T04:13:15.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-10T04:13:15.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 systemd[1]: Stopped Ceph osd.5 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:13:15.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:14 smithi070 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.5.service: Consumed 52.337s CPU time. 2024-09-10T04:13:15.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 systemd[1]: Starting Ceph osd.5 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:13:15.616 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:15 smithi070 ceph-mon[79774]: pgmap v221: 65 pgs: 7 peering, 9 stale+active+clean, 49 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T04:13:15.616 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:15 smithi070 ceph-mon[79774]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-09-10T04:13:15.617 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 podman[96874]: 2024-09-10 04:13:15.265458223 +0000 UTC m=+0.016122668 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:13:15.617 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 podman[96874]: 2024-09-10 04:13:15.379172212 +0000 UTC m=+0.129836657 container create 2fd17dd8371705f6b8c6889696cd9c48fcedfc77964d200ca850a92285293bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T04:13:15.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:15 smithi046 ceph-mon[109506]: pgmap v221: 65 pgs: 7 peering, 9 stale+active+clean, 49 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T04:13:15.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:15 smithi046 ceph-mon[109506]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-09-10T04:13:16.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 podman[96874]: 2024-09-10 04:13:15.615637003 +0000 UTC m=+0.366301465 container init 2fd17dd8371705f6b8c6889696cd9c48fcedfc77964d200ca850a92285293bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T04:13:16.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 podman[96874]: 2024-09-10 04:13:15.619685746 +0000 UTC m=+0.370350193 container start 2fd17dd8371705f6b8c6889696cd9c48fcedfc77964d200ca850a92285293bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T04:13:16.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 podman[96874]: 2024-09-10 04:13:15.698344517 +0000 UTC m=+0.449008972 container attach 2fd17dd8371705f6b8c6889696cd9c48fcedfc77964d200ca850a92285293bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T04:13:16.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:13:16.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 bash[96874]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:13:16.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:13:16.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:15 smithi070 bash[96874]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T04:13:16.779 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-mon[79774]: Health check failed: Degraded data redundancy: 40/312 objects degraded (12.821%), 11 pgs degraded (PG_DEGRADED) 2024-09-10T04:13:16.779 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T04:13:16.779 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 bash[96874]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T04:13:16.780 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T04:13:16.780 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 bash[96874]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T04:13:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:16 smithi046 ceph-mon[109506]: Health check failed: Degraded data redundancy: 40/312 objects degraded (12.821%), 11 pgs degraded (PG_DEGRADED) 2024-09-10T04:13:17.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T04:13:17.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 bash[96874]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T04:13:17.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T04:13:17.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 bash[96874]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T04:13:17.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-10T04:13:17.085 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 bash[96874]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-10T04:13:17.086 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T04:13:17.086 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 bash[96874]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T04:13:17.086 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate[96888]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-10T04:13:17.086 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 bash[96874]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-10T04:13:17.086 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:16 smithi070 podman[96874]: 2024-09-10 04:13:16.807384933 +0000 UTC m=+1.558049381 container died 2fd17dd8371705f6b8c6889696cd9c48fcedfc77964d200ca850a92285293bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:13:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:17 smithi046 ceph-mon[109506]: pgmap v222: 65 pgs: 10 active+undersized, 7 peering, 11 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 40/312 objects degraded (12.821%) 2024-09-10T04:13:17.645 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:17 smithi070 ceph-mon[79774]: pgmap v222: 65 pgs: 10 active+undersized, 7 peering, 11 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 40/312 objects degraded (12.821%) 2024-09-10T04:13:17.646 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:17 smithi070 podman[97035]: 2024-09-10 04:13:17.37719342 +0000 UTC m=+0.563416340 container remove 2fd17dd8371705f6b8c6889696cd9c48fcedfc77964d200ca850a92285293bfb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T04:13:17.646 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:17 smithi070 podman[97091]: 2024-09-10 04:13:17.494573929 +0000 UTC m=+0.015860933 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:13:17.646 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:17 smithi070 podman[97091]: 2024-09-10 04:13:17.644813999 +0000 UTC m=+0.166100994 container create 79aa94488c3a319444330beaa647efd701421d655d9679709f6d305ef38ec020 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:13:18.209 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:17 smithi070 podman[97091]: 2024-09-10 04:13:17.916007071 +0000 UTC m=+0.437294067 container init 79aa94488c3a319444330beaa647efd701421d655d9679709f6d305ef38ec020 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T04:13:18.209 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:17 smithi070 podman[97091]: 2024-09-10 04:13:17.919419031 +0000 UTC m=+0.440706030 container start 79aa94488c3a319444330beaa647efd701421d655d9679709f6d305ef38ec020 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T04:13:18.209 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:17 smithi070 ceph-osd[97110]: -- 172.21.15.70:0/3724933939 <== mon.1 v2:172.21.15.70:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d161bc61e0 con 0x55d160d65c00 2024-09-10T04:13:18.209 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:18 smithi070 bash[97091]: 79aa94488c3a319444330beaa647efd701421d655d9679709f6d305ef38ec020 2024-09-10T04:13:18.209 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:18 smithi070 systemd[1]: Started Ceph osd.5 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:13:19.363 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:19.364 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:19 smithi070 ceph-mon[79774]: pgmap v223: 65 pgs: 10 active+undersized, 7 peering, 11 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 40/312 objects degraded (12.821%) 2024-09-10T04:13:19.364 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:19.364 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:19 smithi046 ceph-mon[109506]: pgmap v223: 65 pgs: 10 active+undersized, 7 peering, 11 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 40/312 objects degraded (12.821%) 2024-09-10T04:13:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:19.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:19.621 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:19 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:13:19.554+0000 7fa72ae1f640 -1 Falling back to public interface 2024-09-10T04:13:20.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:20 smithi070 ceph-mon[79774]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-09-10T04:13:20.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:20 smithi046 ceph-mon[109506]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-09-10T04:13:21.556 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:21 smithi046 ceph-mon[109506]: pgmap v224: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 49/312 objects degraded (15.705%) 2024-09-10T04:13:21.557 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:21 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:21.557 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:21 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:21.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:21 smithi070 ceph-mon[79774]: pgmap v224: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 49/312 objects degraded (15.705%) 2024-09-10T04:13:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:21 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:21 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:22.774 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:22 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:22.775 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:22 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:22 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:22.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:22 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:23 smithi046 ceph-mon[109506]: pgmap v225: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 49/312 objects degraded (15.705%) 2024-09-10T04:13:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:23 smithi070 ceph-mon[79774]: pgmap v225: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 49/312 objects degraded (15.705%) 2024-09-10T04:13:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:24.574 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:24 smithi070 ceph-mon[79774]: from='osd.5 [v2:172.21.15.70:6816/3911965099,v1:172.21.15.70:6817/3911965099]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T04:13:24.575 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:24 smithi070 ceph-mon[79774]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T04:13:24.575 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:24 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:13:24.324+0000 7fa72ae1f640 -1 osd.5 70 log_to_monitors true 2024-09-10T04:13:24.575 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:13:24 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:13:24.423+0000 7fa71ebc1640 -1 osd.5 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:13:24.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:24 smithi046 ceph-mon[109506]: from='osd.5 [v2:172.21.15.70:6816/3911965099,v1:172.21.15.70:6817/3911965099]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T04:13:24.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:24 smithi046 ceph-mon[109506]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T04:13:25.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: pgmap v226: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 49/312 objects degraded (15.705%) 2024-09-10T04:13:25.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T04:13:25.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='osd.5 [v2:172.21.15.70:6816/3911965099,v1:172.21.15.70:6817/3911965099]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: osdmap e73: 6 total, 5 up, 6 in 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.781 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.782 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"} : dispatch 2024-09-10T04:13:25.783 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:25 smithi070 ceph-mon[79774]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:13:25.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: pgmap v226: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 245 MiB data, 983 MiB used, 535 GiB / 536 GiB avail; 49/312 objects degraded (15.705%) 2024-09-10T04:13:25.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T04:13:25.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='osd.5 [v2:172.21.15.70:6816/3911965099,v1:172.21.15.70:6817/3911965099]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:13:25.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: osdmap e73: 6 total, 5 up, 6 in 2024-09-10T04:13:25.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-10T04:13:25.792 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:25.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:25.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"} : dispatch 2024-09-10T04:13:25.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-mon[109506]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T04:13:26.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:25 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:13:25.852+0000 7f3ca7014640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T04:13:26.545 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all osd 2024-09-10T04:13:26.545 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-10T04:13:26.545 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: osd.5 [v2:172.21.15.70:6816/3911965099,v1:172.21.15.70:6817/3911965099] boot 2024-09-10T04:13:26.545 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: osdmap e74: 6 total, 6 up, 6 in 2024-09-10T04:13:26.545 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 49/312 objects degraded (15.705%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.546 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: osdmap e75: 6 total, 6 up, 6 in 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] up:boot 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6828/184543618,v1:172.21.15.46:6829/184543618] up:boot 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"} : dispatch 2024-09-10T04:13:26.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"} : dispatch 2024-09-10T04:13:26.840 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all osd 2024-09-10T04:13:26.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-10T04:13:26.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: osd.5 [v2:172.21.15.70:6816/3911965099,v1:172.21.15.70:6817/3911965099] boot 2024-09-10T04:13:26.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: osdmap e74: 6 total, 6 up, 6 in 2024-09-10T04:13:26.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T04:13:26.891 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi070.ehvjgr=up:active} 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 49/312 objects degraded (15.705%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T04:13:26.892 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: osdmap e75: 6 total, 6 up, 6 in 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] up:boot 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6828/184543618,v1:172.21.15.46:6829/184543618] up:boot 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"} : dispatch 2024-09-10T04:13:26.893 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"} : dispatch 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: pgmap v230: 65 pgs: 2 peering, 13 active+undersized, 13 active+undersized+degraded, 37 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 5 op/s; 46/312 objects degraded (14.744%) 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.70:6824/2786504298,v1:172.21.15.70:6825/2786504298] up:boot 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6826/3836504153,v1:172.21.15.46:6827/3836504153] up:boot 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"} : dispatch 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.hasdap", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:13:27.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:27 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: pgmap v230: 65 pgs: 2 peering, 13 active+undersized, 13 active+undersized+degraded, 37 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 5 op/s; 46/312 objects degraded (14.744%) 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.70:6824/2786504298,v1:172.21.15.70:6825/2786504298] up:boot 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6826/3836504153,v1:172.21.15.46:6827/3836504153] up:boot 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"} : dispatch 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:27.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.hasdap", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:13:27.831 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:27 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:28.673 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:28 smithi046 ceph-mon[109506]: Upgrade: Updating mds.cephfs.smithi046.hasdap 2024-09-10T04:13:28.673 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:28 smithi046 ceph-mon[109506]: Deploying daemon mds.cephfs.smithi046.hasdap on smithi046 2024-09-10T04:13:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:28 smithi070 ceph-mon[79774]: Upgrade: Updating mds.cephfs.smithi046.hasdap 2024-09-10T04:13:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:28 smithi070 ceph-mon[79774]: Deploying daemon mds.cephfs.smithi046.hasdap on smithi046 2024-09-10T04:13:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:29 smithi046 ceph-mon[109506]: pgmap v231: 65 pgs: 2 peering, 13 active+undersized, 13 active+undersized+degraded, 37 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 46/312 objects degraded (14.744%) 2024-09-10T04:13:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:29 smithi070 ceph-mon[79774]: pgmap v231: 65 pgs: 2 peering, 13 active+undersized, 13 active+undersized+degraded, 37 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 46/312 objects degraded (14.744%) 2024-09-10T04:13:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:30 smithi070 ceph-mon[79774]: osdmap e76: 6 total, 6 up, 6 in 2024-09-10T04:13:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:30 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:13:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:30 smithi046 ceph-mon[109506]: osdmap e76: 6 total, 6 up, 6 in 2024-09-10T04:13:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:30 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:13:31.727 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:31 smithi046 ceph-mon[109506]: pgmap v233: 65 pgs: 2 peering, 2 active+undersized, 5 active+undersized+degraded, 56 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 22/312 objects degraded (7.051%) 2024-09-10T04:13:31.727 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:31 smithi046 ceph-mon[109506]: Health check update: Degraded data redundancy: 22/312 objects degraded (7.051%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T04:13:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:31 smithi070 ceph-mon[79774]: pgmap v233: 65 pgs: 2 peering, 2 active+undersized, 5 active+undersized+degraded, 56 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 22/312 objects degraded (7.051%) 2024-09-10T04:13:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:31 smithi070 ceph-mon[79774]: Health check update: Degraded data redundancy: 22/312 objects degraded (7.051%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T04:13:32.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:32 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:13:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:32 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:13:33.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:33 smithi046 ceph-mon[109506]: pgmap v234: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:33.547 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:33 smithi046 ceph-mon[109506]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/312 objects degraded (7.051%), 5 pgs degraded) 2024-09-10T04:13:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:33 smithi070 ceph-mon[79774]: pgmap v234: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:33 smithi070 ceph-mon[79774]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/312 objects degraded (7.051%), 5 pgs degraded) 2024-09-10T04:13:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:35 smithi046 ceph-mon[109506]: pgmap v235: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:35 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:35 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:35 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:35 smithi070 ceph-mon[79774]: pgmap v235: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:35 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:35 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:35 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:36.805 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:36 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:boot 2024-09-10T04:13:36.805 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:36 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:13:36.805 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:36 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:13:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:36 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:boot 2024-09-10T04:13:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:36 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:13:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:36 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:13:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:37 smithi070 ceph-mon[79774]: pgmap v236: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:37 smithi046 ceph-mon[109506]: pgmap v236: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:39.286 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:39.286 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:39.286 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:39.286 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:39 smithi046 ceph-mon[109506]: pgmap v237: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:39 smithi070 ceph-mon[79774]: pgmap v237: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:41.283 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:41.283 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:41.284 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:40 smithi046 ceph-mon[109506]: pgmap v238: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:41.284 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:41.284 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:40 smithi070 ceph-mon[79774]: pgmap v238: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:41.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 -- 172.21.15.46:0/943120438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608102b30 msgr2=0x7fc608102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 --2- 172.21.15.46:0/943120438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608102b30 0x7fc608102f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009880 tx=0x7fc5f802a530 comp rx=0 tx=0).stop 2024-09-10T04:13:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 -- 172.21.15.46:0/943120438 shutdown_connections 2024-09-10T04:13:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 --2- 172.21.15.46:0/943120438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608103450 0x7fc60810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 --2- 172.21.15.46:0/943120438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608102b30 0x7fc608102f10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 -- 172.21.15.46:0/943120438 >> 172.21.15.46:0/943120438 conn(0x7fc6080fc550 msgr2=0x7fc6080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 -- 172.21.15.46:0/943120438 shutdown_connections 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 -- 172.21.15.46:0/943120438 wait complete. 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.994+0000 7fc60dc54640 1 Processor -- start 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc60dc54640 1 -- start start 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc60dc54640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608102b30 0x7fc608100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc60dc54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608103450 0x7fc608100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc60dc54640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc60810ede0 con 0x7fc608103450 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc60dc54640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc60810ec60 con 0x7fc608102b30 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc606ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608103450 0x7fc608100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc606ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608103450 0x7fc608100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37938/0 (socket says 172.21.15.46:37938) 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc606ffd640 1 -- 172.21.15.46:0/2339964819 learned_addr learned my addr 172.21.15.46:0/2339964819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.995+0000 7fc6077fe640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608102b30 0x7fc608100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:42.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc606ffd640 1 -- 172.21.15.46:0/2339964819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608102b30 msgr2=0x7fc608100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc606ffd640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608102b30 0x7fc608100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc606ffd640 1 -- 172.21.15.46:0/2339964819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc608101210 con 0x7fc608103450 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc606ffd640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608103450 0x7fc608100cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00d6f0 tx=0x7fc5fc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc604ff9640 1 -- 172.21.15.46:0/2339964819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc017070 con 0x7fc608103450 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc604ff9640 1 -- 172.21.15.46:0/2339964819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5fc00de00 con 0x7fc608103450 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc608101500 con 0x7fc608103450 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc604ff9640 1 -- 172.21.15.46:0/2339964819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5fc004bb0 con 0x7fc608103450 2024-09-10T04:13:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.996+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6081a3e50 con 0x7fc608103450 2024-09-10T04:13:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.997+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60806a500 con 0x7fc608103450 2024-09-10T04:13:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.998+0000 7fc604ff9640 1 -- 172.21.15.46:0/2339964819 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc5fc009a30 con 0x7fc608103450 2024-09-10T04:13:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.998+0000 7fc604ff9640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5f007ac00 0x7fc5f007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.999+0000 7fc604ff9640 1 -- 172.21.15.46:0/2339964819 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6242+0+0 (secure 0 0 0) 0x7fc5fc09d830 con 0x7fc608103450 2024-09-10T04:13:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.999+0000 7fc6077fe640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5f007ac00 0x7fc5f007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:42.999+0000 7fc6077fe640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5f007ac00 0x7fc5f007d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80096c0 tx=0x7fc5f8004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:43.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.003+0000 7fc604ff9640 1 -- 172.21.15.46:0/2339964819 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc5fc068f70 con 0x7fc608103450 2024-09-10T04:13:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.224+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc608062630 con 0x7fc5f007ac00 2024-09-10T04:13:43.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.225+0000 7fc604ff9640 1 -- 172.21.15.46:0/2339964819 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fc608062630 con 0x7fc5f007ac00 2024-09-10T04:13:43.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5f007ac00 msgr2=0x7fc5f007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5f007ac00 0x7fc5f007d0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5f80096c0 tx=0x7fc5f8004f30 comp rx=0 tx=0).stop 2024-09-10T04:13:43.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608103450 msgr2=0x7fc608100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608103450 0x7fc608100cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00d6f0 tx=0x7fc5fc00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 shutdown_connections 2024-09-10T04:13:43.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5f007ac00 0x7fc5f007d0c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc608103450 0x7fc608100cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 --2- 172.21.15.46:0/2339964819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc608102b30 0x7fc608100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 >> 172.21.15.46:0/2339964819 conn(0x7fc6080fc550 msgr2=0x7fc608108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:43.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 shutdown_connections 2024-09-10T04:13:43.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.227+0000 7fc60dc54640 1 -- 172.21.15.46:0/2339964819 wait complete. 2024-09-10T04:13:43.237 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:13:43.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 -- 172.21.15.46:0/115518498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc98102ac0 msgr2=0x7fcc98102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/115518498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc98102ac0 0x7fcc98102ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009880 tx=0x7fcc8802a530 comp rx=0 tx=0).stop 2024-09-10T04:13:43.315 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:43 smithi046 ceph-mon[109506]: pgmap v239: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:43.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 -- 172.21.15.46:0/115518498 shutdown_connections 2024-09-10T04:13:43.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/115518498 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc981033e0 0x7fcc9810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/115518498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc98102ac0 0x7fcc98102ea0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 -- 172.21.15.46:0/115518498 >> 172.21.15.46:0/115518498 conn(0x7fcc980fc520 msgr2=0x7fcc980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 -- 172.21.15.46:0/115518498 shutdown_connections 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.314+0000 7fcc9cf39640 1 -- 172.21.15.46:0/115518498 wait complete. 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc9cf39640 1 Processor -- start 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc9cf39640 1 -- start start 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc9cf39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc98102ac0 0x7fcc9819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc9cf39640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc981033e0 0x7fcc9819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc9cf39640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9810edf0 con 0x7fcc981033e0 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc9cf39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9810ec70 con 0x7fcc98102ac0 2024-09-10T04:13:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc95d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc981033e0 0x7fcc9819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.315+0000 7fcc95d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc981033e0 0x7fcc9819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37954/0 (socket says 172.21.15.46:37954) 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc95d9b640 1 -- 172.21.15.46:0/740287178 learned_addr learned my addr 172.21.15.46:0/740287178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc9659c640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc98102ac0 0x7fcc9819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc95d9b640 1 -- 172.21.15.46:0/740287178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc98102ac0 msgr2=0x7fcc9819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc95d9b640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc98102ac0 0x7fcc9819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc95d9b640 1 -- 172.21.15.46:0/740287178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc981977c0 con 0x7fcc981033e0 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc9659c640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc98102ac0 0x7fcc9819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc95d9b640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc981033e0 0x7fcc9819db80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d6d0 tx=0x7fcc8c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc877fe640 1 -- 172.21.15.46:0/740287178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcc8c017070 con 0x7fcc981033e0 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc877fe640 1 -- 172.21.15.46:0/740287178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcc8c0054e0 con 0x7fcc981033e0 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc98197ab0 con 0x7fcc981033e0 2024-09-10T04:13:43.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.316+0000 7fcc877fe640 1 -- 172.21.15.46:0/740287178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcc8c004ae0 con 0x7fcc981033e0 2024-09-10T04:13:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.317+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc98198040 con 0x7fcc981033e0 2024-09-10T04:13:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.318+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc9806aa90 con 0x7fcc981033e0 2024-09-10T04:13:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.318+0000 7fcc877fe640 1 -- 172.21.15.46:0/740287178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcc8c004de0 con 0x7fcc981033e0 2024-09-10T04:13:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.319+0000 7fcc877fe640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcc8007abf0 0x7fcc8007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.319+0000 7fcc877fe640 1 -- 172.21.15.46:0/740287178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6242+0+0 (secure 0 0 0) 0x7fcc8c09d870 con 0x7fcc981033e0 2024-09-10T04:13:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.319+0000 7fcc9659c640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcc8007abf0 0x7fcc8007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.319+0000 7fcc9659c640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcc8007abf0 0x7fcc8007d0b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc88000c00 tx=0x7fcc88031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:43.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.323+0000 7fcc877fe640 1 -- 172.21.15.46:0/740287178 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fcc8c068fb0 con 0x7fcc981033e0 2024-09-10T04:13:43.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.531+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fcc981982f0 con 0x7fcc8007abf0 2024-09-10T04:13:43.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.532+0000 7fcc877fe640 1 -- 172.21.15.46:0/740287178 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fcc981982f0 con 0x7fcc8007abf0 2024-09-10T04:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcc8007abf0 msgr2=0x7fcc8007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcc8007abf0 0x7fcc8007d0b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcc88000c00 tx=0x7fcc88031400 comp rx=0 tx=0).stop 2024-09-10T04:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc981033e0 msgr2=0x7fcc9819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc981033e0 0x7fcc9819db80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00d6d0 tx=0x7fcc8c00db90 comp rx=0 tx=0).stop 2024-09-10T04:13:43.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 shutdown_connections 2024-09-10T04:13:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcc8007abf0 0x7fcc8007d0b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc981033e0 0x7fcc9819db80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 --2- 172.21.15.46:0/740287178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc98102ac0 0x7fcc9819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 >> 172.21.15.46:0/740287178 conn(0x7fcc980fc520 msgr2=0x7fcc9810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 shutdown_connections 2024-09-10T04:13:43.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.534+0000 7fcc9cf39640 1 -- 172.21.15.46:0/740287178 wait complete. 2024-09-10T04:13:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:43 smithi070 ceph-mon[79774]: pgmap v239: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:43.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.618+0000 7f629caed640 1 -- 172.21.15.46:0/4195082983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298106800 msgr2=0x7f6298106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.618+0000 7f629caed640 1 --2- 172.21.15.46:0/4195082983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298106800 0x7f6298106be0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6288009880 tx=0x7f628802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 -- 172.21.15.46:0/4195082983 shutdown_connections 2024-09-10T04:13:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 --2- 172.21.15.46:0/4195082983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6298100800 0x7f6298100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 --2- 172.21.15.46:0/4195082983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298106800 0x7f6298106be0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 -- 172.21.15.46:0/4195082983 >> 172.21.15.46:0/4195082983 conn(0x7f62980fc520 msgr2=0x7f62980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 -- 172.21.15.46:0/4195082983 shutdown_connections 2024-09-10T04:13:43.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 -- 172.21.15.46:0/4195082983 wait complete. 2024-09-10T04:13:43.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 Processor -- start 2024-09-10T04:13:43.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.619+0000 7f629caed640 1 -- start start 2024-09-10T04:13:43.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629caed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298100800 0x7f62981987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629caed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6298106800 0x7f6298198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629caed640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f629810dcd0 con 0x7f6298100800 2024-09-10T04:13:43.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629caed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f629810db50 con 0x7f6298106800 2024-09-10T04:13:43.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298100800 0x7f62981987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298100800 0x7f62981987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37980/0 (socket says 172.21.15.46:37980) 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629659c640 1 -- 172.21.15.46:0/2012948592 learned_addr learned my addr 172.21.15.46:0/2012948592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f6295d9b640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6298106800 0x7f6298198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629659c640 1 -- 172.21.15.46:0/2012948592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6298106800 msgr2=0x7f6298198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629659c640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6298106800 0x7f6298198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629659c640 1 -- 172.21.15.46:0/2012948592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6298199360 con 0x7f6298100800 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.620+0000 7f629659c640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298100800 0x7f62981987d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6288031460 tx=0x7f6288004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.621+0000 7f62877fe640 1 -- 172.21.15.46:0/2012948592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f628803f070 con 0x7f6298100800 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.621+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62981995f0 con 0x7f6298100800 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.621+0000 7f62877fe640 1 -- 172.21.15.46:0/2012948592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6288004a90 con 0x7f6298100800 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.621+0000 7f62877fe640 1 -- 172.21.15.46:0/2012948592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6288035030 con 0x7f6298100800 2024-09-10T04:13:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.621+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62981a1ba0 con 0x7f6298100800 2024-09-10T04:13:43.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.622+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6298101f40 con 0x7f6298100800 2024-09-10T04:13:43.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.622+0000 7f62877fe640 1 -- 172.21.15.46:0/2012948592 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6288042070 con 0x7f6298100800 2024-09-10T04:13:43.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.623+0000 7f62877fe640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f628007ac00 0x7f628007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.623+0000 7f6295d9b640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f628007ac00 0x7f628007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.623+0000 7f62877fe640 1 -- 172.21.15.46:0/2012948592 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6242+0+0 (secure 0 0 0) 0x7f62880be6e0 con 0x7f6298100800 2024-09-10T04:13:43.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.624+0000 7f6295d9b640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f628007ac00 0x7f628007d0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f629806ad40 tx=0x7f628c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:43.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.628+0000 7f62877fe640 1 -- 172.21.15.46:0/2012948592 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6288089e20 con 0x7f6298100800 2024-09-10T04:13:43.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.825+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6298062630 con 0x7f628007ac00 2024-09-10T04:13:43.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.836+0000 7f62877fe640 1 -- 172.21.15.46:0/2012948592 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f6298062630 con 0x7f628007ac00 2024-09-10T04:13:43.836 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (5m) 3s ago 16m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (7m) 3s ago 16m 9177k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (7m) 23s ago 15m 9021k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (3m) 3s ago 16m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (3m) 23s ago 15m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (4m) 3s ago 16m 77.3M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (11m) 3s ago 11m 12.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 a853d297aecf 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (9s) 3s ago 11m 11.6M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (11m) 23s ago 11m 26.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 ad63a9851aed 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (11m) 23s ago 11m 25.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (7m) 3s ago 17m 530M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:13:43.837 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (6m) 23s ago 15m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (4m) 3s ago 18m 50.2M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (3m) 23s ago 15m 38.9M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (6m) 3s ago 16m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 23s ago 15m 12.3M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (3m) 3s ago 14m 86.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (2m) 3s ago 14m 90.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (2m) 3s ago 13m 83.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (91s) 23s ago 13m 91.0M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (58s) 23s ago 12m 86.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (26s) 23s ago 12m 12.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:13:43.838 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (5m) 3s ago 16m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:13:43.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f628007ac00 msgr2=0x7f628007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f628007ac00 0x7f628007d0c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f629806ad40 tx=0x7f628c005f90 comp rx=0 tx=0).stop 2024-09-10T04:13:43.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298100800 msgr2=0x7f62981987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298100800 0x7f62981987d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6288031460 tx=0x7f6288004530 comp rx=0 tx=0).stop 2024-09-10T04:13:43.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 shutdown_connections 2024-09-10T04:13:43.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f628007ac00 0x7f628007d0c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6298106800 0x7f6298198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 --2- 172.21.15.46:0/2012948592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6298100800 0x7f62981987d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 >> 172.21.15.46:0/2012948592 conn(0x7f62980fc520 msgr2=0x7f62980fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:43.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 shutdown_connections 2024-09-10T04:13:43.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.838+0000 7f629caed640 1 -- 172.21.15.46:0/2012948592 wait complete. 2024-09-10T04:13:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.924+0000 7fb1aaaae640 1 -- 172.21.15.46:0/1695378819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4102b10 msgr2=0x7fb1a4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.924+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/1695378819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4102b10 0x7fb1a4102ef0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb1a00098b0 tx=0x7fb1a002a560 comp rx=0 tx=0).stop 2024-09-10T04:13:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.924+0000 7fb1aaaae640 1 -- 172.21.15.46:0/1695378819 shutdown_connections 2024-09-10T04:13:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.924+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/1695378819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4103430 0x7fb1a410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.924+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/1695378819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4102b10 0x7fb1a4102ef0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.924+0000 7fb1aaaae640 1 -- 172.21.15.46:0/1695378819 >> 172.21.15.46:0/1695378819 conn(0x7fb1a40fc550 msgr2=0x7fb1a40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.925+0000 7fb1aaaae640 1 -- 172.21.15.46:0/1695378819 shutdown_connections 2024-09-10T04:13:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.925+0000 7fb1aaaae640 1 -- 172.21.15.46:0/1695378819 wait complete. 2024-09-10T04:13:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.925+0000 7fb1aaaae640 1 Processor -- start 2024-09-10T04:13:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.925+0000 7fb1aaaae640 1 -- start start 2024-09-10T04:13:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1aaaae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4102b10 0x7fb1a419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1aaaae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103430 0x7fb1a419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1aaaae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410edc0 con 0x7fb1a4103430 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1aaaae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a410ec40 con 0x7fb1a4102b10 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1a884a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4102b10 0x7fb1a419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1a884a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4102b10 0x7fb1a419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:56658/0 (socket says 172.21.15.46:56658) 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103430 0x7fb1a419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1a884a640 1 -- 172.21.15.46:0/3138485895 learned_addr learned my addr 172.21.15.46:0/3138485895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1a884a640 1 -- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103430 msgr2=0x7fb1a419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1a884a640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103430 0x7fb1a419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.926+0000 7fb1a884a640 1 -- 172.21.15.46:0/3138485895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41a0410 con 0x7fb1a4102b10 2024-09-10T04:13:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.927+0000 7fb1a884a640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4102b10 0x7fb1a419f820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009230 tx=0x7fb1a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.927+0000 7fb199ffb640 1 -- 172.21.15.46:0/3138485895 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb1a003f070 con 0x7fb1a4102b10 2024-09-10T04:13:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.927+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a4199aa0 con 0x7fb1a4102b10 2024-09-10T04:13:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.927+0000 7fb199ffb640 1 -- 172.21.15.46:0/3138485895 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb1a0004ac0 con 0x7fb1a4102b10 2024-09-10T04:13:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.927+0000 7fb199ffb640 1 -- 172.21.15.46:0/3138485895 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb1a0035030 con 0x7fb1a4102b10 2024-09-10T04:13:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.927+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a4199fa0 con 0x7fb1a4102b10 2024-09-10T04:13:43.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.928+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a406a520 con 0x7fb1a4102b10 2024-09-10T04:13:43.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.929+0000 7fb199ffb640 1 -- 172.21.15.46:0/3138485895 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb1a0042070 con 0x7fb1a4102b10 2024-09-10T04:13:43.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.930+0000 7fb199ffb640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18c07ac00 0x7fb18c07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:43.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.930+0000 7fb199ffb640 1 -- 172.21.15.46:0/3138485895 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6242+0+0 (secure 0 0 0) 0x7fb1a00be840 con 0x7fb1a4102b10 2024-09-10T04:13:43.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.930+0000 7fb19bfff640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18c07ac00 0x7fb18c07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.930+0000 7fb19bfff640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18c07ac00 0x7fb18c07d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb194005e10 tx=0x7fb194005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:43.935+0000 7fb199ffb640 1 -- 172.21.15.46:0/3138485895 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb1a0089f80 con 0x7fb1a4102b10 2024-09-10T04:13:44.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.205+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fb1a419af10 con 0x7fb1a4102b10 2024-09-10T04:13:44.207 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:44.207 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:44.207 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:44.207 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:44.207 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:44.207 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:13:44.207 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:44.208 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:44.208 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.208 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.208 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.208 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.206+0000 7fb199ffb640 1 -- 172.21.15.46:0/3138485895 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fb1a008ee30 con 0x7fb1a4102b10 2024-09-10T04:13:44.208 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:13:44.208 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 3, 2024-09-10T04:13:44.209 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 1 2024-09-10T04:13:44.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:13:44.210 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:13:44.210 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 3, 2024-09-10T04:13:44.210 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 11 2024-09-10T04:13:44.210 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:13:44.210 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:13:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18c07ac00 msgr2=0x7fb18c07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18c07ac00 0x7fb18c07d0c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb194005e10 tx=0x7fb194005da0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4102b10 msgr2=0x7fb1a419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4102b10 0x7fb1a419f820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009230 tx=0x7fb1a00046c0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 shutdown_connections 2024-09-10T04:13:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18c07ac00 0x7fb18c07d0c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1a4103430 0x7fb1a419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 --2- 172.21.15.46:0/3138485895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1a4102b10 0x7fb1a419f820 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 >> 172.21.15.46:0/3138485895 conn(0x7fb1a40fc550 msgr2=0x7fb1a4100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:44.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 shutdown_connections 2024-09-10T04:13:44.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.209+0000 7fb1aaaae640 1 -- 172.21.15.46:0/3138485895 wait complete. 2024-09-10T04:13:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.294+0000 7fc4496cc640 1 -- 172.21.15.46:0/706401963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444102b30 msgr2=0x7fc444102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.294+0000 7fc4496cc640 1 --2- 172.21.15.46:0/706401963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444102b30 0x7fc444102f10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc434009c20 tx=0x7fc43402aa00 comp rx=0 tx=0).stop 2024-09-10T04:13:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 -- 172.21.15.46:0/706401963 shutdown_connections 2024-09-10T04:13:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 --2- 172.21.15.46:0/706401963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444103450 0x7fc44410b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 --2- 172.21.15.46:0/706401963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444102b30 0x7fc444102f10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 -- 172.21.15.46:0/706401963 >> 172.21.15.46:0/706401963 conn(0x7fc4440fc550 msgr2=0x7fc4440fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:44.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 -- 172.21.15.46:0/706401963 shutdown_connections 2024-09-10T04:13:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 -- 172.21.15.46:0/706401963 wait complete. 2024-09-10T04:13:44.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 Processor -- start 2024-09-10T04:13:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.295+0000 7fc4496cc640 1 -- start start 2024-09-10T04:13:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4496cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444102b30 0x7fc44419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4496cc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444103450 0x7fc44419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4496cc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410ede0 con 0x7fc444103450 2024-09-10T04:13:44.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4496cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc44410ec60 con 0x7fc444102b30 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4427fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444103450 0x7fc44419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4427fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444103450 0x7fc44419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38010/0 (socket says 172.21.15.46:38010) 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4427fc640 1 -- 172.21.15.46:0/2878934404 learned_addr learned my addr 172.21.15.46:0/2878934404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc442ffd640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444102b30 0x7fc44419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4427fc640 1 -- 172.21.15.46:0/2878934404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444102b30 msgr2=0x7fc44419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4427fc640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444102b30 0x7fc44419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4427fc640 1 -- 172.21.15.46:0/2878934404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4441977d0 con 0x7fc444103450 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc4427fc640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444103450 0x7fc44419db90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c8f0 tx=0x7fc43800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.296+0000 7fc42bfff640 1 -- 172.21.15.46:0/2878934404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc4380079b0 con 0x7fc444103450 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.297+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc444197ac0 con 0x7fc444103450 2024-09-10T04:13:44.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.297+0000 7fc42bfff640 1 -- 172.21.15.46:0/2878934404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4380054e0 con 0x7fc444103450 2024-09-10T04:13:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.297+0000 7fc42bfff640 1 -- 172.21.15.46:0/2878934404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc438004dd0 con 0x7fc444103450 2024-09-10T04:13:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.297+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc444198050 con 0x7fc444103450 2024-09-10T04:13:44.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.298+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc44406a500 con 0x7fc444103450 2024-09-10T04:13:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.298+0000 7fc42bfff640 1 -- 172.21.15.46:0/2878934404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc438005020 con 0x7fc444103450 2024-09-10T04:13:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.299+0000 7fc42bfff640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc42c07ab90 0x7fc42c07d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.299+0000 7fc42bfff640 1 -- 172.21.15.46:0/2878934404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6242+0+0 (secure 0 0 0) 0x7fc43809db90 con 0x7fc444103450 2024-09-10T04:13:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.299+0000 7fc442ffd640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc42c07ab90 0x7fc42c07d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.300+0000 7fc442ffd640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc42c07ab90 0x7fc42c07d050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc434009430 tx=0x7fc43403a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:44.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.303+0000 7fc42bfff640 1 -- 172.21.15.46:0/2878934404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc438014030 con 0x7fc444103450 2024-09-10T04:13:44.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.533+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc444105f90 con 0x7fc444103450 2024-09-10T04:13:44.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.534+0000 7fc42bfff640 1 -- 172.21.15.46:0/2878934404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+1994 (secure 0 0 0) 0x7fc4380692d0 con 0x7fc444103450 2024-09-10T04:13:44.535 INFO:teuthology.orchestra.run.smithi046.stdout:e21 2024-09-10T04:13:44.535 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:13:35:433662+0000 2024-09-10T04:13:44.535 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:13:44.535 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:13:44.535 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:13:44.536 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:13:44.537 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34316} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/184543618,v1:172.21.15.46:6829/184543618] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:44.538 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44303} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/2786504298,v1:172.21.15.70:6825/2786504298] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 21 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.536+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc42c07ab90 msgr2=0x7fc42c07d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc42c07ab90 0x7fc42c07d050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc434009430 tx=0x7fc43403a040 comp rx=0 tx=0).stop 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444103450 msgr2=0x7fc44419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444103450 0x7fc44419db90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c8f0 tx=0x7fc43800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 shutdown_connections 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc42c07ab90 0x7fc42c07d050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc444103450 0x7fc44419db90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 --2- 172.21.15.46:0/2878934404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc444102b30 0x7fc44419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 >> 172.21.15.46:0/2878934404 conn(0x7fc4440fc550 msgr2=0x7fc444108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:44.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 shutdown_connections 2024-09-10T04:13:44.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.537+0000 7fc4496cc640 1 -- 172.21.15.46:0/2878934404 wait complete. 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 -- 172.21.15.46:0/86166254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 msgr2=0x7f24e8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/86166254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e8100c50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24d0009880 tx=0x7f24d002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 -- 172.21.15.46:0/86166254 shutdown_connections 2024-09-10T04:13:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/86166254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e8100c50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/86166254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e81067f0 0x7f24e8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 -- 172.21.15.46:0/86166254 >> 172.21.15.46:0/86166254 conn(0x7f24e80fc550 msgr2=0x7f24e80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 -- 172.21.15.46:0/86166254 shutdown_connections 2024-09-10T04:13:44.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.624+0000 7f24ecfdb640 1 -- 172.21.15.46:0/86166254 wait complete. 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.625+0000 7f24ecfdb640 1 Processor -- start 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.625+0000 7f24ecfdb640 1 -- start start 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.625+0000 7f24ecfdb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e81023c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.625+0000 7f24ecfdb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e81067f0 0x7f24e8102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.625+0000 7f24ecfdb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810dcc0 con 0x7f24e81007f0 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.625+0000 7f24ecfdb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f24e810db40 con 0x7f24e81067f0 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e81023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e81023c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38036/0 (socket says 172.21.15.46:38036) 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e81067f0 0x7f24e8102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e659c640 1 -- 172.21.15.46:0/1679180261 learned_addr learned my addr 172.21.15.46:0/1679180261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:44.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e659c640 1 -- 172.21.15.46:0/1679180261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e81067f0 msgr2=0x7f24e8102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e659c640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e81067f0 0x7f24e8102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e659c640 1 -- 172.21.15.46:0/1679180261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e8102e40 con 0x7f24e81007f0 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e5d9b640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e81067f0 0x7f24e8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24e659c640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e81023c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d6f0 tx=0x7f24d800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24df7fe640 1 -- 172.21.15.46:0/1679180261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24d8017070 con 0x7f24e81007f0 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24df7fe640 1 -- 172.21.15.46:0/1679180261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f24d800de00 con 0x7f24e81007f0 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.626+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e8103130 con 0x7f24e81007f0 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.627+0000 7f24df7fe640 1 -- 172.21.15.46:0/1679180261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f24d8004bb0 con 0x7f24e81007f0 2024-09-10T04:13:44.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.627+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81a3e00 con 0x7f24e81007f0 2024-09-10T04:13:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.628+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e8101f30 con 0x7f24e81007f0 2024-09-10T04:13:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.629+0000 7f24df7fe640 1 -- 172.21.15.46:0/1679180261 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f24d8004e60 con 0x7f24e81007f0 2024-09-10T04:13:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.630+0000 7f24df7fe640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f24d407abf0 0x7f24d407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.630+0000 7f24df7fe640 1 -- 172.21.15.46:0/1679180261 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6242+0+0 (secure 0 0 0) 0x7f24d809d820 con 0x7f24e81007f0 2024-09-10T04:13:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.630+0000 7f24e5d9b640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f24d407abf0 0x7f24d407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.630+0000 7f24e5d9b640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f24d407abf0 0x7f24d407d0b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24e8103e50 tx=0x7f24d0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:44.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.634+0000 7f24df7fe640 1 -- 172.21.15.46:0/1679180261 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f24d8068f60 con 0x7f24e81007f0 2024-09-10T04:13:44.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.842+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f24e8069ca0 con 0x7f24d407abf0 2024-09-10T04:13:44.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.844+0000 7f24df7fe640 1 -- 172.21.15.46:0/1679180261 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f24e8069ca0 con 0x7f24d407abf0 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "13/23 daemons upgraded", 2024-09-10T04:13:44.845 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:13:44.846 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:13:44.846 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:13:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f24d407abf0 msgr2=0x7f24d407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f24d407abf0 0x7f24d407d0b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24e8103e50 tx=0x7f24d0031400 comp rx=0 tx=0).stop 2024-09-10T04:13:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 msgr2=0x7f24e81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e81023c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24d800d6f0 tx=0x7f24d800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 shutdown_connections 2024-09-10T04:13:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f24d407abf0 0x7f24d407d0b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f24e81067f0 0x7f24e8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 --2- 172.21.15.46:0/1679180261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24e81007f0 0x7f24e81023c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 >> 172.21.15.46:0/1679180261 conn(0x7f24e80fc550 msgr2=0x7f24e8104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 shutdown_connections 2024-09-10T04:13:44.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.846+0000 7f24ecfdb640 1 -- 172.21.15.46:0/1679180261 wait complete. 2024-09-10T04:13:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.931+0000 7f947d30c640 1 -- 172.21.15.46:0/1360273985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 msgr2=0x7f9478101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.931+0000 7f947d30c640 1 --2- 172.21.15.46:0/1360273985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f9478101d60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f946c009880 tx=0x7f946c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.931+0000 7f947d30c640 1 -- 172.21.15.46:0/1360273985 shutdown_connections 2024-09-10T04:13:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.931+0000 7f947d30c640 1 --2- 172.21.15.46:0/1360273985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f9478101d60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.931+0000 7f947d30c640 1 --2- 172.21.15.46:0/1360273985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9478107900 0x7f9478107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.932+0000 7f947d30c640 1 -- 172.21.15.46:0/1360273985 >> 172.21.15.46:0/1360273985 conn(0x7f94780fd600 msgr2=0x7f94780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.932+0000 7f947d30c640 1 -- 172.21.15.46:0/1360273985 shutdown_connections 2024-09-10T04:13:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.932+0000 7f947d30c640 1 -- 172.21.15.46:0/1360273985 wait complete. 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.932+0000 7f947d30c640 1 Processor -- start 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.932+0000 7f947d30c640 1 -- start start 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f947d30c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f947819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f947d30c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9478107900 0x7f947819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f947d30c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f947810edd0 con 0x7f9478101900 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f947d30c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f947810ec50 con 0x7f9478107900 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f9476ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f947819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f9476ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f947819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38054/0 (socket says 172.21.15.46:38054) 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f9476ffd640 1 -- 172.21.15.46:0/4099543921 learned_addr learned my addr 172.21.15.46:0/4099543921 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f94767fc640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9478107900 0x7f947819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f9476ffd640 1 -- 172.21.15.46:0/4099543921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9478107900 msgr2=0x7f947819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f9476ffd640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9478107900 0x7f947819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f9476ffd640 1 -- 172.21.15.46:0/4099543921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9478197800 con 0x7f9478101900 2024-09-10T04:13:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.933+0000 7f9476ffd640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f947819d680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f946800e970 tx=0x7f946800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.934+0000 7f945ffff640 1 -- 172.21.15.46:0/4099543921 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f946800cac0 con 0x7f9478101900 2024-09-10T04:13:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.934+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9478197af0 con 0x7f9478101900 2024-09-10T04:13:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.934+0000 7f945ffff640 1 -- 172.21.15.46:0/4099543921 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f94680054e0 con 0x7f9478101900 2024-09-10T04:13:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.934+0000 7f945ffff640 1 -- 172.21.15.46:0/4099543921 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9468010430 con 0x7f9478101900 2024-09-10T04:13:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.934+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9478198050 con 0x7f9478101900 2024-09-10T04:13:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.935+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9478103040 con 0x7f9478101900 2024-09-10T04:13:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.935+0000 7f945ffff640 1 -- 172.21.15.46:0/4099543921 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9468004ec0 con 0x7f9478101900 2024-09-10T04:13:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.936+0000 7f945ffff640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f946007abf0 0x7f946007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:13:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.936+0000 7f94767fc640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f946007abf0 0x7f946007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:13:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.936+0000 7f945ffff640 1 -- 172.21.15.46:0/4099543921 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6242+0+0 (secure 0 0 0) 0x7f946809e340 con 0x7f9478101900 2024-09-10T04:13:44.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.936+0000 7f94767fc640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f946007abf0 0x7f946007d0b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9478198b50 tx=0x7f946c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:13:44.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:44.940+0000 7f945ffff640 1 -- 172.21.15.46:0/4099543921 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9468069a80 con 0x7f9478101900 2024-09-10T04:13:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.206+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9478109e70 con 0x7f9478101900 2024-09-10T04:13:45.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.206+0000 7f945ffff640 1 -- 172.21.15.46:0/4099543921 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f946806e930 con 0x7f9478101900 2024-09-10T04:13:45.207 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:13:45.207 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:13:45.207 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:13:45.208 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:13:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:13:45.208 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:13:45.208 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:13:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f946007abf0 msgr2=0x7f946007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f946007abf0 0x7f946007d0b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9478198b50 tx=0x7f946c031400 comp rx=0 tx=0).stop 2024-09-10T04:13:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 msgr2=0x7f947819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:13:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f947819d680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f946800e970 tx=0x7f946800ed70 comp rx=0 tx=0).stop 2024-09-10T04:13:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 shutdown_connections 2024-09-10T04:13:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f946007abf0 0x7f946007d0b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:45.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9478107900 0x7f947819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 --2- 172.21.15.46:0/4099543921 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9478101900 0x7f947819d680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:13:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 >> 172.21.15.46:0/4099543921 conn(0x7f94780fd600 msgr2=0x7f94780fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:13:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 shutdown_connections 2024-09-10T04:13:45.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:13:45.209+0000 7f947d30c640 1 -- 172.21.15.46:0/4099543921 wait complete. 2024-09-10T04:13:45.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:45 smithi070 ceph-mon[79774]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:45.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:45 smithi070 ceph-mon[79774]: pgmap v240: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:45.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:45 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3138485895' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:45.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:45 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2878934404' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:13:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:45 smithi046 ceph-mon[109506]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:45 smithi046 ceph-mon[109506]: pgmap v240: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:45 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3138485895' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:13:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:45 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2878934404' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:13:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:46 smithi046 ceph-mon[109506]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:46 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4099543921' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:13:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:46 smithi046 ceph-mon[109506]: Upgrade: Updating mds.cephfs.smithi046.cpwyud 2024-09-10T04:13:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.cpwyud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:13:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:46.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:46 smithi046 ceph-mon[109506]: Deploying daemon mds.cephfs.smithi046.cpwyud on smithi046 2024-09-10T04:13:46.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:46 smithi070 ceph-mon[79774]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:13:46.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:46 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4099543921' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:13:46.486 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:46 smithi070 ceph-mon[79774]: Upgrade: Updating mds.cephfs.smithi046.cpwyud 2024-09-10T04:13:46.487 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:46.487 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi046.cpwyud", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:13:46.487 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:13:46.487 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:46 smithi070 ceph-mon[79774]: Deploying daemon mds.cephfs.smithi046.cpwyud on smithi046 2024-09-10T04:13:47.770 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:47 smithi070 ceph-mon[79774]: pgmap v241: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:47.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:47 smithi046 ceph-mon[109506]: pgmap v241: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:48.681 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:48 smithi070 ceph-mon[79774]: osdmap e77: 6 total, 6 up, 6 in 2024-09-10T04:13:48.681 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:48 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:13:48.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:48 smithi046 ceph-mon[109506]: osdmap e77: 6 total, 6 up, 6 in 2024-09-10T04:13:48.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:48 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:13:49.809 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:49 smithi046 ceph-mon[109506]: pgmap v243: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:49.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:49 smithi070 ceph-mon[79774]: pgmap v243: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:51 smithi070 ceph-mon[79774]: pgmap v244: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:51 smithi046 ceph-mon[109506]: pgmap v244: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:52.960 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:52.960 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:52.960 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:52.960 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:52 smithi046 ceph-mon[109506]: pgmap v245: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:52.960 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:52 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:boot 2024-09-10T04:13:52.960 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:52 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:13:52.960 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:52 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"} : dispatch 2024-09-10T04:13:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:13:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:52 smithi070 ceph-mon[79774]: pgmap v245: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:52 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:boot 2024-09-10T04:13:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:52 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:13:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:52 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi046.cpwyud"} : dispatch 2024-09-10T04:13:53.888 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:13:54.917 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:54 smithi046 ceph-mon[109506]: pgmap v246: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:54 smithi070 ceph-mon[79774]: pgmap v246: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:57 smithi070 ceph-mon[79774]: pgmap v247: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:57 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:57 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:57 smithi046 ceph-mon[109506]: pgmap v247: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:57 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:57 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:58.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:58 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:58.830 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:58 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:58 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:58 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:13:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:13:59 smithi070 ceph-mon[79774]: pgmap v248: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:13:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:13:59 smithi046 ceph-mon[109506]: pgmap v248: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: pgmap v249: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:14:01.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:14:01.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:01.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:14:01.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:01.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:01.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:01.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:01 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: pgmap v249: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:01.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:01.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:01 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:02 smithi070 ceph-mon[79774]: Upgrade: Updating mds.cephfs.smithi070.ehvjgr 2024-09-10T04:14:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:02 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:02 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.ehvjgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:14:03.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:02 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:14:03.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:02 smithi070 ceph-mon[79774]: Deploying daemon mds.cephfs.smithi070.ehvjgr on smithi070 2024-09-10T04:14:03.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:02 smithi070 ceph-mon[79774]: pgmap v250: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:03.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:02 smithi046 ceph-mon[109506]: Upgrade: Updating mds.cephfs.smithi070.ehvjgr 2024-09-10T04:14:03.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:02 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:03.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:02 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.ehvjgr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:14:03.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:02 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:14:03.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:02 smithi046 ceph-mon[109506]: Deploying daemon mds.cephfs.smithi070.ehvjgr on smithi070 2024-09-10T04:14:03.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:02 smithi046 ceph-mon[109506]: pgmap v250: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:05.280 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:05 smithi046 ceph-mon[109506]: osdmap e78: 6 total, 6 up, 6 in 2024-09-10T04:14:05.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:05 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:14:05.281 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:05 smithi046 ceph-mon[109506]: pgmap v252: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:05.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:05 smithi070 ceph-mon[79774]: osdmap e78: 6 total, 6 up, 6 in 2024-09-10T04:14:05.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:05 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:14:05.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:05 smithi070 ceph-mon[79774]: pgmap v252: 65 pgs: 65 active+clean; 245 MiB data, 929 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:06 smithi070 ceph-mon[79774]: pgmap v253: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:06 smithi046 ceph-mon[109506]: pgmap v253: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:08.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:08.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:08.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:14:08.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:08 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] up:boot 2024-09-10T04:14:08.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:08 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:14:08.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"} : dispatch 2024-09-10T04:14:08.825 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:14:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:08 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] up:boot 2024-09-10T04:14:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:08 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:14:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.ehvjgr"} : dispatch 2024-09-10T04:14:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:09.811 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:09 smithi070 ceph-mon[79774]: pgmap v254: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:09.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:09 smithi046 ceph-mon[109506]: pgmap v254: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:11.544 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:11 smithi070 ceph-mon[79774]: pgmap v255: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:11.544 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:11.545 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:11.545 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:11.545 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:11 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:11.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:11 smithi046 ceph-mon[109506]: pgmap v255: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:11 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:14:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:13 smithi070 ceph-mon[79774]: pgmap v256: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:13 smithi046 ceph-mon[109506]: pgmap v256: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 -- 172.21.15.46:0/1337022589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 msgr2=0x7f7308106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 --2- 172.21.15.46:0/1337022589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f7308106bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009880 tx=0x7f72f802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 -- 172.21.15.46:0/1337022589 shutdown_connections 2024-09-10T04:14:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 --2- 172.21.15.46:0/1337022589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081007f0 0x7f7308100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 --2- 172.21.15.46:0/1337022589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f7308106bd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 -- 172.21.15.46:0/1337022589 >> 172.21.15.46:0/1337022589 conn(0x7f73080fc550 msgr2=0x7f73080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 -- 172.21.15.46:0/1337022589 shutdown_connections 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.306+0000 7f730ee90640 1 -- 172.21.15.46:0/1337022589 wait complete. 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f730ee90640 1 Processor -- start 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f730ee90640 1 -- start start 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f730ee90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081007f0 0x7f730819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f730ee90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f730819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f730ee90640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f730810dca0 con 0x7f73081067f0 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f730ee90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f730810db20 con 0x7f73081007f0 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f7307fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f730819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f7307fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f730819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54310/0 (socket says 172.21.15.46:54310) 2024-09-10T04:14:15.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f7307fff640 1 -- 172.21.15.46:0/212939027 learned_addr learned my addr 172.21.15.46:0/212939027 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.307+0000 7f730cc2c640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081007f0 0x7f730819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f7307fff640 1 -- 172.21.15.46:0/212939027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081007f0 msgr2=0x7f730819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f7307fff640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081007f0 0x7f730819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f7307fff640 1 -- 172.21.15.46:0/212939027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7308197750 con 0x7f73081067f0 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f730cc2c640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081007f0 0x7f730819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f7307fff640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f730819dbc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c810 tx=0x7f72fc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f7305ffb640 1 -- 172.21.15.46:0/212939027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f72fc004020 con 0x7f73081067f0 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7308197a40 con 0x7f73081067f0 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f7305ffb640 1 -- 172.21.15.46:0/212939027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f72fc0041c0 con 0x7f73081067f0 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.308+0000 7f7305ffb640 1 -- 172.21.15.46:0/212939027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f72fc0047a0 con 0x7f73081067f0 2024-09-10T04:14:15.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.309+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7308197fa0 con 0x7f73081067f0 2024-09-10T04:14:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.310+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7308101f30 con 0x7f73081067f0 2024-09-10T04:14:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.310+0000 7f7305ffb640 1 -- 172.21.15.46:0/212939027 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f72fc0079b0 con 0x7f73081067f0 2024-09-10T04:14:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.311+0000 7f7305ffb640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f007abf0 0x7f72f007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.311+0000 7f7305ffb640 1 -- 172.21.15.46:0/212939027 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f72fc09db80 con 0x7f73081067f0 2024-09-10T04:14:15.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.311+0000 7f730cc2c640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f007abf0 0x7f72f007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.311+0000 7f730cc2c640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f007abf0 0x7f72f007d0b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f72f8031460 tx=0x7f72f8009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:15.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.315+0000 7f7305ffb640 1 -- 172.21.15.46:0/212939027 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f72fc014030 con 0x7f73081067f0 2024-09-10T04:14:15.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.520+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f73081982b0 con 0x7f72f007abf0 2024-09-10T04:14:15.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.522+0000 7f7305ffb640 1 -- 172.21.15.46:0/212939027 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f73081982b0 con 0x7f72f007abf0 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f007abf0 msgr2=0x7f72f007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f007abf0 0x7f72f007d0b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f72f8031460 tx=0x7f72f8009560 comp rx=0 tx=0).stop 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 msgr2=0x7f730819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f730819dbc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72fc00c810 tx=0x7f72fc00ccd0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 shutdown_connections 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f007abf0 0x7f72f007d0b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081067f0 0x7f730819dbc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 --2- 172.21.15.46:0/212939027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081007f0 0x7f730819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 >> 172.21.15.46:0/212939027 conn(0x7f73080fc550 msgr2=0x7f7308069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 shutdown_connections 2024-09-10T04:14:15.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.524+0000 7f730ee90640 1 -- 172.21.15.46:0/212939027 wait complete. 2024-09-10T04:14:15.535 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:14:15.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:15 smithi046 ceph-mon[109506]: pgmap v257: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 -- 172.21.15.46:0/3931086882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 msgr2=0x7febf0100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 --2- 172.21.15.46:0/3931086882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf0100ab0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7febe0009880 tx=0x7febe002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 -- 172.21.15.46:0/3931086882 shutdown_connections 2024-09-10T04:14:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 --2- 172.21.15.46:0/3931086882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf0100ab0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 --2- 172.21.15.46:0/3931086882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf0106650 0x7febf0106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 -- 172.21.15.46:0/3931086882 >> 172.21.15.46:0/3931086882 conn(0x7febf00fc350 msgr2=0x7febf00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 -- 172.21.15.46:0/3931086882 shutdown_connections 2024-09-10T04:14:15.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.610+0000 7febf659e640 1 -- 172.21.15.46:0/3931086882 wait complete. 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.611+0000 7febf659e640 1 Processor -- start 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.611+0000 7febf659e640 1 -- start start 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf659e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf010b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf659e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf0106650 0x7febf010bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf659e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febf010ec00 con 0x7febf0100650 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf659e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7febf010ea80 con 0x7febf0106650 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf559c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf010b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf559c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf010b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54332/0 (socket says 172.21.15.46:54332) 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf559c640 1 -- 172.21.15.46:0/1682473120 learned_addr learned my addr 172.21.15.46:0/1682473120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf4d9b640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf0106650 0x7febf010bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf559c640 1 -- 172.21.15.46:0/1682473120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf0106650 msgr2=0x7febf010bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf559c640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf0106650 0x7febf010bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf559c640 1 -- 172.21.15.46:0/1682473120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf010c060 con 0x7febf0100650 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf4d9b640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf0106650 0x7febf010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf559c640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf010b5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febec00c8a0 tx=0x7febec00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febe67fc640 1 -- 172.21.15.46:0/1682473120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7febec0079b0 con 0x7febf0100650 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febe67fc640 1 -- 172.21.15.46:0/1682473120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7febec007b50 con 0x7febf0100650 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf010c350 con 0x7febf0100650 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.612+0000 7febe67fc640 1 -- 172.21.15.46:0/1682473120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7febec018700 con 0x7febf0100650 2024-09-10T04:14:15.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.613+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf01a3be0 con 0x7febf0100650 2024-09-10T04:14:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.613+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf0101d90 con 0x7febf0100650 2024-09-10T04:14:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.614+0000 7febe67fc640 1 -- 172.21.15.46:0/1682473120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7febec007d00 con 0x7febf0100650 2024-09-10T04:14:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.615+0000 7febe67fc640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7febdc07abf0 0x7febdc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.615+0000 7febe67fc640 1 -- 172.21.15.46:0/1682473120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7febec09df90 con 0x7febf0100650 2024-09-10T04:14:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.615+0000 7febf4d9b640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7febdc07abf0 0x7febdc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.615+0000 7febf4d9b640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7febdc07abf0 0x7febdc07d0b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7febf010d070 tx=0x7febe0009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:15.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.619+0000 7febe67fc640 1 -- 172.21.15.46:0/1682473120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7febec014030 con 0x7febf0100650 2024-09-10T04:14:15.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.823+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7febf0062630 con 0x7febdc07abf0 2024-09-10T04:14:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.824+0000 7febe67fc640 1 -- 172.21.15.46:0/1682473120 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7febf0062630 con 0x7febdc07abf0 2024-09-10T04:14:15.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7febdc07abf0 msgr2=0x7febdc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7febdc07abf0 0x7febdc07d0b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7febf010d070 tx=0x7febe0009560 comp rx=0 tx=0).stop 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 msgr2=0x7febf010b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf010b5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febec00c8a0 tx=0x7febec00cd60 comp rx=0 tx=0).stop 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 shutdown_connections 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7febdc07abf0 0x7febdc07d0b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7febf0106650 0x7febf010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 --2- 172.21.15.46:0/1682473120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf0100650 0x7febf010b5e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 >> 172.21.15.46:0/1682473120 conn(0x7febf00fc350 msgr2=0x7febf00fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 shutdown_connections 2024-09-10T04:14:15.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.827+0000 7febf659e640 1 -- 172.21.15.46:0/1682473120 wait complete. 2024-09-10T04:14:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:15 smithi070 ceph-mon[79774]: pgmap v257: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 -- 172.21.15.46:0/2938038888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 msgr2=0x7f767c10b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 --2- 172.21.15.46:0/2938038888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c10b930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76640098b0 tx=0x7f766402a560 comp rx=0 tx=0).stop 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 -- 172.21.15.46:0/2938038888 shutdown_connections 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 --2- 172.21.15.46:0/2938038888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c10b930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 --2- 172.21.15.46:0/2938038888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c102b20 0x7f767c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 -- 172.21.15.46:0/2938038888 >> 172.21.15.46:0/2938038888 conn(0x7f767c0fc520 msgr2=0x7f767c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 -- 172.21.15.46:0/2938038888 shutdown_connections 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.911+0000 7f768224b640 1 -- 172.21.15.46:0/2938038888 wait complete. 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.912+0000 7f768224b640 1 Processor -- start 2024-09-10T04:14:15.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.912+0000 7f768224b640 1 -- start start 2024-09-10T04:14:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.912+0000 7f768224b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c102b20 0x7f767c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.912+0000 7f768224b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.912+0000 7f768224b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f767c10edf0 con 0x7f767c103440 2024-09-10T04:14:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.912+0000 7f768224b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f767c10ec70 con 0x7f767c102b20 2024-09-10T04:14:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f7673fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f7673fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54344/0 (socket says 172.21.15.46:54344) 2024-09-10T04:14:15.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f7673fff640 1 -- 172.21.15.46:0/691406366 learned_addr learned my addr 172.21.15.46:0/691406366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f767b7fe640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c102b20 0x7f767c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f7673fff640 1 -- 172.21.15.46:0/691406366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c102b20 msgr2=0x7f767c19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f7673fff640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c102b20 0x7f767c19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f7673fff640 1 -- 172.21.15.46:0/691406366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c199a40 con 0x7f767c103440 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f7673fff640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c19fdc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7664031460 tx=0x7f76640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f76797fa640 1 -- 172.21.15.46:0/691406366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f766403f070 con 0x7f767c103440 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.913+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c199cd0 con 0x7f767c103440 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.914+0000 7f76797fa640 1 -- 172.21.15.46:0/691406366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f76640044d0 con 0x7f767c103440 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.914+0000 7f76797fa640 1 -- 172.21.15.46:0/691406366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7664035030 con 0x7f767c103440 2024-09-10T04:14:15.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.914+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c19a200 con 0x7f767c103440 2024-09-10T04:14:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.915+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f767c06aa30 con 0x7f767c103440 2024-09-10T04:14:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.915+0000 7f76797fa640 1 -- 172.21.15.46:0/691406366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7664042070 con 0x7f767c103440 2024-09-10T04:14:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.916+0000 7f76797fa640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f766807abf0 0x7f766807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:15.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.916+0000 7f767b7fe640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f766807abf0 0x7f766807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.916+0000 7f76797fa640 1 -- 172.21.15.46:0/691406366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f76640be610 con 0x7f767c103440 2024-09-10T04:14:15.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.917+0000 7f767b7fe640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f766807abf0 0x7f766807d0b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f766c0058c0 tx=0x7f766c00a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:15.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:15.921+0000 7f76797fa640 1 -- 172.21.15.46:0/691406366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7664004cf0 con 0x7f767c103440 2024-09-10T04:14:16.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.113+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f767c19af70 con 0x7f766807abf0 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.123+0000 7f76797fa640 1 -- 172.21.15.46:0/691406366 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f767c19af70 con 0x7f766807abf0 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (5m) 19s ago 17m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (8m) 19s ago 17m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (8m) 5s ago 16m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (4m) 19s ago 17m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (4m) 5s ago 16m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:14:16.125 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (5m) 19s ago 16m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (24s) 19s ago 11m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (41s) 19s ago 11m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (9s) 5s ago 11m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (11m) 5s ago 11m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (7m) 19s ago 18m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (6m) 5s ago 16m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (4m) 19s ago 18m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (4m) 5s ago 16m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (6m) 19s ago 17m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (6m) 5s ago 16m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (3m) 19s ago 15m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (3m) 19s ago 14m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:14:16.126 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (2m) 19s ago 14m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:14:16.127 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (2m) 5s ago 13m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:14:16.127 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (91s) 5s ago 13m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:14:16.127 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (58s) 5s ago 12m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:14:16.127 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (6m) 19s ago 16m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:14:16.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.125+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f766807abf0 msgr2=0x7f766807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.125+0000 7f768224b640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f766807abf0 0x7f766807d0b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f766c0058c0 tx=0x7f766c00a250 comp rx=0 tx=0).stop 2024-09-10T04:14:16.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.125+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 msgr2=0x7f767c19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.125+0000 7f768224b640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c19fdc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7664031460 tx=0x7f76640048f0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.126+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 shutdown_connections 2024-09-10T04:14:16.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.126+0000 7f768224b640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f766807abf0 0x7f766807d0b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.126+0000 7f768224b640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f767c103440 0x7f767c19fdc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.126+0000 7f768224b640 1 --2- 172.21.15.46:0/691406366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767c102b20 0x7f767c19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.126+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 >> 172.21.15.46:0/691406366 conn(0x7f767c0fc520 msgr2=0x7f767c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:16.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.126+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 shutdown_connections 2024-09-10T04:14:16.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.126+0000 7f768224b640 1 -- 172.21.15.46:0/691406366 wait complete. 2024-09-10T04:14:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/108466440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0103440 msgr2=0x7f0bc010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/108466440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0103440 0x7f0bc010b930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098b0 tx=0x7f0ba802a560 comp rx=0 tx=0).stop 2024-09-10T04:14:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/108466440 shutdown_connections 2024-09-10T04:14:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/108466440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0103440 0x7f0bc010b930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/108466440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bc0102b20 0x7f0bc0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/108466440 >> 172.21.15.46:0/108466440 conn(0x7f0bc00fc520 msgr2=0x7f0bc00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/108466440 shutdown_connections 2024-09-10T04:14:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.209+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/108466440 wait complete. 2024-09-10T04:14:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bc5fa7640 1 Processor -- start 2024-09-10T04:14:16.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bc5fa7640 1 -- start start 2024-09-10T04:14:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bc5fa7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0102b20 0x7f0bc0100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bc5fa7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bc0103440 0x7f0bc0100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bc5fa7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc010edd0 con 0x7f0bc0102b20 2024-09-10T04:14:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bc5fa7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc010ec50 con 0x7f0bc0103440 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0102b20 0x7f0bc0100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bbf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0102b20 0x7f0bc0100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54358/0 (socket says 172.21.15.46:54358) 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bbf7fe640 1 -- 172.21.15.46:0/3361932109 learned_addr learned my addr 172.21.15.46:0/3361932109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.210+0000 7f0bb6ffd640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bc0103440 0x7f0bc0100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bbf7fe640 1 -- 172.21.15.46:0/3361932109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bc0103440 msgr2=0x7f0bc0100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bbf7fe640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bc0103440 0x7f0bc0100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bbf7fe640 1 -- 172.21.15.46:0/3361932109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc0101200 con 0x7f0bc0102b20 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bbf7fe640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0102b20 0x7f0bc0100780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000d8d0 tx=0x7f0bb000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bbd7fa640 1 -- 172.21.15.46:0/3361932109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0bb0017070 con 0x7f0bc0102b20 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc01014f0 con 0x7f0bc0102b20 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bbd7fa640 1 -- 172.21.15.46:0/3361932109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0bb0010460 con 0x7f0bc0102b20 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bbd7fa640 1 -- 172.21.15.46:0/3361932109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0bb000f5f0 con 0x7f0bc0102b20 2024-09-10T04:14:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.211+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc01a3f00 con 0x7f0bc0102b20 2024-09-10T04:14:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.213+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc006aa30 con 0x7f0bc0102b20 2024-09-10T04:14:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.213+0000 7f0bbd7fa640 1 -- 172.21.15.46:0/3361932109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0bb0009a30 con 0x7f0bc0102b20 2024-09-10T04:14:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.214+0000 7f0bbd7fa640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0bac07abf0 0x7f0bac07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.214+0000 7f0bbd7fa640 1 -- 172.21.15.46:0/3361932109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0bb009dab0 con 0x7f0bc0102b20 2024-09-10T04:14:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.214+0000 7f0bb6ffd640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0bac07abf0 0x7f0bac07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.214+0000 7f0bb6ffd640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0bac07abf0 0x7f0bac07d0b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000c00 tx=0x7f0ba8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:16.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.218+0000 7f0bbd7fa640 1 -- 172.21.15.46:0/3361932109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0bb00691f0 con 0x7f0bc0102b20 2024-09-10T04:14:16.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.475+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f0bc0101ce0 con 0x7f0bc0102b20 2024-09-10T04:14:16.477 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:16 smithi046 ceph-mon[109506]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:16.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.475+0000 7f0bbd7fa640 1 -- 172.21.15.46:0/3361932109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f0bc0101ce0 con 0x7f0bc0102b20 2024-09-10T04:14:16.477 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:14:16.478 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:14:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:14:16.479 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:14:16.479 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0bac07abf0 msgr2=0x7f0bac07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0bac07abf0 0x7f0bac07d0b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8000c00 tx=0x7f0ba8031400 comp rx=0 tx=0).stop 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0102b20 msgr2=0x7f0bc0100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0102b20 0x7f0bc0100780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000d8d0 tx=0x7f0bb000dd90 comp rx=0 tx=0).stop 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 shutdown_connections 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0bac07abf0 0x7f0bac07d0b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bc0103440 0x7f0bc0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 --2- 172.21.15.46:0/3361932109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bc0102b20 0x7f0bc0100780 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 >> 172.21.15.46:0/3361932109 conn(0x7f0bc00fc520 msgr2=0x7f0bc0103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 shutdown_connections 2024-09-10T04:14:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.477+0000 7f0bc5fa7640 1 -- 172.21.15.46:0/3361932109 wait complete. 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 -- 172.21.15.46:0/2487818556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 msgr2=0x7f22d8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 --2- 172.21.15.46:0/2487818556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d8107ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009880 tx=0x7f22d402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 -- 172.21.15.46:0/2487818556 shutdown_connections 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 --2- 172.21.15.46:0/2487818556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d8101900 0x7f22d8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 --2- 172.21.15.46:0/2487818556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d8107ce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 -- 172.21.15.46:0/2487818556 >> 172.21.15.46:0/2487818556 conn(0x7f22d80fd600 msgr2=0x7f22d80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 -- 172.21.15.46:0/2487818556 shutdown_connections 2024-09-10T04:14:16.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.562+0000 7f22dfdac640 1 -- 172.21.15.46:0/2487818556 wait complete. 2024-09-10T04:14:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dfdac640 1 Processor -- start 2024-09-10T04:14:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dfdac640 1 -- start start 2024-09-10T04:14:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dfdac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d8101900 0x7f22d819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dfdac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dfdac640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22d810edd0 con 0x7f22d8107900 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dfdac640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f22d810ec50 con 0x7f22d8101900 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dd347640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dd347640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54380/0 (socket says 172.21.15.46:54380) 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dd347640 1 -- 172.21.15.46:0/649283837 learned_addr learned my addr 172.21.15.46:0/649283837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22ddb48640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d8101900 0x7f22d819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.563+0000 7f22dd347640 1 -- 172.21.15.46:0/649283837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d8101900 msgr2=0x7f22d819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22dd347640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d8101900 0x7f22d819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22dd347640 1 -- 172.21.15.46:0/649283837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d81977a0 con 0x7f22d8107900 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22ddb48640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d8101900 0x7f22d819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22dd347640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d819db60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c8f0 tx=0x7f22c800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22ceffd640 1 -- 172.21.15.46:0/649283837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f22c80079b0 con 0x7f22d8107900 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22ceffd640 1 -- 172.21.15.46:0/649283837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f22c80054e0 con 0x7f22d8107900 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d8197a90 con 0x7f22d8107900 2024-09-10T04:14:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22ceffd640 1 -- 172.21.15.46:0/649283837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f22c8004dd0 con 0x7f22d8107900 2024-09-10T04:14:16.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.564+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d8198020 con 0x7f22d8107900 2024-09-10T04:14:16.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.565+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d8103040 con 0x7f22d8107900 2024-09-10T04:14:16.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.566+0000 7f22ceffd640 1 -- 172.21.15.46:0/649283837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f22c8005020 con 0x7f22d8107900 2024-09-10T04:14:16.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.566+0000 7f22ceffd640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f22c407abf0 0x7f22c407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.567+0000 7f22ceffd640 1 -- 172.21.15.46:0/649283837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f22c809dc50 con 0x7f22d8107900 2024-09-10T04:14:16.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.567+0000 7f22ddb48640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f22c407abf0 0x7f22c407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.567+0000 7f22ddb48640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f22c407abf0 0x7f22c407d0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f22d4031460 tx=0x7f22d4009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:16.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.571+0000 7f22ceffd640 1 -- 172.21.15.46:0/649283837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f22c8014030 con 0x7f22d8107900 2024-09-10T04:14:16.736 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:16 smithi070 ceph-mon[79774]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:16.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.797+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f22d8109e70 con 0x7f22d8107900 2024-09-10T04:14:16.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.798+0000 7f22ceffd640 1 -- 172.21.15.46:0/649283837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f22c8069370 con 0x7f22d8107900 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:14:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:14:16.801 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:14:16.802 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:16.803 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:14:16.803 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:16.803 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:16.803 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:16.803 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:16.803 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f22c407abf0 msgr2=0x7f22c407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f22c407abf0 0x7f22c407d0b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f22d4031460 tx=0x7f22d4009560 comp rx=0 tx=0).stop 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 msgr2=0x7f22d819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d819db60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c8f0 tx=0x7f22c800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 shutdown_connections 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f22c407abf0 0x7f22c407d0b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d8107900 0x7f22d819db60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 --2- 172.21.15.46:0/649283837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22d8101900 0x7f22d819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.800+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 >> 172.21.15.46:0/649283837 conn(0x7f22d80fd600 msgr2=0x7f22d80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.801+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 shutdown_connections 2024-09-10T04:14:16.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.801+0000 7f22dfdac640 1 -- 172.21.15.46:0/649283837 wait complete. 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.886+0000 7f414c14e640 1 -- 172.21.15.46:0/1435488701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 msgr2=0x7f4144100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.886+0000 7f414c14e640 1 --2- 172.21.15.46:0/1435488701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f4144100c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4134009880 tx=0x7f413402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.886+0000 7f414c14e640 1 -- 172.21.15.46:0/1435488701 shutdown_connections 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.886+0000 7f414c14e640 1 --2- 172.21.15.46:0/1435488701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f4144100c60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.886+0000 7f414c14e640 1 --2- 172.21.15.46:0/1435488701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4144106800 0x7f4144106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.886+0000 7f414c14e640 1 -- 172.21.15.46:0/1435488701 >> 172.21.15.46:0/1435488701 conn(0x7f41440fc520 msgr2=0x7f41440fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.886+0000 7f414c14e640 1 -- 172.21.15.46:0/1435488701 shutdown_connections 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.887+0000 7f414c14e640 1 -- 172.21.15.46:0/1435488701 wait complete. 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.887+0000 7f414c14e640 1 Processor -- start 2024-09-10T04:14:16.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.887+0000 7f414c14e640 1 -- start start 2024-09-10T04:14:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.887+0000 7f414c14e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f414419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.887+0000 7f414c14e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4144106800 0x7f414419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.887+0000 7f414c14e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f414410dcb0 con 0x7f4144100800 2024-09-10T04:14:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.887+0000 7f414c14e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f414410db30 con 0x7f4144106800 2024-09-10T04:14:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f4149eea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f414419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f4149eea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f414419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54404/0 (socket says 172.21.15.46:54404) 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f4149eea640 1 -- 172.21.15.46:0/4016613683 learned_addr learned my addr 172.21.15.46:0/4016613683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f41496e9640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4144106800 0x7f414419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f4149eea640 1 -- 172.21.15.46:0/4016613683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4144106800 msgr2=0x7f414419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f4149eea640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4144106800 0x7f414419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f4149eea640 1 -- 172.21.15.46:0/4016613683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41441999d0 con 0x7f4144100800 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f4149eea640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f414419f810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f414000d8d0 tx=0x7f414000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f413affd640 1 -- 172.21.15.46:0/4016613683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4140017070 con 0x7f4144100800 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f413affd640 1 -- 172.21.15.46:0/4016613683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4140010460 con 0x7f4144100800 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4144199cc0 con 0x7f4144100800 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f413affd640 1 -- 172.21.15.46:0/4016613683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f414000f5f0 con 0x7f4144100800 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.888+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414419a220 con 0x7f4144100800 2024-09-10T04:14:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.889+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4144101f40 con 0x7f4144100800 2024-09-10T04:14:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.890+0000 7f413affd640 1 -- 172.21.15.46:0/4016613683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4140009a30 con 0x7f4144100800 2024-09-10T04:14:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.891+0000 7f413affd640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f413007ac00 0x7f413007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.891+0000 7f413affd640 1 -- 172.21.15.46:0/4016613683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f414009db60 con 0x7f4144100800 2024-09-10T04:14:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.891+0000 7f41496e9640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f413007ac00 0x7f413007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.891+0000 7f41496e9640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f413007ac00 0x7f413007d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f414419b1c0 tx=0x7f41340095c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:16.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:16.895+0000 7f413affd640 1 -- 172.21.15.46:0/4016613683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f41400692a0 con 0x7f4144100800 2024-09-10T04:14:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.104+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f414419a4d0 con 0x7f413007ac00 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.105+0000 7f413affd640 1 -- 172.21.15.46:0/4016613683 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f414419a4d0 con 0x7f413007ac00 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:14:17.107 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:14:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:14:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:14:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:14:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:14:17.108 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:14:17.108 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:14:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f413007ac00 msgr2=0x7f413007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f413007ac00 0x7f413007d0c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f414419b1c0 tx=0x7f41340095c0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 msgr2=0x7f414419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f414419f810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f414000d8d0 tx=0x7f414000dd90 comp rx=0 tx=0).stop 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 shutdown_connections 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f413007ac00 0x7f413007d0c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4144106800 0x7f414419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 --2- 172.21.15.46:0/4016613683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4144100800 0x7f414419f810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 >> 172.21.15.46:0/4016613683 conn(0x7f41440fc520 msgr2=0x7f41440fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 shutdown_connections 2024-09-10T04:14:17.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.108+0000 7f414c14e640 1 -- 172.21.15.46:0/4016613683 wait complete. 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 -- 172.21.15.46:0/3123071205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54101900 msgr2=0x7f7e54101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/3123071205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54101900 0x7f7e54101d60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009880 tx=0x7f7e3c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 -- 172.21.15.46:0/3123071205 shutdown_connections 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/3123071205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54101900 0x7f7e54101d60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/3123071205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e54107900 0x7f7e54107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 -- 172.21.15.46:0/3123071205 >> 172.21.15.46:0/3123071205 conn(0x7f7e540fd600 msgr2=0x7f7e540ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 -- 172.21.15.46:0/3123071205 shutdown_connections 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.192+0000 7f7e5ab56640 1 -- 172.21.15.46:0/3123071205 wait complete. 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e5ab56640 1 Processor -- start 2024-09-10T04:14:17.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e5ab56640 1 -- start start 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e5ab56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e54101900 0x7f7e5419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e5ab56640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54107900 0x7f7e5419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e5ab56640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410edd0 con 0x7f7e54107900 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e5ab56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5410ec50 con 0x7f7e54101900 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e53fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54107900 0x7f7e5419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e53fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54107900 0x7f7e5419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54420/0 (socket says 172.21.15.46:54420) 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e53fff640 1 -- 172.21.15.46:0/1675890068 learned_addr learned my addr 172.21.15.46:0/1675890068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e588f2640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e54101900 0x7f7e5419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e53fff640 1 -- 172.21.15.46:0/1675890068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e54101900 msgr2=0x7f7e5419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e53fff640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e54101900 0x7f7e5419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.193+0000 7f7e53fff640 1 -- 172.21.15.46:0/1675890068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e54197800 con 0x7f7e54107900 2024-09-10T04:14:17.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.194+0000 7f7e53fff640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54107900 0x7f7e5419dbc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c031460 tx=0x7f7e3c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.194+0000 7f7e51ffb640 1 -- 172.21.15.46:0/1675890068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e3c03f070 con 0x7f7e54107900 2024-09-10T04:14:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.194+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e54197a90 con 0x7f7e54107900 2024-09-10T04:14:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.194+0000 7f7e51ffb640 1 -- 172.21.15.46:0/1675890068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e3c004cf0 con 0x7f7e54107900 2024-09-10T04:14:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.194+0000 7f7e51ffb640 1 -- 172.21.15.46:0/1675890068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e3c035030 con 0x7f7e54107900 2024-09-10T04:14:17.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.194+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e54197fc0 con 0x7f7e54107900 2024-09-10T04:14:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.195+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e54103040 con 0x7f7e54107900 2024-09-10T04:14:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.196+0000 7f7e51ffb640 1 -- 172.21.15.46:0/1675890068 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7e3c042070 con 0x7f7e54107900 2024-09-10T04:14:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.196+0000 7f7e51ffb640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e4007abf0 0x7f7e4007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.197+0000 7f7e51ffb640 1 -- 172.21.15.46:0/1675890068 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f7e3c0be7f0 con 0x7f7e54107900 2024-09-10T04:14:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.197+0000 7f7e588f2640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e4007abf0 0x7f7e4007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:17.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.197+0000 7f7e588f2640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e4007abf0 0x7f7e4007d0b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e44005fd0 tx=0x7f7e44005e20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.201+0000 7f7e51ffb640 1 -- 172.21.15.46:0/1675890068 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7e3c089e90 con 0x7f7e54107900 2024-09-10T04:14:17.438 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:17 smithi046 ceph-mon[109506]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:17.438 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:17 smithi046 ceph-mon[109506]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:17.438 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:17 smithi046 ceph-mon[109506]: pgmap v258: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:17.438 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:17 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3361932109' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:17.438 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:17 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/649283837' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:14:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.463+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f7e54198d30 con 0x7f7e54107900 2024-09-10T04:14:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.463+0000 7f7e51ffb640 1 -- 172.21.15.46:0/1675890068 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f7e3c08ed40 con 0x7f7e54107900 2024-09-10T04:14:17.465 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:14:17.465 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:14:17.465 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:14:17.465 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:14:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:14:17.466 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:14:17.466 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:14:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e4007abf0 msgr2=0x7f7e4007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e4007abf0 0x7f7e4007d0b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7e44005fd0 tx=0x7f7e44005e20 comp rx=0 tx=0).stop 2024-09-10T04:14:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54107900 msgr2=0x7f7e5419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54107900 0x7f7e5419dbc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c031460 tx=0x7f7e3c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 shutdown_connections 2024-09-10T04:14:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e4007abf0 0x7f7e4007d0b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e54107900 0x7f7e5419dbc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 --2- 172.21.15.46:0/1675890068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e54101900 0x7f7e5419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.465+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 >> 172.21.15.46:0/1675890068 conn(0x7f7e540fd600 msgr2=0x7f7e540fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.466+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 shutdown_connections 2024-09-10T04:14:17.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:17.466+0000 7f7e5ab56640 1 -- 172.21.15.46:0/1675890068 wait complete. 2024-09-10T04:14:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:17 smithi070 ceph-mon[79774]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:17 smithi070 ceph-mon[79774]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:17 smithi070 ceph-mon[79774]: pgmap v258: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:17 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3361932109' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:17 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/649283837' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:14:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:18 smithi070 ceph-mon[79774]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:18 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1675890068' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:14:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:18 smithi046 ceph-mon[109506]: from='client.34386 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:18 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1675890068' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:14:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:19 smithi070 ceph-mon[79774]: pgmap v259: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:19.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:19 smithi046 ceph-mon[109506]: pgmap v259: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:21 smithi070 ceph-mon[79774]: pgmap v260: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:21 smithi046 ceph-mon[109506]: pgmap v260: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:23 smithi046 ceph-mon[109506]: pgmap v261: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:23 smithi070 ceph-mon[79774]: pgmap v261: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:25 smithi070 ceph-mon[79774]: pgmap v262: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:25 smithi046 ceph-mon[109506]: pgmap v262: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:27 smithi070 ceph-mon[79774]: pgmap v263: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:27 smithi046 ceph-mon[109506]: pgmap v263: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:29 smithi070 ceph-mon[79774]: pgmap v264: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:29 smithi046 ceph-mon[109506]: pgmap v264: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:31 smithi070 ceph-mon[79774]: pgmap v265: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:31 smithi046 ceph-mon[109506]: pgmap v265: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:33 smithi070 ceph-mon[79774]: pgmap v266: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:33 smithi046 ceph-mon[109506]: pgmap v266: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:35 smithi046 ceph-mon[109506]: pgmap v267: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:35 smithi070 ceph-mon[79774]: pgmap v267: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:37.746 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:37 smithi046 ceph-mon[109506]: pgmap v268: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:37 smithi070 ceph-mon[79774]: pgmap v268: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:39 smithi070 ceph-mon[79774]: pgmap v269: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:39 smithi046 ceph-mon[109506]: pgmap v269: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:41 smithi046 ceph-mon[109506]: pgmap v270: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:41 smithi070 ceph-mon[79774]: pgmap v270: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:43 smithi070 ceph-mon[79774]: pgmap v271: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:43 smithi046 ceph-mon[109506]: pgmap v271: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:45 smithi070 ceph-mon[79774]: pgmap v272: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:45 smithi046 ceph-mon[109506]: pgmap v272: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:47.567 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:47 smithi046 ceph-mon[109506]: pgmap v273: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 -- 172.21.15.46:0/441203072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 msgr2=0x7f9f78100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 --2- 172.21.15.46:0/441203072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f78100c50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9f68009880 tx=0x7f9f6802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 -- 172.21.15.46:0/441203072 shutdown_connections 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 --2- 172.21.15.46:0/441203072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f78100c50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 --2- 172.21.15.46:0/441203072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f781067f0 0x7f9f78106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 -- 172.21.15.46:0/441203072 >> 172.21.15.46:0/441203072 conn(0x7f9f780fc550 msgr2=0x7f9f780fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 -- 172.21.15.46:0/441203072 shutdown_connections 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.564+0000 7f9f807e9640 1 -- 172.21.15.46:0/441203072 wait complete. 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f807e9640 1 Processor -- start 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f807e9640 1 -- start start 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f807e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f7819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f807e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f781067f0 0x7f9f7819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f807e9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7810dcc0 con 0x7f9f781007f0 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f807e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7810db40 con 0x7f9f781067f0 2024-09-10T04:14:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f7e585640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f7819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f7e585640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f7819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36272/0 (socket says 172.21.15.46:36272) 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.565+0000 7f9f7e585640 1 -- 172.21.15.46:0/1848527723 learned_addr learned my addr 172.21.15.46:0/1848527723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f7dd84640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f781067f0 0x7f9f7819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f7e585640 1 -- 172.21.15.46:0/1848527723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f781067f0 msgr2=0x7f9f7819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f7e585640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f781067f0 0x7f9f7819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f7e585640 1 -- 172.21.15.46:0/1848527723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f781966b0 con 0x7f9f781007f0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f7e585640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f7819c5e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400d6f0 tx=0x7f9f7400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f6f7fe640 1 -- 172.21.15.46:0/1848527723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9f74017070 con 0x7f9f781007f0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f6f7fe640 1 -- 172.21.15.46:0/1848527723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9f7400de00 con 0x7f9f781007f0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f781969a0 con 0x7f9f781007f0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f6f7fe640 1 -- 172.21.15.46:0/1848527723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9f74004bb0 con 0x7f9f781007f0 2024-09-10T04:14:47.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.566+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f78196f00 con 0x7f9f781007f0 2024-09-10T04:14:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.567+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f78101f30 con 0x7f9f781007f0 2024-09-10T04:14:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.568+0000 7f9f6f7fe640 1 -- 172.21.15.46:0/1848527723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9f74009a30 con 0x7f9f781007f0 2024-09-10T04:14:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.568+0000 7f9f6f7fe640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9f6407ac40 0x7f9f6407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.569+0000 7f9f6f7fe640 1 -- 172.21.15.46:0/1848527723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f9f7409d890 con 0x7f9f781007f0 2024-09-10T04:14:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.569+0000 7f9f7dd84640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9f6407ac40 0x7f9f6407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.569+0000 7f9f7dd84640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9f6407ac40 0x7f9f6407d100 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f78197f00 tx=0x7f9f68031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:47.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.573+0000 7f9f6f7fe640 1 -- 172.21.15.46:0/1848527723 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9f74068fb0 con 0x7f9f781007f0 2024-09-10T04:14:47.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.780+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9f781971b0 con 0x7f9f6407ac40 2024-09-10T04:14:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.782+0000 7f9f6f7fe640 1 -- 172.21.15.46:0/1848527723 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f9f781971b0 con 0x7f9f6407ac40 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9f6407ac40 msgr2=0x7f9f6407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9f6407ac40 0x7f9f6407d100 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9f78197f00 tx=0x7f9f68031400 comp rx=0 tx=0).stop 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 msgr2=0x7f9f7819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f7819c5e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f7400d6f0 tx=0x7f9f7400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 shutdown_connections 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9f6407ac40 0x7f9f6407d100 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f781067f0 0x7f9f7819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 --2- 172.21.15.46:0/1848527723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f781007f0 0x7f9f7819c5e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 >> 172.21.15.46:0/1848527723 conn(0x7f9f780fc550 msgr2=0x7f9f780fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:47.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 shutdown_connections 2024-09-10T04:14:47.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.784+0000 7f9f807e9640 1 -- 172.21.15.46:0/1848527723 wait complete. 2024-09-10T04:14:47.794 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:14:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:47 smithi070 ceph-mon[79774]: pgmap v273: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 -- 172.21.15.46:0/95912797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 msgr2=0x7f67bc102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 --2- 172.21.15.46:0/95912797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc102ea0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009880 tx=0x7f67ac02a530 comp rx=0 tx=0).stop 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 -- 172.21.15.46:0/95912797 shutdown_connections 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 --2- 172.21.15.46:0/95912797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc1033e0 0x7f67bc10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 --2- 172.21.15.46:0/95912797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc102ea0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 -- 172.21.15.46:0/95912797 >> 172.21.15.46:0/95912797 conn(0x7f67bc0fc520 msgr2=0x7f67bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 -- 172.21.15.46:0/95912797 shutdown_connections 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 -- 172.21.15.46:0/95912797 wait complete. 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 Processor -- start 2024-09-10T04:14:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.870+0000 7f67c21a9640 1 -- start start 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67c21a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67c21a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc1033e0 0x7f67bc100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67c21a9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10edf0 con 0x7f67bc102ac0 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67c21a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc10ec70 con 0x7f67bc1033e0 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67bb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67bb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36302/0 (socket says 172.21.15.46:36302) 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67bb7fe640 1 -- 172.21.15.46:0/3636894064 learned_addr learned my addr 172.21.15.46:0/3636894064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67baffd640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc1033e0 0x7f67bc100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67bb7fe640 1 -- 172.21.15.46:0/3636894064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc1033e0 msgr2=0x7f67bc100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67bb7fe640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc1033e0 0x7f67bc100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67bb7fe640 1 -- 172.21.15.46:0/3636894064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1011a0 con 0x7f67bc102ac0 2024-09-10T04:14:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.871+0000 7f67bb7fe640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc100720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67ac031460 tx=0x7f67ac004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.872+0000 7f67b8ff9640 1 -- 172.21.15.46:0/3636894064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f67ac03f070 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.872+0000 7f67b8ff9640 1 -- 172.21.15.46:0/3636894064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67ac0035f0 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.872+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc101430 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.872+0000 7f67b8ff9640 1 -- 172.21.15.46:0/3636894064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f67ac035030 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.872+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a3dc0 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.873+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc06aa90 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.873+0000 7f67b8ff9640 1 -- 172.21.15.46:0/3636894064 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f67ac042070 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.874+0000 7f67b8ff9640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f67a407ac40 0x7f67a407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.874+0000 7f67b8ff9640 1 -- 172.21.15.46:0/3636894064 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f67ac0be560 con 0x7f67bc102ac0 2024-09-10T04:14:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.874+0000 7f67baffd640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f67a407ac40 0x7f67a407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:47.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.875+0000 7f67baffd640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f67a407ac40 0x7f67a407d100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67bc1021b0 tx=0x7f67b0005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:47.878+0000 7f67b8ff9640 1 -- 172.21.15.46:0/3636894064 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f67ac089d20 con 0x7f67bc102ac0 2024-09-10T04:14:48.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.087+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f67bc062630 con 0x7f67a407ac40 2024-09-10T04:14:48.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.089+0000 7f67b8ff9640 1 -- 172.21.15.46:0/3636894064 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f67bc062630 con 0x7f67a407ac40 2024-09-10T04:14:48.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.090+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f67a407ac40 msgr2=0x7f67a407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.090+0000 7f67c21a9640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f67a407ac40 0x7f67a407d100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f67bc1021b0 tx=0x7f67b0005f90 comp rx=0 tx=0).stop 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.090+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 msgr2=0x7f67bc100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.090+0000 7f67c21a9640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc100720 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67ac031460 tx=0x7f67ac004530 comp rx=0 tx=0).stop 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.091+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 shutdown_connections 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.091+0000 7f67c21a9640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f67a407ac40 0x7f67a407d100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.091+0000 7f67c21a9640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67bc1033e0 0x7f67bc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.091+0000 7f67c21a9640 1 --2- 172.21.15.46:0/3636894064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67bc102ac0 0x7f67bc100720 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.091+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 >> 172.21.15.46:0/3636894064 conn(0x7f67bc0fc520 msgr2=0x7f67bc108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.091+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 shutdown_connections 2024-09-10T04:14:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.091+0000 7f67c21a9640 1 -- 172.21.15.46:0/3636894064 wait complete. 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 -- 172.21.15.46:0/2287008255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc106800 msgr2=0x7feadc106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 --2- 172.21.15.46:0/2287008255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc106800 0x7feadc106be0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fead4009880 tx=0x7fead402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 -- 172.21.15.46:0/2287008255 shutdown_connections 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 --2- 172.21.15.46:0/2287008255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feadc100800 0x7feadc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 --2- 172.21.15.46:0/2287008255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc106800 0x7feadc106be0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 -- 172.21.15.46:0/2287008255 >> 172.21.15.46:0/2287008255 conn(0x7feadc0fc520 msgr2=0x7feadc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 -- 172.21.15.46:0/2287008255 shutdown_connections 2024-09-10T04:14:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 -- 172.21.15.46:0/2287008255 wait complete. 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 Processor -- start 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.176+0000 7feae3e76640 1 -- start start 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae3e76640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc100800 0x7feadc107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae3e76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feadc106800 0x7feadc107c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae3e76640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feadc10dcb0 con 0x7feadc100800 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae3e76640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feadc10db30 con 0x7feadc106800 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1c12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc100800 0x7feadc107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1c12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc100800 0x7feadc107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36326/0 (socket says 172.21.15.46:36326) 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1c12640 1 -- 172.21.15.46:0/3495037092 learned_addr learned my addr 172.21.15.46:0/3495037092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1411640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feadc106800 0x7feadc107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1c12640 1 -- 172.21.15.46:0/3495037092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feadc106800 msgr2=0x7feadc107c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1c12640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feadc106800 0x7feadc107c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1c12640 1 -- 172.21.15.46:0/3495037092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1081b0 con 0x7feadc100800 2024-09-10T04:14:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.177+0000 7feae1c12640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc100800 0x7feadc107730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fead4031460 tx=0x7fead4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.178+0000 7feadaffd640 1 -- 172.21.15.46:0/3495037092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fead403f070 con 0x7feadc100800 2024-09-10T04:14:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.178+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc108440 con 0x7feadc100800 2024-09-10T04:14:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.178+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a3d90 con 0x7feadc100800 2024-09-10T04:14:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.178+0000 7feadaffd640 1 -- 172.21.15.46:0/3495037092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fead40035f0 con 0x7feadc100800 2024-09-10T04:14:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.178+0000 7feadaffd640 1 -- 172.21.15.46:0/3495037092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fead4035030 con 0x7feadc100800 2024-09-10T04:14:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.179+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc101f40 con 0x7feadc100800 2024-09-10T04:14:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.179+0000 7feadaffd640 1 -- 172.21.15.46:0/3495037092 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fead4042070 con 0x7feadc100800 2024-09-10T04:14:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.180+0000 7feadaffd640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feac407ac40 0x7feac407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.180+0000 7feadaffd640 1 -- 172.21.15.46:0/3495037092 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fead40be550 con 0x7feadc100800 2024-09-10T04:14:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.180+0000 7feae1411640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feac407ac40 0x7feac407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.181+0000 7feae1411640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feac407ac40 0x7feac407d100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feadc1091c0 tx=0x7fead0005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:48.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.184+0000 7feadaffd640 1 -- 172.21.15.46:0/3495037092 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fead4089d10 con 0x7feadc100800 2024-09-10T04:14:48.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.377+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7feadc06aa00 con 0x7feac407ac40 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.387+0000 7feadaffd640 1 -- 172.21.15.46:0/3495037092 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7feadc06aa00 con 0x7feac407ac40 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (6m) 51s ago 17m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (8m) 51s ago 17m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (8m) 38s ago 17m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (4m) 51s ago 17m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (4m) 38s ago 17m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:14:48.389 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (5m) 51s ago 17m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (57s) 51s ago 12m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (73s) 51s ago 12m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (41s) 38s ago 12m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (12m) 38s ago 12m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (8m) 51s ago 19m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (7m) 38s ago 16m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (5m) 51s ago 19m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (5m) 38s ago 16m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (7m) 51s ago 17m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 38s ago 16m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (4m) 51s ago 15m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:14:48.390 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (3m) 51s ago 15m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:14:48.391 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (3m) 51s ago 14m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:14:48.391 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (2m) 38s ago 14m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:14:48.391 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (2m) 38s ago 13m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:14:48.391 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (90s) 38s ago 13m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:14:48.391 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (6m) 51s ago 17m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feac407ac40 msgr2=0x7feac407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feac407ac40 0x7feac407d100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feadc1091c0 tx=0x7fead0005f90 comp rx=0 tx=0).stop 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc100800 msgr2=0x7feadc107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc100800 0x7feadc107730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fead4031460 tx=0x7fead4004530 comp rx=0 tx=0).stop 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 shutdown_connections 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feac407ac40 0x7feac407d100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feadc106800 0x7feadc107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 --2- 172.21.15.46:0/3495037092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feadc100800 0x7feadc107730 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.389+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 >> 172.21.15.46:0/3495037092 conn(0x7feadc0fc520 msgr2=0x7feadc06a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.390+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 shutdown_connections 2024-09-10T04:14:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.390+0000 7feae3e76640 1 -- 172.21.15.46:0/3495037092 wait complete. 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 -- 172.21.15.46:0/3495663640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881007f0 msgr2=0x7f8288100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3495663640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881007f0 0x7f8288100c50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f827c009880 tx=0x7f827c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 -- 172.21.15.46:0/3495663640 shutdown_connections 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3495663640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881007f0 0x7f8288100c50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3495663640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82881067f0 0x7f8288106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 -- 172.21.15.46:0/3495663640 >> 172.21.15.46:0/3495663640 conn(0x7f82880fc550 msgr2=0x7f82880fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 -- 172.21.15.46:0/3495663640 shutdown_connections 2024-09-10T04:14:48.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 -- 172.21.15.46:0/3495663640 wait complete. 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.473+0000 7f828fb1c640 1 Processor -- start 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828fb1c640 1 -- start start 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828fb1c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82881007f0 0x7f828819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828fb1c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881067f0 0x7f828819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828fb1c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f828810dca0 con 0x7f82881067f0 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828fb1c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f828810db20 con 0x7f82881007f0 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828d0b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881067f0 0x7f828819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828d0b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881067f0 0x7f828819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36352/0 (socket says 172.21.15.46:36352) 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828d0b7640 1 -- 172.21.15.46:0/3026208468 learned_addr learned my addr 172.21.15.46:0/3026208468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828d8b8640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82881007f0 0x7f828819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828d0b7640 1 -- 172.21.15.46:0/3026208468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82881007f0 msgr2=0x7f828819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828d0b7640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82881007f0 0x7f828819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.474+0000 7f828d0b7640 1 -- 172.21.15.46:0/3026208468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8288197780 con 0x7f82881067f0 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.475+0000 7f828d0b7640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881067f0 0x7f828819db40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f827c031460 tx=0x7f827c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.475+0000 7f827affd640 1 -- 172.21.15.46:0/3026208468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f827c03f070 con 0x7f82881067f0 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.475+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8288197a10 con 0x7f82881067f0 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.475+0000 7f827affd640 1 -- 172.21.15.46:0/3026208468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f827c0035f0 con 0x7f82881067f0 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.475+0000 7f827affd640 1 -- 172.21.15.46:0/3026208468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f827c035030 con 0x7f82881067f0 2024-09-10T04:14:48.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.475+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8288197f40 con 0x7f82881067f0 2024-09-10T04:14:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.476+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8288101f30 con 0x7f82881067f0 2024-09-10T04:14:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.477+0000 7f827affd640 1 -- 172.21.15.46:0/3026208468 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f827c042070 con 0x7f82881067f0 2024-09-10T04:14:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.477+0000 7f827affd640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f827407abf0 0x7f827407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.478+0000 7f827affd640 1 -- 172.21.15.46:0/3026208468 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f827c0be5d0 con 0x7f82881067f0 2024-09-10T04:14:48.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.478+0000 7f828d8b8640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f827407abf0 0x7f827407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.478+0000 7f828d8b8640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f827407abf0 0x7f827407d0b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8284005d00 tx=0x7f828400b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.482+0000 7f827affd640 1 -- 172.21.15.46:0/3026208468 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f827c089d10 con 0x7f82881067f0 2024-09-10T04:14:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.740+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f8288109d90 con 0x7f82881067f0 2024-09-10T04:14:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.741+0000 7f827affd640 1 -- 172.21.15.46:0/3026208468 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f827c08ebc0 con 0x7f82881067f0 2024-09-10T04:14:48.743 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:14:48.744 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:14:48.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:14:48.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:48.744 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:14:48.744 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:14:48.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:48.745 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:14:48.745 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:14:48.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:48.745 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:14:48.745 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:14:48.745 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:14:48.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:14:48.746 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:14:48.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:14:48.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:14:48.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:14:48.746 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:14:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f827407abf0 msgr2=0x7f827407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f827407abf0 0x7f827407d0b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8284005d00 tx=0x7f828400b410 comp rx=0 tx=0).stop 2024-09-10T04:14:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881067f0 msgr2=0x7f828819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881067f0 0x7f828819db40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f827c031460 tx=0x7f827c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 shutdown_connections 2024-09-10T04:14:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f827407abf0 0x7f827407d0b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82881067f0 0x7f828819db40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 --2- 172.21.15.46:0/3026208468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82881007f0 0x7f828819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 >> 172.21.15.46:0/3026208468 conn(0x7f82880fc550 msgr2=0x7f8288069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 shutdown_connections 2024-09-10T04:14:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.743+0000 7f828fb1c640 1 -- 172.21.15.46:0/3026208468 wait complete. 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 -- 172.21.15.46:0/1299559861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68107900 msgr2=0x7efe68107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1299559861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68107900 0x7efe68107ce0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009880 tx=0x7efe5c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 -- 172.21.15.46:0/1299559861 shutdown_connections 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1299559861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe68101900 0x7efe68101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1299559861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68107900 0x7efe68107ce0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 -- 172.21.15.46:0/1299559861 >> 172.21.15.46:0/1299559861 conn(0x7efe680fd600 msgr2=0x7efe680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 -- 172.21.15.46:0/1299559861 shutdown_connections 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 -- 172.21.15.46:0/1299559861 wait complete. 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.828+0000 7efe6eadd640 1 Processor -- start 2024-09-10T04:14:48.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6eadd640 1 -- start start 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6eadd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68101900 0x7efe6819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6eadd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe68107900 0x7efe6819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6eadd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810edd0 con 0x7efe68101900 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6eadd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810ec50 con 0x7efe68107900 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6c879640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68101900 0x7efe6819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6c879640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68101900 0x7efe6819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47836/0 (socket says 172.21.15.46:47836) 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6c879640 1 -- 172.21.15.46:0/1416841867 learned_addr learned my addr 172.21.15.46:0/1416841867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe67fff640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe68107900 0x7efe6819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6c879640 1 -- 172.21.15.46:0/1416841867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe68107900 msgr2=0x7efe6819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6c879640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe68107900 0x7efe6819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6c879640 1 -- 172.21.15.46:0/1416841867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe681977a0 con 0x7efe68101900 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe67fff640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe68107900 0x7efe6819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:14:48.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.829+0000 7efe6c879640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68101900 0x7efe6819d620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe5c031460 tx=0x7efe5c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.830+0000 7efe65ffb640 1 -- 172.21.15.46:0/1416841867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe5c03f070 con 0x7efe68101900 2024-09-10T04:14:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.830+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe68197a30 con 0x7efe68101900 2024-09-10T04:14:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.830+0000 7efe65ffb640 1 -- 172.21.15.46:0/1416841867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe5c0035f0 con 0x7efe68101900 2024-09-10T04:14:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.830+0000 7efe65ffb640 1 -- 172.21.15.46:0/1416841867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe5c035030 con 0x7efe68101900 2024-09-10T04:14:48.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.830+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe68197f60 con 0x7efe68101900 2024-09-10T04:14:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.831+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe68103040 con 0x7efe68101900 2024-09-10T04:14:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.831+0000 7efe65ffb640 1 -- 172.21.15.46:0/1416841867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efe5c042070 con 0x7efe68101900 2024-09-10T04:14:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.832+0000 7efe65ffb640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe5807abf0 0x7efe5807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.832+0000 7efe65ffb640 1 -- 172.21.15.46:0/1416841867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7efe5c0be680 con 0x7efe68101900 2024-09-10T04:14:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.832+0000 7efe67fff640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe5807abf0 0x7efe5807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:48.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.832+0000 7efe67fff640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe5807abf0 0x7efe5807d0b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe680ffc20 tx=0x7efe54005e50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:48.836+0000 7efe65ffb640 1 -- 172.21.15.46:0/1416841867 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efe5c089dc0 con 0x7efe68101900 2024-09-10T04:14:49.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.059+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efe68109e70 con 0x7efe68101900 2024-09-10T04:14:49.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.060+0000 7efe65ffb640 1 -- 172.21.15.46:0/1416841867 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7efe5c08ec70 con 0x7efe68101900 2024-09-10T04:14:49.062 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:14:49.062 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:14:49.062 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:14:49.062 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:14:49.062 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:14:49.062 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:49.062 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:14:49.063 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:14:49.064 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:49.065 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:14:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:14:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe5807abf0 msgr2=0x7efe5807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe5807abf0 0x7efe5807d0b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efe680ffc20 tx=0x7efe54005e50 comp rx=0 tx=0).stop 2024-09-10T04:14:49.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68101900 msgr2=0x7efe6819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68101900 0x7efe6819d620 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe5c031460 tx=0x7efe5c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 shutdown_connections 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe5807abf0 0x7efe5807d0b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe68107900 0x7efe6819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 --2- 172.21.15.46:0/1416841867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe68101900 0x7efe6819d620 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 >> 172.21.15.46:0/1416841867 conn(0x7efe680fd600 msgr2=0x7efe680fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 shutdown_connections 2024-09-10T04:14:49.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.063+0000 7efe6eadd640 1 -- 172.21.15.46:0/1416841867 wait complete. 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.148+0000 7f367a716640 1 -- 172.21.15.46:0/1157794559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 msgr2=0x7f3674101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.148+0000 7f367a716640 1 --2- 172.21.15.46:0/1157794559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f3674101d60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3668009880 tx=0x7f366802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 -- 172.21.15.46:0/1157794559 shutdown_connections 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 --2- 172.21.15.46:0/1157794559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f3674101d60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 --2- 172.21.15.46:0/1157794559 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674107900 0x7f3674107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 -- 172.21.15.46:0/1157794559 >> 172.21.15.46:0/1157794559 conn(0x7f36740fd600 msgr2=0x7f36740ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 -- 172.21.15.46:0/1157794559 shutdown_connections 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 -- 172.21.15.46:0/1157794559 wait complete. 2024-09-10T04:14:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 Processor -- start 2024-09-10T04:14:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.149+0000 7f367a716640 1 -- start start 2024-09-10T04:14:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f367a716640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f367419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f367a716640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674107900 0x7f367419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f367a716640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f367410edd0 con 0x7f3674101900 2024-09-10T04:14:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f367a716640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f367410ec50 con 0x7f3674107900 2024-09-10T04:14:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f3673fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f367419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f3673fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f367419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47862/0 (socket says 172.21.15.46:47862) 2024-09-10T04:14:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f3673fff640 1 -- 172.21.15.46:0/71395438 learned_addr learned my addr 172.21.15.46:0/71395438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f36737fe640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674107900 0x7f367419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f3673fff640 1 -- 172.21.15.46:0/71395438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674107900 msgr2=0x7f367419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f3673fff640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674107900 0x7f367419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f3673fff640 1 -- 172.21.15.46:0/71395438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36741999b0 con 0x7f3674101900 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f36737fe640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674107900 0x7f367419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f3673fff640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f367419f7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f366400d6f0 tx=0x7f366400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f36717fa640 1 -- 172.21.15.46:0/71395438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3664017070 con 0x7f3674101900 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f36717fa640 1 -- 172.21.15.46:0/71395438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f366400de00 con 0x7f3674101900 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.150+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3674199ca0 con 0x7f3674101900 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.151+0000 7f36717fa640 1 -- 172.21.15.46:0/71395438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3664004bb0 con 0x7f3674101900 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.151+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367419a230 con 0x7f3674101900 2024-09-10T04:14:49.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.151+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3674103040 con 0x7f3674101900 2024-09-10T04:14:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.152+0000 7f36717fa640 1 -- 172.21.15.46:0/71395438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3664004e60 con 0x7f3674101900 2024-09-10T04:14:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.153+0000 7f36717fa640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f365c07abf0 0x7f365c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.153+0000 7f36737fe640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f365c07abf0 0x7f365c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.153+0000 7f36717fa640 1 -- 172.21.15.46:0/71395438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f366409d8d0 con 0x7f3674101900 2024-09-10T04:14:49.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.153+0000 7f36737fe640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f365c07abf0 0x7f365c07d0b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3668031460 tx=0x7f36680095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:49.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.157+0000 7f36717fa640 1 -- 172.21.15.46:0/71395438 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3664069010 con 0x7f3674101900 2024-09-10T04:14:49.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.365+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f367419a5a0 con 0x7f365c07abf0 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.366+0000 7f36717fa640 1 -- 172.21.15.46:0/71395438 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f367419a5a0 con 0x7f365c07abf0 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:14:49.368 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:14:49.369 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:14:49.369 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:14:49.369 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:14:49.369 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:14:49.369 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:14:49.369 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:14:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f365c07abf0 msgr2=0x7f365c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f365c07abf0 0x7f365c07d0b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3668031460 tx=0x7f36680095f0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 msgr2=0x7f367419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f367419f7f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f366400d6f0 tx=0x7f366400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 shutdown_connections 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f365c07abf0 0x7f365c07d0b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3674107900 0x7f367419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 --2- 172.21.15.46:0/71395438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3674101900 0x7f367419f7f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.368+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 >> 172.21.15.46:0/71395438 conn(0x7f36740fd600 msgr2=0x7f3674074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.369+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 shutdown_connections 2024-09-10T04:14:49.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.369+0000 7f367a716640 1 -- 172.21.15.46:0/71395438 wait complete. 2024-09-10T04:14:49.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.453+0000 7f7193eae640 1 -- 172.21.15.46:0/2467235053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c100800 msgr2=0x7f718c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.453+0000 7f7193eae640 1 --2- 172.21.15.46:0/2467235053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c100800 0x7f718c100c60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7180009880 tx=0x7f718002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.453+0000 7f7193eae640 1 -- 172.21.15.46:0/2467235053 shutdown_connections 2024-09-10T04:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.453+0000 7f7193eae640 1 --2- 172.21.15.46:0/2467235053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c100800 0x7f718c100c60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.453+0000 7f7193eae640 1 --2- 172.21.15.46:0/2467235053 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c106800 0x7f718c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.453+0000 7f7193eae640 1 -- 172.21.15.46:0/2467235053 >> 172.21.15.46:0/2467235053 conn(0x7f718c0fc520 msgr2=0x7f718c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.454+0000 7f7193eae640 1 -- 172.21.15.46:0/2467235053 shutdown_connections 2024-09-10T04:14:49.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.454+0000 7f7193eae640 1 -- 172.21.15.46:0/2467235053 wait complete. 2024-09-10T04:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.454+0000 7f7193eae640 1 Processor -- start 2024-09-10T04:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.454+0000 7f7193eae640 1 -- start start 2024-09-10T04:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.454+0000 7f7193eae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c100800 0x7f718c0fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7193eae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c106800 0x7f718c0fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:49.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7193eae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10dcd0 con 0x7f718c106800 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7193eae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10db50 con 0x7f718c100800 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191449640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c106800 0x7f718c0fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191449640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c106800 0x7f718c0fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47880/0 (socket says 172.21.15.46:47880) 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191449640 1 -- 172.21.15.46:0/1902947358 learned_addr learned my addr 172.21.15.46:0/1902947358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191c4a640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c100800 0x7f718c0fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191449640 1 -- 172.21.15.46:0/1902947358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c100800 msgr2=0x7f718c0fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191449640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c100800 0x7f718c0fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191449640 1 -- 172.21.15.46:0/1902947358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c0fd670 con 0x7f718c106800 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f7191449640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c106800 0x7f718c0fd130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7180031460 tx=0x7f71800045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f718affd640 1 -- 172.21.15.46:0/1902947358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f718003f070 con 0x7f718c106800 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.455+0000 7f718affd640 1 -- 172.21.15.46:0/1902947358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7180004b10 con 0x7f718c106800 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.456+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c0fd900 con 0x7f718c106800 2024-09-10T04:14:49.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.456+0000 7f718affd640 1 -- 172.21.15.46:0/1902947358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7180035030 con 0x7f718c106800 2024-09-10T04:14:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.456+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c1a3db0 con 0x7f718c106800 2024-09-10T04:14:49.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.457+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c101f40 con 0x7f718c106800 2024-09-10T04:14:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.457+0000 7f718affd640 1 -- 172.21.15.46:0/1902947358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7180042070 con 0x7f718c106800 2024-09-10T04:14:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.458+0000 7f718affd640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407ac50 0x7f717407d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:14:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.458+0000 7f7191c4a640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407ac50 0x7f717407d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:14:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.458+0000 7f718affd640 1 -- 172.21.15.46:0/1902947358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f71800be6f0 con 0x7f718c106800 2024-09-10T04:14:49.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.459+0000 7f7191c4a640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407ac50 0x7f717407d110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f717c005fd0 tx=0x7f717c009680 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:14:49.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.462+0000 7f718affd640 1 -- 172.21.15.46:0/1902947358 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7180089e10 con 0x7f718c106800 2024-09-10T04:14:49.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.729+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f718c109d80 con 0x7f718c106800 2024-09-10T04:14:49.731 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:49 smithi046 ceph-mon[109506]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:49.731 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:49 smithi046 ceph-mon[109506]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:49.731 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:49 smithi046 ceph-mon[109506]: pgmap v274: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:49.731 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:49 smithi046 ceph-mon[109506]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:49.731 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:49 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3026208468' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:49.731 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:49 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1416841867' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.729+0000 7f718affd640 1 -- 172.21.15.46:0/1902947358 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f718008ecc0 con 0x7f718c106800 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:14:49.732 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:14:49.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407ac50 msgr2=0x7f717407d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407ac50 0x7f717407d110 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f717c005fd0 tx=0x7f717c009680 comp rx=0 tx=0).stop 2024-09-10T04:14:49.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c106800 msgr2=0x7f718c0fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:14:49.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c106800 0x7f718c0fd130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7180031460 tx=0x7f71800045e0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 shutdown_connections 2024-09-10T04:14:49.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407ac50 0x7f717407d110 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c106800 0x7f718c0fd130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 --2- 172.21.15.46:0/1902947358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c100800 0x7f718c0fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:14:49.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 >> 172.21.15.46:0/1902947358 conn(0x7f718c0fc520 msgr2=0x7f718c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:14:49.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 shutdown_connections 2024-09-10T04:14:49.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:14:49.731+0000 7f7193eae640 1 -- 172.21.15.46:0/1902947358 wait complete. 2024-09-10T04:14:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:49 smithi070 ceph-mon[79774]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:49 smithi070 ceph-mon[79774]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:49 smithi070 ceph-mon[79774]: pgmap v274: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:49 smithi070 ceph-mon[79774]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:49 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3026208468' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:14:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:49 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1416841867' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:14:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:50 smithi046 ceph-mon[109506]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:50 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1902947358' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:14:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:50 smithi070 ceph-mon[79774]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:14:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:50 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1902947358' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:14:51.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:51 smithi070 ceph-mon[79774]: pgmap v275: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:51.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:51 smithi046 ceph-mon[109506]: pgmap v275: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:53 smithi046 ceph-mon[109506]: pgmap v276: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:53 smithi070 ceph-mon[79774]: pgmap v276: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:14:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:55 smithi070 ceph-mon[79774]: pgmap v277: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:55 smithi046 ceph-mon[109506]: pgmap v277: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:57 smithi070 ceph-mon[79774]: pgmap v278: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:57 smithi046 ceph-mon[109506]: pgmap v278: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:14:59 smithi046 ceph-mon[109506]: pgmap v279: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:14:59.733 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:14:59 smithi070 ceph-mon[79774]: pgmap v279: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:01 smithi070 ceph-mon[79774]: pgmap v280: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:01 smithi046 ceph-mon[109506]: pgmap v280: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:03 smithi070 ceph-mon[79774]: pgmap v281: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:03 smithi046 ceph-mon[109506]: pgmap v281: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:05.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:05 smithi046 ceph-mon[109506]: pgmap v282: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:05 smithi070 ceph-mon[79774]: pgmap v282: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:07 smithi070 ceph-mon[79774]: pgmap v283: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:07 smithi046 ceph-mon[109506]: pgmap v283: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:09 smithi070 ceph-mon[79774]: pgmap v284: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:09 smithi046 ceph-mon[109506]: pgmap v284: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:11 smithi046 ceph-mon[109506]: pgmap v285: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:11 smithi070 ceph-mon[79774]: pgmap v285: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:13.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:13 smithi046 ceph-mon[109506]: pgmap v286: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:13 smithi070 ceph-mon[79774]: pgmap v286: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:15 smithi070 ceph-mon[79774]: pgmap v287: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:15 smithi046 ceph-mon[109506]: pgmap v287: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:17 smithi046 ceph-mon[109506]: pgmap v288: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:17 smithi070 ceph-mon[79774]: pgmap v288: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.830+0000 7f93e2f01640 1 -- 172.21.15.46:0/1683139317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 msgr2=0x7f93dc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.830+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1683139317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc10b8d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009880 tx=0x7f93c802a530 comp rx=0 tx=0).stop 2024-09-10T04:15:19.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.831+0000 7f93e2f01640 1 -- 172.21.15.46:0/1683139317 shutdown_connections 2024-09-10T04:15:19.832 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:19 smithi046 ceph-mon[109506]: pgmap v289: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:19.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.831+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1683139317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc10b8d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:19.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.831+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1683139317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc102ac0 0x7f93dc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:19.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.831+0000 7f93e2f01640 1 -- 172.21.15.46:0/1683139317 >> 172.21.15.46:0/1683139317 conn(0x7f93dc0fc520 msgr2=0x7f93dc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.831+0000 7f93e2f01640 1 -- 172.21.15.46:0/1683139317 shutdown_connections 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.831+0000 7f93e2f01640 1 -- 172.21.15.46:0/1683139317 wait complete. 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.831+0000 7f93e2f01640 1 Processor -- start 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93e2f01640 1 -- start start 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93e2f01640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc102ac0 0x7f93dc19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93e2f01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93e2f01640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc10edd0 con 0x7f93dc1033e0 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93e2f01640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc10ec50 con 0x7f93dc102ac0 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45986/0 (socket says 172.21.15.46:45986) 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93dbfff640 1 -- 172.21.15.46:0/1356513693 learned_addr learned my addr 172.21.15.46:0/1356513693 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:19.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93e0c9d640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc102ac0 0x7f93dc19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93dbfff640 1 -- 172.21.15.46:0/1356513693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc102ac0 msgr2=0x7f93dc19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93dbfff640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc102ac0 0x7f93dc19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.832+0000 7f93dbfff640 1 -- 172.21.15.46:0/1356513693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc1977e0 con 0x7f93dc1033e0 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.833+0000 7f93dbfff640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc19dba0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93c8031460 tx=0x7f93c8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.833+0000 7f93d9ffb640 1 -- 172.21.15.46:0/1356513693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f93c803f070 con 0x7f93dc1033e0 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.833+0000 7f93d9ffb640 1 -- 172.21.15.46:0/1356513693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93c8004a90 con 0x7f93dc1033e0 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.833+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93dc197a70 con 0x7f93dc1033e0 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.833+0000 7f93d9ffb640 1 -- 172.21.15.46:0/1356513693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f93c8035030 con 0x7f93dc1033e0 2024-09-10T04:15:19.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.833+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93dc197fa0 con 0x7f93dc1033e0 2024-09-10T04:15:19.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:19 smithi070 ceph-mon[79774]: pgmap v289: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:19.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.834+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93dc06aa90 con 0x7f93dc1033e0 2024-09-10T04:15:19.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.835+0000 7f93d9ffb640 1 -- 172.21.15.46:0/1356513693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f93c8042070 con 0x7f93dc1033e0 2024-09-10T04:15:19.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.835+0000 7f93d9ffb640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f93cc07aca0 0x7f93cc07d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:19.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.835+0000 7f93d9ffb640 1 -- 172.21.15.46:0/1356513693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f93c80be6c0 con 0x7f93dc1033e0 2024-09-10T04:15:19.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.836+0000 7f93e0c9d640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f93cc07aca0 0x7f93cc07d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:19.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.836+0000 7f93e0c9d640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f93cc07aca0 0x7f93cc07d160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93d000bbe0 tx=0x7f93d000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:19.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:19.840+0000 7f93d9ffb640 1 -- 172.21.15.46:0/1356513693 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f93c8089de0 con 0x7f93dc1033e0 2024-09-10T04:15:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.044+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f93dc1982b0 con 0x7f93cc07aca0 2024-09-10T04:15:20.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.045+0000 7f93d9ffb640 1 -- 172.21.15.46:0/1356513693 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f93dc1982b0 con 0x7f93cc07aca0 2024-09-10T04:15:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f93cc07aca0 msgr2=0x7f93cc07d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f93cc07aca0 0x7f93cc07d160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93d000bbe0 tx=0x7f93d000b390 comp rx=0 tx=0).stop 2024-09-10T04:15:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 msgr2=0x7f93dc19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc19dba0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93c8031460 tx=0x7f93c8004530 comp rx=0 tx=0).stop 2024-09-10T04:15:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 shutdown_connections 2024-09-10T04:15:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f93cc07aca0 0x7f93cc07d160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93dc1033e0 0x7f93dc19dba0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.047+0000 7f93e2f01640 1 --2- 172.21.15.46:0/1356513693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93dc102ac0 0x7f93dc19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.048+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 >> 172.21.15.46:0/1356513693 conn(0x7f93dc0fc520 msgr2=0x7f93dc108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.048+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 shutdown_connections 2024-09-10T04:15:20.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.048+0000 7f93e2f01640 1 -- 172.21.15.46:0/1356513693 wait complete. 2024-09-10T04:15:20.058 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.133+0000 7fdc1b412640 1 -- 172.21.15.46:0/896302948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074510 msgr2=0x7fdc140748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.133+0000 7fdc1b412640 1 --2- 172.21.15.46:0/896302948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074510 0x7fdc140748f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009880 tx=0x7fdc1002a530 comp rx=0 tx=0).stop 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.133+0000 7fdc1b412640 1 -- 172.21.15.46:0/896302948 shutdown_connections 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.133+0000 7fdc1b412640 1 --2- 172.21.15.46:0/896302948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc14074e30 0x7fdc1410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.133+0000 7fdc1b412640 1 --2- 172.21.15.46:0/896302948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074510 0x7fdc140748f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.133+0000 7fdc1b412640 1 -- 172.21.15.46:0/896302948 >> 172.21.15.46:0/896302948 conn(0x7fdc140fd600 msgr2=0x7fdc140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 -- 172.21.15.46:0/896302948 shutdown_connections 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 -- 172.21.15.46:0/896302948 wait complete. 2024-09-10T04:15:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 Processor -- start 2024-09-10T04:15:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 -- start start 2024-09-10T04:15:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc14074510 0x7fdc1419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074e30 0x7fdc1419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14110fe0 con 0x7fdc14074e30 2024-09-10T04:15:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.134+0000 7fdc1b412640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14110e60 con 0x7fdc14074510 2024-09-10T04:15:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc189ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074e30 0x7fdc1419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc189ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074e30 0x7fdc1419fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46014/0 (socket says 172.21.15.46:46014) 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc189ad640 1 -- 172.21.15.46:0/2736465793 learned_addr learned my addr 172.21.15.46:0/2736465793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc191ae640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc14074510 0x7fdc1419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc189ad640 1 -- 172.21.15.46:0/2736465793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc14074510 msgr2=0x7fdc1419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc189ad640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc14074510 0x7fdc1419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc189ad640 1 -- 172.21.15.46:0/2736465793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc14199a10 con 0x7fdc14074e30 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc189ad640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074e30 0x7fdc1419fd90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400e970 tx=0x7fdc0400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc0a7fc640 1 -- 172.21.15.46:0/2736465793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc0400cac0 con 0x7fdc14074e30 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc0a7fc640 1 -- 172.21.15.46:0/2736465793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc040054e0 con 0x7fdc14074e30 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc14199d00 con 0x7fdc14074e30 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc0a7fc640 1 -- 172.21.15.46:0/2736465793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdc04010430 con 0x7fdc14074e30 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.135+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1419a290 con 0x7fdc14074e30 2024-09-10T04:15:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.136+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc14075c50 con 0x7fdc14074e30 2024-09-10T04:15:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.137+0000 7fdc0a7fc640 1 -- 172.21.15.46:0/2736465793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdc04004ec0 con 0x7fdc14074e30 2024-09-10T04:15:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.138+0000 7fdc0a7fc640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc0007ac00 0x7fdc0007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.138+0000 7fdc0a7fc640 1 -- 172.21.15.46:0/2736465793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fdc0409e2d0 con 0x7fdc14074e30 2024-09-10T04:15:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.138+0000 7fdc191ae640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc0007ac00 0x7fdc0007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.138+0000 7fdc191ae640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc0007ac00 0x7fdc0007d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc10000c00 tx=0x7fdc10031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.142+0000 7fdc0a7fc640 1 -- 172.21.15.46:0/2736465793 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdc04069a10 con 0x7fdc14074e30 2024-09-10T04:15:20.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.348+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdc1419af40 con 0x7fdc0007ac00 2024-09-10T04:15:20.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.349+0000 7fdc0a7fc640 1 -- 172.21.15.46:0/2736465793 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fdc1419af40 con 0x7fdc0007ac00 2024-09-10T04:15:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc0007ac00 msgr2=0x7fdc0007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc0007ac00 0x7fdc0007d0c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc10000c00 tx=0x7fdc10031400 comp rx=0 tx=0).stop 2024-09-10T04:15:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074e30 msgr2=0x7fdc1419fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074e30 0x7fdc1419fd90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400e970 tx=0x7fdc0400ed70 comp rx=0 tx=0).stop 2024-09-10T04:15:20.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 shutdown_connections 2024-09-10T04:15:20.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc0007ac00 0x7fdc0007d0c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc14074e30 0x7fdc1419fd90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 --2- 172.21.15.46:0/2736465793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc14074510 0x7fdc1419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.351+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 >> 172.21.15.46:0/2736465793 conn(0x7fdc140fd600 msgr2=0x7fdc1410b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:20.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.352+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 shutdown_connections 2024-09-10T04:15:20.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.352+0000 7fdc1b412640 1 -- 172.21.15.46:0/2736465793 wait complete. 2024-09-10T04:15:20.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 -- 172.21.15.46:0/1899145090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 msgr2=0x7f020810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 --2- 172.21.15.46:0/1899145090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020810b8d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009880 tx=0x7f01fc02a530 comp rx=0 tx=0).stop 2024-09-10T04:15:20.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 -- 172.21.15.46:0/1899145090 shutdown_connections 2024-09-10T04:15:20.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 --2- 172.21.15.46:0/1899145090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020810b8d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 --2- 172.21.15.46:0/1899145090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208102ac0 0x7f0208102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 -- 172.21.15.46:0/1899145090 >> 172.21.15.46:0/1899145090 conn(0x7f02080fc520 msgr2=0x7f02080fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:20.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 -- 172.21.15.46:0/1899145090 shutdown_connections 2024-09-10T04:15:20.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.436+0000 7f020c90a640 1 -- 172.21.15.46:0/1899145090 wait complete. 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.437+0000 7f020c90a640 1 Processor -- start 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f020c90a640 1 -- start start 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f020c90a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208102ac0 0x7f020819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f020c90a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f020c90a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f020810edf0 con 0x7f02081033e0 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f020c90a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f020810ec70 con 0x7f0208102ac0 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f0205d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f0205d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46020/0 (socket says 172.21.15.46:46020) 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f0205d9b640 1 -- 172.21.15.46:0/671962313 learned_addr learned my addr 172.21.15.46:0/671962313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f020659c640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208102ac0 0x7f020819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f0205d9b640 1 -- 172.21.15.46:0/671962313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208102ac0 msgr2=0x7f020819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f0205d9b640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208102ac0 0x7f020819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.438+0000 7f0205d9b640 1 -- 172.21.15.46:0/671962313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0208197800 con 0x7f02081033e0 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.439+0000 7f0205d9b640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020819dbc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f01fc031460 tx=0x7f01fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.439+0000 7f01f77fe640 1 -- 172.21.15.46:0/671962313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01fc03f070 con 0x7f02081033e0 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.439+0000 7f01f77fe640 1 -- 172.21.15.46:0/671962313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01fc004ac0 con 0x7f02081033e0 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.439+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0208197a90 con 0x7f02081033e0 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.439+0000 7f01f77fe640 1 -- 172.21.15.46:0/671962313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01fc035030 con 0x7f02081033e0 2024-09-10T04:15:20.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.439+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0208197fc0 con 0x7f02081033e0 2024-09-10T04:15:20.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.440+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020806aa90 con 0x7f02081033e0 2024-09-10T04:15:20.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.441+0000 7f01f77fe640 1 -- 172.21.15.46:0/671962313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f01fc042070 con 0x7f02081033e0 2024-09-10T04:15:20.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.441+0000 7f01f77fe640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01f007abf0 0x7f01f007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.441+0000 7f01f77fe640 1 -- 172.21.15.46:0/671962313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f01fc0be630 con 0x7f02081033e0 2024-09-10T04:15:20.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.441+0000 7f020659c640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01f007abf0 0x7f01f007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.442+0000 7f020659c640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01f007abf0 0x7f01f007d0b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01f80058c0 tx=0x7f01f800a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:20.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.446+0000 7f01f77fe640 1 -- 172.21.15.46:0/671962313 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f01fc089d70 con 0x7f02081033e0 2024-09-10T04:15:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.640+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0208198330 con 0x7f01f007abf0 2024-09-10T04:15:20.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.649+0000 7f01f77fe640 1 -- 172.21.15.46:0/671962313 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7f0208198330 con 0x7f01f007abf0 2024-09-10T04:15:20.650 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:15:20.650 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (6m) 83s ago 18m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:15:20.650 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (9m) 83s ago 18m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (9m) 70s ago 17m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (5m) 83s ago 18m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (5m) 70s ago 17m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (6m) 83s ago 18m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (89s) 83s ago 12m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (106s) 83s ago 12m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (73s) 70s ago 12m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (12m) 70s ago 12m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (8m) 83s ago 19m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (7m) 70s ago 17m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (5m) 83s ago 19m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (5m) 70s ago 17m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (7m) 83s ago 18m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:15:20.651 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (7m) 70s ago 17m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:15:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (4m) 83s ago 16m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:15:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (4m) 83s ago 15m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:15:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (3m) 83s ago 15m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:15:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (3m) 70s ago 14m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:15:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (2m) 70s ago 14m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:15:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (2m) 70s ago 13m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:15:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (7m) 83s ago 17m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:15:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.651+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01f007abf0 msgr2=0x7f01f007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.651+0000 7f020c90a640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01f007abf0 0x7f01f007d0b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f01f80058c0 tx=0x7f01f800a250 comp rx=0 tx=0).stop 2024-09-10T04:15:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.651+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 msgr2=0x7f020819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.651+0000 7f020c90a640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020819dbc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f01fc031460 tx=0x7f01fc0046c0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.652+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 shutdown_connections 2024-09-10T04:15:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.652+0000 7f020c90a640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01f007abf0 0x7f01f007d0b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.652+0000 7f020c90a640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02081033e0 0x7f020819dbc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.652+0000 7f020c90a640 1 --2- 172.21.15.46:0/671962313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0208102ac0 0x7f020819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.652+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 >> 172.21.15.46:0/671962313 conn(0x7f02080fc520 msgr2=0x7f0208108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.652+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 shutdown_connections 2024-09-10T04:15:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.652+0000 7f020c90a640 1 -- 172.21.15.46:0/671962313 wait complete. 2024-09-10T04:15:20.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.735+0000 7f98045a6640 1 -- 172.21.15.46:0/1203860992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc107900 msgr2=0x7f97fc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.735+0000 7f98045a6640 1 --2- 172.21.15.46:0/1203860992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc107900 0x7f97fc107ce0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009880 tx=0x7f97f802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 -- 172.21.15.46:0/1203860992 shutdown_connections 2024-09-10T04:15:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 --2- 172.21.15.46:0/1203860992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97fc101900 0x7f97fc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 --2- 172.21.15.46:0/1203860992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc107900 0x7f97fc107ce0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 -- 172.21.15.46:0/1203860992 >> 172.21.15.46:0/1203860992 conn(0x7f97fc0fd600 msgr2=0x7f97fc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 -- 172.21.15.46:0/1203860992 shutdown_connections 2024-09-10T04:15:20.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 -- 172.21.15.46:0/1203860992 wait complete. 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 Processor -- start 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.736+0000 7f98045a6640 1 -- start start 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f98045a6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f98045a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97fc107900 0x7f97fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f98045a6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc10edb0 con 0x7f97fc101900 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f98045a6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc10ec30 con 0x7f97fc107900 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9802342640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9802342640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46030/0 (socket says 172.21.15.46:46030) 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9802342640 1 -- 172.21.15.46:0/1025149056 learned_addr learned my addr 172.21.15.46:0/1025149056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:20.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9801b41640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97fc107900 0x7f97fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9802342640 1 -- 172.21.15.46:0/1025149056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97fc107900 msgr2=0x7f97fc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9802342640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97fc107900 0x7f97fc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9802342640 1 -- 172.21.15.46:0/1025149056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc074f70 con 0x7f97fc101900 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.737+0000 7f9802342640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97f8031460 tx=0x7f97f80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.738+0000 7f97f37fe640 1 -- 172.21.15.46:0/1025149056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97f803f070 con 0x7f97fc101900 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.738+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc075200 con 0x7f97fc101900 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.738+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc1a3d80 con 0x7f97fc101900 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.738+0000 7f97f37fe640 1 -- 172.21.15.46:0/1025149056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97f8004b10 con 0x7f97fc101900 2024-09-10T04:15:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.738+0000 7f97f37fe640 1 -- 172.21.15.46:0/1025149056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97f8035030 con 0x7f97fc101900 2024-09-10T04:15:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.739+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e0005290 con 0x7f97fc101900 2024-09-10T04:15:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.740+0000 7f97f37fe640 1 -- 172.21.15.46:0/1025149056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f97f8042070 con 0x7f97fc101900 2024-09-10T04:15:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.740+0000 7f97f37fe640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97e807ac50 0x7f97e807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.740+0000 7f97f37fe640 1 -- 172.21.15.46:0/1025149056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f97f80be6e0 con 0x7f97fc101900 2024-09-10T04:15:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.740+0000 7f9801b41640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97e807ac50 0x7f97e807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.741+0000 7f9801b41640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97e807ac50 0x7f97e807d110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005fd0 tx=0x7f97ec005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:20.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:20.745+0000 7f97f37fe640 1 -- 172.21.15.46:0/1025149056 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f97f8089e00 con 0x7f97fc101900 2024-09-10T04:15:21.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.002+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f97e0005d90 con 0x7f97fc101900 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.003+0000 7f97f37fe640 1 -- 172.21.15.46:0/1025149056 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f97f808ecb0 con 0x7f97fc101900 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:15:21.004 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:15:21.005 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:15:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97e807ac50 msgr2=0x7f97e807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97e807ac50 0x7f97e807d110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005fd0 tx=0x7f97ec005850 comp rx=0 tx=0).stop 2024-09-10T04:15:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc101900 msgr2=0x7f97fc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f97f8031460 tx=0x7f97f80045e0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 shutdown_connections 2024-09-10T04:15:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97e807ac50 0x7f97e807d110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97fc107900 0x7f97fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 --2- 172.21.15.46:0/1025149056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc101900 0x7f97fc0744f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 >> 172.21.15.46:0/1025149056 conn(0x7f97fc0fd600 msgr2=0x7f97fc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 shutdown_connections 2024-09-10T04:15:21.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.005+0000 7f98045a6640 1 -- 172.21.15.46:0/1025149056 wait complete. 2024-09-10T04:15:21.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.089+0000 7f192caa4640 1 -- 172.21.15.46:0/998137636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 msgr2=0x7f1928102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.089+0000 7f192caa4640 1 --2- 172.21.15.46:0/998137636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928102ea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1918009880 tx=0x7f191802a530 comp rx=0 tx=0).stop 2024-09-10T04:15:21.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.089+0000 7f192caa4640 1 -- 172.21.15.46:0/998137636 shutdown_connections 2024-09-10T04:15:21.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.089+0000 7f192caa4640 1 --2- 172.21.15.46:0/998137636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19281033e0 0x7f192810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.089+0000 7f192caa4640 1 --2- 172.21.15.46:0/998137636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928102ea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.090+0000 7f192caa4640 1 -- 172.21.15.46:0/998137636 >> 172.21.15.46:0/998137636 conn(0x7f19280fc520 msgr2=0x7f19280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:21.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.090+0000 7f192caa4640 1 -- 172.21.15.46:0/998137636 shutdown_connections 2024-09-10T04:15:21.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.090+0000 7f192caa4640 1 -- 172.21.15.46:0/998137636 wait complete. 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.090+0000 7f192caa4640 1 Processor -- start 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.090+0000 7f192caa4640 1 -- start start 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192caa4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192caa4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19281033e0 0x7f1928100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192caa4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f192810edd0 con 0x7f1928102ac0 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192caa4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f192810ec50 con 0x7f19281033e0 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46052/0 (socket says 172.21.15.46:46052) 2024-09-10T04:15:21.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192659c640 1 -- 172.21.15.46:0/620973835 learned_addr learned my addr 172.21.15.46:0/620973835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f1925d9b640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19281033e0 0x7f1928100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192659c640 1 -- 172.21.15.46:0/620973835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19281033e0 msgr2=0x7f1928100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192659c640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19281033e0 0x7f1928100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192659c640 1 -- 172.21.15.46:0/620973835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19281011a0 con 0x7f1928102ac0 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f192659c640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928100720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1918031460 tx=0x7f19180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.091+0000 7f19177fe640 1 -- 172.21.15.46:0/620973835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f191803f070 con 0x7f1928102ac0 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.092+0000 7f19177fe640 1 -- 172.21.15.46:0/620973835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1918004b10 con 0x7f1928102ac0 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.092+0000 7f19177fe640 1 -- 172.21.15.46:0/620973835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1918035030 con 0x7f1928102ac0 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.092+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1928101430 con 0x7f1928102ac0 2024-09-10T04:15:21.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.092+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19281a3d80 con 0x7f1928102ac0 2024-09-10T04:15:21.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.093+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f192806aa90 con 0x7f1928102ac0 2024-09-10T04:15:21.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.093+0000 7f19177fe640 1 -- 172.21.15.46:0/620973835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1918042070 con 0x7f1928102ac0 2024-09-10T04:15:21.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.094+0000 7f19177fe640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f191007ac00 0x7f191007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.094+0000 7f19177fe640 1 -- 172.21.15.46:0/620973835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f19180be750 con 0x7f1928102ac0 2024-09-10T04:15:21.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.094+0000 7f1925d9b640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f191007ac00 0x7f191007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.094+0000 7f1925d9b640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f191007ac00 0x7f191007d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19281021b0 tx=0x7f191c00b2e0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:21.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.098+0000 7f19177fe640 1 -- 172.21.15.46:0/620973835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1918089e90 con 0x7f1928102ac0 2024-09-10T04:15:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.320+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1928105f90 con 0x7f1928102ac0 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.321+0000 7f19177fe640 1 -- 172.21.15.46:0/620973835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f191808ed40 con 0x7f1928102ac0 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:15:21.323 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:15:21.324 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:21.325 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:15:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f191007ac00 msgr2=0x7f191007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f191007ac00 0x7f191007d0c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19281021b0 tx=0x7f191c00b2e0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 msgr2=0x7f1928100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928100720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1918031460 tx=0x7f19180045e0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 shutdown_connections 2024-09-10T04:15:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f191007ac00 0x7f191007d0c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19281033e0 0x7f1928100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 --2- 172.21.15.46:0/620973835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1928102ac0 0x7f1928100720 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 >> 172.21.15.46:0/620973835 conn(0x7f19280fc520 msgr2=0x7f1928108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 shutdown_connections 2024-09-10T04:15:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.323+0000 7f192caa4640 1 -- 172.21.15.46:0/620973835 wait complete. 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 -- 172.21.15.46:0/2295012176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 msgr2=0x7fd848107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 --2- 172.21.15.46:0/2295012176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd848107ce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd844009880 tx=0x7fd84402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 -- 172.21.15.46:0/2295012176 shutdown_connections 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 --2- 172.21.15.46:0/2295012176 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848101900 0x7fd848101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 --2- 172.21.15.46:0/2295012176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd848107ce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 -- 172.21.15.46:0/2295012176 >> 172.21.15.46:0/2295012176 conn(0x7fd8480fd600 msgr2=0x7fd8480ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 -- 172.21.15.46:0/2295012176 shutdown_connections 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.408+0000 7fd850327640 1 -- 172.21.15.46:0/2295012176 wait complete. 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd850327640 1 Processor -- start 2024-09-10T04:15:21.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd850327640 1 -- start start 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd850327640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848101900 0x7fd84819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd850327640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd84819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd850327640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810edd0 con 0x7fd848107900 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd850327640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd84810ec50 con 0x7fd848101900 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd84d8c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd84819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd84d8c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd84819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46068/0 (socket says 172.21.15.46:46068) 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.409+0000 7fd84d8c2640 1 -- 172.21.15.46:0/2478995798 learned_addr learned my addr 172.21.15.46:0/2478995798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd84e0c3640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848101900 0x7fd84819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd84d8c2640 1 -- 172.21.15.46:0/2478995798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848101900 msgr2=0x7fd84819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd84d8c2640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848101900 0x7fd84819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd84d8c2640 1 -- 172.21.15.46:0/2478995798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8481977c0 con 0x7fd848107900 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd84d8c2640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd84819db80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c8f0 tx=0x7fd83800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd83f7fe640 1 -- 172.21.15.46:0/2478995798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd8380079b0 con 0x7fd848107900 2024-09-10T04:15:21.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd83f7fe640 1 -- 172.21.15.46:0/2478995798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd8380054e0 con 0x7fd848107900 2024-09-10T04:15:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd848197ab0 con 0x7fd848107900 2024-09-10T04:15:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd83f7fe640 1 -- 172.21.15.46:0/2478995798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd838004dd0 con 0x7fd848107900 2024-09-10T04:15:21.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.410+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd848198040 con 0x7fd848107900 2024-09-10T04:15:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.411+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd848103040 con 0x7fd848107900 2024-09-10T04:15:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.412+0000 7fd83f7fe640 1 -- 172.21.15.46:0/2478995798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd838005020 con 0x7fd848107900 2024-09-10T04:15:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.412+0000 7fd83f7fe640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd83407ab90 0x7fd83407d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.413+0000 7fd84e0c3640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd83407ab90 0x7fd83407d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.413+0000 7fd83f7fe640 1 -- 172.21.15.46:0/2478995798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fd83809dc60 con 0x7fd848107900 2024-09-10T04:15:21.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.413+0000 7fd84e0c3640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd83407ab90 0x7fd83407d050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd844031460 tx=0x7fd844005bd0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.417+0000 7fd83f7fe640 1 -- 172.21.15.46:0/2478995798 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd838014030 con 0x7fd848107900 2024-09-10T04:15:21.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:21 smithi046 ceph-mon[109506]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:21.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:21 smithi046 ceph-mon[109506]: pgmap v290: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:21.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:21 smithi046 ceph-mon[109506]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:21.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:21 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1025149056' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:15:21.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:21 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/620973835' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:15:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.621+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd8481983b0 con 0x7fd83407ab90 2024-09-10T04:15:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.622+0000 7fd83f7fe640 1 -- 172.21.15.46:0/2478995798 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd8481983b0 con 0x7fd83407ab90 2024-09-10T04:15:21.623 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:15:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:15:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:15:21.623 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:15:21.624 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd83407ab90 msgr2=0x7fd83407d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd83407ab90 0x7fd83407d050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd844031460 tx=0x7fd844005bd0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 msgr2=0x7fd84819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd84819db80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c8f0 tx=0x7fd83800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 shutdown_connections 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd83407ab90 0x7fd83407d050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd848107900 0x7fd84819db80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 --2- 172.21.15.46:0/2478995798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848101900 0x7fd84819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 >> 172.21.15.46:0/2478995798 conn(0x7fd8480fd600 msgr2=0x7fd8480fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 shutdown_connections 2024-09-10T04:15:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.625+0000 7fd850327640 1 -- 172.21.15.46:0/2478995798 wait complete. 2024-09-10T04:15:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1299664328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 msgr2=0x7fe6b8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1299664328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b8107ce0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009880 tx=0x7fe6b402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1299664328 shutdown_connections 2024-09-10T04:15:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1299664328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6b8101900 0x7fe6b8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1299664328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b8107ce0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1299664328 >> 172.21.15.46:0/1299664328 conn(0x7fe6b80fd600 msgr2=0x7fe6b80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1299664328 shutdown_connections 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.709+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1299664328 wait complete. 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6c03c8640 1 Processor -- start 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6c03c8640 1 -- start start 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6c03c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6b8101900 0x7fe6b819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6c03c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6c03c8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b810ee30 con 0x7fe6b8107900 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6c03c8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b810ecb0 con 0x7fe6b8101900 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6bd963640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6bd963640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b819dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46074/0 (socket says 172.21.15.46:46074) 2024-09-10T04:15:21.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6bd963640 1 -- 172.21.15.46:0/1557691952 learned_addr learned my addr 172.21.15.46:0/1557691952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.710+0000 7fe6be164640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6b8101900 0x7fe6b819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6bd963640 1 -- 172.21.15.46:0/1557691952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6b8101900 msgr2=0x7fe6b819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6bd963640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6b8101900 0x7fe6b819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6bd963640 1 -- 172.21.15.46:0/1557691952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b8197860 con 0x7fe6b8107900 2024-09-10T04:15:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6be164640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6b8101900 0x7fe6b819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:15:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6bd963640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b819dbe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800b420 tx=0x7fe6a800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6af7fe640 1 -- 172.21.15.46:0/1557691952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe6a8011630 con 0x7fe6b8107900 2024-09-10T04:15:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6af7fe640 1 -- 172.21.15.46:0/1557691952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe6a8016070 con 0x7fe6b8107900 2024-09-10T04:15:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6af7fe640 1 -- 172.21.15.46:0/1557691952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe6a8010d40 con 0x7fe6b8107900 2024-09-10T04:15:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b8197b50 con 0x7fe6b8107900 2024-09-10T04:15:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.711+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b81980e0 con 0x7fe6b8107900 2024-09-10T04:15:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.712+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b8103040 con 0x7fe6b8107900 2024-09-10T04:15:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.713+0000 7fe6af7fe640 1 -- 172.21.15.46:0/1557691952 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe6a801b460 con 0x7fe6b8107900 2024-09-10T04:15:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.713+0000 7fe6af7fe640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe6a407ac40 0x7fe6a407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.713+0000 7fe6af7fe640 1 -- 172.21.15.46:0/1557691952 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fe6a809d370 con 0x7fe6b8107900 2024-09-10T04:15:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.714+0000 7fe6be164640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe6a407ac40 0x7fe6a407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.714+0000 7fe6be164640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe6a407ac40 0x7fe6a407d100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4000c00 tx=0x7fe6b4031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:21.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.718+0000 7fe6af7fe640 1 -- 172.21.15.46:0/1557691952 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe6a8021250 con 0x7fe6b8107900 2024-09-10T04:15:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:21 smithi070 ceph-mon[79774]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:21 smithi070 ceph-mon[79774]: pgmap v290: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:21 smithi070 ceph-mon[79774]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:21 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1025149056' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:15:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:21 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/620973835' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.977+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe6b8198d90 con 0x7fe6b8107900 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.977+0000 7fe6af7fe640 1 -- 172.21.15.46:0/1557691952 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fe6a806be20 con 0x7fe6b8107900 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:15:21.978 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:15:21.979 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:15:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe6a407ac40 msgr2=0x7fe6a407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe6a407ac40 0x7fe6a407d100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4000c00 tx=0x7fe6b4031420 comp rx=0 tx=0).stop 2024-09-10T04:15:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 msgr2=0x7fe6b819dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b819dbe0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800b420 tx=0x7fe6a800b8e0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 shutdown_connections 2024-09-10T04:15:21.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe6a407ac40 0x7fe6a407d100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b8107900 0x7fe6b819dbe0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 --2- 172.21.15.46:0/1557691952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6b8101900 0x7fe6b819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 >> 172.21.15.46:0/1557691952 conn(0x7fe6b80fd600 msgr2=0x7fe6b80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.979+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 shutdown_connections 2024-09-10T04:15:21.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:21.980+0000 7fe6c03c8640 1 -- 172.21.15.46:0/1557691952 wait complete. 2024-09-10T04:15:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:22 smithi070 ceph-mon[79774]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:22 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1557691952' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:15:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:22 smithi046 ceph-mon[109506]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:22 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1557691952' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:15:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:23 smithi046 ceph-mon[109506]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:23 smithi046 ceph-mon[109506]: pgmap v291: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:23 smithi070 ceph-mon[79774]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:23 smithi070 ceph-mon[79774]: pgmap v291: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:25 smithi070 ceph-mon[79774]: pgmap v292: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:25 smithi046 ceph-mon[109506]: pgmap v292: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:27 smithi070 ceph-mon[79774]: pgmap v293: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:27 smithi046 ceph-mon[109506]: pgmap v293: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:29 smithi046 ceph-mon[109506]: pgmap v294: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:29 smithi070 ceph-mon[79774]: pgmap v294: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:31 smithi070 ceph-mon[79774]: pgmap v295: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:31 smithi046 ceph-mon[109506]: pgmap v295: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:33.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:33 smithi070 ceph-mon[79774]: pgmap v296: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:33 smithi046 ceph-mon[109506]: pgmap v296: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:35 smithi070 ceph-mon[79774]: pgmap v297: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:35 smithi046 ceph-mon[109506]: pgmap v297: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:37 smithi070 ceph-mon[79774]: pgmap v298: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:37 smithi046 ceph-mon[109506]: pgmap v298: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:38.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:39 smithi070 ceph-mon[79774]: pgmap v299: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:39 smithi046 ceph-mon[109506]: pgmap v299: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:41 smithi046 ceph-mon[109506]: pgmap v300: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:41 smithi070 ceph-mon[79774]: pgmap v300: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:43.756 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:43 smithi046 ceph-mon[109506]: pgmap v301: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:43 smithi070 ceph-mon[79774]: pgmap v301: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:45 smithi070 ceph-mon[79774]: pgmap v302: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:45 smithi046 ceph-mon[109506]: pgmap v302: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:47 smithi046 ceph-mon[109506]: pgmap v303: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:47 smithi070 ceph-mon[79774]: pgmap v303: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:49 smithi070 ceph-mon[79774]: pgmap v304: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:49.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:49 smithi046 ceph-mon[109506]: pgmap v304: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:51 smithi070 ceph-mon[79774]: pgmap v305: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:51 smithi046 ceph-mon[109506]: pgmap v305: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 -- 172.21.15.46:0/3927044281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4618100a90 msgr2=0x7f46181065d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 --2- 172.21.15.46:0/3927044281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4618100a90 0x7f46181065d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f460c009880 tx=0x7f460c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 -- 172.21.15.46:0/3927044281 shutdown_connections 2024-09-10T04:15:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 --2- 172.21.15.46:0/3927044281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4618100a90 0x7f46181065d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 --2- 172.21.15.46:0/3927044281 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46181000e0 0x7f46181004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 -- 172.21.15.46:0/3927044281 >> 172.21.15.46:0/3927044281 conn(0x7f46180fbc90 msgr2=0x7f46180fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 -- 172.21.15.46:0/3927044281 shutdown_connections 2024-09-10T04:15:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.076+0000 7f461ea2d640 1 -- 172.21.15.46:0/3927044281 wait complete. 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.077+0000 7f461ea2d640 1 Processor -- start 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.077+0000 7f461ea2d640 1 -- start start 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f461ea2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46181000e0 0x7f461819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f461ea2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4618100a90 0x7f461819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f461ea2d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f461810ee30 con 0x7f46181000e0 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f461ea2d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f461810ecb0 con 0x7f4618100a90 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f4617fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46181000e0 0x7f461819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f4617fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46181000e0 0x7f461819d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59374/0 (socket says 172.21.15.46:59374) 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f4617fff640 1 -- 172.21.15.46:0/4130016433 learned_addr learned my addr 172.21.15.46:0/4130016433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f46177fe640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4618100a90 0x7f461819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f4617fff640 1 -- 172.21.15.46:0/4130016433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4618100a90 msgr2=0x7f461819dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f4617fff640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4618100a90 0x7f461819dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f4617fff640 1 -- 172.21.15.46:0/4130016433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461819e2d0 con 0x7f46181000e0 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f46177fe640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4618100a90 0x7f461819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.078+0000 7f4617fff640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46181000e0 0x7f461819d6e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f460800c8a0 tx=0x7f460800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.079+0000 7f46157fa640 1 -- 172.21.15.46:0/4130016433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f46080079b0 con 0x7f46181000e0 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.079+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46181979c0 con 0x7f46181000e0 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.079+0000 7f46157fa640 1 -- 172.21.15.46:0/4130016433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4608007b50 con 0x7f46181000e0 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.079+0000 7f46157fa640 1 -- 172.21.15.46:0/4130016433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4608018700 con 0x7f46181000e0 2024-09-10T04:15:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.079+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4618197f20 con 0x7f46181000e0 2024-09-10T04:15:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.080+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46181031a0 con 0x7f46181000e0 2024-09-10T04:15:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.081+0000 7f46157fa640 1 -- 172.21.15.46:0/4130016433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4608007d00 con 0x7f46181000e0 2024-09-10T04:15:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.081+0000 7f46157fa640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f460007abf0 0x7f460007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.081+0000 7f46157fa640 1 -- 172.21.15.46:0/4130016433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f460809df90 con 0x7f46181000e0 2024-09-10T04:15:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.081+0000 7f46177fe640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f460007abf0 0x7f460007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.082+0000 7f46177fe640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f460007abf0 0x7f460007d0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4618199090 tx=0x7f460c009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.086+0000 7f46157fa640 1 -- 172.21.15.46:0/4130016433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4608014030 con 0x7f46181000e0 2024-09-10T04:15:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.291+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4618198e40 con 0x7f460007abf0 2024-09-10T04:15:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.292+0000 7f46157fa640 1 -- 172.21.15.46:0/4130016433 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f4618198e40 con 0x7f460007abf0 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f460007abf0 msgr2=0x7f460007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f460007abf0 0x7f460007d0b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4618199090 tx=0x7f460c009560 comp rx=0 tx=0).stop 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46181000e0 msgr2=0x7f461819d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46181000e0 0x7f461819d6e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f460800c8a0 tx=0x7f460800cd60 comp rx=0 tx=0).stop 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 shutdown_connections 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f460007abf0 0x7f460007d0b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4618100a90 0x7f461819dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 --2- 172.21.15.46:0/4130016433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46181000e0 0x7f461819d6e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 >> 172.21.15.46:0/4130016433 conn(0x7f46180fbc90 msgr2=0x7f46180fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 shutdown_connections 2024-09-10T04:15:52.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.295+0000 7f461ea2d640 1 -- 172.21.15.46:0/4130016433 wait complete. 2024-09-10T04:15:52.306 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:15:52.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.381+0000 7f9a557f2640 1 -- 172.21.15.46:0/559340426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 msgr2=0x7f9a50107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.381+0000 7f9a557f2640 1 --2- 172.21.15.46:0/559340426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a50107ce0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009880 tx=0x7f9a4002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 -- 172.21.15.46:0/559340426 shutdown_connections 2024-09-10T04:15:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 --2- 172.21.15.46:0/559340426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a50101900 0x7f9a50101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 --2- 172.21.15.46:0/559340426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a50107ce0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 -- 172.21.15.46:0/559340426 >> 172.21.15.46:0/559340426 conn(0x7f9a500fd600 msgr2=0x7f9a500ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 -- 172.21.15.46:0/559340426 shutdown_connections 2024-09-10T04:15:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 -- 172.21.15.46:0/559340426 wait complete. 2024-09-10T04:15:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 Processor -- start 2024-09-10T04:15:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.382+0000 7f9a557f2640 1 -- start start 2024-09-10T04:15:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a557f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a50101900 0x7f9a5019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a557f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a5019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a557f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010edd0 con 0x7f9a50107900 2024-09-10T04:15:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a557f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5010ec50 con 0x7f9a50101900 2024-09-10T04:15:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a5019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a5019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59390/0 (socket says 172.21.15.46:59390) 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a47fff640 1 -- 172.21.15.46:0/2588348077 learned_addr learned my addr 172.21.15.46:0/2588348077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a4effd640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a50101900 0x7f9a5019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a47fff640 1 -- 172.21.15.46:0/2588348077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a50101900 msgr2=0x7f9a5019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a47fff640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a50101900 0x7f9a5019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a47fff640 1 -- 172.21.15.46:0/2588348077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a501977c0 con 0x7f9a50107900 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a47fff640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a5019db80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800d6f0 tx=0x7f9a3800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.383+0000 7f9a4cff9640 1 -- 172.21.15.46:0/2588348077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a38017070 con 0x7f9a50107900 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.384+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a50197ab0 con 0x7f9a50107900 2024-09-10T04:15:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.384+0000 7f9a4cff9640 1 -- 172.21.15.46:0/2588348077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9a3800de00 con 0x7f9a50107900 2024-09-10T04:15:52.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.384+0000 7f9a4cff9640 1 -- 172.21.15.46:0/2588348077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a38004bb0 con 0x7f9a50107900 2024-09-10T04:15:52.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.384+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a50198040 con 0x7f9a50107900 2024-09-10T04:15:52.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.384+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a50103040 con 0x7f9a50107900 2024-09-10T04:15:52.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.385+0000 7f9a4cff9640 1 -- 172.21.15.46:0/2588348077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9a38004e60 con 0x7f9a50107900 2024-09-10T04:15:52.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.386+0000 7f9a4cff9640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a3c07abf0 0x7f9a3c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.386+0000 7f9a4effd640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a3c07abf0 0x7f9a3c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.386+0000 7f9a4cff9640 1 -- 172.21.15.46:0/2588348077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f9a3809d820 con 0x7f9a50107900 2024-09-10T04:15:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.386+0000 7f9a4effd640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a3c07abf0 0x7f9a3c07d0b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a400096c0 tx=0x7f9a40031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.390+0000 7f9a4cff9640 1 -- 172.21.15.46:0/2588348077 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9a38068f60 con 0x7f9a50107900 2024-09-10T04:15:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.595+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9a501982f0 con 0x7f9a3c07abf0 2024-09-10T04:15:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.596+0000 7f9a4cff9640 1 -- 172.21.15.46:0/2588348077 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f9a501982f0 con 0x7f9a3c07abf0 2024-09-10T04:15:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a3c07abf0 msgr2=0x7f9a3c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a3c07abf0 0x7f9a3c07d0b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a400096c0 tx=0x7f9a40031400 comp rx=0 tx=0).stop 2024-09-10T04:15:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 msgr2=0x7f9a5019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a5019db80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800d6f0 tx=0x7f9a3800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 shutdown_connections 2024-09-10T04:15:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a3c07abf0 0x7f9a3c07d0b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a50107900 0x7f9a5019db80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 --2- 172.21.15.46:0/2588348077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a50101900 0x7f9a5019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.598+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 >> 172.21.15.46:0/2588348077 conn(0x7f9a500fd600 msgr2=0x7f9a500fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.599+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 shutdown_connections 2024-09-10T04:15:52.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.599+0000 7f9a557f2640 1 -- 172.21.15.46:0/2588348077 wait complete. 2024-09-10T04:15:52.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.683+0000 7efc21fb1640 1 -- 172.21.15.46:0/3667285968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 msgr2=0x7efc1c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.683+0000 7efc21fb1640 1 --2- 172.21.15.46:0/3667285968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c101d60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc10009880 tx=0x7efc1002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.683+0000 7efc21fb1640 1 -- 172.21.15.46:0/3667285968 shutdown_connections 2024-09-10T04:15:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.683+0000 7efc21fb1640 1 --2- 172.21.15.46:0/3667285968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c101d60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.683+0000 7efc21fb1640 1 --2- 172.21.15.46:0/3667285968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc1c107900 0x7efc1c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.683+0000 7efc21fb1640 1 -- 172.21.15.46:0/3667285968 >> 172.21.15.46:0/3667285968 conn(0x7efc1c0fd600 msgr2=0x7efc1c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.684+0000 7efc21fb1640 1 -- 172.21.15.46:0/3667285968 shutdown_connections 2024-09-10T04:15:52.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.684+0000 7efc21fb1640 1 -- 172.21.15.46:0/3667285968 wait complete. 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.684+0000 7efc21fb1640 1 Processor -- start 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc21fb1640 1 -- start start 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc21fb1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc21fb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc1c107900 0x7efc1c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc21fb1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c10edd0 con 0x7efc1c101900 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc21fb1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efc1c10ec50 con 0x7efc1c107900 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc1b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc1b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59404/0 (socket says 172.21.15.46:59404) 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc1b7fe640 1 -- 172.21.15.46:0/2453987684 learned_addr learned my addr 172.21.15.46:0/2453987684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc1affd640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc1c107900 0x7efc1c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc1b7fe640 1 -- 172.21.15.46:0/2453987684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc1c107900 msgr2=0x7efc1c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc1b7fe640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc1c107900 0x7efc1c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.685+0000 7efc1b7fe640 1 -- 172.21.15.46:0/2453987684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1c074f70 con 0x7efc1c101900 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.686+0000 7efc1b7fe640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c0744f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00ea00 tx=0x7efc0c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.686+0000 7efc18ff9640 1 -- 172.21.15.46:0/2453987684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efc0c00cb80 con 0x7efc1c101900 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.686+0000 7efc18ff9640 1 -- 172.21.15.46:0/2453987684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc0c00cd20 con 0x7efc1c101900 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.686+0000 7efc18ff9640 1 -- 172.21.15.46:0/2453987684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efc0c011470 con 0x7efc1c101900 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.686+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1c075260 con 0x7efc1c101900 2024-09-10T04:15:52.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.686+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc1c1a3e80 con 0x7efc1c101900 2024-09-10T04:15:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.687+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc1c103040 con 0x7efc1c101900 2024-09-10T04:15:52.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.688+0000 7efc18ff9640 1 -- 172.21.15.46:0/2453987684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efc0c005020 con 0x7efc1c101900 2024-09-10T04:15:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.689+0000 7efc18ff9640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc0407ac40 0x7efc0407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.689+0000 7efc18ff9640 1 -- 172.21.15.46:0/2453987684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7efc0c09dfc0 con 0x7efc1c101900 2024-09-10T04:15:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.689+0000 7efc1affd640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc0407ac40 0x7efc0407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.690+0000 7efc1affd640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc0407ac40 0x7efc0407d100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc1c075f80 tx=0x7efc10031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.694+0000 7efc18ff9640 1 -- 172.21.15.46:0/2453987684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efc0c016030 con 0x7efc1c101900 2024-09-10T04:15:52.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.888+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7efc1c062630 con 0x7efc0407ac40 2024-09-10T04:15:52.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.898+0000 7efc18ff9640 1 -- 172.21.15.46:0/2453987684 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7efc1c062630 con 0x7efc0407ac40 2024-09-10T04:15:52.900 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:15:52.901 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (7m) 115s ago 18m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:15:52.901 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (10m) 115s ago 19m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:15:52.901 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (9m) 102s ago 18m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:15:52.901 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (5m) 115s ago 18m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:15:52.901 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (5m) 102s ago 18m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:15:52.901 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (7m) 115s ago 18m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:15:52.901 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (2m) 115s ago 13m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:15:52.902 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (2m) 115s ago 13m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:15:52.902 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (105s) 102s ago 13m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:15:52.902 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (13m) 102s ago 13m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:15:52.902 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (9m) 115s ago 20m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:15:52.902 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (8m) 102s ago 17m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:15:52.902 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (6m) 115s ago 20m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:15:52.902 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (6m) 102s ago 17m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (8m) 115s ago 18m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 102s ago 18m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (5m) 115s ago 16m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (4m) 115s ago 16m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (4m) 115s ago 15m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (3m) 102s ago 15m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (3m) 102s ago 14m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:15:52.903 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (2m) 102s ago 14m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:15:52.904 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (7m) 115s ago 18m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:15:52.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc0407ac40 msgr2=0x7efc0407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc0407ac40 0x7efc0407d100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efc1c075f80 tx=0x7efc10031400 comp rx=0 tx=0).stop 2024-09-10T04:15:52.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 msgr2=0x7efc1c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c0744f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc0c00ea00 tx=0x7efc0c00eec0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 shutdown_connections 2024-09-10T04:15:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc0407ac40 0x7efc0407d100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efc1c107900 0x7efc1c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 --2- 172.21.15.46:0/2453987684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1c101900 0x7efc1c0744f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 >> 172.21.15.46:0/2453987684 conn(0x7efc1c0fd600 msgr2=0x7efc1c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 shutdown_connections 2024-09-10T04:15:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.900+0000 7efc21fb1640 1 -- 172.21.15.46:0/2453987684 wait complete. 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 -- 172.21.15.46:0/898881990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614100800 msgr2=0x7f2614100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 --2- 172.21.15.46:0/898881990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614100800 0x7f2614100c60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009880 tx=0x7f25fc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 -- 172.21.15.46:0/898881990 shutdown_connections 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 --2- 172.21.15.46:0/898881990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614100800 0x7f2614100c60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 --2- 172.21.15.46:0/898881990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2614106800 0x7f2614106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 -- 172.21.15.46:0/898881990 >> 172.21.15.46:0/898881990 conn(0x7f26140fc520 msgr2=0x7f26140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 -- 172.21.15.46:0/898881990 shutdown_connections 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.983+0000 7f261a06f640 1 -- 172.21.15.46:0/898881990 wait complete. 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.984+0000 7f261a06f640 1 Processor -- start 2024-09-10T04:15:52.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.984+0000 7f261a06f640 1 -- start start 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f261a06f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2614100800 0x7f2614198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f261a06f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614106800 0x7f2614198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f261a06f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f261410dcd0 con 0x7f2614106800 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f261a06f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f261410db50 con 0x7f2614100800 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f2612ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614106800 0x7f2614198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f2612ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614106800 0x7f2614198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59424/0 (socket says 172.21.15.46:59424) 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f2612ffd640 1 -- 172.21.15.46:0/2231419949 learned_addr learned my addr 172.21.15.46:0/2231419949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f26137fe640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2614100800 0x7f2614198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f2612ffd640 1 -- 172.21.15.46:0/2231419949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2614100800 msgr2=0x7f2614198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f2612ffd640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2614100800 0x7f2614198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f2612ffd640 1 -- 172.21.15.46:0/2231419949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2614199360 con 0x7f2614106800 2024-09-10T04:15:52.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f26137fe640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2614100800 0x7f2614198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:15:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.985+0000 7f2612ffd640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614106800 0x7f2614198d60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f25fc031460 tx=0x7f25fc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.986+0000 7f2610ff9640 1 -- 172.21.15.46:0/2231419949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f25fc03f070 con 0x7f2614106800 2024-09-10T04:15:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.986+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141995f0 con 0x7f2614106800 2024-09-10T04:15:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.986+0000 7f2610ff9640 1 -- 172.21.15.46:0/2231419949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25fc0035f0 con 0x7f2614106800 2024-09-10T04:15:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.986+0000 7f2610ff9640 1 -- 172.21.15.46:0/2231419949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f25fc035030 con 0x7f2614106800 2024-09-10T04:15:52.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.986+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141a1ba0 con 0x7f2614106800 2024-09-10T04:15:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.987+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2614101f40 con 0x7f2614106800 2024-09-10T04:15:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.988+0000 7f2610ff9640 1 -- 172.21.15.46:0/2231419949 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f25fc042070 con 0x7f2614106800 2024-09-10T04:15:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.988+0000 7f2610ff9640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f260007abf0 0x7f260007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.989+0000 7f2610ff9640 1 -- 172.21.15.46:0/2231419949 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f25fc0be5b0 con 0x7f2614106800 2024-09-10T04:15:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.989+0000 7f26137fe640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f260007abf0 0x7f260007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:52.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.989+0000 7f26137fe640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f260007abf0 0x7f260007d0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2604009fc0 tx=0x7f2604005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:52.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:52.993+0000 7f2610ff9640 1 -- 172.21.15.46:0/2231419949 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f25fc089cf0 con 0x7f2614106800 2024-09-10T04:15:53.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.260+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f2614108d70 con 0x7f2614106800 2024-09-10T04:15:53.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.261+0000 7f2610ff9640 1 -- 172.21.15.46:0/2231419949 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f25fc08eba0 con 0x7f2614106800 2024-09-10T04:15:53.263 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:15:53.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:15:53.265 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:15:53.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.263+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f260007abf0 msgr2=0x7f260007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.263+0000 7f261a06f640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f260007abf0 0x7f260007d0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2604009fc0 tx=0x7f2604005f90 comp rx=0 tx=0).stop 2024-09-10T04:15:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.263+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614106800 msgr2=0x7f2614198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.263+0000 7f261a06f640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614106800 0x7f2614198d60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f25fc031460 tx=0x7f25fc004530 comp rx=0 tx=0).stop 2024-09-10T04:15:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.264+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 shutdown_connections 2024-09-10T04:15:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.264+0000 7f261a06f640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f260007abf0 0x7f260007d0b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.264+0000 7f261a06f640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2614106800 0x7f2614198d60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.264+0000 7f261a06f640 1 --2- 172.21.15.46:0/2231419949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2614100800 0x7f2614198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.264+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 >> 172.21.15.46:0/2231419949 conn(0x7f26140fc520 msgr2=0x7f26140fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:53.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.264+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 shutdown_connections 2024-09-10T04:15:53.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.264+0000 7f261a06f640 1 -- 172.21.15.46:0/2231419949 wait complete. 2024-09-10T04:15:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.348+0000 7f25fcf59640 1 -- 172.21.15.46:0/1477908386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8107900 msgr2=0x7f25f8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.348+0000 7f25fcf59640 1 --2- 172.21.15.46:0/1477908386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8107900 0x7f25f8107ce0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009880 tx=0x7f25e802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 -- 172.21.15.46:0/1477908386 shutdown_connections 2024-09-10T04:15:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 --2- 172.21.15.46:0/1477908386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8101900 0x7f25f8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 --2- 172.21.15.46:0/1477908386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8107900 0x7f25f8107ce0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 -- 172.21.15.46:0/1477908386 >> 172.21.15.46:0/1477908386 conn(0x7f25f80fd600 msgr2=0x7f25f80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 -- 172.21.15.46:0/1477908386 shutdown_connections 2024-09-10T04:15:53.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 -- 172.21.15.46:0/1477908386 wait complete. 2024-09-10T04:15:53.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 Processor -- start 2024-09-10T04:15:53.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.349+0000 7f25fcf59640 1 -- start start 2024-09-10T04:15:53.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25fcf59640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8101900 0x7f25f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25fcf59640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8107900 0x7f25f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25fcf59640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810edd0 con 0x7f25f8101900 2024-09-10T04:15:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25fcf59640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f25f810ec50 con 0x7f25f8107900 2024-09-10T04:15:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8101900 0x7f25f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8101900 0x7f25f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59450/0 (socket says 172.21.15.46:59450) 2024-09-10T04:15:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f659c640 1 -- 172.21.15.46:0/3960371536 learned_addr learned my addr 172.21.15.46:0/3960371536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f5d9b640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8107900 0x7f25f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f659c640 1 -- 172.21.15.46:0/3960371536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8107900 msgr2=0x7f25f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f659c640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8107900 0x7f25f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f659c640 1 -- 172.21.15.46:0/3960371536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f8074f70 con 0x7f25f8101900 2024-09-10T04:15:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25f659c640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8101900 0x7f25f80744f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25e8031460 tx=0x7f25e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.350+0000 7f25e77fe640 1 -- 172.21.15.46:0/3960371536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f25e803f070 con 0x7f25f8101900 2024-09-10T04:15:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.351+0000 7f25e77fe640 1 -- 172.21.15.46:0/3960371536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25e8004b10 con 0x7f25f8101900 2024-09-10T04:15:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.351+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f8075200 con 0x7f25f8101900 2024-09-10T04:15:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.351+0000 7f25e77fe640 1 -- 172.21.15.46:0/3960371536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f25e8035030 con 0x7f25f8101900 2024-09-10T04:15:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.351+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81a3d60 con 0x7f25f8101900 2024-09-10T04:15:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.352+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f8103040 con 0x7f25f8101900 2024-09-10T04:15:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.352+0000 7f25e77fe640 1 -- 172.21.15.46:0/3960371536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f25e8042070 con 0x7f25f8101900 2024-09-10T04:15:53.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.353+0000 7f25e77fe640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f25e007ac50 0x7f25e007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.353+0000 7f25e77fe640 1 -- 172.21.15.46:0/3960371536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f25e80be700 con 0x7f25f8101900 2024-09-10T04:15:53.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.353+0000 7f25f5d9b640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f25e007ac50 0x7f25e007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.353+0000 7f25f5d9b640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f25e007ac50 0x7f25e007d110 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005fd0 tx=0x7f25ec005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:53.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.357+0000 7f25e77fe640 1 -- 172.21.15.46:0/3960371536 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f25e8089ec0 con 0x7f25f8101900 2024-09-10T04:15:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.585+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f25f8075a20 con 0x7f25f8101900 2024-09-10T04:15:53.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.585+0000 7f25e77fe640 1 -- 172.21.15.46:0/3960371536 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f25e808e020 con 0x7f25f8101900 2024-09-10T04:15:53.587 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:15:53.588 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:15:53.589 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:53.590 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:53.591 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:15:53.591 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:15:53.591 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:53.591 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:53.591 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:53.591 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f25e007ac50 msgr2=0x7f25e007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f25e007ac50 0x7f25e007d110 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f25ec005fd0 tx=0x7f25ec005850 comp rx=0 tx=0).stop 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8101900 msgr2=0x7f25f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8101900 0x7f25f80744f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f25e8031460 tx=0x7f25e80045e0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 shutdown_connections 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f25e007ac50 0x7f25e007d110 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25f8107900 0x7f25f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 --2- 172.21.15.46:0/3960371536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f8101900 0x7f25f80744f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.587+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 >> 172.21.15.46:0/3960371536 conn(0x7f25f80fd600 msgr2=0x7f25f80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:53.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.588+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 shutdown_connections 2024-09-10T04:15:53.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.588+0000 7f25fcf59640 1 -- 172.21.15.46:0/3960371536 wait complete. 2024-09-10T04:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.672+0000 7fc516354640 1 -- 172.21.15.46:0/983386687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 msgr2=0x7fc510101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.672+0000 7fc516354640 1 --2- 172.21.15.46:0/983386687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc510101d60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc504009880 tx=0x7fc50402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.672+0000 7fc516354640 1 -- 172.21.15.46:0/983386687 shutdown_connections 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.672+0000 7fc516354640 1 --2- 172.21.15.46:0/983386687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc510101d60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.672+0000 7fc516354640 1 --2- 172.21.15.46:0/983386687 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc510107900 0x7fc510107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.672+0000 7fc516354640 1 -- 172.21.15.46:0/983386687 >> 172.21.15.46:0/983386687 conn(0x7fc5100fd600 msgr2=0x7fc5100ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.673+0000 7fc516354640 1 -- 172.21.15.46:0/983386687 shutdown_connections 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.673+0000 7fc516354640 1 -- 172.21.15.46:0/983386687 wait complete. 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.673+0000 7fc516354640 1 Processor -- start 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.673+0000 7fc516354640 1 -- start start 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc516354640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc51019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc516354640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc510107900 0x7fc51019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc516354640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010edd0 con 0x7fc510101900 2024-09-10T04:15:53.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc516354640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc51010ec50 con 0x7fc510107900 2024-09-10T04:15:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc51019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc51019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59468/0 (socket says 172.21.15.46:59468) 2024-09-10T04:15:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50ffff640 1 -- 172.21.15.46:0/1969002142 learned_addr learned my addr 172.21.15.46:0/1969002142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50f7fe640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc510107900 0x7fc51019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50ffff640 1 -- 172.21.15.46:0/1969002142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc510107900 msgr2=0x7fc51019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50ffff640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc510107900 0x7fc51019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50ffff640 1 -- 172.21.15.46:0/1969002142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5101977a0 con 0x7fc510101900 2024-09-10T04:15:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50ffff640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc51019d620 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc50000d6f0 tx=0x7fc50000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50d7fa640 1 -- 172.21.15.46:0/1969002142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc500017070 con 0x7fc510101900 2024-09-10T04:15:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc510197a90 con 0x7fc510101900 2024-09-10T04:15:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50d7fa640 1 -- 172.21.15.46:0/1969002142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc50000de00 con 0x7fc510101900 2024-09-10T04:15:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc50d7fa640 1 -- 172.21.15.46:0/1969002142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc500004bb0 con 0x7fc510101900 2024-09-10T04:15:53.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.674+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc510197ff0 con 0x7fc510101900 2024-09-10T04:15:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.675+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc510103040 con 0x7fc510101900 2024-09-10T04:15:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.676+0000 7fc50d7fa640 1 -- 172.21.15.46:0/1969002142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc500004e60 con 0x7fc510101900 2024-09-10T04:15:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.677+0000 7fc50d7fa640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f807abf0 0x7fc4f807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.677+0000 7fc50d7fa640 1 -- 172.21.15.46:0/1969002142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc50009d820 con 0x7fc510101900 2024-09-10T04:15:53.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.677+0000 7fc50f7fe640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f807abf0 0x7fc4f807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.677+0000 7fc50f7fe640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f807abf0 0x7fc4f807d0b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc510198af0 tx=0x7fc504031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:53.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.681+0000 7fc50d7fa640 1 -- 172.21.15.46:0/1969002142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc500068f60 con 0x7fc510101900 2024-09-10T04:15:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:53 smithi070 ceph-mon[79774]: pgmap v306: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:53 smithi070 ceph-mon[79774]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:53 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2231419949' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:15:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:53 smithi046 ceph-mon[109506]: pgmap v306: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:53 smithi046 ceph-mon[109506]: from='client.34450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:15:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:53 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2231419949' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:15:53.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.885+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc5101982a0 con 0x7fc4f807abf0 2024-09-10T04:15:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.886+0000 7fc50d7fa640 1 -- 172.21.15.46:0/1969002142 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fc5101982a0 con 0x7fc4f807abf0 2024-09-10T04:15:53.888 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:15:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:15:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:15:53.889 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:15:53.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f807abf0 msgr2=0x7fc4f807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f807abf0 0x7fc4f807d0b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc510198af0 tx=0x7fc504031400 comp rx=0 tx=0).stop 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 msgr2=0x7fc51019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc51019d620 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc50000d6f0 tx=0x7fc50000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 shutdown_connections 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f807abf0 0x7fc4f807d0b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc510107900 0x7fc51019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 --2- 172.21.15.46:0/1969002142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc510101900 0x7fc51019d620 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 >> 172.21.15.46:0/1969002142 conn(0x7fc5100fd600 msgr2=0x7fc5100fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 shutdown_connections 2024-09-10T04:15:53.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.889+0000 7fc516354640 1 -- 172.21.15.46:0/1969002142 wait complete. 2024-09-10T04:15:53.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.974+0000 7f12e689f640 1 -- 172.21.15.46:0/1104617828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01007f0 msgr2=0x7f12e0100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.974+0000 7f12e689f640 1 --2- 172.21.15.46:0/1104617828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01007f0 0x7f12e0100c50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f12d4009880 tx=0x7f12d402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.974+0000 7f12e689f640 1 -- 172.21.15.46:0/1104617828 shutdown_connections 2024-09-10T04:15:53.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.974+0000 7f12e689f640 1 --2- 172.21.15.46:0/1104617828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01007f0 0x7f12e0100c50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.974+0000 7f12e689f640 1 --2- 172.21.15.46:0/1104617828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e01067f0 0x7f12e0106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.974+0000 7f12e689f640 1 -- 172.21.15.46:0/1104617828 >> 172.21.15.46:0/1104617828 conn(0x7f12e00fc550 msgr2=0x7f12e00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.975+0000 7f12e689f640 1 -- 172.21.15.46:0/1104617828 shutdown_connections 2024-09-10T04:15:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.975+0000 7f12e689f640 1 -- 172.21.15.46:0/1104617828 wait complete. 2024-09-10T04:15:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.975+0000 7f12e689f640 1 Processor -- start 2024-09-10T04:15:53.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.975+0000 7f12e689f640 1 -- start start 2024-09-10T04:15:53.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12e689f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e01007f0 0x7f12e019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12e689f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01067f0 0x7f12e019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12e689f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12e010dcc0 con 0x7f12e01067f0 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12e689f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f12e010db40 con 0x7f12e01007f0 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01067f0 0x7f12e019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12df7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01067f0 0x7f12e019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59500/0 (socket says 172.21.15.46:59500) 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12df7fe640 1 -- 172.21.15.46:0/2732167378 learned_addr learned my addr 172.21.15.46:0/2732167378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12dffff640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e01007f0 0x7f12e019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12df7fe640 1 -- 172.21.15.46:0/2732167378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e01007f0 msgr2=0x7f12e019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12df7fe640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e01007f0 0x7f12e019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12df7fe640 1 -- 172.21.15.46:0/2732167378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e01977e0 con 0x7f12e01067f0 2024-09-10T04:15:53.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.976+0000 7f12df7fe640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01067f0 0x7f12e019dba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f12d4031460 tx=0x7f12d4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.977+0000 7f12dd7fa640 1 -- 172.21.15.46:0/2732167378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f12d403f070 con 0x7f12e01067f0 2024-09-10T04:15:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.977+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e0197a70 con 0x7f12e01067f0 2024-09-10T04:15:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.977+0000 7f12dd7fa640 1 -- 172.21.15.46:0/2732167378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f12d40035f0 con 0x7f12e01067f0 2024-09-10T04:15:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.977+0000 7f12dd7fa640 1 -- 172.21.15.46:0/2732167378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f12d4035030 con 0x7f12e01067f0 2024-09-10T04:15:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.977+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e0197fa0 con 0x7f12e01067f0 2024-09-10T04:15:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.978+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e0101f30 con 0x7f12e01067f0 2024-09-10T04:15:53.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.979+0000 7f12dd7fa640 1 -- 172.21.15.46:0/2732167378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f12d4042070 con 0x7f12e01067f0 2024-09-10T04:15:53.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.979+0000 7f12dd7fa640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f12c807ac40 0x7f12c807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:15:53.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.979+0000 7f12dd7fa640 1 -- 172.21.15.46:0/2732167378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f12d40be4a0 con 0x7f12e01067f0 2024-09-10T04:15:53.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.979+0000 7f12dffff640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f12c807ac40 0x7f12c807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:15:53.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.980+0000 7f12dffff640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f12c807ac40 0x7f12c807d100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009bd0 tx=0x7f12d0005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:15:53.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:53.984+0000 7f12dd7fa640 1 -- 172.21.15.46:0/2732167378 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f12d4089c60 con 0x7f12e01067f0 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.242+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f12e0198d10 con 0x7f12e01067f0 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.242+0000 7f12dd7fa640 1 -- 172.21.15.46:0/2732167378 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f12d408eb10 con 0x7f12e01067f0 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:15:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:15:54.245 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:15:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f12c807ac40 msgr2=0x7f12c807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f12c807ac40 0x7f12c807d100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009bd0 tx=0x7f12d0005f90 comp rx=0 tx=0).stop 2024-09-10T04:15:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01067f0 msgr2=0x7f12e019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01067f0 0x7f12e019dba0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f12d4031460 tx=0x7f12d4004530 comp rx=0 tx=0).stop 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 shutdown_connections 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f12c807ac40 0x7f12c807d100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e01067f0 0x7f12e019dba0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 --2- 172.21.15.46:0/2732167378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12e01007f0 0x7f12e019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.244+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 >> 172.21.15.46:0/2732167378 conn(0x7f12e00fc550 msgr2=0x7f12e0069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.245+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 shutdown_connections 2024-09-10T04:15:54.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:15:54.245+0000 7f12e689f640 1 -- 172.21.15.46:0/2732167378 wait complete. 2024-09-10T04:15:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:54 smithi070 ceph-mon[79774]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:54 smithi070 ceph-mon[79774]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:54 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3960371536' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:15:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:54 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2732167378' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:15:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:54 smithi046 ceph-mon[109506]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:54 smithi046 ceph-mon[109506]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:54 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3960371536' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:15:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:54 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2732167378' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:15:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:55 smithi070 ceph-mon[79774]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:55 smithi070 ceph-mon[79774]: pgmap v307: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:55 smithi046 ceph-mon[109506]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:15:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:55 smithi046 ceph-mon[109506]: pgmap v307: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:57 smithi070 ceph-mon[79774]: pgmap v308: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:57 smithi046 ceph-mon[109506]: pgmap v308: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:15:59 smithi070 ceph-mon[79774]: pgmap v309: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:15:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:15:59 smithi046 ceph-mon[109506]: pgmap v309: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:01 smithi070 ceph-mon[79774]: pgmap v310: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:01 smithi046 ceph-mon[109506]: pgmap v310: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:03 smithi070 ceph-mon[79774]: pgmap v311: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:03 smithi046 ceph-mon[109506]: pgmap v311: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:05 smithi070 ceph-mon[79774]: pgmap v312: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:05.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:05 smithi046 ceph-mon[109506]: pgmap v312: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:07 smithi070 ceph-mon[79774]: pgmap v313: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:07 smithi046 ceph-mon[109506]: pgmap v313: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:09 smithi070 ceph-mon[79774]: pgmap v314: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:09 smithi046 ceph-mon[109506]: pgmap v314: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:11 smithi070 ceph-mon[79774]: pgmap v315: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:11 smithi046 ceph-mon[109506]: pgmap v315: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:13 smithi070 ceph-mon[79774]: pgmap v316: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:13 smithi046 ceph-mon[109506]: pgmap v316: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:15 smithi070 ceph-mon[79774]: pgmap v317: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:15 smithi046 ceph-mon[109506]: pgmap v317: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:17 smithi070 ceph-mon[79774]: pgmap v318: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:17 smithi046 ceph-mon[109506]: pgmap v318: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:19 smithi070 ceph-mon[79774]: pgmap v319: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:19 smithi046 ceph-mon[109506]: pgmap v319: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:21 smithi070 ceph-mon[79774]: pgmap v320: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:21 smithi046 ceph-mon[109506]: pgmap v320: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:23 smithi070 ceph-mon[79774]: pgmap v321: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:23 smithi046 ceph-mon[109506]: pgmap v321: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:24.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.342+0000 7fe5d3cec640 1 -- 172.21.15.46:0/2151701565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 msgr2=0x7fe5cc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.342+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/2151701565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc100a30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009880 tx=0x7fe5c802a490 comp rx=0 tx=0).stop 2024-09-10T04:16:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.342+0000 7fe5d3cec640 1 -- 172.21.15.46:0/2151701565 shutdown_connections 2024-09-10T04:16:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.342+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/2151701565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5cc101000 0x7fe5cc10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.342+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/2151701565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc100a30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.342+0000 7fe5d3cec640 1 -- 172.21.15.46:0/2151701565 >> 172.21.15.46:0/2151701565 conn(0x7fe5cc0fc520 msgr2=0x7fe5cc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.343+0000 7fe5d3cec640 1 -- 172.21.15.46:0/2151701565 shutdown_connections 2024-09-10T04:16:24.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.343+0000 7fe5d3cec640 1 -- 172.21.15.46:0/2151701565 wait complete. 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.343+0000 7fe5d3cec640 1 Processor -- start 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.343+0000 7fe5d3cec640 1 -- start start 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d3cec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d3cec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5cc101000 0x7fe5cc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d3cec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc110fb0 con 0x7fe5cc100650 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d3cec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc110e30 con 0x7fe5cc101000 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1a88640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1a88640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49962/0 (socket says 172.21.15.46:49962) 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1a88640 1 -- 172.21.15.46:0/1135981273 learned_addr learned my addr 172.21.15.46:0/1135981273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1287640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5cc101000 0x7fe5cc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1a88640 1 -- 172.21.15.46:0/1135981273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5cc101000 msgr2=0x7fe5cc19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1a88640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5cc101000 0x7fe5cc19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1a88640 1 -- 172.21.15.46:0/1135981273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc1999a0 con 0x7fe5cc100650 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5d1a88640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc19f7e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8031460 tx=0x7fe5c8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5c2ffd640 1 -- 172.21.15.46:0/1135981273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5c803f070 con 0x7fe5cc100650 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.344+0000 7fe5c2ffd640 1 -- 172.21.15.46:0/1135981273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5c8004a90 con 0x7fe5cc100650 2024-09-10T04:16:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.345+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc199c30 con 0x7fe5cc100650 2024-09-10T04:16:24.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.345+0000 7fe5c2ffd640 1 -- 172.21.15.46:0/1135981273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe5c8035030 con 0x7fe5cc100650 2024-09-10T04:16:24.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.345+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc19a1c0 con 0x7fe5cc100650 2024-09-10T04:16:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.346+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc06aa90 con 0x7fe5cc100650 2024-09-10T04:16:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.346+0000 7fe5c2ffd640 1 -- 172.21.15.46:0/1135981273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe5c8042070 con 0x7fe5cc100650 2024-09-10T04:16:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.347+0000 7fe5c2ffd640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5b807ac00 0x7fe5b807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.347+0000 7fe5d1287640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5b807ac00 0x7fe5b807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.347+0000 7fe5c2ffd640 1 -- 172.21.15.46:0/1135981273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fe5c80be7b0 con 0x7fe5cc100650 2024-09-10T04:16:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.347+0000 7fe5d1287640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5b807ac00 0x7fe5b807d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc102520 tx=0x7fe5bc005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:24.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.351+0000 7fe5c2ffd640 1 -- 172.21.15.46:0/1135981273 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe5c8089ef0 con 0x7fe5cc100650 2024-09-10T04:16:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.605+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe5cc19aed0 con 0x7fe5b807ac00 2024-09-10T04:16:24.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.607+0000 7fe5c2ffd640 1 -- 172.21.15.46:0/1135981273 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fe5cc19aed0 con 0x7fe5b807ac00 2024-09-10T04:16:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5b807ac00 msgr2=0x7fe5b807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5b807ac00 0x7fe5b807d0c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc102520 tx=0x7fe5bc005f90 comp rx=0 tx=0).stop 2024-09-10T04:16:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 msgr2=0x7fe5cc19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc19f7e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8031460 tx=0x7fe5c8004530 comp rx=0 tx=0).stop 2024-09-10T04:16:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 shutdown_connections 2024-09-10T04:16:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe5b807ac00 0x7fe5b807d0c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5cc101000 0x7fe5cc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 --2- 172.21.15.46:0/1135981273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5cc100650 0x7fe5cc19f7e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 >> 172.21.15.46:0/1135981273 conn(0x7fe5cc0fc520 msgr2=0x7fe5cc0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 shutdown_connections 2024-09-10T04:16:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.609+0000 7fe5d3cec640 1 -- 172.21.15.46:0/1135981273 wait complete. 2024-09-10T04:16:24.620 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 -- 172.21.15.46:0/1056517048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 msgr2=0x7f9d54100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1056517048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d54100c60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009880 tx=0x7f9d4402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 -- 172.21.15.46:0/1056517048 shutdown_connections 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1056517048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d54100c60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1056517048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d54106800 0x7f9d54106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 -- 172.21.15.46:0/1056517048 >> 172.21.15.46:0/1056517048 conn(0x7f9d540fc520 msgr2=0x7f9d540fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 -- 172.21.15.46:0/1056517048 shutdown_connections 2024-09-10T04:16:24.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.695+0000 7f9d5b311640 1 -- 172.21.15.46:0/1056517048 wait complete. 2024-09-10T04:16:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d5b311640 1 Processor -- start 2024-09-10T04:16:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d5b311640 1 -- start start 2024-09-10T04:16:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d5b311640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d540fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d5b311640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d54106800 0x7f9d540fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d5b311640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5410dcd0 con 0x7f9d54100800 2024-09-10T04:16:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d5b311640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5410db50 con 0x7f9d54106800 2024-09-10T04:16:24.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d590ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d540fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d590ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d540fcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49976/0 (socket says 172.21.15.46:49976) 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d590ad640 1 -- 172.21.15.46:0/1986322424 learned_addr learned my addr 172.21.15.46:0/1986322424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d588ac640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d54106800 0x7f9d540fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d590ad640 1 -- 172.21.15.46:0/1986322424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d54106800 msgr2=0x7f9d540fd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d590ad640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d54106800 0x7f9d540fd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.696+0000 7f9d590ad640 1 -- 172.21.15.46:0/1986322424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d540fd630 con 0x7f9d54100800 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.697+0000 7f9d590ad640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d540fcbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c8f0 tx=0x7f9d5000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.697+0000 7f9d4a7fc640 1 -- 172.21.15.46:0/1986322424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9d500079b0 con 0x7f9d54100800 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.697+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d540fd920 con 0x7f9d54100800 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.697+0000 7f9d4a7fc640 1 -- 172.21.15.46:0/1986322424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9d500054e0 con 0x7f9d54100800 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.697+0000 7f9d4a7fc640 1 -- 172.21.15.46:0/1986322424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9d50004dd0 con 0x7f9d54100800 2024-09-10T04:16:24.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.697+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541a3e70 con 0x7f9d54100800 2024-09-10T04:16:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.698+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d54101f40 con 0x7f9d54100800 2024-09-10T04:16:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.699+0000 7f9d4a7fc640 1 -- 172.21.15.46:0/1986322424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9d50005020 con 0x7f9d54100800 2024-09-10T04:16:24.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.699+0000 7f9d4a7fc640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9d4007ab90 0x7f9d4007d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.699+0000 7f9d4a7fc640 1 -- 172.21.15.46:0/1986322424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f9d5009dc50 con 0x7f9d54100800 2024-09-10T04:16:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.699+0000 7f9d588ac640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9d4007ab90 0x7f9d4007d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:24.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.700+0000 7f9d588ac640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9d4007ab90 0x7f9d4007d050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d540fe640 tx=0x7f9d44031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:24.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.704+0000 7f9d4a7fc640 1 -- 172.21.15.46:0/1986322424 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9d50014030 con 0x7f9d54100800 2024-09-10T04:16:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.907+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9d54062630 con 0x7f9d4007ab90 2024-09-10T04:16:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.909+0000 7f9d4a7fc640 1 -- 172.21.15.46:0/1986322424 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f9d54062630 con 0x7f9d4007ab90 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9d4007ab90 msgr2=0x7f9d4007d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9d4007ab90 0x7f9d4007d050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d540fe640 tx=0x7f9d44031420 comp rx=0 tx=0).stop 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 msgr2=0x7f9d540fcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d540fcbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c8f0 tx=0x7f9d5000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 shutdown_connections 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9d4007ab90 0x7f9d4007d050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d54106800 0x7f9d540fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 --2- 172.21.15.46:0/1986322424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d54100800 0x7f9d540fcbb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 >> 172.21.15.46:0/1986322424 conn(0x7f9d540fc520 msgr2=0x7f9d5406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 shutdown_connections 2024-09-10T04:16:24.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.911+0000 7f9d5b311640 1 -- 172.21.15.46:0/1986322424 wait complete. 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 -- 172.21.15.46:0/4141397979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 msgr2=0x7f6948102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 --2- 172.21.15.46:0/4141397979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f6948102e90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6938009880 tx=0x7f693802a530 comp rx=0 tx=0).stop 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 -- 172.21.15.46:0/4141397979 shutdown_connections 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 --2- 172.21.15.46:0/4141397979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69481033d0 0x7f694810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 --2- 172.21.15.46:0/4141397979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f6948102e90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 -- 172.21.15.46:0/4141397979 >> 172.21.15.46:0/4141397979 conn(0x7f69480fc550 msgr2=0x7f69480fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 -- 172.21.15.46:0/4141397979 shutdown_connections 2024-09-10T04:16:24.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.996+0000 7f694ca42640 1 -- 172.21.15.46:0/4141397979 wait complete. 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694ca42640 1 Processor -- start 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694ca42640 1 -- start start 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694ca42640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f694819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694ca42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69481033d0 0x7f694819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694ca42640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f694810ede0 con 0x7f6948102ab0 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694ca42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f694810ec60 con 0x7f69481033d0 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f694819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f694819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49992/0 (socket says 172.21.15.46:49992) 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694659c640 1 -- 172.21.15.46:0/2133178154 learned_addr learned my addr 172.21.15.46:0/2133178154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f6945d9b640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69481033d0 0x7f694819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694659c640 1 -- 172.21.15.46:0/2133178154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69481033d0 msgr2=0x7f694819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694659c640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69481033d0 0x7f694819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.997+0000 7f694659c640 1 -- 172.21.15.46:0/2133178154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69481977b0 con 0x7f6948102ab0 2024-09-10T04:16:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.998+0000 7f694659c640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f694819d630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6938031460 tx=0x7f69380048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.998+0000 7f69377fe640 1 -- 172.21.15.46:0/2133178154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f693803f070 con 0x7f6948102ab0 2024-09-10T04:16:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.998+0000 7f69377fe640 1 -- 172.21.15.46:0/2133178154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6938004cf0 con 0x7f6948102ab0 2024-09-10T04:16:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.998+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6948197a40 con 0x7f6948102ab0 2024-09-10T04:16:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.998+0000 7f69377fe640 1 -- 172.21.15.46:0/2133178154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6938035030 con 0x7f6948102ab0 2024-09-10T04:16:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.998+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6948197f70 con 0x7f6948102ab0 2024-09-10T04:16:25.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.999+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694806a580 con 0x7f6948102ab0 2024-09-10T04:16:25.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:24.999+0000 7f69377fe640 1 -- 172.21.15.46:0/2133178154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6938042070 con 0x7f6948102ab0 2024-09-10T04:16:25.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.000+0000 7f69377fe640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f693007abf0 0x7f693007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.000+0000 7f6945d9b640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f693007abf0 0x7f693007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.000+0000 7f69377fe640 1 -- 172.21.15.46:0/2133178154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f69380be740 con 0x7f6948102ab0 2024-09-10T04:16:25.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.001+0000 7f6945d9b640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f693007abf0 0x7f693007d0b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69481088f0 tx=0x7f693c005f50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.005+0000 7f69377fe640 1 -- 172.21.15.46:0/2133178154 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6938089de0 con 0x7f6948102ab0 2024-09-10T04:16:25.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.200+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6948198220 con 0x7f693007abf0 2024-09-10T04:16:25.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.210+0000 7f69377fe640 1 -- 172.21.15.46:0/2133178154 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f6948198220 con 0x7f693007abf0 2024-09-10T04:16:25.211 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (8m) 2m ago 19m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (10m) 2m ago 19m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (10m) 2m ago 18m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (6m) 2m ago 19m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (6m) 2m ago 18m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (7m) 2m ago 19m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (2m) 2m ago 13m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (2m) 2m ago 14m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (2m) 2m ago 13m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (13m) 2m ago 13m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (10m) 2m ago 20m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (9m) 2m ago 18m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (6m) 2m ago 20m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (6m) 2m ago 18m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:16:25.212 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (8m) 2m ago 19m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (8m) 2m ago 18m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (5m) 2m ago 17m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (5m) 2m ago 16m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (4m) 2m ago 16m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (4m) 2m ago 15m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (3m) 2m ago 15m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (3m) 2m ago 15m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:16:25.213 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (8m) 2m ago 18m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:16:25.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f693007abf0 msgr2=0x7f693007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f693007abf0 0x7f693007d0b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f69481088f0 tx=0x7f693c005f50 comp rx=0 tx=0).stop 2024-09-10T04:16:25.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 msgr2=0x7f694819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f694819d630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6938031460 tx=0x7f69380048f0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 shutdown_connections 2024-09-10T04:16:25.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f693007abf0 0x7f693007d0b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69481033d0 0x7f694819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 --2- 172.21.15.46:0/2133178154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6948102ab0 0x7f694819d630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 >> 172.21.15.46:0/2133178154 conn(0x7f69480fc550 msgr2=0x7f694810aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:25.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 shutdown_connections 2024-09-10T04:16:25.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.212+0000 7f694ca42640 1 -- 172.21.15.46:0/2133178154 wait complete. 2024-09-10T04:16:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 -- 172.21.15.46:0/3275770929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294107900 msgr2=0x7fd294107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/3275770929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294107900 0x7fd294107ce0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd284009880 tx=0x7fd28402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 -- 172.21.15.46:0/3275770929 shutdown_connections 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/3275770929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd294101900 0x7fd294101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/3275770929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294107900 0x7fd294107ce0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 -- 172.21.15.46:0/3275770929 >> 172.21.15.46:0/3275770929 conn(0x7fd2940fd600 msgr2=0x7fd2940ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 -- 172.21.15.46:0/3275770929 shutdown_connections 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.297+0000 7fd29a8f6640 1 -- 172.21.15.46:0/3275770929 wait complete. 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd29a8f6640 1 Processor -- start 2024-09-10T04:16:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd29a8f6640 1 -- start start 2024-09-10T04:16:25.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd29a8f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294101900 0x7fd29419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd29a8f6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd294107900 0x7fd29419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd29a8f6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd29410edd0 con 0x7fd294101900 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd29a8f6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd29410ec50 con 0x7fd294107900 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd293fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294101900 0x7fd29419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd293fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294101900 0x7fd29419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50010/0 (socket says 172.21.15.46:50010) 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd293fff640 1 -- 172.21.15.46:0/2959201359 learned_addr learned my addr 172.21.15.46:0/2959201359 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.298+0000 7fd2937fe640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd294107900 0x7fd29419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd293fff640 1 -- 172.21.15.46:0/2959201359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd294107900 msgr2=0x7fd29419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd293fff640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd294107900 0x7fd29419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd293fff640 1 -- 172.21.15.46:0/2959201359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2941977a0 con 0x7fd294101900 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd293fff640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294101900 0x7fd29419d620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd284031460 tx=0x7fd284004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd2917fa640 1 -- 172.21.15.46:0/2959201359 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd28403f070 con 0x7fd294101900 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd2917fa640 1 -- 172.21.15.46:0/2959201359 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd2840035f0 con 0x7fd294101900 2024-09-10T04:16:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd2917fa640 1 -- 172.21.15.46:0/2959201359 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd284035030 con 0x7fd294101900 2024-09-10T04:16:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd294197a30 con 0x7fd294101900 2024-09-10T04:16:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.299+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd294197f60 con 0x7fd294101900 2024-09-10T04:16:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.300+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd294103040 con 0x7fd294101900 2024-09-10T04:16:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.301+0000 7fd2917fa640 1 -- 172.21.15.46:0/2959201359 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd284042070 con 0x7fd294101900 2024-09-10T04:16:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.301+0000 7fd2917fa640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd27c07ac40 0x7fd27c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.302+0000 7fd2917fa640 1 -- 172.21.15.46:0/2959201359 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fd2840be4a0 con 0x7fd294101900 2024-09-10T04:16:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.302+0000 7fd2937fe640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd27c07ac40 0x7fd27c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.302+0000 7fd2937fe640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd27c07ac40 0x7fd27c07d100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd2940ffc20 tx=0x7fd288005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.306+0000 7fd2917fa640 1 -- 172.21.15.46:0/2959201359 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd284089c60 con 0x7fd294101900 2024-09-10T04:16:25.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.565+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fd294109e70 con 0x7fd294101900 2024-09-10T04:16:25.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.565+0000 7fd2917fa640 1 -- 172.21.15.46:0/2959201359 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fd28408eb10 con 0x7fd294101900 2024-09-10T04:16:25.567 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:16:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:16:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:16:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:25.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:16:25.568 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:16:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd27c07ac40 msgr2=0x7fd27c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd27c07ac40 0x7fd27c07d100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd2940ffc20 tx=0x7fd288005f90 comp rx=0 tx=0).stop 2024-09-10T04:16:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294101900 msgr2=0x7fd29419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294101900 0x7fd29419d620 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd284031460 tx=0x7fd284004530 comp rx=0 tx=0).stop 2024-09-10T04:16:25.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 shutdown_connections 2024-09-10T04:16:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd27c07ac40 0x7fd27c07d100 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd294107900 0x7fd29419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 --2- 172.21.15.46:0/2959201359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd294101900 0x7fd29419d620 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 >> 172.21.15.46:0/2959201359 conn(0x7fd2940fd600 msgr2=0x7fd2940fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 shutdown_connections 2024-09-10T04:16:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.568+0000 7fd29a8f6640 1 -- 172.21.15.46:0/2959201359 wait complete. 2024-09-10T04:16:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 -- 172.21.15.46:0/2611252202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 msgr2=0x7fcd00107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/2611252202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd00107ce0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009880 tx=0x7fccf002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 -- 172.21.15.46:0/2611252202 shutdown_connections 2024-09-10T04:16:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/2611252202 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd00101900 0x7fcd00101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/2611252202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd00107ce0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 -- 172.21.15.46:0/2611252202 >> 172.21.15.46:0/2611252202 conn(0x7fcd000fd600 msgr2=0x7fcd000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 -- 172.21.15.46:0/2611252202 shutdown_connections 2024-09-10T04:16:25.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 -- 172.21.15.46:0/2611252202 wait complete. 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.653+0000 7fcd05ccd640 1 Processor -- start 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fcd05ccd640 1 -- start start 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fcd05ccd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd00101900 0x7fcd0019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fcd05ccd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd0019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fcd05ccd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010edd0 con 0x7fcd00107900 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fcd05ccd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0010ec50 con 0x7fcd00101900 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fccfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd0019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fccfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd0019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50036/0 (socket says 172.21.15.46:50036) 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fccfeffd640 1 -- 172.21.15.46:0/4109187078 learned_addr learned my addr 172.21.15.46:0/4109187078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fccff7fe640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd00101900 0x7fcd0019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fccfeffd640 1 -- 172.21.15.46:0/4109187078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd00101900 msgr2=0x7fcd0019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fccfeffd640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd00101900 0x7fcd0019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.654+0000 7fccfeffd640 1 -- 172.21.15.46:0/4109187078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd001977a0 con 0x7fcd00107900 2024-09-10T04:16:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.655+0000 7fccfeffd640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd0019db60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d6f0 tx=0x7fccf400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.655+0000 7fccfcff9640 1 -- 172.21.15.46:0/4109187078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fccf4017070 con 0x7fcd00107900 2024-09-10T04:16:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.655+0000 7fccfcff9640 1 -- 172.21.15.46:0/4109187078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fccf400de00 con 0x7fcd00107900 2024-09-10T04:16:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.655+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd00197a90 con 0x7fcd00107900 2024-09-10T04:16:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.655+0000 7fccfcff9640 1 -- 172.21.15.46:0/4109187078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fccf4004bb0 con 0x7fcd00107900 2024-09-10T04:16:25.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.655+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd00198020 con 0x7fcd00107900 2024-09-10T04:16:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.656+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd00103040 con 0x7fcd00107900 2024-09-10T04:16:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.657+0000 7fccfcff9640 1 -- 172.21.15.46:0/4109187078 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fccf4009a30 con 0x7fcd00107900 2024-09-10T04:16:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.657+0000 7fccfcff9640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcce807ac40 0x7fcce807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.657+0000 7fccfcff9640 1 -- 172.21.15.46:0/4109187078 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fccf409d7f0 con 0x7fcd00107900 2024-09-10T04:16:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.657+0000 7fccff7fe640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcce807ac40 0x7fcce807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.658+0000 7fccff7fe640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcce807ac40 0x7fcce807d100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fccf0031460 tx=0x7fccf0009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.662+0000 7fccfcff9640 1 -- 172.21.15.46:0/4109187078 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fccf4068fb0 con 0x7fcd00107900 2024-09-10T04:16:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:25 smithi070 ceph-mon[79774]: pgmap v322: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:25 smithi046 ceph-mon[109506]: pgmap v322: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.886+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fcd00198d00 con 0x7fcd00107900 2024-09-10T04:16:25.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.886+0000 7fccfcff9640 1 -- 172.21.15.46:0/4109187078 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7fcd00198d00 con 0x7fcd00107900 2024-09-10T04:16:25.888 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:16:25.889 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:16:25.890 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:16:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcce807ac40 msgr2=0x7fcce807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcce807ac40 0x7fcce807d100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fccf0031460 tx=0x7fccf0009560 comp rx=0 tx=0).stop 2024-09-10T04:16:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 msgr2=0x7fcd0019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd0019db60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fccf400d6f0 tx=0x7fccf400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 shutdown_connections 2024-09-10T04:16:25.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcce807ac40 0x7fcce807d100 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd00107900 0x7fcd0019db60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 --2- 172.21.15.46:0/4109187078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd00101900 0x7fcd0019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 >> 172.21.15.46:0/4109187078 conn(0x7fcd000fd600 msgr2=0x7fcd000fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 shutdown_connections 2024-09-10T04:16:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.889+0000 7fcd05ccd640 1 -- 172.21.15.46:0/4109187078 wait complete. 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 -- 172.21.15.46:0/2768327566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 msgr2=0x7f8518101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 --2- 172.21.15.46:0/2768327566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f8518101d60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8504009880 tx=0x7f850402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 -- 172.21.15.46:0/2768327566 shutdown_connections 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 --2- 172.21.15.46:0/2768327566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f8518101d60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 --2- 172.21.15.46:0/2768327566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8518107900 0x7f8518107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 -- 172.21.15.46:0/2768327566 >> 172.21.15.46:0/2768327566 conn(0x7f85180fd600 msgr2=0x7f85180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 -- 172.21.15.46:0/2768327566 shutdown_connections 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.974+0000 7f851f020640 1 -- 172.21.15.46:0/2768327566 wait complete. 2024-09-10T04:16:25.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851f020640 1 Processor -- start 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851f020640 1 -- start start 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851f020640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f851819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851f020640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8518107900 0x7f851819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851f020640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f851810edd0 con 0x7f8518101900 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851f020640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f851810ec50 con 0x7f8518107900 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851cdbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f851819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851cdbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f851819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50046/0 (socket says 172.21.15.46:50046) 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.975+0000 7f851cdbc640 1 -- 172.21.15.46:0/390855229 learned_addr learned my addr 172.21.15.46:0/390855229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f8517fff640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8518107900 0x7f851819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f851cdbc640 1 -- 172.21.15.46:0/390855229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8518107900 msgr2=0x7f851819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f851cdbc640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8518107900 0x7f851819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:25.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f851cdbc640 1 -- 172.21.15.46:0/390855229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85181977a0 con 0x7f8518101900 2024-09-10T04:16:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f851cdbc640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f851819d620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e970 tx=0x7f850c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f8515ffb640 1 -- 172.21.15.46:0/390855229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f850c00cac0 con 0x7f8518101900 2024-09-10T04:16:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f8515ffb640 1 -- 172.21.15.46:0/390855229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f850c0054e0 con 0x7f8518101900 2024-09-10T04:16:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8518197a90 con 0x7f8518101900 2024-09-10T04:16:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f8515ffb640 1 -- 172.21.15.46:0/390855229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f850c010430 con 0x7f8518101900 2024-09-10T04:16:25.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.976+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8518197ff0 con 0x7f8518101900 2024-09-10T04:16:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.977+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8518103040 con 0x7f8518101900 2024-09-10T04:16:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.978+0000 7f8515ffb640 1 -- 172.21.15.46:0/390855229 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f850c004f00 con 0x7f8518101900 2024-09-10T04:16:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.978+0000 7f8515ffb640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f850807ac40 0x7f850807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.978+0000 7f8515ffb640 1 -- 172.21.15.46:0/390855229 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f850c09e370 con 0x7f8518101900 2024-09-10T04:16:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.979+0000 7f8517fff640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f850807ac40 0x7f850807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:25.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.979+0000 7f8517fff640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f850807ac40 0x7f850807d100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8518198af0 tx=0x7f8504031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:25.983+0000 7f8515ffb640 1 -- 172.21.15.46:0/390855229 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f850c069a90 con 0x7f8518101900 2024-09-10T04:16:26.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.186+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8518198300 con 0x7f850807ac40 2024-09-10T04:16:26.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.187+0000 7f8515ffb640 1 -- 172.21.15.46:0/390855229 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f8518198300 con 0x7f850807ac40 2024-09-10T04:16:26.189 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:16:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:16:26.189 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:16:26.190 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:16:26.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.189+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f850807ac40 msgr2=0x7f850807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:26.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.189+0000 7f851f020640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f850807ac40 0x7f850807d100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8518198af0 tx=0x7f8504031400 comp rx=0 tx=0).stop 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.189+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 msgr2=0x7f851819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.189+0000 7f851f020640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f851819d620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f850c00e970 tx=0x7f850c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.190+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 shutdown_connections 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.190+0000 7f851f020640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f850807ac40 0x7f850807d100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.190+0000 7f851f020640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8518107900 0x7f851819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.190+0000 7f851f020640 1 --2- 172.21.15.46:0/390855229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8518101900 0x7f851819d620 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.190+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 >> 172.21.15.46:0/390855229 conn(0x7f85180fd600 msgr2=0x7f85180fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.190+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 shutdown_connections 2024-09-10T04:16:26.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.190+0000 7f851f020640 1 -- 172.21.15.46:0/390855229 wait complete. 2024-09-10T04:16:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 -- 172.21.15.46:0/2494213954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 msgr2=0x7f5f5010b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2494213954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5010b990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f400098b0 tx=0x7f5f4002a560 comp rx=0 tx=0).stop 2024-09-10T04:16:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 -- 172.21.15.46:0/2494213954 shutdown_connections 2024-09-10T04:16:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2494213954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5010b990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2494213954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f50102b80 0x7f5f50102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 -- 172.21.15.46:0/2494213954 >> 172.21.15.46:0/2494213954 conn(0x7f5f500fc580 msgr2=0x7f5f500fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:26.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 -- 172.21.15.46:0/2494213954 shutdown_connections 2024-09-10T04:16:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 -- 172.21.15.46:0/2494213954 wait complete. 2024-09-10T04:16:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 Processor -- start 2024-09-10T04:16:26.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.276+0000 7f5f583f8640 1 -- start start 2024-09-10T04:16:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f583f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f50102b80 0x7f5f5019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f583f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f583f8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5010ee30 con 0x7f5f501034a0 2024-09-10T04:16:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f583f8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5010ecb0 con 0x7f5f50102b80 2024-09-10T04:16:26.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f55993640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f55993640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5019fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50062/0 (socket says 172.21.15.46:50062) 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f55993640 1 -- 172.21.15.46:0/2295019685 learned_addr learned my addr 172.21.15.46:0/2295019685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f56194640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f50102b80 0x7f5f5019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f55993640 1 -- 172.21.15.46:0/2295019685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f50102b80 msgr2=0x7f5f5019f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f55993640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f50102b80 0x7f5f5019f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f55993640 1 -- 172.21.15.46:0/2295019685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f501a04c0 con 0x7f5f501034a0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.277+0000 7f5f55993640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5019fe10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f40031460 tx=0x7f5f400048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.278+0000 7f5f477fe640 1 -- 172.21.15.46:0/2295019685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5f4003f070 con 0x7f5f501034a0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.278+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f50199b50 con 0x7f5f501034a0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.278+0000 7f5f477fe640 1 -- 172.21.15.46:0/2295019685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f400044d0 con 0x7f5f501034a0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.278+0000 7f5f477fe640 1 -- 172.21.15.46:0/2295019685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5f40035030 con 0x7f5f501034a0 2024-09-10T04:16:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.278+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5019a050 con 0x7f5f501034a0 2024-09-10T04:16:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.279+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5006ad40 con 0x7f5f501034a0 2024-09-10T04:16:26.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.279+0000 7f5f477fe640 1 -- 172.21.15.46:0/2295019685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5f40042070 con 0x7f5f501034a0 2024-09-10T04:16:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.280+0000 7f5f477fe640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f3c07abf0 0x7f5f3c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.280+0000 7f5f477fe640 1 -- 172.21.15.46:0/2295019685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f5f400be6d0 con 0x7f5f501034a0 2024-09-10T04:16:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.280+0000 7f5f56194640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f3c07abf0 0x7f5f3c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:26.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.281+0000 7f5f56194640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f3c07abf0 0x7f5f3c07d0b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c005e10 tx=0x7f5f4c005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:26.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.288+0000 7f5f477fe640 1 -- 172.21.15.46:0/2295019685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5f40004cf0 con 0x7f5f501034a0 2024-09-10T04:16:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.546+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f5f5019afc0 con 0x7f5f501034a0 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.547+0000 7f5f477fe640 1 -- 172.21.15.46:0/2295019685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f5f40089e10 con 0x7f5f501034a0 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:16:26.549 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f3c07abf0 msgr2=0x7f5f3c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f3c07abf0 0x7f5f3c07d0b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c005e10 tx=0x7f5f4c005da0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 msgr2=0x7f5f5019fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5019fe10 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f40031460 tx=0x7f5f400048f0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 shutdown_connections 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f3c07abf0 0x7f5f3c07d0b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f501034a0 0x7f5f5019fe10 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 --2- 172.21.15.46:0/2295019685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f50102b80 0x7f5f5019f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 >> 172.21.15.46:0/2295019685 conn(0x7f5f500fc580 msgr2=0x7f5f50100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 shutdown_connections 2024-09-10T04:16:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:26.549+0000 7f5f583f8640 1 -- 172.21.15.46:0/2295019685 wait complete. 2024-09-10T04:16:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:26 smithi070 ceph-mon[79774]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:26 smithi070 ceph-mon[79774]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:26 smithi070 ceph-mon[79774]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:26 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2959201359' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:16:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:26 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4109187078' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:16:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:26 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2295019685' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:16:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:26 smithi046 ceph-mon[109506]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:26 smithi046 ceph-mon[109506]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:26 smithi046 ceph-mon[109506]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:26 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2959201359' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:16:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:26 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4109187078' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:16:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:26 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2295019685' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:16:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:27 smithi070 ceph-mon[79774]: pgmap v323: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:27 smithi070 ceph-mon[79774]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:27.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:27 smithi046 ceph-mon[109506]: pgmap v323: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:27.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:27 smithi046 ceph-mon[109506]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:29 smithi070 ceph-mon[79774]: pgmap v324: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:29.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:29 smithi046 ceph-mon[109506]: pgmap v324: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:31 smithi070 ceph-mon[79774]: pgmap v325: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:31 smithi046 ceph-mon[109506]: pgmap v325: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:33 smithi070 ceph-mon[79774]: pgmap v326: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:33 smithi046 ceph-mon[109506]: pgmap v326: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:35.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:35 smithi046 ceph-mon[109506]: pgmap v327: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:36.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:35 smithi070 ceph-mon[79774]: pgmap v327: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:38.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:37 smithi070 ceph-mon[79774]: pgmap v328: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:38.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:37 smithi046 ceph-mon[109506]: pgmap v328: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:40.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:39 smithi070 ceph-mon[79774]: pgmap v329: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:39 smithi046 ceph-mon[109506]: pgmap v329: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:42.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:41 smithi070 ceph-mon[79774]: pgmap v330: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:42.197 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:41 smithi046 ceph-mon[109506]: pgmap v330: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:43 smithi046 ceph-mon[109506]: pgmap v331: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:43 smithi070 ceph-mon[79774]: pgmap v331: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:45 smithi070 ceph-mon[79774]: pgmap v332: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:45 smithi046 ceph-mon[109506]: pgmap v332: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:46 smithi070 ceph-mon[79774]: pgmap v333: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:46 smithi046 ceph-mon[109506]: pgmap v333: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:49 smithi070 ceph-mon[79774]: pgmap v334: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:49 smithi046 ceph-mon[109506]: pgmap v334: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:51.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:51 smithi070 ceph-mon[79774]: pgmap v335: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:51.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:51 smithi046 ceph-mon[109506]: pgmap v335: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:53.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:53 smithi046 ceph-mon[109506]: pgmap v336: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:53.484 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:53 smithi070 ceph-mon[79774]: pgmap v336: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:16:55.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:55 smithi070 ceph-mon[79774]: pgmap v337: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:55.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:55 smithi046 ceph-mon[109506]: pgmap v337: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.648+0000 7fde27bba640 1 -- 172.21.15.46:0/3822363484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20106800 msgr2=0x7fde20106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.648+0000 7fde27bba640 1 --2- 172.21.15.46:0/3822363484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20106800 0x7fde20106be0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009c20 tx=0x7fde1c02a970 comp rx=0 tx=0).stop 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.648+0000 7fde27bba640 1 -- 172.21.15.46:0/3822363484 shutdown_connections 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.648+0000 7fde27bba640 1 --2- 172.21.15.46:0/3822363484 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde20100800 0x7fde20100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.648+0000 7fde27bba640 1 --2- 172.21.15.46:0/3822363484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20106800 0x7fde20106be0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.648+0000 7fde27bba640 1 -- 172.21.15.46:0/3822363484 >> 172.21.15.46:0/3822363484 conn(0x7fde200fc520 msgr2=0x7fde200fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.648+0000 7fde27bba640 1 -- 172.21.15.46:0/3822363484 shutdown_connections 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.649+0000 7fde27bba640 1 -- 172.21.15.46:0/3822363484 wait complete. 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.649+0000 7fde27bba640 1 Processor -- start 2024-09-10T04:16:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.649+0000 7fde27bba640 1 -- start start 2024-09-10T04:16:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.649+0000 7fde27bba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20100800 0x7fde200fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:56.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.649+0000 7fde27bba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde20106800 0x7fde200fd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde27bba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010dcd0 con 0x7fde20100800 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde27bba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fde2010db50 con 0x7fde20106800 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25956640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20100800 0x7fde200fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25956640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20100800 0x7fde200fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33444/0 (socket says 172.21.15.46:33444) 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25956640 1 -- 172.21.15.46:0/1304668845 learned_addr learned my addr 172.21.15.46:0/1304668845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25155640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde20106800 0x7fde200fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25956640 1 -- 172.21.15.46:0/1304668845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde20106800 msgr2=0x7fde200fd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25956640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde20106800 0x7fde200fd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25956640 1 -- 172.21.15.46:0/1304668845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde200fd680 con 0x7fde20100800 2024-09-10T04:16:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde25956640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20100800 0x7fde200fcc00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006010 tx=0x7fde1c02fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde16ffd640 1 -- 172.21.15.46:0/1304668845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fde1c047070 con 0x7fde20100800 2024-09-10T04:16:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde16ffd640 1 -- 172.21.15.46:0/1304668845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde1c003e30 con 0x7fde20100800 2024-09-10T04:16:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde200fd910 con 0x7fde20100800 2024-09-10T04:16:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.650+0000 7fde16ffd640 1 -- 172.21.15.46:0/1304668845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fde1c034070 con 0x7fde20100800 2024-09-10T04:16:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.651+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde201a3da0 con 0x7fde20100800 2024-09-10T04:16:56.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.652+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde20101f40 con 0x7fde20100800 2024-09-10T04:16:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.652+0000 7fde16ffd640 1 -- 172.21.15.46:0/1304668845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fde1c04a070 con 0x7fde20100800 2024-09-10T04:16:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.653+0000 7fde16ffd640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde0c07ac50 0x7fde0c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.653+0000 7fde16ffd640 1 -- 172.21.15.46:0/1304668845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fde1c0be8b0 con 0x7fde20100800 2024-09-10T04:16:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.653+0000 7fde25155640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde0c07ac50 0x7fde0c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:56.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.653+0000 7fde25155640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde0c07ac50 0x7fde0c07d110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde200fe690 tx=0x7fde10005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:56.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.658+0000 7fde16ffd640 1 -- 172.21.15.46:0/1304668845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fde1c03f070 con 0x7fde20100800 2024-09-10T04:16:56.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.861+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fde20062630 con 0x7fde0c07ac50 2024-09-10T04:16:56.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.862+0000 7fde16ffd640 1 -- 172.21.15.46:0/1304668845 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fde20062630 con 0x7fde0c07ac50 2024-09-10T04:16:56.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde0c07ac50 msgr2=0x7fde0c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:56.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde0c07ac50 0x7fde0c07d110 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fde200fe690 tx=0x7fde10005f90 comp rx=0 tx=0).stop 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20100800 msgr2=0x7fde200fcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20100800 0x7fde200fcc00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006010 tx=0x7fde1c02fd60 comp rx=0 tx=0).stop 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 shutdown_connections 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde0c07ac50 0x7fde0c07d110 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde20106800 0x7fde200fd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 --2- 172.21.15.46:0/1304668845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde20100800 0x7fde200fcc00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.864+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 >> 172.21.15.46:0/1304668845 conn(0x7fde200fc520 msgr2=0x7fde2006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.865+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 shutdown_connections 2024-09-10T04:16:56.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.865+0000 7fde27bba640 1 -- 172.21.15.46:0/1304668845 wait complete. 2024-09-10T04:16:56.874 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:16:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3210222983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 msgr2=0x7f42ac102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3210222983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac102e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009880 tx=0x7f42a802a530 comp rx=0 tx=0).stop 2024-09-10T04:16:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3210222983 shutdown_connections 2024-09-10T04:16:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3210222983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac1033d0 0x7f42ac10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3210222983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac102e90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3210222983 >> 172.21.15.46:0/3210222983 conn(0x7f42ac0fc550 msgr2=0x7f42ac0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:56.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3210222983 shutdown_connections 2024-09-10T04:16:56.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.951+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3210222983 wait complete. 2024-09-10T04:16:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b2cfb640 1 Processor -- start 2024-09-10T04:16:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b2cfb640 1 -- start start 2024-09-10T04:16:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b2cfb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b2cfb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac1033d0 0x7f42ac19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:56.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b2cfb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac10ede0 con 0x7f42ac102ab0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b2cfb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac10ec60 con 0x7f42ac1033d0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b0a97640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b0a97640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33468/0 (socket says 172.21.15.46:33468) 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b0a97640 1 -- 172.21.15.46:0/3729655599 learned_addr learned my addr 172.21.15.46:0/3729655599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42a3fff640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac1033d0 0x7f42ac19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b0a97640 1 -- 172.21.15.46:0/3729655599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac1033d0 msgr2=0x7f42ac19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.952+0000 7f42b0a97640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac1033d0 0x7f42ac19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.953+0000 7f42b0a97640 1 -- 172.21.15.46:0/3729655599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac1977b0 con 0x7f42ac102ab0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.953+0000 7f42b0a97640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac19d630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f42a8031460 tx=0x7f42a80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.953+0000 7f42a1ffb640 1 -- 172.21.15.46:0/3729655599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42a803f070 con 0x7f42ac102ab0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.953+0000 7f42a1ffb640 1 -- 172.21.15.46:0/3729655599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42a8004cf0 con 0x7f42ac102ab0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.953+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac197a40 con 0x7f42ac102ab0 2024-09-10T04:16:56.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.953+0000 7f42a1ffb640 1 -- 172.21.15.46:0/3729655599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42a8035030 con 0x7f42ac102ab0 2024-09-10T04:16:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.953+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac197f70 con 0x7f42ac102ab0 2024-09-10T04:16:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.954+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ac06a580 con 0x7f42ac102ab0 2024-09-10T04:16:56.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.955+0000 7f42a1ffb640 1 -- 172.21.15.46:0/3729655599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f42a8042070 con 0x7f42ac102ab0 2024-09-10T04:16:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.956+0000 7f42a1ffb640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f429407abf0 0x7f429407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.956+0000 7f42a1ffb640 1 -- 172.21.15.46:0/3729655599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f42a80be6a0 con 0x7f42ac102ab0 2024-09-10T04:16:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.956+0000 7f42a3fff640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f429407abf0 0x7f429407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.956+0000 7f42a3fff640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f429407abf0 0x7f429407d0b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42ac1088f0 tx=0x7f429c005f50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:56.960+0000 7f42a1ffb640 1 -- 172.21.15.46:0/3729655599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f42a8089de0 con 0x7f42ac102ab0 2024-09-10T04:16:57.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.162+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f42ac1982e0 con 0x7f429407abf0 2024-09-10T04:16:57.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.163+0000 7f42a1ffb640 1 -- 172.21.15.46:0/3729655599 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f42ac1982e0 con 0x7f429407abf0 2024-09-10T04:16:57.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f429407abf0 msgr2=0x7f429407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f429407abf0 0x7f429407d0b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42ac1088f0 tx=0x7f429c005f50 comp rx=0 tx=0).stop 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 msgr2=0x7f42ac19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac19d630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f42a8031460 tx=0x7f42a80048f0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 shutdown_connections 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f429407abf0 0x7f429407d0b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42ac1033d0 0x7f42ac19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 --2- 172.21.15.46:0/3729655599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42ac102ab0 0x7f42ac19d630 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 >> 172.21.15.46:0/3729655599 conn(0x7f42ac0fc550 msgr2=0x7f42ac10aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 shutdown_connections 2024-09-10T04:16:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.166+0000 7f42b2cfb640 1 -- 172.21.15.46:0/3729655599 wait complete. 2024-09-10T04:16:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 -- 172.21.15.46:0/2614095226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 msgr2=0x7f28cc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 --2- 172.21.15.46:0/2614095226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc10b8d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009880 tx=0x7f28bc02a540 comp rx=0 tx=0).stop 2024-09-10T04:16:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 -- 172.21.15.46:0/2614095226 shutdown_connections 2024-09-10T04:16:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 --2- 172.21.15.46:0/2614095226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc10b8d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 --2- 172.21.15.46:0/2614095226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28cc102ac0 0x7f28cc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 -- 172.21.15.46:0/2614095226 >> 172.21.15.46:0/2614095226 conn(0x7f28cc0fc520 msgr2=0x7f28cc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 -- 172.21.15.46:0/2614095226 shutdown_connections 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.251+0000 7f28d45c7640 1 -- 172.21.15.46:0/2614095226 wait complete. 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d45c7640 1 Processor -- start 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d45c7640 1 -- start start 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d45c7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28cc102ac0 0x7f28cc19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d45c7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d45c7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc10edd0 con 0x7f28cc1033e0 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d45c7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc10ec50 con 0x7f28cc102ac0 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d1b62640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d1b62640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33488/0 (socket says 172.21.15.46:33488) 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.252+0000 7f28d1b62640 1 -- 172.21.15.46:0/3914488353 learned_addr learned my addr 172.21.15.46:0/3914488353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:57.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28d2363640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28cc102ac0 0x7f28cc19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28d1b62640 1 -- 172.21.15.46:0/3914488353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28cc102ac0 msgr2=0x7f28cc19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28d1b62640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28cc102ac0 0x7f28cc19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28d1b62640 1 -- 172.21.15.46:0/3914488353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc1977e0 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28d1b62640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc19dba0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28bc031460 tx=0x7f28bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28c37fe640 1 -- 172.21.15.46:0/3914488353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f28bc03f070 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28c37fe640 1 -- 172.21.15.46:0/3914488353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f28bc004ac0 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc197a70 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28c37fe640 1 -- 172.21.15.46:0/3914488353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f28bc035030 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.253+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc197fa0 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.254+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc06aa90 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.255+0000 7f28c37fe640 1 -- 172.21.15.46:0/3914488353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f28bc042070 con 0x7f28cc1033e0 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.255+0000 7f28c37fe640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f28b807ac40 0x7f28b807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.255+0000 7f28d2363640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f28b807ac40 0x7f28b807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.255+0000 7f28c37fe640 1 -- 172.21.15.46:0/3914488353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f28bc0be780 con 0x7f28cc1033e0 2024-09-10T04:16:57.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.256+0000 7f28d2363640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f28b807ac40 0x7f28b807d100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005e10 tx=0x7f28c8005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:57.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.260+0000 7f28c37fe640 1 -- 172.21.15.46:0/3914488353 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f28bc089e20 con 0x7f28cc1033e0 2024-09-10T04:16:57.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.452+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f28cc198d10 con 0x7f28b807ac40 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.462+0000 7f28c37fe640 1 -- 172.21.15.46:0/3914488353 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f28cc198d10 con 0x7f28b807ac40 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (8m) 3m ago 19m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (11m) 3m ago 20m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (10m) 2m ago 19m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (6m) 3m ago 20m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (6m) 2m ago 19m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (8m) 3m ago 19m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:16:57.464 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (3m) 3m ago 14m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (3m) 3m ago 14m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (2m) 2m ago 14m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (14m) 2m ago 14m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (10m) 3m ago 21m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (9m) 2m ago 19m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (7m) 3m ago 21m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (7m) 2m ago 18m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (9m) 3m ago 19m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (9m) 2m ago 19m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (6m) 3m ago 17m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (5m) 3m ago 17m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (5m) 3m ago 16m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (4m) 2m ago 16m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:16:57.465 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (4m) 2m ago 15m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:16:57.466 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (3m) 2m ago 15m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:16:57.466 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (8m) 3m ago 19m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:16:57.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f28b807ac40 msgr2=0x7f28b807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f28b807ac40 0x7f28b807d100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005e10 tx=0x7f28c8005da0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 msgr2=0x7f28cc19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc19dba0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28bc031460 tx=0x7f28bc0046c0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 shutdown_connections 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f28b807ac40 0x7f28b807d100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28cc1033e0 0x7f28cc19dba0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 --2- 172.21.15.46:0/3914488353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28cc102ac0 0x7f28cc19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 >> 172.21.15.46:0/3914488353 conn(0x7f28cc0fc520 msgr2=0x7f28cc108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 shutdown_connections 2024-09-10T04:16:57.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.464+0000 7f28d45c7640 1 -- 172.21.15.46:0/3914488353 wait complete. 2024-09-10T04:16:57.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 -- 172.21.15.46:0/2838260963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 msgr2=0x7fb258107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/2838260963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb258107ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb248009880 tx=0x7fb24802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.549 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:57 smithi046 ceph-mon[109506]: pgmap v338: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:57.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 -- 172.21.15.46:0/2838260963 shutdown_connections 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/2838260963 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb258101900 0x7fb258101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/2838260963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb258107ce0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 -- 172.21.15.46:0/2838260963 >> 172.21.15.46:0/2838260963 conn(0x7fb2580fd600 msgr2=0x7fb2580ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 -- 172.21.15.46:0/2838260963 shutdown_connections 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.547+0000 7fb25ccb7640 1 -- 172.21.15.46:0/2838260963 wait complete. 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.548+0000 7fb25ccb7640 1 Processor -- start 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.548+0000 7fb25ccb7640 1 -- start start 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.548+0000 7fb25ccb7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb258101900 0x7fb25819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb25ccb7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb25819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb25ccb7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb25810edd0 con 0x7fb258107900 2024-09-10T04:16:57.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb25ccb7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb25810ec50 con 0x7fb258101900 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb255d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb25819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb255d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb25819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33512/0 (socket says 172.21.15.46:33512) 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb255d9b640 1 -- 172.21.15.46:0/1906038519 learned_addr learned my addr 172.21.15.46:0/1906038519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb25659c640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb258101900 0x7fb25819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb255d9b640 1 -- 172.21.15.46:0/1906038519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb258101900 msgr2=0x7fb25819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb255d9b640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb258101900 0x7fb25819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb255d9b640 1 -- 172.21.15.46:0/1906038519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb258199a30 con 0x7fb258107900 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb255d9b640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb25819fdb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d6f0 tx=0x7fb24c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:57.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.549+0000 7fb2477fe640 1 -- 172.21.15.46:0/1906038519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb24c017070 con 0x7fb258107900 2024-09-10T04:16:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.550+0000 7fb2477fe640 1 -- 172.21.15.46:0/1906038519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb24c00de00 con 0x7fb258107900 2024-09-10T04:16:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.550+0000 7fb2477fe640 1 -- 172.21.15.46:0/1906038519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb24c004bb0 con 0x7fb258107900 2024-09-10T04:16:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.550+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb258199d20 con 0x7fb258107900 2024-09-10T04:16:57.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.550+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25819a2b0 con 0x7fb258107900 2024-09-10T04:16:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.551+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb258103040 con 0x7fb258107900 2024-09-10T04:16:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.551+0000 7fb2477fe640 1 -- 172.21.15.46:0/1906038519 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb24c009a30 con 0x7fb258107900 2024-09-10T04:16:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.552+0000 7fb2477fe640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb24007ac40 0x7fb24007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.552+0000 7fb2477fe640 1 -- 172.21.15.46:0/1906038519 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fb24c09d740 con 0x7fb258107900 2024-09-10T04:16:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.552+0000 7fb25659c640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb24007ac40 0x7fb24007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.553+0000 7fb25659c640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb24007ac40 0x7fb24007d100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb248000c00 tx=0x7fb248031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:57.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.557+0000 7fb2477fe640 1 -- 172.21.15.46:0/1906038519 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb24c068f00 con 0x7fb258107900 2024-09-10T04:16:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:57 smithi070 ceph-mon[79774]: pgmap v338: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:57.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.813+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fb25819af60 con 0x7fb258107900 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.814+0000 7fb2477fe640 1 -- 172.21.15.46:0/1906038519 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fb24c06ddb0 con 0x7fb258107900 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:16:57.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:16:57.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:16:57.818 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:16:57.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb24007ac40 msgr2=0x7fb24007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb24007ac40 0x7fb24007d100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb248000c00 tx=0x7fb248031400 comp rx=0 tx=0).stop 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 msgr2=0x7fb25819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb25819fdb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d6f0 tx=0x7fb24c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 shutdown_connections 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb24007ac40 0x7fb24007d100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb258107900 0x7fb25819fdb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 --2- 172.21.15.46:0/1906038519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb258101900 0x7fb25819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 >> 172.21.15.46:0/1906038519 conn(0x7fb2580fd600 msgr2=0x7fb258074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 shutdown_connections 2024-09-10T04:16:57.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.816+0000 7fb25ccb7640 1 -- 172.21.15.46:0/1906038519 wait complete. 2024-09-10T04:16:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 -- 172.21.15.46:0/650891805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4107900 msgr2=0x7faec4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 --2- 172.21.15.46:0/650891805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4107900 0x7faec4107ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009880 tx=0x7faeb402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 -- 172.21.15.46:0/650891805 shutdown_connections 2024-09-10T04:16:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 --2- 172.21.15.46:0/650891805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec4101900 0x7faec4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 --2- 172.21.15.46:0/650891805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4107900 0x7faec4107ce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 -- 172.21.15.46:0/650891805 >> 172.21.15.46:0/650891805 conn(0x7faec40fd600 msgr2=0x7faec40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 -- 172.21.15.46:0/650891805 shutdown_connections 2024-09-10T04:16:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.901+0000 7faeca6ae640 1 -- 172.21.15.46:0/650891805 wait complete. 2024-09-10T04:16:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faeca6ae640 1 Processor -- start 2024-09-10T04:16:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faeca6ae640 1 -- start start 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faeca6ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4101900 0x7faec419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faeca6ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec4107900 0x7faec419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faeca6ae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faec410ee30 con 0x7faec4101900 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faeca6ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7faec410ecb0 con 0x7faec4107900 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faec3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4101900 0x7faec419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faec3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4101900 0x7faec419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33528/0 (socket says 172.21.15.46:33528) 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faec3fff640 1 -- 172.21.15.46:0/3264005699 learned_addr learned my addr 172.21.15.46:0/3264005699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.902+0000 7faec37fe640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec4107900 0x7faec419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faec3fff640 1 -- 172.21.15.46:0/3264005699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec4107900 msgr2=0x7faec419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faec3fff640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec4107900 0x7faec419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faec3fff640 1 -- 172.21.15.46:0/3264005699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec4199a10 con 0x7faec4101900 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faec3fff640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4101900 0x7faec419f850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faeb4031460 tx=0x7faeb40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faec17fa640 1 -- 172.21.15.46:0/3264005699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faeb403f070 con 0x7faec4101900 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faec17fa640 1 -- 172.21.15.46:0/3264005699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faeb4004cf0 con 0x7faec4101900 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec4199ca0 con 0x7faec4101900 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faec17fa640 1 -- 172.21.15.46:0/3264005699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faeb4035030 con 0x7faec4101900 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.903+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec419a1d0 con 0x7faec4101900 2024-09-10T04:16:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.904+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec4103040 con 0x7faec4101900 2024-09-10T04:16:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.905+0000 7faec17fa640 1 -- 172.21.15.46:0/3264005699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7faeb4042070 con 0x7faec4101900 2024-09-10T04:16:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.905+0000 7faec17fa640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faeac07abf0 0x7faeac07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.906+0000 7faec17fa640 1 -- 172.21.15.46:0/3264005699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7faeb40be850 con 0x7faec4101900 2024-09-10T04:16:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.906+0000 7faec37fe640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faeac07abf0 0x7faeac07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.906+0000 7faec37fe640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faeac07abf0 0x7faeac07d0b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faec40ffa40 tx=0x7faeb8008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:57.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:57.910+0000 7faec17fa640 1 -- 172.21.15.46:0/3264005699 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7faeb4089ef0 con 0x7faec4101900 2024-09-10T04:16:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.136+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7faec419b010 con 0x7faec4101900 2024-09-10T04:16:58.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.136+0000 7faec17fa640 1 -- 172.21.15.46:0/3264005699 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7faeb408eda0 con 0x7faec4101900 2024-09-10T04:16:58.138 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:16:58.139 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:16:58.140 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:58.141 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:16:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:16:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faeac07abf0 msgr2=0x7faeac07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faeac07abf0 0x7faeac07d0b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faec40ffa40 tx=0x7faeb8008040 comp rx=0 tx=0).stop 2024-09-10T04:16:58.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4101900 msgr2=0x7faec419f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4101900 0x7faec419f850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faeb4031460 tx=0x7faeb40048f0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 shutdown_connections 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faeac07abf0 0x7faeac07d0b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faec4107900 0x7faec419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 --2- 172.21.15.46:0/3264005699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec4101900 0x7faec419f850 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 >> 172.21.15.46:0/3264005699 conn(0x7faec40fd600 msgr2=0x7faec4074450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 shutdown_connections 2024-09-10T04:16:58.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.139+0000 7faeca6ae640 1 -- 172.21.15.46:0/3264005699 wait complete. 2024-09-10T04:16:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 -- 172.21.15.46:0/415138883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 msgr2=0x7f61a010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 --2- 172.21.15.46:0/415138883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a010b930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f618c0098b0 tx=0x7f618c02a560 comp rx=0 tx=0).stop 2024-09-10T04:16:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 -- 172.21.15.46:0/415138883 shutdown_connections 2024-09-10T04:16:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 --2- 172.21.15.46:0/415138883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a010b930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 --2- 172.21.15.46:0/415138883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a0102b20 0x7f61a0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 -- 172.21.15.46:0/415138883 >> 172.21.15.46:0/415138883 conn(0x7f61a00fc520 msgr2=0x7f61a00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 -- 172.21.15.46:0/415138883 shutdown_connections 2024-09-10T04:16:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.224+0000 7f61a7015640 1 -- 172.21.15.46:0/415138883 wait complete. 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a7015640 1 Processor -- start 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a7015640 1 -- start start 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a7015640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a0102b20 0x7f61a0100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a7015640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a0100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a7015640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f61a010edf0 con 0x7f61a0103440 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a7015640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f61a010ec70 con 0x7f61a0102b20 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f619ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a0100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f619ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a0100cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33556/0 (socket says 172.21.15.46:33556) 2024-09-10T04:16:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f619ffff640 1 -- 172.21.15.46:0/4104419511 learned_addr learned my addr 172.21.15.46:0/4104419511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a4db1640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a0102b20 0x7f61a0100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f619ffff640 1 -- 172.21.15.46:0/4104419511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a0102b20 msgr2=0x7f61a0100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f619ffff640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a0102b20 0x7f61a0100780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f619ffff640 1 -- 172.21.15.46:0/4104419511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a0101200 con 0x7f61a0103440 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.225+0000 7f61a4db1640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a0102b20 0x7f61a0100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.226+0000 7f619ffff640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a0100cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f618c031460 tx=0x7f618c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.226+0000 7f619dffb640 1 -- 172.21.15.46:0/4104419511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f618c03f070 con 0x7f61a0103440 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.226+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a0101490 con 0x7f61a0103440 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.226+0000 7f619dffb640 1 -- 172.21.15.46:0/4104419511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f618c0035f0 con 0x7f61a0103440 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.226+0000 7f619dffb640 1 -- 172.21.15.46:0/4104419511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f618c035030 con 0x7f61a0103440 2024-09-10T04:16:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.226+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a01a3e60 con 0x7f61a0103440 2024-09-10T04:16:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.227+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a006aa30 con 0x7f61a0103440 2024-09-10T04:16:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.228+0000 7f619dffb640 1 -- 172.21.15.46:0/4104419511 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f618c042070 con 0x7f61a0103440 2024-09-10T04:16:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.228+0000 7f619dffb640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f619007abf0 0x7f619007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.228+0000 7f619dffb640 1 -- 172.21.15.46:0/4104419511 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f618c0be6a0 con 0x7f61a0103440 2024-09-10T04:16:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.228+0000 7f61a4db1640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f619007abf0 0x7f619007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.229+0000 7f61a4db1640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f619007abf0 0x7f619007d0b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61940058c0 tx=0x7f6194005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:58.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.235+0000 7f619dffb640 1 -- 172.21.15.46:0/4104419511 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f618c089d40 con 0x7f61a0103440 2024-09-10T04:16:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.439+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f61a0062630 con 0x7f619007abf0 2024-09-10T04:16:58.441 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:58 smithi046 ceph-mon[109506]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:58.441 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:58 smithi046 ceph-mon[109506]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:58.441 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:58 smithi046 ceph-mon[109506]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:58.441 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:58 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1906038519' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:16:58.442 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:58 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3264005699' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:16:58.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.440+0000 7f619dffb640 1 -- 172.21.15.46:0/4104419511 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f61a0062630 con 0x7f619007abf0 2024-09-10T04:16:58.442 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:16:58.442 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:16:58.443 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:16:58.444 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:16:58.444 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:16:58.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f619007abf0 msgr2=0x7f619007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f619007abf0 0x7f619007d0b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61940058c0 tx=0x7f6194005850 comp rx=0 tx=0).stop 2024-09-10T04:16:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 msgr2=0x7f61a0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a0100cc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f618c031460 tx=0x7f618c0045c0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 shutdown_connections 2024-09-10T04:16:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f619007abf0 0x7f619007d0b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61a0103440 0x7f61a0100cc0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 --2- 172.21.15.46:0/4104419511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a0102b20 0x7f61a0100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.443+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 >> 172.21.15.46:0/4104419511 conn(0x7f61a00fc520 msgr2=0x7f61a0103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:58.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.444+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 shutdown_connections 2024-09-10T04:16:58.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.444+0000 7f61a7015640 1 -- 172.21.15.46:0/4104419511 wait complete. 2024-09-10T04:16:58.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.528+0000 7f690077d640 1 -- 172.21.15.46:0/3123340052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81007f0 msgr2=0x7f68f8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.528+0000 7f690077d640 1 --2- 172.21.15.46:0/3123340052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81007f0 0x7f68f8100c50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009880 tx=0x7f68ec02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 -- 172.21.15.46:0/3123340052 shutdown_connections 2024-09-10T04:16:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 --2- 172.21.15.46:0/3123340052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81007f0 0x7f68f8100c50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 --2- 172.21.15.46:0/3123340052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68f81067f0 0x7f68f8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 -- 172.21.15.46:0/3123340052 >> 172.21.15.46:0/3123340052 conn(0x7f68f80fc550 msgr2=0x7f68f80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 -- 172.21.15.46:0/3123340052 shutdown_connections 2024-09-10T04:16:58.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 -- 172.21.15.46:0/3123340052 wait complete. 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 Processor -- start 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.529+0000 7f690077d640 1 -- start start 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f690077d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68f81007f0 0x7f68f80fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f690077d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81067f0 0x7f68f80fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f690077d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68f810dcc0 con 0x7f68f81067f0 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f690077d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f68f810db40 con 0x7f68f81007f0 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fdd18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81067f0 0x7f68f80fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:58.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fdd18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81067f0 0x7f68f80fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33580/0 (socket says 172.21.15.46:33580) 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fdd18640 1 -- 172.21.15.46:0/2253345727 learned_addr learned my addr 172.21.15.46:0/2253345727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fe519640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68f81007f0 0x7f68f80fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fdd18640 1 -- 172.21.15.46:0/2253345727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68f81007f0 msgr2=0x7f68f80fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fdd18640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68f81007f0 0x7f68f80fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fdd18640 1 -- 172.21.15.46:0/2253345727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f80fd750 con 0x7f68f81067f0 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fe519640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68f81007f0 0x7f68f80fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68fdd18640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81067f0 0x7f68f80fd210 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68ec031460 tx=0x7f68ec004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.530+0000 7f68f77fe640 1 -- 172.21.15.46:0/2253345727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f68ec03f070 con 0x7f68f81067f0 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.531+0000 7f68f77fe640 1 -- 172.21.15.46:0/2253345727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68ec004a90 con 0x7f68f81067f0 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.531+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f80fd9e0 con 0x7f68f81067f0 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.531+0000 7f68f77fe640 1 -- 172.21.15.46:0/2253345727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f68ec035030 con 0x7f68f81067f0 2024-09-10T04:16:58.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.531+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f81a3da0 con 0x7f68f81067f0 2024-09-10T04:16:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.532+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f8101f30 con 0x7f68f81067f0 2024-09-10T04:16:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.532+0000 7f68f77fe640 1 -- 172.21.15.46:0/2253345727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f68ec042070 con 0x7f68f81067f0 2024-09-10T04:16:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.533+0000 7f68f77fe640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68e007ac50 0x7f68e007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:16:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.533+0000 7f68fe519640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68e007ac50 0x7f68e007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:16:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.533+0000 7f68f77fe640 1 -- 172.21.15.46:0/2253345727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f68ec0be6d0 con 0x7f68f81067f0 2024-09-10T04:16:58.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.533+0000 7f68fe519640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68e007ac50 0x7f68e007d110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009b50 tx=0x7f68e8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:16:58.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.537+0000 7f68f77fe640 1 -- 172.21.15.46:0/2253345727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f68ec089e90 con 0x7f68f81067f0 2024-09-10T04:16:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:58 smithi070 ceph-mon[79774]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:58 smithi070 ceph-mon[79774]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:58 smithi070 ceph-mon[79774]: from='client.34514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:58 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1906038519' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:16:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:58 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3264005699' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:16:58.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.793+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f68f8109d70 con 0x7f68f81067f0 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.794+0000 7f68f77fe640 1 -- 172.21.15.46:0/2253345727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f68ec08ed40 con 0x7f68f81067f0 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:16:58.796 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68e007ac50 msgr2=0x7f68e007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68e007ac50 0x7f68e007d110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009b50 tx=0x7f68e8005f90 comp rx=0 tx=0).stop 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81067f0 msgr2=0x7f68f80fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81067f0 0x7f68f80fd210 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f68ec031460 tx=0x7f68ec004530 comp rx=0 tx=0).stop 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 shutdown_connections 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68e007ac50 0x7f68e007d110 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68f81067f0 0x7f68f80fd210 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 --2- 172.21.15.46:0/2253345727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68f81007f0 0x7f68f80fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 >> 172.21.15.46:0/2253345727 conn(0x7f68f80fc550 msgr2=0x7f68f8069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 shutdown_connections 2024-09-10T04:16:58.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:16:58.796+0000 7f690077d640 1 -- 172.21.15.46:0/2253345727 wait complete. 2024-09-10T04:16:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:59 smithi070 ceph-mon[79774]: pgmap v339: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:59 smithi070 ceph-mon[79774]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:16:59 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2253345727' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:16:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:59 smithi046 ceph-mon[109506]: pgmap v339: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:16:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:59 smithi046 ceph-mon[109506]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:16:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:16:59 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2253345727' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:17:01.533 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:01 smithi070 ceph-mon[79774]: pgmap v340: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:01 smithi046 ceph-mon[109506]: pgmap v340: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:03 smithi070 ceph-mon[79774]: pgmap v341: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:03 smithi046 ceph-mon[109506]: pgmap v341: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:05.285 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:05 smithi046 ceph-mon[109506]: pgmap v342: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:05.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:05 smithi070 ceph-mon[79774]: pgmap v342: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:07.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:07 smithi070 ceph-mon[79774]: pgmap v343: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:07.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:07 smithi046 ceph-mon[109506]: pgmap v343: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:08.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:09.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:09 smithi070 ceph-mon[79774]: pgmap v344: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:09.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:09 smithi046 ceph-mon[109506]: pgmap v344: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:11 smithi070 ceph-mon[79774]: pgmap v345: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:11.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:11 smithi046 ceph-mon[109506]: pgmap v345: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:13 smithi070 ceph-mon[79774]: pgmap v346: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:13.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:13 smithi046 ceph-mon[109506]: pgmap v346: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:15.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:15 smithi070 ceph-mon[79774]: pgmap v347: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:15.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:15 smithi046 ceph-mon[109506]: pgmap v347: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:17.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:17 smithi070 ceph-mon[79774]: pgmap v348: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:17 smithi046 ceph-mon[109506]: pgmap v348: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:19.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:19 smithi070 ceph-mon[79774]: pgmap v349: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:19.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:19 smithi046 ceph-mon[109506]: pgmap v349: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:21 smithi070 ceph-mon[79774]: pgmap v350: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:21.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:21 smithi046 ceph-mon[109506]: pgmap v350: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:23 smithi070 ceph-mon[79774]: pgmap v351: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:23 smithi046 ceph-mon[109506]: pgmap v351: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:25.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:25 smithi070 ceph-mon[79774]: pgmap v352: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:25.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:25 smithi046 ceph-mon[109506]: pgmap v352: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:27 smithi070 ceph-mon[79774]: pgmap v353: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:27.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:27 smithi046 ceph-mon[109506]: pgmap v353: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 -- 172.21.15.46:0/1924252298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100250 msgr2=0x7efca0100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 --2- 172.21.15.46:0/1924252298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100250 0x7efca0100630 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc9000a990 tx=0x7efc9002b7d0 comp rx=0 tx=0).stop 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 -- 172.21.15.46:0/1924252298 shutdown_connections 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 --2- 172.21.15.46:0/1924252298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efca0100c00 0x7efca0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 --2- 172.21.15.46:0/1924252298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100250 0x7efca0100630 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 -- 172.21.15.46:0/1924252298 >> 172.21.15.46:0/1924252298 conn(0x7efca00fbc90 msgr2=0x7efca00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 -- 172.21.15.46:0/1924252298 shutdown_connections 2024-09-10T04:17:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.894+0000 7efca4ad2640 1 -- 172.21.15.46:0/1924252298 wait complete. 2024-09-10T04:17:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efca4ad2640 1 Processor -- start 2024-09-10T04:17:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efca4ad2640 1 -- start start 2024-09-10T04:17:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efca4ad2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efca0100250 0x7efca019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efca4ad2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100c00 0x7efca019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efca4ad2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efca010cbd0 con 0x7efca0100c00 2024-09-10T04:17:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efca4ad2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efca010ca50 con 0x7efca0100250 2024-09-10T04:17:28.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efc97fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100c00 0x7efca019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efc97fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100c00 0x7efca019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49008/0 (socket says 172.21.15.46:49008) 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efc97fff640 1 -- 172.21.15.46:0/651040913 learned_addr learned my addr 172.21.15.46:0/651040913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.895+0000 7efc9e59c640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efca0100250 0x7efca019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efc97fff640 1 -- 172.21.15.46:0/651040913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efca0100250 msgr2=0x7efca019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efc97fff640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efca0100250 0x7efca019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efc97fff640 1 -- 172.21.15.46:0/651040913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca0197810 con 0x7efca0100c00 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efc97fff640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100c00 0x7efca019db80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc8800d6f0 tx=0x7efc8800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efc977fe640 1 -- 172.21.15.46:0/651040913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efc88017070 con 0x7efca0100c00 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca0197b00 con 0x7efca0100c00 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efc977fe640 1 -- 172.21.15.46:0/651040913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc8800de00 con 0x7efca0100c00 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efc977fe640 1 -- 172.21.15.46:0/651040913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efc88004bb0 con 0x7efca0100c00 2024-09-10T04:17:28.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.896+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca0198090 con 0x7efca0100c00 2024-09-10T04:17:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.897+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca0101730 con 0x7efca0100c00 2024-09-10T04:17:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.898+0000 7efc977fe640 1 -- 172.21.15.46:0/651040913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efc88004e60 con 0x7efca0100c00 2024-09-10T04:17:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.899+0000 7efc977fe640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc8c07abf0 0x7efc8c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.899+0000 7efc977fe640 1 -- 172.21.15.46:0/651040913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7efc8809d820 con 0x7efca0100c00 2024-09-10T04:17:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.899+0000 7efc9e59c640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc8c07abf0 0x7efc8c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:28.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.899+0000 7efc9e59c640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc8c07abf0 0x7efc8c07d0b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc9000a340 tx=0x7efc90005fd0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:28.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:28.903+0000 7efc977fe640 1 -- 172.21.15.46:0/651040913 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efc88068f60 con 0x7efca0100c00 2024-09-10T04:17:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.107+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7efca0198340 con 0x7efc8c07abf0 2024-09-10T04:17:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.108+0000 7efc977fe640 1 -- 172.21.15.46:0/651040913 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7efca0198340 con 0x7efc8c07abf0 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc8c07abf0 msgr2=0x7efc8c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc8c07abf0 0x7efc8c07d0b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7efc9000a340 tx=0x7efc90005fd0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100c00 msgr2=0x7efca019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100c00 0x7efca019db80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc8800d6f0 tx=0x7efc8800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 shutdown_connections 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efc8c07abf0 0x7efc8c07d0b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efca0100c00 0x7efca019db80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 --2- 172.21.15.46:0/651040913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efca0100250 0x7efca019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 >> 172.21.15.46:0/651040913 conn(0x7efca00fbc90 msgr2=0x7efca00fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 shutdown_connections 2024-09-10T04:17:29.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.110+0000 7efca4ad2640 1 -- 172.21.15.46:0/651040913 wait complete. 2024-09-10T04:17:29.121 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 -- 172.21.15.46:0/627903903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 msgr2=0x7f66dc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/627903903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc101d60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009880 tx=0x7f66d002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 -- 172.21.15.46:0/627903903 shutdown_connections 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/627903903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc101d60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/627903903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66dc107900 0x7f66dc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 -- 172.21.15.46:0/627903903 >> 172.21.15.46:0/627903903 conn(0x7f66dc0fd600 msgr2=0x7f66dc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 -- 172.21.15.46:0/627903903 shutdown_connections 2024-09-10T04:17:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.196+0000 7f66e0ee3640 1 -- 172.21.15.46:0/627903903 wait complete. 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66e0ee3640 1 Processor -- start 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66e0ee3640 1 -- start start 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66e0ee3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66e0ee3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66dc107900 0x7f66dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66e0ee3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc10edd0 con 0x7f66dc101900 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66e0ee3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc10ec50 con 0x7f66dc107900 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66da59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66da59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49028/0 (socket says 172.21.15.46:49028) 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66da59c640 1 -- 172.21.15.46:0/2958761466 learned_addr learned my addr 172.21.15.46:0/2958761466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.197+0000 7f66d9d9b640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66dc107900 0x7f66dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66da59c640 1 -- 172.21.15.46:0/2958761466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66dc107900 msgr2=0x7f66dc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66da59c640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66dc107900 0x7f66dc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66da59c640 1 -- 172.21.15.46:0/2958761466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc074f70 con 0x7f66dc101900 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66da59c640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc0744f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e970 tx=0x7f66cc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66cb7fe640 1 -- 172.21.15.46:0/2958761466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66cc00cac0 con 0x7f66dc101900 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66cb7fe640 1 -- 172.21.15.46:0/2958761466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f66cc0054e0 con 0x7f66dc101900 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66cb7fe640 1 -- 172.21.15.46:0/2958761466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f66cc010430 con 0x7f66dc101900 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc075260 con 0x7f66dc101900 2024-09-10T04:17:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.198+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc1a3e20 con 0x7f66dc101900 2024-09-10T04:17:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.200+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66dc103040 con 0x7f66dc101900 2024-09-10T04:17:29.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.200+0000 7f66cb7fe640 1 -- 172.21.15.46:0/2958761466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f66cc004e80 con 0x7f66dc101900 2024-09-10T04:17:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.201+0000 7f66cb7fe640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66c407abf0 0x7f66c407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.201+0000 7f66cb7fe640 1 -- 172.21.15.46:0/2958761466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f66cc09e350 con 0x7f66dc101900 2024-09-10T04:17:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.201+0000 7f66d9d9b640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66c407abf0 0x7f66c407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.202+0000 7f66d9d9b640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66c407abf0 0x7f66c407d0b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66dc075f80 tx=0x7f66d0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:29.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.205+0000 7f66cb7fe640 1 -- 172.21.15.46:0/2958761466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f66cc0699f0 con 0x7f66dc101900 2024-09-10T04:17:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.413+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f66dc062630 con 0x7f66c407abf0 2024-09-10T04:17:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.415+0000 7f66cb7fe640 1 -- 172.21.15.46:0/2958761466 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f66dc062630 con 0x7f66c407abf0 2024-09-10T04:17:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66c407abf0 msgr2=0x7f66c407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66c407abf0 0x7f66c407d0b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f66dc075f80 tx=0x7f66d0031400 comp rx=0 tx=0).stop 2024-09-10T04:17:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 msgr2=0x7f66dc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc0744f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00e970 tx=0x7f66cc00ed70 comp rx=0 tx=0).stop 2024-09-10T04:17:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 shutdown_connections 2024-09-10T04:17:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f66c407abf0 0x7f66c407d0b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66dc107900 0x7f66dc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 --2- 172.21.15.46:0/2958761466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66dc101900 0x7f66dc0744f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 >> 172.21.15.46:0/2958761466 conn(0x7f66dc0fd600 msgr2=0x7f66dc0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 shutdown_connections 2024-09-10T04:17:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.417+0000 7f66e0ee3640 1 -- 172.21.15.46:0/2958761466 wait complete. 2024-09-10T04:17:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.501+0000 7f5e903b9640 1 -- 172.21.15.46:0/4065547511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 msgr2=0x7f5e88101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.501+0000 7f5e903b9640 1 --2- 172.21.15.46:0/4065547511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e88101d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009880 tx=0x7f5e7802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.504 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:29 smithi046 ceph-mon[109506]: pgmap v354: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:29.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 -- 172.21.15.46:0/4065547511 shutdown_connections 2024-09-10T04:17:29.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 --2- 172.21.15.46:0/4065547511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e88101d60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 --2- 172.21.15.46:0/4065547511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e88107900 0x7f5e88107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 -- 172.21.15.46:0/4065547511 >> 172.21.15.46:0/4065547511 conn(0x7f5e880fd600 msgr2=0x7f5e880ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 -- 172.21.15.46:0/4065547511 shutdown_connections 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 -- 172.21.15.46:0/4065547511 wait complete. 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 Processor -- start 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 -- start start 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.502+0000 7f5e903b9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e8819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e903b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e88107900 0x7f5e8819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e903b9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8810edb0 con 0x7f5e88101900 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e903b9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8810ec30 con 0x7f5e88107900 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8e155640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e8819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8e155640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e8819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49038/0 (socket says 172.21.15.46:49038) 2024-09-10T04:17:29.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8e155640 1 -- 172.21.15.46:0/3739311827 learned_addr learned my addr 172.21.15.46:0/3739311827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8d954640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e88107900 0x7f5e8819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8e155640 1 -- 172.21.15.46:0/3739311827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e88107900 msgr2=0x7f5e8819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8e155640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e88107900 0x7f5e8819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8e155640 1 -- 172.21.15.46:0/3739311827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e88197800 con 0x7f5e88101900 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e8e155640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e8819d680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400d8d0 tx=0x7f5e8400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e7f7fe640 1 -- 172.21.15.46:0/3739311827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e84017030 con 0x7f5e88101900 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e7f7fe640 1 -- 172.21.15.46:0/3739311827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5e84004020 con 0x7f5e88101900 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e7f7fe640 1 -- 172.21.15.46:0/3739311827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5e840047a0 con 0x7f5e88101900 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.503+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e88197af0 con 0x7f5e88101900 2024-09-10T04:17:29.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.504+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e88198050 con 0x7f5e88101900 2024-09-10T04:17:29.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.505+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e88103040 con 0x7f5e88101900 2024-09-10T04:17:29.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.505+0000 7f5e7f7fe640 1 -- 172.21.15.46:0/3739311827 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5e840041d0 con 0x7f5e88101900 2024-09-10T04:17:29.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.506+0000 7f5e7f7fe640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e7407ac40 0x7f5e7407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.506+0000 7f5e8d954640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e7407ac40 0x7f5e7407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.506+0000 7f5e7f7fe640 1 -- 172.21.15.46:0/3739311827 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f5e84018070 con 0x7f5e88101900 2024-09-10T04:17:29.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.507+0000 7f5e8d954640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e7407ac40 0x7f5e7407d100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5e88198b50 tx=0x7f5e78031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:29.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.511+0000 7f5e7f7fe640 1 -- 172.21.15.46:0/3739311827 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5e84069130 con 0x7f5e88101900 2024-09-10T04:17:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:29 smithi070 ceph-mon[79774]: pgmap v354: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.704+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5e88198360 con 0x7f5e7407ac40 2024-09-10T04:17:29.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.714+0000 7f5e7f7fe640 1 -- 172.21.15.46:0/3739311827 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f5e88198360 con 0x7f5e7407ac40 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (9m) 3m ago 20m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (11m) 3m ago 20m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (11m) 3m ago 19m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (7m) 3m ago 20m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (7m) 3m ago 19m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (8m) 3m ago 20m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (3m) 3m ago 14m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (3m) 3m ago 15m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:17:29.716 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (3m) 3m ago 15m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (14m) 3m ago 14m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (11m) 3m ago 21m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (10m) 3m ago 19m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (7m) 3m ago 21m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (7m) 3m ago 19m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (9m) 3m ago 20m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (9m) 3m ago 19m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (7m) 3m ago 18m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (6m) 3m ago 17m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:17:29.717 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (5m) 3m ago 17m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:17:29.718 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (5m) 3m ago 16m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:17:29.718 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (4m) 3m ago 16m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:17:29.718 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (4m) 3m ago 16m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:17:29.718 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (9m) 3m ago 20m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e7407ac40 msgr2=0x7f5e7407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e7407ac40 0x7f5e7407d100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5e88198b50 tx=0x7f5e78031400 comp rx=0 tx=0).stop 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 msgr2=0x7f5e8819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e8819d680 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e8400d8d0 tx=0x7f5e8400dd90 comp rx=0 tx=0).stop 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 shutdown_connections 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5e7407ac40 0x7f5e7407d100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e88107900 0x7f5e8819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 --2- 172.21.15.46:0/3739311827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e88101900 0x7f5e8819d680 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 >> 172.21.15.46:0/3739311827 conn(0x7f5e880fd600 msgr2=0x7f5e880fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:29.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 shutdown_connections 2024-09-10T04:17:29.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.716+0000 7f5e903b9640 1 -- 172.21.15.46:0/3739311827 wait complete. 2024-09-10T04:17:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.799+0000 7f5d8c428640 1 -- 172.21.15.46:0/3503369581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100c00 msgr2=0x7f5d84104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.799+0000 7f5d8c428640 1 --2- 172.21.15.46:0/3503369581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100c00 0x7f5d84104c50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400a990 tx=0x7f5d7402b7d0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 -- 172.21.15.46:0/3503369581 shutdown_connections 2024-09-10T04:17:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 --2- 172.21.15.46:0/3503369581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100c00 0x7f5d84104c50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 --2- 172.21.15.46:0/3503369581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d84100250 0x7f5d84100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 -- 172.21.15.46:0/3503369581 >> 172.21.15.46:0/3503369581 conn(0x7f5d840fbc90 msgr2=0x7f5d840fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 -- 172.21.15.46:0/3503369581 shutdown_connections 2024-09-10T04:17:29.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 -- 172.21.15.46:0/3503369581 wait complete. 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 Processor -- start 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.800+0000 7f5d8c428640 1 -- start start 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8c428640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100250 0x7f5d84101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8c428640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d84100c00 0x7f5d84102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8c428640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410cbd0 con 0x7f5d84100250 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8c428640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8410ca50 con 0x7f5d84100c00 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8a1c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100250 0x7f5d84101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8a1c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100250 0x7f5d84101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49056/0 (socket says 172.21.15.46:49056) 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8a1c4640 1 -- 172.21.15.46:0/2235661288 learned_addr learned my addr 172.21.15.46:0/2235661288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d899c3640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d84100c00 0x7f5d84102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8a1c4640 1 -- 172.21.15.46:0/2235661288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d84100c00 msgr2=0x7f5d84102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8a1c4640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d84100c00 0x7f5d84102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8a1c4640 1 -- 172.21.15.46:0/2235661288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d84102770 con 0x7f5d84100250 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d8a1c4640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100250 0x7f5d84101cf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000ea00 tx=0x7f5d8000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d7b7fe640 1 -- 172.21.15.46:0/2235661288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d8000cb80 con 0x7f5d84100250 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.801+0000 7f5d7b7fe640 1 -- 172.21.15.46:0/2235661288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d8000cd20 con 0x7f5d84100250 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.802+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d84102a60 con 0x7f5d84100250 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.802+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a3e50 con 0x7f5d84100250 2024-09-10T04:17:29.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.802+0000 7f5d7b7fe640 1 -- 172.21.15.46:0/2235661288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d80011470 con 0x7f5d84100250 2024-09-10T04:17:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.803+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d84101730 con 0x7f5d84100250 2024-09-10T04:17:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.804+0000 7f5d7b7fe640 1 -- 172.21.15.46:0/2235661288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5d80011770 con 0x7f5d84100250 2024-09-10T04:17:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.804+0000 7f5d7b7fe640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d7007abf0 0x7f5d7007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.804+0000 7f5d7b7fe640 1 -- 172.21.15.46:0/2235661288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f5d8009e4f0 con 0x7f5d84100250 2024-09-10T04:17:29.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.804+0000 7f5d899c3640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d7007abf0 0x7f5d7007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:29.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.805+0000 7f5d899c3640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d7007abf0 0x7f5d7007d0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400a340 tx=0x7f5d74005fd0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:29.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:29.808+0000 7f5d7b7fe640 1 -- 172.21.15.46:0/2235661288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5d80016030 con 0x7f5d84100250 2024-09-10T04:17:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.070+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f5d84100630 con 0x7f5d84100250 2024-09-10T04:17:30.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.071+0000 7f5d7b7fe640 1 -- 172.21.15.46:0/2235661288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f5d84100630 con 0x7f5d84100250 2024-09-10T04:17:30.072 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:17:30.073 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:17:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:17:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:17:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:17:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:17:30.074 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:17:30.074 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:17:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d7007abf0 msgr2=0x7f5d7007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d7007abf0 0x7f5d7007d0b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400a340 tx=0x7f5d74005fd0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100250 msgr2=0x7f5d84101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100250 0x7f5d84101cf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000ea00 tx=0x7f5d8000eec0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 shutdown_connections 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d7007abf0 0x7f5d7007d0b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d84100c00 0x7f5d84102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 --2- 172.21.15.46:0/2235661288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d84100250 0x7f5d84101cf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 >> 172.21.15.46:0/2235661288 conn(0x7f5d840fbc90 msgr2=0x7f5d840fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 shutdown_connections 2024-09-10T04:17:30.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.073+0000 7f5d8c428640 1 -- 172.21.15.46:0/2235661288 wait complete. 2024-09-10T04:17:30.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.157+0000 7f5638f01640 1 -- 172.21.15.46:0/1755485073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 msgr2=0x7f5634100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.157+0000 7f5638f01640 1 --2- 172.21.15.46:0/1755485073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f5634100630 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5624009880 tx=0x7f562402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 -- 172.21.15.46:0/1755485073 shutdown_connections 2024-09-10T04:17:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 --2- 172.21.15.46:0/1755485073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5634100c00 0x7f5634104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 --2- 172.21.15.46:0/1755485073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f5634100630 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 -- 172.21.15.46:0/1755485073 >> 172.21.15.46:0/1755485073 conn(0x7f56340fbc90 msgr2=0x7f56340fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 -- 172.21.15.46:0/1755485073 shutdown_connections 2024-09-10T04:17:30.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 -- 172.21.15.46:0/1755485073 wait complete. 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 Processor -- start 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.158+0000 7f5638f01640 1 -- start start 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f5638f01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f563419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f5638f01640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5634100c00 0x7f563419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f5638f01640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f563410ee90 con 0x7f5634100250 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f5638f01640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f563410ed10 con 0x7f5634100c00 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f563259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f563419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f563259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f563419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49082/0 (socket says 172.21.15.46:49082) 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f563259c640 1 -- 172.21.15.46:0/2580779057 learned_addr learned my addr 172.21.15.46:0/2580779057 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f5631d9b640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5634100c00 0x7f563419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f563259c640 1 -- 172.21.15.46:0/2580779057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5634100c00 msgr2=0x7f563419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f563259c640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5634100c00 0x7f563419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f563259c640 1 -- 172.21.15.46:0/2580779057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5634197860 con 0x7f5634100250 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f563259c640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f563419d6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5624031460 tx=0x7f5624004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.159+0000 7f56237fe640 1 -- 172.21.15.46:0/2580779057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f562403f070 con 0x7f5634100250 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.160+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5634197af0 con 0x7f5634100250 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.160+0000 7f56237fe640 1 -- 172.21.15.46:0/2580779057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f56240035f0 con 0x7f5634100250 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.160+0000 7f56237fe640 1 -- 172.21.15.46:0/2580779057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5624035030 con 0x7f5634100250 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.160+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5634198020 con 0x7f5634100250 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.161+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5634101730 con 0x7f5634100250 2024-09-10T04:17:30.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.161+0000 7f56237fe640 1 -- 172.21.15.46:0/2580779057 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5624042070 con 0x7f5634100250 2024-09-10T04:17:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.162+0000 7f56237fe640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f561c07abf0 0x7f561c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.162+0000 7f56237fe640 1 -- 172.21.15.46:0/2580779057 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f56240be5b0 con 0x7f5634100250 2024-09-10T04:17:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.162+0000 7f5631d9b640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f561c07abf0 0x7f561c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.162+0000 7f5631d9b640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f561c07abf0 0x7f561c07d0b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56340fe220 tx=0x7f5628005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:30.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.166+0000 7f56237fe640 1 -- 172.21.15.46:0/2580779057 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5624089cf0 con 0x7f5634100250 2024-09-10T04:17:30.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:30 smithi046 ceph-mon[109506]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:30.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:30 smithi046 ceph-mon[109506]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:30.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:30 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2235661288' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:17:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.394+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5634108560 con 0x7f5634100250 2024-09-10T04:17:30.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.395+0000 7f56237fe640 1 -- 172.21.15.46:0/2580779057 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f562408eba0 con 0x7f5634100250 2024-09-10T04:17:30.397 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:17:30.397 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:17:30.397 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:17:30.397 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:17:30.397 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:17:30.397 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:17:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:17:30.399 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:17:30.400 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:17:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:17:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f561c07abf0 msgr2=0x7f561c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f561c07abf0 0x7f561c07d0b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f56340fe220 tx=0x7f5628005f90 comp rx=0 tx=0).stop 2024-09-10T04:17:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 msgr2=0x7f563419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f563419d6a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5624031460 tx=0x7f5624004530 comp rx=0 tx=0).stop 2024-09-10T04:17:30.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 shutdown_connections 2024-09-10T04:17:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f561c07abf0 0x7f561c07d0b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5634100c00 0x7f563419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 --2- 172.21.15.46:0/2580779057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5634100250 0x7f563419d6a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 >> 172.21.15.46:0/2580779057 conn(0x7f56340fbc90 msgr2=0x7f56340fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 shutdown_connections 2024-09-10T04:17:30.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.397+0000 7f5638f01640 1 -- 172.21.15.46:0/2580779057 wait complete. 2024-09-10T04:17:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/3713578728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 msgr2=0x7fdd58100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/3713578728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd58100c50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009880 tx=0x7fdd4802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/3713578728 shutdown_connections 2024-09-10T04:17:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/3713578728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd58100c50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/3713578728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd581067f0 0x7fdd58106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/3713578728 >> 172.21.15.46:0/3713578728 conn(0x7fdd580fc550 msgr2=0x7fdd580fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/3713578728 shutdown_connections 2024-09-10T04:17:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.485+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/3713578728 wait complete. 2024-09-10T04:17:30.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5f4e9640 1 Processor -- start 2024-09-10T04:17:30.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5f4e9640 1 -- start start 2024-09-10T04:17:30.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5f4e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd5819c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5f4e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd581067f0 0x7fdd5819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5f4e9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810dcc0 con 0x7fdd581007f0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5f4e9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5810db40 con 0x7fdd581067f0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5d285640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd5819c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5d285640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd5819c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49096/0 (socket says 172.21.15.46:49096) 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.486+0000 7fdd5d285640 1 -- 172.21.15.46:0/617029543 learned_addr learned my addr 172.21.15.46:0/617029543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd5ca84640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd581067f0 0x7fdd5819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd5d285640 1 -- 172.21.15.46:0/617029543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd581067f0 msgr2=0x7fdd5819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd5d285640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd581067f0 0x7fdd5819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd5d285640 1 -- 172.21.15.46:0/617029543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581966b0 con 0x7fdd581007f0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd5d285640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd5819c5e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400d6f0 tx=0x7fdd5400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd4e7fc640 1 -- 172.21.15.46:0/617029543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdd54017070 con 0x7fdd581007f0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd581969a0 con 0x7fdd581007f0 2024-09-10T04:17:30.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd4e7fc640 1 -- 172.21.15.46:0/617029543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd5400de00 con 0x7fdd581007f0 2024-09-10T04:17:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd4e7fc640 1 -- 172.21.15.46:0/617029543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdd54004bb0 con 0x7fdd581007f0 2024-09-10T04:17:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.487+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd58196f00 con 0x7fdd581007f0 2024-09-10T04:17:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.488+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd58101f30 con 0x7fdd581007f0 2024-09-10T04:17:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.489+0000 7fdd4e7fc640 1 -- 172.21.15.46:0/617029543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdd54004e60 con 0x7fdd581007f0 2024-09-10T04:17:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.489+0000 7fdd4e7fc640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd4407abf0 0x7fdd4407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.490+0000 7fdd4e7fc640 1 -- 172.21.15.46:0/617029543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fdd5409d820 con 0x7fdd581007f0 2024-09-10T04:17:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.490+0000 7fdd5ca84640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd4407abf0 0x7fdd4407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.490+0000 7fdd5ca84640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd4407abf0 0x7fdd4407d0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd58197f00 tx=0x7fdd48031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:30.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.494+0000 7fdd4e7fc640 1 -- 172.21.15.46:0/617029543 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdd54068f60 con 0x7fdd581007f0 2024-09-10T04:17:30.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:30 smithi070 ceph-mon[79774]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:30.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:30 smithi070 ceph-mon[79774]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:30.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:30 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2235661288' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:17:30.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.698+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdd581971b0 con 0x7fdd4407abf0 2024-09-10T04:17:30.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.700+0000 7fdd4e7fc640 1 -- 172.21.15.46:0/617029543 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fdd581971b0 con 0x7fdd4407abf0 2024-09-10T04:17:30.701 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:17:30.702 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:17:30.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd4407abf0 msgr2=0x7fdd4407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd4407abf0 0x7fdd4407d0b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdd58197f00 tx=0x7fdd48031400 comp rx=0 tx=0).stop 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 msgr2=0x7fdd5819c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd5819c5e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400d6f0 tx=0x7fdd5400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 shutdown_connections 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd4407abf0 0x7fdd4407d0b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd581067f0 0x7fdd5819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 --2- 172.21.15.46:0/617029543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd581007f0 0x7fdd5819c5e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 >> 172.21.15.46:0/617029543 conn(0x7fdd580fc550 msgr2=0x7fdd580fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 shutdown_connections 2024-09-10T04:17:30.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.702+0000 7fdd5f4e9640 1 -- 172.21.15.46:0/617029543 wait complete. 2024-09-10T04:17:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.786+0000 7f82a32f2640 1 -- 172.21.15.46:0/1708373560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 msgr2=0x7f829c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.786+0000 7f82a32f2640 1 --2- 172.21.15.46:0/1708373560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c106be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8298009880 tx=0x7f829802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.786+0000 7f82a32f2640 1 -- 172.21.15.46:0/1708373560 shutdown_connections 2024-09-10T04:17:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.786+0000 7f82a32f2640 1 --2- 172.21.15.46:0/1708373560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f829c100800 0x7f829c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.786+0000 7f82a32f2640 1 --2- 172.21.15.46:0/1708373560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c106be0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.786+0000 7f82a32f2640 1 -- 172.21.15.46:0/1708373560 >> 172.21.15.46:0/1708373560 conn(0x7f829c0fc520 msgr2=0x7f829c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.786+0000 7f82a32f2640 1 -- 172.21.15.46:0/1708373560 shutdown_connections 2024-09-10T04:17:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.787+0000 7f82a32f2640 1 -- 172.21.15.46:0/1708373560 wait complete. 2024-09-10T04:17:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.787+0000 7f82a32f2640 1 Processor -- start 2024-09-10T04:17:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.787+0000 7f82a32f2640 1 -- start start 2024-09-10T04:17:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.787+0000 7f82a32f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f829c100800 0x7f829c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.787+0000 7f82a32f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.787+0000 7f82a32f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10dcd0 con 0x7f829c106800 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.787+0000 7f82a32f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f829c10db50 con 0x7f829c100800 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a088d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a088d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49112/0 (socket says 172.21.15.46:49112) 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a088d640 1 -- 172.21.15.46:0/3698107040 learned_addr learned my addr 172.21.15.46:0/3698107040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a108e640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f829c100800 0x7f829c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a088d640 1 -- 172.21.15.46:0/3698107040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f829c100800 msgr2=0x7f829c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a088d640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f829c100800 0x7f829c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a088d640 1 -- 172.21.15.46:0/3698107040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c199350 con 0x7f829c106800 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a088d640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c198d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f828c00d6f0 tx=0x7f828c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82927fc640 1 -- 172.21.15.46:0/3698107040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f828c017070 con 0x7f829c106800 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82927fc640 1 -- 172.21.15.46:0/3698107040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f828c00de00 con 0x7f829c106800 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c199640 con 0x7f829c106800 2024-09-10T04:17:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82927fc640 1 -- 172.21.15.46:0/3698107040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f828c004bb0 con 0x7f829c106800 2024-09-10T04:17:30.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.788+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c1a1c40 con 0x7f829c106800 2024-09-10T04:17:30.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.789+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f829c101f40 con 0x7f829c106800 2024-09-10T04:17:30.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.790+0000 7f82927fc640 1 -- 172.21.15.46:0/3698107040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f828c009a30 con 0x7f829c106800 2024-09-10T04:17:30.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.791+0000 7f82927fc640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f828807ac40 0x7f828807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:17:30.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.791+0000 7f82a108e640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f828807ac40 0x7f828807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:17:30.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.791+0000 7f82927fc640 1 -- 172.21.15.46:0/3698107040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f828c09d740 con 0x7f829c106800 2024-09-10T04:17:30.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.791+0000 7f82a108e640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f828807ac40 0x7f828807d100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82980096c0 tx=0x7f8298031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:17:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:30.795+0000 7f82927fc640 1 -- 172.21.15.46:0/3698107040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f829c199640 con 0x7f829c106800 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.055+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f829c108d70 con 0x7f829c106800 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.055+0000 7f82927fc640 1 -- 172.21.15.46:0/3698107040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f828c068f00 con 0x7f829c106800 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:17:31.057 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:17:31.058 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:17:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f828807ac40 msgr2=0x7f828807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f828807ac40 0x7f828807d100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f82980096c0 tx=0x7f8298031400 comp rx=0 tx=0).stop 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 msgr2=0x7f829c198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c198d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f828c00d6f0 tx=0x7f828c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 shutdown_connections 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f828807ac40 0x7f828807d100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c106800 0x7f829c198d50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 --2- 172.21.15.46:0/3698107040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f829c100800 0x7f829c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 >> 172.21.15.46:0/3698107040 conn(0x7f829c0fc520 msgr2=0x7f829c0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 shutdown_connections 2024-09-10T04:17:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:17:31.058+0000 7f82a32f2640 1 -- 172.21.15.46:0/3698107040 wait complete. 2024-09-10T04:17:31.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:31 smithi046 ceph-mon[109506]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:31.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:31 smithi046 ceph-mon[109506]: pgmap v355: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:31.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:31 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2580779057' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:17:31.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:31 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3698107040' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:17:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:31 smithi070 ceph-mon[79774]: from='client.34542 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:31 smithi070 ceph-mon[79774]: pgmap v355: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:31 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2580779057' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:17:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:31 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3698107040' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:17:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:32 smithi070 ceph-mon[79774]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:32.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:32 smithi046 ceph-mon[109506]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:17:33.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:33 smithi070 ceph-mon[79774]: pgmap v356: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:33.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:33 smithi046 ceph-mon[109506]: pgmap v356: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:35 smithi070 ceph-mon[79774]: pgmap v357: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:35 smithi046 ceph-mon[109506]: pgmap v357: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:36 smithi070 ceph-mon[79774]: pgmap v358: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:36 smithi046 ceph-mon[109506]: pgmap v358: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:38.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:38.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:39 smithi070 ceph-mon[79774]: pgmap v359: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:39.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:39 smithi046 ceph-mon[109506]: pgmap v359: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:41 smithi070 ceph-mon[79774]: pgmap v360: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:41 smithi046 ceph-mon[109506]: pgmap v360: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:43 smithi070 ceph-mon[79774]: pgmap v361: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:43.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:43 smithi046 ceph-mon[109506]: pgmap v361: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:45.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:45 smithi070 ceph-mon[79774]: pgmap v362: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:45 smithi046 ceph-mon[109506]: pgmap v362: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:47.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:47 smithi070 ceph-mon[79774]: pgmap v363: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:47 smithi046 ceph-mon[109506]: pgmap v363: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:49 smithi070 ceph-mon[79774]: pgmap v364: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:49 smithi046 ceph-mon[109506]: pgmap v364: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:51.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:51 smithi070 ceph-mon[79774]: pgmap v365: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:51.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:51 smithi046 ceph-mon[109506]: pgmap v365: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:53 smithi070 ceph-mon[79774]: pgmap v366: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:53 smithi046 ceph-mon[109506]: pgmap v366: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:17:55.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:55 smithi070 ceph-mon[79774]: pgmap v367: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:55.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:55 smithi046 ceph-mon[109506]: pgmap v367: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:57 smithi070 ceph-mon[79774]: pgmap v368: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:57.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:57 smithi046 ceph-mon[109506]: pgmap v368: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:17:59 smithi070 ceph-mon[79774]: pgmap v369: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:17:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:17:59 smithi046 ceph-mon[109506]: pgmap v369: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.152+0000 7f3a0230b640 1 -- 172.21.15.46:0/2332544906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc101900 msgr2=0x7f39fc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.152+0000 7f3a0230b640 1 --2- 172.21.15.46:0/2332544906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc101900 0x7f39fc101d60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009880 tx=0x7f39f002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.152+0000 7f3a0230b640 1 -- 172.21.15.46:0/2332544906 shutdown_connections 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.152+0000 7f3a0230b640 1 --2- 172.21.15.46:0/2332544906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc101900 0x7f39fc101d60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.152+0000 7f3a0230b640 1 --2- 172.21.15.46:0/2332544906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39fc107900 0x7f39fc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.152+0000 7f3a0230b640 1 -- 172.21.15.46:0/2332544906 >> 172.21.15.46:0/2332544906 conn(0x7f39fc0fd600 msgr2=0x7f39fc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.153+0000 7f3a0230b640 1 -- 172.21.15.46:0/2332544906 shutdown_connections 2024-09-10T04:18:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.153+0000 7f3a0230b640 1 -- 172.21.15.46:0/2332544906 wait complete. 2024-09-10T04:18:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.153+0000 7f3a0230b640 1 Processor -- start 2024-09-10T04:18:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.153+0000 7f3a0230b640 1 -- start start 2024-09-10T04:18:01.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f3a0230b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39fc101900 0x7f39fc19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f3a0230b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc107900 0x7f39fc19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f3a0230b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc10edd0 con 0x7f39fc107900 2024-09-10T04:18:01.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f3a0230b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc10ec50 con 0x7f39fc101900 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc107900 0x7f39fc19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc107900 0x7f39fc19db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34104/0 (socket says 172.21.15.46:34104) 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fb7fe640 1 -- 172.21.15.46:0/4076855246 learned_addr learned my addr 172.21.15.46:0/4076855246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fbfff640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39fc101900 0x7f39fc19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fb7fe640 1 -- 172.21.15.46:0/4076855246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39fc101900 msgr2=0x7f39fc19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fb7fe640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39fc101900 0x7f39fc19d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fb7fe640 1 -- 172.21.15.46:0/4076855246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc1977a0 con 0x7f39fc107900 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.154+0000 7f39fb7fe640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc107900 0x7f39fc19db60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39f0031460 tx=0x7f39f00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.155+0000 7f39f97fa640 1 -- 172.21.15.46:0/4076855246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f39f003f070 con 0x7f39fc107900 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.155+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc197a30 con 0x7f39fc107900 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.155+0000 7f39f97fa640 1 -- 172.21.15.46:0/4076855246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f39f00035f0 con 0x7f39fc107900 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.155+0000 7f39f97fa640 1 -- 172.21.15.46:0/4076855246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f39f0035030 con 0x7f39fc107900 2024-09-10T04:18:01.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.155+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc197f60 con 0x7f39fc107900 2024-09-10T04:18:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.156+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc103040 con 0x7f39fc107900 2024-09-10T04:18:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.157+0000 7f39f97fa640 1 -- 172.21.15.46:0/4076855246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f39f0042070 con 0x7f39fc107900 2024-09-10T04:18:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.157+0000 7f39f97fa640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f39e407abf0 0x7f39e407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.157+0000 7f39f97fa640 1 -- 172.21.15.46:0/4076855246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f39f00be5d0 con 0x7f39fc107900 2024-09-10T04:18:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.157+0000 7f39fbfff640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f39e407abf0 0x7f39e407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.158+0000 7f39fbfff640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f39e407abf0 0x7f39e407d0b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f39ec005fd0 tx=0x7f39ec00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:01.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.162+0000 7f39f97fa640 1 -- 172.21.15.46:0/4076855246 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f39f0089d10 con 0x7f39fc107900 2024-09-10T04:18:01.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.366+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f39fc198210 con 0x7f39e407abf0 2024-09-10T04:18:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.367+0000 7f39f97fa640 1 -- 172.21.15.46:0/4076855246 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f39fc198210 con 0x7f39e407abf0 2024-09-10T04:18:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f39e407abf0 msgr2=0x7f39e407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f39e407abf0 0x7f39e407d0b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f39ec005fd0 tx=0x7f39ec00b390 comp rx=0 tx=0).stop 2024-09-10T04:18:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc107900 msgr2=0x7f39fc19db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc107900 0x7f39fc19db60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f39f0031460 tx=0x7f39f00045e0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 shutdown_connections 2024-09-10T04:18:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f39e407abf0 0x7f39e407d0b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39fc107900 0x7f39fc19db60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 --2- 172.21.15.46:0/4076855246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39fc101900 0x7f39fc19d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 >> 172.21.15.46:0/4076855246 conn(0x7f39fc0fd600 msgr2=0x7f39fc0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 shutdown_connections 2024-09-10T04:18:01.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.370+0000 7f3a0230b640 1 -- 172.21.15.46:0/4076855246 wait complete. 2024-09-10T04:18:01.380 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:18:01.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 -- 172.21.15.46:0/807030960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 msgr2=0x7f77a4100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/807030960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a4100c50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7798009880 tx=0x7f779802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 -- 172.21.15.46:0/807030960 shutdown_connections 2024-09-10T04:18:01.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/807030960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a4100c50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/807030960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77a41067f0 0x7f77a4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 -- 172.21.15.46:0/807030960 >> 172.21.15.46:0/807030960 conn(0x7f77a40fc550 msgr2=0x7f77a40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:01.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 -- 172.21.15.46:0/807030960 shutdown_connections 2024-09-10T04:18:01.458 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:01 smithi046 ceph-mon[109506]: pgmap v370: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:01.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.455+0000 7f77aa0a9640 1 -- 172.21.15.46:0/807030960 wait complete. 2024-09-10T04:18:01.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.456+0000 7f77aa0a9640 1 Processor -- start 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.456+0000 7f77aa0a9640 1 -- start start 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.456+0000 7f77aa0a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.456+0000 7f77aa0a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77a41067f0 0x7f77a419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.456+0000 7f77aa0a9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410dca0 con 0x7f77a41007f0 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.456+0000 7f77aa0a9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410db20 con 0x7f77a41067f0 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.456+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34124/0 (socket says 172.21.15.46:34124) 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a37fe640 1 -- 172.21.15.46:0/2090988645 learned_addr learned my addr 172.21.15.46:0/2090988645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a2ffd640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77a41067f0 0x7f77a419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a37fe640 1 -- 172.21.15.46:0/2090988645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77a41067f0 msgr2=0x7f77a419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a37fe640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77a41067f0 0x7f77a419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a37fe640 1 -- 172.21.15.46:0/2090988645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a4197780 con 0x7f77a41007f0 2024-09-10T04:18:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a37fe640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a419d600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f779400c810 tx=0x7f779400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a0ff9640 1 -- 172.21.15.46:0/2090988645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7794004020 con 0x7f77a41007f0 2024-09-10T04:18:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a0ff9640 1 -- 172.21.15.46:0/2090988645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f77940041c0 con 0x7f77a41007f0 2024-09-10T04:18:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a4197a70 con 0x7f77a41007f0 2024-09-10T04:18:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77a0ff9640 1 -- 172.21.15.46:0/2090988645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f77940047a0 con 0x7f77a41007f0 2024-09-10T04:18:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.457+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a4197fd0 con 0x7f77a41007f0 2024-09-10T04:18:01.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.459+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a4101f30 con 0x7f77a41007f0 2024-09-10T04:18:01.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.459+0000 7f77a0ff9640 1 -- 172.21.15.46:0/2090988645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f77940079b0 con 0x7f77a41007f0 2024-09-10T04:18:01.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.460+0000 7f77a0ff9640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f778c07ac40 0x7f778c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.460+0000 7f77a0ff9640 1 -- 172.21.15.46:0/2090988645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f779409daf0 con 0x7f77a41007f0 2024-09-10T04:18:01.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.460+0000 7f77a2ffd640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f778c07ac40 0x7f778c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.460+0000 7f77a2ffd640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f778c07ac40 0x7f778c07d100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77a4198ad0 tx=0x7f7798031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:01.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.465+0000 7f77a0ff9640 1 -- 172.21.15.46:0/2090988645 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7794014030 con 0x7f77a41007f0 2024-09-10T04:18:01.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:01 smithi070 ceph-mon[79774]: pgmap v370: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.670+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f77a41982e0 con 0x7f778c07ac40 2024-09-10T04:18:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.672+0000 7f77a0ff9640 1 -- 172.21.15.46:0/2090988645 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f77a41982e0 con 0x7f778c07ac40 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f778c07ac40 msgr2=0x7f778c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f778c07ac40 0x7f778c07d100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77a4198ad0 tx=0x7f7798031400 comp rx=0 tx=0).stop 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 msgr2=0x7f77a419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a419d600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f779400c810 tx=0x7f779400ccd0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 shutdown_connections 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f778c07ac40 0x7f778c07d100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77a41067f0 0x7f77a419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 --2- 172.21.15.46:0/2090988645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77a41007f0 0x7f77a419d600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 >> 172.21.15.46:0/2090988645 conn(0x7f77a40fc550 msgr2=0x7f77a4069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 shutdown_connections 2024-09-10T04:18:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.674+0000 7f77aa0a9640 1 -- 172.21.15.46:0/2090988645 wait complete. 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 -- 172.21.15.46:0/3102334686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 msgr2=0x7fc68810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 --2- 172.21.15.46:0/3102334686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68810b8c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc670009880 tx=0x7fc67002a530 comp rx=0 tx=0).stop 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 -- 172.21.15.46:0/3102334686 shutdown_connections 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 --2- 172.21.15.46:0/3102334686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68810b8c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 --2- 172.21.15.46:0/3102334686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc688102ab0 0x7fc688102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 -- 172.21.15.46:0/3102334686 >> 172.21.15.46:0/3102334686 conn(0x7fc6880fc550 msgr2=0x7fc6880fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 -- 172.21.15.46:0/3102334686 shutdown_connections 2024-09-10T04:18:01.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.759+0000 7fc68dcba640 1 -- 172.21.15.46:0/3102334686 wait complete. 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc68dcba640 1 Processor -- start 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc68dcba640 1 -- start start 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc68dcba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc688102ab0 0x7fc68819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc68dcba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc68dcba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc68810edc0 con 0x7fc6881033d0 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc68dcba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc68810ec40 con 0x7fc688102ab0 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc686ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc686ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34150/0 (socket says 172.21.15.46:34150) 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc686ffd640 1 -- 172.21.15.46:0/4186381829 learned_addr learned my addr 172.21.15.46:0/4186381829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.760+0000 7fc6877fe640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc688102ab0 0x7fc68819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.761+0000 7fc686ffd640 1 -- 172.21.15.46:0/4186381829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc688102ab0 msgr2=0x7fc68819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.761+0000 7fc686ffd640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc688102ab0 0x7fc68819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.761+0000 7fc686ffd640 1 -- 172.21.15.46:0/4186381829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc688197790 con 0x7fc6881033d0 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.761+0000 7fc686ffd640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68819db50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc670031460 tx=0x7fc6700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.761+0000 7fc684ff9640 1 -- 172.21.15.46:0/4186381829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc67003f070 con 0x7fc6881033d0 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.761+0000 7fc684ff9640 1 -- 172.21.15.46:0/4186381829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc670004ac0 con 0x7fc6881033d0 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.761+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc688197a20 con 0x7fc6881033d0 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.762+0000 7fc684ff9640 1 -- 172.21.15.46:0/4186381829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc670035030 con 0x7fc6881033d0 2024-09-10T04:18:01.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.762+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc688197f50 con 0x7fc6881033d0 2024-09-10T04:18:01.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.763+0000 7fc684ff9640 1 -- 172.21.15.46:0/4186381829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc670042070 con 0x7fc6881033d0 2024-09-10T04:18:01.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.763+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68806a580 con 0x7fc6881033d0 2024-09-10T04:18:01.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.764+0000 7fc684ff9640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc67407ac50 0x7fc67407d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:01.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.764+0000 7fc684ff9640 1 -- 172.21.15.46:0/4186381829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc6700be560 con 0x7fc6881033d0 2024-09-10T04:18:01.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.764+0000 7fc6877fe640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc67407ac50 0x7fc67407d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:01.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.764+0000 7fc6877fe640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc67407ac50 0x7fc67407d110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc678007ec0 tx=0x7fc6780075b0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.768+0000 7fc684ff9640 1 -- 172.21.15.46:0/4186381829 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc670089d20 con 0x7fc6881033d0 2024-09-10T04:18:01.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.963+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc688198200 con 0x7fc67407ac50 2024-09-10T04:18:01.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.973+0000 7fc684ff9640 1 -- 172.21.15.46:0/4186381829 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fc688198200 con 0x7fc67407ac50 2024-09-10T04:18:01.973 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (9m) 4m ago 20m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (12m) 4m ago 21m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (11m) 3m ago 20m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (8m) 4m ago 21m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (7m) 3m ago 20m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (9m) 4m ago 20m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (4m) 4m ago 15m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (4m) 4m ago 15m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (3m) 3m ago 15m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (15m) 3m ago 15m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (11m) 4m ago 22m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (10m) 3m ago 20m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (8m) 4m ago 22m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:18:01.974 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (8m) 3m ago 20m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (10m) 4m ago 21m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (10m) 3m ago 20m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (7m) 4m ago 18m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (7m) 4m ago 18m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (6m) 4m ago 17m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (5m) 3m ago 17m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (5m) 3m ago 17m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (4m) 3m ago 16m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:18:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (9m) 4m ago 20m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:18:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc67407ac50 msgr2=0x7fc67407d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc67407ac50 0x7fc67407d110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc678007ec0 tx=0x7fc6780075b0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 msgr2=0x7fc68819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68819db50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc670031460 tx=0x7fc6700046c0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 shutdown_connections 2024-09-10T04:18:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc67407ac50 0x7fc67407d110 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6881033d0 0x7fc68819db50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 --2- 172.21.15.46:0/4186381829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc688102ab0 0x7fc68819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 >> 172.21.15.46:0/4186381829 conn(0x7fc6880fc550 msgr2=0x7fc68810aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 shutdown_connections 2024-09-10T04:18:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:01.975+0000 7fc68dcba640 1 -- 172.21.15.46:0/4186381829 wait complete. 2024-09-10T04:18:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 -- 172.21.15.46:0/558337346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 msgr2=0x7fc204101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/558337346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc204101d60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009880 tx=0x7fc1f002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 -- 172.21.15.46:0/558337346 shutdown_connections 2024-09-10T04:18:02.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/558337346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc204101d60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/558337346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc204107900 0x7fc204107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 -- 172.21.15.46:0/558337346 >> 172.21.15.46:0/558337346 conn(0x7fc2040fd600 msgr2=0x7fc2040ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 -- 172.21.15.46:0/558337346 shutdown_connections 2024-09-10T04:18:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.058+0000 7fc20b0f3640 1 -- 172.21.15.46:0/558337346 wait complete. 2024-09-10T04:18:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.059+0000 7fc20b0f3640 1 Processor -- start 2024-09-10T04:18:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.059+0000 7fc20b0f3640 1 -- start start 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.059+0000 7fc20b0f3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc20419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.059+0000 7fc20b0f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc204107900 0x7fc20419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.059+0000 7fc20b0f3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc20410edb0 con 0x7fc204101900 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.059+0000 7fc20b0f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc20410ec30 con 0x7fc204107900 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.059+0000 7fc208e8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc20419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc208e8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc20419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34174/0 (socket says 172.21.15.46:34174) 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc208e8f640 1 -- 172.21.15.46:0/1373047866 learned_addr learned my addr 172.21.15.46:0/1373047866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc203fff640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc204107900 0x7fc20419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc208e8f640 1 -- 172.21.15.46:0/1373047866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc204107900 msgr2=0x7fc20419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc208e8f640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc204107900 0x7fc20419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc208e8f640 1 -- 172.21.15.46:0/1373047866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc204197800 con 0x7fc204101900 2024-09-10T04:18:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc203fff640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc204107900 0x7fc20419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:18:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc208e8f640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc20419d680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800c810 tx=0x7fc1f800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc201ffb640 1 -- 172.21.15.46:0/1373047866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1f8004020 con 0x7fc204101900 2024-09-10T04:18:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc204197af0 con 0x7fc204101900 2024-09-10T04:18:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc201ffb640 1 -- 172.21.15.46:0/1373047866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc1f80041c0 con 0x7fc204101900 2024-09-10T04:18:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.060+0000 7fc201ffb640 1 -- 172.21.15.46:0/1373047866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1f80047a0 con 0x7fc204101900 2024-09-10T04:18:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.061+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc204198050 con 0x7fc204101900 2024-09-10T04:18:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.062+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc204103040 con 0x7fc204101900 2024-09-10T04:18:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.062+0000 7fc201ffb640 1 -- 172.21.15.46:0/1373047866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc1f80079b0 con 0x7fc204101900 2024-09-10T04:18:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.063+0000 7fc201ffb640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1f407f000 0x7fc1f40814c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.063+0000 7fc203fff640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1f407f000 0x7fc1f40814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.063+0000 7fc201ffb640 1 -- 172.21.15.46:0/1373047866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc1f809dba0 con 0x7fc204101900 2024-09-10T04:18:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.064+0000 7fc203fff640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1f407f000 0x7fc1f40814c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc204198b50 tx=0x7fc1f0009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.067+0000 7fc201ffb640 1 -- 172.21.15.46:0/1373047866 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc1f8014030 con 0x7fc204101900 2024-09-10T04:18:02.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.329+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc204109e90 con 0x7fc204101900 2024-09-10T04:18:02.330 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:02 smithi046 ceph-mon[109506]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:02.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.330+0000 7fc201ffb640 1 -- 172.21.15.46:0/1373047866 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fc1f80692e0 con 0x7fc204101900 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:18:02.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:18:02.332 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:18:02.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:02.332 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:18:02.332 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:18:02.332 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:18:02.332 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:18:02.332 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1f407f000 msgr2=0x7fc1f40814c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1f407f000 0x7fc1f40814c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc204198b50 tx=0x7fc1f0009560 comp rx=0 tx=0).stop 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 msgr2=0x7fc20419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc20419d680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800c810 tx=0x7fc1f800ccd0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 shutdown_connections 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1f407f000 0x7fc1f40814c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc204107900 0x7fc20419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 --2- 172.21.15.46:0/1373047866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc204101900 0x7fc20419d680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 >> 172.21.15.46:0/1373047866 conn(0x7fc2040fd600 msgr2=0x7fc2040fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:02.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 shutdown_connections 2024-09-10T04:18:02.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.332+0000 7fc20b0f3640 1 -- 172.21.15.46:0/1373047866 wait complete. 2024-09-10T04:18:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 -- 172.21.15.46:0/2771139783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 msgr2=0x7f55b4102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 --2- 172.21.15.46:0/2771139783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4102f00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f55a40098b0 tx=0x7f55a402a560 comp rx=0 tx=0).stop 2024-09-10T04:18:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 -- 172.21.15.46:0/2771139783 shutdown_connections 2024-09-10T04:18:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 --2- 172.21.15.46:0/2771139783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55b4103440 0x7f55b410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 --2- 172.21.15.46:0/2771139783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4102f00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 -- 172.21.15.46:0/2771139783 >> 172.21.15.46:0/2771139783 conn(0x7f55b40fc520 msgr2=0x7f55b40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 -- 172.21.15.46:0/2771139783 shutdown_connections 2024-09-10T04:18:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.416+0000 7f55ba68b640 1 -- 172.21.15.46:0/2771139783 wait complete. 2024-09-10T04:18:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55ba68b640 1 Processor -- start 2024-09-10T04:18:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55ba68b640 1 -- start start 2024-09-10T04:18:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55ba68b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55ba68b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55b4103440 0x7f55b4100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55ba68b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55b410edd0 con 0x7f55b4102b20 2024-09-10T04:18:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55ba68b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f55b410ec50 con 0x7f55b4103440 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55b3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55b3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34196/0 (socket says 172.21.15.46:34196) 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.417+0000 7f55b3fff640 1 -- 172.21.15.46:0/739154685 learned_addr learned my addr 172.21.15.46:0/739154685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b37fe640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55b4103440 0x7f55b4100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b3fff640 1 -- 172.21.15.46:0/739154685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55b4103440 msgr2=0x7f55b4100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b3fff640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55b4103440 0x7f55b4100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b3fff640 1 -- 172.21.15.46:0/739154685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b4101200 con 0x7f55b4102b20 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b3fff640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4100780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009230 tx=0x7f55a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b17fa640 1 -- 172.21.15.46:0/739154685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f55a403f070 con 0x7f55b4102b20 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b4101490 con 0x7f55b4102b20 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b17fa640 1 -- 172.21.15.46:0/739154685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f55a4004ac0 con 0x7f55b4102b20 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55b17fa640 1 -- 172.21.15.46:0/739154685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f55a4035030 con 0x7f55b4102b20 2024-09-10T04:18:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.418+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b41a3d70 con 0x7f55b4102b20 2024-09-10T04:18:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.419+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b406aa30 con 0x7f55b4102b20 2024-09-10T04:18:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.420+0000 7f55b17fa640 1 -- 172.21.15.46:0/739154685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f55a4042070 con 0x7f55b4102b20 2024-09-10T04:18:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.420+0000 7f55b17fa640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f559c07ac00 0x7f559c07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.421+0000 7f55b17fa640 1 -- 172.21.15.46:0/739154685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f55a40be6f0 con 0x7f55b4102b20 2024-09-10T04:18:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.421+0000 7f55b37fe640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f559c07ac00 0x7f559c07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.421+0000 7f55b37fe640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f559c07ac00 0x7f559c07d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55b4102210 tx=0x7f55a800a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:02.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.425+0000 7f55b17fa640 1 -- 172.21.15.46:0/739154685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f55a4089e30 con 0x7f55b4102b20 2024-09-10T04:18:02.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:02 smithi070 ceph-mon[79774]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:02.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.648+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f55b4105ff0 con 0x7f55b4102b20 2024-09-10T04:18:02.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.648+0000 7f55b17fa640 1 -- 172.21.15.46:0/739154685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f55a408ece0 con 0x7f55b4102b20 2024-09-10T04:18:02.649 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:18:02.649 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:18:02.650 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:18:02.651 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:02.652 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.650+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f559c07ac00 msgr2=0x7f559c07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.650+0000 7f55ba68b640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f559c07ac00 0x7f559c07d0c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f55b4102210 tx=0x7f55a800a2d0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.650+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 msgr2=0x7f55b4100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.650+0000 7f55ba68b640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4100780 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55a4009230 tx=0x7f55a40046c0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.651+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 shutdown_connections 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.651+0000 7f55ba68b640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f559c07ac00 0x7f559c07d0c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.651+0000 7f55ba68b640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55b4103440 0x7f55b4100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.651+0000 7f55ba68b640 1 --2- 172.21.15.46:0/739154685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b4102b20 0x7f55b4100780 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.651+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 >> 172.21.15.46:0/739154685 conn(0x7f55b40fc520 msgr2=0x7f55b4103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:02.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.651+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 shutdown_connections 2024-09-10T04:18:02.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.651+0000 7f55ba68b640 1 -- 172.21.15.46:0/739154685 wait complete. 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 -- 172.21.15.46:0/885177511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 msgr2=0x7f1a30100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 --2- 172.21.15.46:0/885177511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a30100c50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009880 tx=0x7f1a2402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 -- 172.21.15.46:0/885177511 shutdown_connections 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 --2- 172.21.15.46:0/885177511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a30100c50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 --2- 172.21.15.46:0/885177511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a301067f0 0x7f1a30106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 -- 172.21.15.46:0/885177511 >> 172.21.15.46:0/885177511 conn(0x7f1a300fc550 msgr2=0x7f1a300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 -- 172.21.15.46:0/885177511 shutdown_connections 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 -- 172.21.15.46:0/885177511 wait complete. 2024-09-10T04:18:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 Processor -- start 2024-09-10T04:18:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.736+0000 7f1a37086640 1 -- start start 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a37086640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a3019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a37086640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a301067f0 0x7f1a3019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a37086640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3010dca0 con 0x7f1a301007f0 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a37086640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3010db20 con 0x7f1a301067f0 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a34e22640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a3019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a34e22640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a3019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34204/0 (socket says 172.21.15.46:34204) 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a34e22640 1 -- 172.21.15.46:0/3909446862 learned_addr learned my addr 172.21.15.46:0/3909446862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a301067f0 0x7f1a3019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a34e22640 1 -- 172.21.15.46:0/3909446862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a301067f0 msgr2=0x7f1a3019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a34e22640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a301067f0 0x7f1a3019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a34e22640 1 -- 172.21.15.46:0/3909446862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a30197780 con 0x7f1a301007f0 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.737+0000 7f1a34e22640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a3019d600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c8a0 tx=0x7f1a2000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.738+0000 7f1a2dffb640 1 -- 172.21.15.46:0/3909446862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1a200079b0 con 0x7f1a301007f0 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.738+0000 7f1a2dffb640 1 -- 172.21.15.46:0/3909446862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1a20007b50 con 0x7f1a301007f0 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.738+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a30197a70 con 0x7f1a301007f0 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.738+0000 7f1a2dffb640 1 -- 172.21.15.46:0/3909446862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1a20018700 con 0x7f1a301007f0 2024-09-10T04:18:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.738+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a30197fd0 con 0x7f1a301007f0 2024-09-10T04:18:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.739+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a30101f30 con 0x7f1a301007f0 2024-09-10T04:18:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.739+0000 7f1a2dffb640 1 -- 172.21.15.46:0/3909446862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1a20007d00 con 0x7f1a301007f0 2024-09-10T04:18:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.740+0000 7f1a2dffb640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1a1807ac40 0x7f1a1807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.740+0000 7f1a2dffb640 1 -- 172.21.15.46:0/3909446862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f1a2009de00 con 0x7f1a301007f0 2024-09-10T04:18:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.740+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1a1807ac40 0x7f1a1807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.741+0000 7f1a2ffff640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1a1807ac40 0x7f1a1807d100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1a30198ad0 tx=0x7f1a24031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.745+0000 7f1a2dffb640 1 -- 172.21.15.46:0/3909446862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1a20014030 con 0x7f1a301007f0 2024-09-10T04:18:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.948+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1a301982e0 con 0x7f1a1807ac40 2024-09-10T04:18:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.950+0000 7f1a2dffb640 1 -- 172.21.15.46:0/3909446862 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f1a301982e0 con 0x7f1a1807ac40 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:18:02.951 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1a1807ac40 msgr2=0x7f1a1807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1a1807ac40 0x7f1a1807d100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1a30198ad0 tx=0x7f1a24031400 comp rx=0 tx=0).stop 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 msgr2=0x7f1a3019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a3019d600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c8a0 tx=0x7f1a2000cd60 comp rx=0 tx=0).stop 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 shutdown_connections 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1a1807ac40 0x7f1a1807d100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a301067f0 0x7f1a3019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 --2- 172.21.15.46:0/3909446862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a301007f0 0x7f1a3019d600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:02.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 >> 172.21.15.46:0/3909446862 conn(0x7f1a300fc550 msgr2=0x7f1a30069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:02.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 shutdown_connections 2024-09-10T04:18:02.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:02.952+0000 7f1a37086640 1 -- 172.21.15.46:0/3909446862 wait complete. 2024-09-10T04:18:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 -- 172.21.15.46:0/3274605311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 msgr2=0x7fee00102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 --2- 172.21.15.46:0/3274605311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00102e90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fedf0009880 tx=0x7fedf002a530 comp rx=0 tx=0).stop 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 -- 172.21.15.46:0/3274605311 shutdown_connections 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 --2- 172.21.15.46:0/3274605311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee001033d0 0x7fee0010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 --2- 172.21.15.46:0/3274605311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00102e90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 -- 172.21.15.46:0/3274605311 >> 172.21.15.46:0/3274605311 conn(0x7fee000fc550 msgr2=0x7fee000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 -- 172.21.15.46:0/3274605311 shutdown_connections 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.036+0000 7fee054ec640 1 -- 172.21.15.46:0/3274605311 wait complete. 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fee054ec640 1 Processor -- start 2024-09-10T04:18:03.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fee054ec640 1 -- start start 2024-09-10T04:18:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fee054ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fee054ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee001033d0 0x7fee00100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fee054ec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee0010ede0 con 0x7fee00102ab0 2024-09-10T04:18:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fee054ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fee0010ec60 con 0x7fee001033d0 2024-09-10T04:18:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fedfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:03.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fedfeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34222/0 (socket says 172.21.15.46:34222) 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.037+0000 7fedfeffd640 1 -- 172.21.15.46:0/3748737987 learned_addr learned my addr 172.21.15.46:0/3748737987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedf7fff640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee001033d0 0x7fee00100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedfeffd640 1 -- 172.21.15.46:0/3748737987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee001033d0 msgr2=0x7fee00100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedfeffd640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee001033d0 0x7fee00100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedfeffd640 1 -- 172.21.15.46:0/3748737987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee00101190 con 0x7fee00102ab0 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedfeffd640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00100710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fedf0031460 tx=0x7fedf0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedfcff9640 1 -- 172.21.15.46:0/3748737987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fedf003f070 con 0x7fee00102ab0 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee00101420 con 0x7fee00102ab0 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedfcff9640 1 -- 172.21.15.46:0/3748737987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fedf00035f0 con 0x7fee00102ab0 2024-09-10T04:18:03.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fedfcff9640 1 -- 172.21.15.46:0/3748737987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fedf0035030 con 0x7fee00102ab0 2024-09-10T04:18:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.038+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee001a3d70 con 0x7fee00102ab0 2024-09-10T04:18:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.039+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0006a580 con 0x7fee00102ab0 2024-09-10T04:18:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.040+0000 7fedfcff9640 1 -- 172.21.15.46:0/3748737987 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fedf0042070 con 0x7fee00102ab0 2024-09-10T04:18:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.040+0000 7fedfcff9640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fedec07abf0 0x7fedec07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:03.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.041+0000 7fedf7fff640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fedec07abf0 0x7fedec07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.041+0000 7fedfcff9640 1 -- 172.21.15.46:0/3748737987 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fedf00be5b0 con 0x7fee00102ab0 2024-09-10T04:18:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.041+0000 7fedf7fff640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fedec07abf0 0x7fedec07d0b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee001021a0 tx=0x7fede8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:03.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.045+0000 7fedfcff9640 1 -- 172.21.15.46:0/3748737987 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fedf0089cf0 con 0x7fee00102ab0 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.304+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fee00101c40 con 0x7fee00102ab0 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.304+0000 7fedfcff9640 1 -- 172.21.15.46:0/3748737987 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fedf008eba0 con 0x7fee00102ab0 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:18:03.305 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:18:03.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fedec07abf0 msgr2=0x7fedec07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:03.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fedec07abf0 0x7fedec07d0b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee001021a0 tx=0x7fede8005f90 comp rx=0 tx=0).stop 2024-09-10T04:18:03.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 msgr2=0x7fee00100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:03.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00100710 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fedf0031460 tx=0x7fedf0004530 comp rx=0 tx=0).stop 2024-09-10T04:18:03.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 shutdown_connections 2024-09-10T04:18:03.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fedec07abf0 0x7fedec07d0b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:03.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee001033d0 0x7fee00100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:03.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 --2- 172.21.15.46:0/3748737987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee00102ab0 0x7fee00100710 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:03.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.306+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 >> 172.21.15.46:0/3748737987 conn(0x7fee000fc550 msgr2=0x7fee0010aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:03.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.307+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 shutdown_connections 2024-09-10T04:18:03.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:03.307+0000 7fee054ec640 1 -- 172.21.15.46:0/3748737987 wait complete. 2024-09-10T04:18:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:03 smithi070 ceph-mon[79774]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:03 smithi070 ceph-mon[79774]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:03 smithi070 ceph-mon[79774]: pgmap v371: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:03 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1373047866' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:18:03.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:03 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/739154685' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:18:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:03 smithi046 ceph-mon[109506]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:03.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:03 smithi046 ceph-mon[109506]: from='client.34570 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:03 smithi046 ceph-mon[109506]: pgmap v371: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:03 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1373047866' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:18:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:03 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/739154685' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:18:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:04 smithi070 ceph-mon[79774]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:04 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3748737987' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:18:04.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:04 smithi046 ceph-mon[109506]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:04.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:04 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3748737987' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:18:05.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:05 smithi070 ceph-mon[79774]: pgmap v372: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:05.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:05 smithi046 ceph-mon[109506]: pgmap v372: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:07.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:07 smithi070 ceph-mon[79774]: pgmap v373: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:07.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:07 smithi046 ceph-mon[109506]: pgmap v373: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:08.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:09.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:09 smithi070 ceph-mon[79774]: pgmap v374: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:09.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:09 smithi046 ceph-mon[109506]: pgmap v374: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:11.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:11 smithi070 ceph-mon[79774]: pgmap v375: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:11.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:11 smithi046 ceph-mon[109506]: pgmap v375: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:13.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:13 smithi070 ceph-mon[79774]: pgmap v376: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:13.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:13 smithi046 ceph-mon[109506]: pgmap v376: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:15.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:15 smithi070 ceph-mon[79774]: pgmap v377: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:15.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:15 smithi046 ceph-mon[109506]: pgmap v377: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:17.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:17 smithi070 ceph-mon[79774]: pgmap v378: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:17 smithi046 ceph-mon[109506]: pgmap v378: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:19.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:19 smithi070 ceph-mon[79774]: pgmap v379: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:19.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:19 smithi046 ceph-mon[109506]: pgmap v379: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:21 smithi070 ceph-mon[79774]: pgmap v380: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:21.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:21 smithi046 ceph-mon[109506]: pgmap v380: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:23 smithi070 ceph-mon[79774]: pgmap v381: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:23 smithi046 ceph-mon[109506]: pgmap v381: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:25.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:25 smithi070 ceph-mon[79774]: pgmap v382: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:25.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:25 smithi046 ceph-mon[109506]: pgmap v382: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:27 smithi070 ceph-mon[79774]: pgmap v383: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:27.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:27 smithi046 ceph-mon[109506]: pgmap v383: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:29 smithi070 ceph-mon[79774]: pgmap v384: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:29 smithi046 ceph-mon[109506]: pgmap v384: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:31 smithi070 ceph-mon[79774]: pgmap v385: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:31.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:31 smithi046 ceph-mon[109506]: pgmap v385: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 -- 172.21.15.46:0/2847252396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 msgr2=0x7f86d0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 --2- 172.21.15.46:0/2847252396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d0102ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009880 tx=0x7f86c002a530 comp rx=0 tx=0).stop 2024-09-10T04:18:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 -- 172.21.15.46:0/2847252396 shutdown_connections 2024-09-10T04:18:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 --2- 172.21.15.46:0/2847252396 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d01033e0 0x7f86d010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 --2- 172.21.15.46:0/2847252396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d0102ea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 -- 172.21.15.46:0/2847252396 >> 172.21.15.46:0/2847252396 conn(0x7f86d00fc520 msgr2=0x7f86d00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 -- 172.21.15.46:0/2847252396 shutdown_connections 2024-09-10T04:18:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 -- 172.21.15.46:0/2847252396 wait complete. 2024-09-10T04:18:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.405+0000 7f86d60ae640 1 Processor -- start 2024-09-10T04:18:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86d60ae640 1 -- start start 2024-09-10T04:18:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86d60ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86d60ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d01033e0 0x7f86d019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86d60ae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010edf0 con 0x7f86d0102ac0 2024-09-10T04:18:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86d60ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f86d010ec70 con 0x7f86d01033e0 2024-09-10T04:18:33.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37016/0 (socket says 172.21.15.46:37016) 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86cf7fe640 1 -- 172.21.15.46:0/3256719956 learned_addr learned my addr 172.21.15.46:0/3256719956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.406+0000 7f86ceffd640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d01033e0 0x7f86d019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86cf7fe640 1 -- 172.21.15.46:0/3256719956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d01033e0 msgr2=0x7f86d019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86cf7fe640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d01033e0 0x7f86d019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86cf7fe640 1 -- 172.21.15.46:0/3256719956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d01977c0 con 0x7f86d0102ac0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86cf7fe640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d019d640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86c0031460 tx=0x7f86c00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86ccff9640 1 -- 172.21.15.46:0/3256719956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f86c003f070 con 0x7f86d0102ac0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86ccff9640 1 -- 172.21.15.46:0/3256719956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f86c00044d0 con 0x7f86d0102ac0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d0197a50 con 0x7f86d0102ac0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86ccff9640 1 -- 172.21.15.46:0/3256719956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f86c0035030 con 0x7f86d0102ac0 2024-09-10T04:18:33.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.407+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d0197f80 con 0x7f86d0102ac0 2024-09-10T04:18:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.408+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d006aa90 con 0x7f86d0102ac0 2024-09-10T04:18:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.409+0000 7f86ccff9640 1 -- 172.21.15.46:0/3256719956 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f86c0042070 con 0x7f86d0102ac0 2024-09-10T04:18:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.410+0000 7f86ccff9640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f86b807ac50 0x7f86b807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:33.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.410+0000 7f86ccff9640 1 -- 172.21.15.46:0/3256719956 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f86c00be770 con 0x7f86d0102ac0 2024-09-10T04:18:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.410+0000 7f86ceffd640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f86b807ac50 0x7f86b807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:33.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.410+0000 7f86ceffd640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f86b807ac50 0x7f86b807d110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86d0108900 tx=0x7f86c4005ec0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:33.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.414+0000 7f86ccff9640 1 -- 172.21.15.46:0/3256719956 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f86c0004cf0 con 0x7f86d0102ac0 2024-09-10T04:18:33.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:33 smithi070 ceph-mon[79774]: pgmap v386: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:33.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:33 smithi046 ceph-mon[109506]: pgmap v386: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:33.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.619+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f86d0198230 con 0x7f86b807ac50 2024-09-10T04:18:33.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.621+0000 7f86ccff9640 1 -- 172.21.15.46:0/3256719956 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f86d0198230 con 0x7f86b807ac50 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f86b807ac50 msgr2=0x7f86b807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f86b807ac50 0x7f86b807d110 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f86d0108900 tx=0x7f86c4005ec0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 msgr2=0x7f86d019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d019d640 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f86c0031460 tx=0x7f86c00048f0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 shutdown_connections 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f86b807ac50 0x7f86b807d110 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d01033e0 0x7f86d019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 --2- 172.21.15.46:0/3256719956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86d0102ac0 0x7f86d019d640 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 >> 172.21.15.46:0/3256719956 conn(0x7f86d00fc520 msgr2=0x7f86d010af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 shutdown_connections 2024-09-10T04:18:33.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.623+0000 7f86d60ae640 1 -- 172.21.15.46:0/3256719956 wait complete. 2024-09-10T04:18:33.634 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:18:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 -- 172.21.15.46:0/2696021686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c103480 msgr2=0x7f224c10b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2696021686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c103480 0x7f224c10b970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2240009880 tx=0x7f224002a530 comp rx=0 tx=0).stop 2024-09-10T04:18:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 -- 172.21.15.46:0/2696021686 shutdown_connections 2024-09-10T04:18:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2696021686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c103480 0x7f224c10b970 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2696021686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c102b60 0x7f224c102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 -- 172.21.15.46:0/2696021686 >> 172.21.15.46:0/2696021686 conn(0x7f224c0fc520 msgr2=0x7f224c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 -- 172.21.15.46:0/2696021686 shutdown_connections 2024-09-10T04:18:33.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.708+0000 7f2251cd9640 1 -- 172.21.15.46:0/2696021686 wait complete. 2024-09-10T04:18:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f2251cd9640 1 Processor -- start 2024-09-10T04:18:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f2251cd9640 1 -- start start 2024-09-10T04:18:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f2251cd9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c102b60 0x7f224c19d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:33.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f2251cd9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c103480 0x7f224c19dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f2251cd9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10edf0 con 0x7f224c102b60 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f2251cd9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f224c10ec70 con 0x7f224c103480 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f224b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c102b60 0x7f224c19d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f224b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c102b60 0x7f224c19d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37024/0 (socket says 172.21.15.46:37024) 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f224b7fe640 1 -- 172.21.15.46:0/2970331007 learned_addr learned my addr 172.21.15.46:0/2970331007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.709+0000 7f224affd640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c103480 0x7f224c19dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f224b7fe640 1 -- 172.21.15.46:0/2970331007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c103480 msgr2=0x7f224c19dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f224b7fe640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c103480 0x7f224c19dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f224b7fe640 1 -- 172.21.15.46:0/2970331007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c1978a0 con 0x7f224c102b60 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f224b7fe640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c102b60 0x7f224c19d6e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f223c00d6f0 tx=0x7f223c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f2248ff9640 1 -- 172.21.15.46:0/2970331007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f223c017070 con 0x7f224c102b60 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c197b90 con 0x7f224c102b60 2024-09-10T04:18:33.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f2248ff9640 1 -- 172.21.15.46:0/2970331007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f223c00de00 con 0x7f224c102b60 2024-09-10T04:18:33.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f2248ff9640 1 -- 172.21.15.46:0/2970331007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f223c004bb0 con 0x7f224c102b60 2024-09-10T04:18:33.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.710+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c1980f0 con 0x7f224c102b60 2024-09-10T04:18:33.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.711+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c06a9f0 con 0x7f224c102b60 2024-09-10T04:18:33.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.712+0000 7f2248ff9640 1 -- 172.21.15.46:0/2970331007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f223c009a30 con 0x7f224c102b60 2024-09-10T04:18:33.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.712+0000 7f2248ff9640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f223407ac40 0x7f223407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.713+0000 7f224affd640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f223407ac40 0x7f223407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.713+0000 7f2248ff9640 1 -- 172.21.15.46:0/2970331007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f223c09d740 con 0x7f224c102b60 2024-09-10T04:18:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.713+0000 7f224affd640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f223407ac40 0x7f223407d100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f224c199090 tx=0x7f2240004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.717+0000 7f2248ff9640 1 -- 172.21.15.46:0/2970331007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f223c068f00 con 0x7f224c102b60 2024-09-10T04:18:33.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.921+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f224c198dd0 con 0x7f223407ac40 2024-09-10T04:18:33.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.922+0000 7f2248ff9640 1 -- 172.21.15.46:0/2970331007 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f224c198dd0 con 0x7f223407ac40 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f223407ac40 msgr2=0x7f223407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f223407ac40 0x7f223407d100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f224c199090 tx=0x7f2240004f30 comp rx=0 tx=0).stop 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c102b60 msgr2=0x7f224c19d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c102b60 0x7f224c19d6e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f223c00d6f0 tx=0x7f223c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 shutdown_connections 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f223407ac40 0x7f223407d100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f224c103480 0x7f224c19dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 --2- 172.21.15.46:0/2970331007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f224c102b60 0x7f224c19d6e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 >> 172.21.15.46:0/2970331007 conn(0x7f224c0fc520 msgr2=0x7f224c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 shutdown_connections 2024-09-10T04:18:33.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:33.925+0000 7f2251cd9640 1 -- 172.21.15.46:0/2970331007 wait complete. 2024-09-10T04:18:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 -- 172.21.15.46:0/3946284157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 msgr2=0x7f0720106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 --2- 172.21.15.46:0/3946284157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f0720106be0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f071c009880 tx=0x7f071c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 -- 172.21.15.46:0/3946284157 shutdown_connections 2024-09-10T04:18:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 --2- 172.21.15.46:0/3946284157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0720100800 0x7f0720100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 --2- 172.21.15.46:0/3946284157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f0720106be0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 -- 172.21.15.46:0/3946284157 >> 172.21.15.46:0/3946284157 conn(0x7f07200fc520 msgr2=0x7f07200fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 -- 172.21.15.46:0/3946284157 shutdown_connections 2024-09-10T04:18:34.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.009+0000 7f0727d10640 1 -- 172.21.15.46:0/3946284157 wait complete. 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f0727d10640 1 Processor -- start 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f0727d10640 1 -- start start 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f0727d10640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0720100800 0x7f072019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f0727d10640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f072019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f0727d10640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f072010dcb0 con 0x7f0720106800 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f0727d10640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f072010db30 con 0x7f0720100800 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f07252ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f072019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f07252ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f072019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37032/0 (socket says 172.21.15.46:37032) 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.010+0000 7f07252ab640 1 -- 172.21.15.46:0/1361295983 learned_addr learned my addr 172.21.15.46:0/1361295983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f0725aac640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0720100800 0x7f072019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f07252ab640 1 -- 172.21.15.46:0/1361295983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0720100800 msgr2=0x7f072019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f07252ab640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0720100800 0x7f072019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f07252ab640 1 -- 172.21.15.46:0/1361295983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0720197840 con 0x7f0720106800 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f07252ab640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f072019dc00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f071000d8d0 tx=0x7f071000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f0716ffd640 1 -- 172.21.15.46:0/1361295983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0710017030 con 0x7f0720106800 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0720197b30 con 0x7f0720106800 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f0716ffd640 1 -- 172.21.15.46:0/1361295983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0710004020 con 0x7f0720106800 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f0716ffd640 1 -- 172.21.15.46:0/1361295983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f07100047a0 con 0x7f0720106800 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.011+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07201980c0 con 0x7f0720106800 2024-09-10T04:18:34.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.012+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0720101f40 con 0x7f0720106800 2024-09-10T04:18:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.013+0000 7f0716ffd640 1 -- 172.21.15.46:0/1361295983 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f07100041d0 con 0x7f0720106800 2024-09-10T04:18:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.014+0000 7f0716ffd640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f070c07abf0 0x7f070c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.014+0000 7f0725aac640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f070c07abf0 0x7f070c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.014+0000 7f0716ffd640 1 -- 172.21.15.46:0/1361295983 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0710018070 con 0x7f0720106800 2024-09-10T04:18:34.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.014+0000 7f0725aac640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f070c07abf0 0x7f070c07d0b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f071c0096c0 tx=0x7f071c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.018+0000 7f0716ffd640 1 -- 172.21.15.46:0/1361295983 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0710069110 con 0x7f0720106800 2024-09-10T04:18:34.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.212+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f07201983d0 con 0x7f070c07abf0 2024-09-10T04:18:34.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.223+0000 7f0716ffd640 1 -- 172.21.15.46:0/1361295983 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f07201983d0 con 0x7f070c07abf0 2024-09-10T04:18:34.224 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:18:34.224 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (10m) 4m ago 21m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (12m) 4m ago 21m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (12m) 4m ago 20m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (8m) 4m ago 21m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (8m) 4m ago 20m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (9m) 4m ago 21m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (4m) 4m ago 16m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (4m) 4m ago 16m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (4m) 4m ago 16m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (16m) 4m ago 16m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (12m) 4m ago 22m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (11m) 4m ago 20m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (9m) 4m ago 23m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (8m) 4m ago 20m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:18:34.225 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (10m) 4m ago 21m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (10m) 4m ago 20m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 4m ago 19m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (7m) 4m ago 18m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (6m) 4m ago 18m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (6m) 4m ago 18m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (5m) 4m ago 17m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (5m) 4m ago 17m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:18:34.226 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (10m) 4m ago 21m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f070c07abf0 msgr2=0x7f070c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f070c07abf0 0x7f070c07d0b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f071c0096c0 tx=0x7f071c031400 comp rx=0 tx=0).stop 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 msgr2=0x7f072019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f072019dc00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f071000d8d0 tx=0x7f071000dd90 comp rx=0 tx=0).stop 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 shutdown_connections 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f070c07abf0 0x7f070c07d0b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0720106800 0x7f072019dc00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 --2- 172.21.15.46:0/1361295983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0720100800 0x7f072019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 >> 172.21.15.46:0/1361295983 conn(0x7f07200fc520 msgr2=0x7f072006a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 shutdown_connections 2024-09-10T04:18:34.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.225+0000 7f0727d10640 1 -- 172.21.15.46:0/1361295983 wait complete. 2024-09-10T04:18:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 -- 172.21.15.46:0/522598392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 msgr2=0x7ff344102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/522598392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344102ea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff340009880 tx=0x7ff34002a530 comp rx=0 tx=0).stop 2024-09-10T04:18:34.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 -- 172.21.15.46:0/522598392 shutdown_connections 2024-09-10T04:18:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/522598392 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3441033e0 0x7ff34410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/522598392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344102ea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 -- 172.21.15.46:0/522598392 >> 172.21.15.46:0/522598392 conn(0x7ff3440fc520 msgr2=0x7ff3440fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 -- 172.21.15.46:0/522598392 shutdown_connections 2024-09-10T04:18:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.308+0000 7ff34c7b6640 1 -- 172.21.15.46:0/522598392 wait complete. 2024-09-10T04:18:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34c7b6640 1 Processor -- start 2024-09-10T04:18:34.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34c7b6640 1 -- start start 2024-09-10T04:18:34.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34c7b6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34c7b6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3441033e0 0x7ff344100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34c7b6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff34410edf0 con 0x7ff344102ac0 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34c7b6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff34410ec70 con 0x7ff3441033e0 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34a552640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34a552640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37058/0 (socket says 172.21.15.46:37058) 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34a552640 1 -- 172.21.15.46:0/3457794321 learned_addr learned my addr 172.21.15.46:0/3457794321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff349d51640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3441033e0 0x7ff344100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34a552640 1 -- 172.21.15.46:0/3457794321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3441033e0 msgr2=0x7ff344100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34a552640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3441033e0 0x7ff344100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.309+0000 7ff34a552640 1 -- 172.21.15.46:0/3457794321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3441011a0 con 0x7ff344102ac0 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.310+0000 7ff34a552640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344100720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff340031460 tx=0x7ff340004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.310+0000 7ff33b7fe640 1 -- 172.21.15.46:0/3457794321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff34003f070 con 0x7ff344102ac0 2024-09-10T04:18:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.310+0000 7ff33b7fe640 1 -- 172.21.15.46:0/3457794321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff3400035f0 con 0x7ff344102ac0 2024-09-10T04:18:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.310+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff344101430 con 0x7ff344102ac0 2024-09-10T04:18:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.310+0000 7ff33b7fe640 1 -- 172.21.15.46:0/3457794321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff340035030 con 0x7ff344102ac0 2024-09-10T04:18:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.310+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3441a3da0 con 0x7ff344102ac0 2024-09-10T04:18:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.311+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34406aa90 con 0x7ff344102ac0 2024-09-10T04:18:34.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.312+0000 7ff33b7fe640 1 -- 172.21.15.46:0/3457794321 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff340042070 con 0x7ff344102ac0 2024-09-10T04:18:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.312+0000 7ff33b7fe640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33007ac40 0x7ff33007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.312+0000 7ff33b7fe640 1 -- 172.21.15.46:0/3457794321 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ff3400be4a0 con 0x7ff344102ac0 2024-09-10T04:18:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.312+0000 7ff349d51640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33007ac40 0x7ff33007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.313+0000 7ff349d51640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33007ac40 0x7ff33007d100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff3441021b0 tx=0x7ff334005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.317+0000 7ff33b7fe640 1 -- 172.21.15.46:0/3457794321 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff340089c60 con 0x7ff344102ac0 2024-09-10T04:18:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.578+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff344101c50 con 0x7ff344102ac0 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.579+0000 7ff33b7fe640 1 -- 172.21.15.46:0/3457794321 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7ff34008eb10 con 0x7ff344102ac0 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:18:34.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:18:34.582 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33007ac40 msgr2=0x7ff33007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33007ac40 0x7ff33007d100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff3441021b0 tx=0x7ff334005f90 comp rx=0 tx=0).stop 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 msgr2=0x7ff344100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344100720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff340031460 tx=0x7ff340004530 comp rx=0 tx=0).stop 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 shutdown_connections 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33007ac40 0x7ff33007d100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3441033e0 0x7ff344100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 --2- 172.21.15.46:0/3457794321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff344102ac0 0x7ff344100720 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 >> 172.21.15.46:0/3457794321 conn(0x7ff3440fc520 msgr2=0x7ff344108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 shutdown_connections 2024-09-10T04:18:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.581+0000 7ff34c7b6640 1 -- 172.21.15.46:0/3457794321 wait complete. 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 -- 172.21.15.46:0/3852798970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d041033e0 msgr2=0x7f7d0410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 --2- 172.21.15.46:0/3852798970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d041033e0 0x7f7d0410b8d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009880 tx=0x7f7cf802a530 comp rx=0 tx=0).stop 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 -- 172.21.15.46:0/3852798970 shutdown_connections 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 --2- 172.21.15.46:0/3852798970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d041033e0 0x7f7d0410b8d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 --2- 172.21.15.46:0/3852798970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d04102ac0 0x7f7d04102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 -- 172.21.15.46:0/3852798970 >> 172.21.15.46:0/3852798970 conn(0x7f7d040fc520 msgr2=0x7f7d040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 -- 172.21.15.46:0/3852798970 shutdown_connections 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.666+0000 7f7d0a725640 1 -- 172.21.15.46:0/3852798970 wait complete. 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.667+0000 7f7d0a725640 1 Processor -- start 2024-09-10T04:18:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.667+0000 7f7d0a725640 1 -- start start 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.667+0000 7f7d0a725640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.667+0000 7f7d0a725640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d041033e0 0x7f7d0419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.667+0000 7f7d0a725640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410edf0 con 0x7f7d04102ac0 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.667+0000 7f7d0a725640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0410ec70 con 0x7f7d041033e0 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37068/0 (socket says 172.21.15.46:37068) 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d03fff640 1 -- 172.21.15.46:0/1745316213 learned_addr learned my addr 172.21.15.46:0/1745316213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d037fe640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d041033e0 0x7f7d0419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d03fff640 1 -- 172.21.15.46:0/1745316213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d041033e0 msgr2=0x7f7d0419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d03fff640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d041033e0 0x7f7d0419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d03fff640 1 -- 172.21.15.46:0/1745316213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041977e0 con 0x7f7d04102ac0 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d03fff640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400d6f0 tx=0x7f7cf400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d017fa640 1 -- 172.21.15.46:0/1745316213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cf4017070 con 0x7f7d04102ac0 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d017fa640 1 -- 172.21.15.46:0/1745316213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cf400de00 con 0x7f7d04102ac0 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d04197ad0 con 0x7f7d04102ac0 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d017fa640 1 -- 172.21.15.46:0/1745316213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cf4004bb0 con 0x7f7d04102ac0 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.668+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d04198030 con 0x7f7d04102ac0 2024-09-10T04:18:34.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.669+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0406aa90 con 0x7f7d04102ac0 2024-09-10T04:18:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.670+0000 7f7d017fa640 1 -- 172.21.15.46:0/1745316213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7cf4009a30 con 0x7f7d04102ac0 2024-09-10T04:18:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.671+0000 7f7d017fa640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cec07ac40 0x7f7cec07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.671+0000 7f7d037fe640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cec07ac40 0x7f7cec07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.671+0000 7f7d017fa640 1 -- 172.21.15.46:0/1745316213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f7cf409d740 con 0x7f7d04102ac0 2024-09-10T04:18:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.671+0000 7f7d037fe640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cec07ac40 0x7f7cec07d100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d04198b30 tx=0x7f7cf8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.675+0000 7f7d017fa640 1 -- 172.21.15.46:0/1745316213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7cf4068f00 con 0x7f7d04102ac0 2024-09-10T04:18:34.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.898+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7d04105f60 con 0x7f7d04102ac0 2024-09-10T04:18:34.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.898+0000 7f7d017fa640 1 -- 172.21.15.46:0/1745316213 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f7cf406ddb0 con 0x7f7d04102ac0 2024-09-10T04:18:34.900 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:18:34.900 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:18:34.900 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:18:34.901 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:18:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:34.903 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:34.904 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:18:34.904 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:18:34.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cec07ac40 msgr2=0x7f7cec07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cec07ac40 0x7f7cec07d100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d04198b30 tx=0x7f7cf8031400 comp rx=0 tx=0).stop 2024-09-10T04:18:34.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04102ac0 msgr2=0x7f7d0419d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400d6f0 tx=0x7f7cf400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 shutdown_connections 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cec07ac40 0x7f7cec07d100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d041033e0 0x7f7d0419dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 --2- 172.21.15.46:0/1745316213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d04102ac0 0x7f7d0419d660 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.900+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 >> 172.21.15.46:0/1745316213 conn(0x7f7d040fc520 msgr2=0x7f7d04108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.901+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 shutdown_connections 2024-09-10T04:18:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.901+0000 7f7d0a725640 1 -- 172.21.15.46:0/1745316213 wait complete. 2024-09-10T04:18:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.985+0000 7f40cd559640 1 -- 172.21.15.46:0/1429399450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8103470 msgr2=0x7f40c810b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.985+0000 7f40cd559640 1 --2- 172.21.15.46:0/1429399450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8103470 0x7f40c810b960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009880 tx=0x7f40bc02a530 comp rx=0 tx=0).stop 2024-09-10T04:18:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.986+0000 7f40cd559640 1 -- 172.21.15.46:0/1429399450 shutdown_connections 2024-09-10T04:18:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.986+0000 7f40cd559640 1 --2- 172.21.15.46:0/1429399450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8103470 0x7f40c810b960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.986+0000 7f40cd559640 1 --2- 172.21.15.46:0/1429399450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c8102b50 0x7f40c8102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.986+0000 7f40cd559640 1 -- 172.21.15.46:0/1429399450 >> 172.21.15.46:0/1429399450 conn(0x7f40c80fc550 msgr2=0x7f40c80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.986+0000 7f40cd559640 1 -- 172.21.15.46:0/1429399450 shutdown_connections 2024-09-10T04:18:34.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.986+0000 7f40cd559640 1 -- 172.21.15.46:0/1429399450 wait complete. 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.986+0000 7f40cd559640 1 Processor -- start 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40cd559640 1 -- start start 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40cd559640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8102b50 0x7f40c81007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40cd559640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c8103470 0x7f40c8100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40cd559640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40c810edc0 con 0x7f40c8102b50 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40cd559640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f40c810ec40 con 0x7f40c8103470 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8102b50 0x7f40c81007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8102b50 0x7f40c81007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37088/0 (socket says 172.21.15.46:37088) 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c6ffd640 1 -- 172.21.15.46:0/3641782919 learned_addr learned my addr 172.21.15.46:0/3641782919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c67fc640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c8103470 0x7f40c8100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c6ffd640 1 -- 172.21.15.46:0/3641782919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c8103470 msgr2=0x7f40c8100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c6ffd640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c8103470 0x7f40c8100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c6ffd640 1 -- 172.21.15.46:0/3641782919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c8101230 con 0x7f40c8102b50 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c67fc640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c8103470 0x7f40c8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.987+0000 7f40c6ffd640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8102b50 0x7f40c81007b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f40b800d8d0 tx=0x7f40b800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.988+0000 7f40affff640 1 -- 172.21.15.46:0/3641782919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f40b8017070 con 0x7f40c8102b50 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.988+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c8101520 con 0x7f40c8102b50 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.988+0000 7f40affff640 1 -- 172.21.15.46:0/3641782919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f40b8010460 con 0x7f40c8102b50 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.988+0000 7f40affff640 1 -- 172.21.15.46:0/3641782919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f40b800f5f0 con 0x7f40c8102b50 2024-09-10T04:18:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.988+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c81a3e10 con 0x7f40c8102b50 2024-09-10T04:18:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.989+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c806a4e0 con 0x7f40c8102b50 2024-09-10T04:18:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.989+0000 7f40affff640 1 -- 172.21.15.46:0/3641782919 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f40b8009a30 con 0x7f40c8102b50 2024-09-10T04:18:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.990+0000 7f40affff640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f40b007ac00 0x7f40b007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.990+0000 7f40c67fc640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f40b007ac00 0x7f40b007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.990+0000 7f40affff640 1 -- 172.21.15.46:0/3641782919 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f40b809db40 con 0x7f40c8102b50 2024-09-10T04:18:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.991+0000 7f40c67fc640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f40b007ac00 0x7f40b007d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40bc004f90 tx=0x7f40bc0041a0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:34.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:34.994+0000 7f40affff640 1 -- 172.21.15.46:0/3641782919 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f40b8069280 con 0x7f40c8102b50 2024-09-10T04:18:35.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.197+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f40c8062630 con 0x7f40b007ac00 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.198+0000 7f40affff640 1 -- 172.21.15.46:0/3641782919 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f40c8062630 con 0x7f40b007ac00 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:18:35.200 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:18:35.201 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:18:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f40b007ac00 msgr2=0x7f40b007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f40b007ac00 0x7f40b007d0c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40bc004f90 tx=0x7f40bc0041a0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8102b50 msgr2=0x7f40c81007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8102b50 0x7f40c81007b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f40b800d8d0 tx=0x7f40b800dd90 comp rx=0 tx=0).stop 2024-09-10T04:18:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 shutdown_connections 2024-09-10T04:18:35.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f40b007ac00 0x7f40b007d0c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c8103470 0x7f40c8100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 --2- 172.21.15.46:0/3641782919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c8102b50 0x7f40c81007b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.200+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 >> 172.21.15.46:0/3641782919 conn(0x7f40c80fc550 msgr2=0x7f40c8108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.201+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 shutdown_connections 2024-09-10T04:18:35.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.201+0000 7f40cd559640 1 -- 172.21.15.46:0/3641782919 wait complete. 2024-09-10T04:18:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 -- 172.21.15.46:0/2448539455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 msgr2=0x7f0ff8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/2448539455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff8101d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009880 tx=0x7f0fec02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 -- 172.21.15.46:0/2448539455 shutdown_connections 2024-09-10T04:18:35.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/2448539455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff8101d60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/2448539455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ff8107900 0x7f0ff8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 -- 172.21.15.46:0/2448539455 >> 172.21.15.46:0/2448539455 conn(0x7f0ff80fd620 msgr2=0x7f0ff80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 -- 172.21.15.46:0/2448539455 shutdown_connections 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.285+0000 7f0ffcc45640 1 -- 172.21.15.46:0/2448539455 wait complete. 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ffcc45640 1 Processor -- start 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ffcc45640 1 -- start start 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ffcc45640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ffcc45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ff8107900 0x7f0ff8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ffcc45640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff810edb0 con 0x7f0ff8101900 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ffcc45640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff810ec30 con 0x7f0ff8107900 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ff659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ff659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37120/0 (socket says 172.21.15.46:37120) 2024-09-10T04:18:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.286+0000 7f0ff659c640 1 -- 172.21.15.46:0/3678297844 learned_addr learned my addr 172.21.15.46:0/3678297844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0ff5d9b640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ff8107900 0x7f0ff8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0ff659c640 1 -- 172.21.15.46:0/3678297844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ff8107900 msgr2=0x7f0ff8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0ff659c640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ff8107900 0x7f0ff8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0ff659c640 1 -- 172.21.15.46:0/3678297844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff8074f70 con 0x7f0ff8101900 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0ff659c640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff80744f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800d8d0 tx=0x7f0fe800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0fe77fe640 1 -- 172.21.15.46:0/3678297844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0fe8017070 con 0x7f0ff8101900 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0fe77fe640 1 -- 172.21.15.46:0/3678297844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0fe8010460 con 0x7f0ff8101900 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff8075260 con 0x7f0ff8101900 2024-09-10T04:18:35.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0fe77fe640 1 -- 172.21.15.46:0/3678297844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0fe800f5f0 con 0x7f0ff8101900 2024-09-10T04:18:35.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.287+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff81a3e20 con 0x7f0ff8101900 2024-09-10T04:18:35.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.288+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff8103040 con 0x7f0ff8101900 2024-09-10T04:18:35.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.289+0000 7f0fe77fe640 1 -- 172.21.15.46:0/3678297844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0fe8009a30 con 0x7f0ff8101900 2024-09-10T04:18:35.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.289+0000 7f0fe77fe640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe007ac00 0x7f0fe007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:18:35.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.290+0000 7f0fe77fe640 1 -- 172.21.15.46:0/3678297844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0fe809da90 con 0x7f0ff8101900 2024-09-10T04:18:35.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.290+0000 7f0ff5d9b640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe007ac00 0x7f0fe007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:18:35.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.290+0000 7f0ff5d9b640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe007ac00 0x7f0fe007d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8075f80 tx=0x7f0fec031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:18:35.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.294+0000 7f0fe77fe640 1 -- 172.21.15.46:0/3678297844 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0fe80691d0 con 0x7f0ff8101900 2024-09-10T04:18:35.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.557+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f0ff8101d60 con 0x7f0ff8101900 2024-09-10T04:18:35.559 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:35 smithi046 ceph-mon[109506]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:35.559 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:35 smithi046 ceph-mon[109506]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:35.560 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:35 smithi046 ceph-mon[109506]: pgmap v387: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:35.560 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:35 smithi046 ceph-mon[109506]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:35.560 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:35 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3457794321' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:18:35.560 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:35 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1745316213' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:18:35.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.558+0000 7f0fe77fe640 1 -- 172.21.15.46:0/3678297844 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f0fe806e080 con 0x7f0ff8101900 2024-09-10T04:18:35.560 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:18:35.561 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:18:35.561 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:18:35.561 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:18:35.561 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:18:35.561 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:18:35.561 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:18:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe007ac00 msgr2=0x7f0fe007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe007ac00 0x7f0fe007d0c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8075f80 tx=0x7f0fec031400 comp rx=0 tx=0).stop 2024-09-10T04:18:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 msgr2=0x7f0ff80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:18:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff80744f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800d8d0 tx=0x7f0fe800dd90 comp rx=0 tx=0).stop 2024-09-10T04:18:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 shutdown_connections 2024-09-10T04:18:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe007ac00 0x7f0fe007d0c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ff8107900 0x7f0ff8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 --2- 172.21.15.46:0/3678297844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff8101900 0x7f0ff80744f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:18:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 >> 172.21.15.46:0/3678297844 conn(0x7f0ff80fd620 msgr2=0x7f0ff80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:18:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 shutdown_connections 2024-09-10T04:18:35.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:18:35.560+0000 7f0ffcc45640 1 -- 172.21.15.46:0/3678297844 wait complete. 2024-09-10T04:18:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:35 smithi070 ceph-mon[79774]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:35 smithi070 ceph-mon[79774]: from='client.34594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:35 smithi070 ceph-mon[79774]: pgmap v387: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:35 smithi070 ceph-mon[79774]: from='client.34598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:35.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:35 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3457794321' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:18:35.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:35 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1745316213' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:18:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:36 smithi070 ceph-mon[79774]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:36.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:36 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3678297844' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:18:36.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:36 smithi046 ceph-mon[109506]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:18:36.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:36 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3678297844' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:18:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:37 smithi070 ceph-mon[79774]: pgmap v388: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:37.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:37 smithi046 ceph-mon[109506]: pgmap v388: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:38.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:39.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:39 smithi070 ceph-mon[79774]: pgmap v389: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:39 smithi046 ceph-mon[109506]: pgmap v389: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:41 smithi046 ceph-mon[109506]: pgmap v390: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:41 smithi070 ceph-mon[79774]: pgmap v390: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:43 smithi070 ceph-mon[79774]: pgmap v391: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:43 smithi046 ceph-mon[109506]: pgmap v391: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:45 smithi070 ceph-mon[79774]: pgmap v392: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:45 smithi046 ceph-mon[109506]: pgmap v392: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:47.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:47 smithi046 ceph-mon[109506]: pgmap v393: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:47.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:47 smithi070 ceph-mon[79774]: pgmap v393: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:49.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:49 smithi070 ceph-mon[79774]: pgmap v394: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:49.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:49 smithi046 ceph-mon[109506]: pgmap v394: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:51.653 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:51 smithi046 ceph-mon[109506]: pgmap v395: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:51 smithi070 ceph-mon[79774]: pgmap v395: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:53 smithi070 ceph-mon[79774]: pgmap v396: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:53.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:53 smithi046 ceph-mon[109506]: pgmap v396: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:18:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:55 smithi070 ceph-mon[79774]: pgmap v397: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:55 smithi046 ceph-mon[109506]: pgmap v397: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:57 smithi070 ceph-mon[79774]: pgmap v398: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:57 smithi046 ceph-mon[109506]: pgmap v398: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:18:59 smithi046 ceph-mon[109506]: pgmap v399: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:18:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:18:59 smithi070 ceph-mon[79774]: pgmap v399: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:01.807 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:01 smithi046 ceph-mon[109506]: pgmap v400: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:01 smithi070 ceph-mon[79774]: pgmap v400: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:03 smithi070 ceph-mon[79774]: pgmap v401: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:03 smithi046 ceph-mon[109506]: pgmap v401: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:05.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:05 smithi046 ceph-mon[109506]: pgmap v402: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:05.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 -- 172.21.15.46:0/1432912689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 msgr2=0x7f0ffc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:05.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1432912689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc100c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009880 tx=0x7f0fec02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 -- 172.21.15.46:0/1432912689 shutdown_connections 2024-09-10T04:19:05.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1432912689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc100c50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1432912689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc1067f0 0x7f0ffc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 -- 172.21.15.46:0/1432912689 >> 172.21.15.46:0/1432912689 conn(0x7f0ffc0fc550 msgr2=0x7f0ffc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:05.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 -- 172.21.15.46:0/1432912689 shutdown_connections 2024-09-10T04:19:05.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.658+0000 7f1003aaf640 1 -- 172.21.15.46:0/1432912689 wait complete. 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.659+0000 7f1003aaf640 1 Processor -- start 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.659+0000 7f1003aaf640 1 -- start start 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f1003aaf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f1003aaf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc1067f0 0x7f0ffc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f1003aaf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10dca0 con 0x7f0ffc1007f0 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f1003aaf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0ffc10db20 con 0x7f0ffc1067f0 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100184b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100184b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60062/0 (socket says 172.21.15.46:60062) 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100184b640 1 -- 172.21.15.46:0/1059017012 learned_addr learned my addr 172.21.15.46:0/1059017012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100104a640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc1067f0 0x7f0ffc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:05.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100184b640 1 -- 172.21.15.46:0/1059017012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc1067f0 msgr2=0x7f0ffc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100184b640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc1067f0 0x7f0ffc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100184b640 1 -- 172.21.15.46:0/1059017012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ffc0fd700 con 0x7f0ffc1007f0 2024-09-10T04:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f100184b640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc0fcc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800d8d0 tx=0x7f0ff800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.660+0000 7f0ff2ffd640 1 -- 172.21.15.46:0/1059017012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ff8017070 con 0x7f0ffc1007f0 2024-09-10T04:19:05.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.661+0000 7f0ff2ffd640 1 -- 172.21.15.46:0/1059017012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ff8010460 con 0x7f0ffc1007f0 2024-09-10T04:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.661+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ffc0fd9f0 con 0x7f0ffc1007f0 2024-09-10T04:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.661+0000 7f0ff2ffd640 1 -- 172.21.15.46:0/1059017012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ff800f5f0 con 0x7f0ffc1007f0 2024-09-10T04:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.661+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ffc1a3e00 con 0x7f0ffc1007f0 2024-09-10T04:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.662+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ffc101f30 con 0x7f0ffc1007f0 2024-09-10T04:19:05.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.662+0000 7f0ff2ffd640 1 -- 172.21.15.46:0/1059017012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0ff8009a30 con 0x7f0ffc1007f0 2024-09-10T04:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.663+0000 7f0ff2ffd640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe807abf0 0x7f0fe807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.663+0000 7f0ff2ffd640 1 -- 172.21.15.46:0/1059017012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0ff809da90 con 0x7f0ffc1007f0 2024-09-10T04:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.663+0000 7f100104a640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe807abf0 0x7f0fe807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:05.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.664+0000 7f100104a640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe807abf0 0x7f0fe807d0b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0fe710 tx=0x7f0fec031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:05.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.667+0000 7f0ff2ffd640 1 -- 172.21.15.46:0/1059017012 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0ff80691d0 con 0x7f0ffc1007f0 2024-09-10T04:19:05.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:05 smithi070 ceph-mon[79774]: pgmap v402: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:05.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.874+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0ffc062630 con 0x7f0fe807abf0 2024-09-10T04:19:05.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.875+0000 7f0ff2ffd640 1 -- 172.21.15.46:0/1059017012 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0ffc062630 con 0x7f0fe807abf0 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe807abf0 msgr2=0x7f0fe807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe807abf0 0x7f0fe807d0b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0fe710 tx=0x7f0fec031400 comp rx=0 tx=0).stop 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 msgr2=0x7f0ffc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc0fcc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800d8d0 tx=0x7f0ff800dd90 comp rx=0 tx=0).stop 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 shutdown_connections 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fe807abf0 0x7f0fe807d0b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ffc1067f0 0x7f0ffc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 --2- 172.21.15.46:0/1059017012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ffc1007f0 0x7f0ffc0fcc80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 >> 172.21.15.46:0/1059017012 conn(0x7f0ffc0fc550 msgr2=0x7f0ffc069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 shutdown_connections 2024-09-10T04:19:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.878+0000 7f1003aaf640 1 -- 172.21.15.46:0/1059017012 wait complete. 2024-09-10T04:19:05.889 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:19:05.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 -- 172.21.15.46:0/3078199376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981067f0 msgr2=0x7f1898106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:05.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 --2- 172.21.15.46:0/3078199376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981067f0 0x7f1898106bd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1888009880 tx=0x7f188802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 -- 172.21.15.46:0/3078199376 shutdown_connections 2024-09-10T04:19:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 --2- 172.21.15.46:0/3078199376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18981007f0 0x7f1898100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 --2- 172.21.15.46:0/3078199376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981067f0 0x7f1898106bd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 -- 172.21.15.46:0/3078199376 >> 172.21.15.46:0/3078199376 conn(0x7f18980fc550 msgr2=0x7f18980fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 -- 172.21.15.46:0/3078199376 shutdown_connections 2024-09-10T04:19:05.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 -- 172.21.15.46:0/3078199376 wait complete. 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.964+0000 7f189e3d1640 1 Processor -- start 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f189e3d1640 1 -- start start 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f189e3d1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981007f0 0x7f18981987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f189e3d1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18981067f0 0x7f1898198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f189e3d1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f189810dcc0 con 0x7f18981007f0 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f189e3d1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f189810db40 con 0x7f18981067f0 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f1897fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981007f0 0x7f18981987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f1897fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981007f0 0x7f18981987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60092/0 (socket says 172.21.15.46:60092) 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f1897fff640 1 -- 172.21.15.46:0/34819879 learned_addr learned my addr 172.21.15.46:0/34819879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f18977fe640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18981067f0 0x7f1898198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f1897fff640 1 -- 172.21.15.46:0/34819879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18981067f0 msgr2=0x7f1898198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:05.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f1897fff640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18981067f0 0x7f1898198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.965+0000 7f1897fff640 1 -- 172.21.15.46:0/34819879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1898199360 con 0x7f18981007f0 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.966+0000 7f1897fff640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981007f0 0x7f18981987d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1888031460 tx=0x7f18880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.966+0000 7f18957fa640 1 -- 172.21.15.46:0/34819879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f188803f070 con 0x7f18981007f0 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.966+0000 7f18957fa640 1 -- 172.21.15.46:0/34819879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1888004cf0 con 0x7f18981007f0 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.966+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18981995f0 con 0x7f18981007f0 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.966+0000 7f18957fa640 1 -- 172.21.15.46:0/34819879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1888035030 con 0x7f18981007f0 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.966+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18981a1ba0 con 0x7f18981007f0 2024-09-10T04:19:05.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.967+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1898101f30 con 0x7f18981007f0 2024-09-10T04:19:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.968+0000 7f18957fa640 1 -- 172.21.15.46:0/34819879 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1888042070 con 0x7f18981007f0 2024-09-10T04:19:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.969+0000 7f18957fa640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f188007ac40 0x7f188007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.969+0000 7f18977fe640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f188007ac40 0x7f188007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:05.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.969+0000 7f18957fa640 1 -- 172.21.15.46:0/34819879 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f18880be6b0 con 0x7f18981007f0 2024-09-10T04:19:05.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.969+0000 7f18977fe640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f188007ac40 0x7f188007d100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f188c005fd0 tx=0x7f188c005e20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:05.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:05.973+0000 7f18957fa640 1 -- 172.21.15.46:0/34819879 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1888089e70 con 0x7f18981007f0 2024-09-10T04:19:06.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.176+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1898062630 con 0x7f188007ac40 2024-09-10T04:19:06.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.177+0000 7f18957fa640 1 -- 172.21.15.46:0/34819879 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f1898062630 con 0x7f188007ac40 2024-09-10T04:19:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f188007ac40 msgr2=0x7f188007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f188007ac40 0x7f188007d100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f188c005fd0 tx=0x7f188c005e20 comp rx=0 tx=0).stop 2024-09-10T04:19:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981007f0 msgr2=0x7f18981987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981007f0 0x7f18981987d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1888031460 tx=0x7f18880048f0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 shutdown_connections 2024-09-10T04:19:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f188007ac40 0x7f188007d100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18981067f0 0x7f1898198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 --2- 172.21.15.46:0/34819879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18981007f0 0x7f18981987d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 >> 172.21.15.46:0/34819879 conn(0x7f18980fc550 msgr2=0x7f18980fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 shutdown_connections 2024-09-10T04:19:06.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.179+0000 7f189e3d1640 1 -- 172.21.15.46:0/34819879 wait complete. 2024-09-10T04:19:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 -- 172.21.15.46:0/576569229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 msgr2=0x7f30ac100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 --2- 172.21.15.46:0/576569229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac100c60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3094009880 tx=0x7f309402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 -- 172.21.15.46:0/576569229 shutdown_connections 2024-09-10T04:19:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 --2- 172.21.15.46:0/576569229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac100c60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 --2- 172.21.15.46:0/576569229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30ac106800 0x7f30ac106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 -- 172.21.15.46:0/576569229 >> 172.21.15.46:0/576569229 conn(0x7f30ac0fc520 msgr2=0x7f30ac0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:06.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 -- 172.21.15.46:0/576569229 shutdown_connections 2024-09-10T04:19:06.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.264+0000 7f30b09c5640 1 -- 172.21.15.46:0/576569229 wait complete. 2024-09-10T04:19:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.265+0000 7f30b09c5640 1 Processor -- start 2024-09-10T04:19:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.265+0000 7f30b09c5640 1 -- start start 2024-09-10T04:19:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.265+0000 7f30b09c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.265+0000 7f30b09c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30ac106800 0x7f30ac19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.265+0000 7f30b09c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac10dcd0 con 0x7f30ac100800 2024-09-10T04:19:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.265+0000 7f30b09c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac10db50 con 0x7f30ac106800 2024-09-10T04:19:06.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30aa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30aa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60104/0 (socket says 172.21.15.46:60104) 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30aa59c640 1 -- 172.21.15.46:0/3200252458 learned_addr learned my addr 172.21.15.46:0/3200252458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30a3fff640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30ac106800 0x7f30ac19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30aa59c640 1 -- 172.21.15.46:0/3200252458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30ac106800 msgr2=0x7f30ac19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30aa59c640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30ac106800 0x7f30ac19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30aa59c640 1 -- 172.21.15.46:0/3200252458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac199a00 con 0x7f30ac100800 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30aa59c640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac19f840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f309c00d6f0 tx=0x7f309c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30a37fe640 1 -- 172.21.15.46:0/3200252458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f309c017070 con 0x7f30ac100800 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30a37fe640 1 -- 172.21.15.46:0/3200252458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f309c00de00 con 0x7f30ac100800 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac199cf0 con 0x7f30ac100800 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.266+0000 7f30a37fe640 1 -- 172.21.15.46:0/3200252458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f309c004bb0 con 0x7f30ac100800 2024-09-10T04:19:06.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.267+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac19a250 con 0x7f30ac100800 2024-09-10T04:19:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.268+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ac101f40 con 0x7f30ac100800 2024-09-10T04:19:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.268+0000 7f30a37fe640 1 -- 172.21.15.46:0/3200252458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f309c004e60 con 0x7f30ac100800 2024-09-10T04:19:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.269+0000 7f30a37fe640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f309807abf0 0x7f309807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.269+0000 7f30a3fff640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f309807abf0 0x7f309807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.269+0000 7f30a37fe640 1 -- 172.21.15.46:0/3200252458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f309c09d820 con 0x7f30ac100800 2024-09-10T04:19:06.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.269+0000 7f30a3fff640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f309807abf0 0x7f309807d0b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f30ac19b1f0 tx=0x7f3094031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:06.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.273+0000 7f30a37fe640 1 -- 172.21.15.46:0/3200252458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f309c068f60 con 0x7f30ac100800 2024-09-10T04:19:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.466+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f30ac19a500 con 0x7f309807abf0 2024-09-10T04:19:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.476+0000 7f30a37fe640 1 -- 172.21.15.46:0/3200252458 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f30ac19a500 con 0x7f309807abf0 2024-09-10T04:19:06.477 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:19:06.477 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (10m) 5m ago 21m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (13m) 5m ago 22m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (13m) 4m ago 21m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (9m) 5m ago 22m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (9m) 4m ago 21m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (10m) 5m ago 21m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (5m) 5m ago 16m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (5m) 5m ago 16m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (4m) 4m ago 16m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (16m) 4m ago 16m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (12m) 5m ago 23m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (11m) 4m ago 21m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (9m) 5m ago 23m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (9m) 4m ago 21m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:19:06.478 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (11m) 5m ago 22m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (11m) 4m ago 21m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 5m ago 20m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (8m) 5m ago 19m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (7m) 5m ago 19m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (6m) 4m ago 18m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (6m) 4m ago 18m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (5m) 4m ago 17m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:19:06.479 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (11m) 5m ago 21m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:19:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f309807abf0 msgr2=0x7f309807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f309807abf0 0x7f309807d0b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f30ac19b1f0 tx=0x7f3094031400 comp rx=0 tx=0).stop 2024-09-10T04:19:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 msgr2=0x7f30ac19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac19f840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f309c00d6f0 tx=0x7f309c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 shutdown_connections 2024-09-10T04:19:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f309807abf0 0x7f309807d0b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30ac106800 0x7f30ac19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 --2- 172.21.15.46:0/3200252458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30ac100800 0x7f30ac19f840 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.478+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 >> 172.21.15.46:0/3200252458 conn(0x7f30ac0fc520 msgr2=0x7f30ac06a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.479+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 shutdown_connections 2024-09-10T04:19:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.479+0000 7f30b09c5640 1 -- 172.21.15.46:0/3200252458 wait complete. 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 -- 172.21.15.46:0/193768524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 msgr2=0x7f1280102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 --2- 172.21.15.46:0/193768524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f1280102e90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1270009880 tx=0x7f127002a530 comp rx=0 tx=0).stop 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 -- 172.21.15.46:0/193768524 shutdown_connections 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 --2- 172.21.15.46:0/193768524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12801033d0 0x7f128010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 --2- 172.21.15.46:0/193768524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f1280102e90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 -- 172.21.15.46:0/193768524 >> 172.21.15.46:0/193768524 conn(0x7f12800fc550 msgr2=0x7f12800fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 -- 172.21.15.46:0/193768524 shutdown_connections 2024-09-10T04:19:06.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.562+0000 7f12867c5640 1 -- 172.21.15.46:0/193768524 wait complete. 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f12867c5640 1 Processor -- start 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f12867c5640 1 -- start start 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f12867c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f128019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f12867c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12801033d0 0x7f128019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f12867c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f128010ede0 con 0x7f1280102ab0 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f12867c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f128010ec60 con 0x7f12801033d0 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f127ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f128019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f127ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f128019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60134/0 (socket says 172.21.15.46:60134) 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.563+0000 7f127ffff640 1 -- 172.21.15.46:0/3214923442 learned_addr learned my addr 172.21.15.46:0/3214923442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f127f7fe640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12801033d0 0x7f128019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f127ffff640 1 -- 172.21.15.46:0/3214923442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12801033d0 msgr2=0x7f128019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f127ffff640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12801033d0 0x7f128019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f127ffff640 1 -- 172.21.15.46:0/3214923442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12801977b0 con 0x7f1280102ab0 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f127f7fe640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12801033d0 0x7f128019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f127ffff640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f128019d630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1270031460 tx=0x7f1270004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f127d7fa640 1 -- 172.21.15.46:0/3214923442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f127003f070 con 0x7f1280102ab0 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1280197a40 con 0x7f1280102ab0 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.564+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1280197f70 con 0x7f1280102ab0 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.565+0000 7f127d7fa640 1 -- 172.21.15.46:0/3214923442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f12700035f0 con 0x7f1280102ab0 2024-09-10T04:19:06.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.565+0000 7f127d7fa640 1 -- 172.21.15.46:0/3214923442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1270035030 con 0x7f1280102ab0 2024-09-10T04:19:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.566+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126c005290 con 0x7f1280102ab0 2024-09-10T04:19:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.566+0000 7f127d7fa640 1 -- 172.21.15.46:0/3214923442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1270042070 con 0x7f1280102ab0 2024-09-10T04:19:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.567+0000 7f127d7fa640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f126807ac40 0x7f126807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.567+0000 7f127d7fa640 1 -- 172.21.15.46:0/3214923442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f12700be490 con 0x7f1280102ab0 2024-09-10T04:19:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.567+0000 7f127f7fe640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f126807ac40 0x7f126807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.567+0000 7f127f7fe640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f126807ac40 0x7f126807d100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12801088f0 tx=0x7f1274005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.571+0000 7f127d7fa640 1 -- 172.21.15.46:0/3214923442 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1270089c50 con 0x7f1280102ab0 2024-09-10T04:19:06.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.834+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f126c005d90 con 0x7f1280102ab0 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.835+0000 7f127d7fa640 1 -- 172.21.15.46:0/3214923442 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f127008eb00 con 0x7f1280102ab0 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:19:06.837 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:19:06.838 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:19:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.837+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f126807ac40 msgr2=0x7f126807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f126807ac40 0x7f126807d100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f12801088f0 tx=0x7f1274005f90 comp rx=0 tx=0).stop 2024-09-10T04:19:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 msgr2=0x7f128019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f128019d630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1270031460 tx=0x7f1270004530 comp rx=0 tx=0).stop 2024-09-10T04:19:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 shutdown_connections 2024-09-10T04:19:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f126807ac40 0x7f126807d100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12801033d0 0x7f128019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 --2- 172.21.15.46:0/3214923442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1280102ab0 0x7f128019d630 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 >> 172.21.15.46:0/3214923442 conn(0x7f12800fc550 msgr2=0x7f128010aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 shutdown_connections 2024-09-10T04:19:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.838+0000 7f12867c5640 1 -- 172.21.15.46:0/3214923442 wait complete. 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 -- 172.21.15.46:0/3109899197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 msgr2=0x7f10b4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3109899197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4107ce0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009c20 tx=0x7f10a402a970 comp rx=0 tx=0).stop 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 -- 172.21.15.46:0/3109899197 shutdown_connections 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3109899197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b4101900 0x7f10b4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3109899197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4107ce0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 -- 172.21.15.46:0/3109899197 >> 172.21.15.46:0/3109899197 conn(0x7f10b40fd600 msgr2=0x7f10b40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 -- 172.21.15.46:0/3109899197 shutdown_connections 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.926+0000 7f10bc61f640 1 -- 172.21.15.46:0/3109899197 wait complete. 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10bc61f640 1 Processor -- start 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10bc61f640 1 -- start start 2024-09-10T04:19:06.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10bc61f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b4101900 0x7f10b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10bc61f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10bc61f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410edb0 con 0x7f10b4107900 2024-09-10T04:19:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10bc61f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410ec30 con 0x7f10b4101900 2024-09-10T04:19:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10b9bba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10b9bba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60152/0 (socket says 172.21.15.46:60152) 2024-09-10T04:19:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10b9bba640 1 -- 172.21.15.46:0/3891770064 learned_addr learned my addr 172.21.15.46:0/3891770064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:06.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10ba3bb640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b4101900 0x7f10b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10b9bba640 1 -- 172.21.15.46:0/3891770064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b4101900 msgr2=0x7f10b40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10b9bba640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b4101900 0x7f10b40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.927+0000 7f10b9bba640 1 -- 172.21.15.46:0/3891770064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b4074f70 con 0x7f10b4107900 2024-09-10T04:19:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.928+0000 7f10b9bba640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4074a30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c8a0 tx=0x7f10a800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.928+0000 7f10b37fe640 1 -- 172.21.15.46:0/3891770064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f10a80079b0 con 0x7f10b4107900 2024-09-10T04:19:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.928+0000 7f10b37fe640 1 -- 172.21.15.46:0/3891770064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f10a8007b50 con 0x7f10b4107900 2024-09-10T04:19:06.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.928+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b4075260 con 0x7f10b4107900 2024-09-10T04:19:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.928+0000 7f10b37fe640 1 -- 172.21.15.46:0/3891770064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f10a8018700 con 0x7f10b4107900 2024-09-10T04:19:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.928+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b41a3da0 con 0x7f10b4107900 2024-09-10T04:19:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.929+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b4103040 con 0x7f10b4107900 2024-09-10T04:19:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.930+0000 7f10b37fe640 1 -- 172.21.15.46:0/3891770064 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f10a8007d00 con 0x7f10b4107900 2024-09-10T04:19:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.930+0000 7f10b37fe640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f109c07ac40 0x7f109c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:06.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.930+0000 7f10b37fe640 1 -- 172.21.15.46:0/3891770064 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f10a809de00 con 0x7f10b4107900 2024-09-10T04:19:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.930+0000 7f10ba3bb640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f109c07ac40 0x7f109c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:06.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.931+0000 7f10ba3bb640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f109c07ac40 0x7f109c07d100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009430 tx=0x7f10a403c040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:06.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:06.935+0000 7f10b37fe640 1 -- 172.21.15.46:0/3891770064 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f10a8014030 con 0x7f10b4107900 2024-09-10T04:19:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.157+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f10b4104b10 con 0x7f10b4107900 2024-09-10T04:19:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.157+0000 7f10b37fe640 1 -- 172.21.15.46:0/3891770064 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f10a80695c0 con 0x7f10b4107900 2024-09-10T04:19:07.159 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:19:07.160 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:19:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:07.162 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:19:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f109c07ac40 msgr2=0x7f109c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f109c07ac40 0x7f109c07d100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009430 tx=0x7f10a403c040 comp rx=0 tx=0).stop 2024-09-10T04:19:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 msgr2=0x7f10b4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4074a30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c8a0 tx=0x7f10a800cd60 comp rx=0 tx=0).stop 2024-09-10T04:19:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 shutdown_connections 2024-09-10T04:19:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f109c07ac40 0x7f109c07d100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10b4107900 0x7f10b4074a30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 --2- 172.21.15.46:0/3891770064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b4101900 0x7f10b40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 >> 172.21.15.46:0/3891770064 conn(0x7f10b40fd600 msgr2=0x7f10b40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 shutdown_connections 2024-09-10T04:19:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.160+0000 7f10bc61f640 1 -- 172.21.15.46:0/3891770064 wait complete. 2024-09-10T04:19:07.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 -- 172.21.15.46:0/1394001104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101067f0 msgr2=0x7f7f10106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1394001104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101067f0 0x7f7f10106bd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f00009880 tx=0x7f7f0002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 -- 172.21.15.46:0/1394001104 shutdown_connections 2024-09-10T04:19:07.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1394001104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f101007f0 0x7f7f10100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1394001104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101067f0 0x7f7f10106bd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 -- 172.21.15.46:0/1394001104 >> 172.21.15.46:0/1394001104 conn(0x7f7f100fc550 msgr2=0x7f7f100fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 -- 172.21.15.46:0/1394001104 shutdown_connections 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.246+0000 7f7f168a5640 1 -- 172.21.15.46:0/1394001104 wait complete. 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f168a5640 1 Processor -- start 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f168a5640 1 -- start start 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f168a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101007f0 0x7f7f10107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f168a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f101067f0 0x7f7f10107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f168a5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1010dca0 con 0x7f7f101007f0 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f168a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1010db20 con 0x7f7f101067f0 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f0ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101007f0 0x7f7f10107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f0ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101007f0 0x7f7f10107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60170/0 (socket says 172.21.15.46:60170) 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f0ffff640 1 -- 172.21.15.46:0/1706218905 learned_addr learned my addr 172.21.15.46:0/1706218905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f07fff640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f101067f0 0x7f7f10107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:07.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f0ffff640 1 -- 172.21.15.46:0/1706218905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f101067f0 msgr2=0x7f7f10107c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f0ffff640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f101067f0 0x7f7f10107c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.247+0000 7f7f0ffff640 1 -- 172.21.15.46:0/1706218905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f101081a0 con 0x7f7f101007f0 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.248+0000 7f7f0ffff640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101007f0 0x7f7f10107720 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7f00031460 tx=0x7f7f000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.248+0000 7f7f0dffb640 1 -- 172.21.15.46:0/1706218905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7f0003f070 con 0x7f7f101007f0 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.248+0000 7f7f0dffb640 1 -- 172.21.15.46:0/1706218905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f000035f0 con 0x7f7f101007f0 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.248+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f10108430 con 0x7f7f101007f0 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.248+0000 7f7f0dffb640 1 -- 172.21.15.46:0/1706218905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7f00035030 con 0x7f7f101007f0 2024-09-10T04:19:07.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.248+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f101a3d10 con 0x7f7f101007f0 2024-09-10T04:19:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.249+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f10101f30 con 0x7f7f101007f0 2024-09-10T04:19:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.250+0000 7f7f0dffb640 1 -- 172.21.15.46:0/1706218905 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7f00042070 con 0x7f7f101007f0 2024-09-10T04:19:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.250+0000 7f7f0dffb640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7efc07ac40 0x7f7efc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.250+0000 7f7f0dffb640 1 -- 172.21.15.46:0/1706218905 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f7f000be4f0 con 0x7f7f101007f0 2024-09-10T04:19:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.250+0000 7f7f07fff640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7efc07ac40 0x7f7efc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:07.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.251+0000 7f7f07fff640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7efc07ac40 0x7f7efc07d100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7f101091b0 tx=0x7f7ef800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:07.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.255+0000 7f7f0dffb640 1 -- 172.21.15.46:0/1706218905 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7f00089cb0 con 0x7f7f101007f0 2024-09-10T04:19:07.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.458+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7f1006a520 con 0x7f7efc07ac40 2024-09-10T04:19:07.461 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:07 smithi046 ceph-mon[109506]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:07.461 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:07 smithi046 ceph-mon[109506]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:07.461 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:07 smithi046 ceph-mon[109506]: pgmap v403: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:07.461 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:07 smithi046 ceph-mon[109506]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:07.461 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:07 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3214923442' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:19:07.461 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:07 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3891770064' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.460+0000 7f7f0dffb640 1 -- 172.21.15.46:0/1706218905 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f7f1006a520 con 0x7f7efc07ac40 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:19:07.462 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:19:07.463 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:19:07.463 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:19:07.463 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:19:07.463 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7efc07ac40 msgr2=0x7f7efc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7efc07ac40 0x7f7efc07d100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7f101091b0 tx=0x7f7ef800b390 comp rx=0 tx=0).stop 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101007f0 msgr2=0x7f7f10107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101007f0 0x7f7f10107720 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7f00031460 tx=0x7f7f000045e0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 shutdown_connections 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7efc07ac40 0x7f7efc07d100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f101067f0 0x7f7f10107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 --2- 172.21.15.46:0/1706218905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f101007f0 0x7f7f10107720 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 >> 172.21.15.46:0/1706218905 conn(0x7f7f100fc550 msgr2=0x7f7f10069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:07.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 shutdown_connections 2024-09-10T04:19:07.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.463+0000 7f7f168a5640 1 -- 172.21.15.46:0/1706218905 wait complete. 2024-09-10T04:19:07.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.547+0000 7f6ae10e7640 1 -- 172.21.15.46:0/4052455812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1007f0 msgr2=0x7f6adc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.547+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/4052455812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1007f0 0x7f6adc100c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009880 tx=0x7f6ad002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.547+0000 7f6ae10e7640 1 -- 172.21.15.46:0/4052455812 shutdown_connections 2024-09-10T04:19:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.547+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/4052455812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1007f0 0x7f6adc100c50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.547+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/4052455812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc1067f0 0x7f6adc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.547+0000 7f6ae10e7640 1 -- 172.21.15.46:0/4052455812 >> 172.21.15.46:0/4052455812 conn(0x7f6adc0fc550 msgr2=0x7f6adc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.547+0000 7f6ae10e7640 1 -- 172.21.15.46:0/4052455812 shutdown_connections 2024-09-10T04:19:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.548+0000 7f6ae10e7640 1 -- 172.21.15.46:0/4052455812 wait complete. 2024-09-10T04:19:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.548+0000 7f6ae10e7640 1 Processor -- start 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.548+0000 7f6ae10e7640 1 -- start start 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ae10e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc1007f0 0x7f6adc19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ae10e7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1067f0 0x7f6adc19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ae10e7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10dca0 con 0x7f6adc1067f0 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ae10e7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc10db20 con 0x7f6adc1007f0 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ada59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1067f0 0x7f6adc19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ada59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1067f0 0x7f6adc19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60186/0 (socket says 172.21.15.46:60186) 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ada59c640 1 -- 172.21.15.46:0/945174412 learned_addr learned my addr 172.21.15.46:0/945174412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6adad9d640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc1007f0 0x7f6adc19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ada59c640 1 -- 172.21.15.46:0/945174412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc1007f0 msgr2=0x7f6adc19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ada59c640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc1007f0 0x7f6adc19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ada59c640 1 -- 172.21.15.46:0/945174412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc197780 con 0x7f6adc1067f0 2024-09-10T04:19:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.549+0000 7f6ada59c640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1067f0 0x7f6adc19db40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0031460 tx=0x7f6ad00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.550+0000 7f6ac3fff640 1 -- 172.21.15.46:0/945174412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ad003f070 con 0x7f6adc1067f0 2024-09-10T04:19:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.550+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc197a10 con 0x7f6adc1067f0 2024-09-10T04:19:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.550+0000 7f6ac3fff640 1 -- 172.21.15.46:0/945174412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ad0004cf0 con 0x7f6adc1067f0 2024-09-10T04:19:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.550+0000 7f6ac3fff640 1 -- 172.21.15.46:0/945174412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ad0035030 con 0x7f6adc1067f0 2024-09-10T04:19:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.550+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc197f40 con 0x7f6adc1067f0 2024-09-10T04:19:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.551+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc101f30 con 0x7f6adc1067f0 2024-09-10T04:19:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.552+0000 7f6ac3fff640 1 -- 172.21.15.46:0/945174412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6ad0042070 con 0x7f6adc1067f0 2024-09-10T04:19:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.552+0000 7f6ac3fff640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ac407abf0 0x7f6ac407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.552+0000 7f6ac3fff640 1 -- 172.21.15.46:0/945174412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f6ad00be690 con 0x7f6adc1067f0 2024-09-10T04:19:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.552+0000 7f6adad9d640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ac407abf0 0x7f6ac407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.553+0000 7f6adad9d640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ac407abf0 0x7f6ac407d0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00abe0 tx=0x7f6acc00a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.557+0000 7f6ac3fff640 1 -- 172.21.15.46:0/945174412 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6ad0089dd0 con 0x7f6adc1067f0 2024-09-10T04:19:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.815+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6adc198cb0 con 0x7f6adc1067f0 2024-09-10T04:19:07.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.815+0000 7f6ac3fff640 1 -- 172.21.15.46:0/945174412 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f6ad008ec80 con 0x7f6adc1067f0 2024-09-10T04:19:07.817 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:19:07.817 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:19:07.818 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:19:07.818 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:19:07.818 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:19:07.818 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:19:07.818 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ac407abf0 msgr2=0x7f6ac407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ac407abf0 0x7f6ac407d0b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00abe0 tx=0x7f6acc00a2d0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1067f0 msgr2=0x7f6adc19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1067f0 0x7f6adc19db40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0031460 tx=0x7f6ad00048f0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 shutdown_connections 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ac407abf0 0x7f6ac407d0b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6adc1067f0 0x7f6adc19db40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 --2- 172.21.15.46:0/945174412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6adc1007f0 0x7f6adc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 >> 172.21.15.46:0/945174412 conn(0x7f6adc0fc550 msgr2=0x7f6adc069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:07.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 shutdown_connections 2024-09-10T04:19:07.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:07.819+0000 7f6ae10e7640 1 -- 172.21.15.46:0/945174412 wait complete. 2024-09-10T04:19:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:07 smithi070 ceph-mon[79774]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:07 smithi070 ceph-mon[79774]: from='client.34622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:07 smithi070 ceph-mon[79774]: pgmap v403: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:07 smithi070 ceph-mon[79774]: from='client.34626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:07 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3214923442' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:19:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:07 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3891770064' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:19:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:08 smithi046 ceph-mon[109506]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:08 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/945174412' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:19:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:08 smithi070 ceph-mon[79774]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:08 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/945174412' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:19:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:09 smithi070 ceph-mon[79774]: pgmap v404: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:09 smithi046 ceph-mon[109506]: pgmap v404: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:11.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:11 smithi046 ceph-mon[109506]: pgmap v405: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:11 smithi070 ceph-mon[79774]: pgmap v405: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:13 smithi070 ceph-mon[79774]: pgmap v406: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:13 smithi046 ceph-mon[109506]: pgmap v406: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:15 smithi070 ceph-mon[79774]: pgmap v407: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:15 smithi046 ceph-mon[109506]: pgmap v407: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:17 smithi070 ceph-mon[79774]: pgmap v408: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:17.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:17 smithi046 ceph-mon[109506]: pgmap v408: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:19 smithi070 ceph-mon[79774]: pgmap v409: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:19 smithi046 ceph-mon[109506]: pgmap v409: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:21 smithi070 ceph-mon[79774]: pgmap v410: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:21 smithi046 ceph-mon[109506]: pgmap v410: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:23 smithi046 ceph-mon[109506]: pgmap v411: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:23 smithi070 ceph-mon[79774]: pgmap v411: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:25 smithi070 ceph-mon[79774]: pgmap v412: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:25 smithi046 ceph-mon[109506]: pgmap v412: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:27.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:27 smithi046 ceph-mon[109506]: pgmap v413: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:27 smithi070 ceph-mon[79774]: pgmap v413: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:29 smithi046 ceph-mon[109506]: pgmap v414: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:29 smithi070 ceph-mon[79774]: pgmap v414: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:31 smithi070 ceph-mon[79774]: pgmap v415: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:31 smithi046 ceph-mon[109506]: pgmap v415: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:33.794 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:33 smithi070 ceph-mon[79774]: pgmap v416: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:33 smithi046 ceph-mon[109506]: pgmap v416: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:35 smithi046 ceph-mon[109506]: pgmap v417: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:35 smithi070 ceph-mon[79774]: pgmap v417: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:37 smithi070 ceph-mon[79774]: pgmap v418: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:37 smithi046 ceph-mon[109506]: pgmap v418: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 -- 172.21.15.46:0/2963585839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc102b50 msgr2=0x7f29bc102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 --2- 172.21.15.46:0/2963585839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc102b50 0x7f29bc102f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009880 tx=0x7f29ac02a530 comp rx=0 tx=0).stop 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 -- 172.21.15.46:0/2963585839 shutdown_connections 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 --2- 172.21.15.46:0/2963585839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc103470 0x7f29bc10b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 --2- 172.21.15.46:0/2963585839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc102b50 0x7f29bc102f30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 -- 172.21.15.46:0/2963585839 >> 172.21.15.46:0/2963585839 conn(0x7f29bc0fc550 msgr2=0x7f29bc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 -- 172.21.15.46:0/2963585839 shutdown_connections 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.919+0000 7f29c2985640 1 -- 172.21.15.46:0/2963585839 wait complete. 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.920+0000 7f29c2985640 1 Processor -- start 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.920+0000 7f29c2985640 1 -- start start 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.920+0000 7f29c2985640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc102b50 0x7f29bc1007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.920+0000 7f29c2985640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc103470 0x7f29bc100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.920+0000 7f29c2985640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10ede0 con 0x7f29bc103470 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.920+0000 7f29c2985640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f29bc10ec60 con 0x7f29bc102b50 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc103470 0x7f29bc100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc103470 0x7f29bc100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44244/0 (socket says 172.21.15.46:44244) 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bb7fe640 1 -- 172.21.15.46:0/995180224 learned_addr learned my addr 172.21.15.46:0/995180224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bbfff640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc102b50 0x7f29bc1007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bb7fe640 1 -- 172.21.15.46:0/995180224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc102b50 msgr2=0x7f29bc1007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bb7fe640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc102b50 0x7f29bc1007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bb7fe640 1 -- 172.21.15.46:0/995180224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29bc101230 con 0x7f29bc103470 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29bb7fe640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc103470 0x7f29bc100cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f29b000d6f0 tx=0x7f29b000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29b97fa640 1 -- 172.21.15.46:0/995180224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f29b0017070 con 0x7f29bc103470 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29b97fa640 1 -- 172.21.15.46:0/995180224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f29b000de00 con 0x7f29bc103470 2024-09-10T04:19:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29bc101520 con 0x7f29bc103470 2024-09-10T04:19:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29b97fa640 1 -- 172.21.15.46:0/995180224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f29b0004bb0 con 0x7f29bc103470 2024-09-10T04:19:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.921+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29bc1a3e50 con 0x7f29bc103470 2024-09-10T04:19:37.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.922+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29bc06a4e0 con 0x7f29bc103470 2024-09-10T04:19:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.923+0000 7f29b97fa640 1 -- 172.21.15.46:0/995180224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f29b0009a30 con 0x7f29bc103470 2024-09-10T04:19:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.924+0000 7f29b97fa640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f29a407ac50 0x7f29a407d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.924+0000 7f29b97fa640 1 -- 172.21.15.46:0/995180224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f29b009d7e0 con 0x7f29bc103470 2024-09-10T04:19:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.924+0000 7f29bbfff640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f29a407ac50 0x7f29a407d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.924+0000 7f29bbfff640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f29a407ac50 0x7f29a407d110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0096c0 tx=0x7f29ac004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:37.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:37.928+0000 7f29b97fa640 1 -- 172.21.15.46:0/995180224 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f29b0068f00 con 0x7f29bc103470 2024-09-10T04:19:38.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.133+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f29bc062630 con 0x7f29a407ac50 2024-09-10T04:19:38.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.134+0000 7f29b97fa640 1 -- 172.21.15.46:0/995180224 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f29bc062630 con 0x7f29a407ac50 2024-09-10T04:19:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f29a407ac50 msgr2=0x7f29a407d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f29a407ac50 0x7f29a407d110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0096c0 tx=0x7f29ac004f30 comp rx=0 tx=0).stop 2024-09-10T04:19:38.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc103470 msgr2=0x7f29bc100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc103470 0x7f29bc100cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f29b000d6f0 tx=0x7f29b000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 shutdown_connections 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f29a407ac50 0x7f29a407d110 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29bc103470 0x7f29bc100cf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 --2- 172.21.15.46:0/995180224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29bc102b50 0x7f29bc1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 >> 172.21.15.46:0/995180224 conn(0x7f29bc0fc550 msgr2=0x7f29bc108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.136+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 shutdown_connections 2024-09-10T04:19:38.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.137+0000 7f29c2985640 1 -- 172.21.15.46:0/995180224 wait complete. 2024-09-10T04:19:38.147 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:19:38.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 -- 172.21.15.46:0/2733500491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 msgr2=0x7f2aac10b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/2733500491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac10b960 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009880 tx=0x7f2aa002a530 comp rx=0 tx=0).stop 2024-09-10T04:19:38.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 -- 172.21.15.46:0/2733500491 shutdown_connections 2024-09-10T04:19:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/2733500491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac10b960 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/2733500491 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac102b50 0x7f2aac102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 -- 172.21.15.46:0/2733500491 >> 172.21.15.46:0/2733500491 conn(0x7f2aac0fc550 msgr2=0x7f2aac0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 -- 172.21.15.46:0/2733500491 shutdown_connections 2024-09-10T04:19:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 -- 172.21.15.46:0/2733500491 wait complete. 2024-09-10T04:19:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.223+0000 7f2ab16fd640 1 Processor -- start 2024-09-10T04:19:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2ab16fd640 1 -- start start 2024-09-10T04:19:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2ab16fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac102b50 0x7f2aac19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2ab16fd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2ab16fd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac10edc0 con 0x7f2aac103470 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2ab16fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac10ec40 con 0x7f2aac102b50 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2aaa7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2aaa7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac19db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44270/0 (socket says 172.21.15.46:44270) 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2aaa7fc640 1 -- 172.21.15.46:0/3530166798 learned_addr learned my addr 172.21.15.46:0/3530166798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2aaaffd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac102b50 0x7f2aac19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2aaa7fc640 1 -- 172.21.15.46:0/3530166798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac102b50 msgr2=0x7f2aac19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2aaa7fc640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac102b50 0x7f2aac19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.224+0000 7f2aaa7fc640 1 -- 172.21.15.46:0/3530166798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac197790 con 0x7f2aac103470 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.225+0000 7f2aaa7fc640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac19db50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0004f90 tx=0x7f2aa00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.225+0000 7f2a93fff640 1 -- 172.21.15.46:0/3530166798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2aa003f070 con 0x7f2aac103470 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.225+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aac197a20 con 0x7f2aac103470 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.225+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aac197f50 con 0x7f2aac103470 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.225+0000 7f2a93fff640 1 -- 172.21.15.46:0/3530166798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2aa0004c70 con 0x7f2aac103470 2024-09-10T04:19:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.225+0000 7f2a93fff640 1 -- 172.21.15.46:0/3530166798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2aa0035030 con 0x7f2aac103470 2024-09-10T04:19:38.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.226+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a98005290 con 0x7f2aac103470 2024-09-10T04:19:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.227+0000 7f2a93fff640 1 -- 172.21.15.46:0/3530166798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2aa0042070 con 0x7f2aac103470 2024-09-10T04:19:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.227+0000 7f2a93fff640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9407abf0 0x7f2a9407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.227+0000 7f2a93fff640 1 -- 172.21.15.46:0/3530166798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f2aa00be580 con 0x7f2aac103470 2024-09-10T04:19:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.227+0000 7f2aaaffd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9407abf0 0x7f2a9407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.228+0000 7f2aaaffd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9407abf0 0x7f2a9407d0b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c005d00 tx=0x7f2a9c00b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:38.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.232+0000 7f2a93fff640 1 -- 172.21.15.46:0/3530166798 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2aa0089cc0 con 0x7f2aac103470 2024-09-10T04:19:38.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.438+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2a98000bc0 con 0x7f2a9407abf0 2024-09-10T04:19:38.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.439+0000 7f2a93fff640 1 -- 172.21.15.46:0/3530166798 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f2a98000bc0 con 0x7f2a9407abf0 2024-09-10T04:19:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9407abf0 msgr2=0x7f2a9407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9407abf0 0x7f2a9407d0b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c005d00 tx=0x7f2a9c00b410 comp rx=0 tx=0).stop 2024-09-10T04:19:38.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 msgr2=0x7f2aac19db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac19db50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0004f90 tx=0x7f2aa00048f0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 shutdown_connections 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9407abf0 0x7f2a9407d0b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac103470 0x7f2aac19db50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 --2- 172.21.15.46:0/3530166798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac102b50 0x7f2aac19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.441+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 >> 172.21.15.46:0/3530166798 conn(0x7f2aac0fc550 msgr2=0x7f2aac108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.442+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 shutdown_connections 2024-09-10T04:19:38.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.442+0000 7f2ab16fd640 1 -- 172.21.15.46:0/3530166798 wait complete. 2024-09-10T04:19:38.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 -- 172.21.15.46:0/793224743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 msgr2=0x7f02b8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 --2- 172.21.15.46:0/793224743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b8106be0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009880 tx=0x7f02a802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 -- 172.21.15.46:0/793224743 shutdown_connections 2024-09-10T04:19:38.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 --2- 172.21.15.46:0/793224743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b8100800 0x7f02b8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 --2- 172.21.15.46:0/793224743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b8106be0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 -- 172.21.15.46:0/793224743 >> 172.21.15.46:0/793224743 conn(0x7f02b80fc520 msgr2=0x7f02b80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:38.528 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:38.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 -- 172.21.15.46:0/793224743 shutdown_connections 2024-09-10T04:19:38.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.526+0000 7f02bcb24640 1 -- 172.21.15.46:0/793224743 wait complete. 2024-09-10T04:19:38.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02bcb24640 1 Processor -- start 2024-09-10T04:19:38.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02bcb24640 1 -- start start 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02bcb24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b8100800 0x7f02b819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02bcb24640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02bcb24640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02b810dcb0 con 0x7f02b8106800 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02bcb24640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f02b810db30 con 0x7f02b8100800 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02affff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02affff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44288/0 (socket says 172.21.15.46:44288) 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02affff640 1 -- 172.21.15.46:0/4239037155 learned_addr learned my addr 172.21.15.46:0/4239037155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.527+0000 7f02b659c640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b8100800 0x7f02b819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02affff640 1 -- 172.21.15.46:0/4239037155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b8100800 msgr2=0x7f02b819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02affff640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b8100800 0x7f02b819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02affff640 1 -- 172.21.15.46:0/4239037155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b81a03f0 con 0x7f02b8106800 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02affff640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b819fd40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c8a0 tx=0x7f02a000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02af7fe640 1 -- 172.21.15.46:0/4239037155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02a00079b0 con 0x7f02b8106800 2024-09-10T04:19:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b8199ae0 con 0x7f02b8106800 2024-09-10T04:19:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02af7fe640 1 -- 172.21.15.46:0/4239037155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02a0007b50 con 0x7f02b8106800 2024-09-10T04:19:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b819a040 con 0x7f02b8106800 2024-09-10T04:19:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.528+0000 7f02af7fe640 1 -- 172.21.15.46:0/4239037155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02a0018700 con 0x7f02b8106800 2024-09-10T04:19:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.529+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0298005290 con 0x7f02b8106800 2024-09-10T04:19:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.530+0000 7f02af7fe640 1 -- 172.21.15.46:0/4239037155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f02a0018980 con 0x7f02b8106800 2024-09-10T04:19:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.530+0000 7f02af7fe640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02a407abf0 0x7f02a407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.531+0000 7f02af7fe640 1 -- 172.21.15.46:0/4239037155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f02a009de10 con 0x7f02b8106800 2024-09-10T04:19:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.531+0000 7f02b659c640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02a407abf0 0x7f02a407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.531+0000 7f02b659c640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02a407abf0 0x7f02a407d0b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02a8000c00 tx=0x7f02a8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:38.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.535+0000 7f02af7fe640 1 -- 172.21.15.46:0/4239037155 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f02a0014030 con 0x7f02b8106800 2024-09-10T04:19:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.731+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0298000bc0 con 0x7f02a407abf0 2024-09-10T04:19:38.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.740+0000 7f02af7fe640 1 -- 172.21.15.46:0/4239037155 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f0298000bc0 con 0x7f02a407abf0 2024-09-10T04:19:38.741 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:19:38.741 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (11m) 5m ago 22m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:19:38.741 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (13m) 5m ago 22m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:19:38.741 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (13m) 5m ago 21m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:19:38.741 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (9m) 5m ago 22m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:19:38.741 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (9m) 5m ago 21m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (10m) 5m ago 22m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (5m) 5m ago 17m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (6m) 5m ago 17m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (5m) 5m ago 17m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (17m) 5m ago 17m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (13m) 5m ago 23m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (12m) 5m ago 21m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (10m) 5m ago 24m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (9m) 5m ago 21m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (12m) 5m ago 22m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (11m) 5m ago 21m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (9m) 5m ago 20m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (8m) 5m ago 20m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:19:38.742 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (8m) 5m ago 19m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:19:38.743 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (7m) 5m ago 19m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:19:38.743 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (6m) 5m ago 18m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:19:38.743 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (6m) 5m ago 18m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:19:38.743 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (11m) 5m ago 22m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.742+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02a407abf0 msgr2=0x7f02a407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.742+0000 7f02bcb24640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02a407abf0 0x7f02a407d0b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f02a8000c00 tx=0x7f02a8031400 comp rx=0 tx=0).stop 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.742+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 msgr2=0x7f02b819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.742+0000 7f02bcb24640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b819fd40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f02a000c8a0 tx=0x7f02a000cd60 comp rx=0 tx=0).stop 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.742+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 shutdown_connections 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.742+0000 7f02bcb24640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02a407abf0 0x7f02a407d0b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.743+0000 7f02bcb24640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02b8106800 0x7f02b819fd40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.743+0000 7f02bcb24640 1 --2- 172.21.15.46:0/4239037155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02b8100800 0x7f02b819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.743+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 >> 172.21.15.46:0/4239037155 conn(0x7f02b80fc520 msgr2=0x7f02b806a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.743+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 shutdown_connections 2024-09-10T04:19:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.743+0000 7f02bcb24640 1 -- 172.21.15.46:0/4239037155 wait complete. 2024-09-10T04:19:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 -- 172.21.15.46:0/4071787289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00102ab0 msgr2=0x7f6f00102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 --2- 172.21.15.46:0/4071787289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00102ab0 0x7f6f00102e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009880 tx=0x7f6ef002a540 comp rx=0 tx=0).stop 2024-09-10T04:19:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 -- 172.21.15.46:0/4071787289 shutdown_connections 2024-09-10T04:19:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 --2- 172.21.15.46:0/4071787289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f001033d0 0x7f6f0010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 --2- 172.21.15.46:0/4071787289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00102ab0 0x7f6f00102e90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 -- 172.21.15.46:0/4071787289 >> 172.21.15.46:0/4071787289 conn(0x7f6f000fc550 msgr2=0x7f6f000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 -- 172.21.15.46:0/4071787289 shutdown_connections 2024-09-10T04:19:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 -- 172.21.15.46:0/4071787289 wait complete. 2024-09-10T04:19:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.828+0000 7f6f05cba640 1 Processor -- start 2024-09-10T04:19:38.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6f05cba640 1 -- start start 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6f05cba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00102ab0 0x7f6f0019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6f05cba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f001033d0 0x7f6f0019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6f05cba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0010ede0 con 0x7f6f001033d0 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6f05cba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0010ec60 con 0x7f6f00102ab0 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6efeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f001033d0 0x7f6f0019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6efeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f001033d0 0x7f6f0019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48264/0 (socket says 172.21.15.46:48264) 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6efeffd640 1 -- 172.21.15.46:0/2320154310 learned_addr learned my addr 172.21.15.46:0/2320154310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:38.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6eff7fe640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00102ab0 0x7f6f0019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6efeffd640 1 -- 172.21.15.46:0/2320154310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00102ab0 msgr2=0x7f6f0019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6efeffd640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00102ab0 0x7f6f0019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.829+0000 7f6efeffd640 1 -- 172.21.15.46:0/2320154310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f00199a20 con 0x7f6f001033d0 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.830+0000 7f6efeffd640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f001033d0 0x7f6f0019fd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400d6d0 tx=0x7f6ef400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.830+0000 7f6efcff9640 1 -- 172.21.15.46:0/2320154310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ef4017070 con 0x7f6f001033d0 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.830+0000 7f6efcff9640 1 -- 172.21.15.46:0/2320154310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ef40054e0 con 0x7f6f001033d0 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.830+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f00199d10 con 0x7f6f001033d0 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.830+0000 7f6efcff9640 1 -- 172.21.15.46:0/2320154310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ef4004ae0 con 0x7f6f001033d0 2024-09-10T04:19:38.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.830+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0019a2a0 con 0x7f6f001033d0 2024-09-10T04:19:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.831+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0006a580 con 0x7f6f001033d0 2024-09-10T04:19:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.832+0000 7f6efcff9640 1 -- 172.21.15.46:0/2320154310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6ef4004de0 con 0x7f6f001033d0 2024-09-10T04:19:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.832+0000 7f6efcff9640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ee807abf0 0x7f6ee807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.832+0000 7f6eff7fe640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ee807abf0 0x7f6ee807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:38.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.832+0000 7f6efcff9640 1 -- 172.21.15.46:0/2320154310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f6ef409d7c0 con 0x7f6f001033d0 2024-09-10T04:19:38.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.833+0000 7f6eff7fe640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ee807abf0 0x7f6ee807d0b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0000c00 tx=0x7f6ef0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:38.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:38.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:38.837+0000 7f6efcff9640 1 -- 172.21.15.46:0/2320154310 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6ef4068f00 con 0x7f6f001033d0 2024-09-10T04:19:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.094+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f6f0019af00 con 0x7f6f001033d0 2024-09-10T04:19:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.094+0000 7f6efcff9640 1 -- 172.21.15.46:0/2320154310 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f6ef406ddb0 con 0x7f6f001033d0 2024-09-10T04:19:39.095 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:19:39.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:19:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:19:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:19:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:19:39.097 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:19:39.097 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.096+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ee807abf0 msgr2=0x7f6ee807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.096+0000 7f6f05cba640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ee807abf0 0x7f6ee807d0b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0000c00 tx=0x7f6ef0031400 comp rx=0 tx=0).stop 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f001033d0 msgr2=0x7f6f0019fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f001033d0 0x7f6f0019fd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6ef400d6d0 tx=0x7f6ef400db90 comp rx=0 tx=0).stop 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 shutdown_connections 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6ee807abf0 0x7f6ee807d0b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f001033d0 0x7f6f0019fd50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 --2- 172.21.15.46:0/2320154310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00102ab0 0x7f6f0019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 >> 172.21.15.46:0/2320154310 conn(0x7f6f000fc550 msgr2=0x7f6f00100b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 shutdown_connections 2024-09-10T04:19:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.097+0000 7f6f05cba640 1 -- 172.21.15.46:0/2320154310 wait complete. 2024-09-10T04:19:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.181+0000 7f82f7733640 1 -- 172.21.15.46:0/2188783613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 msgr2=0x7f82f01056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.181+0000 7f82f7733640 1 --2- 172.21.15.46:0/2188783613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f01056a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009880 tx=0x7f82e402a530 comp rx=0 tx=0).stop 2024-09-10T04:19:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.181+0000 7f82f7733640 1 -- 172.21.15.46:0/2188783613 shutdown_connections 2024-09-10T04:19:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.181+0000 7f82f7733640 1 --2- 172.21.15.46:0/2188783613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f01056a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.181+0000 7f82f7733640 1 --2- 172.21.15.46:0/2188783613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 0x7f82f0100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.181+0000 7f82f7733640 1 -- 172.21.15.46:0/2188783613 >> 172.21.15.46:0/2188783613 conn(0x7f82f00fc520 msgr2=0x7f82f00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 -- 172.21.15.46:0/2188783613 shutdown_connections 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 -- 172.21.15.46:0/2188783613 wait complete. 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 Processor -- start 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 -- start start 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 0x7f82f019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01153d0 con 0x7f82f01011d0 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.182+0000 7f82f7733640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0115250 con 0x7f82f01008b0 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f4cce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f54cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 0x7f82f019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f54cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 0x7f82f019dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.46:57530/0 (socket says 172.21.15.46:57530) 2024-09-10T04:19:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f4cce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f019e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48290/0 (socket says 172.21.15.46:48290) 2024-09-10T04:19:39.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f54cf640 1 -- 172.21.15.46:0/1202635287 learned_addr learned my addr 172.21.15.46:0/1202635287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f4cce640 1 -- 172.21.15.46:0/1202635287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 msgr2=0x7f82f019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f4cce640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 0x7f82f019dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f4cce640 1 -- 172.21.15.46:0/1202635287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f01a2880 con 0x7f82f01011d0 2024-09-10T04:19:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f54cf640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 0x7f82f019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:19:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f4cce640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f019e4f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f82e4006010 tx=0x7f82e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82e27fc640 1 -- 172.21.15.46:0/1202635287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f82e403f070 con 0x7f82f01011d0 2024-09-10T04:19:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f01a2b10 con 0x7f82f01011d0 2024-09-10T04:19:39.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82e27fc640 1 -- 172.21.15.46:0/1202635287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f82e40035f0 con 0x7f82f01011d0 2024-09-10T04:19:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82e27fc640 1 -- 172.21.15.46:0/1202635287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f82e4035030 con 0x7f82f01011d0 2024-09-10T04:19:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.183+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f01a3040 con 0x7f82f01011d0 2024-09-10T04:19:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.184+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f006aa90 con 0x7f82f01011d0 2024-09-10T04:19:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.185+0000 7f82e27fc640 1 -- 172.21.15.46:0/1202635287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f82e4042070 con 0x7f82f01011d0 2024-09-10T04:19:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.186+0000 7f82e27fc640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f82dc07abf0 0x7f82dc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.186+0000 7f82e27fc640 1 -- 172.21.15.46:0/1202635287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f82e40be4c0 con 0x7f82f01011d0 2024-09-10T04:19:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.186+0000 7f82f54cf640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f82dc07abf0 0x7f82dc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.186+0000 7f82f54cf640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f82dc07abf0 0x7f82dc07d0b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005fd0 tx=0x7f82ec00b480 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.190+0000 7f82e27fc640 1 -- 172.21.15.46:0/1202635287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f82e4089c00 con 0x7f82f01011d0 2024-09-10T04:19:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.413+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f82f019f340 con 0x7f82f01011d0 2024-09-10T04:19:39.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.413+0000 7f82e27fc640 1 -- 172.21.15.46:0/1202635287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f82f019f340 con 0x7f82f01011d0 2024-09-10T04:19:39.414 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:19:39.415 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:19:39.416 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:39.417 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:19:39.418 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:19:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:39.418 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f82dc07abf0 msgr2=0x7f82dc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f82dc07abf0 0x7f82dc07d0b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005fd0 tx=0x7f82ec00b480 comp rx=0 tx=0).stop 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 msgr2=0x7f82f019e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f019e4f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f82e4006010 tx=0x7f82e40045e0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 shutdown_connections 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f82dc07abf0 0x7f82dc07d0b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82f01011d0 0x7f82f019e4f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 --2- 172.21.15.46:0/1202635287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f01008b0 0x7f82f019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.416+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 >> 172.21.15.46:0/1202635287 conn(0x7f82f00fc520 msgr2=0x7f82f0112c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:39.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.417+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 shutdown_connections 2024-09-10T04:19:39.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.417+0000 7f82f7733640 1 -- 172.21.15.46:0/1202635287 wait complete. 2024-09-10T04:19:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.502+0000 7f0f2cb22640 1 -- 172.21.15.46:0/259465222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28103430 msgr2=0x7f0f2810b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.502+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/259465222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28103430 0x7f0f2810b920 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c0098b0 tx=0x7f0f1c02a560 comp rx=0 tx=0).stop 2024-09-10T04:19:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.503+0000 7f0f2cb22640 1 -- 172.21.15.46:0/259465222 shutdown_connections 2024-09-10T04:19:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.503+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/259465222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28103430 0x7f0f2810b920 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.503+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/259465222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f28102b10 0x7f0f28102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.503+0000 7f0f2cb22640 1 -- 172.21.15.46:0/259465222 >> 172.21.15.46:0/259465222 conn(0x7f0f280fc550 msgr2=0x7f0f280fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:39.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.503+0000 7f0f2cb22640 1 -- 172.21.15.46:0/259465222 shutdown_connections 2024-09-10T04:19:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.503+0000 7f0f2cb22640 1 -- 172.21.15.46:0/259465222 wait complete. 2024-09-10T04:19:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.503+0000 7f0f2cb22640 1 Processor -- start 2024-09-10T04:19:39.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2cb22640 1 -- start start 2024-09-10T04:19:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2cb22640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28102b10 0x7f0f2819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2cb22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f28103430 0x7f0f2819fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2cb22640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810ede0 con 0x7f0f28102b10 2024-09-10T04:19:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2cb22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2810ec60 con 0x7f0f28103430 2024-09-10T04:19:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28102b10 0x7f0f2819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28102b10 0x7f0f2819f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48308/0 (socket says 172.21.15.46:48308) 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2659c640 1 -- 172.21.15.46:0/2339127917 learned_addr learned my addr 172.21.15.46:0/2339127917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f25d9b640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f28103430 0x7f0f2819fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2659c640 1 -- 172.21.15.46:0/2339127917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f28103430 msgr2=0x7f0f2819fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2659c640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f28103430 0x7f0f2819fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2659c640 1 -- 172.21.15.46:0/2339127917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f28199aa0 con 0x7f0f28102b10 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.504+0000 7f0f2659c640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28102b10 0x7f0f2819f8e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800c8f0 tx=0x7f0f1800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.505+0000 7f0f177fe640 1 -- 172.21.15.46:0/2339127917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f180079b0 con 0x7f0f28102b10 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.505+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f28199d90 con 0x7f0f28102b10 2024-09-10T04:19:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.505+0000 7f0f177fe640 1 -- 172.21.15.46:0/2339127917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f180054e0 con 0x7f0f28102b10 2024-09-10T04:19:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.505+0000 7f0f177fe640 1 -- 172.21.15.46:0/2339127917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f18004dd0 con 0x7f0f28102b10 2024-09-10T04:19:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.505+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2819a2f0 con 0x7f0f28102b10 2024-09-10T04:19:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.506+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2806a520 con 0x7f0f28102b10 2024-09-10T04:19:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.507+0000 7f0f177fe640 1 -- 172.21.15.46:0/2339127917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0f18005020 con 0x7f0f28102b10 2024-09-10T04:19:39.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.507+0000 7f0f177fe640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f1007ab90 0x7f0f1007d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.507+0000 7f0f25d9b640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f1007ab90 0x7f0f1007d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.507+0000 7f0f177fe640 1 -- 172.21.15.46:0/2339127917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0f1809dba0 con 0x7f0f28102b10 2024-09-10T04:19:39.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.508+0000 7f0f25d9b640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f1007ab90 0x7f0f1007d050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c000c00 tx=0x7f0f1c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.512+0000 7f0f177fe640 1 -- 172.21.15.46:0/2339127917 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0f18014030 con 0x7f0f28102b10 2024-09-10T04:19:39.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.717+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0f2819afd0 con 0x7f0f1007ab90 2024-09-10T04:19:39.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:39 smithi046 ceph-mon[109506]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:39.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:39 smithi046 ceph-mon[109506]: pgmap v419: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:39.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:39 smithi046 ceph-mon[109506]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:39.719 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:39 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2320154310' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:19:39.719 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:39 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1202635287' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:19:39.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.719+0000 7f0f177fe640 1 -- 172.21.15.46:0/2339127917 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0f2819afd0 con 0x7f0f1007ab90 2024-09-10T04:19:39.719 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:19:39.719 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:19:39.719 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:19:39.720 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f1007ab90 msgr2=0x7f0f1007d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f1007ab90 0x7f0f1007d050 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c000c00 tx=0x7f0f1c031400 comp rx=0 tx=0).stop 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28102b10 msgr2=0x7f0f2819f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28102b10 0x7f0f2819f8e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800c8f0 tx=0x7f0f1800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 shutdown_connections 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f1007ab90 0x7f0f1007d050 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f28103430 0x7f0f2819fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 --2- 172.21.15.46:0/2339127917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f28102b10 0x7f0f2819f8e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 >> 172.21.15.46:0/2339127917 conn(0x7f0f280fc550 msgr2=0x7f0f28100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 shutdown_connections 2024-09-10T04:19:39.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.721+0000 7f0f2cb22640 1 -- 172.21.15.46:0/2339127917 wait complete. 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3350231148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0107900 msgr2=0x7fd4b0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3350231148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0107900 0x7fd4b0107ce0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac009880 tx=0x7fd4ac02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3350231148 shutdown_connections 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3350231148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b0101900 0x7fd4b0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3350231148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0107900 0x7fd4b0107ce0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3350231148 >> 172.21.15.46:0/3350231148 conn(0x7fd4b00fd600 msgr2=0x7fd4b00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3350231148 shutdown_connections 2024-09-10T04:19:39.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.805+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3350231148 wait complete. 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b7bbf640 1 Processor -- start 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b7bbf640 1 -- start start 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b7bbf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0101900 0x7fd4b019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b7bbf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b0107900 0x7fd4b019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b7bbf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b010edd0 con 0x7fd4b0101900 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b7bbf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd4b010ec50 con 0x7fd4b0107900 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b595b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0101900 0x7fd4b019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b595b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0101900 0x7fd4b019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48326/0 (socket says 172.21.15.46:48326) 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b595b640 1 -- 172.21.15.46:0/3652263040 learned_addr learned my addr 172.21.15.46:0/3652263040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b515a640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b0107900 0x7fd4b019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b595b640 1 -- 172.21.15.46:0/3652263040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b0107900 msgr2=0x7fd4b019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b595b640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b0107900 0x7fd4b019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:39.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.806+0000 7fd4b595b640 1 -- 172.21.15.46:0/3652263040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4b01977a0 con 0x7fd4b0101900 2024-09-10T04:19:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.807+0000 7fd4b595b640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0101900 0x7fd4b019d620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac031460 tx=0x7fd4ac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.807+0000 7fd4a6ffd640 1 -- 172.21.15.46:0/3652263040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd4ac03f070 con 0x7fd4b0101900 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.807+0000 7fd4a6ffd640 1 -- 172.21.15.46:0/3652263040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd4ac004b10 con 0x7fd4b0101900 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.807+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4b0197a30 con 0x7fd4b0101900 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.807+0000 7fd4a6ffd640 1 -- 172.21.15.46:0/3652263040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd4ac035030 con 0x7fd4b0101900 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.807+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4b0197f60 con 0x7fd4b0101900 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.808+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4b0103040 con 0x7fd4b0101900 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.809+0000 7fd4a6ffd640 1 -- 172.21.15.46:0/3652263040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd4ac042070 con 0x7fd4b0101900 2024-09-10T04:19:39.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.809+0000 7fd4a6ffd640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd49c07ac00 0x7fd49c07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:19:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.809+0000 7fd4a6ffd640 1 -- 172.21.15.46:0/3652263040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fd4ac0be750 con 0x7fd4b0101900 2024-09-10T04:19:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.809+0000 7fd4b515a640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd49c07ac00 0x7fd49c07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:19:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.810+0000 7fd4b515a640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd49c07ac00 0x7fd49c07d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00ffc20 tx=0x7fd4a000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:19:39.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:39.814+0000 7fd4a6ffd640 1 -- 172.21.15.46:0/3652263040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd4ac089e90 con 0x7fd4b0101900 2024-09-10T04:19:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:39 smithi070 ceph-mon[79774]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:39 smithi070 ceph-mon[79774]: pgmap v419: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:39 smithi070 ceph-mon[79774]: from='client.34650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:39 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2320154310' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:19:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:39 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1202635287' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:19:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.074+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fd4b0109e70 con 0x7fd4b0101900 2024-09-10T04:19:40.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.074+0000 7fd4a6ffd640 1 -- 172.21.15.46:0/3652263040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fd4ac08ed40 con 0x7fd4b0101900 2024-09-10T04:19:40.075 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:19:40.075 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:19:40.075 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:19:40.076 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:19:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:19:40.076 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:19:40.076 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:19:40.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.077+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd49c07ac00 msgr2=0x7fd49c07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.077+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd49c07ac00 0x7fd49c07d0c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd4b00ffc20 tx=0x7fd4a000b390 comp rx=0 tx=0).stop 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.077+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0101900 msgr2=0x7fd4b019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.077+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0101900 0x7fd4b019d620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac031460 tx=0x7fd4ac0045e0 comp rx=0 tx=0).stop 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.078+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 shutdown_connections 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.078+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd49c07ac00 0x7fd49c07d0c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.078+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4b0107900 0x7fd4b019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.078+0000 7fd4b7bbf640 1 --2- 172.21.15.46:0/3652263040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4b0101900 0x7fd4b019d620 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.078+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 >> 172.21.15.46:0/3652263040 conn(0x7fd4b00fd600 msgr2=0x7fd4b00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.078+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 shutdown_connections 2024-09-10T04:19:40.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:19:40.078+0000 7fd4b7bbf640 1 -- 172.21.15.46:0/3652263040 wait complete. 2024-09-10T04:19:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:40 smithi070 ceph-mon[79774]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:40 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3652263040' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:19:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:40 smithi046 ceph-mon[109506]: from='client.34654 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:40 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3652263040' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:19:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:41 smithi046 ceph-mon[109506]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:41 smithi046 ceph-mon[109506]: pgmap v420: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:41 smithi070 ceph-mon[79774]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:19:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:41 smithi070 ceph-mon[79774]: pgmap v420: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:43 smithi070 ceph-mon[79774]: pgmap v421: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:43 smithi046 ceph-mon[109506]: pgmap v421: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:45 smithi070 ceph-mon[79774]: pgmap v422: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:45 smithi046 ceph-mon[109506]: pgmap v422: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:47 smithi046 ceph-mon[109506]: pgmap v423: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:47 smithi070 ceph-mon[79774]: pgmap v423: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:49 smithi070 ceph-mon[79774]: pgmap v424: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:49.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:49 smithi046 ceph-mon[109506]: pgmap v424: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:51 smithi070 ceph-mon[79774]: pgmap v425: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:51 smithi046 ceph-mon[109506]: pgmap v425: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:53 smithi046 ceph-mon[109506]: pgmap v426: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:53 smithi070 ceph-mon[79774]: pgmap v426: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:19:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:55 smithi070 ceph-mon[79774]: pgmap v427: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:55 smithi046 ceph-mon[109506]: pgmap v427: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:57 smithi070 ceph-mon[79774]: pgmap v428: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:57 smithi046 ceph-mon[109506]: pgmap v428: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:19:59 smithi046 ceph-mon[109506]: pgmap v429: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:19:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:19:59 smithi070 ceph-mon[79774]: pgmap v429: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:19:59.998+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:20:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:19:59.998+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:20:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:19:59.998+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-10T04:20:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:19:59.998+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:20:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:19:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:20:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:19:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:20:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:19:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-10T04:20:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:00 smithi070 ceph-mon[79774]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:20:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:00 smithi070 ceph-mon[79774]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:20:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:00 smithi070 ceph-mon[79774]: fs cephfs is degraded 2024-09-10T04:20:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:00 smithi070 ceph-mon[79774]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:20:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:00 smithi070 ceph-mon[79774]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:20:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:00 smithi070 ceph-mon[79774]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:20:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:00 smithi070 ceph-mon[79774]: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:20:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-mon[109506]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:20:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-mon[109506]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:20:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-mon[109506]: fs cephfs is degraded 2024-09-10T04:20:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-mon[109506]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:20:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-mon[109506]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:20:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-mon[109506]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:20:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:00 smithi046 ceph-mon[109506]: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:20:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:01 smithi070 ceph-mon[79774]: pgmap v430: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:01 smithi046 ceph-mon[109506]: pgmap v430: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:03 smithi070 ceph-mon[79774]: pgmap v431: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:03 smithi046 ceph-mon[109506]: pgmap v431: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:05.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:05 smithi046 ceph-mon[109506]: pgmap v432: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:05 smithi070 ceph-mon[79774]: pgmap v432: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:07.809 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:07 smithi046 ceph-mon[109506]: pgmap v433: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:07.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:07 smithi070 ceph-mon[79774]: pgmap v433: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:09.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:09 smithi070 ceph-mon[79774]: pgmap v434: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:09 smithi046 ceph-mon[109506]: pgmap v434: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 -- 172.21.15.46:0/3042921979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f08102ac0 msgr2=0x7f8f08102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/3042921979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f08102ac0 0x7f8f08102ea0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009880 tx=0x7f8ef802a530 comp rx=0 tx=0).stop 2024-09-10T04:20:10.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 -- 172.21.15.46:0/3042921979 shutdown_connections 2024-09-10T04:20:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/3042921979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f081033e0 0x7f8f0810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/3042921979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f08102ac0 0x7f8f08102ea0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 -- 172.21.15.46:0/3042921979 >> 172.21.15.46:0/3042921979 conn(0x7f8f080fc520 msgr2=0x7f8f080fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 -- 172.21.15.46:0/3042921979 shutdown_connections 2024-09-10T04:20:10.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.175+0000 7f8f0d10f640 1 -- 172.21.15.46:0/3042921979 wait complete. 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.176+0000 7f8f0d10f640 1 Processor -- start 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.176+0000 7f8f0d10f640 1 -- start start 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0d10f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f08102ac0 0x7f8f0819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0d10f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f081033e0 0x7f8f0819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0d10f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810edd0 con 0x7f8f081033e0 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0d10f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0810ec50 con 0x7f8f08102ac0 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f081033e0 0x7f8f0819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f081033e0 0x7f8f0819dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39986/0 (socket says 172.21.15.46:39986) 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0659c640 1 -- 172.21.15.46:0/2340386030 learned_addr learned my addr 172.21.15.46:0/2340386030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f06d9d640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f08102ac0 0x7f8f0819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0659c640 1 -- 172.21.15.46:0/2340386030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f08102ac0 msgr2=0x7f8f0819d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0659c640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f08102ac0 0x7f8f0819d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0659c640 1 -- 172.21.15.46:0/2340386030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0819e310 con 0x7f8f081033e0 2024-09-10T04:20:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.177+0000 7f8f0659c640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f081033e0 0x7f8f0819dc60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c8f0 tx=0x7f8efc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.178+0000 7f8eeffff640 1 -- 172.21.15.46:0/2340386030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8efc0079b0 con 0x7f8f081033e0 2024-09-10T04:20:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.178+0000 7f8eeffff640 1 -- 172.21.15.46:0/2340386030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8efc007b50 con 0x7f8f081033e0 2024-09-10T04:20:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.178+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f08197a00 con 0x7f8f081033e0 2024-09-10T04:20:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.178+0000 7f8eeffff640 1 -- 172.21.15.46:0/2340386030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8efc018610 con 0x7f8f081033e0 2024-09-10T04:20:10.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.178+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f08197f60 con 0x7f8f081033e0 2024-09-10T04:20:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.179+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0806aa90 con 0x7f8f081033e0 2024-09-10T04:20:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.180+0000 7f8eeffff640 1 -- 172.21.15.46:0/2340386030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8efc007d00 con 0x7f8f081033e0 2024-09-10T04:20:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.180+0000 7f8eeffff640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ef007abf0 0x7f8ef007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.181+0000 7f8f06d9d640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ef007abf0 0x7f8ef007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.181+0000 7f8eeffff640 1 -- 172.21.15.46:0/2340386030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f8efc09de10 con 0x7f8f081033e0 2024-09-10T04:20:10.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.181+0000 7f8f06d9d640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ef007abf0 0x7f8ef007d0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:10.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.185+0000 7f8eeffff640 1 -- 172.21.15.46:0/2340386030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8efc014030 con 0x7f8f081033e0 2024-09-10T04:20:10.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.393+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8f08198e10 con 0x7f8ef007abf0 2024-09-10T04:20:10.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.394+0000 7f8eeffff640 1 -- 172.21.15.46:0/2340386030 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f8f08198e10 con 0x7f8ef007abf0 2024-09-10T04:20:10.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ef007abf0 msgr2=0x7f8ef007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ef007abf0 0x7f8ef007d0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000c00 tx=0x7f8ef8031400 comp rx=0 tx=0).stop 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f081033e0 msgr2=0x7f8f0819dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f081033e0 0x7f8f0819dc60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c8f0 tx=0x7f8efc00cdb0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 shutdown_connections 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ef007abf0 0x7f8ef007d0b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f081033e0 0x7f8f0819dc60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 --2- 172.21.15.46:0/2340386030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f08102ac0 0x7f8f0819d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.396+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 >> 172.21.15.46:0/2340386030 conn(0x7f8f080fc520 msgr2=0x7f8f08108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.397+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 shutdown_connections 2024-09-10T04:20:10.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.397+0000 7f8f0d10f640 1 -- 172.21.15.46:0/2340386030 wait complete. 2024-09-10T04:20:10.407 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:20:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 -- 172.21.15.46:0/1885728478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 msgr2=0x7f733410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 --2- 172.21.15.46:0/1885728478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733410dc00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7328009880 tx=0x7f732802a530 comp rx=0 tx=0).stop 2024-09-10T04:20:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 -- 172.21.15.46:0/1885728478 shutdown_connections 2024-09-10T04:20:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 --2- 172.21.15.46:0/1885728478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733410dc00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 --2- 172.21.15.46:0/1885728478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7334074620 0x7f7334074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 -- 172.21.15.46:0/1885728478 >> 172.21.15.46:0/1885728478 conn(0x7f73340fd600 msgr2=0x7f73340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 -- 172.21.15.46:0/1885728478 shutdown_connections 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.482+0000 7f733c219640 1 -- 172.21.15.46:0/1885728478 wait complete. 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f733c219640 1 Processor -- start 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f733c219640 1 -- start start 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f733c219640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7334074620 0x7f733419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f733c219640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f733c219640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7334110ff0 con 0x7f7334074f40 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f733c219640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7334110e70 con 0x7f7334074620 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f73397b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f73397b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40008/0 (socket says 172.21.15.46:40008) 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f73397b4640 1 -- 172.21.15.46:0/1412846022 learned_addr learned my addr 172.21.15.46:0/1412846022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:10.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.483+0000 7f7339fb5640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7334074620 0x7f733419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f7339fb5640 1 -- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 msgr2=0x7f733419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f7339fb5640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f7339fb5640 1 -- 172.21.15.46:0/1412846022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7334199a00 con 0x7f7334074620 2024-09-10T04:20:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f73397b4640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:20:10.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f7339fb5640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7334074620 0x7f733419f840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7324009fd0 tx=0x7f732400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f7332ffd640 1 -- 172.21.15.46:0/1412846022 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f732400cc20 con 0x7f7334074620 2024-09-10T04:20:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7334199cf0 con 0x7f7334074620 2024-09-10T04:20:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f733419a250 con 0x7f7334074620 2024-09-10T04:20:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f7332ffd640 1 -- 172.21.15.46:0/1412846022 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f732400cdc0 con 0x7f7334074620 2024-09-10T04:20:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.484+0000 7f7332ffd640 1 -- 172.21.15.46:0/1412846022 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7324004400 con 0x7f7334074620 2024-09-10T04:20:10.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.486+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7320005290 con 0x7f7334074620 2024-09-10T04:20:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.486+0000 7f7332ffd640 1 -- 172.21.15.46:0/1412846022 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7324004680 con 0x7f7334074620 2024-09-10T04:20:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.487+0000 7f7332ffd640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f731c07abf0 0x7f731c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.487+0000 7f73397b4640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f731c07abf0 0x7f731c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.488+0000 7f7332ffd640 1 -- 172.21.15.46:0/1412846022 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f732409c5e0 con 0x7f7334074620 2024-09-10T04:20:10.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.488+0000 7f73397b4640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f731c07abf0 0x7f731c07d0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f733419b1f0 tx=0x7f73280058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:10.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.493+0000 7f7332ffd640 1 -- 172.21.15.46:0/1412846022 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7324069980 con 0x7f7334074620 2024-09-10T04:20:10.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.698+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7320000bc0 con 0x7f731c07abf0 2024-09-10T04:20:10.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.699+0000 7f7332ffd640 1 -- 172.21.15.46:0/1412846022 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f7320000bc0 con 0x7f731c07abf0 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f731c07abf0 msgr2=0x7f731c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f731c07abf0 0x7f731c07d0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f733419b1f0 tx=0x7f73280058d0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7334074620 msgr2=0x7f733419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7334074620 0x7f733419f840 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7324009fd0 tx=0x7f732400ef10 comp rx=0 tx=0).stop 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 shutdown_connections 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f731c07abf0 0x7f731c07d0b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7334074f40 0x7f733419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 --2- 172.21.15.46:0/1412846022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7334074620 0x7f733419f840 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 >> 172.21.15.46:0/1412846022 conn(0x7f73340fd600 msgr2=0x7f733410b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 shutdown_connections 2024-09-10T04:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.701+0000 7f733c219640 1 -- 172.21.15.46:0/1412846022 wait complete. 2024-09-10T04:20:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1524722506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 msgr2=0x7f9b34101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1524722506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b34101d60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009880 tx=0x7f9b2402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1524722506 shutdown_connections 2024-09-10T04:20:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1524722506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b34101d60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1524722506 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b34107900 0x7f9b34107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1524722506 >> 172.21.15.46:0/1524722506 conn(0x7f9b340fd600 msgr2=0x7f9b340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:10.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1524722506 shutdown_connections 2024-09-10T04:20:10.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.785+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1524722506 wait complete. 2024-09-10T04:20:10.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b3b6a4640 1 Processor -- start 2024-09-10T04:20:10.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b3b6a4640 1 -- start start 2024-09-10T04:20:10.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b3b6a4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b3419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b3b6a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b34107900 0x7f9b3419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b3b6a4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3410edb0 con 0x7f9b34101900 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b3b6a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3410ec30 con 0x7f9b34107900 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b39440640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b3419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b39440640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b3419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40024/0 (socket says 172.21.15.46:40024) 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.786+0000 7f9b39440640 1 -- 172.21.15.46:0/1311830633 learned_addr learned my addr 172.21.15.46:0/1311830633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b38c3f640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b34107900 0x7f9b3419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b39440640 1 -- 172.21.15.46:0/1311830633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b34107900 msgr2=0x7f9b3419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b39440640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b34107900 0x7f9b3419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b39440640 1 -- 172.21.15.46:0/1311830633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b34197800 con 0x7f9b34101900 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b38c3f640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b34107900 0x7f9b3419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b39440640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b3419d680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c8a0 tx=0x7f9b3000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b2a7fc640 1 -- 172.21.15.46:0/1311830633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b300079b0 con 0x7f9b34101900 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b34197af0 con 0x7f9b34101900 2024-09-10T04:20:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b34198050 con 0x7f9b34101900 2024-09-10T04:20:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b2a7fc640 1 -- 172.21.15.46:0/1311830633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9b30007b50 con 0x7f9b34101900 2024-09-10T04:20:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.787+0000 7f9b2a7fc640 1 -- 172.21.15.46:0/1311830633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9b30018700 con 0x7f9b34101900 2024-09-10T04:20:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.788+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b18005290 con 0x7f9b34101900 2024-09-10T04:20:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.789+0000 7f9b2a7fc640 1 -- 172.21.15.46:0/1311830633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9b30007d00 con 0x7f9b34101900 2024-09-10T04:20:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.790+0000 7f9b2a7fc640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b2007ac40 0x7f9b2007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.790+0000 7f9b2a7fc640 1 -- 172.21.15.46:0/1311830633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f9b3009de60 con 0x7f9b34101900 2024-09-10T04:20:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.790+0000 7f9b38c3f640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b2007ac40 0x7f9b2007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:10.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.790+0000 7f9b38c3f640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b2007ac40 0x7f9b2007d100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b34198b50 tx=0x7f9b24009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:10.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.794+0000 7f9b2a7fc640 1 -- 172.21.15.46:0/1311830633 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9b30014030 con 0x7f9b34101900 2024-09-10T04:20:10.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.989+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9b18000bc0 con 0x7f9b2007ac40 2024-09-10T04:20:11.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:10.999+0000 7f9b2a7fc640 1 -- 172.21.15.46:0/1311830633 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f9b18000bc0 con 0x7f9b2007ac40 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (11m) 6m ago 23m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (14m) 6m ago 23m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (14m) 6m ago 22m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (10m) 6m ago 23m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (10m) 6m ago 22m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (11m) 6m ago 22m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (6m) 6m ago 17m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (6m) 6m ago 17m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:20:11.001 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (6m) 6m ago 17m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (17m) 6m ago 17m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (13m) 6m ago 24m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (12m) 6m ago 22m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (10m) 6m ago 24m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (10m) 6m ago 22m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (12m) 6m ago 23m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (12m) 6m ago 22m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (9m) 6m ago 21m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (9m) 6m ago 20m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (8m) 6m ago 20m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (7m) 6m ago 19m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (7m) 6m ago 19m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (6m) 6m ago 18m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:20:11.002 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (12m) 6m ago 22m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:20:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.001+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b2007ac40 msgr2=0x7f9b2007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.001+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b2007ac40 0x7f9b2007d100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9b34198b50 tx=0x7f9b24009560 comp rx=0 tx=0).stop 2024-09-10T04:20:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.001+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 msgr2=0x7f9b3419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.001+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b3419d680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c8a0 tx=0x7f9b3000cd60 comp rx=0 tx=0).stop 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.001+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 shutdown_connections 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.001+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9b2007ac40 0x7f9b2007d100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.001+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9b34107900 0x7f9b3419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.002+0000 7f9b3b6a4640 1 --2- 172.21.15.46:0/1311830633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b34101900 0x7f9b3419d680 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.002+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 >> 172.21.15.46:0/1311830633 conn(0x7f9b340fd600 msgr2=0x7f9b340fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.002+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 shutdown_connections 2024-09-10T04:20:11.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.002+0000 7f9b3b6a4640 1 -- 172.21.15.46:0/1311830633 wait complete. 2024-09-10T04:20:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 -- 172.21.15.46:0/3737353751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 msgr2=0x7f70dc106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 --2- 172.21.15.46:0/3737353751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc106bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009880 tx=0x7f70d802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 -- 172.21.15.46:0/3737353751 shutdown_connections 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 --2- 172.21.15.46:0/3737353751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70dc1007f0 0x7f70dc100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 --2- 172.21.15.46:0/3737353751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc106bd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 -- 172.21.15.46:0/3737353751 >> 172.21.15.46:0/3737353751 conn(0x7f70dc0fc550 msgr2=0x7f70dc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 -- 172.21.15.46:0/3737353751 shutdown_connections 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 -- 172.21.15.46:0/3737353751 wait complete. 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.085+0000 7f70e36b7640 1 Processor -- start 2024-09-10T04:20:11.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e36b7640 1 -- start start 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e36b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70dc1007f0 0x7f70dc0fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e36b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc0fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e36b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc10dcc0 con 0x7f70dc1067f0 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e36b7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f70dc10db40 con 0x7f70dc1007f0 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e0c52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc0fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e0c52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc0fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40038/0 (socket says 172.21.15.46:40038) 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e0c52640 1 -- 172.21.15.46:0/1963562453 learned_addr learned my addr 172.21.15.46:0/1963562453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e1453640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70dc1007f0 0x7f70dc0fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e0c52640 1 -- 172.21.15.46:0/1963562453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70dc1007f0 msgr2=0x7f70dc0fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e0c52640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70dc1007f0 0x7f70dc0fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.086+0000 7f70e0c52640 1 -- 172.21.15.46:0/1963562453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70dc0fd750 con 0x7f70dc1067f0 2024-09-10T04:20:11.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.087+0000 7f70e0c52640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc0fd210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c8f0 tx=0x7f70cc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.087+0000 7f70d27fc640 1 -- 172.21.15.46:0/1963562453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f70cc0079b0 con 0x7f70dc1067f0 2024-09-10T04:20:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.087+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70dc0fda40 con 0x7f70dc1067f0 2024-09-10T04:20:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.087+0000 7f70d27fc640 1 -- 172.21.15.46:0/1963562453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f70cc0054e0 con 0x7f70dc1067f0 2024-09-10T04:20:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.087+0000 7f70d27fc640 1 -- 172.21.15.46:0/1963562453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f70cc004dd0 con 0x7f70dc1067f0 2024-09-10T04:20:11.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.087+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70dc1a3e30 con 0x7f70dc1067f0 2024-09-10T04:20:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.088+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70dc101f30 con 0x7f70dc1067f0 2024-09-10T04:20:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.089+0000 7f70d27fc640 1 -- 172.21.15.46:0/1963562453 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f70cc005020 con 0x7f70dc1067f0 2024-09-10T04:20:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.089+0000 7f70d27fc640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f70c807ab90 0x7f70c807d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.090+0000 7f70d27fc640 1 -- 172.21.15.46:0/1963562453 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f70cc09dba0 con 0x7f70dc1067f0 2024-09-10T04:20:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.090+0000 7f70e1453640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f70c807ab90 0x7f70c807d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.090+0000 7f70e1453640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f70c807ab90 0x7f70c807d050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70d8000c00 tx=0x7f70d8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:11.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.094+0000 7f70d27fc640 1 -- 172.21.15.46:0/1963562453 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f70cc014030 con 0x7f70dc1067f0 2024-09-10T04:20:11.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.356+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f70dc109d70 con 0x7f70dc1067f0 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.356+0000 7f70d27fc640 1 -- 172.21.15.46:0/1963562453 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f70cc0692e0 con 0x7f70dc1067f0 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:20:11.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:20:11.359 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:20:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f70c807ab90 msgr2=0x7f70c807d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f70c807ab90 0x7f70c807d050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f70d8000c00 tx=0x7f70d8031400 comp rx=0 tx=0).stop 2024-09-10T04:20:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 msgr2=0x7f70dc0fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc0fd210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00c8f0 tx=0x7f70cc00cdb0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 shutdown_connections 2024-09-10T04:20:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f70c807ab90 0x7f70c807d050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70dc1067f0 0x7f70dc0fd210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 --2- 172.21.15.46:0/1963562453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70dc1007f0 0x7f70dc0fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.358+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 >> 172.21.15.46:0/1963562453 conn(0x7f70dc0fc550 msgr2=0x7f70dc069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.359+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 shutdown_connections 2024-09-10T04:20:11.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.359+0000 7f70e36b7640 1 -- 172.21.15.46:0/1963562453 wait complete. 2024-09-10T04:20:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 -- 172.21.15.46:0/3117503629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 msgr2=0x7f25a8102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3117503629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a8102f00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f25980098b0 tx=0x7f259802a560 comp rx=0 tx=0).stop 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 -- 172.21.15.46:0/3117503629 shutdown_connections 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3117503629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a8103440 0x7f25a810b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3117503629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a8102f00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 -- 172.21.15.46:0/3117503629 >> 172.21.15.46:0/3117503629 conn(0x7f25a80fc520 msgr2=0x7f25a80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 -- 172.21.15.46:0/3117503629 shutdown_connections 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 -- 172.21.15.46:0/3117503629 wait complete. 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 Processor -- start 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.443+0000 7f25ad33c640 1 -- start start 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25ad33c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25ad33c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a8103440 0x7f25a819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25ad33c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25a810edf0 con 0x7f25a8102b20 2024-09-10T04:20:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25ad33c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f25a810ec70 con 0x7f25a8103440 2024-09-10T04:20:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40064/0 (socket says 172.21.15.46:40064) 2024-09-10T04:20:11.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a6ffd640 1 -- 172.21.15.46:0/3127285568 learned_addr learned my addr 172.21.15.46:0/3127285568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a67fc640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a8103440 0x7f25a819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a6ffd640 1 -- 172.21.15.46:0/3127285568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a8103440 msgr2=0x7f25a819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a6ffd640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a8103440 0x7f25a819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a6ffd640 1 -- 172.21.15.46:0/3127285568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a8197820 con 0x7f25a8102b20 2024-09-10T04:20:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a67fc640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a8103440 0x7f25a819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:20:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f25a6ffd640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a819d6a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2598031460 tx=0x7f25980045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:11.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.444+0000 7f259ffff640 1 -- 172.21.15.46:0/3127285568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f259803f070 con 0x7f25a8102b20 2024-09-10T04:20:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.445+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a8197ab0 con 0x7f25a8102b20 2024-09-10T04:20:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.445+0000 7f259ffff640 1 -- 172.21.15.46:0/3127285568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f25980035f0 con 0x7f25a8102b20 2024-09-10T04:20:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.445+0000 7f259ffff640 1 -- 172.21.15.46:0/3127285568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2598035030 con 0x7f25a8102b20 2024-09-10T04:20:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.445+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a8197fe0 con 0x7f25a8102b20 2024-09-10T04:20:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.446+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a806aa30 con 0x7f25a8102b20 2024-09-10T04:20:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.446+0000 7f259ffff640 1 -- 172.21.15.46:0/3127285568 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2598042070 con 0x7f25a8102b20 2024-09-10T04:20:11.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.447+0000 7f259ffff640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f259407abf0 0x7f259407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.447+0000 7f259ffff640 1 -- 172.21.15.46:0/3127285568 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f25980be6a0 con 0x7f25a8102b20 2024-09-10T04:20:11.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.447+0000 7f25a67fc640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f259407abf0 0x7f259407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.447+0000 7f25a67fc640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f259407abf0 0x7f259407d0b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f25a8108960 tx=0x7f2590005e50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.451+0000 7f259ffff640 1 -- 172.21.15.46:0/3127285568 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2598089d40 con 0x7f25a8102b20 2024-09-10T04:20:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.674+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f25a8198d80 con 0x7f25a8102b20 2024-09-10T04:20:11.677 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:11 smithi046 ceph-mon[109506]: pgmap v435: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:11.677 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:11 smithi046 ceph-mon[109506]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:11.677 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:11 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1963562453' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:20:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.675+0000 7f259ffff640 1 -- 172.21.15.46:0/3127285568 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f25a8198d80 con 0x7f25a8102b20 2024-09-10T04:20:11.677 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:20:11.678 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:20:11.679 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:11.681 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:20:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f259407abf0 msgr2=0x7f259407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f259407abf0 0x7f259407d0b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f25a8108960 tx=0x7f2590005e50 comp rx=0 tx=0).stop 2024-09-10T04:20:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 msgr2=0x7f25a819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a819d6a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2598031460 tx=0x7f25980045c0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 shutdown_connections 2024-09-10T04:20:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f259407abf0 0x7f259407d0b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25a8103440 0x7f25a819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 --2- 172.21.15.46:0/3127285568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a8102b20 0x7f25a819d6a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 >> 172.21.15.46:0/3127285568 conn(0x7f25a80fc520 msgr2=0x7f25a8103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.677+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 shutdown_connections 2024-09-10T04:20:11.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.678+0000 7f25ad33c640 1 -- 172.21.15.46:0/3127285568 wait complete. 2024-09-10T04:20:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 -- 172.21.15.46:0/1788394093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8107900 msgr2=0x7f63e8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 --2- 172.21.15.46:0/1788394093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8107900 0x7f63e8107ce0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009880 tx=0x7f63e402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 -- 172.21.15.46:0/1788394093 shutdown_connections 2024-09-10T04:20:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 --2- 172.21.15.46:0/1788394093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63e8101900 0x7f63e8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 --2- 172.21.15.46:0/1788394093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8107900 0x7f63e8107ce0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 -- 172.21.15.46:0/1788394093 >> 172.21.15.46:0/1788394093 conn(0x7f63e80fd600 msgr2=0x7f63e80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 -- 172.21.15.46:0/1788394093 shutdown_connections 2024-09-10T04:20:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.763+0000 7f63f0573640 1 -- 172.21.15.46:0/1788394093 wait complete. 2024-09-10T04:20:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.764+0000 7f63f0573640 1 Processor -- start 2024-09-10T04:20:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.764+0000 7f63f0573640 1 -- start start 2024-09-10T04:20:11.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.764+0000 7f63f0573640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8101900 0x7f63e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.764+0000 7f63f0573640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63e8107900 0x7f63e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.764+0000 7f63f0573640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f63e810edb0 con 0x7f63e8101900 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.764+0000 7f63f0573640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f63e810ec30 con 0x7f63e8107900 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.764+0000 7f63ee30f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8101900 0x7f63e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63ee30f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8101900 0x7f63e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40084/0 (socket says 172.21.15.46:40084) 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63ee30f640 1 -- 172.21.15.46:0/2833233774 learned_addr learned my addr 172.21.15.46:0/2833233774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63edb0e640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63e8107900 0x7f63e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63ee30f640 1 -- 172.21.15.46:0/2833233774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63e8107900 msgr2=0x7f63e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63ee30f640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63e8107900 0x7f63e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63ee30f640 1 -- 172.21.15.46:0/2833233774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e8074f70 con 0x7f63e8101900 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63ee30f640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8101900 0x7f63e80744f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f63e4031460 tx=0x7f63e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63df7fe640 1 -- 172.21.15.46:0/2833233774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f63e403f070 con 0x7f63e8101900 2024-09-10T04:20:11.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63df7fe640 1 -- 172.21.15.46:0/2833233774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63e40035f0 con 0x7f63e8101900 2024-09-10T04:20:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63df7fe640 1 -- 172.21.15.46:0/2833233774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f63e4035030 con 0x7f63e8101900 2024-09-10T04:20:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e8075200 con 0x7f63e8101900 2024-09-10T04:20:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.765+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e81a3da0 con 0x7f63e8101900 2024-09-10T04:20:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.767+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e8103040 con 0x7f63e8101900 2024-09-10T04:20:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.767+0000 7f63df7fe640 1 -- 172.21.15.46:0/2833233774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f63e4042070 con 0x7f63e8101900 2024-09-10T04:20:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.768+0000 7f63df7fe640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f63d407abf0 0x7f63d407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.768+0000 7f63edb0e640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f63d407abf0 0x7f63d407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.768+0000 7f63df7fe640 1 -- 172.21.15.46:0/2833233774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f63e40be5d0 con 0x7f63e8101900 2024-09-10T04:20:11.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.768+0000 7f63edb0e640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f63d407abf0 0x7f63d407d0b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63e8075f80 tx=0x7f63d800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:11.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.772+0000 7f63df7fe640 1 -- 172.21.15.46:0/2833233774 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f63e4089d10 con 0x7f63e8101900 2024-09-10T04:20:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:11 smithi070 ceph-mon[79774]: pgmap v435: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:11 smithi070 ceph-mon[79774]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:11 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1963562453' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:20:11.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.980+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f63e8062630 con 0x7f63d407abf0 2024-09-10T04:20:11.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.981+0000 7f63df7fe640 1 -- 172.21.15.46:0/2833233774 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f63e8062630 con 0x7f63d407abf0 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:20:11.988 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:20:11.989 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:20:11.989 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:20:11.989 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:20:11.989 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:20:11.989 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f63d407abf0 msgr2=0x7f63d407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f63d407abf0 0x7f63d407d0b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f63e8075f80 tx=0x7f63d800b390 comp rx=0 tx=0).stop 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8101900 msgr2=0x7f63e80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8101900 0x7f63e80744f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f63e4031460 tx=0x7f63e40045e0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 shutdown_connections 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f63d407abf0 0x7f63d407d0b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63e8107900 0x7f63e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 --2- 172.21.15.46:0/2833233774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63e8101900 0x7f63e80744f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:11.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 >> 172.21.15.46:0/2833233774 conn(0x7f63e80fd600 msgr2=0x7f63e80fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:11.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 shutdown_connections 2024-09-10T04:20:11.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:11.984+0000 7f63f0573640 1 -- 172.21.15.46:0/2833233774 wait complete. 2024-09-10T04:20:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 -- 172.21.15.46:0/3655326866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 msgr2=0x7f78e8105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/3655326866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8105c40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009880 tx=0x7f78d802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:20:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 -- 172.21.15.46:0/3655326866 shutdown_connections 2024-09-10T04:20:12.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/3655326866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8101010 0x7f78e8101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/3655326866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8105c40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 -- 172.21.15.46:0/3655326866 >> 172.21.15.46:0/3655326866 conn(0x7f78e80fbc90 msgr2=0x7f78e80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 -- 172.21.15.46:0/3655326866 shutdown_connections 2024-09-10T04:20:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.068+0000 7f78ee5e6640 1 -- 172.21.15.46:0/3655326866 wait complete. 2024-09-10T04:20:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78ee5e6640 1 Processor -- start 2024-09-10T04:20:12.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78ee5e6640 1 -- start start 2024-09-10T04:20:12.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78ee5e6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8101010 0x7f78e8196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78ee5e6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78ee5e6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810de20 con 0x7f78e8105860 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78ee5e6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f78e810dca0 con 0x7f78e8101010 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78e77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78e77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40106/0 (socket says 172.21.15.46:40106) 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78e77fe640 1 -- 172.21.15.46:0/4247746748 learned_addr learned my addr 172.21.15.46:0/4247746748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.069+0000 7f78e7fff640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8101010 0x7f78e8196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78e77fe640 1 -- 172.21.15.46:0/4247746748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8101010 msgr2=0x7f78e8196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78e77fe640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8101010 0x7f78e8196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78e77fe640 1 -- 172.21.15.46:0/4247746748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e81971a0 con 0x7f78e8105860 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78e77fe640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8196c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c8a0 tx=0x7f78dc00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78e57fa640 1 -- 172.21.15.46:0/4247746748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f78dc0079b0 con 0x7f78e8105860 2024-09-10T04:20:12.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78e57fa640 1 -- 172.21.15.46:0/4247746748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f78dc007b50 con 0x7f78e8105860 2024-09-10T04:20:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e8197490 con 0x7f78e8105860 2024-09-10T04:20:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78e57fa640 1 -- 172.21.15.46:0/4247746748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f78dc018700 con 0x7f78e8105860 2024-09-10T04:20:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.070+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81a3f60 con 0x7f78e8105860 2024-09-10T04:20:12.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.071+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e8102020 con 0x7f78e8105860 2024-09-10T04:20:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.072+0000 7f78e57fa640 1 -- 172.21.15.46:0/4247746748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f78dc007d00 con 0x7f78e8105860 2024-09-10T04:20:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.072+0000 7f78e57fa640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78d007abf0 0x7f78d007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.072+0000 7f78e57fa640 1 -- 172.21.15.46:0/4247746748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f78dc09def0 con 0x7f78e8105860 2024-09-10T04:20:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.072+0000 7f78e7fff640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78d007abf0 0x7f78d007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:12.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.073+0000 7f78e7fff640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78d007abf0 0x7f78d007d0b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78d80096c0 tx=0x7f78d8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.077+0000 7f78e57fa640 1 -- 172.21.15.46:0/4247746748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f78dc014030 con 0x7f78e8105860 2024-09-10T04:20:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.337+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f78e8108d80 con 0x7f78e8105860 2024-09-10T04:20:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.337+0000 7f78e57fa640 1 -- 172.21.15.46:0/4247746748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f78dc069630 con 0x7f78e8105860 2024-09-10T04:20:12.339 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:20:12.339 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:20:12.339 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:20:12.340 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:20:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:20:12.340 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:20:12.340 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:20:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.339+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78d007abf0 msgr2=0x7f78d007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.339+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78d007abf0 0x7f78d007d0b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f78d80096c0 tx=0x7f78d8031400 comp rx=0 tx=0).stop 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 msgr2=0x7f78e8196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8196c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00c8a0 tx=0x7f78dc00cd60 comp rx=0 tx=0).stop 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 shutdown_connections 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78d007abf0 0x7f78d007d0b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e8105860 0x7f78e8196c60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 --2- 172.21.15.46:0/4247746748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e8101010 0x7f78e8196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 >> 172.21.15.46:0/4247746748 conn(0x7f78e80fbc90 msgr2=0x7f78e80fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 shutdown_connections 2024-09-10T04:20:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:12.340+0000 7f78ee5e6640 1 -- 172.21.15.46:0/4247746748 wait complete. 2024-09-10T04:20:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:12 smithi070 ceph-mon[79774]: from='client.44593 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:12 smithi070 ceph-mon[79774]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:12 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3127285568' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:20:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:12 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4247746748' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:20:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:12 smithi046 ceph-mon[109506]: from='client.44593 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:12 smithi046 ceph-mon[109506]: from='client.34682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:12 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3127285568' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:20:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:12 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4247746748' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:20:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:13 smithi070 ceph-mon[79774]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:13 smithi070 ceph-mon[79774]: pgmap v436: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:13 smithi046 ceph-mon[109506]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:13.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:13 smithi046 ceph-mon[109506]: pgmap v436: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:15 smithi070 ceph-mon[79774]: pgmap v437: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:15 smithi046 ceph-mon[109506]: pgmap v437: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:17.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:17 smithi046 ceph-mon[109506]: pgmap v438: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:17 smithi070 ceph-mon[79774]: pgmap v438: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:19 smithi070 ceph-mon[79774]: pgmap v439: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:19 smithi046 ceph-mon[109506]: pgmap v439: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:21 smithi070 ceph-mon[79774]: pgmap v440: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:21 smithi046 ceph-mon[109506]: pgmap v440: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:23 smithi070 ceph-mon[79774]: pgmap v441: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:23 smithi046 ceph-mon[109506]: pgmap v441: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:25 smithi070 ceph-mon[79774]: pgmap v442: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:25 smithi046 ceph-mon[109506]: pgmap v442: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:27 smithi070 ceph-mon[79774]: pgmap v443: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:27 smithi046 ceph-mon[109506]: pgmap v443: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:29 smithi046 ceph-mon[109506]: pgmap v444: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:29 smithi070 ceph-mon[79774]: pgmap v444: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:31 smithi070 ceph-mon[79774]: pgmap v445: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:31 smithi046 ceph-mon[109506]: pgmap v445: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:33 smithi070 ceph-mon[79774]: pgmap v446: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:33 smithi046 ceph-mon[109506]: pgmap v446: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:35 smithi046 ceph-mon[109506]: pgmap v447: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:35 smithi070 ceph-mon[79774]: pgmap v447: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:37 smithi070 ceph-mon[79774]: pgmap v448: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:37 smithi046 ceph-mon[109506]: pgmap v448: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:39 smithi070 ceph-mon[79774]: pgmap v449: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:39 smithi046 ceph-mon[109506]: pgmap v449: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:41.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:41 smithi070 ceph-mon[79774]: pgmap v450: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:41.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:41 smithi046 ceph-mon[109506]: pgmap v450: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:42.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 -- 172.21.15.46:0/1815932908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10103480 msgr2=0x7f8c1010b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1815932908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10103480 0x7f8c1010b970 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098b0 tx=0x7f8c0402a560 comp rx=0 tx=0).stop 2024-09-10T04:20:42.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 -- 172.21.15.46:0/1815932908 shutdown_connections 2024-09-10T04:20:42.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1815932908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10103480 0x7f8c1010b970 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1815932908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10102b60 0x7f8c10102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 -- 172.21.15.46:0/1815932908 >> 172.21.15.46:0/1815932908 conn(0x7f8c100fc580 msgr2=0x7f8c100fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:42.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 -- 172.21.15.46:0/1815932908 shutdown_connections 2024-09-10T04:20:42.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.438+0000 7f8c159a8640 1 -- 172.21.15.46:0/1815932908 wait complete. 2024-09-10T04:20:42.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c159a8640 1 Processor -- start 2024-09-10T04:20:42.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c159a8640 1 -- start start 2024-09-10T04:20:42.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c159a8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10102b60 0x7f8c1019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:42.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c159a8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10103480 0x7f8c1019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:42.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c159a8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010ee30 con 0x7f8c10102b60 2024-09-10T04:20:42.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c159a8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010ecb0 con 0x7f8c10103480 2024-09-10T04:20:42.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c0effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10102b60 0x7f8c1019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c0effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10102b60 0x7f8c1019f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44654/0 (socket says 172.21.15.46:44654) 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.439+0000 7f8c0effd640 1 -- 172.21.15.46:0/1647308230 learned_addr learned my addr 172.21.15.46:0/1647308230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c0e7fc640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10103480 0x7f8c1019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c0effd640 1 -- 172.21.15.46:0/1647308230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10103480 msgr2=0x7f8c1019fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c0effd640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10103480 0x7f8c1019fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c0effd640 1 -- 172.21.15.46:0/1647308230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c10199a50 con 0x7f8c10102b60 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c0effd640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10102b60 0x7f8c1019f890 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000d8d0 tx=0x7f8c0000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c149a6640 1 -- 172.21.15.46:0/1647308230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8c00017070 con 0x7f8c10102b60 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c10199d40 con 0x7f8c10102b60 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c149a6640 1 -- 172.21.15.46:0/1647308230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c00010460 con 0x7f8c10102b60 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c149a6640 1 -- 172.21.15.46:0/1647308230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8c0000f5f0 con 0x7f8c10102b60 2024-09-10T04:20:42.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.440+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1019a2a0 con 0x7f8c10102b60 2024-09-10T04:20:42.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.441+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1006ad60 con 0x7f8c10102b60 2024-09-10T04:20:42.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.442+0000 7f8c149a6640 1 -- 172.21.15.46:0/1647308230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8c00009a30 con 0x7f8c10102b60 2024-09-10T04:20:42.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.443+0000 7f8c149a6640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bf807abf0 0x7f8bf807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:42.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.443+0000 7f8c0e7fc640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bf807abf0 0x7f8bf807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:42.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.443+0000 7f8c149a6640 1 -- 172.21.15.46:0/1647308230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f8c0009db60 con 0x7f8c10102b60 2024-09-10T04:20:42.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.443+0000 7f8c0e7fc640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bf807abf0 0x7f8bf807d0b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c04000c00 tx=0x7f8c04031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:42.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.447+0000 7f8c149a6640 1 -- 172.21.15.46:0/1647308230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8c000692a0 con 0x7f8c10102b60 2024-09-10T04:20:42.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.650+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8c1019af80 con 0x7f8bf807abf0 2024-09-10T04:20:42.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.651+0000 7f8c149a6640 1 -- 172.21.15.46:0/1647308230 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f8c1019af80 con 0x7f8bf807abf0 2024-09-10T04:20:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bf807abf0 msgr2=0x7f8bf807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bf807abf0 0x7f8bf807d0b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8c04000c00 tx=0x7f8c04031400 comp rx=0 tx=0).stop 2024-09-10T04:20:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10102b60 msgr2=0x7f8c1019f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10102b60 0x7f8c1019f890 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000d8d0 tx=0x7f8c0000dd90 comp rx=0 tx=0).stop 2024-09-10T04:20:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 shutdown_connections 2024-09-10T04:20:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bf807abf0 0x7f8bf807d0b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c10103480 0x7f8c1019fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 --2- 172.21.15.46:0/1647308230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c10102b60 0x7f8c1019f890 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.653+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 >> 172.21.15.46:0/1647308230 conn(0x7f8c100fc580 msgr2=0x7f8c1010b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.654+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 shutdown_connections 2024-09-10T04:20:42.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.654+0000 7f8c159a8640 1 -- 172.21.15.46:0/1647308230 wait complete. 2024-09-10T04:20:42.664 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.739+0000 7f20faef4640 1 -- 172.21.15.46:0/3891308422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 msgr2=0x7f20f4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.739+0000 7f20faef4640 1 --2- 172.21.15.46:0/3891308422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f4100c60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009880 tx=0x7f20e002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.739+0000 7f20faef4640 1 -- 172.21.15.46:0/3891308422 shutdown_connections 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.739+0000 7f20faef4640 1 --2- 172.21.15.46:0/3891308422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f4100c60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.739+0000 7f20faef4640 1 --2- 172.21.15.46:0/3891308422 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20f4106800 0x7f20f4106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.739+0000 7f20faef4640 1 -- 172.21.15.46:0/3891308422 >> 172.21.15.46:0/3891308422 conn(0x7f20f40fc520 msgr2=0x7f20f40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 -- 172.21.15.46:0/3891308422 shutdown_connections 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 -- 172.21.15.46:0/3891308422 wait complete. 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 Processor -- start 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 -- start start 2024-09-10T04:20:42.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20f4106800 0x7f20f419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20f410dcb0 con 0x7f20f4100800 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.740+0000 7f20faef4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f20f410db30 con 0x7f20f4106800 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f8c90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f8c90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44662/0 (socket says 172.21.15.46:44662) 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f8c90640 1 -- 172.21.15.46:0/3648017153 learned_addr learned my addr 172.21.15.46:0/3648017153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f3fff640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20f4106800 0x7f20f419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f8c90640 1 -- 172.21.15.46:0/3648017153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20f4106800 msgr2=0x7f20f419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f8c90640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20f4106800 0x7f20f419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f8c90640 1 -- 172.21.15.46:0/3648017153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f41977a0 con 0x7f20f4100800 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f8c90640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f419d620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c8a0 tx=0x7f20e800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:42.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f1ffb640 1 -- 172.21.15.46:0/3648017153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f20e80079b0 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f4197a90 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f1ffb640 1 -- 172.21.15.46:0/3648017153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20e8007b50 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.741+0000 7f20f1ffb640 1 -- 172.21.15.46:0/3648017153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f20e8018700 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.742+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f4197ff0 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.743+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f4101f40 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.743+0000 7f20f1ffb640 1 -- 172.21.15.46:0/3648017153 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f20e8007d00 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.744+0000 7f20f1ffb640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f20e407ac00 0x7f20e407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.744+0000 7f20f1ffb640 1 -- 172.21.15.46:0/3648017153 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f20e809df90 con 0x7f20f4100800 2024-09-10T04:20:42.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.744+0000 7f20f3fff640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f20e407ac00 0x7f20e407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:42.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.745+0000 7f20f3fff640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f20e407ac00 0x7f20e407d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20f4198af0 tx=0x7f20e0009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:42.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.749+0000 7f20f1ffb640 1 -- 172.21.15.46:0/3648017153 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f20e8014030 con 0x7f20f4100800 2024-09-10T04:20:42.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.950+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f20f4198300 con 0x7f20e407ac00 2024-09-10T04:20:42.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.952+0000 7f20f1ffb640 1 -- 172.21.15.46:0/3648017153 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f20f4198300 con 0x7f20e407ac00 2024-09-10T04:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f20e407ac00 msgr2=0x7f20e407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f20e407ac00 0x7f20e407d0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20f4198af0 tx=0x7f20e0009560 comp rx=0 tx=0).stop 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 msgr2=0x7f20f419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f419d620 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20e800c8a0 tx=0x7f20e800cd60 comp rx=0 tx=0).stop 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 shutdown_connections 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f20e407ac00 0x7f20e407d0c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f20f4106800 0x7f20f419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 --2- 172.21.15.46:0/3648017153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f4100800 0x7f20f419d620 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 >> 172.21.15.46:0/3648017153 conn(0x7f20f40fc520 msgr2=0x7f20f406a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 shutdown_connections 2024-09-10T04:20:42.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:42.954+0000 7f20faef4640 1 -- 172.21.15.46:0/3648017153 wait complete. 2024-09-10T04:20:43.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.038+0000 7f08125e1640 1 -- 172.21.15.46:0/515458138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c1033e0 msgr2=0x7f080c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.038+0000 7f08125e1640 1 --2- 172.21.15.46:0/515458138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c1033e0 0x7f080c10b8d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0800009880 tx=0x7f080002a530 comp rx=0 tx=0).stop 2024-09-10T04:20:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.038+0000 7f08125e1640 1 -- 172.21.15.46:0/515458138 shutdown_connections 2024-09-10T04:20:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.038+0000 7f08125e1640 1 --2- 172.21.15.46:0/515458138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c1033e0 0x7f080c10b8d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.038+0000 7f08125e1640 1 --2- 172.21.15.46:0/515458138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080c102ac0 0x7f080c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.038+0000 7f08125e1640 1 -- 172.21.15.46:0/515458138 >> 172.21.15.46:0/515458138 conn(0x7f080c0fc520 msgr2=0x7f080c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.039+0000 7f08125e1640 1 -- 172.21.15.46:0/515458138 shutdown_connections 2024-09-10T04:20:43.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.039+0000 7f08125e1640 1 -- 172.21.15.46:0/515458138 wait complete. 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.039+0000 7f08125e1640 1 Processor -- start 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.039+0000 7f08125e1640 1 -- start start 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f08125e1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c102ac0 0x7f080c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f08125e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080c1033e0 0x7f080c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f08125e1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f080c10edf0 con 0x7f080c102ac0 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f08125e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f080c10ec70 con 0x7f080c1033e0 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c102ac0 0x7f080c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c102ac0 0x7f080c19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44684/0 (socket says 172.21.15.46:44684) 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080bfff640 1 -- 172.21.15.46:0/1853423115 learned_addr learned my addr 172.21.15.46:0/1853423115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080b7fe640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080c1033e0 0x7f080c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080bfff640 1 -- 172.21.15.46:0/1853423115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080c1033e0 msgr2=0x7f080c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080bfff640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080c1033e0 0x7f080c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080bfff640 1 -- 172.21.15.46:0/1853423115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c197800 con 0x7f080c102ac0 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.040+0000 7f080b7fe640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080c1033e0 0x7f080c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.041+0000 7f080bfff640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c102ac0 0x7f080c19d680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00e970 tx=0x7f07fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.041+0000 7f08097fa640 1 -- 172.21.15.46:0/1853423115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f07fc00cac0 con 0x7f080c102ac0 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.041+0000 7f08097fa640 1 -- 172.21.15.46:0/1853423115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f07fc0054e0 con 0x7f080c102ac0 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.041+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c197af0 con 0x7f080c102ac0 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.041+0000 7f08097fa640 1 -- 172.21.15.46:0/1853423115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f07fc010430 con 0x7f080c102ac0 2024-09-10T04:20:43.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.041+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c198050 con 0x7f080c102ac0 2024-09-10T04:20:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.042+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c06aa90 con 0x7f080c102ac0 2024-09-10T04:20:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.043+0000 7f08097fa640 1 -- 172.21.15.46:0/1853423115 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f07fc004e80 con 0x7f080c102ac0 2024-09-10T04:20:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.043+0000 7f08097fa640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f407abf0 0x7f07f407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.043+0000 7f08097fa640 1 -- 172.21.15.46:0/1853423115 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f07fc09e370 con 0x7f080c102ac0 2024-09-10T04:20:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.043+0000 7f080b7fe640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f407abf0 0x7f07f407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.044+0000 7f080b7fe640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f407abf0 0x7f07f407d0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f080c198b50 tx=0x7f0800031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:43.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.048+0000 7f08097fa640 1 -- 172.21.15.46:0/1853423115 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f07fc069ab0 con 0x7f080c102ac0 2024-09-10T04:20:43.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.245+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f080c198360 con 0x7f07f407abf0 2024-09-10T04:20:43.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.255+0000 7f08097fa640 1 -- 172.21.15.46:0/1853423115 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f080c198360 con 0x7f07f407abf0 2024-09-10T04:20:43.256 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:20:43.256 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (12m) 6m ago 23m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (14m) 6m ago 23m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (14m) 6m ago 22m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (10m) 6m ago 23m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (10m) 6m ago 22m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (11m) 6m ago 23m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (6m) 6m ago 18m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (7m) 6m ago 18m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (6m) 6m ago 18m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (18m) 6m ago 18m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (14m) 6m ago 24m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (13m) 6m ago 22m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (11m) 6m ago 25m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (10m) 6m ago 22m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:20:43.257 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (13m) 6m ago 23m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (13m) 6m ago 22m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (10m) 6m ago 21m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (9m) 6m ago 21m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (9m) 6m ago 20m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (8m) 6m ago 20m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (7m) 6m ago 19m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (7m) 6m ago 19m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:20:43.258 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (12m) 6m ago 23m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f407abf0 msgr2=0x7f07f407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f407abf0 0x7f07f407d0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f080c198b50 tx=0x7f0800031400 comp rx=0 tx=0).stop 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c102ac0 msgr2=0x7f080c19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c102ac0 0x7f080c19d680 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00e970 tx=0x7f07fc00ed70 comp rx=0 tx=0).stop 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 shutdown_connections 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f407abf0 0x7f07f407d0b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080c1033e0 0x7f080c19dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 --2- 172.21.15.46:0/1853423115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f080c102ac0 0x7f080c19d680 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 >> 172.21.15.46:0/1853423115 conn(0x7f080c0fc520 msgr2=0x7f080c108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:43.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 shutdown_connections 2024-09-10T04:20:43.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.257+0000 7f08125e1640 1 -- 172.21.15.46:0/1853423115 wait complete. 2024-09-10T04:20:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 -- 172.21.15.46:0/1005043909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8107900 msgr2=0x7fa0b8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 --2- 172.21.15.46:0/1005043909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8107900 0x7fa0b8107ce0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009880 tx=0x7fa0b402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 -- 172.21.15.46:0/1005043909 shutdown_connections 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 --2- 172.21.15.46:0/1005043909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8101900 0x7fa0b8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 --2- 172.21.15.46:0/1005043909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8107900 0x7fa0b8107ce0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 -- 172.21.15.46:0/1005043909 >> 172.21.15.46:0/1005043909 conn(0x7fa0b80fd600 msgr2=0x7fa0b80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 -- 172.21.15.46:0/1005043909 shutdown_connections 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 -- 172.21.15.46:0/1005043909 wait complete. 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.340+0000 7fa0c0640640 1 Processor -- start 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0c0640640 1 -- start start 2024-09-10T04:20:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0c0640640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101900 0x7fa0b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0c0640640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8107900 0x7fa0b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0c0640640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b810edd0 con 0x7fa0b8101900 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0c0640640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b810ec50 con 0x7fa0b8107900 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0be3dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101900 0x7fa0b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0be3dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101900 0x7fa0b80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44696/0 (socket says 172.21.15.46:44696) 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0be3dc640 1 -- 172.21.15.46:0/266389836 learned_addr learned my addr 172.21.15.46:0/266389836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0bdbdb640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8107900 0x7fa0b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0be3dc640 1 -- 172.21.15.46:0/266389836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8107900 msgr2=0x7fa0b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0be3dc640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8107900 0x7fa0b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0be3dc640 1 -- 172.21.15.46:0/266389836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b8074f70 con 0x7fa0b8101900 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0bdbdb640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8107900 0x7fa0b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.341+0000 7fa0be3dc640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101900 0x7fa0b80744f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4031460 tx=0x7fa0b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.342+0000 7fa0af7fe640 1 -- 172.21.15.46:0/266389836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0b403f070 con 0x7fa0b8101900 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.342+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b8075200 con 0x7fa0b8101900 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.342+0000 7fa0af7fe640 1 -- 172.21.15.46:0/266389836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa0b4004cf0 con 0x7fa0b8101900 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.342+0000 7fa0af7fe640 1 -- 172.21.15.46:0/266389836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa0b4035030 con 0x7fa0b8101900 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.342+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b81a3d80 con 0x7fa0b8101900 2024-09-10T04:20:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.343+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b8103040 con 0x7fa0b8101900 2024-09-10T04:20:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.344+0000 7fa0af7fe640 1 -- 172.21.15.46:0/266389836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa0b4042070 con 0x7fa0b8101900 2024-09-10T04:20:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.344+0000 7fa0af7fe640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a407abf0 0x7fa0a407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.344+0000 7fa0af7fe640 1 -- 172.21.15.46:0/266389836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fa0b40be750 con 0x7fa0b8101900 2024-09-10T04:20:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.344+0000 7fa0bdbdb640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a407abf0 0x7fa0a407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.345+0000 7fa0bdbdb640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a407abf0 0x7fa0a407d0b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8005fd0 tx=0x7fa0a8005ea0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.349+0000 7fa0af7fe640 1 -- 172.21.15.46:0/266389836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa0b4089e90 con 0x7fa0b8101900 2024-09-10T04:20:43.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.610+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fa0b8109e70 con 0x7fa0b8101900 2024-09-10T04:20:43.612 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:43 smithi046 ceph-mon[109506]: pgmap v451: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:43.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.610+0000 7fa0af7fe640 1 -- 172.21.15.46:0/266389836 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fa0b408c3b0 con 0x7fa0b8101900 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:20:43.613 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:20:43.614 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:20:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.612+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a407abf0 msgr2=0x7fa0a407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.612+0000 7fa0c0640640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a407abf0 0x7fa0a407d0b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8005fd0 tx=0x7fa0a8005ea0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.612+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101900 msgr2=0x7fa0b80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.612+0000 7fa0c0640640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101900 0x7fa0b80744f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4031460 tx=0x7fa0b40048f0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.613+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 shutdown_connections 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.613+0000 7fa0c0640640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa0a407abf0 0x7fa0a407d0b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.613+0000 7fa0c0640640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa0b8107900 0x7fa0b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.613+0000 7fa0c0640640 1 --2- 172.21.15.46:0/266389836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0b8101900 0x7fa0b80744f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.613+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 >> 172.21.15.46:0/266389836 conn(0x7fa0b80fd600 msgr2=0x7fa0b80fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.613+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 shutdown_connections 2024-09-10T04:20:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.613+0000 7fa0c0640640 1 -- 172.21.15.46:0/266389836 wait complete. 2024-09-10T04:20:43.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.696+0000 7f95f3f40640 1 -- 172.21.15.46:0/1422352450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 msgr2=0x7f95ec074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.696+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1422352450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec074970 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f95e8009880 tx=0x7f95e802a530 comp rx=0 tx=0).stop 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 -- 172.21.15.46:0/1422352450 shutdown_connections 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1422352450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ec074eb0 0x7f95ec1123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1422352450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec074970 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 -- 172.21.15.46:0/1422352450 >> 172.21.15.46:0/1422352450 conn(0x7f95ec0fd600 msgr2=0x7f95ec0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 -- 172.21.15.46:0/1422352450 shutdown_connections 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 -- 172.21.15.46:0/1422352450 wait complete. 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 Processor -- start 2024-09-10T04:20:43.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.697+0000 7f95f3f40640 1 -- start start 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f3f40640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f3f40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ec074eb0 0x7f95ec19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f3f40640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec1153d0 con 0x7f95ec074590 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f3f40640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f95ec115250 con 0x7f95ec074eb0 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f1cdc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f1cdc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec19dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44716/0 (socket says 172.21.15.46:44716) 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f1cdc640 1 -- 172.21.15.46:0/1505582892 learned_addr learned my addr 172.21.15.46:0/1505582892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f14db640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ec074eb0 0x7f95ec19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f1cdc640 1 -- 172.21.15.46:0/1505582892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ec074eb0 msgr2=0x7f95ec19e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f1cdc640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ec074eb0 0x7f95ec19e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f1cdc640 1 -- 172.21.15.46:0/1505582892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ec19ebc0 con 0x7f95ec074590 2024-09-10T04:20:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95f1cdc640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec19dfd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f95e8031460 tx=0x7f95e80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.698+0000 7f95e2ffd640 1 -- 172.21.15.46:0/1505582892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f95e803f070 con 0x7f95ec074590 2024-09-10T04:20:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.699+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ec1a2900 con 0x7f95ec074590 2024-09-10T04:20:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.699+0000 7f95e2ffd640 1 -- 172.21.15.46:0/1505582892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f95e8004cf0 con 0x7f95ec074590 2024-09-10T04:20:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.699+0000 7f95e2ffd640 1 -- 172.21.15.46:0/1505582892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f95e8035030 con 0x7f95ec074590 2024-09-10T04:20:43.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.699+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ec1a2e30 con 0x7f95ec074590 2024-09-10T04:20:43.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.700+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95ec075cd0 con 0x7f95ec074590 2024-09-10T04:20:43.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.700+0000 7f95e2ffd640 1 -- 172.21.15.46:0/1505582892 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f95e8042070 con 0x7f95ec074590 2024-09-10T04:20:43.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.701+0000 7f95e2ffd640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f95d807abf0 0x7f95d807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:43.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.701+0000 7f95e2ffd640 1 -- 172.21.15.46:0/1505582892 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f95e80be730 con 0x7f95ec074590 2024-09-10T04:20:43.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.701+0000 7f95f14db640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f95d807abf0 0x7f95d807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:43.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.701+0000 7f95f14db640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f95d807abf0 0x7f95d807d0b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f95ec104880 tx=0x7f95dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:43.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.705+0000 7f95e2ffd640 1 -- 172.21.15.46:0/1505582892 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f95e8089dd0 con 0x7f95ec074590 2024-09-10T04:20:43.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:43 smithi070 ceph-mon[79774]: pgmap v451: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.930+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f95ec10fac0 con 0x7f95ec074590 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.931+0000 7f95e2ffd640 1 -- 172.21.15.46:0/1505582892 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f95e808ec80 con 0x7f95ec074590 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:43.933 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:20:43.934 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:20:43.935 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:20:43.935 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:20:43.935 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:20:43.935 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:20:43.935 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:20:43.935 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:43.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:43.937 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:20:43.937 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:20:43.937 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:43.937 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:43.937 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:43.937 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f95d807abf0 msgr2=0x7f95d807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f95d807abf0 0x7f95d807d0b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f95ec104880 tx=0x7f95dc00a2d0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 msgr2=0x7f95ec19dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec19dfd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f95e8031460 tx=0x7f95e80048f0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 shutdown_connections 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f95d807abf0 0x7f95d807d0b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95ec074eb0 0x7f95ec19e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 --2- 172.21.15.46:0/1505582892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95ec074590 0x7f95ec19dfd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 >> 172.21.15.46:0/1505582892 conn(0x7f95ec0fd600 msgr2=0x7f95ec104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 shutdown_connections 2024-09-10T04:20:43.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:43.933+0000 7f95f3f40640 1 -- 172.21.15.46:0/1505582892 wait complete. 2024-09-10T04:20:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 -- 172.21.15.46:0/4165689292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec102b20 msgr2=0x7f68ec102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 --2- 172.21.15.46:0/4165689292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec102b20 0x7f68ec102f00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0098b0 tx=0x7f68dc02a560 comp rx=0 tx=0).stop 2024-09-10T04:20:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 -- 172.21.15.46:0/4165689292 shutdown_connections 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 --2- 172.21.15.46:0/4165689292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68ec103440 0x7f68ec10b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 --2- 172.21.15.46:0/4165689292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec102b20 0x7f68ec102f00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 -- 172.21.15.46:0/4165689292 >> 172.21.15.46:0/4165689292 conn(0x7f68ec0fc520 msgr2=0x7f68ec0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 -- 172.21.15.46:0/4165689292 shutdown_connections 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.018+0000 7f68f24c0640 1 -- 172.21.15.46:0/4165689292 wait complete. 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.019+0000 7f68f24c0640 1 Processor -- start 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.019+0000 7f68f24c0640 1 -- start start 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.019+0000 7f68f24c0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68ec102b20 0x7f68ec19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.019+0000 7f68f24c0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec103440 0x7f68ec19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:44.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.019+0000 7f68f24c0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec10edf0 con 0x7f68ec103440 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.019+0000 7f68f24c0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec10ec70 con 0x7f68ec102b20 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec103440 0x7f68ec19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68eb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec103440 0x7f68ec19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44734/0 (socket says 172.21.15.46:44734) 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68eb7fe640 1 -- 172.21.15.46:0/766270697 learned_addr learned my addr 172.21.15.46:0/766270697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68ebfff640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68ec102b20 0x7f68ec19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68eb7fe640 1 -- 172.21.15.46:0/766270697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68ec102b20 msgr2=0x7f68ec19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68eb7fe640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68ec102b20 0x7f68ec19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68eb7fe640 1 -- 172.21.15.46:0/766270697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec197880 con 0x7f68ec103440 2024-09-10T04:20:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68eb7fe640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec103440 0x7f68ec19dc00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c8f0 tx=0x7f68e000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68e97fa640 1 -- 172.21.15.46:0/766270697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f68e00079b0 con 0x7f68ec103440 2024-09-10T04:20:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68e97fa640 1 -- 172.21.15.46:0/766270697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68e00054e0 con 0x7f68ec103440 2024-09-10T04:20:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec197b70 con 0x7f68ec103440 2024-09-10T04:20:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68e97fa640 1 -- 172.21.15.46:0/766270697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f68e0004dd0 con 0x7f68ec103440 2024-09-10T04:20:44.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.020+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec198100 con 0x7f68ec103440 2024-09-10T04:20:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.021+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ec06aa30 con 0x7f68ec103440 2024-09-10T04:20:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.022+0000 7f68e97fa640 1 -- 172.21.15.46:0/766270697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f68e0005020 con 0x7f68ec103440 2024-09-10T04:20:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.023+0000 7f68e97fa640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68d407abf0 0x7f68d407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.023+0000 7f68ebfff640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68d407abf0 0x7f68d407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.023+0000 7f68e97fa640 1 -- 172.21.15.46:0/766270697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f68e009db00 con 0x7f68ec103440 2024-09-10T04:20:44.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.023+0000 7f68ebfff640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68d407abf0 0x7f68d407d0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009230 tx=0x7f68dc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.027+0000 7f68e97fa640 1 -- 172.21.15.46:0/766270697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f68e0014030 con 0x7f68ec103440 2024-09-10T04:20:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.232+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f68ec198db0 con 0x7f68d407abf0 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.233+0000 7f68e97fa640 1 -- 172.21.15.46:0/766270697 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f68ec198db0 con 0x7f68d407abf0 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:20:44.236 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:20:44.236 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:20:44.236 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:20:44.236 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:20:44.236 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:20:44.236 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:20:44.236 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:20:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.236+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68d407abf0 msgr2=0x7f68d407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.236+0000 7f68f24c0640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68d407abf0 0x7f68d407d0b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009230 tx=0x7f68dc031400 comp rx=0 tx=0).stop 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.236+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec103440 msgr2=0x7f68ec19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.236+0000 7f68f24c0640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec103440 0x7f68ec19dc00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f68e000c8f0 tx=0x7f68e000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.236+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 shutdown_connections 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.237+0000 7f68f24c0640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f68d407abf0 0x7f68d407d0b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.237+0000 7f68f24c0640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68ec103440 0x7f68ec19dc00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.237+0000 7f68f24c0640 1 --2- 172.21.15.46:0/766270697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68ec102b20 0x7f68ec19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.237+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 >> 172.21.15.46:0/766270697 conn(0x7f68ec0fc520 msgr2=0x7f68ec107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.237+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 shutdown_connections 2024-09-10T04:20:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.237+0000 7f68f24c0640 1 -- 172.21.15.46:0/766270697 wait complete. 2024-09-10T04:20:44.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 -- 172.21.15.46:0/195202167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 msgr2=0x7f6f00107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 --2- 172.21.15.46:0/195202167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f00107ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009880 tx=0x7f6efc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 -- 172.21.15.46:0/195202167 shutdown_connections 2024-09-10T04:20:44.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 --2- 172.21.15.46:0/195202167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00101900 0x7f6f00101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 --2- 172.21.15.46:0/195202167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f00107ce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 -- 172.21.15.46:0/195202167 >> 172.21.15.46:0/195202167 conn(0x7f6f000fd600 msgr2=0x7f6f000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:44.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 -- 172.21.15.46:0/195202167 shutdown_connections 2024-09-10T04:20:44.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 -- 172.21.15.46:0/195202167 wait complete. 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.322+0000 7f6f080a4640 1 Processor -- start 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f080a4640 1 -- start start 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f080a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00101900 0x7f6f0019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f080a4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f0019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f080a4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0010edb0 con 0x7f6f00107900 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f080a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0010ec30 con 0x7f6f00101900 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f0563f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f0019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f0563f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f0019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44744/0 (socket says 172.21.15.46:44744) 2024-09-10T04:20:44.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f0563f640 1 -- 172.21.15.46:0/3721912275 learned_addr learned my addr 172.21.15.46:0/3721912275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f05e40640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00101900 0x7f6f0019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f0563f640 1 -- 172.21.15.46:0/3721912275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00101900 msgr2=0x7f6f0019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f0563f640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00101900 0x7f6f0019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f0563f640 1 -- 172.21.15.46:0/3721912275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f001977c0 con 0x7f6f00107900 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.323+0000 7f6f0563f640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f0019db80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000d8d0 tx=0x7f6ef000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.324+0000 7f6ef6ffd640 1 -- 172.21.15.46:0/3721912275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ef0017030 con 0x7f6f00107900 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.324+0000 7f6ef6ffd640 1 -- 172.21.15.46:0/3721912275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6ef0004020 con 0x7f6f00107900 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.324+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f00197ab0 con 0x7f6f00107900 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.324+0000 7f6ef6ffd640 1 -- 172.21.15.46:0/3721912275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6ef00047a0 con 0x7f6f00107900 2024-09-10T04:20:44.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.324+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f00198040 con 0x7f6f00107900 2024-09-10T04:20:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.325+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f00103040 con 0x7f6f00107900 2024-09-10T04:20:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.326+0000 7f6ef6ffd640 1 -- 172.21.15.46:0/3721912275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6ef00041d0 con 0x7f6f00107900 2024-09-10T04:20:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.326+0000 7f6ef6ffd640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6eec07ac40 0x7f6eec07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:20:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.326+0000 7f6ef6ffd640 1 -- 172.21.15.46:0/3721912275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f6ef0018070 con 0x7f6f00107900 2024-09-10T04:20:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.326+0000 7f6f05e40640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6eec07ac40 0x7f6eec07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:20:44.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.327+0000 7f6f05e40640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6eec07ac40 0x7f6eec07d100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0096c0 tx=0x7f6efc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:20:44.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.331+0000 7f6ef6ffd640 1 -- 172.21.15.46:0/3721912275 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6ef0069190 con 0x7f6f00107900 2024-09-10T04:20:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.588+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6f00104b10 con 0x7f6f00107900 2024-09-10T04:20:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.588+0000 7f6ef6ffd640 1 -- 172.21.15.46:0/3721912275 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f6ef006e040 con 0x7f6f00107900 2024-09-10T04:20:44.590 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:20:44.590 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:20:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:20:44.591 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:20:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:20:44.591 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:20:44.591 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:20:44.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.590+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6eec07ac40 msgr2=0x7f6eec07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.590+0000 7f6f080a4640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6eec07ac40 0x7f6eec07d100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0096c0 tx=0x7f6efc031400 comp rx=0 tx=0).stop 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.590+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 msgr2=0x7f6f0019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.590+0000 7f6f080a4640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f0019db80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000d8d0 tx=0x7f6ef000dd90 comp rx=0 tx=0).stop 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.591+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 shutdown_connections 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.591+0000 7f6f080a4640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6eec07ac40 0x7f6eec07d100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.591+0000 7f6f080a4640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f00107900 0x7f6f0019db80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.591+0000 7f6f080a4640 1 --2- 172.21.15.46:0/3721912275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f00101900 0x7f6f0019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.591+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 >> 172.21.15.46:0/3721912275 conn(0x7f6f000fd600 msgr2=0x7f6f000ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.591+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 shutdown_connections 2024-09-10T04:20:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:20:44.591+0000 7f6f080a4640 1 -- 172.21.15.46:0/3721912275 wait complete. 2024-09-10T04:20:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:44 smithi070 ceph-mon[79774]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:44 smithi070 ceph-mon[79774]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:44 smithi070 ceph-mon[79774]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:44.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:44 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/266389836' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:20:44.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:44 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1505582892' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:20:44.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:44 smithi046 ceph-mon[109506]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:44.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:44 smithi046 ceph-mon[109506]: from='client.34706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:44.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:44 smithi046 ceph-mon[109506]: from='client.34710 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:44.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:44 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/266389836' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:20:44.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:44 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1505582892' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:20:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:45 smithi070 ceph-mon[79774]: pgmap v452: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:45 smithi070 ceph-mon[79774]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:45 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3721912275' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:20:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:45 smithi046 ceph-mon[109506]: pgmap v452: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:45 smithi046 ceph-mon[109506]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:20:45.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:45 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3721912275' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:20:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:47 smithi070 ceph-mon[79774]: pgmap v453: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:47.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:47 smithi046 ceph-mon[109506]: pgmap v453: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:49 smithi070 ceph-mon[79774]: pgmap v454: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:49.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:49 smithi046 ceph-mon[109506]: pgmap v454: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:51 smithi070 ceph-mon[79774]: pgmap v455: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:51 smithi046 ceph-mon[109506]: pgmap v455: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:53 smithi070 ceph-mon[79774]: pgmap v456: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:53 smithi046 ceph-mon[109506]: pgmap v456: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:20:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:55 smithi070 ceph-mon[79774]: pgmap v457: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:55 smithi046 ceph-mon[109506]: pgmap v457: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:57 smithi070 ceph-mon[79774]: pgmap v458: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:57 smithi046 ceph-mon[109506]: pgmap v458: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:20:59 smithi070 ceph-mon[79774]: pgmap v459: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:20:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:20:59 smithi046 ceph-mon[109506]: pgmap v459: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:01 smithi070 ceph-mon[79774]: pgmap v460: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:01 smithi046 ceph-mon[109506]: pgmap v460: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:03 smithi070 ceph-mon[79774]: pgmap v461: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:03 smithi046 ceph-mon[109506]: pgmap v461: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:05 smithi070 ceph-mon[79774]: pgmap v462: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:05.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:05 smithi046 ceph-mon[109506]: pgmap v462: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:07 smithi070 ceph-mon[79774]: pgmap v463: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:07 smithi046 ceph-mon[109506]: pgmap v463: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:09 smithi070 ceph-mon[79774]: pgmap v464: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:10.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:09 smithi046 ceph-mon[109506]: pgmap v464: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:12.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:11 smithi070 ceph-mon[79774]: pgmap v465: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:11 smithi046 ceph-mon[109506]: pgmap v465: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:13.813 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:13 smithi046 ceph-mon[109506]: pgmap v466: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:13 smithi070 ceph-mon[79774]: pgmap v466: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:14.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 -- 172.21.15.46:0/2224592719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c102ac0 msgr2=0x7ff04c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:14.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 --2- 172.21.15.46:0/2224592719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c102ac0 0x7ff04c102ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009880 tx=0x7ff03c02a530 comp rx=0 tx=0).stop 2024-09-10T04:21:14.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 -- 172.21.15.46:0/2224592719 shutdown_connections 2024-09-10T04:21:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 --2- 172.21.15.46:0/2224592719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff04c1033e0 0x7ff04c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 --2- 172.21.15.46:0/2224592719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c102ac0 0x7ff04c102ea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 -- 172.21.15.46:0/2224592719 >> 172.21.15.46:0/2224592719 conn(0x7ff04c0fc520 msgr2=0x7ff04c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 -- 172.21.15.46:0/2224592719 shutdown_connections 2024-09-10T04:21:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.689+0000 7ff05162f640 1 -- 172.21.15.46:0/2224592719 wait complete. 2024-09-10T04:21:14.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.690+0000 7ff05162f640 1 Processor -- start 2024-09-10T04:21:14.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.690+0000 7ff05162f640 1 -- start start 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.690+0000 7ff05162f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff04c102ac0 0x7ff04c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.690+0000 7ff05162f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c1033e0 0x7ff04c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.690+0000 7ff05162f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c10edd0 con 0x7ff04c1033e0 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.690+0000 7ff05162f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c10ec50 con 0x7ff04c102ac0 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c1033e0 0x7ff04c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c1033e0 0x7ff04c100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46776/0 (socket says 172.21.15.46:46776) 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04a7fc640 1 -- 172.21.15.46:0/3611013887 learned_addr learned my addr 172.21.15.46:0/3611013887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04affd640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff04c102ac0 0x7ff04c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04a7fc640 1 -- 172.21.15.46:0/3611013887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff04c102ac0 msgr2=0x7ff04c100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04a7fc640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff04c102ac0 0x7ff04c100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04a7fc640 1 -- 172.21.15.46:0/3611013887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c1011a0 con 0x7ff04c1033e0 2024-09-10T04:21:14.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff04a7fc640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c1033e0 0x7ff04c100c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c8f0 tx=0x7ff04000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff033fff640 1 -- 172.21.15.46:0/3611013887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff0400079b0 con 0x7ff04c1033e0 2024-09-10T04:21:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff033fff640 1 -- 172.21.15.46:0/3611013887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff040007b50 con 0x7ff04c1033e0 2024-09-10T04:21:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04c101490 con 0x7ff04c1033e0 2024-09-10T04:21:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff033fff640 1 -- 172.21.15.46:0/3611013887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff040018610 con 0x7ff04c1033e0 2024-09-10T04:21:14.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.691+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04c1a3f00 con 0x7ff04c1033e0 2024-09-10T04:21:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.692+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04c06aa90 con 0x7ff04c1033e0 2024-09-10T04:21:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.693+0000 7ff033fff640 1 -- 172.21.15.46:0/3611013887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff040007d00 con 0x7ff04c1033e0 2024-09-10T04:21:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.694+0000 7ff033fff640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff03407ac40 0x7ff03407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.694+0000 7ff033fff640 1 -- 172.21.15.46:0/3611013887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ff04009dd40 con 0x7ff04c1033e0 2024-09-10T04:21:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.694+0000 7ff04affd640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff03407ac40 0x7ff03407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:14.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.694+0000 7ff04affd640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff03407ac40 0x7ff03407d100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff03c000c00 tx=0x7ff03c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:14.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.699+0000 7ff033fff640 1 -- 172.21.15.46:0/3611013887 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff040014030 con 0x7ff04c1033e0 2024-09-10T04:21:14.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.906+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff04c062630 con 0x7ff03407ac40 2024-09-10T04:21:14.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.907+0000 7ff033fff640 1 -- 172.21.15.46:0/3611013887 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7ff04c062630 con 0x7ff03407ac40 2024-09-10T04:21:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff03407ac40 msgr2=0x7ff03407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:14.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff03407ac40 0x7ff03407d100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff03c000c00 tx=0x7ff03c031400 comp rx=0 tx=0).stop 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c1033e0 msgr2=0x7ff04c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c1033e0 0x7ff04c100c60 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c8f0 tx=0x7ff04000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 shutdown_connections 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff03407ac40 0x7ff03407d100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff04c1033e0 0x7ff04c100c60 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 --2- 172.21.15.46:0/3611013887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff04c102ac0 0x7ff04c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.909+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 >> 172.21.15.46:0/3611013887 conn(0x7ff04c0fc520 msgr2=0x7ff04c108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.910+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 shutdown_connections 2024-09-10T04:21:14.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.910+0000 7ff05162f640 1 -- 172.21.15.46:0/3611013887 wait complete. 2024-09-10T04:21:14.919 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:21:14.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.995+0000 7fc5bdc80640 1 -- 172.21.15.46:0/3894916419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8102ac0 msgr2=0x7fc5b8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.995+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/3894916419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8102ac0 0x7fc5b8102ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009880 tx=0x7fc5a802a530 comp rx=0 tx=0).stop 2024-09-10T04:21:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.995+0000 7fc5bdc80640 1 -- 172.21.15.46:0/3894916419 shutdown_connections 2024-09-10T04:21:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.996+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/3894916419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5b81033e0 0x7fc5b810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.996+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/3894916419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b8102ac0 0x7fc5b8102ea0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.996+0000 7fc5bdc80640 1 -- 172.21.15.46:0/3894916419 >> 172.21.15.46:0/3894916419 conn(0x7fc5b80fc520 msgr2=0x7fc5b80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.996+0000 7fc5bdc80640 1 -- 172.21.15.46:0/3894916419 shutdown_connections 2024-09-10T04:21:14.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.996+0000 7fc5bdc80640 1 -- 172.21.15.46:0/3894916419 wait complete. 2024-09-10T04:21:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.996+0000 7fc5bdc80640 1 Processor -- start 2024-09-10T04:21:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.996+0000 7fc5bdc80640 1 -- start start 2024-09-10T04:21:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5bdc80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5b8102ac0 0x7fc5b819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5bdc80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b81033e0 0x7fc5b819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5bdc80640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b810edf0 con 0x7fc5b81033e0 2024-09-10T04:21:14.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5bdc80640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b810ec70 con 0x7fc5b8102ac0 2024-09-10T04:21:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b81033e0 0x7fc5b819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5b6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b81033e0 0x7fc5b819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46786/0 (socket says 172.21.15.46:46786) 2024-09-10T04:21:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5b6ffd640 1 -- 172.21.15.46:0/1361410518 learned_addr learned my addr 172.21.15.46:0/1361410518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5b77fe640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5b8102ac0 0x7fc5b819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5b6ffd640 1 -- 172.21.15.46:0/1361410518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5b8102ac0 msgr2=0x7fc5b819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5b6ffd640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5b8102ac0 0x7fc5b819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.997+0000 7fc5b6ffd640 1 -- 172.21.15.46:0/1361410518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b81977c0 con 0x7fc5b81033e0 2024-09-10T04:21:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.998+0000 7fc5b6ffd640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b81033e0 0x7fc5b819db80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00e970 tx=0x7fc5ac00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.998+0000 7fc5b4ff9640 1 -- 172.21.15.46:0/1361410518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5ac00cac0 con 0x7fc5b81033e0 2024-09-10T04:21:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.998+0000 7fc5b4ff9640 1 -- 172.21.15.46:0/1361410518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5ac0054e0 con 0x7fc5b81033e0 2024-09-10T04:21:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.998+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b8197ab0 con 0x7fc5b81033e0 2024-09-10T04:21:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.998+0000 7fc5b4ff9640 1 -- 172.21.15.46:0/1361410518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5ac004dd0 con 0x7fc5b81033e0 2024-09-10T04:21:15.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.998+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b8198040 con 0x7fc5b81033e0 2024-09-10T04:21:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:14.999+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b806aa90 con 0x7fc5b81033e0 2024-09-10T04:21:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.000+0000 7fc5b4ff9640 1 -- 172.21.15.46:0/1361410518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc5ac005020 con 0x7fc5b81033e0 2024-09-10T04:21:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.000+0000 7fc5b4ff9640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5a007ab90 0x7fc5a007d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.001+0000 7fc5b4ff9640 1 -- 172.21.15.46:0/1361410518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc5ac09e290 con 0x7fc5b81033e0 2024-09-10T04:21:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.001+0000 7fc5b77fe640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5a007ab90 0x7fc5a007d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.001+0000 7fc5b77fe640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5a007ab90 0x7fc5a007d050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80096c0 tx=0x7fc5a8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.006+0000 7fc5b4ff9640 1 -- 172.21.15.46:0/1361410518 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc5ac0699d0 con 0x7fc5b81033e0 2024-09-10T04:21:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.212+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc5b8198350 con 0x7fc5a007ab90 2024-09-10T04:21:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.213+0000 7fc5b4ff9640 1 -- 172.21.15.46:0/1361410518 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fc5b8198350 con 0x7fc5a007ab90 2024-09-10T04:21:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5a007ab90 msgr2=0x7fc5a007d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5a007ab90 0x7fc5a007d050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80096c0 tx=0x7fc5a8031400 comp rx=0 tx=0).stop 2024-09-10T04:21:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b81033e0 msgr2=0x7fc5b819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b81033e0 0x7fc5b819db80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00e970 tx=0x7fc5ac00ee30 comp rx=0 tx=0).stop 2024-09-10T04:21:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 shutdown_connections 2024-09-10T04:21:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5a007ab90 0x7fc5a007d050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5b81033e0 0x7fc5b819db80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 --2- 172.21.15.46:0/1361410518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5b8102ac0 0x7fc5b819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 >> 172.21.15.46:0/1361410518 conn(0x7fc5b80fc520 msgr2=0x7fc5b810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.215+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 shutdown_connections 2024-09-10T04:21:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.216+0000 7fc5bdc80640 1 -- 172.21.15.46:0/1361410518 wait complete. 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 -- 172.21.15.46:0/3936353050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70107900 msgr2=0x7f9a70107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 --2- 172.21.15.46:0/3936353050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70107900 0x7f9a70107ce0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009880 tx=0x7f9a6c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 -- 172.21.15.46:0/3936353050 shutdown_connections 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 --2- 172.21.15.46:0/3936353050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a70101900 0x7f9a70101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 --2- 172.21.15.46:0/3936353050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70107900 0x7f9a70107ce0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 -- 172.21.15.46:0/3936353050 >> 172.21.15.46:0/3936353050 conn(0x7f9a700fd600 msgr2=0x7f9a700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 -- 172.21.15.46:0/3936353050 shutdown_connections 2024-09-10T04:21:15.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.300+0000 7f9a781ca640 1 -- 172.21.15.46:0/3936353050 wait complete. 2024-09-10T04:21:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a781ca640 1 Processor -- start 2024-09-10T04:21:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a781ca640 1 -- start start 2024-09-10T04:21:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a781ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70101900 0x7f9a7019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a781ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a70107900 0x7f9a7019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a781ca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010edd0 con 0x7f9a70101900 2024-09-10T04:21:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a781ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f9a7010ec50 con 0x7f9a70107900 2024-09-10T04:21:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a75f66640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70101900 0x7f9a7019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a75f66640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70101900 0x7f9a7019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46796/0 (socket says 172.21.15.46:46796) 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.301+0000 7f9a75f66640 1 -- 172.21.15.46:0/4157437847 learned_addr learned my addr 172.21.15.46:0/4157437847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a75765640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a70107900 0x7f9a7019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a75f66640 1 -- 172.21.15.46:0/4157437847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a70107900 msgr2=0x7f9a7019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a75f66640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a70107900 0x7f9a7019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a75f66640 1 -- 172.21.15.46:0/4157437847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a701977a0 con 0x7f9a70101900 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a75765640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a70107900 0x7f9a7019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a75f66640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70101900 0x7f9a7019d620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c031460 tx=0x7f9a6c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a66ffd640 1 -- 172.21.15.46:0/4157437847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a6c03f070 con 0x7f9a70101900 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a66ffd640 1 -- 172.21.15.46:0/4157437847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9a6c0035f0 con 0x7f9a70101900 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a70197a30 con 0x7f9a70101900 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a70197f60 con 0x7f9a70101900 2024-09-10T04:21:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.302+0000 7f9a66ffd640 1 -- 172.21.15.46:0/4157437847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9a6c035030 con 0x7f9a70101900 2024-09-10T04:21:15.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.303+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70103040 con 0x7f9a70101900 2024-09-10T04:21:15.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.304+0000 7f9a66ffd640 1 -- 172.21.15.46:0/4157437847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9a6c042070 con 0x7f9a70101900 2024-09-10T04:21:15.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.305+0000 7f9a66ffd640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a5c07ac40 0x7f9a5c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.305+0000 7f9a66ffd640 1 -- 172.21.15.46:0/4157437847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f9a6c0be640 con 0x7f9a70101900 2024-09-10T04:21:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.305+0000 7f9a75765640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a5c07ac40 0x7f9a5c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.305+0000 7f9a75765640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a5c07ac40 0x7f9a5c07d100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9a600058c0 tx=0x7f9a60005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.309+0000 7f9a66ffd640 1 -- 172.21.15.46:0/4157437847 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9a6c089d60 con 0x7f9a70101900 2024-09-10T04:21:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.502+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9a70198210 con 0x7f9a5c07ac40 2024-09-10T04:21:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.512+0000 7f9a66ffd640 1 -- 172.21.15.46:0/4157437847 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f9a70198210 con 0x7f9a5c07ac40 2024-09-10T04:21:15.512 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:21:15.512 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (12m) 7m ago 24m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (15m) 7m ago 24m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (15m) 7m ago 23m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (11m) 7m ago 24m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (11m) 7m ago 23m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (12m) 7m ago 23m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (7m) 7m ago 18m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (7m) 7m ago 18m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (7m) 7m ago 18m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (18m) 7m ago 18m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (14m) 7m ago 25m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (13m) 7m ago 23m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (11m) 7m ago 25m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:21:15.513 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (11m) 7m ago 23m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (13m) 7m ago 24m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (13m) 7m ago 23m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (10m) 7m ago 22m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (10m) 7m ago 21m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (9m) 7m ago 21m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (9m) 7m ago 20m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (8m) 7m ago 20m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (7m) 7m ago 19m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:21:15.514 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (13m) 7m ago 23m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:21:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a5c07ac40 msgr2=0x7f9a5c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a5c07ac40 0x7f9a5c07d100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9a600058c0 tx=0x7f9a60005850 comp rx=0 tx=0).stop 2024-09-10T04:21:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70101900 msgr2=0x7f9a7019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70101900 0x7f9a7019d620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c031460 tx=0x7f9a6c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 shutdown_connections 2024-09-10T04:21:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f9a5c07ac40 0x7f9a5c07d100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a70107900 0x7f9a7019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 --2- 172.21.15.46:0/4157437847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a70101900 0x7f9a7019d620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 >> 172.21.15.46:0/4157437847 conn(0x7f9a700fd600 msgr2=0x7f9a700fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 shutdown_connections 2024-09-10T04:21:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.514+0000 7f9a781ca640 1 -- 172.21.15.46:0/4157437847 wait complete. 2024-09-10T04:21:15.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 -- 172.21.15.46:0/191039529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 msgr2=0x7f62f8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 --2- 172.21.15.46:0/191039529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f8106be0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009880 tx=0x7f62e802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 -- 172.21.15.46:0/191039529 shutdown_connections 2024-09-10T04:21:15.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 --2- 172.21.15.46:0/191039529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62f8100800 0x7f62f8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 --2- 172.21.15.46:0/191039529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f8106be0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 -- 172.21.15.46:0/191039529 >> 172.21.15.46:0/191039529 conn(0x7f62f80fc520 msgr2=0x7f62f80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 -- 172.21.15.46:0/191039529 shutdown_connections 2024-09-10T04:21:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.599+0000 7f62fe24d640 1 -- 172.21.15.46:0/191039529 wait complete. 2024-09-10T04:21:15.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62fe24d640 1 Processor -- start 2024-09-10T04:21:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62fe24d640 1 -- start start 2024-09-10T04:21:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62fe24d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62f8100800 0x7f62f819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62fe24d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62fe24d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62f810cbe0 con 0x7f62f8106800 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62fe24d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f62f810ca60 con 0x7f62f8100800 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62f6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f819b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46810/0 (socket says 172.21.15.46:46810) 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62f6ffd640 1 -- 172.21.15.46:0/132251707 learned_addr learned my addr 172.21.15.46:0/132251707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.600+0000 7f62f77fe640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62f8100800 0x7f62f819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62f6ffd640 1 -- 172.21.15.46:0/132251707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62f8100800 msgr2=0x7f62f819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62f6ffd640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62f8100800 0x7f62f819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62f6ffd640 1 -- 172.21.15.46:0/132251707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f81955e0 con 0x7f62f8106800 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62f6ffd640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f819b960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f62ec010ad0 tx=0x7f62ec010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62f4ff9640 1 -- 172.21.15.46:0/132251707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62ec01a030 con 0x7f62f8106800 2024-09-10T04:21:15.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62f4ff9640 1 -- 172.21.15.46:0/132251707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f62ec00ee50 con 0x7f62f8106800 2024-09-10T04:21:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f81958d0 con 0x7f62f8106800 2024-09-10T04:21:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62f4ff9640 1 -- 172.21.15.46:0/132251707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62ec004b70 con 0x7f62f8106800 2024-09-10T04:21:15.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.601+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f8195e60 con 0x7f62f8106800 2024-09-10T04:21:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.602+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f8101f40 con 0x7f62f8106800 2024-09-10T04:21:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.603+0000 7f62f4ff9640 1 -- 172.21.15.46:0/132251707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f62ec01d070 con 0x7f62f8106800 2024-09-10T04:21:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.604+0000 7f62f4ff9640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62e007ac00 0x7f62e007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.604+0000 7f62f4ff9640 1 -- 172.21.15.46:0/132251707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f62ec01b070 con 0x7f62f8106800 2024-09-10T04:21:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.604+0000 7f62f77fe640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62e007ac00 0x7f62e007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.604+0000 7f62f77fe640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62e007ac00 0x7f62e007d0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62e80096c0 tx=0x7f62e8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.608+0000 7f62f4ff9640 1 -- 172.21.15.46:0/132251707 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f62ec012040 con 0x7f62f8106800 2024-09-10T04:21:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.869+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f62f8108d70 con 0x7f62f8106800 2024-09-10T04:21:15.870 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:15 smithi046 ceph-mon[109506]: pgmap v467: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.870+0000 7f62f4ff9640 1 -- 172.21.15.46:0/132251707 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f62ec06b8b0 con 0x7f62f8106800 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:21:15.871 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:21:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:21:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:21:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:21:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:21:15.872 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:21:15.872 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:21:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.872+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62e007ac00 msgr2=0x7f62e007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.872+0000 7f62fe24d640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62e007ac00 0x7f62e007d0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62e80096c0 tx=0x7f62e8031400 comp rx=0 tx=0).stop 2024-09-10T04:21:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.872+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 msgr2=0x7f62f819b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.872+0000 7f62fe24d640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f819b960 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f62ec010ad0 tx=0x7f62ec010f90 comp rx=0 tx=0).stop 2024-09-10T04:21:15.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.873+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 shutdown_connections 2024-09-10T04:21:15.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.873+0000 7f62fe24d640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62e007ac00 0x7f62e007d0c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.873+0000 7f62fe24d640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62f8106800 0x7f62f819b960 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.873+0000 7f62fe24d640 1 --2- 172.21.15.46:0/132251707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62f8100800 0x7f62f819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.873+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 >> 172.21.15.46:0/132251707 conn(0x7f62f80fc520 msgr2=0x7f62f80fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:15.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.873+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 shutdown_connections 2024-09-10T04:21:15.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.873+0000 7f62fe24d640 1 -- 172.21.15.46:0/132251707 wait complete. 2024-09-10T04:21:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.957+0000 7f3e74586640 1 -- 172.21.15.46:0/2361985583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1067f0 msgr2=0x7f3e6c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.957+0000 7f3e74586640 1 --2- 172.21.15.46:0/2361985583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1067f0 0x7f3e6c106bd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009880 tx=0x7f3e6802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 -- 172.21.15.46:0/2361985583 shutdown_connections 2024-09-10T04:21:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 --2- 172.21.15.46:0/2361985583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e6c1007f0 0x7f3e6c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 --2- 172.21.15.46:0/2361985583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1067f0 0x7f3e6c106bd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 -- 172.21.15.46:0/2361985583 >> 172.21.15.46:0/2361985583 conn(0x7f3e6c0fc550 msgr2=0x7f3e6c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 -- 172.21.15.46:0/2361985583 shutdown_connections 2024-09-10T04:21:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 -- 172.21.15.46:0/2361985583 wait complete. 2024-09-10T04:21:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 Processor -- start 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.958+0000 7f3e74586640 1 -- start start 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e74586640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1007f0 0x7f3e6c0fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e74586640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e6c1067f0 0x7f3e6c0fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e74586640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c10dd20 con 0x7f3e6c1007f0 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e74586640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c10dba0 con 0x7f3e6c1067f0 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e72322640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1007f0 0x7f3e6c0fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e72322640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1007f0 0x7f3e6c0fcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46826/0 (socket says 172.21.15.46:46826) 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e72322640 1 -- 172.21.15.46:0/1947091600 learned_addr learned my addr 172.21.15.46:0/1947091600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:15.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e71b21640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e6c1067f0 0x7f3e6c0fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e72322640 1 -- 172.21.15.46:0/1947091600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e6c1067f0 msgr2=0x7f3e6c0fd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e72322640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e6c1067f0 0x7f3e6c0fd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e72322640 1 -- 172.21.15.46:0/1947091600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c0fd7a0 con 0x7f3e6c1007f0 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e71b21640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e6c1067f0 0x7f3e6c0fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e72322640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1007f0 0x7f3e6c0fcd20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3e68031460 tx=0x7f3e680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e637fe640 1 -- 172.21.15.46:0/1947091600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3e6803f070 con 0x7f3e6c1007f0 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c0fda30 con 0x7f3e6c1007f0 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c1a3dc0 con 0x7f3e6c1007f0 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.959+0000 7f3e637fe640 1 -- 172.21.15.46:0/1947091600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3e68004cf0 con 0x7f3e6c1007f0 2024-09-10T04:21:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.960+0000 7f3e637fe640 1 -- 172.21.15.46:0/1947091600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3e68035030 con 0x7f3e6c1007f0 2024-09-10T04:21:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.960+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e50005290 con 0x7f3e6c1007f0 2024-09-10T04:21:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.961+0000 7f3e637fe640 1 -- 172.21.15.46:0/1947091600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3e68042070 con 0x7f3e6c1007f0 2024-09-10T04:21:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.962+0000 7f3e637fe640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e5807ac40 0x7f3e5807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.962+0000 7f3e637fe640 1 -- 172.21.15.46:0/1947091600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f3e680be560 con 0x7f3e6c1007f0 2024-09-10T04:21:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.962+0000 7f3e71b21640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e5807ac40 0x7f3e5807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.962+0000 7f3e71b21640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e5807ac40 0x7f3e5807d100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0fe7b0 tx=0x7f3e5c008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:15.966+0000 7f3e637fe640 1 -- 172.21.15.46:0/1947091600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3e68089d20 con 0x7f3e6c1007f0 2024-09-10T04:21:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:15 smithi070 ceph-mon[79774]: pgmap v467: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.193+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3e50005d90 con 0x7f3e6c1007f0 2024-09-10T04:21:16.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.194+0000 7f3e637fe640 1 -- 172.21.15.46:0/1947091600 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f3e6808ebd0 con 0x7f3e6c1007f0 2024-09-10T04:21:16.195 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:21:16.195 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:21:16.195 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:21:16.196 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:21:16.197 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:16.198 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:21:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.196+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e5807ac40 msgr2=0x7f3e5807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.196+0000 7f3e74586640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e5807ac40 0x7f3e5807d100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0fe7b0 tx=0x7f3e5c008040 comp rx=0 tx=0).stop 2024-09-10T04:21:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.196+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1007f0 msgr2=0x7f3e6c0fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.196+0000 7f3e74586640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1007f0 0x7f3e6c0fcd20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3e68031460 tx=0x7f3e680048f0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.197+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 shutdown_connections 2024-09-10T04:21:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.197+0000 7f3e74586640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3e5807ac40 0x7f3e5807d100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.197+0000 7f3e74586640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e6c1067f0 0x7f3e6c0fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.197+0000 7f3e74586640 1 --2- 172.21.15.46:0/1947091600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e6c1007f0 0x7f3e6c0fcd20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.197+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 >> 172.21.15.46:0/1947091600 conn(0x7f3e6c0fc550 msgr2=0x7f3e6c069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.197+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 shutdown_connections 2024-09-10T04:21:16.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.197+0000 7f3e74586640 1 -- 172.21.15.46:0/1947091600 wait complete. 2024-09-10T04:21:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 -- 172.21.15.46:0/4129175192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 msgr2=0x7f773010b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 --2- 172.21.15.46:0/4129175192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773010b960 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7718009880 tx=0x7f771802a530 comp rx=0 tx=0).stop 2024-09-10T04:21:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 -- 172.21.15.46:0/4129175192 shutdown_connections 2024-09-10T04:21:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 --2- 172.21.15.46:0/4129175192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773010b960 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 --2- 172.21.15.46:0/4129175192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7730102b50 0x7f7730102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 -- 172.21.15.46:0/4129175192 >> 172.21.15.46:0/4129175192 conn(0x7f77300fc550 msgr2=0x7f77300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 -- 172.21.15.46:0/4129175192 shutdown_connections 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.283+0000 7f7734a71640 1 -- 172.21.15.46:0/4129175192 wait complete. 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f7734a71640 1 Processor -- start 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f7734a71640 1 -- start start 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f7734a71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7730102b50 0x7f773019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f7734a71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f7734a71640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f773010edc0 con 0x7f7730103470 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f7734a71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f773010ec40 con 0x7f7730102b50 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f772dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f772dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46832/0 (socket says 172.21.15.46:46832) 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f772dd9b640 1 -- 172.21.15.46:0/2231750149 learned_addr learned my addr 172.21.15.46:0/2231750149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f772e59c640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7730102b50 0x7f773019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:16.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f772dd9b640 1 -- 172.21.15.46:0/2231750149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7730102b50 msgr2=0x7f773019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f772dd9b640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7730102b50 0x7f773019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.284+0000 7f772dd9b640 1 -- 172.21.15.46:0/2231750149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77301a0430 con 0x7f7730103470 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.285+0000 7f772dd9b640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773019fd80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7718004f90 tx=0x7f77180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.285+0000 7f77277fe640 1 -- 172.21.15.46:0/2231750149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f771803f070 con 0x7f7730103470 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.285+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7730199ac0 con 0x7f7730103470 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.285+0000 7f77277fe640 1 -- 172.21.15.46:0/2231750149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7718004cf0 con 0x7f7730103470 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.285+0000 7f77277fe640 1 -- 172.21.15.46:0/2231750149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7718035030 con 0x7f7730103470 2024-09-10T04:21:16.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.285+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7730199fc0 con 0x7f7730103470 2024-09-10T04:21:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.286+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773006a4e0 con 0x7f7730103470 2024-09-10T04:21:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.287+0000 7f77277fe640 1 -- 172.21.15.46:0/2231750149 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7718042070 con 0x7f7730103470 2024-09-10T04:21:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.287+0000 7f77277fe640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f771c07abf0 0x7f771c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.288+0000 7f77277fe640 1 -- 172.21.15.46:0/2231750149 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f77180be6b0 con 0x7f7730103470 2024-09-10T04:21:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.288+0000 7f772e59c640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f771c07abf0 0x7f771c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:16.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.288+0000 7f772e59c640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f771c07abf0 0x7f771c07d0b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f773019ab40 tx=0x7f772800a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:16.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.292+0000 7f77277fe640 1 -- 172.21.15.46:0/2231750149 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7718089df0 con 0x7f7730103470 2024-09-10T04:21:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.494+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f773019afa0 con 0x7f771c07abf0 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.495+0000 7f77277fe640 1 -- 172.21.15.46:0/2231750149 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f773019afa0 con 0x7f771c07abf0 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:21:16.496 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:21:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:21:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:21:16.497 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:21:16.497 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:21:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.498+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f771c07abf0 msgr2=0x7f771c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.498+0000 7f7734a71640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f771c07abf0 0x7f771c07d0b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f773019ab40 tx=0x7f772800a2d0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.498+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 msgr2=0x7f773019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.498+0000 7f7734a71640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773019fd80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7718004f90 tx=0x7f77180048f0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.499+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 shutdown_connections 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.499+0000 7f7734a71640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f771c07abf0 0x7f771c07d0b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.499+0000 7f7734a71640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7730103470 0x7f773019fd80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.499+0000 7f7734a71640 1 --2- 172.21.15.46:0/2231750149 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7730102b50 0x7f773019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.499+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 >> 172.21.15.46:0/2231750149 conn(0x7f77300fc550 msgr2=0x7f7730100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.499+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 shutdown_connections 2024-09-10T04:21:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.499+0000 7f7734a71640 1 -- 172.21.15.46:0/2231750149 wait complete. 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 -- 172.21.15.46:0/2191248583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 msgr2=0x7fa56c105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 --2- 172.21.15.46:0/2191248583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c105c40 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa568009880 tx=0x7fa56802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 -- 172.21.15.46:0/2191248583 shutdown_connections 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 --2- 172.21.15.46:0/2191248583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa56c101010 0x7fa56c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 --2- 172.21.15.46:0/2191248583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c105c40 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 -- 172.21.15.46:0/2191248583 >> 172.21.15.46:0/2191248583 conn(0x7fa56c0fbc90 msgr2=0x7fa56c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 -- 172.21.15.46:0/2191248583 shutdown_connections 2024-09-10T04:21:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 -- 172.21.15.46:0/2191248583 wait complete. 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 Processor -- start 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.583+0000 7fa573430640 1 -- start start 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa573430640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa56c101010 0x7fa56c0fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa573430640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c0fc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa573430640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c10de40 con 0x7fa56c105860 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa573430640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c10dcc0 con 0x7fa56c101010 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5709cb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c0fc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5709cb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c0fc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46844/0 (socket says 172.21.15.46:46844) 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5709cb640 1 -- 172.21.15.46:0/120075185 learned_addr learned my addr 172.21.15.46:0/120075185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5711cc640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa56c101010 0x7fa56c0fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:16.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5709cb640 1 -- 172.21.15.46:0/120075185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa56c101010 msgr2=0x7fa56c0fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5709cb640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa56c101010 0x7fa56c0fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5709cb640 1 -- 172.21.15.46:0/120075185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c0fce60 con 0x7fa56c105860 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.584+0000 7fa5709cb640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c0fc920 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00d6f0 tx=0x7fa55c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.585+0000 7fa5627fc640 1 -- 172.21.15.46:0/120075185 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa55c017070 con 0x7fa56c105860 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.585+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c0fd150 con 0x7fa56c105860 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.585+0000 7fa5627fc640 1 -- 172.21.15.46:0/120075185 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa55c00de00 con 0x7fa56c105860 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.585+0000 7fa5627fc640 1 -- 172.21.15.46:0/120075185 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa55c004bb0 con 0x7fa56c105860 2024-09-10T04:21:16.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.585+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c1a3f20 con 0x7fa56c105860 2024-09-10T04:21:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.586+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c102020 con 0x7fa56c105860 2024-09-10T04:21:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.586+0000 7fa5627fc640 1 -- 172.21.15.46:0/120075185 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa55c004e60 con 0x7fa56c105860 2024-09-10T04:21:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.587+0000 7fa5627fc640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.587+0000 7fa5627fc640 1 -- 172.21.15.46:0/120075185 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fa55c09d8c0 con 0x7fa56c105860 2024-09-10T04:21:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.587+0000 7fa5711cc640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.587+0000 7fa5711cc640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa5680096c0 tx=0x7fa568031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:16.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.591+0000 7fa5627fc640 1 -- 172.21.15.46:0/120075185 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa55c068f60 con 0x7fa56c105860 2024-09-10T04:21:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:16 smithi046 ceph-mon[109506]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:16 smithi046 ceph-mon[109506]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:16.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:16 smithi046 ceph-mon[109506]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:16.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:16 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/132251707' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:21:16.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:16 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1947091600' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:21:16.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.853+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa56c109d90 con 0x7fa56c105860 2024-09-10T04:21:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.853+0000 7fa5627fc640 1 -- 172.21.15.46:0/120075185 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fa55c06de10 con 0x7fa56c105860 2024-09-10T04:21:16.854 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:21:16.854 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:21:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:21:16.854 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:21:16.854 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:21:16.854 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:21:16.855 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:21:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 msgr2=0x7fa55807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa5680096c0 tx=0x7fa568031400 comp rx=0 tx=0).stop 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 msgr2=0x7fa56c0fc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c0fc920 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00d6f0 tx=0x7fa55c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 shutdown_connections 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa55807abf0 0x7fa55807d0b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56c105860 0x7fa56c0fc920 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 --2- 172.21.15.46:0/120075185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa56c101010 0x7fa56c0fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 >> 172.21.15.46:0/120075185 conn(0x7fa56c0fbc90 msgr2=0x7fa56c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 shutdown_connections 2024-09-10T04:21:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:16.856+0000 7fa573430640 1 -- 172.21.15.46:0/120075185 wait complete. 2024-09-10T04:21:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:16 smithi070 ceph-mon[79774]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:16 smithi070 ceph-mon[79774]: from='client.34734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:16 smithi070 ceph-mon[79774]: from='client.34738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:16 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/132251707' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:21:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:16 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1947091600' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:21:17.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:17 smithi070 ceph-mon[79774]: pgmap v468: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:17.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:17 smithi070 ceph-mon[79774]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:17.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:17 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/120075185' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:21:18.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:17 smithi046 ceph-mon[109506]: pgmap v468: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:18.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:17 smithi046 ceph-mon[109506]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:18.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:17 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/120075185' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:21:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:19 smithi070 ceph-mon[79774]: pgmap v469: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:19 smithi046 ceph-mon[109506]: pgmap v469: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:21 smithi070 ceph-mon[79774]: pgmap v470: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:21 smithi046 ceph-mon[109506]: pgmap v470: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:23.968 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:23 smithi046 ceph-mon[109506]: pgmap v471: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:23.968 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:23 smithi070 ceph-mon[79774]: pgmap v471: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:25 smithi070 ceph-mon[79774]: pgmap v472: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:26.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:25 smithi046 ceph-mon[109506]: pgmap v472: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:27.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:26 smithi070 ceph-mon[79774]: pgmap v473: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:27.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:26 smithi046 ceph-mon[109506]: pgmap v473: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:29.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:29 smithi046 ceph-mon[109506]: pgmap v474: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:29.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:29 smithi070 ceph-mon[79774]: pgmap v474: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:31.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:31 smithi070 ceph-mon[79774]: pgmap v475: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:31.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:31 smithi046 ceph-mon[109506]: pgmap v475: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:33.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:33 smithi070 ceph-mon[79774]: pgmap v476: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:33.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:33 smithi046 ceph-mon[109506]: pgmap v476: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:35.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:35 smithi070 ceph-mon[79774]: pgmap v477: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:35 smithi046 ceph-mon[109506]: pgmap v477: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:37.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:37 smithi070 ceph-mon[79774]: pgmap v478: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:37.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:37 smithi046 ceph-mon[109506]: pgmap v478: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:38.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:39.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:39 smithi070 ceph-mon[79774]: pgmap v479: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:39.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:39 smithi046 ceph-mon[109506]: pgmap v479: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:41.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:41 smithi070 ceph-mon[79774]: pgmap v480: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:41 smithi046 ceph-mon[109506]: pgmap v480: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:43.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:43 smithi070 ceph-mon[79774]: pgmap v481: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:43.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:43 smithi046 ceph-mon[109506]: pgmap v481: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:45.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:45 smithi070 ceph-mon[79774]: pgmap v482: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:45 smithi046 ceph-mon[109506]: pgmap v482: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 -- 172.21.15.46:0/3025808939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830100800 msgr2=0x7f3830100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 --2- 172.21.15.46:0/3025808939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830100800 0x7f3830100c60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3824009880 tx=0x7f382402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 -- 172.21.15.46:0/3025808939 shutdown_connections 2024-09-10T04:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 --2- 172.21.15.46:0/3025808939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830100800 0x7f3830100c60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 --2- 172.21.15.46:0/3025808939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3830106800 0x7f3830106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:46.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 -- 172.21.15.46:0/3025808939 >> 172.21.15.46:0/3025808939 conn(0x7f38300fc520 msgr2=0x7f38300fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 -- 172.21.15.46:0/3025808939 shutdown_connections 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.953+0000 7f3836b5d640 1 -- 172.21.15.46:0/3025808939 wait complete. 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.954+0000 7f3836b5d640 1 Processor -- start 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.954+0000 7f3836b5d640 1 -- start start 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.954+0000 7f3836b5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3830100800 0x7f38300fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.954+0000 7f3836b5d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830106800 0x7f38300fd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.954+0000 7f3836b5d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f383010dcb0 con 0x7f3830106800 2024-09-10T04:21:46.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.954+0000 7f3836b5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f383010db30 con 0x7f3830100800 2024-09-10T04:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830106800 0x7f38300fd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830106800 0x7f38300fd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50208/0 (socket says 172.21.15.46:50208) 2024-09-10T04:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382ffff640 1 -- 172.21.15.46:0/1974883540 learned_addr learned my addr 172.21.15.46:0/1974883540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f38348f9640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3830100800 0x7f38300fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382ffff640 1 -- 172.21.15.46:0/1974883540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3830100800 msgr2=0x7f38300fcbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382ffff640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3830100800 0x7f38300fcbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:46.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382ffff640 1 -- 172.21.15.46:0/1974883540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38300fd670 con 0x7f3830106800 2024-09-10T04:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382ffff640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830106800 0x7f38300fd130 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3824031460 tx=0x7f38240045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382dffb640 1 -- 172.21.15.46:0/1974883540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f382403f070 con 0x7f3830106800 2024-09-10T04:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38300fd900 con 0x7f3830106800 2024-09-10T04:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382dffb640 1 -- 172.21.15.46:0/1974883540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38240035f0 con 0x7f3830106800 2024-09-10T04:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.955+0000 7f382dffb640 1 -- 172.21.15.46:0/1974883540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3824035030 con 0x7f3830106800 2024-09-10T04:21:46.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.956+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38301a3d60 con 0x7f3830106800 2024-09-10T04:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.957+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3830101f40 con 0x7f3830106800 2024-09-10T04:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.958+0000 7f382dffb640 1 -- 172.21.15.46:0/1974883540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3824042070 con 0x7f3830106800 2024-09-10T04:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.958+0000 7f382dffb640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f381807abf0 0x7f381807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.958+0000 7f382dffb640 1 -- 172.21.15.46:0/1974883540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f38240be5d0 con 0x7f3830106800 2024-09-10T04:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.958+0000 7f38348f9640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f381807abf0 0x7f381807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:46.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.959+0000 7f38348f9640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f381807abf0 0x7f381807d0b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3820005fd0 tx=0x7f3820009760 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:46.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:46.963+0000 7f382dffb640 1 -- 172.21.15.46:0/1974883540 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3824089d10 con 0x7f3830106800 2024-09-10T04:21:47.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.171+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3830062630 con 0x7f381807abf0 2024-09-10T04:21:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.173+0000 7f382dffb640 1 -- 172.21.15.46:0/1974883540 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f3830062630 con 0x7f381807abf0 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f381807abf0 msgr2=0x7f381807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f381807abf0 0x7f381807d0b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3820005fd0 tx=0x7f3820009760 comp rx=0 tx=0).stop 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830106800 msgr2=0x7f38300fd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830106800 0x7f38300fd130 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f3824031460 tx=0x7f38240045e0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 shutdown_connections 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f381807abf0 0x7f381807d0b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3830106800 0x7f38300fd130 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 --2- 172.21.15.46:0/1974883540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3830100800 0x7f38300fcbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 >> 172.21.15.46:0/1974883540 conn(0x7f38300fc520 msgr2=0x7f383006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 shutdown_connections 2024-09-10T04:21:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.175+0000 7f3836b5d640 1 -- 172.21.15.46:0/1974883540 wait complete. 2024-09-10T04:21:47.185 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:21:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 -- 172.21.15.46:0/4153429733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c107900 msgr2=0x7ff96c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 --2- 172.21.15.46:0/4153429733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c107900 0x7ff96c107ce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff968009880 tx=0x7ff96802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 -- 172.21.15.46:0/4153429733 shutdown_connections 2024-09-10T04:21:47.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 --2- 172.21.15.46:0/4153429733 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff96c101900 0x7ff96c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 --2- 172.21.15.46:0/4153429733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c107900 0x7ff96c107ce0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 -- 172.21.15.46:0/4153429733 >> 172.21.15.46:0/4153429733 conn(0x7ff96c0fd600 msgr2=0x7ff96c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 -- 172.21.15.46:0/4153429733 shutdown_connections 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.260+0000 7ff9741ad640 1 -- 172.21.15.46:0/4153429733 wait complete. 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.261+0000 7ff9741ad640 1 Processor -- start 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.261+0000 7ff9741ad640 1 -- start start 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.261+0000 7ff9741ad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c101900 0x7ff96c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.261+0000 7ff9741ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff96c107900 0x7ff96c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.261+0000 7ff9741ad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c10edd0 con 0x7ff96c101900 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.261+0000 7ff9741ad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c10ec50 con 0x7ff96c107900 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971f49640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c101900 0x7ff96c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971f49640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c101900 0x7ff96c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50224/0 (socket says 172.21.15.46:50224) 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971f49640 1 -- 172.21.15.46:0/3281807445 learned_addr learned my addr 172.21.15.46:0/3281807445 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971748640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff96c107900 0x7ff96c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971f49640 1 -- 172.21.15.46:0/3281807445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff96c107900 msgr2=0x7ff96c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971f49640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff96c107900 0x7ff96c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971f49640 1 -- 172.21.15.46:0/3281807445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c1977c0 con 0x7ff96c101900 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff971f49640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c101900 0x7ff96c19d640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff968031460 tx=0x7ff968004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff962ffd640 1 -- 172.21.15.46:0/3281807445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff96803f070 con 0x7ff96c101900 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c197a50 con 0x7ff96c101900 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff962ffd640 1 -- 172.21.15.46:0/3281807445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9680035f0 con 0x7ff96c101900 2024-09-10T04:21:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff962ffd640 1 -- 172.21.15.46:0/3281807445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff968035030 con 0x7ff96c101900 2024-09-10T04:21:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.262+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c197f80 con 0x7ff96c101900 2024-09-10T04:21:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.263+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c103040 con 0x7ff96c101900 2024-09-10T04:21:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.265+0000 7ff962ffd640 1 -- 172.21.15.46:0/3281807445 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff968042070 con 0x7ff96c101900 2024-09-10T04:21:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.265+0000 7ff962ffd640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff95807abf0 0x7ff95807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.265+0000 7ff962ffd640 1 -- 172.21.15.46:0/3281807445 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ff9680be660 con 0x7ff96c101900 2024-09-10T04:21:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.265+0000 7ff971748640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff95807abf0 0x7ff95807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.266+0000 7ff971748640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff95807abf0 0x7ff95807d0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0ffbf0 tx=0x7ff95c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:47.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.270+0000 7ff962ffd640 1 -- 172.21.15.46:0/3281807445 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff968089da0 con 0x7ff96c101900 2024-09-10T04:21:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.473+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff96c1982f0 con 0x7ff95807abf0 2024-09-10T04:21:47.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.474+0000 7ff962ffd640 1 -- 172.21.15.46:0/3281807445 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7ff96c1982f0 con 0x7ff95807abf0 2024-09-10T04:21:47.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.476+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff95807abf0 msgr2=0x7ff95807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.476+0000 7ff9741ad640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff95807abf0 0x7ff95807d0b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0ffbf0 tx=0x7ff95c005f90 comp rx=0 tx=0).stop 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.476+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c101900 msgr2=0x7ff96c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.476+0000 7ff9741ad640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c101900 0x7ff96c19d640 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff968031460 tx=0x7ff968004530 comp rx=0 tx=0).stop 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.476+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 shutdown_connections 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.477+0000 7ff9741ad640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff95807abf0 0x7ff95807d0b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.477+0000 7ff9741ad640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff96c107900 0x7ff96c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.477+0000 7ff9741ad640 1 --2- 172.21.15.46:0/3281807445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff96c101900 0x7ff96c19d640 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.477+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 >> 172.21.15.46:0/3281807445 conn(0x7ff96c0fd600 msgr2=0x7ff96c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.477+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 shutdown_connections 2024-09-10T04:21:47.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.477+0000 7ff9741ad640 1 -- 172.21.15.46:0/3281807445 wait complete. 2024-09-10T04:21:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.561+0000 7f3700ed2640 1 -- 172.21.15.46:0/4082945477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 msgr2=0x7f36fc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 --2- 172.21.15.46:0/4082945477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc100c50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f36e4009880 tx=0x7f36e402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 -- 172.21.15.46:0/4082945477 shutdown_connections 2024-09-10T04:21:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 --2- 172.21.15.46:0/4082945477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc100c50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 --2- 172.21.15.46:0/4082945477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36fc1067f0 0x7f36fc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 -- 172.21.15.46:0/4082945477 >> 172.21.15.46:0/4082945477 conn(0x7f36fc0fc550 msgr2=0x7f36fc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:47.564 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:47 smithi046 ceph-mon[109506]: pgmap v483: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 -- 172.21.15.46:0/4082945477 shutdown_connections 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 -- 172.21.15.46:0/4082945477 wait complete. 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.562+0000 7f3700ed2640 1 Processor -- start 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f3700ed2640 1 -- start start 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f3700ed2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f3700ed2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36fc1067f0 0x7f36fc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f3700ed2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc10dcc0 con 0x7f36fc1007f0 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f3700ed2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f36fc10db40 con 0x7f36fc1067f0 2024-09-10T04:21:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f36fa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f36fa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50230/0 (socket says 172.21.15.46:50230) 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f36fa59c640 1 -- 172.21.15.46:0/3357920268 learned_addr learned my addr 172.21.15.46:0/3357920268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f36f3fff640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36fc1067f0 0x7f36fc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f36fa59c640 1 -- 172.21.15.46:0/3357920268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36fc1067f0 msgr2=0x7f36fc19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f36fa59c640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36fc1067f0 0x7f36fc19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.563+0000 7f36fa59c640 1 -- 172.21.15.46:0/3357920268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36fc1999b0 con 0x7f36fc1007f0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.564+0000 7f36fa59c640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc19f7f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00d6f0 tx=0x7f36ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.564+0000 7f36f37fe640 1 -- 172.21.15.46:0/3357920268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f36ec017070 con 0x7f36fc1007f0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.564+0000 7f36f37fe640 1 -- 172.21.15.46:0/3357920268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f36ec00de00 con 0x7f36fc1007f0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.564+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36fc199ca0 con 0x7f36fc1007f0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.564+0000 7f36f37fe640 1 -- 172.21.15.46:0/3357920268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f36ec004bb0 con 0x7f36fc1007f0 2024-09-10T04:21:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.564+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36fc19a200 con 0x7f36fc1007f0 2024-09-10T04:21:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.565+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36fc101f30 con 0x7f36fc1007f0 2024-09-10T04:21:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.566+0000 7f36f37fe640 1 -- 172.21.15.46:0/3357920268 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f36ec009a30 con 0x7f36fc1007f0 2024-09-10T04:21:47.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.566+0000 7f36f37fe640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36e807ac50 0x7f36e807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.566+0000 7f36f37fe640 1 -- 172.21.15.46:0/3357920268 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f36ec09d740 con 0x7f36fc1007f0 2024-09-10T04:21:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.566+0000 7f36f3fff640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36e807ac50 0x7f36e807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.567+0000 7f36f3fff640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36e807ac50 0x7f36e807d110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36fc19b1a0 tx=0x7f36e4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.570+0000 7f36f37fe640 1 -- 172.21.15.46:0/3357920268 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f36ec068f00 con 0x7f36fc1007f0 2024-09-10T04:21:47.584 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:47 smithi070 ceph-mon[79774]: pgmap v483: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:47.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.766+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f36fc19a4b0 con 0x7f36e807ac50 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.776+0000 7f36f37fe640 1 -- 172.21.15.46:0/3357920268 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f36fc19a4b0 con 0x7f36e807ac50 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (13m) 7m ago 24m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (15m) 7m ago 24m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (15m) 7m ago 24m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (11m) 7m ago 24m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (11m) 7m ago 24m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (12m) 7m ago 24m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (7m) 7m ago 19m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:21:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (8m) 7m ago 19m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (7m) 7m ago 19m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (19m) 7m ago 19m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (15m) 7m ago 25m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (14m) 7m ago 23m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (12m) 7m ago 26m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (12m) 7m ago 23m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (14m) 7m ago 24m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (14m) 7m ago 23m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (11m) 7m ago 22m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (10m) 7m ago 22m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (10m) 7m ago 21m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:21:47.779 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (9m) 7m ago 21m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:21:47.780 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (9m) 7m ago 20m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:21:47.780 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (8m) 7m ago 20m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:21:47.780 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (13m) 7m ago 24m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:21:47.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36e807ac50 msgr2=0x7f36e807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36e807ac50 0x7f36e807d110 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f36fc19b1a0 tx=0x7f36e4031400 comp rx=0 tx=0).stop 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 msgr2=0x7f36fc19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc19f7f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f36ec00d6f0 tx=0x7f36ec00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 shutdown_connections 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36e807ac50 0x7f36e807d110 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f36fc1067f0 0x7f36fc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 --2- 172.21.15.46:0/3357920268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc1007f0 0x7f36fc19f7f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.778+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 >> 172.21.15.46:0/3357920268 conn(0x7f36fc0fc550 msgr2=0x7f36fc069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.779+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 shutdown_connections 2024-09-10T04:21:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.779+0000 7f3700ed2640 1 -- 172.21.15.46:0/3357920268 wait complete. 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.862+0000 7f4f69a3f640 1 -- 172.21.15.46:0/342489583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64107900 msgr2=0x7f4f64107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.862+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/342489583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64107900 0x7f4f64107ce0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009880 tx=0x7f4f5402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 -- 172.21.15.46:0/342489583 shutdown_connections 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/342489583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f64101900 0x7f4f64101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/342489583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64107900 0x7f4f64107ce0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 -- 172.21.15.46:0/342489583 >> 172.21.15.46:0/342489583 conn(0x7f4f640fd600 msgr2=0x7f4f640ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 -- 172.21.15.46:0/342489583 shutdown_connections 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 -- 172.21.15.46:0/342489583 wait complete. 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 Processor -- start 2024-09-10T04:21:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.863+0000 7f4f69a3f640 1 -- start start 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f69a3f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64101900 0x7f4f6419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f69a3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f64107900 0x7f4f6419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f69a3f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6410edd0 con 0x7f4f64101900 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f69a3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6410ec50 con 0x7f4f64107900 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f62ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64101900 0x7f4f6419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f62ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64101900 0x7f4f6419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50250/0 (socket says 172.21.15.46:50250) 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f62ffd640 1 -- 172.21.15.46:0/381233421 learned_addr learned my addr 172.21.15.46:0/381233421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f5bfff640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f64107900 0x7f4f6419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f62ffd640 1 -- 172.21.15.46:0/381233421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f64107900 msgr2=0x7f4f6419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f62ffd640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f64107900 0x7f4f6419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f62ffd640 1 -- 172.21.15.46:0/381233421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f641977a0 con 0x7f4f64101900 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f5bfff640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f64107900 0x7f4f6419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f62ffd640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64101900 0x7f4f6419d620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4f54031460 tx=0x7f4f54004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f68a3d640 1 -- 172.21.15.46:0/381233421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f5403f070 con 0x7f4f64101900 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f68a3d640 1 -- 172.21.15.46:0/381233421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f54004a90 con 0x7f4f64101900 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f64197a30 con 0x7f4f64101900 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f68a3d640 1 -- 172.21.15.46:0/381233421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f54035030 con 0x7f4f64101900 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.864+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f64197f60 con 0x7f4f64101900 2024-09-10T04:21:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.865+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f64103040 con 0x7f4f64101900 2024-09-10T04:21:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.866+0000 7f4f68a3d640 1 -- 172.21.15.46:0/381233421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4f54042070 con 0x7f4f64101900 2024-09-10T04:21:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.867+0000 7f4f68a3d640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f5007ac50 0x7f4f5007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.867+0000 7f4f68a3d640 1 -- 172.21.15.46:0/381233421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f4f540be730 con 0x7f4f64101900 2024-09-10T04:21:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.867+0000 7f4f5bfff640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f5007ac50 0x7f4f5007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.867+0000 7f4f5bfff640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f5007ac50 0x7f4f5007d110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4f640ffc20 tx=0x7f4f4c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:47.872+0000 7f4f68a3d640 1 -- 172.21.15.46:0/381233421 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4f54089ef0 con 0x7f4f64101900 2024-09-10T04:21:48.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.133+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f4f64198cd0 con 0x7f4f64101900 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.134+0000 7f4f68a3d640 1 -- 172.21.15.46:0/381233421 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f4f5408c410 con 0x7f4f64101900 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:21:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:21:48.137 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:21:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f5007ac50 msgr2=0x7f4f5007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f5007ac50 0x7f4f5007d110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4f640ffc20 tx=0x7f4f4c005f90 comp rx=0 tx=0).stop 2024-09-10T04:21:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64101900 msgr2=0x7f4f6419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64101900 0x7f4f6419d620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4f54031460 tx=0x7f4f54004530 comp rx=0 tx=0).stop 2024-09-10T04:21:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 shutdown_connections 2024-09-10T04:21:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f5007ac50 0x7f4f5007d110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f64107900 0x7f4f6419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 --2- 172.21.15.46:0/381233421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f64101900 0x7f4f6419d620 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 >> 172.21.15.46:0/381233421 conn(0x7f4f640fd600 msgr2=0x7f4f640fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 shutdown_connections 2024-09-10T04:21:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.137+0000 7f4f69a3f640 1 -- 172.21.15.46:0/381233421 wait complete. 2024-09-10T04:21:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 -- 172.21.15.46:0/2579845410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 msgr2=0x7f3abc104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 --2- 172.21.15.46:0/2579845410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc104c50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009880 tx=0x7f3aac02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 -- 172.21.15.46:0/2579845410 shutdown_connections 2024-09-10T04:21:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 --2- 172.21.15.46:0/2579845410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc104c50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 --2- 172.21.15.46:0/2579845410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc100250 0x7f3abc100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 -- 172.21.15.46:0/2579845410 >> 172.21.15.46:0/2579845410 conn(0x7f3abc0fbc90 msgr2=0x7f3abc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:48.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 -- 172.21.15.46:0/2579845410 shutdown_connections 2024-09-10T04:21:48.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.222+0000 7f3ac420c640 1 -- 172.21.15.46:0/2579845410 wait complete. 2024-09-10T04:21:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac420c640 1 Processor -- start 2024-09-10T04:21:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac420c640 1 -- start start 2024-09-10T04:21:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac420c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc100250 0x7f3abc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac420c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac420c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10cc40 con 0x7f3abc100c00 2024-09-10T04:21:48.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac420c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10cac0 con 0x7f3abc100250 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac17a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac17a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50268/0 (socket says 172.21.15.46:50268) 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac17a7640 1 -- 172.21.15.46:0/1557391558 learned_addr learned my addr 172.21.15.46:0/1557391558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.223+0000 7f3ac1fa8640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc100250 0x7f3abc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ac17a7640 1 -- 172.21.15.46:0/1557391558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc100250 msgr2=0x7f3abc19f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ac17a7640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc100250 0x7f3abc19f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ac17a7640 1 -- 172.21.15.46:0/1557391558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc199a40 con 0x7f3abc100c00 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ac17a7640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc19fdc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3aac031460 tx=0x7f3aac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ab2ffd640 1 -- 172.21.15.46:0/1557391558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3aac03f070 con 0x7f3abc100c00 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc199cd0 con 0x7f3abc100c00 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ab2ffd640 1 -- 172.21.15.46:0/1557391558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3aac0035f0 con 0x7f3abc100c00 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ab2ffd640 1 -- 172.21.15.46:0/1557391558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3aac035030 con 0x7f3abc100c00 2024-09-10T04:21:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.224+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc19a200 con 0x7f3abc100c00 2024-09-10T04:21:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.225+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc101730 con 0x7f3abc100c00 2024-09-10T04:21:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.226+0000 7f3ab2ffd640 1 -- 172.21.15.46:0/1557391558 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3aac042070 con 0x7f3abc100c00 2024-09-10T04:21:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.226+0000 7f3ab2ffd640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3aa807abf0 0x7f3aa807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.227+0000 7f3ab2ffd640 1 -- 172.21.15.46:0/1557391558 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f3aac0be670 con 0x7f3abc100c00 2024-09-10T04:21:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.227+0000 7f3ac1fa8640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3aa807abf0 0x7f3aa807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.227+0000 7f3ac1fa8640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3aa807abf0 0x7f3aa807d0b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3abc19ab80 tx=0x7f3ab800b490 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:48.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.231+0000 7f3ab2ffd640 1 -- 172.21.15.46:0/1557391558 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3aac089d10 con 0x7f3abc100c00 2024-09-10T04:21:48.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.453+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3abc108560 con 0x7f3abc100c00 2024-09-10T04:21:48.455 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:48 smithi046 ceph-mon[109506]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:48.455 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:48 smithi046 ceph-mon[109506]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:48.455 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:48 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/381233421' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:21:48.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.453+0000 7f3ab2ffd640 1 -- 172.21.15.46:0/1557391558 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f3aac08ebc0 con 0x7f3abc100c00 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:21:48.456 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:21:48.457 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:21:48.457 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:21:48.457 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:21:48.457 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:21:48.457 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:21:48.458 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:21:48.458 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:21:48.458 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:21:48.458 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:21:48.458 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:21:48.458 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:21:48.458 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:21:48.459 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:21:48.460 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:21:48.461 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:21:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:48.461 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:21:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:21:48.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:48.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:48.462 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:48.462 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:21:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:21:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3aa807abf0 msgr2=0x7f3aa807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3aa807abf0 0x7f3aa807d0b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3abc19ab80 tx=0x7f3ab800b490 comp rx=0 tx=0).stop 2024-09-10T04:21:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 msgr2=0x7f3abc19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc19fdc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3aac031460 tx=0x7f3aac0045e0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 shutdown_connections 2024-09-10T04:21:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3aa807abf0 0x7f3aa807d0b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3abc100c00 0x7f3abc19fdc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 --2- 172.21.15.46:0/1557391558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc100250 0x7f3abc19f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 >> 172.21.15.46:0/1557391558 conn(0x7f3abc0fbc90 msgr2=0x7f3abc0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 shutdown_connections 2024-09-10T04:21:48.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.455+0000 7f3ac420c640 1 -- 172.21.15.46:0/1557391558 wait complete. 2024-09-10T04:21:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 -- 172.21.15.46:0/1863401000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 msgr2=0x7f81f4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 --2- 172.21.15.46:0/1863401000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4106be0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009880 tx=0x7f81e802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 -- 172.21.15.46:0/1863401000 shutdown_connections 2024-09-10T04:21:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 --2- 172.21.15.46:0/1863401000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f4100800 0x7f81f4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 --2- 172.21.15.46:0/1863401000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4106be0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 -- 172.21.15.46:0/1863401000 >> 172.21.15.46:0/1863401000 conn(0x7f81f40fc520 msgr2=0x7f81f40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 -- 172.21.15.46:0/1863401000 shutdown_connections 2024-09-10T04:21:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 -- 172.21.15.46:0/1863401000 wait complete. 2024-09-10T04:21:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.541+0000 7f81faa67640 1 Processor -- start 2024-09-10T04:21:48.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81faa67640 1 -- start start 2024-09-10T04:21:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81faa67640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f4100800 0x7f81f41987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81faa67640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81faa67640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81f410dcd0 con 0x7f81f4106800 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81faa67640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f81f410db50 con 0x7f81f4100800 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50284/0 (socket says 172.21.15.46:50284) 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f3fff640 1 -- 172.21.15.46:0/3542204008 learned_addr learned my addr 172.21.15.46:0/3542204008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f8803640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f4100800 0x7f81f41987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f3fff640 1 -- 172.21.15.46:0/3542204008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f4100800 msgr2=0x7f81f41987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f3fff640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f4100800 0x7f81f41987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f3fff640 1 -- 172.21.15.46:0/3542204008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f4199320 con 0x7f81f4106800 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f8803640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f4100800 0x7f81f41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.542+0000 7f81f3fff640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4198d20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f81e000e970 tx=0x7f81e000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.543+0000 7f81f1ffb640 1 -- 172.21.15.46:0/3542204008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f81e000cac0 con 0x7f81f4106800 2024-09-10T04:21:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.543+0000 7f81f1ffb640 1 -- 172.21.15.46:0/3542204008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81e00054e0 con 0x7f81f4106800 2024-09-10T04:21:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.543+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f4199610 con 0x7f81f4106800 2024-09-10T04:21:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.543+0000 7f81f1ffb640 1 -- 172.21.15.46:0/3542204008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f81e0010430 con 0x7f81f4106800 2024-09-10T04:21:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.543+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f41a1c10 con 0x7f81f4106800 2024-09-10T04:21:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.544+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f4101f40 con 0x7f81f4106800 2024-09-10T04:21:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.545+0000 7f81f1ffb640 1 -- 172.21.15.46:0/3542204008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f81e0004f00 con 0x7f81f4106800 2024-09-10T04:21:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.545+0000 7f81f1ffb640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f81e407ac40 0x7f81e407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.545+0000 7f81f1ffb640 1 -- 172.21.15.46:0/3542204008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f81e009e370 con 0x7f81f4106800 2024-09-10T04:21:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.545+0000 7f81f8803640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f81e407ac40 0x7f81e407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.546+0000 7f81f8803640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f81e407ac40 0x7f81e407d100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81e8006010 tx=0x7f81e80095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.550+0000 7f81f1ffb640 1 -- 172.21.15.46:0/3542204008 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f81f4199610 con 0x7f81f4106800 2024-09-10T04:21:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:48 smithi070 ceph-mon[79774]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:48 smithi070 ceph-mon[79774]: from='client.34762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:48.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:48 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/381233421' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:21:48.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.753+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f81f4062630 con 0x7f81e407ac40 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.754+0000 7f81f1ffb640 1 -- 172.21.15.46:0/3542204008 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f81f4062630 con 0x7f81e407ac40 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:21:48.756 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:21:48.757 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:21:48.757 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:21:48.757 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:21:48.757 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:21:48.757 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:21:48.757 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f81e407ac40 msgr2=0x7f81e407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f81e407ac40 0x7f81e407d100 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f81e8006010 tx=0x7f81e80095f0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 msgr2=0x7f81f4198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4198d20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f81e000e970 tx=0x7f81e000ed70 comp rx=0 tx=0).stop 2024-09-10T04:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 shutdown_connections 2024-09-10T04:21:48.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f81e407ac40 0x7f81e407d100 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f4106800 0x7f81f4198d20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 --2- 172.21.15.46:0/3542204008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f4100800 0x7f81f41987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.757+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 >> 172.21.15.46:0/3542204008 conn(0x7f81f40fc520 msgr2=0x7f81f40fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.758+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 shutdown_connections 2024-09-10T04:21:48.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.758+0000 7f81faa67640 1 -- 172.21.15.46:0/3542204008 wait complete. 2024-09-10T04:21:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 -- 172.21.15.46:0/2610728636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe194101ba0 msgr2=0x7fe194105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 --2- 172.21.15.46:0/2610728636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe194101ba0 0x7fe194105480 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009880 tx=0x7fe17c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 -- 172.21.15.46:0/2610728636 shutdown_connections 2024-09-10T04:21:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 --2- 172.21.15.46:0/2610728636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe194101ba0 0x7fe194105480 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 --2- 172.21.15.46:0/2610728636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1941011f0 0x7fe1941015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 -- 172.21.15.46:0/2610728636 >> 172.21.15.46:0/2610728636 conn(0x7fe1940fbc90 msgr2=0x7fe1940fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 -- 172.21.15.46:0/2610728636 shutdown_connections 2024-09-10T04:21:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.842+0000 7fe199e4c640 1 -- 172.21.15.46:0/2610728636 wait complete. 2024-09-10T04:21:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe199e4c640 1 Processor -- start 2024-09-10T04:21:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe199e4c640 1 -- start start 2024-09-10T04:21:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe199e4c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1941011f0 0x7fe194102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe199e4c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194101ba0 0x7fe1941029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe199e4c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe19410dd20 con 0x7fe1941011f0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe199e4c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe19410dba0 con 0x7fe194101ba0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe1937fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1941011f0 0x7fe194102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe1937fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1941011f0 0x7fe194102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34566/0 (socket says 172.21.15.46:34566) 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe1937fe640 1 -- 172.21.15.46:0/3511023434 learned_addr learned my addr 172.21.15.46:0/3511023434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe192ffd640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194101ba0 0x7fe1941029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe1937fe640 1 -- 172.21.15.46:0/3511023434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194101ba0 msgr2=0x7fe1941029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe1937fe640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194101ba0 0x7fe1941029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.843+0000 7fe1937fe640 1 -- 172.21.15.46:0/3511023434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe194102f10 con 0x7fe1941011f0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.844+0000 7fe1937fe640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1941011f0 0x7fe194102490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe18400d8d0 tx=0x7fe18400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.844+0000 7fe190ff9640 1 -- 172.21.15.46:0/3511023434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe184017070 con 0x7fe1941011f0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.844+0000 7fe190ff9640 1 -- 172.21.15.46:0/3511023434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe184010460 con 0x7fe1941011f0 2024-09-10T04:21:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.844+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe194103200 con 0x7fe1941011f0 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.844+0000 7fe190ff9640 1 -- 172.21.15.46:0/3511023434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe18400f5f0 con 0x7fe1941011f0 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.844+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941a3eb0 con 0x7fe1941011f0 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.845+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1941021b0 con 0x7fe1941011f0 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.846+0000 7fe190ff9640 1 -- 172.21.15.46:0/3511023434 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe184009a30 con 0x7fe1941011f0 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.846+0000 7fe190ff9640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe18007abf0 0x7fe18007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.847+0000 7fe192ffd640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe18007abf0 0x7fe18007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.847+0000 7fe190ff9640 1 -- 172.21.15.46:0/3511023434 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fe18409db40 con 0x7fe1941011f0 2024-09-10T04:21:48.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.847+0000 7fe192ffd640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe18007abf0 0x7fe18007d0b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:21:48.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:48.851+0000 7fe190ff9640 1 -- 172.21.15.46:0/3511023434 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe184069280 con 0x7fe1941011f0 2024-09-10T04:21:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.113+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe1941a4140 con 0x7fe1941011f0 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.114+0000 7fe190ff9640 1 -- 172.21.15.46:0/3511023434 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fe18406e130 con 0x7fe1941011f0 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:21:49.116 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:21:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe18007abf0 msgr2=0x7fe18007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe18007abf0 0x7fe18007d0b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c031400 comp rx=0 tx=0).stop 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1941011f0 msgr2=0x7fe194102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1941011f0 0x7fe194102490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe18400d8d0 tx=0x7fe18400dd90 comp rx=0 tx=0).stop 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 shutdown_connections 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe18007abf0 0x7fe18007d0b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe194101ba0 0x7fe1941029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 --2- 172.21.15.46:0/3511023434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe1941011f0 0x7fe194102490 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 >> 172.21.15.46:0/3511023434 conn(0x7fe1940fbc90 msgr2=0x7fe1940fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 shutdown_connections 2024-09-10T04:21:49.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:21:49.116+0000 7fe199e4c640 1 -- 172.21.15.46:0/3511023434 wait complete. 2024-09-10T04:21:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:49 smithi070 ceph-mon[79774]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:49 smithi070 ceph-mon[79774]: pgmap v484: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:49 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1557391558' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:21:49.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:49 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3511023434' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:21:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:49 smithi046 ceph-mon[109506]: from='client.34766 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:49 smithi046 ceph-mon[109506]: pgmap v484: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:49 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1557391558' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:21:49.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:49 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3511023434' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:21:50.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:50 smithi070 ceph-mon[79774]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:50.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:50 smithi046 ceph-mon[109506]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:21:51.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:51 smithi070 ceph-mon[79774]: pgmap v485: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:51.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:51 smithi046 ceph-mon[109506]: pgmap v485: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:53 smithi046 ceph-mon[109506]: pgmap v486: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:53.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:53 smithi070 ceph-mon[79774]: pgmap v486: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:53.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:21:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:55 smithi070 ceph-mon[79774]: pgmap v487: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:55 smithi046 ceph-mon[109506]: pgmap v487: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:57 smithi070 ceph-mon[79774]: pgmap v488: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:57 smithi046 ceph-mon[109506]: pgmap v488: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:21:59 smithi046 ceph-mon[109506]: pgmap v489: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:21:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:21:59 smithi070 ceph-mon[79774]: pgmap v489: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:01.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:01 smithi046 ceph-mon[109506]: pgmap v490: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:01 smithi070 ceph-mon[79774]: pgmap v490: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:03 smithi070 ceph-mon[79774]: pgmap v491: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:03 smithi046 ceph-mon[109506]: pgmap v491: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:05.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:05 smithi046 ceph-mon[109506]: pgmap v492: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:05 smithi070 ceph-mon[79774]: pgmap v492: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:07 smithi070 ceph-mon[79774]: pgmap v493: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:07 smithi046 ceph-mon[109506]: pgmap v493: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:09.654 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:09 smithi046 ceph-mon[109506]: pgmap v494: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:09 smithi070 ceph-mon[79774]: pgmap v494: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:11.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:11 smithi070 ceph-mon[79774]: pgmap v495: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:11.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:11 smithi046 ceph-mon[109506]: pgmap v495: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:13 smithi070 ceph-mon[79774]: pgmap v496: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:13 smithi046 ceph-mon[109506]: pgmap v496: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:15 smithi070 ceph-mon[79774]: pgmap v497: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:15 smithi046 ceph-mon[109506]: pgmap v497: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:17 smithi046 ceph-mon[109506]: pgmap v498: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:17 smithi070 ceph-mon[79774]: pgmap v498: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1363531812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c41000e0 msgr2=0x7ff9c41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1363531812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c41000e0 0x7ff9c41004c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009880 tx=0x7ff9b402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1363531812 shutdown_connections 2024-09-10T04:22:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1363531812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c4100a90 0x7ff9c41065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1363531812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c41000e0 0x7ff9c41004c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1363531812 >> 172.21.15.46:0/1363531812 conn(0x7ff9c40fbc90 msgr2=0x7ff9c40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1363531812 shutdown_connections 2024-09-10T04:22:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.214+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1363531812 wait complete. 2024-09-10T04:22:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c8ede640 1 Processor -- start 2024-09-10T04:22:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c8ede640 1 -- start start 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c8ede640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c41000e0 0x7ff9c4101060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c8ede640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a90 0x7ff9c41015a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c8ede640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410ee50 con 0x7ff9c4100a90 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c8ede640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c410ecd0 con 0x7ff9c41000e0 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c1d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a90 0x7ff9c41015a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c1d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a90 0x7ff9c41015a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33204/0 (socket says 172.21.15.46:33204) 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.215+0000 7ff9c1d9b640 1 -- 172.21.15.46:0/1279482661 learned_addr learned my addr 172.21.15.46:0/1279482661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9c259c640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c41000e0 0x7ff9c4101060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9c1d9b640 1 -- 172.21.15.46:0/1279482661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c41000e0 msgr2=0x7ff9c4101060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9c1d9b640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c41000e0 0x7ff9c4101060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9c1d9b640 1 -- 172.21.15.46:0/1279482661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c4101ae0 con 0x7ff9c4100a90 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9c1d9b640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a90 0x7ff9c41015a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800d6f0 tx=0x7ff9b800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9b37fe640 1 -- 172.21.15.46:0/1279482661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff9b8017070 con 0x7ff9c4100a90 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c4101dd0 con 0x7ff9c4100a90 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9b37fe640 1 -- 172.21.15.46:0/1279482661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff9b800de00 con 0x7ff9c4100a90 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a3f20 con 0x7ff9c4100a90 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.216+0000 7ff9b37fe640 1 -- 172.21.15.46:0/1279482661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff9b8004bb0 con 0x7ff9c4100a90 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.217+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a4005290 con 0x7ff9c4100a90 2024-09-10T04:22:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.218+0000 7ff9b37fe640 1 -- 172.21.15.46:0/1279482661 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff9b8009a30 con 0x7ff9c4100a90 2024-09-10T04:22:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.218+0000 7ff9b37fe640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff9ac07ac40 0x7ff9ac07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.219+0000 7ff9b37fe640 1 -- 172.21.15.46:0/1279482661 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ff9b809d6b0 con 0x7ff9c4100a90 2024-09-10T04:22:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.219+0000 7ff9c259c640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff9ac07ac40 0x7ff9ac07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.219+0000 7ff9c259c640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff9ac07ac40 0x7ff9ac07d100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40096c0 tx=0x7ff9b4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.223+0000 7ff9b37fe640 1 -- 172.21.15.46:0/1279482661 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff9b8068e70 con 0x7ff9c4100a90 2024-09-10T04:22:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.428+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff9a4000bc0 con 0x7ff9ac07ac40 2024-09-10T04:22:19.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.430+0000 7ff9b37fe640 1 -- 172.21.15.46:0/1279482661 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7ff9a4000bc0 con 0x7ff9ac07ac40 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff9ac07ac40 msgr2=0x7ff9ac07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff9ac07ac40 0x7ff9ac07d100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40096c0 tx=0x7ff9b4031400 comp rx=0 tx=0).stop 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a90 msgr2=0x7ff9c41015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a90 0x7ff9c41015a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800d6f0 tx=0x7ff9b800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 shutdown_connections 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff9ac07ac40 0x7ff9ac07d100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9c4100a90 0x7ff9c41015a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 --2- 172.21.15.46:0/1279482661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9c41000e0 0x7ff9c4101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 >> 172.21.15.46:0/1279482661 conn(0x7ff9c40fbc90 msgr2=0x7ff9c40fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 shutdown_connections 2024-09-10T04:22:19.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.432+0000 7ff9c8ede640 1 -- 172.21.15.46:0/1279482661 wait complete. 2024-09-10T04:22:19.443 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:22:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 -- 172.21.15.46:0/1504361035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 msgr2=0x7facb4100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 --2- 172.21.15.46:0/1504361035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4100630 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7facb0009880 tx=0x7facb002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 -- 172.21.15.46:0/1504361035 shutdown_connections 2024-09-10T04:22:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 --2- 172.21.15.46:0/1504361035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facb4100c00 0x7facb4104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 --2- 172.21.15.46:0/1504361035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4100630 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 -- 172.21.15.46:0/1504361035 >> 172.21.15.46:0/1504361035 conn(0x7facb40fbc90 msgr2=0x7facb40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:19.521 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:19 smithi046 ceph-mon[109506]: pgmap v499: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 -- 172.21.15.46:0/1504361035 shutdown_connections 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.518+0000 7facbb6d1640 1 -- 172.21.15.46:0/1504361035 wait complete. 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facbb6d1640 1 Processor -- start 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facbb6d1640 1 -- start start 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facbb6d1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facbb6d1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facb4100c00 0x7facb4102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facbb6d1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7facb410dd30 con 0x7facb4100250 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facbb6d1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7facb410dbb0 con 0x7facb4100c00 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facb946d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facb946d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33210/0 (socket says 172.21.15.46:33210) 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.519+0000 7facb946d640 1 -- 172.21.15.46:0/32634718 learned_addr learned my addr 172.21.15.46:0/32634718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facb8c6c640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facb4100c00 0x7facb4102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facb946d640 1 -- 172.21.15.46:0/32634718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facb4100c00 msgr2=0x7facb4102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facb946d640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facb4100c00 0x7facb4102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facb946d640 1 -- 172.21.15.46:0/32634718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb4102770 con 0x7facb4100250 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facb946d640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4101cf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facb0031460 tx=0x7facb00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facaa7fc640 1 -- 172.21.15.46:0/32634718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7facb003f070 con 0x7facb4100250 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facaa7fc640 1 -- 172.21.15.46:0/32634718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7facb0004cf0 con 0x7facb4100250 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facb4102a00 con 0x7facb4100250 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facaa7fc640 1 -- 172.21.15.46:0/32634718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7facb0035030 con 0x7facb4100250 2024-09-10T04:22:19.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.520+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facb41a3e20 con 0x7facb4100250 2024-09-10T04:22:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.522+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facb4101730 con 0x7facb4100250 2024-09-10T04:22:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.522+0000 7facaa7fc640 1 -- 172.21.15.46:0/32634718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7facb0042070 con 0x7facb4100250 2024-09-10T04:22:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.523+0000 7facaa7fc640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faca007ac40 0x7faca007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.523+0000 7facb8c6c640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faca007ac40 0x7faca007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.523+0000 7facaa7fc640 1 -- 172.21.15.46:0/32634718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7facb00be640 con 0x7facb4100250 2024-09-10T04:22:19.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.524+0000 7facb8c6c640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faca007ac40 0x7faca007d100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facb4103780 tx=0x7faca400a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:19.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.528+0000 7facaa7fc640 1 -- 172.21.15.46:0/32634718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7facb0089da0 con 0x7facb4100250 2024-09-10T04:22:19.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.732+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7facb40fd840 con 0x7faca007ac40 2024-09-10T04:22:19.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.733+0000 7facaa7fc640 1 -- 172.21.15.46:0/32634718 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7facb40fd840 con 0x7faca007ac40 2024-09-10T04:22:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.735+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faca007ac40 msgr2=0x7faca007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faca007ac40 0x7faca007d100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facb4103780 tx=0x7faca400a2d0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 msgr2=0x7facb4101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4101cf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7facb0031460 tx=0x7facb00048f0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 shutdown_connections 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faca007ac40 0x7faca007d100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facb4100c00 0x7facb4102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 --2- 172.21.15.46:0/32634718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facb4100250 0x7facb4101cf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 >> 172.21.15.46:0/32634718 conn(0x7facb40fbc90 msgr2=0x7facb410c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 shutdown_connections 2024-09-10T04:22:19.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.736+0000 7facbb6d1640 1 -- 172.21.15.46:0/32634718 wait complete. 2024-09-10T04:22:19.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.823+0000 7f81966c2640 1 -- 172.21.15.46:0/1617747198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 msgr2=0x7f8190107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.823+0000 7f81966c2640 1 --2- 172.21.15.46:0/1617747198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f8190107ce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8180009880 tx=0x7f818002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 -- 172.21.15.46:0/1617747198 shutdown_connections 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 --2- 172.21.15.46:0/1617747198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8190101900 0x7f8190101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 --2- 172.21.15.46:0/1617747198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f8190107ce0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 -- 172.21.15.46:0/1617747198 >> 172.21.15.46:0/1617747198 conn(0x7f81900fd600 msgr2=0x7f81900ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 -- 172.21.15.46:0/1617747198 shutdown_connections 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 -- 172.21.15.46:0/1617747198 wait complete. 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 Processor -- start 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.824+0000 7f81966c2640 1 -- start start 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f81966c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8190101900 0x7f819019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f81966c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f819019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f81966c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f819010edd0 con 0x7f8190107900 2024-09-10T04:22:19.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f81966c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f819010ec50 con 0x7f8190101900 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f8187fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f819019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f8187fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f819019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33224/0 (socket says 172.21.15.46:33224) 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f8187fff640 1 -- 172.21.15.46:0/2713756980 learned_addr learned my addr 172.21.15.46:0/2713756980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f818ffff640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8190101900 0x7f819019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f8187fff640 1 -- 172.21.15.46:0/2713756980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8190101900 msgr2=0x7f819019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f8187fff640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8190101900 0x7f819019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f8187fff640 1 -- 172.21.15.46:0/2713756980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81901977a0 con 0x7f8190107900 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.825+0000 7f8187fff640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f819019db60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f817800d6f0 tx=0x7f817800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.826+0000 7f818dffb640 1 -- 172.21.15.46:0/2713756980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8178017070 con 0x7f8190107900 2024-09-10T04:22:19.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.826+0000 7f818dffb640 1 -- 172.21.15.46:0/2713756980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f817800de00 con 0x7f8190107900 2024-09-10T04:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.826+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8190197a90 con 0x7f8190107900 2024-09-10T04:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.826+0000 7f818dffb640 1 -- 172.21.15.46:0/2713756980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8178004bb0 con 0x7f8190107900 2024-09-10T04:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.826+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8190198020 con 0x7f8190107900 2024-09-10T04:22:19.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.827+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8190103040 con 0x7f8190107900 2024-09-10T04:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.828+0000 7f818dffb640 1 -- 172.21.15.46:0/2713756980 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8178004e60 con 0x7f8190107900 2024-09-10T04:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.828+0000 7f818dffb640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f817c07abf0 0x7f817c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.828+0000 7f818dffb640 1 -- 172.21.15.46:0/2713756980 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f817809d820 con 0x7f8190107900 2024-09-10T04:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.828+0000 7f818ffff640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f817c07abf0 0x7f817c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:19.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.829+0000 7f818ffff640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f817c07abf0 0x7f817c07d0b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81800096c0 tx=0x7f8180031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:19.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:19.833+0000 7f818dffb640 1 -- 172.21.15.46:0/2713756980 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8178068f60 con 0x7f8190107900 2024-09-10T04:22:19.834 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:19 smithi070 ceph-mon[79774]: pgmap v499: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:20.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.031+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f81901982d0 con 0x7f817c07abf0 2024-09-10T04:22:20.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.040+0000 7f818dffb640 1 -- 172.21.15.46:0/2713756980 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f81901982d0 con 0x7f817c07abf0 2024-09-10T04:22:20.042 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:22:20.042 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (13m) 8m ago 25m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:22:20.042 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (16m) 8m ago 25m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:22:20.042 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (16m) 8m ago 24m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (12m) 8m ago 25m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (12m) 8m ago 24m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (13m) 8m ago 25m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (8m) 8m ago 19m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (8m) 8m ago 19m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (8m) 8m ago 19m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (19m) 8m ago 19m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (15m) 8m ago 26m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (14m) 8m ago 24m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (12m) 8m ago 26m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (12m) 8m ago 24m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (14m) 8m ago 25m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:22:20.043 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (14m) 8m ago 24m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:22:20.044 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (11m) 8m ago 23m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:22:20.044 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (11m) 8m ago 22m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:22:20.044 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (10m) 8m ago 22m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:22:20.044 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (10m) 8m ago 21m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:22:20.044 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (9m) 8m ago 21m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:22:20.044 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (9m) 8m ago 20m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:22:20.044 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (14m) 8m ago 24m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f817c07abf0 msgr2=0x7f817c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f817c07abf0 0x7f817c07d0b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f81800096c0 tx=0x7f8180031400 comp rx=0 tx=0).stop 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 msgr2=0x7f819019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f819019db60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f817800d6f0 tx=0x7f817800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 shutdown_connections 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f817c07abf0 0x7f817c07d0b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8190107900 0x7f819019db60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 --2- 172.21.15.46:0/2713756980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8190101900 0x7f819019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.042+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 >> 172.21.15.46:0/2713756980 conn(0x7f81900fd600 msgr2=0x7f81900fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.043+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 shutdown_connections 2024-09-10T04:22:20.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.043+0000 7f81966c2640 1 -- 172.21.15.46:0/2713756980 wait complete. 2024-09-10T04:22:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.126+0000 7f84185b2640 1 -- 172.21.15.46:0/3918358722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410100800 msgr2=0x7f8410100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.126+0000 7f84185b2640 1 --2- 172.21.15.46:0/3918358722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410100800 0x7f8410100c60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8400009880 tx=0x7f840002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.126+0000 7f84185b2640 1 -- 172.21.15.46:0/3918358722 shutdown_connections 2024-09-10T04:22:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.126+0000 7f84185b2640 1 --2- 172.21.15.46:0/3918358722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410100800 0x7f8410100c60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.126+0000 7f84185b2640 1 --2- 172.21.15.46:0/3918358722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8410106800 0x7f8410106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.126+0000 7f84185b2640 1 -- 172.21.15.46:0/3918358722 >> 172.21.15.46:0/3918358722 conn(0x7f84100fc520 msgr2=0x7f84100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 -- 172.21.15.46:0/3918358722 shutdown_connections 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 -- 172.21.15.46:0/3918358722 wait complete. 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 Processor -- start 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 -- start start 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8410100800 0x7f84100fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410106800 0x7f84100fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f841010dcd0 con 0x7f8410106800 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.127+0000 7f84185b2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f841010db50 con 0x7f8410100800 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f8415b4d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410106800 0x7f84100fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f8415b4d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410106800 0x7f84100fd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33250/0 (socket says 172.21.15.46:33250) 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f8415b4d640 1 -- 172.21.15.46:0/2793597245 learned_addr learned my addr 172.21.15.46:0/2793597245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f841634e640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8410100800 0x7f84100fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f8415b4d640 1 -- 172.21.15.46:0/2793597245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8410100800 msgr2=0x7f84100fcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f8415b4d640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8410100800 0x7f84100fcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f8415b4d640 1 -- 172.21.15.46:0/2793597245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84100fd6b0 con 0x7f8410106800 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f8415b4d640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410106800 0x7f84100fd170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8400031460 tx=0x7f84000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f84077fe640 1 -- 172.21.15.46:0/2793597245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f840003f070 con 0x7f8410106800 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84100fd940 con 0x7f8410106800 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f84077fe640 1 -- 172.21.15.46:0/2793597245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8400004b10 con 0x7f8410106800 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f84077fe640 1 -- 172.21.15.46:0/2793597245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8400035030 con 0x7f8410106800 2024-09-10T04:22:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.128+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84101a3db0 con 0x7f8410106800 2024-09-10T04:22:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.129+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8410101f40 con 0x7f8410106800 2024-09-10T04:22:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.130+0000 7f84077fe640 1 -- 172.21.15.46:0/2793597245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8400042070 con 0x7f8410106800 2024-09-10T04:22:20.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.130+0000 7f84077fe640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f83fc07ac00 0x7f83fc07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.131+0000 7f84077fe640 1 -- 172.21.15.46:0/2793597245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f84000be7f0 con 0x7f8410106800 2024-09-10T04:22:20.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.131+0000 7f841634e640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f83fc07ac00 0x7f83fc07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.131+0000 7f841634e640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f83fc07ac00 0x7f83fc07d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f840c005fd0 tx=0x7f840c00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.135+0000 7f84077fe640 1 -- 172.21.15.46:0/2793597245 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8400089e90 con 0x7f8410106800 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.395+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f8410109d80 con 0x7f8410106800 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.395+0000 7f84077fe640 1 -- 172.21.15.46:0/2793597245 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f840008c3b0 con 0x7f8410106800 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:22:20.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:22:20.399 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:22:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f83fc07ac00 msgr2=0x7f83fc07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f83fc07ac00 0x7f83fc07d0c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f840c005fd0 tx=0x7f840c00b390 comp rx=0 tx=0).stop 2024-09-10T04:22:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410106800 msgr2=0x7f84100fd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410106800 0x7f84100fd170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8400031460 tx=0x7f84000045e0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 shutdown_connections 2024-09-10T04:22:20.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f83fc07ac00 0x7f83fc07d0c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410106800 0x7f84100fd170 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 --2- 172.21.15.46:0/2793597245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8410100800 0x7f84100fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 >> 172.21.15.46:0/2793597245 conn(0x7f84100fc520 msgr2=0x7f841006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 shutdown_connections 2024-09-10T04:22:20.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.397+0000 7f84185b2640 1 -- 172.21.15.46:0/2793597245 wait complete. 2024-09-10T04:22:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.481+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4270057178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8103490 msgr2=0x7f03b810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4270057178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8103490 0x7f03b810b980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009880 tx=0x7f03a802a530 comp rx=0 tx=0).stop 2024-09-10T04:22:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4270057178 shutdown_connections 2024-09-10T04:22:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4270057178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8103490 0x7f03b810b980 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4270057178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b8102b70 0x7f03b8102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4270057178 >> 172.21.15.46:0/4270057178 conn(0x7f03b80fc550 msgr2=0x7f03b80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4270057178 shutdown_connections 2024-09-10T04:22:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4270057178 wait complete. 2024-09-10T04:22:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 Processor -- start 2024-09-10T04:22:20.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.482+0000 7f03bf7bb640 1 -- start start 2024-09-10T04:22:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bf7bb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8102b70 0x7f03b81007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bf7bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b8103490 0x7f03b8100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bf7bb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03b810edc0 con 0x7f03b8102b70 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bf7bb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f03b810ec40 con 0x7f03b8103490 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bd557640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8102b70 0x7f03b81007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bd557640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8102b70 0x7f03b81007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33266/0 (socket says 172.21.15.46:33266) 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bd557640 1 -- 172.21.15.46:0/4127155361 learned_addr learned my addr 172.21.15.46:0/4127155361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bcd56640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b8103490 0x7f03b8100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bd557640 1 -- 172.21.15.46:0/4127155361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b8103490 msgr2=0x7f03b8100d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bd557640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b8103490 0x7f03b8100d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bd557640 1 -- 172.21.15.46:0/4127155361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b8101250 con 0x7f03b8102b70 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bcd56640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b8103490 0x7f03b8100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bd557640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8102b70 0x7f03b81007d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f03b400db80 tx=0x7f03b400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03ae7fc640 1 -- 172.21.15.46:0/4127155361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f03b4017070 con 0x7f03b8102b70 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03ae7fc640 1 -- 172.21.15.46:0/4127155361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f03b4003e30 con 0x7f03b8102b70 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b8101540 con 0x7f03b8102b70 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03ae7fc640 1 -- 172.21.15.46:0/4127155361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f03b4010460 con 0x7f03b8102b70 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.483+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b81a3e10 con 0x7f03b8102b70 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.484+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b806a4c0 con 0x7f03b8102b70 2024-09-10T04:22:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.485+0000 7f03ae7fc640 1 -- 172.21.15.46:0/4127155361 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f03b400dc70 con 0x7f03b8102b70 2024-09-10T04:22:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.486+0000 7f03ae7fc640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f03a407ac40 0x7f03a407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.486+0000 7f03ae7fc640 1 -- 172.21.15.46:0/4127155361 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f03b409e0b0 con 0x7f03b8102b70 2024-09-10T04:22:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.486+0000 7f03bcd56640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f03a407ac40 0x7f03a407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.486+0000 7f03bcd56640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f03a407ac40 0x7f03a407d100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03b8102260 tx=0x7f03a8004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:20.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.490+0000 7f03ae7fc640 1 -- 172.21.15.46:0/4127155361 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f03b4069870 con 0x7f03b8102b70 2024-09-10T04:22:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.713+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f03b8106000 con 0x7f03b8102b70 2024-09-10T04:22:20.715 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:20 smithi046 ceph-mon[109506]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.714+0000 7f03ae7fc640 1 -- 172.21.15.46:0/4127155361 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f03b406e720 con 0x7f03b8102b70 2024-09-10T04:22:20.716 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:22:20.716 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:22:20.716 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:22:20.716 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:22:20.716 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:22:20.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:22:20.717 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:22:20.718 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:20.719 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:22:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f03a407ac40 msgr2=0x7f03a407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f03a407ac40 0x7f03a407d100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03b8102260 tx=0x7f03a8004f30 comp rx=0 tx=0).stop 2024-09-10T04:22:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8102b70 msgr2=0x7f03b81007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8102b70 0x7f03b81007d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f03b400db80 tx=0x7f03b400b630 comp rx=0 tx=0).stop 2024-09-10T04:22:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 shutdown_connections 2024-09-10T04:22:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f03a407ac40 0x7f03a407d100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03b8103490 0x7f03b8100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 --2- 172.21.15.46:0/4127155361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8102b70 0x7f03b81007d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 >> 172.21.15.46:0/4127155361 conn(0x7f03b80fc550 msgr2=0x7f03b8103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 shutdown_connections 2024-09-10T04:22:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.717+0000 7f03bf7bb640 1 -- 172.21.15.46:0/4127155361 wait complete. 2024-09-10T04:22:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 -- 172.21.15.46:0/2865889945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810100800 msgr2=0x7f0810100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 --2- 172.21.15.46:0/2865889945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810100800 0x7f0810100c60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0804009880 tx=0x7f080402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 -- 172.21.15.46:0/2865889945 shutdown_connections 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 --2- 172.21.15.46:0/2865889945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810100800 0x7f0810100c60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 --2- 172.21.15.46:0/2865889945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0810106800 0x7f0810106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 -- 172.21.15.46:0/2865889945 >> 172.21.15.46:0/2865889945 conn(0x7f08100fc520 msgr2=0x7f08100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 -- 172.21.15.46:0/2865889945 shutdown_connections 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.804+0000 7f08149d6640 1 -- 172.21.15.46:0/2865889945 wait complete. 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.805+0000 7f08149d6640 1 Processor -- start 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.805+0000 7f08149d6640 1 -- start start 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.805+0000 7f08149d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0810100800 0x7f081019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.805+0000 7f08149d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810106800 0x7f081019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.805+0000 7f08149d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f081010dcb0 con 0x7f0810106800 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.805+0000 7f08149d6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f081010db30 con 0x7f0810100800 2024-09-10T04:22:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810106800 0x7f081019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810106800 0x7f081019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33286/0 (socket says 172.21.15.46:33286) 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080dd9b640 1 -- 172.21.15.46:0/2052076347 learned_addr learned my addr 172.21.15.46:0/2052076347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080e59c640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0810100800 0x7f081019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080dd9b640 1 -- 172.21.15.46:0/2052076347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0810100800 msgr2=0x7f081019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080dd9b640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0810100800 0x7f081019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080dd9b640 1 -- 172.21.15.46:0/2052076347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08101977a0 con 0x7f0810106800 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f080dd9b640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810106800 0x7f081019db60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0804031460 tx=0x7f08040048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f07ff7fe640 1 -- 172.21.15.46:0/2052076347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f080403f070 con 0x7f0810106800 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0810197a30 con 0x7f0810106800 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f07ff7fe640 1 -- 172.21.15.46:0/2052076347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08040044d0 con 0x7f0810106800 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f07ff7fe640 1 -- 172.21.15.46:0/2052076347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0804035030 con 0x7f0810106800 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.806+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0810197f60 con 0x7f0810106800 2024-09-10T04:22:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.807+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0810101f40 con 0x7f0810106800 2024-09-10T04:22:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.808+0000 7f07ff7fe640 1 -- 172.21.15.46:0/2052076347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0804042070 con 0x7f0810106800 2024-09-10T04:22:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.809+0000 7f07ff7fe640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f807ac00 0x7f07f807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.809+0000 7f07ff7fe640 1 -- 172.21.15.46:0/2052076347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f08040be730 con 0x7f0810106800 2024-09-10T04:22:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.809+0000 7f080e59c640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f807ac00 0x7f07f807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.809+0000 7f080e59c640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f807ac00 0x7f07f807d0c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f080000abe0 tx=0x7f080000a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:20.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:20.813+0000 7f07ff7fe640 1 -- 172.21.15.46:0/2052076347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0804004cf0 con 0x7f0810106800 2024-09-10T04:22:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:20 smithi070 ceph-mon[79774]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:21.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.014+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0810198210 con 0x7f07f807ac00 2024-09-10T04:22:21.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.016+0000 7f07ff7fe640 1 -- 172.21.15.46:0/2052076347 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0810198210 con 0x7f07f807ac00 2024-09-10T04:22:21.017 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:22:21.017 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:22:21.018 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:22:21.019 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:22:21.019 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:22:21.019 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f807ac00 msgr2=0x7f07f807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f807ac00 0x7f07f807d0c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f080000abe0 tx=0x7f080000a2d0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810106800 msgr2=0x7f081019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810106800 0x7f081019db60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0804031460 tx=0x7f08040048f0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 shutdown_connections 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f07f807ac00 0x7f07f807d0c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0810106800 0x7f081019db60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 --2- 172.21.15.46:0/2052076347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0810100800 0x7f081019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 >> 172.21.15.46:0/2052076347 conn(0x7f08100fc520 msgr2=0x7f081006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 shutdown_connections 2024-09-10T04:22:21.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.018+0000 7f08149d6640 1 -- 172.21.15.46:0/2052076347 wait complete. 2024-09-10T04:22:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 -- 172.21.15.46:0/623956508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 msgr2=0x7f6a0c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 --2- 172.21.15.46:0/623956508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c1007a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009880 tx=0x7f69fc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 -- 172.21.15.46:0/623956508 shutdown_connections 2024-09-10T04:22:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 --2- 172.21.15.46:0/623956508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a0c100d70 0x7f6a0c1065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 --2- 172.21.15.46:0/623956508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c1007a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 -- 172.21.15.46:0/623956508 >> 172.21.15.46:0/623956508 conn(0x7f6a0c077150 msgr2=0x7f6a0c077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 -- 172.21.15.46:0/623956508 shutdown_connections 2024-09-10T04:22:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 -- 172.21.15.46:0/623956508 wait complete. 2024-09-10T04:22:21.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.103+0000 7f6a1119d640 1 Processor -- start 2024-09-10T04:22:21.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a1119d640 1 -- start start 2024-09-10T04:22:21.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a1119d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:21.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a1119d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a0c100d70 0x7f6a0c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a1119d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10ede0 con 0x7f6a0c1003c0 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a1119d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10ec60 con 0x7f6a0c100d70 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a0ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a0ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c19d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33308/0 (socket says 172.21.15.46:33308) 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a0ad9d640 1 -- 172.21.15.46:0/2091199340 learned_addr learned my addr 172.21.15.46:0/2091199340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a0a59c640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a0c100d70 0x7f6a0c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a0ad9d640 1 -- 172.21.15.46:0/2091199340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a0c100d70 msgr2=0x7f6a0c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a0ad9d640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a0c100d70 0x7f6a0c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.104+0000 7f6a0ad9d640 1 -- 172.21.15.46:0/2091199340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c197790 con 0x7f6a0c1003c0 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.105+0000 7f6a0ad9d640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c19d610 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69fc031460 tx=0x7f69fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.105+0000 7f69f3fff640 1 -- 172.21.15.46:0/2091199340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f69fc03f070 con 0x7f6a0c1003c0 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.105+0000 7f69f3fff640 1 -- 172.21.15.46:0/2091199340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f69fc0035f0 con 0x7f6a0c1003c0 2024-09-10T04:22:21.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.105+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c197a20 con 0x7f6a0c1003c0 2024-09-10T04:22:21.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.105+0000 7f69f3fff640 1 -- 172.21.15.46:0/2091199340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f69fc035030 con 0x7f6a0c1003c0 2024-09-10T04:22:21.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.105+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c197f50 con 0x7f6a0c1003c0 2024-09-10T04:22:21.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.106+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c069d90 con 0x7f6a0c1003c0 2024-09-10T04:22:21.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.107+0000 7f69f3fff640 1 -- 172.21.15.46:0/2091199340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f69fc042070 con 0x7f6a0c1003c0 2024-09-10T04:22:21.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.107+0000 7f69f3fff640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f69f407abf0 0x7f69f407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:21.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.107+0000 7f6a0a59c640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f69f407abf0 0x7f69f407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:21.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.107+0000 7f69f3fff640 1 -- 172.21.15.46:0/2091199340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f69fc0be600 con 0x7f6a0c1003c0 2024-09-10T04:22:21.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.108+0000 7f6a0a59c640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f69f407abf0 0x7f69f407d0b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0ffc50 tx=0x7f6a0000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:21.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.112+0000 7f69f3fff640 1 -- 172.21.15.46:0/2091199340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f69fc089d40 con 0x7f6a0c1003c0 2024-09-10T04:22:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.369+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6a0c109ea0 con 0x7f6a0c1003c0 2024-09-10T04:22:21.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.369+0000 7f69f3fff640 1 -- 172.21.15.46:0/2091199340 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f69fc08ebf0 con 0x7f6a0c1003c0 2024-09-10T04:22:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:22:21.371 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:22:21.372 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:22:21.372 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:22:21.372 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:22:21.372 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:22:21.372 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f69f407abf0 msgr2=0x7f69f407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f69f407abf0 0x7f69f407d0b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c0ffc50 tx=0x7f6a0000b390 comp rx=0 tx=0).stop 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 msgr2=0x7f6a0c19d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c19d610 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69fc031460 tx=0x7f69fc0045e0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 shutdown_connections 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f69f407abf0 0x7f69f407d0b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a0c100d70 0x7f6a0c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 --2- 172.21.15.46:0/2091199340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a0c1003c0 0x7f6a0c19d610 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 >> 172.21.15.46:0/2091199340 conn(0x7f6a0c077150 msgr2=0x7f6a0c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 shutdown_connections 2024-09-10T04:22:21.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:21.372+0000 7f6a1119d640 1 -- 172.21.15.46:0/2091199340 wait complete. 2024-09-10T04:22:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:21 smithi070 ceph-mon[79774]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:21 smithi070 ceph-mon[79774]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:21 smithi070 ceph-mon[79774]: pgmap v500: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:21 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2793597245' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:22:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:21 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4127155361' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:22:21.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:21 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2091199340' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:22:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:21 smithi046 ceph-mon[109506]: from='client.34790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:21 smithi046 ceph-mon[109506]: from='client.34794 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:21 smithi046 ceph-mon[109506]: pgmap v500: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:21 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2793597245' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:22:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:21 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4127155361' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:22:21.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:21 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2091199340' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:22:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:22 smithi070 ceph-mon[79774]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:22 smithi046 ceph-mon[109506]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:23 smithi046 ceph-mon[109506]: pgmap v501: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:23 smithi070 ceph-mon[79774]: pgmap v501: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:25 smithi070 ceph-mon[79774]: pgmap v502: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:25 smithi046 ceph-mon[109506]: pgmap v502: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:27 smithi070 ceph-mon[79774]: pgmap v503: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:27 smithi046 ceph-mon[109506]: pgmap v503: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:29 smithi046 ceph-mon[109506]: pgmap v504: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:29 smithi070 ceph-mon[79774]: pgmap v504: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:31 smithi070 ceph-mon[79774]: pgmap v505: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:31 smithi046 ceph-mon[109506]: pgmap v505: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:33 smithi070 ceph-mon[79774]: pgmap v506: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:33 smithi046 ceph-mon[109506]: pgmap v506: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:35 smithi070 ceph-mon[79774]: pgmap v507: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:35.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:35 smithi046 ceph-mon[109506]: pgmap v507: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:37 smithi070 ceph-mon[79774]: pgmap v508: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:37 smithi046 ceph-mon[109506]: pgmap v508: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:39 smithi070 ceph-mon[79774]: pgmap v509: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:39 smithi046 ceph-mon[109506]: pgmap v509: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:41 smithi046 ceph-mon[109506]: pgmap v510: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:41 smithi070 ceph-mon[79774]: pgmap v510: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:43 smithi070 ceph-mon[79774]: pgmap v511: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:43 smithi046 ceph-mon[109506]: pgmap v511: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:45.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:45 smithi046 ceph-mon[109506]: pgmap v512: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:45 smithi070 ceph-mon[79774]: pgmap v512: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:47 smithi046 ceph-mon[109506]: pgmap v513: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:47.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:47 smithi070 ceph-mon[79774]: pgmap v513: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:49 smithi070 ceph-mon[79774]: pgmap v514: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:49.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:49 smithi046 ceph-mon[109506]: pgmap v514: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/4072963024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e4102ac0 msgr2=0x7fd2e4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/4072963024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e4102ac0 0x7fd2e4102ea0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009c20 tx=0x7fd2d402aa00 comp rx=0 tx=0).stop 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/4072963024 shutdown_connections 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/4072963024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e41033e0 0x7fd2e410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/4072963024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e4102ac0 0x7fd2e4102ea0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/4072963024 >> 172.21.15.46:0/4072963024 conn(0x7fd2e40fc520 msgr2=0x7fd2e40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/4072963024 shutdown_connections 2024-09-10T04:22:51.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/4072963024 wait complete. 2024-09-10T04:22:51.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 Processor -- start 2024-09-10T04:22:51.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.473+0000 7fd2ec6ca640 1 -- start start 2024-09-10T04:22:51.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2ec6ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e4102ac0 0x7fd2e419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:51.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2ec6ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e41033e0 0x7fd2e419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:51.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2ec6ca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410edd0 con 0x7fd2e41033e0 2024-09-10T04:22:51.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2ec6ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410ec50 con 0x7fd2e4102ac0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2e9c65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e41033e0 0x7fd2e419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2e9c65640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e41033e0 0x7fd2e419fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49626/0 (socket says 172.21.15.46:49626) 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2e9c65640 1 -- 172.21.15.46:0/3803539043 learned_addr learned my addr 172.21.15.46:0/3803539043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2ea466640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e4102ac0 0x7fd2e419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2e9c65640 1 -- 172.21.15.46:0/3803539043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e4102ac0 msgr2=0x7fd2e419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2e9c65640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e4102ac0 0x7fd2e419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2e9c65640 1 -- 172.21.15.46:0/3803539043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e4199a80 con 0x7fd2e41033e0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.474+0000 7fd2ea466640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e4102ac0 0x7fd2e419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.475+0000 7fd2e9c65640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e41033e0 0x7fd2e419fd60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800d6c0 tx=0x7fd2d800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.475+0000 7fd2e37fe640 1 -- 172.21.15.46:0/3803539043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd2d8017070 con 0x7fd2e41033e0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.475+0000 7fd2e37fe640 1 -- 172.21.15.46:0/3803539043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd2d8009ce0 con 0x7fd2e41033e0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.475+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e4199d10 con 0x7fd2e41033e0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.475+0000 7fd2e37fe640 1 -- 172.21.15.46:0/3803539043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd2d800f3c0 con 0x7fd2e41033e0 2024-09-10T04:22:51.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.475+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e419a2a0 con 0x7fd2e41033e0 2024-09-10T04:22:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.476+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e406aa90 con 0x7fd2e41033e0 2024-09-10T04:22:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.477+0000 7fd2e37fe640 1 -- 172.21.15.46:0/3803539043 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd2d8010460 con 0x7fd2e41033e0 2024-09-10T04:22:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.477+0000 7fd2e37fe640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd2cc07ac50 0x7fd2cc07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:51.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.478+0000 7fd2e37fe640 1 -- 172.21.15.46:0/3803539043 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fd2d809da20 con 0x7fd2e41033e0 2024-09-10T04:22:51.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.478+0000 7fd2ea466640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd2cc07ac50 0x7fd2cc07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:51.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.478+0000 7fd2ea466640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd2cc07ac50 0x7fd2cc07d110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005e30 tx=0x7fd2d403a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:51.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.482+0000 7fd2e37fe640 1 -- 172.21.15.46:0/3803539043 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd2d8069140 con 0x7fd2e41033e0 2024-09-10T04:22:51.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.687+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd2e419af10 con 0x7fd2cc07ac50 2024-09-10T04:22:51.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.688+0000 7fd2e37fe640 1 -- 172.21.15.46:0/3803539043 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd2d40481b0 con 0x7fd2cc07ac50 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd2cc07ac50 msgr2=0x7fd2cc07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd2cc07ac50 0x7fd2cc07d110 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4005e30 tx=0x7fd2d403a040 comp rx=0 tx=0).stop 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e41033e0 msgr2=0x7fd2e419fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e41033e0 0x7fd2e419fd60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800d6c0 tx=0x7fd2d800db80 comp rx=0 tx=0).stop 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 shutdown_connections 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd2cc07ac50 0x7fd2cc07d110 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2e41033e0 0x7fd2e419fd60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 --2- 172.21.15.46:0/3803539043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2e4102ac0 0x7fd2e419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 >> 172.21.15.46:0/3803539043 conn(0x7fd2e40fc520 msgr2=0x7fd2e4100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:51.692 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:51 smithi046 ceph-mon[109506]: pgmap v515: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:51.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 shutdown_connections 2024-09-10T04:22:51.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.690+0000 7fd2ec6ca640 1 -- 172.21.15.46:0/3803539043 wait complete. 2024-09-10T04:22:51.700 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:22:51.739 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:51 smithi070 ceph-mon[79774]: pgmap v515: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.775+0000 7f298f474640 1 -- 172.21.15.46:0/3038942340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29881033e0 msgr2=0x7f298810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.775+0000 7f298f474640 1 --2- 172.21.15.46:0/3038942340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29881033e0 0x7f298810b8d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f297c009880 tx=0x7f297c02a530 comp rx=0 tx=0).stop 2024-09-10T04:22:51.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.775+0000 7f298f474640 1 -- 172.21.15.46:0/3038942340 shutdown_connections 2024-09-10T04:22:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.775+0000 7f298f474640 1 --2- 172.21.15.46:0/3038942340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29881033e0 0x7f298810b8d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.775+0000 7f298f474640 1 --2- 172.21.15.46:0/3038942340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2988102ac0 0x7f2988102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.775+0000 7f298f474640 1 -- 172.21.15.46:0/3038942340 >> 172.21.15.46:0/3038942340 conn(0x7f29880fc520 msgr2=0x7f29880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.775+0000 7f298f474640 1 -- 172.21.15.46:0/3038942340 shutdown_connections 2024-09-10T04:22:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298f474640 1 -- 172.21.15.46:0/3038942340 wait complete. 2024-09-10T04:22:51.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298f474640 1 Processor -- start 2024-09-10T04:22:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298f474640 1 -- start start 2024-09-10T04:22:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298f474640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2988102ac0 0x7f298819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298f474640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29881033e0 0x7f298819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298f474640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f298810edd0 con 0x7f2988102ac0 2024-09-10T04:22:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298f474640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f298810ec50 con 0x7f29881033e0 2024-09-10T04:22:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298d210640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2988102ac0 0x7f298819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:51.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298d210640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2988102ac0 0x7f298819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49652/0 (socket says 172.21.15.46:49652) 2024-09-10T04:22:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.776+0000 7f298d210640 1 -- 172.21.15.46:0/3085112736 learned_addr learned my addr 172.21.15.46:0/3085112736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f298ca0f640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29881033e0 0x7f298819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f298d210640 1 -- 172.21.15.46:0/3085112736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29881033e0 msgr2=0x7f298819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f298d210640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29881033e0 0x7f298819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f298d210640 1 -- 172.21.15.46:0/3085112736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29881977e0 con 0x7f2988102ac0 2024-09-10T04:22:51.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f298d210640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2988102ac0 0x7f298819d660 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f298400d8d0 tx=0x7f298400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f297a7fc640 1 -- 172.21.15.46:0/3085112736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2984017030 con 0x7f2988102ac0 2024-09-10T04:22:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f297a7fc640 1 -- 172.21.15.46:0/3085112736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2984004020 con 0x7f2988102ac0 2024-09-10T04:22:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2988197ad0 con 0x7f2988102ac0 2024-09-10T04:22:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f297a7fc640 1 -- 172.21.15.46:0/3085112736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f29840047a0 con 0x7f2988102ac0 2024-09-10T04:22:51.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.777+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2988198030 con 0x7f2988102ac0 2024-09-10T04:22:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.778+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298806aa90 con 0x7f2988102ac0 2024-09-10T04:22:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.779+0000 7f297a7fc640 1 -- 172.21.15.46:0/3085112736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2984009a30 con 0x7f2988102ac0 2024-09-10T04:22:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.779+0000 7f297a7fc640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f297407ac40 0x7f297407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.779+0000 7f297a7fc640 1 -- 172.21.15.46:0/3085112736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f2984018070 con 0x7f2988102ac0 2024-09-10T04:22:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.780+0000 7f298ca0f640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f297407ac40 0x7f297407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:51.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.780+0000 7f298ca0f640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f297407ac40 0x7f297407d100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2988198b30 tx=0x7f297c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:51.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.784+0000 7f297a7fc640 1 -- 172.21.15.46:0/3085112736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f29840690c0 con 0x7f2988102ac0 2024-09-10T04:22:51.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.988+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2988198340 con 0x7f297407ac40 2024-09-10T04:22:51.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.989+0000 7f297a7fc640 1 -- 172.21.15.46:0/3085112736 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f2988198340 con 0x7f297407ac40 2024-09-10T04:22:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f297407ac40 msgr2=0x7f297407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f297407ac40 0x7f297407d100 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2988198b30 tx=0x7f297c031400 comp rx=0 tx=0).stop 2024-09-10T04:22:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2988102ac0 msgr2=0x7f298819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2988102ac0 0x7f298819d660 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f298400d8d0 tx=0x7f298400dd90 comp rx=0 tx=0).stop 2024-09-10T04:22:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 shutdown_connections 2024-09-10T04:22:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f297407ac40 0x7f297407d100 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29881033e0 0x7f298819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 --2- 172.21.15.46:0/3085112736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2988102ac0 0x7f298819d660 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:51.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 >> 172.21.15.46:0/3085112736 conn(0x7f29880fc520 msgr2=0x7f2988108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:51.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 shutdown_connections 2024-09-10T04:22:51.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:51.991+0000 7f298f474640 1 -- 172.21.15.46:0/3085112736 wait complete. 2024-09-10T04:22:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 -- 172.21.15.46:0/1905225989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 msgr2=0x7f2414100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 --2- 172.21.15.46:0/1905225989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f2414100a30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2404009880 tx=0x7f240402a490 comp rx=0 tx=0).stop 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 -- 172.21.15.46:0/1905225989 shutdown_connections 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 --2- 172.21.15.46:0/1905225989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2414101000 0x7f241410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 --2- 172.21.15.46:0/1905225989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f2414100a30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 -- 172.21.15.46:0/1905225989 >> 172.21.15.46:0/1905225989 conn(0x7f24140fc520 msgr2=0x7f24140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 -- 172.21.15.46:0/1905225989 shutdown_connections 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.076+0000 7f24198bc640 1 -- 172.21.15.46:0/1905225989 wait complete. 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f24198bc640 1 Processor -- start 2024-09-10T04:22:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f24198bc640 1 -- start start 2024-09-10T04:22:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f24198bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f241419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f24198bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2414101000 0x7f241419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f24198bc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2414110fb0 con 0x7f2414100650 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f24198bc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2414110e30 con 0x7f2414101000 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f2412ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f241419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f2412ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f241419f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49670/0 (socket says 172.21.15.46:49670) 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f2412ffd640 1 -- 172.21.15.46:0/3734611286 learned_addr learned my addr 172.21.15.46:0/3734611286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.077+0000 7f240bfff640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2414101000 0x7f241419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f2412ffd640 1 -- 172.21.15.46:0/3734611286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2414101000 msgr2=0x7f241419fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f2412ffd640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2414101000 0x7f241419fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f2412ffd640 1 -- 172.21.15.46:0/3734611286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2414199a40 con 0x7f2414100650 2024-09-10T04:22:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f2412ffd640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f241419f880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2404031460 tx=0x7f24040045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f24188ba640 1 -- 172.21.15.46:0/3734611286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f240403f070 con 0x7f2414100650 2024-09-10T04:22:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f24188ba640 1 -- 172.21.15.46:0/3734611286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2404004b10 con 0x7f2414100650 2024-09-10T04:22:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2414199cd0 con 0x7f2414100650 2024-09-10T04:22:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f24188ba640 1 -- 172.21.15.46:0/3734611286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2404035030 con 0x7f2414100650 2024-09-10T04:22:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.078+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241419a260 con 0x7f2414100650 2024-09-10T04:22:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.079+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241406aa90 con 0x7f2414100650 2024-09-10T04:22:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.080+0000 7f24188ba640 1 -- 172.21.15.46:0/3734611286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2404042070 con 0x7f2414100650 2024-09-10T04:22:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.080+0000 7f24188ba640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f240007ac50 0x7f240007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.081+0000 7f24188ba640 1 -- 172.21.15.46:0/3734611286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f24040be6a0 con 0x7f2414100650 2024-09-10T04:22:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.081+0000 7f240bfff640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f240007ac50 0x7f240007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.081+0000 7f240bfff640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f240007ac50 0x7f240007d110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2414102520 tx=0x7f23fc00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:52.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.085+0000 7f24188ba640 1 -- 172.21.15.46:0/3734611286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2404089e60 con 0x7f2414100650 2024-09-10T04:22:52.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.279+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f241419af70 con 0x7f240007ac50 2024-09-10T04:22:52.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.289+0000 7f24188ba640 1 -- 172.21.15.46:0/3734611286 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f241419af70 con 0x7f240007ac50 2024-09-10T04:22:52.290 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (14m) 8m ago 25m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (17m) 8m ago 26m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (16m) 8m ago 25m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (12m) 8m ago 25m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (12m) 8m ago 25m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (14m) 8m ago 25m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (9m) 8m ago 20m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (9m) 8m ago 20m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (8m) 8m ago 20m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (20m) 8m ago 20m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (16m) 8m ago 27m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (15m) 8m ago 24m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:22:52.291 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (13m) 8m ago 27m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (13m) 8m ago 24m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (15m) 8m ago 25m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (15m) 8m ago 25m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (12m) 8m ago 23m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (11m) 8m ago 23m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (11m) 8m ago 22m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (10m) 8m ago 22m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (10m) 8m ago 21m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (9m) 8m ago 21m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:22:52.292 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (14m) 8m ago 25m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f240007ac50 msgr2=0x7f240007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f240007ac50 0x7f240007d110 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2414102520 tx=0x7f23fc00b390 comp rx=0 tx=0).stop 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 msgr2=0x7f241419f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f241419f880 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2404031460 tx=0x7f24040045e0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 shutdown_connections 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f240007ac50 0x7f240007d110 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2414101000 0x7f241419fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 --2- 172.21.15.46:0/3734611286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2414100650 0x7f241419f880 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 >> 172.21.15.46:0/3734611286 conn(0x7f24140fc520 msgr2=0x7f24140fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 shutdown_connections 2024-09-10T04:22:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.292+0000 7f24198bc640 1 -- 172.21.15.46:0/3734611286 wait complete. 2024-09-10T04:22:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 -- 172.21.15.46:0/4203513674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 msgr2=0x7f1af8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 --2- 172.21.15.46:0/4203513674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8102ea0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009880 tx=0x7f1af402a530 comp rx=0 tx=0).stop 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 -- 172.21.15.46:0/4203513674 shutdown_connections 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 --2- 172.21.15.46:0/4203513674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af81033e0 0x7f1af810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 --2- 172.21.15.46:0/4203513674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8102ea0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 -- 172.21.15.46:0/4203513674 >> 172.21.15.46:0/4203513674 conn(0x7f1af80fc520 msgr2=0x7f1af80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 -- 172.21.15.46:0/4203513674 shutdown_connections 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.375+0000 7f1affaba640 1 -- 172.21.15.46:0/4203513674 wait complete. 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1affaba640 1 Processor -- start 2024-09-10T04:22:52.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1affaba640 1 -- start start 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1affaba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1affaba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af81033e0 0x7f1af8100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1affaba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1af810ee50 con 0x7f1af8102ac0 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1affaba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1af810ecd0 con 0x7f1af81033e0 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1afd856640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1afd856640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49692/0 (socket says 172.21.15.46:49692) 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1afd856640 1 -- 172.21.15.46:0/3087962613 learned_addr learned my addr 172.21.15.46:0/3087962613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1afd055640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af81033e0 0x7f1af8100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.376+0000 7f1afd856640 1 -- 172.21.15.46:0/3087962613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af81033e0 msgr2=0x7f1af8100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1afd856640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af81033e0 0x7f1af8100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1afd856640 1 -- 172.21.15.46:0/3087962613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af81011a0 con 0x7f1af8102ac0 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1afd055640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af81033e0 0x7f1af8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:22:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1afd856640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8100720 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1af4031460 tx=0x7f1af40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1aeeffd640 1 -- 172.21.15.46:0/3087962613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1af403f070 con 0x7f1af8102ac0 2024-09-10T04:22:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1aeeffd640 1 -- 172.21.15.46:0/3087962613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1af40044d0 con 0x7f1af8102ac0 2024-09-10T04:22:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1aeeffd640 1 -- 172.21.15.46:0/3087962613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1af4035030 con 0x7f1af8102ac0 2024-09-10T04:22:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af8101430 con 0x7f1af8102ac0 2024-09-10T04:22:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.377+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af81a3e20 con 0x7f1af8102ac0 2024-09-10T04:22:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.378+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af806aa90 con 0x7f1af8102ac0 2024-09-10T04:22:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.379+0000 7f1aeeffd640 1 -- 172.21.15.46:0/3087962613 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1af4042070 con 0x7f1af8102ac0 2024-09-10T04:22:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.379+0000 7f1aeeffd640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ae407ac00 0x7f1ae407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.379+0000 7f1aeeffd640 1 -- 172.21.15.46:0/3087962613 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f1af40be820 con 0x7f1af8102ac0 2024-09-10T04:22:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.379+0000 7f1afd055640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ae407ac00 0x7f1ae407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.380+0000 7f1afd055640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ae407ac00 0x7f1ae407d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006fd0 tx=0x7f1ae8008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:52.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.384+0000 7f1aeeffd640 1 -- 172.21.15.46:0/3087962613 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1af4004cf0 con 0x7f1af8102ac0 2024-09-10T04:22:52.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.646+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1af81a4110 con 0x7f1af8102ac0 2024-09-10T04:22:52.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.646+0000 7f1aeeffd640 1 -- 172.21.15.46:0/3087962613 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f1af4089f60 con 0x7f1af8102ac0 2024-09-10T04:22:52.648 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:22:52.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:22:52.648 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:22:52.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:52.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:22:52.648 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:22:52.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:52.649 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:22:52.649 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:22:52.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:52.649 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:22:52.649 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:22:52.650 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:22:52.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:22:52.650 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:22:52.650 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:22:52.650 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:22:52.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:22:52.651 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:22:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ae407ac00 msgr2=0x7f1ae407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ae407ac00 0x7f1ae407d0c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006fd0 tx=0x7f1ae8008040 comp rx=0 tx=0).stop 2024-09-10T04:22:52.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 msgr2=0x7f1af8100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8100720 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1af4031460 tx=0x7f1af40048f0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 shutdown_connections 2024-09-10T04:22:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ae407ac00 0x7f1ae407d0c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1af81033e0 0x7f1af8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 --2- 172.21.15.46:0/3087962613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1af8102ac0 0x7f1af8100720 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 >> 172.21.15.46:0/3087962613 conn(0x7f1af80fc520 msgr2=0x7f1af810af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 shutdown_connections 2024-09-10T04:22:52.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.649+0000 7f1affaba640 1 -- 172.21.15.46:0/3087962613 wait complete. 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 -- 172.21.15.46:0/3035215916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 msgr2=0x7f169810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 --2- 172.21.15.46:0/3035215916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f169810b980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1680009880 tx=0x7f168002a530 comp rx=0 tx=0).stop 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 -- 172.21.15.46:0/3035215916 shutdown_connections 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 --2- 172.21.15.46:0/3035215916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f169810b980 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 --2- 172.21.15.46:0/3035215916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1698102b70 0x7f1698102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 -- 172.21.15.46:0/3035215916 >> 172.21.15.46:0/3035215916 conn(0x7f16980fc550 msgr2=0x7f16980fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 -- 172.21.15.46:0/3035215916 shutdown_connections 2024-09-10T04:22:52.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.733+0000 7f169c89c640 1 -- 172.21.15.46:0/3035215916 wait complete. 2024-09-10T04:22:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f169c89c640 1 Processor -- start 2024-09-10T04:22:52.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f169c89c640 1 -- start start 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f169c89c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1698102b70 0x7f16981007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f169c89c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f1698100d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f169c89c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f169810edc0 con 0x7f1698103490 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f169c89c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f169810ec40 con 0x7f1698102b70 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f168ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f1698100d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f168ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f1698100d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49706/0 (socket says 172.21.15.46:49706) 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f168ffff640 1 -- 172.21.15.46:0/3838403709 learned_addr learned my addr 172.21.15.46:0/3838403709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f169659c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1698102b70 0x7f16981007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f168ffff640 1 -- 172.21.15.46:0/3838403709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1698102b70 msgr2=0x7f16981007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f168ffff640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1698102b70 0x7f16981007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.734+0000 7f168ffff640 1 -- 172.21.15.46:0/3838403709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1698101250 con 0x7f1698103490 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.735+0000 7f168ffff640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f1698100d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1680004f90 tx=0x7f16800048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:52.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.735+0000 7f168f7fe640 1 -- 172.21.15.46:0/3838403709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f168003f070 con 0x7f1698103490 2024-09-10T04:22:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.735+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16981014e0 con 0x7f1698103490 2024-09-10T04:22:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.735+0000 7f168f7fe640 1 -- 172.21.15.46:0/3838403709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1680004cf0 con 0x7f1698103490 2024-09-10T04:22:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.735+0000 7f168f7fe640 1 -- 172.21.15.46:0/3838403709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1680035030 con 0x7f1698103490 2024-09-10T04:22:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.735+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16981a3d00 con 0x7f1698103490 2024-09-10T04:22:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.736+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169806a4c0 con 0x7f1698103490 2024-09-10T04:22:52.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.737+0000 7f168f7fe640 1 -- 172.21.15.46:0/3838403709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1680042070 con 0x7f1698103490 2024-09-10T04:22:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.737+0000 7f168f7fe640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f168407ac00 0x7f168407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.737+0000 7f168f7fe640 1 -- 172.21.15.46:0/3838403709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f16800be870 con 0x7f1698103490 2024-09-10T04:22:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.737+0000 7f169659c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f168407ac00 0x7f168407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:52.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.738+0000 7f169659c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f168407ac00 0x7f168407d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1688005e10 tx=0x7f1688005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.742+0000 7f168f7fe640 1 -- 172.21.15.46:0/3838403709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1680089f10 con 0x7f1698103490 2024-09-10T04:22:52.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.965+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1698106000 con 0x7f1698103490 2024-09-10T04:22:52.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.966+0000 7f168f7fe640 1 -- 172.21.15.46:0/3838403709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f168008edc0 con 0x7f1698103490 2024-09-10T04:22:52.967 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:22:52.967 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:22:52.967 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:22:52.967 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:22:52.967 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:22:52.967 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:22:52.968 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:22:52.969 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:52.970 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f168407ac00 msgr2=0x7f168407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f168407ac00 0x7f168407d0c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1688005e10 tx=0x7f1688005da0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 msgr2=0x7f1698100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f1698100d10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1680004f90 tx=0x7f16800048f0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 shutdown_connections 2024-09-10T04:22:52.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f168407ac00 0x7f168407d0c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1698103490 0x7f1698100d10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 --2- 172.21.15.46:0/3838403709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1698102b70 0x7f16981007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:52.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 >> 172.21.15.46:0/3838403709 conn(0x7f16980fc550 msgr2=0x7f1698103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:52.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 shutdown_connections 2024-09-10T04:22:52.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:52.968+0000 7f169c89c640 1 -- 172.21.15.46:0/3838403709 wait complete. 2024-09-10T04:22:53.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 -- 172.21.15.46:0/2212193996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 msgr2=0x7f38d0100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 --2- 172.21.15.46:0/2212193996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d0100a30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009880 tx=0x7f38c002a490 comp rx=0 tx=0).stop 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 -- 172.21.15.46:0/2212193996 shutdown_connections 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 --2- 172.21.15.46:0/2212193996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38d0101000 0x7f38d010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 --2- 172.21.15.46:0/2212193996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d0100a30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 -- 172.21.15.46:0/2212193996 >> 172.21.15.46:0/2212193996 conn(0x7f38d00fc520 msgr2=0x7f38d00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 -- 172.21.15.46:0/2212193996 shutdown_connections 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.055+0000 7f38d56c7640 1 -- 172.21.15.46:0/2212193996 wait complete. 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38d56c7640 1 Processor -- start 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38d56c7640 1 -- start start 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38d56c7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:53.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38d56c7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38d0101000 0x7f38d019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38d56c7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0110f90 con 0x7f38d0100650 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38d56c7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0110e10 con 0x7f38d0101000 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38ceffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38ceffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49722/0 (socket says 172.21.15.46:49722) 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38ceffd640 1 -- 172.21.15.46:0/1656608007 learned_addr learned my addr 172.21.15.46:0/1656608007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.056+0000 7f38c7fff640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38d0101000 0x7f38d019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38ceffd640 1 -- 172.21.15.46:0/1656608007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38d0101000 msgr2=0x7f38d019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38ceffd640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38d0101000 0x7f38d019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38ceffd640 1 -- 172.21.15.46:0/1656608007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d01a03b0 con 0x7f38d0100650 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38c7fff640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38d0101000 0x7f38d019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38ceffd640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d019f7c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f38c0031460 tx=0x7f38c00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:53.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38ccff9640 1 -- 172.21.15.46:0/1656608007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f38c003f070 con 0x7f38d0100650 2024-09-10T04:22:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38ccff9640 1 -- 172.21.15.46:0/1656608007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38c00044d0 con 0x7f38d0100650 2024-09-10T04:22:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d0199a40 con 0x7f38d0100650 2024-09-10T04:22:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38ccff9640 1 -- 172.21.15.46:0/1656608007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f38c0035030 con 0x7f38d0100650 2024-09-10T04:22:53.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.057+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d0199fa0 con 0x7f38d0100650 2024-09-10T04:22:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.058+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d006aa90 con 0x7f38d0100650 2024-09-10T04:22:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.059+0000 7f38ccff9640 1 -- 172.21.15.46:0/1656608007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f38c0042070 con 0x7f38d0100650 2024-09-10T04:22:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.059+0000 7f38ccff9640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38bc07ac50 0x7f38bc07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.060+0000 7f38ccff9640 1 -- 172.21.15.46:0/1656608007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f38c00be750 con 0x7f38d0100650 2024-09-10T04:22:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.060+0000 7f38c7fff640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38bc07ac50 0x7f38bc07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:53.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.060+0000 7f38c7fff640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38bc07ac50 0x7f38bc07d110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f38d0102530 tx=0x7f38b800a5c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:53.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.064+0000 7f38ccff9640 1 -- 172.21.15.46:0/1656608007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f38c0004cf0 con 0x7f38d0100650 2024-09-10T04:22:53.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.265+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f38d019afc0 con 0x7f38bc07ac50 2024-09-10T04:22:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.267+0000 7f38ccff9640 1 -- 172.21.15.46:0/1656608007 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f38d019afc0 con 0x7f38bc07ac50 2024-09-10T04:22:53.267 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:22:53.267 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:22:53.268 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:22:53.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38bc07ac50 msgr2=0x7f38bc07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38bc07ac50 0x7f38bc07d110 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f38d0102530 tx=0x7f38b800a5c0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 msgr2=0x7f38d019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d019f7c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f38c0031460 tx=0x7f38c00048f0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 shutdown_connections 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38bc07ac50 0x7f38bc07d110 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38d0101000 0x7f38d019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 --2- 172.21.15.46:0/1656608007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d0100650 0x7f38d019f7c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 >> 172.21.15.46:0/1656608007 conn(0x7f38d00fc520 msgr2=0x7f38d00fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 shutdown_connections 2024-09-10T04:22:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.269+0000 7f38d56c7640 1 -- 172.21.15.46:0/1656608007 wait complete. 2024-09-10T04:22:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/1218705324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100ce0 msgr2=0x7fa8f81065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/1218705324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100ce0 0x7fa8f81065a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec009880 tx=0x7fa8ec02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/1218705324 shutdown_connections 2024-09-10T04:22:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/1218705324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100ce0 0x7fa8f81065a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/1218705324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f8100330 0x7fa8f8100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/1218705324 >> 172.21.15.46:0/1218705324 conn(0x7fa8f8077270 msgr2=0x7fa8f8077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/1218705324 shutdown_connections 2024-09-10T04:22:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.354+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/1218705324 wait complete. 2024-09-10T04:22:53.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8fdf0f640 1 Processor -- start 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8fdf0f640 1 -- start start 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8fdf0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100330 0x7fa8f81012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8fdf0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f8100ce0 0x7fa8f81017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8fdf0f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f810edf0 con 0x7fa8f8100330 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8fdf0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f810ec70 con 0x7fa8f8100ce0 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100330 0x7fa8f81012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100330 0x7fa8f81012b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49752/0 (socket says 172.21.15.46:49752) 2024-09-10T04:22:53.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f77fe640 1 -- 172.21.15.46:0/2628081373 learned_addr learned my addr 172.21.15.46:0/2628081373 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f6ffd640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f8100ce0 0x7fa8f81017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f77fe640 1 -- 172.21.15.46:0/2628081373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f8100ce0 msgr2=0x7fa8f81017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f77fe640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f8100ce0 0x7fa8f81017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f77fe640 1 -- 172.21.15.46:0/2628081373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f8101d30 con 0x7fa8f8100330 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.355+0000 7fa8f6ffd640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f8100ce0 0x7fa8f81017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.356+0000 7fa8f77fe640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100330 0x7fa8f81012b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800d6f0 tx=0x7fa8e800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.356+0000 7fa8f4ff9640 1 -- 172.21.15.46:0/2628081373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa8e8017070 con 0x7fa8f8100330 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.356+0000 7fa8f4ff9640 1 -- 172.21.15.46:0/2628081373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8e800de00 con 0x7fa8f8100330 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.356+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f8102020 con 0x7fa8f8100330 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.356+0000 7fa8f4ff9640 1 -- 172.21.15.46:0/2628081373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa8e8004bb0 con 0x7fa8f8100330 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.356+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f81a3e50 con 0x7fa8f8100330 2024-09-10T04:22:53.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.357+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f8069e00 con 0x7fa8f8100330 2024-09-10T04:22:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.358+0000 7fa8f4ff9640 1 -- 172.21.15.46:0/2628081373 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa8e8009a30 con 0x7fa8f8100330 2024-09-10T04:22:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.358+0000 7fa8f4ff9640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa8e007ac40 0x7fa8e007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:22:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.358+0000 7fa8f4ff9640 1 -- 172.21.15.46:0/2628081373 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fa8e809d850 con 0x7fa8f8100330 2024-09-10T04:22:53.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.358+0000 7fa8f6ffd640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa8e007ac40 0x7fa8e007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:22:53.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.359+0000 7fa8f6ffd640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa8e007ac40 0x7fa8e007d100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec000c00 tx=0x7fa8ec031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:22:53.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.363+0000 7fa8f4ff9640 1 -- 172.21.15.46:0/2628081373 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa8e8068fb0 con 0x7fa8f8100330 2024-09-10T04:22:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:53 smithi046 ceph-mon[109506]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:53 smithi046 ceph-mon[109506]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:53 smithi046 ceph-mon[109506]: pgmap v516: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:53 smithi046 ceph-mon[109506]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:53 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3087962613' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:22:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:53 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3838403709' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:22:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.626+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa8f8109e80 con 0x7fa8f8100330 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.626+0000 7fa8f4ff9640 1 -- 172.21.15.46:0/2628081373 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fa8e806de60 con 0x7fa8f8100330 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:22:53.627 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.628+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa8e007ac40 msgr2=0x7fa8e007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa8e007ac40 0x7fa8e007d100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec000c00 tx=0x7fa8ec031400 comp rx=0 tx=0).stop 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100330 msgr2=0x7fa8f81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100330 0x7fa8f81012b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800d6f0 tx=0x7fa8e800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 shutdown_connections 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa8e007ac40 0x7fa8e007d100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8f8100ce0 0x7fa8f81017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 --2- 172.21.15.46:0/2628081373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8f8100330 0x7fa8f81012b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:22:53.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 >> 172.21.15.46:0/2628081373 conn(0x7fa8f8077270 msgr2=0x7fa8f80fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:22:53.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 shutdown_connections 2024-09-10T04:22:53.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:22:53.629+0000 7fa8fdf0f640 1 -- 172.21.15.46:0/2628081373 wait complete. 2024-09-10T04:22:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:53 smithi070 ceph-mon[79774]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:53 smithi070 ceph-mon[79774]: from='client.34818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:53 smithi070 ceph-mon[79774]: pgmap v516: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:53 smithi070 ceph-mon[79774]: from='client.34822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:53 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3087962613' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:22:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:53 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3838403709' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:22:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:22:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:54 smithi070 ceph-mon[79774]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:54 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2628081373' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:22:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:54 smithi046 ceph-mon[109506]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:22:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:54 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2628081373' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:22:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:55 smithi070 ceph-mon[79774]: pgmap v517: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:55.842 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:55 smithi046 ceph-mon[109506]: pgmap v517: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:57 smithi070 ceph-mon[79774]: pgmap v518: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:57 smithi046 ceph-mon[109506]: pgmap v518: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:22:59 smithi046 ceph-mon[109506]: pgmap v519: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:22:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:22:59 smithi070 ceph-mon[79774]: pgmap v519: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:01 smithi070 ceph-mon[79774]: pgmap v520: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:01 smithi046 ceph-mon[109506]: pgmap v520: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:03 smithi070 ceph-mon[79774]: pgmap v521: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:03 smithi046 ceph-mon[109506]: pgmap v521: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:05.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:05 smithi046 ceph-mon[109506]: pgmap v522: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:05 smithi070 ceph-mon[79774]: pgmap v522: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:07 smithi070 ceph-mon[79774]: pgmap v523: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:07 smithi046 ceph-mon[109506]: pgmap v523: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:09 smithi070 ceph-mon[79774]: pgmap v524: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:09 smithi046 ceph-mon[109506]: pgmap v524: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:11 smithi046 ceph-mon[109506]: pgmap v525: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:11 smithi070 ceph-mon[79774]: pgmap v525: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:13 smithi070 ceph-mon[79774]: pgmap v526: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:13 smithi046 ceph-mon[109506]: pgmap v526: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:15 smithi070 ceph-mon[79774]: pgmap v527: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:15 smithi046 ceph-mon[109506]: pgmap v527: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:17 smithi046 ceph-mon[109506]: pgmap v528: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:17 smithi070 ceph-mon[79774]: pgmap v528: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:19 smithi070 ceph-mon[79774]: pgmap v529: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:19 smithi046 ceph-mon[109506]: pgmap v529: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:21 smithi070 ceph-mon[79774]: pgmap v530: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:21 smithi046 ceph-mon[109506]: pgmap v530: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:23 smithi046 ceph-mon[109506]: pgmap v531: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.725+0000 7f5f20218640 1 -- 172.21.15.46:0/237404304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 msgr2=0x7f5f18107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.725+0000 7f5f20218640 1 --2- 172.21.15.46:0/237404304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f18107ce0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009880 tx=0x7f5f1402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 -- 172.21.15.46:0/237404304 shutdown_connections 2024-09-10T04:23:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 --2- 172.21.15.46:0/237404304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f18101900 0x7f5f18101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 --2- 172.21.15.46:0/237404304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f18107ce0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:23.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 -- 172.21.15.46:0/237404304 >> 172.21.15.46:0/237404304 conn(0x7f5f180fd600 msgr2=0x7f5f180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 -- 172.21.15.46:0/237404304 shutdown_connections 2024-09-10T04:23:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 -- 172.21.15.46:0/237404304 wait complete. 2024-09-10T04:23:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 Processor -- start 2024-09-10T04:23:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.726+0000 7f5f20218640 1 -- start start 2024-09-10T04:23:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f20218640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f18101900 0x7f5f1819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f20218640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f1819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f20218640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1810edd0 con 0x7f5f18107900 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f20218640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1810ec50 con 0x7f5f18101900 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1d7b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f1819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1d7b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f1819fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38562/0 (socket says 172.21.15.46:38562) 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1d7b3640 1 -- 172.21.15.46:0/2369097466 learned_addr learned my addr 172.21.15.46:0/2369097466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1dfb4640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f18101900 0x7f5f1819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1d7b3640 1 -- 172.21.15.46:0/2369097466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f18101900 msgr2=0x7f5f1819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1d7b3640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f18101900 0x7f5f1819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1d7b3640 1 -- 172.21.15.46:0/2369097466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f18199a10 con 0x7f5f18107900 2024-09-10T04:23:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.727+0000 7f5f1d7b3640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f1819fd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e970 tx=0x7f5f0800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.728+0000 7f5f0effd640 1 -- 172.21.15.46:0/2369097466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5f0800cac0 con 0x7f5f18107900 2024-09-10T04:23:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.728+0000 7f5f0effd640 1 -- 172.21.15.46:0/2369097466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5f080054e0 con 0x7f5f18107900 2024-09-10T04:23:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.728+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f18199d00 con 0x7f5f18107900 2024-09-10T04:23:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.728+0000 7f5f0effd640 1 -- 172.21.15.46:0/2369097466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5f08010430 con 0x7f5f18107900 2024-09-10T04:23:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.728+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1819a290 con 0x7f5f18107900 2024-09-10T04:23:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.729+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f18103040 con 0x7f5f18107900 2024-09-10T04:23:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.729+0000 7f5f0effd640 1 -- 172.21.15.46:0/2369097466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5f08004f00 con 0x7f5f18107900 2024-09-10T04:23:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.730+0000 7f5f0effd640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f0407ac40 0x7f5f0407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.730+0000 7f5f0effd640 1 -- 172.21.15.46:0/2369097466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f5f0809e280 con 0x7f5f18107900 2024-09-10T04:23:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.730+0000 7f5f1dfb4640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f0407ac40 0x7f5f0407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.731+0000 7f5f1dfb4640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f0407ac40 0x7f5f0407d100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f14000c00 tx=0x7f5f14031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:23.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.735+0000 7f5f0effd640 1 -- 172.21.15.46:0/2369097466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5f080699e0 con 0x7f5f18107900 2024-09-10T04:23:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:23 smithi070 ceph-mon[79774]: pgmap v531: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.940+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5f1819b010 con 0x7f5f0407ac40 2024-09-10T04:23:23.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.941+0000 7f5f0effd640 1 -- 172.21.15.46:0/2369097466 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f5f1819b010 con 0x7f5f0407ac40 2024-09-10T04:23:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f0407ac40 msgr2=0x7f5f0407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f0407ac40 0x7f5f0407d100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f14000c00 tx=0x7f5f14031400 comp rx=0 tx=0).stop 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 msgr2=0x7f5f1819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f1819fd90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800e970 tx=0x7f5f0800ed70 comp rx=0 tx=0).stop 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 shutdown_connections 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5f0407ac40 0x7f5f0407d100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5f18107900 0x7f5f1819fd90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 --2- 172.21.15.46:0/2369097466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f18101900 0x7f5f1819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 >> 172.21.15.46:0/2369097466 conn(0x7f5f180fd600 msgr2=0x7f5f18074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 shutdown_connections 2024-09-10T04:23:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:23.944+0000 7f5f20218640 1 -- 172.21.15.46:0/2369097466 wait complete. 2024-09-10T04:23:23.954 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:23:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 -- 172.21.15.46:0/292867322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c100800 msgr2=0x7fe76c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 --2- 172.21.15.46:0/292867322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c100800 0x7fe76c100c60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe760009880 tx=0x7fe76002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 -- 172.21.15.46:0/292867322 shutdown_connections 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 --2- 172.21.15.46:0/292867322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c100800 0x7fe76c100c60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 --2- 172.21.15.46:0/292867322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe76c106800 0x7fe76c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 -- 172.21.15.46:0/292867322 >> 172.21.15.46:0/292867322 conn(0x7fe76c0fc520 msgr2=0x7fe76c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 -- 172.21.15.46:0/292867322 shutdown_connections 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.029+0000 7fe771ce9640 1 -- 172.21.15.46:0/292867322 wait complete. 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe771ce9640 1 Processor -- start 2024-09-10T04:23:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe771ce9640 1 -- start start 2024-09-10T04:23:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe771ce9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe76c100800 0x7fe76c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe771ce9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c106800 0x7fe76c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe771ce9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c10dcb0 con 0x7fe76c106800 2024-09-10T04:23:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe771ce9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c10db30 con 0x7fe76c100800 2024-09-10T04:23:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe76affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c106800 0x7fe76c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe76affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c106800 0x7fe76c0fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38586/0 (socket says 172.21.15.46:38586) 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.030+0000 7fe76affd640 1 -- 172.21.15.46:0/3578941724 learned_addr learned my addr 172.21.15.46:0/3578941724 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe76b7fe640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe76c100800 0x7fe76c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe76affd640 1 -- 172.21.15.46:0/3578941724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe76c100800 msgr2=0x7fe76c0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe76affd640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe76c100800 0x7fe76c0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe76affd640 1 -- 172.21.15.46:0/3578941724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c0fd650 con 0x7fe76c106800 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe76affd640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c106800 0x7fe76c0fd110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe760031460 tx=0x7fe7600045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe768ff9640 1 -- 172.21.15.46:0/3578941724 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe76003f070 con 0x7fe76c106800 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe768ff9640 1 -- 172.21.15.46:0/3578941724 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7600035f0 con 0x7fe76c106800 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c0fd8e0 con 0x7fe76c106800 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe768ff9640 1 -- 172.21.15.46:0/3578941724 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe760035030 con 0x7fe76c106800 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.031+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c1a3d60 con 0x7fe76c106800 2024-09-10T04:23:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.032+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76c101f40 con 0x7fe76c106800 2024-09-10T04:23:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.033+0000 7fe768ff9640 1 -- 172.21.15.46:0/3578941724 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe760042070 con 0x7fe76c106800 2024-09-10T04:23:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.033+0000 7fe768ff9640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe75407ac40 0x7fe75407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.034+0000 7fe768ff9640 1 -- 172.21.15.46:0/3578941724 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fe7600be4f0 con 0x7fe76c106800 2024-09-10T04:23:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.034+0000 7fe76b7fe640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe75407ac40 0x7fe75407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.034+0000 7fe76b7fe640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe75407ac40 0x7fe75407d100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005d00 tx=0x7fe75c00b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.038+0000 7fe768ff9640 1 -- 172.21.15.46:0/3578941724 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe760089cb0 con 0x7fe76c106800 2024-09-10T04:23:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.242+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe76c062630 con 0x7fe75407ac40 2024-09-10T04:23:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.243+0000 7fe768ff9640 1 -- 172.21.15.46:0/3578941724 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fe76c062630 con 0x7fe75407ac40 2024-09-10T04:23:24.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.245+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe75407ac40 msgr2=0x7fe75407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe75407ac40 0x7fe75407d100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005d00 tx=0x7fe75c00b410 comp rx=0 tx=0).stop 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c106800 msgr2=0x7fe76c0fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c106800 0x7fe76c0fd110 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe760031460 tx=0x7fe7600045e0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 shutdown_connections 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe75407ac40 0x7fe75407d100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe76c106800 0x7fe76c0fd110 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 --2- 172.21.15.46:0/3578941724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe76c100800 0x7fe76c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 >> 172.21.15.46:0/3578941724 conn(0x7fe76c0fc520 msgr2=0x7fe76c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 shutdown_connections 2024-09-10T04:23:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.246+0000 7fe771ce9640 1 -- 172.21.15.46:0/3578941724 wait complete. 2024-09-10T04:23:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 -- 172.21.15.46:0/593776900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 msgr2=0x7f3c2c106c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 --2- 172.21.15.46:0/593776900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c106c40 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009880 tx=0x7f3c1c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 -- 172.21.15.46:0/593776900 shutdown_connections 2024-09-10T04:23:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 --2- 172.21.15.46:0/593776900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c100860 0x7f3c2c100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 --2- 172.21.15.46:0/593776900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c106c40 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 -- 172.21.15.46:0/593776900 >> 172.21.15.46:0/593776900 conn(0x7f3c2c0fc580 msgr2=0x7f3c2c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 -- 172.21.15.46:0/593776900 shutdown_connections 2024-09-10T04:23:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 -- 172.21.15.46:0/593776900 wait complete. 2024-09-10T04:23:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.330+0000 7f3c3315b640 1 Processor -- start 2024-09-10T04:23:24.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c3315b640 1 -- start start 2024-09-10T04:23:24.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c3315b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c100860 0x7f3c2c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c3315b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c3315b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c10dd10 con 0x7f3c2c106860 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c3315b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2c10db90 con 0x7f3c2c100860 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c2bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c2bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38614/0 (socket says 172.21.15.46:38614) 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c2bfff640 1 -- 172.21.15.46:0/602186205 learned_addr learned my addr 172.21.15.46:0/602186205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c30ef7640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c100860 0x7f3c2c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c2bfff640 1 -- 172.21.15.46:0/602186205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c100860 msgr2=0x7f3c2c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c2bfff640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c100860 0x7f3c2c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c2bfff640 1 -- 172.21.15.46:0/602186205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2c1993a0 con 0x7f3c2c106860 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.331+0000 7f3c2bfff640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c198da0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000d8d0 tx=0x7f3c2000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:24.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.332+0000 7f3c29ffb640 1 -- 172.21.15.46:0/602186205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3c20017070 con 0x7f3c2c106860 2024-09-10T04:23:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.332+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2c199690 con 0x7f3c2c106860 2024-09-10T04:23:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.332+0000 7f3c29ffb640 1 -- 172.21.15.46:0/602186205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c20010460 con 0x7f3c2c106860 2024-09-10T04:23:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.332+0000 7f3c29ffb640 1 -- 172.21.15.46:0/602186205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3c2000f5f0 con 0x7f3c2c106860 2024-09-10T04:23:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.332+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2c1a1c90 con 0x7f3c2c106860 2024-09-10T04:23:24.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.333+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2c101fa0 con 0x7f3c2c106860 2024-09-10T04:23:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.333+0000 7f3c29ffb640 1 -- 172.21.15.46:0/602186205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3c20009a30 con 0x7f3c2c106860 2024-09-10T04:23:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.334+0000 7f3c29ffb640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c1407ac50 0x7f3c1407d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.334+0000 7f3c29ffb640 1 -- 172.21.15.46:0/602186205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f3c2009d9c0 con 0x7f3c2c106860 2024-09-10T04:23:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.334+0000 7f3c30ef7640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c1407ac50 0x7f3c1407d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.335+0000 7f3c30ef7640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c1407ac50 0x7f3c1407d110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0096c0 tx=0x7f3c1c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:24.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.338+0000 7f3c29ffb640 1 -- 172.21.15.46:0/602186205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3c2c199690 con 0x7f3c2c106860 2024-09-10T04:23:24.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.531+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3c2c062630 con 0x7f3c1407ac50 2024-09-10T04:23:24.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.541+0000 7f3c29ffb640 1 -- 172.21.15.46:0/602186205 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f3c2c062630 con 0x7f3c1407ac50 2024-09-10T04:23:24.543 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:23:24.543 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (15m) 9m ago 26m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:23:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (17m) 9m ago 26m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:23:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (17m) 9m ago 25m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:23:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (13m) 9m ago 26m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:23:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (13m) 9m ago 25m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:23:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (14m) 9m ago 26m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:23:24.544 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (9m) 9m ago 20m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (9m) 9m ago 20m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (9m) 9m ago 20m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (20m) 9m ago 20m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (16m) 9m ago 27m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (16m) 9m ago 25m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (13m) 9m ago 27m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (13m) 9m ago 25m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (15m) 9m ago 26m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (15m) 9m ago 25m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (12m) 9m ago 24m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (12m) 9m ago 23m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (11m) 9m ago 23m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:23:24.545 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (11m) 9m ago 22m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:23:24.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (10m) 9m ago 22m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:23:24.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (10m) 9m ago 21m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:23:24.546 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (15m) 9m ago 25m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:23:24.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c1407ac50 msgr2=0x7f3c1407d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c1407ac50 0x7f3c1407d110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0096c0 tx=0x7f3c1c031400 comp rx=0 tx=0).stop 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 msgr2=0x7f3c2c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c198da0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000d8d0 tx=0x7f3c2000dd90 comp rx=0 tx=0).stop 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 shutdown_connections 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c1407ac50 0x7f3c1407d110 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c2c106860 0x7f3c2c198da0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 --2- 172.21.15.46:0/602186205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c100860 0x7f3c2c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 >> 172.21.15.46:0/602186205 conn(0x7f3c2c0fc580 msgr2=0x7f3c2c0fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 shutdown_connections 2024-09-10T04:23:24.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.543+0000 7f3c3315b640 1 -- 172.21.15.46:0/602186205 wait complete. 2024-09-10T04:23:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 -- 172.21.15.46:0/1235851241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 msgr2=0x7fbba0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 --2- 172.21.15.46:0/1235851241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba0100c60 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009880 tx=0x7fbb9402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 -- 172.21.15.46:0/1235851241 shutdown_connections 2024-09-10T04:23:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 --2- 172.21.15.46:0/1235851241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba0100c60 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 --2- 172.21.15.46:0/1235851241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbba0106800 0x7fbba0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 -- 172.21.15.46:0/1235851241 >> 172.21.15.46:0/1235851241 conn(0x7fbba00fc520 msgr2=0x7fbba00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 -- 172.21.15.46:0/1235851241 shutdown_connections 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.627+0000 7fbba537e640 1 -- 172.21.15.46:0/1235851241 wait complete. 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbba537e640 1 Processor -- start 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbba537e640 1 -- start start 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbba537e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbba537e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbba0106800 0x7fbba019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbba537e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbba010dcb0 con 0x7fbba0100800 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbba537e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbba010db30 con 0x7fbba0106800 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbb9effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbb9effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38636/0 (socket says 172.21.15.46:38636) 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbb9effd640 1 -- 172.21.15.46:0/3928344838 learned_addr learned my addr 172.21.15.46:0/3928344838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.628+0000 7fbb9e7fc640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbba0106800 0x7fbba019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbb9effd640 1 -- 172.21.15.46:0/3928344838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbba0106800 msgr2=0x7fbba019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbb9effd640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbba0106800 0x7fbba019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbb9effd640 1 -- 172.21.15.46:0/3928344838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba01977f0 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbb9effd640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba019d670 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cc60 tx=0x7fbb9000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbb87fff640 1 -- 172.21.15.46:0/3928344838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb9000ce10 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbb87fff640 1 -- 172.21.15.46:0/3928344838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb9000f910 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba0197ae0 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbb87fff640 1 -- 172.21.15.46:0/3928344838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb90018ae0 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.629+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba0198040 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.630+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbba0101f40 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.631+0000 7fbb87fff640 1 -- 172.21.15.46:0/3928344838 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbb90018c80 con 0x7fbba0100800 2024-09-10T04:23:24.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.631+0000 7fbb87fff640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb8807ac50 0x7fbb8807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.631+0000 7fbb9e7fc640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb8807ac50 0x7fbb8807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.631+0000 7fbb87fff640 1 -- 172.21.15.46:0/3928344838 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fbb9009e210 con 0x7fbba0100800 2024-09-10T04:23:24.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.632+0000 7fbb9e7fc640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb8807ac50 0x7fbb8807d110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbba0198b40 tx=0x7fbb94031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:24.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.636+0000 7fbb87fff640 1 -- 172.21.15.46:0/3928344838 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbb90014030 con 0x7fbba0100800 2024-09-10T04:23:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.902+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fbba0198d50 con 0x7fbba0100800 2024-09-10T04:23:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.903+0000 7fbb87fff640 1 -- 172.21.15.46:0/3928344838 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fbba0198d50 con 0x7fbba0100800 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:23:24.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:23:24.906 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb8807ac50 msgr2=0x7fbb8807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb8807ac50 0x7fbb8807d110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbba0198b40 tx=0x7fbb94031400 comp rx=0 tx=0).stop 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 msgr2=0x7fbba019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba019d670 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000cc60 tx=0x7fbb9000a620 comp rx=0 tx=0).stop 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 shutdown_connections 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb8807ac50 0x7fbb8807d110 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbba0106800 0x7fbba019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 --2- 172.21.15.46:0/3928344838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbba0100800 0x7fbba019d670 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 >> 172.21.15.46:0/3928344838 conn(0x7fbba00fc520 msgr2=0x7fbba006a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 shutdown_connections 2024-09-10T04:23:24.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.905+0000 7fbba537e640 1 -- 172.21.15.46:0/3928344838 wait complete. 2024-09-10T04:23:24.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.989+0000 7fde051af640 1 -- 172.21.15.46:0/423881223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001067f0 msgr2=0x7fde00106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.989+0000 7fde051af640 1 --2- 172.21.15.46:0/423881223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001067f0 0x7fde00106bd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009c20 tx=0x7fddf002a970 comp rx=0 tx=0).stop 2024-09-10T04:23:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.990+0000 7fde051af640 1 -- 172.21.15.46:0/423881223 shutdown_connections 2024-09-10T04:23:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.990+0000 7fde051af640 1 --2- 172.21.15.46:0/423881223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde001007f0 0x7fde00100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.990+0000 7fde051af640 1 --2- 172.21.15.46:0/423881223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001067f0 0x7fde00106bd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.990+0000 7fde051af640 1 -- 172.21.15.46:0/423881223 >> 172.21.15.46:0/423881223 conn(0x7fde000fc550 msgr2=0x7fde000fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.990+0000 7fde051af640 1 -- 172.21.15.46:0/423881223 shutdown_connections 2024-09-10T04:23:24.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.990+0000 7fde051af640 1 -- 172.21.15.46:0/423881223 wait complete. 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.990+0000 7fde051af640 1 Processor -- start 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fde051af640 1 -- start start 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fde051af640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001007f0 0x7fde000fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fde051af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde001067f0 0x7fde000fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fde051af640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde0010dcc0 con 0x7fde001007f0 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fde051af640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fde0010db40 con 0x7fde001067f0 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fddfed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001007f0 0x7fde000fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fddfed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001007f0 0x7fde000fcd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38656/0 (socket says 172.21.15.46:38656) 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fddfed9d640 1 -- 172.21.15.46:0/221335071 learned_addr learned my addr 172.21.15.46:0/221335071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:24.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fddfe59c640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde001067f0 0x7fde000fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fddfed9d640 1 -- 172.21.15.46:0/221335071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde001067f0 msgr2=0x7fde000fd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fddfed9d640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde001067f0 0x7fde000fd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.991+0000 7fddfed9d640 1 -- 172.21.15.46:0/221335071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde000fd7b0 con 0x7fde001007f0 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.992+0000 7fddfed9d640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001007f0 0x7fde000fcd30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fddf002ae90 tx=0x7fddf002f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.992+0000 7fddf7fff640 1 -- 172.21.15.46:0/221335071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fddf0047070 con 0x7fde001007f0 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.992+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde000fda40 con 0x7fde001007f0 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.992+0000 7fddf7fff640 1 -- 172.21.15.46:0/221335071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fddf002f7e0 con 0x7fde001007f0 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.992+0000 7fddf7fff640 1 -- 172.21.15.46:0/221335071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fddf0034070 con 0x7fde001007f0 2024-09-10T04:23:24.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.992+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde001a3d70 con 0x7fde001007f0 2024-09-10T04:23:24.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.993+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde00101f30 con 0x7fde001007f0 2024-09-10T04:23:24.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.994+0000 7fddf7fff640 1 -- 172.21.15.46:0/221335071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fddf004a070 con 0x7fde001007f0 2024-09-10T04:23:24.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.994+0000 7fddf7fff640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddec07abf0 0x7fddec07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:24.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.995+0000 7fddf7fff640 1 -- 172.21.15.46:0/221335071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fddf00be9c0 con 0x7fde001007f0 2024-09-10T04:23:24.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.995+0000 7fddfe59c640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddec07abf0 0x7fddec07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:24.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.995+0000 7fddfe59c640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddec07abf0 0x7fddec07d0b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde000fe7c0 tx=0x7fdde800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:25.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:24.999+0000 7fddf7fff640 1 -- 172.21.15.46:0/221335071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fddf003f070 con 0x7fde001007f0 2024-09-10T04:23:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.225+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fde0002d7c0 con 0x7fde001007f0 2024-09-10T04:23:25.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.225+0000 7fddf7fff640 1 -- 172.21.15.46:0/221335071 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7fddf008a060 con 0x7fde001007f0 2024-09-10T04:23:25.227 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:23:25.228 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:23:25.229 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:23:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddec07abf0 msgr2=0x7fddec07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddec07abf0 0x7fddec07d0b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde000fe7c0 tx=0x7fdde800b390 comp rx=0 tx=0).stop 2024-09-10T04:23:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001007f0 msgr2=0x7fde000fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001007f0 0x7fde000fcd30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fddf002ae90 tx=0x7fddf002f5f0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 shutdown_connections 2024-09-10T04:23:25.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddec07abf0 0x7fddec07d0b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde001067f0 0x7fde000fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 --2- 172.21.15.46:0/221335071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde001007f0 0x7fde000fcd30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 >> 172.21.15.46:0/221335071 conn(0x7fde000fc550 msgr2=0x7fde00069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:25.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 shutdown_connections 2024-09-10T04:23:25.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.228+0000 7fde051af640 1 -- 172.21.15.46:0/221335071 wait complete. 2024-09-10T04:23:25.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 -- 172.21.15.46:0/1274267439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 msgr2=0x7f3ce410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 --2- 172.21.15.46:0/1274267439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce410b8c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009880 tx=0x7f3cd802a530 comp rx=0 tx=0).stop 2024-09-10T04:23:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 -- 172.21.15.46:0/1274267439 shutdown_connections 2024-09-10T04:23:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 --2- 172.21.15.46:0/1274267439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce410b8c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 --2- 172.21.15.46:0/1274267439 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ce4102ab0 0x7f3ce4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 -- 172.21.15.46:0/1274267439 >> 172.21.15.46:0/1274267439 conn(0x7f3ce40fc550 msgr2=0x7f3ce40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 -- 172.21.15.46:0/1274267439 shutdown_connections 2024-09-10T04:23:25.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 -- 172.21.15.46:0/1274267439 wait complete. 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.314+0000 7f3cea219640 1 Processor -- start 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3cea219640 1 -- start start 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3cea219640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ce4102ab0 0x7f3ce419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3cea219640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3cea219640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce410edc0 con 0x7f3ce41033d0 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3cea219640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce410ec40 con 0x7f3ce4102ab0 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3ce2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3ce2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38666/0 (socket says 172.21.15.46:38666) 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3ce2ffd640 1 -- 172.21.15.46:0/541555922 learned_addr learned my addr 172.21.15.46:0/541555922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3ce37fe640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ce4102ab0 0x7f3ce419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:25.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3ce2ffd640 1 -- 172.21.15.46:0/541555922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ce4102ab0 msgr2=0x7f3ce419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3ce2ffd640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ce4102ab0 0x7f3ce419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.315+0000 7f3ce2ffd640 1 -- 172.21.15.46:0/541555922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce4197790 con 0x7f3ce41033d0 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.316+0000 7f3ce2ffd640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce419db50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8031460 tx=0x7f3cd8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.316+0000 7f3ce0ff9640 1 -- 172.21.15.46:0/541555922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3cd803f070 con 0x7f3ce41033d0 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.316+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce4197a20 con 0x7f3ce41033d0 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.316+0000 7f3ce0ff9640 1 -- 172.21.15.46:0/541555922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3cd80035f0 con 0x7f3ce41033d0 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.316+0000 7f3ce0ff9640 1 -- 172.21.15.46:0/541555922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3cd8035030 con 0x7f3ce41033d0 2024-09-10T04:23:25.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.316+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce4197f50 con 0x7f3ce41033d0 2024-09-10T04:23:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.317+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce406a580 con 0x7f3ce41033d0 2024-09-10T04:23:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.317+0000 7f3ce0ff9640 1 -- 172.21.15.46:0/541555922 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3cd8042070 con 0x7f3ce41033d0 2024-09-10T04:23:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.318+0000 7f3ce0ff9640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ccc07ac40 0x7f3ccc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.318+0000 7f3ce0ff9640 1 -- 172.21.15.46:0/541555922 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f3cd80be4a0 con 0x7f3ce41033d0 2024-09-10T04:23:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.318+0000 7f3ce37fe640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ccc07ac40 0x7f3ccc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:25.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.318+0000 7f3ce37fe640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ccc07ac40 0x7f3ccc07d100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd4009760 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:25.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.322+0000 7f3ce0ff9640 1 -- 172.21.15.46:0/541555922 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3cd8089c60 con 0x7f3ce41033d0 2024-09-10T04:23:25.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.528+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3ce4198200 con 0x7f3ccc07ac40 2024-09-10T04:23:25.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.529+0000 7f3ce0ff9640 1 -- 172.21.15.46:0/541555922 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f3ce4198200 con 0x7f3ccc07ac40 2024-09-10T04:23:25.530 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:23:25.531 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:23:25.532 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:23:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ccc07ac40 msgr2=0x7f3ccc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ccc07ac40 0x7f3ccc07d100 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4005fd0 tx=0x7f3cd4009760 comp rx=0 tx=0).stop 2024-09-10T04:23:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 msgr2=0x7f3ce419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce419db50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8031460 tx=0x7f3cd8004530 comp rx=0 tx=0).stop 2024-09-10T04:23:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 shutdown_connections 2024-09-10T04:23:25.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ccc07ac40 0x7f3ccc07d100 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce41033d0 0x7f3ce419db50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 --2- 172.21.15.46:0/541555922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ce4102ab0 0x7f3ce419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 >> 172.21.15.46:0/541555922 conn(0x7f3ce40fc550 msgr2=0x7f3ce410aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 shutdown_connections 2024-09-10T04:23:25.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.531+0000 7f3cea219640 1 -- 172.21.15.46:0/541555922 wait complete. 2024-09-10T04:23:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 -- 172.21.15.46:0/3714301769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 msgr2=0x7fa968106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 --2- 172.21.15.46:0/3714301769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa968106be0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009880 tx=0x7fa95c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 -- 172.21.15.46:0/3714301769 shutdown_connections 2024-09-10T04:23:25.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 --2- 172.21.15.46:0/3714301769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa968100800 0x7fa968100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 --2- 172.21.15.46:0/3714301769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa968106be0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 -- 172.21.15.46:0/3714301769 >> 172.21.15.46:0/3714301769 conn(0x7fa9680fc520 msgr2=0x7fa9680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 -- 172.21.15.46:0/3714301769 shutdown_connections 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.615+0000 7fa96eb53640 1 -- 172.21.15.46:0/3714301769 wait complete. 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa96eb53640 1 Processor -- start 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa96eb53640 1 -- start start 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa96eb53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa968100800 0x7fa96819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa96eb53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa96819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa96eb53640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa96810dcb0 con 0x7fa968106800 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa96eb53640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa96810db30 con 0x7fa968100800 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa967fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa96819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa967fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa96819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38688/0 (socket says 172.21.15.46:38688) 2024-09-10T04:23:25.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.616+0000 7fa967fff640 1 -- 172.21.15.46:0/2834095683 learned_addr learned my addr 172.21.15.46:0/2834095683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa96c8ef640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa968100800 0x7fa96819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa967fff640 1 -- 172.21.15.46:0/2834095683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa968100800 msgr2=0x7fa96819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa967fff640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa968100800 0x7fa96819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa967fff640 1 -- 172.21.15.46:0/2834095683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa968197780 con 0x7fa968106800 2024-09-10T04:23:25.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa967fff640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa96819db40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa95400d8d0 tx=0x7fa95400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa965ffb640 1 -- 172.21.15.46:0/2834095683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa954017030 con 0x7fa968106800 2024-09-10T04:23:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa965ffb640 1 -- 172.21.15.46:0/2834095683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa954004020 con 0x7fa968106800 2024-09-10T04:23:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa968197a70 con 0x7fa968106800 2024-09-10T04:23:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa965ffb640 1 -- 172.21.15.46:0/2834095683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa9540047a0 con 0x7fa968106800 2024-09-10T04:23:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.617+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa968198000 con 0x7fa968106800 2024-09-10T04:23:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.618+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa968101f40 con 0x7fa968106800 2024-09-10T04:23:25.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.619+0000 7fa965ffb640 1 -- 172.21.15.46:0/2834095683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa9540041d0 con 0x7fa968106800 2024-09-10T04:23:25.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.619+0000 7fa965ffb640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa95807ac40 0x7fa95807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:25.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.620+0000 7fa965ffb640 1 -- 172.21.15.46:0/2834095683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fa954018070 con 0x7fa968106800 2024-09-10T04:23:25.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.620+0000 7fa96c8ef640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa95807ac40 0x7fa95807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:25.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.620+0000 7fa96c8ef640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa95807ac40 0x7fa95807d100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0096c0 tx=0x7fa95c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:25.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.624+0000 7fa965ffb640 1 -- 172.21.15.46:0/2834095683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa954069190 con 0x7fa968106800 2024-09-10T04:23:25.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:25 smithi046 ceph-mon[109506]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:25.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:25 smithi046 ceph-mon[109506]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:25.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:25 smithi046 ceph-mon[109506]: pgmap v532: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:25.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:25 smithi046 ceph-mon[109506]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:25.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:25 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3928344838' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:23:25.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:25 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/221335071' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:23:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:25 smithi070 ceph-mon[79774]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:25 smithi070 ceph-mon[79774]: from='client.34846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:25 smithi070 ceph-mon[79774]: pgmap v532: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:25 smithi070 ceph-mon[79774]: from='client.34850 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:25 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3928344838' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:23:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:25 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/221335071' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:23:25.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.888+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa968198310 con 0x7fa968106800 2024-09-10T04:23:25.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.889+0000 7fa965ffb640 1 -- 172.21.15.46:0/2834095683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fa95406e040 con 0x7fa968106800 2024-09-10T04:23:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:23:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:23:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:23:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:23:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:23:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:23:25.891 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:23:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.891+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa95807ac40 msgr2=0x7fa95807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.891+0000 7fa96eb53640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa95807ac40 0x7fa95807d100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0096c0 tx=0x7fa95c031400 comp rx=0 tx=0).stop 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.891+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 msgr2=0x7fa96819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.891+0000 7fa96eb53640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa96819db40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa95400d8d0 tx=0x7fa95400dd90 comp rx=0 tx=0).stop 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.892+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 shutdown_connections 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.892+0000 7fa96eb53640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa95807ac40 0x7fa95807d100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.892+0000 7fa96eb53640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa968106800 0x7fa96819db40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.892+0000 7fa96eb53640 1 --2- 172.21.15.46:0/2834095683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa968100800 0x7fa96819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.892+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 >> 172.21.15.46:0/2834095683 conn(0x7fa9680fc520 msgr2=0x7fa968109e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.892+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 shutdown_connections 2024-09-10T04:23:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:25.892+0000 7fa96eb53640 1 -- 172.21.15.46:0/2834095683 wait complete. 2024-09-10T04:23:26.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:26 smithi046 ceph-mon[109506]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:26.613 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:26 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2834095683' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:23:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:26 smithi070 ceph-mon[79774]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:26 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2834095683' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:23:27.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:27 smithi070 ceph-mon[79774]: pgmap v533: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:27 smithi046 ceph-mon[109506]: pgmap v533: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:29.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:29 smithi046 ceph-mon[109506]: pgmap v534: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:29.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:29 smithi070 ceph-mon[79774]: pgmap v534: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:31 smithi070 ceph-mon[79774]: pgmap v535: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:31 smithi046 ceph-mon[109506]: pgmap v535: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:33 smithi070 ceph-mon[79774]: pgmap v536: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:33 smithi046 ceph-mon[109506]: pgmap v536: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:35 smithi046 ceph-mon[109506]: pgmap v537: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:35 smithi070 ceph-mon[79774]: pgmap v537: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:37 smithi070 ceph-mon[79774]: pgmap v538: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:37 smithi046 ceph-mon[109506]: pgmap v538: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:39 smithi070 ceph-mon[79774]: pgmap v539: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:39 smithi046 ceph-mon[109506]: pgmap v539: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:41 smithi070 ceph-mon[79774]: pgmap v540: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:41.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:41 smithi046 ceph-mon[109506]: pgmap v540: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:43 smithi070 ceph-mon[79774]: pgmap v541: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:43 smithi046 ceph-mon[109506]: pgmap v541: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:45 smithi070 ceph-mon[79774]: pgmap v542: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:45 smithi046 ceph-mon[109506]: pgmap v542: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:47 smithi070 ceph-mon[79774]: pgmap v543: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:47.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:47 smithi046 ceph-mon[109506]: pgmap v543: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:49 smithi070 ceph-mon[79774]: pgmap v544: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:49.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:49 smithi046 ceph-mon[109506]: pgmap v544: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:51 smithi070 ceph-mon[79774]: pgmap v545: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:51 smithi046 ceph-mon[109506]: pgmap v545: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:53 smithi070 ceph-mon[79774]: pgmap v546: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:53 smithi046 ceph-mon[109506]: pgmap v546: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:23:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:55 smithi070 ceph-mon[79774]: pgmap v547: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:55 smithi046 ceph-mon[109506]: pgmap v547: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 -- 172.21.15.46:0/3318515607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2181033e0 msgr2=0x7fe21810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 --2- 172.21.15.46:0/3318515607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2181033e0 0x7fe21810b8d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe208009880 tx=0x7fe20802a530 comp rx=0 tx=0).stop 2024-09-10T04:23:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 -- 172.21.15.46:0/3318515607 shutdown_connections 2024-09-10T04:23:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 --2- 172.21.15.46:0/3318515607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2181033e0 0x7fe21810b8d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 --2- 172.21.15.46:0/3318515607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe218102ac0 0x7fe218102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 -- 172.21.15.46:0/3318515607 >> 172.21.15.46:0/3318515607 conn(0x7fe2180fc520 msgr2=0x7fe2180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 -- 172.21.15.46:0/3318515607 shutdown_connections 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 -- 172.21.15.46:0/3318515607 wait complete. 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.992+0000 7fe2207b4640 1 Processor -- start 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe2207b4640 1 -- start start 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe2207b4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe218102ac0 0x7fe218100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe2207b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2181033e0 0x7fe218100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe2207b4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe21810edd0 con 0x7fe218102ac0 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe2207b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe21810ec50 con 0x7fe2181033e0 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe21e550640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe218102ac0 0x7fe218100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe21e550640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe218102ac0 0x7fe218100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35220/0 (socket says 172.21.15.46:35220) 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe21e550640 1 -- 172.21.15.46:0/364353453 learned_addr learned my addr 172.21.15.46:0/364353453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:55.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe21dd4f640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2181033e0 0x7fe218100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe21e550640 1 -- 172.21.15.46:0/364353453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2181033e0 msgr2=0x7fe218100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe21e550640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2181033e0 0x7fe218100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.993+0000 7fe21e550640 1 -- 172.21.15.46:0/364353453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2181011a0 con 0x7fe218102ac0 2024-09-10T04:23:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.994+0000 7fe21e550640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe218102ac0 0x7fe218100720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe21400d8d0 tx=0x7fe21400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.994+0000 7fe20f7fe640 1 -- 172.21.15.46:0/364353453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe214017030 con 0x7fe218102ac0 2024-09-10T04:23:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.994+0000 7fe20f7fe640 1 -- 172.21.15.46:0/364353453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe214004020 con 0x7fe218102ac0 2024-09-10T04:23:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.994+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe218101490 con 0x7fe218102ac0 2024-09-10T04:23:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.994+0000 7fe20f7fe640 1 -- 172.21.15.46:0/364353453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe2140047a0 con 0x7fe218102ac0 2024-09-10T04:23:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.994+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181a3e60 con 0x7fe218102ac0 2024-09-10T04:23:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.995+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21806aa90 con 0x7fe218102ac0 2024-09-10T04:23:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.996+0000 7fe20f7fe640 1 -- 172.21.15.46:0/364353453 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe2140041d0 con 0x7fe218102ac0 2024-09-10T04:23:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.996+0000 7fe20f7fe640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe20407ac00 0x7fe20407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.996+0000 7fe20f7fe640 1 -- 172.21.15.46:0/364353453 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fe214018070 con 0x7fe218102ac0 2024-09-10T04:23:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.996+0000 7fe21dd4f640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe20407ac00 0x7fe20407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:55.997+0000 7fe21dd4f640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe20407ac00 0x7fe20407d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe2181021b0 tx=0x7fe208031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:56.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.002+0000 7fe20f7fe640 1 -- 172.21.15.46:0/364353453 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe214069110 con 0x7fe218102ac0 2024-09-10T04:23:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.207+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe218062630 con 0x7fe20407ac00 2024-09-10T04:23:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.209+0000 7fe20f7fe640 1 -- 172.21.15.46:0/364353453 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fe218062630 con 0x7fe20407ac00 2024-09-10T04:23:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.211+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe20407ac00 msgr2=0x7fe20407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.211+0000 7fe2207b4640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe20407ac00 0x7fe20407d0c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe2181021b0 tx=0x7fe208031400 comp rx=0 tx=0).stop 2024-09-10T04:23:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.211+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe218102ac0 msgr2=0x7fe218100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.211+0000 7fe2207b4640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe218102ac0 0x7fe218100720 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe21400d8d0 tx=0x7fe21400dd90 comp rx=0 tx=0).stop 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.211+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 shutdown_connections 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.212+0000 7fe2207b4640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe20407ac00 0x7fe20407d0c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.212+0000 7fe2207b4640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2181033e0 0x7fe218100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.212+0000 7fe2207b4640 1 --2- 172.21.15.46:0/364353453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe218102ac0 0x7fe218100720 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.212+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 >> 172.21.15.46:0/364353453 conn(0x7fe2180fc520 msgr2=0x7fe218108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.212+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 shutdown_connections 2024-09-10T04:23:56.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.212+0000 7fe2207b4640 1 -- 172.21.15.46:0/364353453 wait complete. 2024-09-10T04:23:56.222 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:23:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 -- 172.21.15.46:0/3502189381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 msgr2=0x7fee98106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 --2- 172.21.15.46:0/3502189381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee98106bd0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fee88009880 tx=0x7fee8802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 -- 172.21.15.46:0/3502189381 shutdown_connections 2024-09-10T04:23:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 --2- 172.21.15.46:0/3502189381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee981007f0 0x7fee98100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 --2- 172.21.15.46:0/3502189381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee98106bd0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 -- 172.21.15.46:0/3502189381 >> 172.21.15.46:0/3502189381 conn(0x7fee980fc550 msgr2=0x7fee980fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 -- 172.21.15.46:0/3502189381 shutdown_connections 2024-09-10T04:23:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.299+0000 7fee9d88d640 1 -- 172.21.15.46:0/3502189381 wait complete. 2024-09-10T04:23:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee9d88d640 1 Processor -- start 2024-09-10T04:23:56.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee9d88d640 1 -- start start 2024-09-10T04:23:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee9d88d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee981007f0 0x7fee9819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee9d88d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee9819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee9d88d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810dcc0 con 0x7fee981067f0 2024-09-10T04:23:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee9d88d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810db40 con 0x7fee981007f0 2024-09-10T04:23:56.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee9819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee9819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35248/0 (socket says 172.21.15.46:35248) 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee8ffff640 1 -- 172.21.15.46:0/134165672 learned_addr learned my addr 172.21.15.46:0/134165672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee96ffd640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee981007f0 0x7fee9819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee8ffff640 1 -- 172.21.15.46:0/134165672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee981007f0 msgr2=0x7fee9819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee8ffff640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee981007f0 0x7fee9819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.300+0000 7fee8ffff640 1 -- 172.21.15.46:0/134165672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee981999d0 con 0x7fee981067f0 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.301+0000 7fee8ffff640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee9819fd50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fee8000e970 tx=0x7fee8000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.301+0000 7fee9c88b640 1 -- 172.21.15.46:0/134165672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee8000cac0 con 0x7fee981067f0 2024-09-10T04:23:56.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.301+0000 7fee9c88b640 1 -- 172.21.15.46:0/134165672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee800054e0 con 0x7fee981067f0 2024-09-10T04:23:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.301+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee98199cc0 con 0x7fee981067f0 2024-09-10T04:23:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.301+0000 7fee9c88b640 1 -- 172.21.15.46:0/134165672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee80010430 con 0x7fee981067f0 2024-09-10T04:23:56.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.301+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee9819a250 con 0x7fee981067f0 2024-09-10T04:23:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.302+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee98101f30 con 0x7fee981067f0 2024-09-10T04:23:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.303+0000 7fee9c88b640 1 -- 172.21.15.46:0/134165672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fee80004e80 con 0x7fee981067f0 2024-09-10T04:23:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.303+0000 7fee9c88b640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407abf0 0x7fee8407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.303+0000 7fee9c88b640 1 -- 172.21.15.46:0/134165672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fee8009e370 con 0x7fee981067f0 2024-09-10T04:23:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.304+0000 7fee96ffd640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407abf0 0x7fee8407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.304+0000 7fee96ffd640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407abf0 0x7fee8407d0b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee88031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:56.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.308+0000 7fee9c88b640 1 -- 172.21.15.46:0/134165672 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fee80069ab0 con 0x7fee981067f0 2024-09-10T04:23:56.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.555+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fee9819a560 con 0x7fee8407abf0 2024-09-10T04:23:56.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.557+0000 7fee9c88b640 1 -- 172.21.15.46:0/134165672 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fee9819a560 con 0x7fee8407abf0 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407abf0 msgr2=0x7fee8407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407abf0 0x7fee8407d0b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fee88000c00 tx=0x7fee88031400 comp rx=0 tx=0).stop 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 msgr2=0x7fee9819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee9819fd50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fee8000e970 tx=0x7fee8000ed70 comp rx=0 tx=0).stop 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 shutdown_connections 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407abf0 0x7fee8407d0b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee981067f0 0x7fee9819fd50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 --2- 172.21.15.46:0/134165672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee981007f0 0x7fee9819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 >> 172.21.15.46:0/134165672 conn(0x7fee980fc550 msgr2=0x7fee98069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:56.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 shutdown_connections 2024-09-10T04:23:56.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.559+0000 7fee9d88d640 1 -- 172.21.15.46:0/134165672 wait complete. 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 -- 172.21.15.46:0/782696090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1618102ab0 msgr2=0x7f1618102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 --2- 172.21.15.46:0/782696090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1618102ab0 0x7f1618102e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f160c0097e0 tx=0x7f160c02a480 comp rx=0 tx=0).stop 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 -- 172.21.15.46:0/782696090 shutdown_connections 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 --2- 172.21.15.46:0/782696090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 0x7f161810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 --2- 172.21.15.46:0/782696090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1618102ab0 0x7f1618102e90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 -- 172.21.15.46:0/782696090 >> 172.21.15.46:0/782696090 conn(0x7f16180fc550 msgr2=0x7f16180fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 -- 172.21.15.46:0/782696090 shutdown_connections 2024-09-10T04:23:56.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.644+0000 7f161d647640 1 -- 172.21.15.46:0/782696090 wait complete. 2024-09-10T04:23:56.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f161d647640 1 Processor -- start 2024-09-10T04:23:56.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f161d647640 1 -- start start 2024-09-10T04:23:56.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f161d647640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1618102ab0 0x7f161819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f161d647640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 0x7f161819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f161d647640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1618102500 con 0x7f16181033d0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f161d647640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1618102380 con 0x7f1618102ab0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f16167fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 0x7f161819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f16167fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 0x7f161819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35258/0 (socket says 172.21.15.46:35258) 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f16167fc640 1 -- 172.21.15.46:0/1090930865 learned_addr learned my addr 172.21.15.46:0/1090930865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f16167fc640 1 -- 172.21.15.46:0/1090930865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1618102ab0 msgr2=0x7f161819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.645+0000 7f16167fc640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1618102ab0 0x7f161819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.646+0000 7f16167fc640 1 -- 172.21.15.46:0/1090930865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16181999d0 con 0x7f16181033d0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.646+0000 7f16167fc640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 0x7f161819fd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f160800e890 tx=0x7f160800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.646+0000 7f15fffff640 1 -- 172.21.15.46:0/1090930865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f160800cac0 con 0x7f16181033d0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.646+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1618199cc0 con 0x7f16181033d0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.646+0000 7f15fffff640 1 -- 172.21.15.46:0/1090930865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f160800cc60 con 0x7f16181033d0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.646+0000 7f15fffff640 1 -- 172.21.15.46:0/1090930865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1608010430 con 0x7f16181033d0 2024-09-10T04:23:56.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.646+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f161819a250 con 0x7f16181033d0 2024-09-10T04:23:56.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.647+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161806a580 con 0x7f16181033d0 2024-09-10T04:23:56.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.648+0000 7f15fffff640 1 -- 172.21.15.46:0/1090930865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1608004020 con 0x7f16181033d0 2024-09-10T04:23:56.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.648+0000 7f15fffff640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f160007abf0 0x7f160007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.648+0000 7f1616ffd640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f160007abf0 0x7f160007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.648+0000 7f15fffff640 1 -- 172.21.15.46:0/1090930865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f160809e120 con 0x7f16181033d0 2024-09-10T04:23:56.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.649+0000 7f1616ffd640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f160007abf0 0x7f160007d0b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f160c0095f0 tx=0x7f160c030890 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:56.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.653+0000 7f15fffff640 1 -- 172.21.15.46:0/1090930865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1608069860 con 0x7f16181033d0 2024-09-10T04:23:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.846+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f161819af00 con 0x7f160007abf0 2024-09-10T04:23:56.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.856+0000 7f15fffff640 1 -- 172.21.15.46:0/1090930865 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f161819af00 con 0x7f160007abf0 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (15m) 9m ago 26m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (18m) 9m ago 27m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (17m) 9m ago 26m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (13m) 9m ago 27m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (13m) 9m ago 26m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (15m) 9m ago 26m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (10m) 9m ago 21m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (10m) 9m ago 21m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:23:56.857 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (9m) 9m ago 21m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (21m) 9m ago 21m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (17m) 9m ago 28m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (16m) 9m ago 26m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (14m) 9m ago 28m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (14m) 9m ago 25m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (16m) 9m ago 26m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (16m) 9m ago 26m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (13m) 9m ago 24m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (12m) 9m ago 24m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (12m) 9m ago 23m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (11m) 9m ago 23m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (11m) 9m ago 22m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:23:56.858 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (10m) 9m ago 22m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:23:56.859 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (15m) 9m ago 26m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f160007abf0 msgr2=0x7f160007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f160007abf0 0x7f160007d0b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f160c0095f0 tx=0x7f160c030890 comp rx=0 tx=0).stop 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 msgr2=0x7f161819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 0x7f161819fd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f160800e890 tx=0x7f160800ed50 comp rx=0 tx=0).stop 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 shutdown_connections 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f160007abf0 0x7f160007d0b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16181033d0 0x7f161819fd50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 --2- 172.21.15.46:0/1090930865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1618102ab0 0x7f161819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 >> 172.21.15.46:0/1090930865 conn(0x7f16180fc550 msgr2=0x7f1618108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:56.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 shutdown_connections 2024-09-10T04:23:56.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.858+0000 7f161d647640 1 -- 172.21.15.46:0/1090930865 wait complete. 2024-09-10T04:23:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 -- 172.21.15.46:0/4043400999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 msgr2=0x7feb34106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 --2- 172.21.15.46:0/4043400999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb34106bd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feb24009880 tx=0x7feb2402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 -- 172.21.15.46:0/4043400999 shutdown_connections 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 --2- 172.21.15.46:0/4043400999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb341007f0 0x7feb34100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 --2- 172.21.15.46:0/4043400999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb34106bd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 -- 172.21.15.46:0/4043400999 >> 172.21.15.46:0/4043400999 conn(0x7feb340fc550 msgr2=0x7feb340fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 -- 172.21.15.46:0/4043400999 shutdown_connections 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.941+0000 7feb396de640 1 -- 172.21.15.46:0/4043400999 wait complete. 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.942+0000 7feb396de640 1 Processor -- start 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.942+0000 7feb396de640 1 -- start start 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.942+0000 7feb396de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb341007f0 0x7feb3419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.942+0000 7feb396de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb3419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.942+0000 7feb396de640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410dcc0 con 0x7feb341067f0 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.942+0000 7feb396de640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feb3410db40 con 0x7feb341007f0 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb327fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb3419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb327fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb3419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35286/0 (socket says 172.21.15.46:35286) 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb327fc640 1 -- 172.21.15.46:0/1268503433 learned_addr learned my addr 172.21.15.46:0/1268503433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb32ffd640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb341007f0 0x7feb3419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb327fc640 1 -- 172.21.15.46:0/1268503433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb341007f0 msgr2=0x7feb3419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb327fc640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb341007f0 0x7feb3419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb327fc640 1 -- 172.21.15.46:0/1268503433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb341999b0 con 0x7feb341067f0 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb327fc640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb3419fd30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e970 tx=0x7feb2800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb1bfff640 1 -- 172.21.15.46:0/1268503433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb2800cac0 con 0x7feb341067f0 2024-09-10T04:23:56.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb1bfff640 1 -- 172.21.15.46:0/1268503433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb280054e0 con 0x7feb341067f0 2024-09-10T04:23:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb34199ca0 con 0x7feb341067f0 2024-09-10T04:23:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb1bfff640 1 -- 172.21.15.46:0/1268503433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb28010430 con 0x7feb341067f0 2024-09-10T04:23:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.943+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3419a230 con 0x7feb341067f0 2024-09-10T04:23:56.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.944+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb34101f30 con 0x7feb341067f0 2024-09-10T04:23:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.945+0000 7feb1bfff640 1 -- 172.21.15.46:0/1268503433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7feb28004f00 con 0x7feb341067f0 2024-09-10T04:23:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.946+0000 7feb1bfff640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb1c07ac50 0x7feb1c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.946+0000 7feb32ffd640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb1c07ac50 0x7feb1c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.946+0000 7feb1bfff640 1 -- 172.21.15.46:0/1268503433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7feb2809e220 con 0x7feb341067f0 2024-09-10T04:23:56.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.946+0000 7feb32ffd640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb1c07ac50 0x7feb1c07d110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feb24000c00 tx=0x7feb24031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:56.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:56.950+0000 7feb1bfff640 1 -- 172.21.15.46:0/1268503433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7feb280699e0 con 0x7feb341067f0 2024-09-10T04:23:57.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.210+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7feb3419afb0 con 0x7feb341067f0 2024-09-10T04:23:57.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.210+0000 7feb1bfff640 1 -- 172.21.15.46:0/1268503433 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7feb2806e890 con 0x7feb341067f0 2024-09-10T04:23:57.211 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:23:57.211 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:23:57.212 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:23:57.213 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:23:57.213 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:23:57.213 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:23:57.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.213+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb1c07ac50 msgr2=0x7feb1c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.213+0000 7feb396de640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb1c07ac50 0x7feb1c07d110 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feb24000c00 tx=0x7feb24031400 comp rx=0 tx=0).stop 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.213+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 msgr2=0x7feb3419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.213+0000 7feb396de640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb3419fd30 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7feb2800e970 tx=0x7feb2800ed70 comp rx=0 tx=0).stop 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.214+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 shutdown_connections 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.214+0000 7feb396de640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb1c07ac50 0x7feb1c07d110 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.214+0000 7feb396de640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb341067f0 0x7feb3419fd30 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.214+0000 7feb396de640 1 --2- 172.21.15.46:0/1268503433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb341007f0 0x7feb3419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.214+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 >> 172.21.15.46:0/1268503433 conn(0x7feb340fc550 msgr2=0x7feb34069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.214+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 shutdown_connections 2024-09-10T04:23:57.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.214+0000 7feb396de640 1 -- 172.21.15.46:0/1268503433 wait complete. 2024-09-10T04:23:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 -- 172.21.15.46:0/1046662928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828102ac0 msgr2=0x7f0828102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 --2- 172.21.15.46:0/1046662928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828102ac0 0x7f0828102ea0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0818009880 tx=0x7f081802a530 comp rx=0 tx=0).stop 2024-09-10T04:23:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 -- 172.21.15.46:0/1046662928 shutdown_connections 2024-09-10T04:23:57.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 --2- 172.21.15.46:0/1046662928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08281033e0 0x7f082810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 --2- 172.21.15.46:0/1046662928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0828102ac0 0x7f0828102ea0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 -- 172.21.15.46:0/1046662928 >> 172.21.15.46:0/1046662928 conn(0x7f08280fc520 msgr2=0x7f08280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 -- 172.21.15.46:0/1046662928 shutdown_connections 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 -- 172.21.15.46:0/1046662928 wait complete. 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 Processor -- start 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.300+0000 7f082e7b9640 1 -- start start 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f082e7b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828102ac0 0x7f0828100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f082e7b9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08281033e0 0x7f0828100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f082e7b9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f082810edd0 con 0x7f08281033e0 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f082e7b9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f082810ec50 con 0x7f0828102ac0 2024-09-10T04:23:57.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f081ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08281033e0 0x7f0828100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f081ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08281033e0 0x7f0828100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35306/0 (socket says 172.21.15.46:35306) 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f081ffff640 1 -- 172.21.15.46:0/923044781 learned_addr learned my addr 172.21.15.46:0/923044781 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f0827fff640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828102ac0 0x7f0828100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f081ffff640 1 -- 172.21.15.46:0/923044781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828102ac0 msgr2=0x7f0828100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f081ffff640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828102ac0 0x7f0828100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f081ffff640 1 -- 172.21.15.46:0/923044781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08281011a0 con 0x7f08281033e0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f0827fff640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828102ac0 0x7f0828100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f081ffff640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08281033e0 0x7f0828100c60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f081000e970 tx=0x7f081000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f0825ffb640 1 -- 172.21.15.46:0/923044781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f08100096c0 con 0x7f08281033e0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f0825ffb640 1 -- 172.21.15.46:0/923044781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0810004020 con 0x7f08281033e0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0828101490 con 0x7f08281033e0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.301+0000 7f0825ffb640 1 -- 172.21.15.46:0/923044781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f08100049d0 con 0x7f08281033e0 2024-09-10T04:23:57.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.302+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08281a3e40 con 0x7f08281033e0 2024-09-10T04:23:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.303+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f082806aa90 con 0x7f08281033e0 2024-09-10T04:23:57.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.303+0000 7f0825ffb640 1 -- 172.21.15.46:0/923044781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f08100041d0 con 0x7f08281033e0 2024-09-10T04:23:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.304+0000 7f0825ffb640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f081407ac40 0x7f081407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.304+0000 7f0825ffb640 1 -- 172.21.15.46:0/923044781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f081009dc60 con 0x7f08281033e0 2024-09-10T04:23:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.304+0000 7f0827fff640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f081407ac40 0x7f081407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.304+0000 7f0827fff640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f081407ac40 0x7f081407d100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f0818031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:57.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.308+0000 7f0825ffb640 1 -- 172.21.15.46:0/923044781 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0810069420 con 0x7f08281033e0 2024-09-10T04:23:57.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.530+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0828105f90 con 0x7f08281033e0 2024-09-10T04:23:57.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.531+0000 7f0825ffb640 1 -- 172.21.15.46:0/923044781 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f081006e2d0 con 0x7f08281033e0 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:23:57.532 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:23:57.533 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:23:57.534 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:57.535 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:23:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:23:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f081407ac40 msgr2=0x7f081407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f081407ac40 0x7f081407d100 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0818000c00 tx=0x7f0818031400 comp rx=0 tx=0).stop 2024-09-10T04:23:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08281033e0 msgr2=0x7f0828100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08281033e0 0x7f0828100c60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f081000e970 tx=0x7f081000ee30 comp rx=0 tx=0).stop 2024-09-10T04:23:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 shutdown_connections 2024-09-10T04:23:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f081407ac40 0x7f081407d100 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08281033e0 0x7f0828100c60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 --2- 172.21.15.46:0/923044781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0828102ac0 0x7f0828100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 >> 172.21.15.46:0/923044781 conn(0x7f08280fc520 msgr2=0x7f0828108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 shutdown_connections 2024-09-10T04:23:57.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.533+0000 7f082e7b9640 1 -- 172.21.15.46:0/923044781 wait complete. 2024-09-10T04:23:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 -- 172.21.15.46:0/642662301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 msgr2=0x7f204c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 --2- 172.21.15.46:0/642662301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c106be0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f203c009880 tx=0x7f203c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 -- 172.21.15.46:0/642662301 shutdown_connections 2024-09-10T04:23:57.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 --2- 172.21.15.46:0/642662301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c100800 0x7f204c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 --2- 172.21.15.46:0/642662301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c106be0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 -- 172.21.15.46:0/642662301 >> 172.21.15.46:0/642662301 conn(0x7f204c0fc520 msgr2=0x7f204c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 -- 172.21.15.46:0/642662301 shutdown_connections 2024-09-10T04:23:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 -- 172.21.15.46:0/642662301 wait complete. 2024-09-10T04:23:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.619+0000 7f20526ac640 1 Processor -- start 2024-09-10T04:23:57.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f20526ac640 1 -- start start 2024-09-10T04:23:57.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f20526ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c100800 0x7f204c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f20526ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f20526ac640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10dcd0 con 0x7f204c106800 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f20526ac640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f204c10db50 con 0x7f204c100800 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35322/0 (socket says 172.21.15.46:35322) 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f204b7fe640 1 -- 172.21.15.46:0/2931413077 learned_addr learned my addr 172.21.15.46:0/2931413077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f204bfff640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c100800 0x7f204c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f204b7fe640 1 -- 172.21.15.46:0/2931413077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c100800 msgr2=0x7f204c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f204b7fe640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c100800 0x7f204c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.620+0000 7f204b7fe640 1 -- 172.21.15.46:0/2931413077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204c199320 con 0x7f204c106800 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.621+0000 7f204b7fe640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c198d20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f204000d6f0 tx=0x7f204000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.621+0000 7f20497fa640 1 -- 172.21.15.46:0/2931413077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2040017070 con 0x7f204c106800 2024-09-10T04:23:57.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.621+0000 7f20497fa640 1 -- 172.21.15.46:0/2931413077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f204000de00 con 0x7f204c106800 2024-09-10T04:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.621+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204c199610 con 0x7f204c106800 2024-09-10T04:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.621+0000 7f20497fa640 1 -- 172.21.15.46:0/2931413077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2040004bb0 con 0x7f204c106800 2024-09-10T04:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.621+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204c1a1c10 con 0x7f204c106800 2024-09-10T04:23:57.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.622+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f204c101f40 con 0x7f204c106800 2024-09-10T04:23:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.623+0000 7f20497fa640 1 -- 172.21.15.46:0/2931413077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2040009a30 con 0x7f204c106800 2024-09-10T04:23:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.623+0000 7f20497fa640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f203407ac40 0x7f203407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.623+0000 7f20497fa640 1 -- 172.21.15.46:0/2931413077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f204009d7e0 con 0x7f204c106800 2024-09-10T04:23:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.623+0000 7f204bfff640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f203407ac40 0x7f203407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.624+0000 7f204bfff640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f203407ac40 0x7f203407d100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f203c0096c0 tx=0x7f203c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:57.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.628+0000 7f20497fa640 1 -- 172.21.15.46:0/2931413077 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f204c199610 con 0x7f204c106800 2024-09-10T04:23:57.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.830+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f204c062630 con 0x7f203407ac40 2024-09-10T04:23:57.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:57 smithi046 ceph-mon[109506]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:57.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:57 smithi046 ceph-mon[109506]: pgmap v548: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:57.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:57 smithi046 ceph-mon[109506]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:57.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:57 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1268503433' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:23:57.831 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:57 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/923044781' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:23:57.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.831+0000 7f20497fa640 1 -- 172.21.15.46:0/2931413077 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f204c062630 con 0x7f203407ac40 2024-09-10T04:23:57.832 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:23:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:23:57.832 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:23:57.833 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:23:57.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f203407ac40 msgr2=0x7f203407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f203407ac40 0x7f203407d100 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f203c0096c0 tx=0x7f203c031400 comp rx=0 tx=0).stop 2024-09-10T04:23:57.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 msgr2=0x7f204c198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c198d20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f204000d6f0 tx=0x7f204000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 shutdown_connections 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f203407ac40 0x7f203407d100 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f204c106800 0x7f204c198d20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 --2- 172.21.15.46:0/2931413077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c100800 0x7f204c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 >> 172.21.15.46:0/2931413077 conn(0x7f204c0fc520 msgr2=0x7f204c0fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 shutdown_connections 2024-09-10T04:23:57.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.833+0000 7f20526ac640 1 -- 172.21.15.46:0/2931413077 wait complete. 2024-09-10T04:23:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:57 smithi070 ceph-mon[79774]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:57 smithi070 ceph-mon[79774]: pgmap v548: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:57.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:57 smithi070 ceph-mon[79774]: from='client.34874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:57.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:57 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1268503433' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:23:57.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:57 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/923044781' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:23:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.918+0000 7fc72a712640 1 -- 172.21.15.46:0/1415674702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724107900 msgr2=0x7fc724107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.918+0000 7fc72a712640 1 --2- 172.21.15.46:0/1415674702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724107900 0x7fc724107ce0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc714009880 tx=0x7fc71402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 -- 172.21.15.46:0/1415674702 shutdown_connections 2024-09-10T04:23:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 --2- 172.21.15.46:0/1415674702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724101900 0x7fc724101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 --2- 172.21.15.46:0/1415674702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724107900 0x7fc724107ce0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 -- 172.21.15.46:0/1415674702 >> 172.21.15.46:0/1415674702 conn(0x7fc7240fd600 msgr2=0x7fc7240ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:57.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 -- 172.21.15.46:0/1415674702 shutdown_connections 2024-09-10T04:23:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 -- 172.21.15.46:0/1415674702 wait complete. 2024-09-10T04:23:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 Processor -- start 2024-09-10T04:23:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.919+0000 7fc72a712640 1 -- start start 2024-09-10T04:23:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc72a712640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724101900 0x7fc7240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc72a712640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724107900 0x7fc724074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc72a712640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc72410edb0 con 0x7fc724101900 2024-09-10T04:23:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc72a712640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc72410ec30 con 0x7fc724107900 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc723fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724101900 0x7fc7240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc723fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724101900 0x7fc7240744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35350/0 (socket says 172.21.15.46:35350) 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc723fff640 1 -- 172.21.15.46:0/3717645404 learned_addr learned my addr 172.21.15.46:0/3717645404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc7237fe640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724107900 0x7fc724074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc723fff640 1 -- 172.21.15.46:0/3717645404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724107900 msgr2=0x7fc724074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc723fff640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724107900 0x7fc724074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc723fff640 1 -- 172.21.15.46:0/3717645404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc724074f70 con 0x7fc724101900 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc7237fe640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724107900 0x7fc724074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.920+0000 7fc723fff640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724101900 0x7fc7240744f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc714031460 tx=0x7fc714004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.921+0000 7fc7217fa640 1 -- 172.21.15.46:0/3717645404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc71403f070 con 0x7fc724101900 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.921+0000 7fc7217fa640 1 -- 172.21.15.46:0/3717645404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc7140035f0 con 0x7fc724101900 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.921+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc724075200 con 0x7fc724101900 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.921+0000 7fc7217fa640 1 -- 172.21.15.46:0/3717645404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc714035030 con 0x7fc724101900 2024-09-10T04:23:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.921+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7241a3d80 con 0x7fc724101900 2024-09-10T04:23:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.922+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc724103040 con 0x7fc724101900 2024-09-10T04:23:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.922+0000 7fc7217fa640 1 -- 172.21.15.46:0/3717645404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc714042070 con 0x7fc724101900 2024-09-10T04:23:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.923+0000 7fc7217fa640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc70c07abf0 0x7fc70c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:23:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.923+0000 7fc7217fa640 1 -- 172.21.15.46:0/3717645404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc7140be660 con 0x7fc724101900 2024-09-10T04:23:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.923+0000 7fc7237fe640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc70c07abf0 0x7fc70c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:23:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.923+0000 7fc7237fe640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc70c07abf0 0x7fc70c07d0b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc724075f80 tx=0x7fc718005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:23:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:57.927+0000 7fc7217fa640 1 -- 172.21.15.46:0/3717645404 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc714089da0 con 0x7fc724101900 2024-09-10T04:23:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.185+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fc724104b10 con 0x7fc724101900 2024-09-10T04:23:58.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.185+0000 7fc7217fa640 1 -- 172.21.15.46:0/3717645404 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fc71408ec50 con 0x7fc724101900 2024-09-10T04:23:58.187 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:23:58.187 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:23:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:23:58.187 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:23:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:23:58.187 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:23:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:23:58.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc70c07abf0 msgr2=0x7fc70c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc70c07abf0 0x7fc70c07d0b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc724075f80 tx=0x7fc718005f90 comp rx=0 tx=0).stop 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724101900 msgr2=0x7fc7240744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724101900 0x7fc7240744f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc714031460 tx=0x7fc714004530 comp rx=0 tx=0).stop 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 shutdown_connections 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc70c07abf0 0x7fc70c07d0b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc724107900 0x7fc724074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 --2- 172.21.15.46:0/3717645404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc724101900 0x7fc7240744f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 >> 172.21.15.46:0/3717645404 conn(0x7fc7240fd600 msgr2=0x7fc7240ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 shutdown_connections 2024-09-10T04:23:58.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:23:58.188+0000 7fc72a712640 1 -- 172.21.15.46:0/3717645404 wait complete. 2024-09-10T04:23:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:58 smithi070 ceph-mon[79774]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:58 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3717645404' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:23:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:58 smithi046 ceph-mon[109506]: from='client.34878 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:58 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3717645404' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:23:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:59 smithi070 ceph-mon[79774]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:23:59 smithi070 ceph-mon[79774]: pgmap v549: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:23:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:59 smithi046 ceph-mon[109506]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:23:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:23:59 smithi046 ceph-mon[109506]: pgmap v549: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:01 smithi070 ceph-mon[79774]: pgmap v550: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:01 smithi046 ceph-mon[109506]: pgmap v550: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:03 smithi070 ceph-mon[79774]: pgmap v551: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:03 smithi046 ceph-mon[109506]: pgmap v551: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:05 smithi070 ceph-mon[79774]: pgmap v552: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:05.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:05 smithi046 ceph-mon[109506]: pgmap v552: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:07 smithi070 ceph-mon[79774]: pgmap v553: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:07 smithi046 ceph-mon[109506]: pgmap v553: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:09 smithi070 ceph-mon[79774]: pgmap v554: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:09 smithi046 ceph-mon[109506]: pgmap v554: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:11 smithi070 ceph-mon[79774]: pgmap v555: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:11.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:11 smithi046 ceph-mon[109506]: pgmap v555: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:13 smithi046 ceph-mon[109506]: pgmap v556: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:13 smithi070 ceph-mon[79774]: pgmap v556: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:15 smithi070 ceph-mon[79774]: pgmap v557: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:16.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:15 smithi046 ceph-mon[109506]: pgmap v557: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:17.887 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:17 smithi070 ceph-mon[79774]: pgmap v558: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:18.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:17 smithi046 ceph-mon[109506]: pgmap v558: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:19 smithi070 ceph-mon[79774]: pgmap v559: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:19 smithi046 ceph-mon[109506]: pgmap v559: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:21 smithi070 ceph-mon[79774]: pgmap v560: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:21 smithi046 ceph-mon[109506]: pgmap v560: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:23 smithi070 ceph-mon[79774]: pgmap v561: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:23 smithi046 ceph-mon[109506]: pgmap v561: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:26.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:25 smithi070 ceph-mon[79774]: pgmap v562: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:26.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:25 smithi046 ceph-mon[109506]: pgmap v562: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:27 smithi070 ceph-mon[79774]: pgmap v563: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:28.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:27 smithi046 ceph-mon[109506]: pgmap v563: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.285+0000 7ff038b12640 1 -- 172.21.15.46:0/3960302312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 msgr2=0x7ff034100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.285+0000 7ff038b12640 1 --2- 172.21.15.46:0/3960302312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff034100c60 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009880 tx=0x7ff01c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.286+0000 7ff038b12640 1 -- 172.21.15.46:0/3960302312 shutdown_connections 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.286+0000 7ff038b12640 1 --2- 172.21.15.46:0/3960302312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff034100c60 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.286+0000 7ff038b12640 1 --2- 172.21.15.46:0/3960302312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff034106800 0x7ff034106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.286+0000 7ff038b12640 1 -- 172.21.15.46:0/3960302312 >> 172.21.15.46:0/3960302312 conn(0x7ff0340fc520 msgr2=0x7ff0340fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.286+0000 7ff038b12640 1 -- 172.21.15.46:0/3960302312 shutdown_connections 2024-09-10T04:24:28.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.286+0000 7ff038b12640 1 -- 172.21.15.46:0/3960302312 wait complete. 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.286+0000 7ff038b12640 1 Processor -- start 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff038b12640 1 -- start start 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff038b12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff0341987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff038b12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff034106800 0x7ff034198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff038b12640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff03410dcd0 con 0x7ff034100800 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff038b12640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff03410db50 con 0x7ff034106800 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff03259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff0341987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff03259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff0341987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39832/0 (socket says 172.21.15.46:39832) 2024-09-10T04:24:28.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff03259c640 1 -- 172.21.15.46:0/4112137147 learned_addr learned my addr 172.21.15.46:0/4112137147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff02bfff640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff034106800 0x7ff034198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff03259c640 1 -- 172.21.15.46:0/4112137147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff034106800 msgr2=0x7ff034198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff03259c640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff034106800 0x7ff034198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.287+0000 7ff03259c640 1 -- 172.21.15.46:0/4112137147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff034199320 con 0x7ff034100800 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.288+0000 7ff03259c640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff0341987e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff02400e970 tx=0x7ff02400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.288+0000 7ff02b7fe640 1 -- 172.21.15.46:0/4112137147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff02400cac0 con 0x7ff034100800 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.288+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff034199610 con 0x7ff034100800 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.288+0000 7ff02b7fe640 1 -- 172.21.15.46:0/4112137147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff0240054e0 con 0x7ff034100800 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.288+0000 7ff02b7fe640 1 -- 172.21.15.46:0/4112137147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff024010430 con 0x7ff034100800 2024-09-10T04:24:28.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.288+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0341a1c10 con 0x7ff034100800 2024-09-10T04:24:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.289+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff034101f40 con 0x7ff034100800 2024-09-10T04:24:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.290+0000 7ff02b7fe640 1 -- 172.21.15.46:0/4112137147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff024004f00 con 0x7ff034100800 2024-09-10T04:24:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.290+0000 7ff02b7fe640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff02007abf0 0x7ff02007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.290+0000 7ff02b7fe640 1 -- 172.21.15.46:0/4112137147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ff02409e320 con 0x7ff034100800 2024-09-10T04:24:28.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.290+0000 7ff02bfff640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff02007abf0 0x7ff02007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.291+0000 7ff02bfff640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff02007abf0 0x7ff02007d0b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff03406ad00 tx=0x7ff01c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:28.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.295+0000 7ff02b7fe640 1 -- 172.21.15.46:0/4112137147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff034199610 con 0x7ff034100800 2024-09-10T04:24:28.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.499+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff034062630 con 0x7ff02007abf0 2024-09-10T04:24:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.500+0000 7ff02b7fe640 1 -- 172.21.15.46:0/4112137147 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7ff034062630 con 0x7ff02007abf0 2024-09-10T04:24:28.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff02007abf0 msgr2=0x7ff02007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff02007abf0 0x7ff02007d0b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff03406ad00 tx=0x7ff01c031400 comp rx=0 tx=0).stop 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 msgr2=0x7ff0341987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff0341987e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff02400e970 tx=0x7ff02400ed70 comp rx=0 tx=0).stop 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 shutdown_connections 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff02007abf0 0x7ff02007d0b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff034106800 0x7ff034198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.502+0000 7ff038b12640 1 --2- 172.21.15.46:0/4112137147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034100800 0x7ff0341987e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.503+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 >> 172.21.15.46:0/4112137147 conn(0x7ff0340fc520 msgr2=0x7ff0340fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.503+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 shutdown_connections 2024-09-10T04:24:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.503+0000 7ff038b12640 1 -- 172.21.15.46:0/4112137147 wait complete. 2024-09-10T04:24:28.514 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/4214936732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 msgr2=0x7f0ad010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/4214936732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad010b8d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009880 tx=0x7f0abc02a530 comp rx=0 tx=0).stop 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/4214936732 shutdown_connections 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/4214936732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad010b8d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/4214936732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0102ac0 0x7f0ad0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/4214936732 >> 172.21.15.46:0/4214936732 conn(0x7f0ad00fc520 msgr2=0x7f0ad00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/4214936732 shutdown_connections 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.589+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/4214936732 wait complete. 2024-09-10T04:24:28.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0ad6bc9640 1 Processor -- start 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0ad6bc9640 1 -- start start 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0ad6bc9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0102ac0 0x7f0ad0100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0ad6bc9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad0100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0ad6bc9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad010edf0 con 0x7f0ad01033e0 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0ad6bc9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad010ec70 con 0x7f0ad0102ac0 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0acffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad0100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0acffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad0100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39850/0 (socket says 172.21.15.46:39850) 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0acffff640 1 -- 172.21.15.46:0/2845105017 learned_addr learned my addr 172.21.15.46:0/2845105017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0ad4965640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0102ac0 0x7f0ad0100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0acffff640 1 -- 172.21.15.46:0/2845105017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0102ac0 msgr2=0x7f0ad0100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0acffff640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0102ac0 0x7f0ad0100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.590+0000 7f0acffff640 1 -- 172.21.15.46:0/2845105017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad01011a0 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.591+0000 7f0acffff640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad0100c60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0abc031460 tx=0x7f0abc0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.591+0000 7f0acdffb640 1 -- 172.21.15.46:0/2845105017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0abc03f070 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.591+0000 7f0acdffb640 1 -- 172.21.15.46:0/2845105017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0abc0035f0 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.591+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad0101430 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.591+0000 7f0acdffb640 1 -- 172.21.15.46:0/2845105017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0abc035030 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.591+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad01a3de0 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.592+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad006aa90 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.593+0000 7f0acdffb640 1 -- 172.21.15.46:0/2845105017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0abc042070 con 0x7f0ad01033e0 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.593+0000 7f0acdffb640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ac007abf0 0x7f0ac007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.593+0000 7f0acdffb640 1 -- 172.21.15.46:0/2845105017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0abc0be4b0 con 0x7f0ad01033e0 2024-09-10T04:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.593+0000 7f0ad4965640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ac007abf0 0x7f0ac007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.594+0000 7f0ad4965640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ac007abf0 0x7f0ac007d0b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009bd0 tx=0x7f0ac4005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:28.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.598+0000 7f0acdffb640 1 -- 172.21.15.46:0/2845105017 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0abc089bf0 con 0x7f0ad01033e0 2024-09-10T04:24:28.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.801+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0ad0062630 con 0x7f0ac007abf0 2024-09-10T04:24:28.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.802+0000 7f0acdffb640 1 -- 172.21.15.46:0/2845105017 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0ad0062630 con 0x7f0ac007abf0 2024-09-10T04:24:28.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ac007abf0 msgr2=0x7f0ac007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ac007abf0 0x7f0ac007d0b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009bd0 tx=0x7f0ac4005f90 comp rx=0 tx=0).stop 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 msgr2=0x7f0ad0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad0100c60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0abc031460 tx=0x7f0abc0045c0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 shutdown_connections 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ac007abf0 0x7f0ac007d0b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ad01033e0 0x7f0ad0100c60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 --2- 172.21.15.46:0/2845105017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ad0102ac0 0x7f0ad0100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 >> 172.21.15.46:0/2845105017 conn(0x7f0ad00fc520 msgr2=0x7f0ad0108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 shutdown_connections 2024-09-10T04:24:28.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.805+0000 7f0ad6bc9640 1 -- 172.21.15.46:0/2845105017 wait complete. 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 -- 172.21.15.46:0/2293273822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 msgr2=0x7f8f40102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/2293273822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f40102ea0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009880 tx=0x7f8f3002a530 comp rx=0 tx=0).stop 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 -- 172.21.15.46:0/2293273822 shutdown_connections 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/2293273822 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f401033e0 0x7f8f4010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/2293273822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f40102ea0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 -- 172.21.15.46:0/2293273822 >> 172.21.15.46:0/2293273822 conn(0x7f8f400fc520 msgr2=0x7f8f400fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 -- 172.21.15.46:0/2293273822 shutdown_connections 2024-09-10T04:24:28.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.889+0000 7f8f45ae4640 1 -- 172.21.15.46:0/2293273822 wait complete. 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f45ae4640 1 Processor -- start 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f45ae4640 1 -- start start 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f45ae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f4019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f45ae4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f401033e0 0x7f8f4019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f45ae4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4010edf0 con 0x7f8f40102ac0 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f45ae4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4010ec70 con 0x7f8f401033e0 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f3f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f4019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f3f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f4019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40376/0 (socket says 172.21.15.46:40376) 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.890+0000 7f8f3f7fe640 1 -- 172.21.15.46:0/4052149854 learned_addr learned my addr 172.21.15.46:0/4052149854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3effd640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f401033e0 0x7f8f4019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3f7fe640 1 -- 172.21.15.46:0/4052149854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f401033e0 msgr2=0x7f8f4019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3f7fe640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f401033e0 0x7f8f4019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3f7fe640 1 -- 172.21.15.46:0/4052149854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f401977e0 con 0x7f8f40102ac0 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3f7fe640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f4019d660 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f30031460 tx=0x7f8f300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3cff9640 1 -- 172.21.15.46:0/4052149854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f3003f070 con 0x7f8f40102ac0 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f40197a70 con 0x7f8f40102ac0 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3cff9640 1 -- 172.21.15.46:0/4052149854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f300044d0 con 0x7f8f40102ac0 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f3cff9640 1 -- 172.21.15.46:0/4052149854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f30035030 con 0x7f8f40102ac0 2024-09-10T04:24:28.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.891+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f40197fa0 con 0x7f8f40102ac0 2024-09-10T04:24:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.892+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4006aa90 con 0x7f8f40102ac0 2024-09-10T04:24:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.893+0000 7f8f3cff9640 1 -- 172.21.15.46:0/4052149854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8f30042070 con 0x7f8f40102ac0 2024-09-10T04:24:28.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.894+0000 7f8f3cff9640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2c07ac00 0x7f8f2c07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.894+0000 7f8f3cff9640 1 -- 172.21.15.46:0/4052149854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f8f300be750 con 0x7f8f40102ac0 2024-09-10T04:24:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.894+0000 7f8f3effd640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2c07ac00 0x7f8f2c07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:28.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.895+0000 7f8f3effd640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2c07ac00 0x7f8f2c07d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8f40108900 tx=0x7f8f28005f50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:28.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:28.898+0000 7f8f3cff9640 1 -- 172.21.15.46:0/4052149854 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8f30004cf0 con 0x7f8f40102ac0 2024-09-10T04:24:29.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.095+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8f40198310 con 0x7f8f2c07ac00 2024-09-10T04:24:29.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.104+0000 7f8f3cff9640 1 -- 172.21.15.46:0/4052149854 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f8f40198310 con 0x7f8f2c07ac00 2024-09-10T04:24:29.106 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:24:29.106 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (16m) 10m ago 27m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (18m) 10m ago 27m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (18m) 10m ago 26m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (14m) 10m ago 27m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (14m) 10m ago 26m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (15m) 10m ago 27m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (10m) 10m ago 21m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (10m) 10m ago 22m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (10m) 10m ago 22m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (21m) 10m ago 21m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (18m) 10m ago 28m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (17m) 10m ago 26m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (14m) 10m ago 28m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (14m) 10m ago 26m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:24:29.107 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (16m) 10m ago 27m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (16m) 10m ago 26m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (14m) 10m ago 25m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (13m) 10m ago 24m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (12m) 10m ago 24m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (12m) 10m ago 23m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (11m) 10m ago 23m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (11m) 10m ago 23m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:24:29.108 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (16m) 10m ago 27m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2c07ac00 msgr2=0x7f8f2c07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2c07ac00 0x7f8f2c07d0c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8f40108900 tx=0x7f8f28005f50 comp rx=0 tx=0).stop 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 msgr2=0x7f8f4019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f4019d660 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f30031460 tx=0x7f8f300048f0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 shutdown_connections 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f2c07ac00 0x7f8f2c07d0c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f401033e0 0x7f8f4019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 --2- 172.21.15.46:0/4052149854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f40102ac0 0x7f8f4019d660 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 >> 172.21.15.46:0/4052149854 conn(0x7f8f400fc520 msgr2=0x7f8f40108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 shutdown_connections 2024-09-10T04:24:29.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.107+0000 7f8f45ae4640 1 -- 172.21.15.46:0/4052149854 wait complete. 2024-09-10T04:24:29.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 -- 172.21.15.46:0/2129711034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 msgr2=0x7fdf74102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/2129711034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74102ea0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdf64009880 tx=0x7fdf6402a530 comp rx=0 tx=0).stop 2024-09-10T04:24:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 -- 172.21.15.46:0/2129711034 shutdown_connections 2024-09-10T04:24:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/2129711034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf741033e0 0x7fdf7410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/2129711034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74102ea0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 -- 172.21.15.46:0/2129711034 >> 172.21.15.46:0/2129711034 conn(0x7fdf740fc520 msgr2=0x7fdf740fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 -- 172.21.15.46:0/2129711034 shutdown_connections 2024-09-10T04:24:29.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.190+0000 7fdf78a5e640 1 -- 172.21.15.46:0/2129711034 wait complete. 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.191+0000 7fdf78a5e640 1 Processor -- start 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.191+0000 7fdf78a5e640 1 -- start start 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.191+0000 7fdf78a5e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.191+0000 7fdf78a5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf741033e0 0x7fdf74100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.191+0000 7fdf78a5e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410edd0 con 0x7fdf74102ac0 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.191+0000 7fdf78a5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7410ec50 con 0x7fdf741033e0 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf7259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf7259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40396/0 (socket says 172.21.15.46:40396) 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf7259c640 1 -- 172.21.15.46:0/3112590407 learned_addr learned my addr 172.21.15.46:0/3112590407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf71d9b640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf741033e0 0x7fdf74100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf7259c640 1 -- 172.21.15.46:0/3112590407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf741033e0 msgr2=0x7fdf74100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf7259c640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf741033e0 0x7fdf74100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf7259c640 1 -- 172.21.15.46:0/3112590407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf741011a0 con 0x7fdf74102ac0 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf7259c640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74100720 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf64031460 tx=0x7fdf640048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf637fe640 1 -- 172.21.15.46:0/3112590407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf6403f070 con 0x7fdf74102ac0 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf637fe640 1 -- 172.21.15.46:0/3112590407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf64004cf0 con 0x7fdf74102ac0 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf74101430 con 0x7fdf74102ac0 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf637fe640 1 -- 172.21.15.46:0/3112590407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf64035030 con 0x7fdf74102ac0 2024-09-10T04:24:29.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.192+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a3d80 con 0x7fdf74102ac0 2024-09-10T04:24:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.194+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7406aa90 con 0x7fdf74102ac0 2024-09-10T04:24:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.194+0000 7fdf637fe640 1 -- 172.21.15.46:0/3112590407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdf64042070 con 0x7fdf74102ac0 2024-09-10T04:24:29.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.195+0000 7fdf637fe640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf5c07abf0 0x7fdf5c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.195+0000 7fdf637fe640 1 -- 172.21.15.46:0/3112590407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fdf640be6a0 con 0x7fdf74102ac0 2024-09-10T04:24:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.195+0000 7fdf71d9b640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf5c07abf0 0x7fdf5c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.195+0000 7fdf71d9b640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf5c07abf0 0x7fdf5c07d0b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf741021b0 tx=0x7fdf6800a4e0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:29.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.200+0000 7fdf637fe640 1 -- 172.21.15.46:0/3112590407 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdf64089de0 con 0x7fdf74102ac0 2024-09-10T04:24:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.462+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fdf74105f90 con 0x7fdf74102ac0 2024-09-10T04:24:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.462+0000 7fdf637fe640 1 -- 172.21.15.46:0/3112590407 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fdf6408ec90 con 0x7fdf74102ac0 2024-09-10T04:24:29.464 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:24:29.465 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf5c07abf0 msgr2=0x7fdf5c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf5c07abf0 0x7fdf5c07d0b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdf741021b0 tx=0x7fdf6800a4e0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 msgr2=0x7fdf74100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74100720 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdf64031460 tx=0x7fdf640048f0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 shutdown_connections 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf5c07abf0 0x7fdf5c07d0b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf741033e0 0x7fdf74100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 --2- 172.21.15.46:0/3112590407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf74102ac0 0x7fdf74100720 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.464+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 >> 172.21.15.46:0/3112590407 conn(0x7fdf740fc520 msgr2=0x7fdf74108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.465+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 shutdown_connections 2024-09-10T04:24:29.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.465+0000 7fdf78a5e640 1 -- 172.21.15.46:0/3112590407 wait complete. 2024-09-10T04:24:29.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 -- 172.21.15.46:0/4014788318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea4102ab0 msgr2=0x7f0ea4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 --2- 172.21.15.46:0/4014788318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea4102ab0 0x7f0ea4102e90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0e94009880 tx=0x7f0e9402a530 comp rx=0 tx=0).stop 2024-09-10T04:24:29.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 -- 172.21.15.46:0/4014788318 shutdown_connections 2024-09-10T04:24:29.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 --2- 172.21.15.46:0/4014788318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ea41033d0 0x7f0ea410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 --2- 172.21.15.46:0/4014788318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea4102ab0 0x7f0ea4102e90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 -- 172.21.15.46:0/4014788318 >> 172.21.15.46:0/4014788318 conn(0x7f0ea40fc550 msgr2=0x7f0ea40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 -- 172.21.15.46:0/4014788318 shutdown_connections 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 -- 172.21.15.46:0/4014788318 wait complete. 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 Processor -- start 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.549+0000 7f0eaa535640 1 -- start start 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0eaa535640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ea4102ab0 0x7f0ea419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0eaa535640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea41033d0 0x7f0ea419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0eaa535640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea410ede0 con 0x7f0ea41033d0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0eaa535640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea410ec60 con 0x7f0ea4102ab0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea41033d0 0x7f0ea419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea41033d0 0x7f0ea419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40416/0 (socket says 172.21.15.46:40416) 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea37fe640 1 -- 172.21.15.46:0/3960906016 learned_addr learned my addr 172.21.15.46:0/3960906016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea3fff640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ea4102ab0 0x7f0ea419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea37fe640 1 -- 172.21.15.46:0/3960906016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ea4102ab0 msgr2=0x7f0ea419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea37fe640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ea4102ab0 0x7f0ea419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea37fe640 1 -- 172.21.15.46:0/3960906016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea41977b0 con 0x7f0ea41033d0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.550+0000 7f0ea37fe640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea41033d0 0x7f0ea419db70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800d6d0 tx=0x7f0e9800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.551+0000 7f0ea17fa640 1 -- 172.21.15.46:0/3960906016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0e98017070 con 0x7f0ea41033d0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.551+0000 7f0ea17fa640 1 -- 172.21.15.46:0/3960906016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0e980054e0 con 0x7f0ea41033d0 2024-09-10T04:24:29.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.551+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea4197aa0 con 0x7f0ea41033d0 2024-09-10T04:24:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.551+0000 7f0ea17fa640 1 -- 172.21.15.46:0/3960906016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0e98004ae0 con 0x7f0ea41033d0 2024-09-10T04:24:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.551+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea4198030 con 0x7f0ea41033d0 2024-09-10T04:24:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.552+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea406a580 con 0x7f0ea41033d0 2024-09-10T04:24:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.552+0000 7f0ea17fa640 1 -- 172.21.15.46:0/3960906016 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0e98004d90 con 0x7f0ea41033d0 2024-09-10T04:24:29.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.553+0000 7f0ea17fa640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0e8c07ac50 0x7f0e8c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.553+0000 7f0ea17fa640 1 -- 172.21.15.46:0/3960906016 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0e9809d720 con 0x7f0ea41033d0 2024-09-10T04:24:29.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.553+0000 7f0ea3fff640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0e8c07ac50 0x7f0e8c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.553+0000 7f0ea3fff640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0e8c07ac50 0x7f0e8c07d110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e940096c0 tx=0x7f0e94031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:29.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.557+0000 7f0ea17fa640 1 -- 172.21.15.46:0/3960906016 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0e98068ee0 con 0x7f0ea41033d0 2024-09-10T04:24:29.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.780+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0ea41982e0 con 0x7f0ea41033d0 2024-09-10T04:24:29.782 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:29 smithi046 ceph-mon[109506]: pgmap v564: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:29.783 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:29 smithi046 ceph-mon[109506]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:29.783 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:29 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3112590407' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:24:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.781+0000 7f0ea17fa640 1 -- 172.21.15.46:0/3960906016 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f0e9806dd90 con 0x7f0ea41033d0 2024-09-10T04:24:29.783 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:24:29.783 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:24:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:24:29.785 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:24:29.786 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:24:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:24:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:24:29.787 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:24:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:24:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0e8c07ac50 msgr2=0x7f0e8c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0e8c07ac50 0x7f0e8c07d110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e940096c0 tx=0x7f0e94031400 comp rx=0 tx=0).stop 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea41033d0 msgr2=0x7f0ea419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea41033d0 0x7f0ea419db70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800d6d0 tx=0x7f0e9800db90 comp rx=0 tx=0).stop 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 shutdown_connections 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0e8c07ac50 0x7f0e8c07d110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ea41033d0 0x7f0ea419db70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 --2- 172.21.15.46:0/3960906016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ea4102ab0 0x7f0ea419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 >> 172.21.15.46:0/3960906016 conn(0x7f0ea40fc550 msgr2=0x7f0ea410aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 shutdown_connections 2024-09-10T04:24:29.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.783+0000 7f0eaa535640 1 -- 172.21.15.46:0/3960906016 wait complete. 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 -- 172.21.15.46:0/2584799189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460102ac0 msgr2=0x7fc460102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 --2- 172.21.15.46:0/2584799189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460102ac0 0x7fc460102ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc450009880 tx=0x7fc45002a530 comp rx=0 tx=0).stop 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 -- 172.21.15.46:0/2584799189 shutdown_connections 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 --2- 172.21.15.46:0/2584799189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4601033e0 0x7fc46010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 --2- 172.21.15.46:0/2584799189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc460102ac0 0x7fc460102ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 -- 172.21.15.46:0/2584799189 >> 172.21.15.46:0/2584799189 conn(0x7fc4600fc520 msgr2=0x7fc4600fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 -- 172.21.15.46:0/2584799189 shutdown_connections 2024-09-10T04:24:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 -- 172.21.15.46:0/2584799189 wait complete. 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.869+0000 7fc465246640 1 Processor -- start 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc465246640 1 -- start start 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc465246640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc460102ac0 0x7fc460100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc465246640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601033e0 0x7fc460100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc465246640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc46010edd0 con 0x7fc4601033e0 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc465246640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc46010ec50 con 0x7fc460102ac0 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc45e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601033e0 0x7fc460100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc45e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601033e0 0x7fc460100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40440/0 (socket says 172.21.15.46:40440) 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc45e59c640 1 -- 172.21.15.46:0/3226798071 learned_addr learned my addr 172.21.15.46:0/3226798071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc45ed9d640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc460102ac0 0x7fc460100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc45e59c640 1 -- 172.21.15.46:0/3226798071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc460102ac0 msgr2=0x7fc460100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc45e59c640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc460102ac0 0x7fc460100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.870+0000 7fc45e59c640 1 -- 172.21.15.46:0/3226798071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4601011a0 con 0x7fc4601033e0 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.871+0000 7fc45e59c640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601033e0 0x7fc460100c60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d6c0 tx=0x7fc45400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.871+0000 7fc447fff640 1 -- 172.21.15.46:0/3226798071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc454017070 con 0x7fc4601033e0 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.871+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc460101490 con 0x7fc4601033e0 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.871+0000 7fc447fff640 1 -- 172.21.15.46:0/3226798071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc454009ce0 con 0x7fc4601033e0 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.871+0000 7fc447fff640 1 -- 172.21.15.46:0/3226798071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc45400f3c0 con 0x7fc4601033e0 2024-09-10T04:24:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.871+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a3f60 con 0x7fc4601033e0 2024-09-10T04:24:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.872+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46006aa90 con 0x7fc4601033e0 2024-09-10T04:24:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.873+0000 7fc447fff640 1 -- 172.21.15.46:0/3226798071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc454010460 con 0x7fc4601033e0 2024-09-10T04:24:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.873+0000 7fc447fff640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc44807abf0 0x7fc44807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.873+0000 7fc447fff640 1 -- 172.21.15.46:0/3226798071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc45409d9b0 con 0x7fc4601033e0 2024-09-10T04:24:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.873+0000 7fc45ed9d640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc44807abf0 0x7fc44807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:29.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.874+0000 7fc45ed9d640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc44807abf0 0x7fc44807d0b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4500096c0 tx=0x7fc450031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:29.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:29.878+0000 7fc447fff640 1 -- 172.21.15.46:0/3226798071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc4540690f0 con 0x7fc4601033e0 2024-09-10T04:24:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:29 smithi070 ceph-mon[79774]: pgmap v564: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:29 smithi070 ceph-mon[79774]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:29 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3112590407' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:24:30.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.084+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc460101c50 con 0x7fc44807abf0 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.085+0000 7fc447fff640 1 -- 172.21.15.46:0/3226798071 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fc460101c50 con 0x7fc44807abf0 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:24:30.087 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:24:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:24:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:24:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:24:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:24:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:24:30.088 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:24:30.088 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:24:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc44807abf0 msgr2=0x7fc44807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc44807abf0 0x7fc44807d0b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc4500096c0 tx=0x7fc450031400 comp rx=0 tx=0).stop 2024-09-10T04:24:30.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601033e0 msgr2=0x7fc460100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601033e0 0x7fc460100c60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc45400d6c0 tx=0x7fc45400db80 comp rx=0 tx=0).stop 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 shutdown_connections 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc44807abf0 0x7fc44807d0b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4601033e0 0x7fc460100c60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 --2- 172.21.15.46:0/3226798071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc460102ac0 0x7fc460100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.087+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 >> 172.21.15.46:0/3226798071 conn(0x7fc4600fc520 msgr2=0x7fc460108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.088+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 shutdown_connections 2024-09-10T04:24:30.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.088+0000 7fc465246640 1 -- 172.21.15.46:0/3226798071 wait complete. 2024-09-10T04:24:30.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 -- 172.21.15.46:0/3171184528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c102b10 msgr2=0x7f718c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:30.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 --2- 172.21.15.46:0/3171184528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c102b10 0x7f718c102ef0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f717c009c50 tx=0x7f717c02aa30 comp rx=0 tx=0).stop 2024-09-10T04:24:30.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 -- 172.21.15.46:0/3171184528 shutdown_connections 2024-09-10T04:24:30.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 --2- 172.21.15.46:0/3171184528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c103430 0x7f718c10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 --2- 172.21.15.46:0/3171184528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c102b10 0x7f718c102ef0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 -- 172.21.15.46:0/3171184528 >> 172.21.15.46:0/3171184528 conn(0x7f718c0fc550 msgr2=0x7f718c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 -- 172.21.15.46:0/3171184528 shutdown_connections 2024-09-10T04:24:30.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.171+0000 7f7191021640 1 -- 172.21.15.46:0/3171184528 wait complete. 2024-09-10T04:24:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.172+0000 7f7191021640 1 Processor -- start 2024-09-10T04:24:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.172+0000 7f7191021640 1 -- start start 2024-09-10T04:24:30.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f7191021640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c102b10 0x7f718c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f7191021640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c103430 0x7f718c19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f7191021640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10ee40 con 0x7f718c103430 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f7191021640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f718c10ecc0 con 0x7f718c102b10 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c103430 0x7f718c19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c103430 0x7f718c19dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40452/0 (socket says 172.21.15.46:40452) 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718a59c640 1 -- 172.21.15.46:0/4143911609 learned_addr learned my addr 172.21.15.46:0/4143911609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718ad9d640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c102b10 0x7f718c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718a59c640 1 -- 172.21.15.46:0/4143911609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c102b10 msgr2=0x7f718c19d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718a59c640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c102b10 0x7f718c19d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718a59c640 1 -- 172.21.15.46:0/4143911609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c1978f0 con 0x7f718c103430 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f718a59c640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c103430 0x7f718c19dc70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f718000b420 tx=0x7f718000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.173+0000 7f7173fff640 1 -- 172.21.15.46:0/4143911609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7180011630 con 0x7f718c103430 2024-09-10T04:24:30.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.174+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c197be0 con 0x7f718c103430 2024-09-10T04:24:30.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.174+0000 7f7173fff640 1 -- 172.21.15.46:0/4143911609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7180016070 con 0x7f718c103430 2024-09-10T04:24:30.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.174+0000 7f7173fff640 1 -- 172.21.15.46:0/4143911609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7180010d40 con 0x7f718c103430 2024-09-10T04:24:30.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.174+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c198170 con 0x7f718c103430 2024-09-10T04:24:30.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.175+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c06a520 con 0x7f718c103430 2024-09-10T04:24:30.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.175+0000 7f7173fff640 1 -- 172.21.15.46:0/4143911609 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f71800117d0 con 0x7f718c103430 2024-09-10T04:24:30.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.176+0000 7f7173fff640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407abf0 0x7f717407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:24:30.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.176+0000 7f7173fff640 1 -- 172.21.15.46:0/4143911609 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f718009d420 con 0x7f718c103430 2024-09-10T04:24:30.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.176+0000 7f718ad9d640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407abf0 0x7f717407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:24:30.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.176+0000 7f718ad9d640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407abf0 0x7f717407d0b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f717c000f80 tx=0x7f717c03a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:24:30.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.180+0000 7f7173fff640 1 -- 172.21.15.46:0/4143911609 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7180068b60 con 0x7f718c103430 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.440+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f718c198e20 con 0x7f718c103430 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.440+0000 7f7173fff640 1 -- 172.21.15.46:0/4143911609 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f718006da10 con 0x7f718c103430 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:24:30.442 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:24:30.443 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:24:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407abf0 msgr2=0x7f717407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407abf0 0x7f717407d0b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f717c000f80 tx=0x7f717c03a040 comp rx=0 tx=0).stop 2024-09-10T04:24:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c103430 msgr2=0x7f718c19dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c103430 0x7f718c19dc70 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f718000b420 tx=0x7f718000b8e0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 shutdown_connections 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f717407abf0 0x7f717407d0b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f718c103430 0x7f718c19dc70 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 --2- 172.21.15.46:0/4143911609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f718c102b10 0x7f718c19d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.442+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 >> 172.21.15.46:0/4143911609 conn(0x7f718c0fc550 msgr2=0x7f718c103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.443+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 shutdown_connections 2024-09-10T04:24:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:24:30.443+0000 7f7191021640 1 -- 172.21.15.46:0/4143911609 wait complete. 2024-09-10T04:24:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:30 smithi046 ceph-mon[109506]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:30 smithi046 ceph-mon[109506]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:30 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3960906016' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:24:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:30 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4143911609' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:24:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:30 smithi070 ceph-mon[79774]: from='client.34902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:30 smithi070 ceph-mon[79774]: from='client.34906 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:30 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3960906016' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:24:31.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:30 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4143911609' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:24:31.868 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:31 smithi046 ceph-mon[109506]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:31.868 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:31 smithi046 ceph-mon[109506]: pgmap v565: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:32.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:31 smithi070 ceph-mon[79774]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:24:32.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:31 smithi070 ceph-mon[79774]: pgmap v565: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:33 smithi046 ceph-mon[109506]: pgmap v566: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:33 smithi070 ceph-mon[79774]: pgmap v566: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:36.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:35 smithi070 ceph-mon[79774]: pgmap v567: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:36.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:35 smithi046 ceph-mon[109506]: pgmap v567: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:38.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:37 smithi070 ceph-mon[79774]: pgmap v568: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:38.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:37 smithi046 ceph-mon[109506]: pgmap v568: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:40.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:39 smithi070 ceph-mon[79774]: pgmap v569: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:39 smithi046 ceph-mon[109506]: pgmap v569: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:42.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:41 smithi046 ceph-mon[109506]: pgmap v570: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:42.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:41 smithi070 ceph-mon[79774]: pgmap v570: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:43 smithi046 ceph-mon[109506]: pgmap v571: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:43 smithi070 ceph-mon[79774]: pgmap v571: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:45 smithi070 ceph-mon[79774]: pgmap v572: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:45 smithi046 ceph-mon[109506]: pgmap v572: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:47 smithi070 ceph-mon[79774]: pgmap v573: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:47 smithi046 ceph-mon[109506]: pgmap v573: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:50.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:49 smithi070 ceph-mon[79774]: pgmap v574: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:50.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:49 smithi046 ceph-mon[109506]: pgmap v574: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:51 smithi070 ceph-mon[79774]: pgmap v575: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:52.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:51 smithi046 ceph-mon[109506]: pgmap v575: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:53 smithi070 ceph-mon[79774]: pgmap v576: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:54.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:53 smithi046 ceph-mon[109506]: pgmap v576: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:54.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:24:56.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:55 smithi070 ceph-mon[79774]: pgmap v577: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:55 smithi046 ceph-mon[109506]: pgmap v577: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:58.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:57 smithi070 ceph-mon[79774]: pgmap v578: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:24:58.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:57 smithi046 ceph-mon[109506]: pgmap v578: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:00.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:24:59 smithi070 ceph-mon[79774]: pgmap v579: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:00.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:24:59 smithi046 ceph-mon[109506]: pgmap v579: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.540+0000 7fef7311f640 1 -- 172.21.15.46:0/253183046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 msgr2=0x7fef6c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.540+0000 7fef7311f640 1 --2- 172.21.15.46:0/253183046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c10b8d0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fef58009880 tx=0x7fef5802a530 comp rx=0 tx=0).stop 2024-09-10T04:25:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 -- 172.21.15.46:0/253183046 shutdown_connections 2024-09-10T04:25:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 --2- 172.21.15.46:0/253183046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c10b8d0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 --2- 172.21.15.46:0/253183046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef6c102ac0 0x7fef6c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 -- 172.21.15.46:0/253183046 >> 172.21.15.46:0/253183046 conn(0x7fef6c0fc520 msgr2=0x7fef6c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 -- 172.21.15.46:0/253183046 shutdown_connections 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 -- 172.21.15.46:0/253183046 wait complete. 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 Processor -- start 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.541+0000 7fef7311f640 1 -- start start 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef7311f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef6c102ac0 0x7fef6c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef7311f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef7311f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10edd0 con 0x7fef6c1033e0 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef7311f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c10ec50 con 0x7fef6c102ac0 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35780/0 (socket says 172.21.15.46:35780) 2024-09-10T04:25:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef6bfff640 1 -- 172.21.15.46:0/3823411542 learned_addr learned my addr 172.21.15.46:0/3823411542 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef70ebb640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef6c102ac0 0x7fef6c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef6bfff640 1 -- 172.21.15.46:0/3823411542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef6c102ac0 msgr2=0x7fef6c19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef6bfff640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef6c102ac0 0x7fef6c19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef6bfff640 1 -- 172.21.15.46:0/3823411542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c1977e0 con 0x7fef6c1033e0 2024-09-10T04:25:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef6bfff640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c19dba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fef58031460 tx=0x7fef580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.542+0000 7fef69ffb640 1 -- 172.21.15.46:0/3823411542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fef5803f070 con 0x7fef6c1033e0 2024-09-10T04:25:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.543+0000 7fef69ffb640 1 -- 172.21.15.46:0/3823411542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fef58004ac0 con 0x7fef6c1033e0 2024-09-10T04:25:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.543+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c197a70 con 0x7fef6c1033e0 2024-09-10T04:25:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.543+0000 7fef69ffb640 1 -- 172.21.15.46:0/3823411542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fef58035030 con 0x7fef6c1033e0 2024-09-10T04:25:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.543+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c197fa0 con 0x7fef6c1033e0 2024-09-10T04:25:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.544+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef6c06aa90 con 0x7fef6c1033e0 2024-09-10T04:25:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.544+0000 7fef69ffb640 1 -- 172.21.15.46:0/3823411542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fef58042070 con 0x7fef6c1033e0 2024-09-10T04:25:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.545+0000 7fef69ffb640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fef5c07abf0 0x7fef5c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.545+0000 7fef69ffb640 1 -- 172.21.15.46:0/3823411542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fef580be6e0 con 0x7fef6c1033e0 2024-09-10T04:25:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.545+0000 7fef70ebb640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fef5c07abf0 0x7fef5c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.546+0000 7fef70ebb640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fef5c07abf0 0x7fef5c07d0b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fef60005e10 tx=0x7fef60005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:00.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.550+0000 7fef69ffb640 1 -- 172.21.15.46:0/3823411542 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fef58089e20 con 0x7fef6c1033e0 2024-09-10T04:25:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.753+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fef6c198250 con 0x7fef5c07abf0 2024-09-10T04:25:00.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.754+0000 7fef69ffb640 1 -- 172.21.15.46:0/3823411542 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fef6c198250 con 0x7fef5c07abf0 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.756+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fef5c07abf0 msgr2=0x7fef5c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.756+0000 7fef7311f640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fef5c07abf0 0x7fef5c07d0b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fef60005e10 tx=0x7fef60005da0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 msgr2=0x7fef6c19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c19dba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fef58031460 tx=0x7fef580046c0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 shutdown_connections 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fef5c07abf0 0x7fef5c07d0b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef6c1033e0 0x7fef6c19dba0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 --2- 172.21.15.46:0/3823411542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef6c102ac0 0x7fef6c19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 >> 172.21.15.46:0/3823411542 conn(0x7fef6c0fc520 msgr2=0x7fef6c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 shutdown_connections 2024-09-10T04:25:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.757+0000 7fef7311f640 1 -- 172.21.15.46:0/3823411542 wait complete. 2024-09-10T04:25:00.768 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.842+0000 7fd190a52640 1 -- 172.21.15.46:0/3141957714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 msgr2=0x7fd18c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.842+0000 7fd190a52640 1 --2- 172.21.15.46:0/3141957714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c10b8d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd180009880 tx=0x7fd18002a530 comp rx=0 tx=0).stop 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 -- 172.21.15.46:0/3141957714 shutdown_connections 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 --2- 172.21.15.46:0/3141957714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c10b8d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 --2- 172.21.15.46:0/3141957714 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c102ac0 0x7fd18c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 -- 172.21.15.46:0/3141957714 >> 172.21.15.46:0/3141957714 conn(0x7fd18c0fc520 msgr2=0x7fd18c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 -- 172.21.15.46:0/3141957714 shutdown_connections 2024-09-10T04:25:00.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 -- 172.21.15.46:0/3141957714 wait complete. 2024-09-10T04:25:00.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 Processor -- start 2024-09-10T04:25:00.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.843+0000 7fd190a52640 1 -- start start 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd190a52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c102ac0 0x7fd18c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd190a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd190a52640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c10edf0 con 0x7fd18c1033e0 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd190a52640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c10ec70 con 0x7fd18c102ac0 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd189d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd189d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35790/0 (socket says 172.21.15.46:35790) 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd189d9b640 1 -- 172.21.15.46:0/3314263268 learned_addr learned my addr 172.21.15.46:0/3314263268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:00.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd18a59c640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c102ac0 0x7fd18c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd189d9b640 1 -- 172.21.15.46:0/3314263268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c102ac0 msgr2=0x7fd18c100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd189d9b640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c102ac0 0x7fd18c100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd189d9b640 1 -- 172.21.15.46:0/3314263268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c1011a0 con 0x7fd18c1033e0 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.844+0000 7fd189d9b640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c100c60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd180031460 tx=0x7fd1800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.845+0000 7fd17b7fe640 1 -- 172.21.15.46:0/3314263268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd18003f070 con 0x7fd18c1033e0 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.845+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c101430 con 0x7fd18c1033e0 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.845+0000 7fd17b7fe640 1 -- 172.21.15.46:0/3314263268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd180004ac0 con 0x7fd18c1033e0 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.845+0000 7fd17b7fe640 1 -- 172.21.15.46:0/3314263268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd180035030 con 0x7fd18c1033e0 2024-09-10T04:25:00.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.845+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a3de0 con 0x7fd18c1033e0 2024-09-10T04:25:00.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.846+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c06aa90 con 0x7fd18c1033e0 2024-09-10T04:25:00.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.846+0000 7fd17b7fe640 1 -- 172.21.15.46:0/3314263268 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd180042070 con 0x7fd18c1033e0 2024-09-10T04:25:00.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.847+0000 7fd17b7fe640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17407ac00 0x7fd17407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:00.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.847+0000 7fd17b7fe640 1 -- 172.21.15.46:0/3314263268 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fd1800be660 con 0x7fd18c1033e0 2024-09-10T04:25:00.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.847+0000 7fd18a59c640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17407ac00 0x7fd17407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:00.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.847+0000 7fd18a59c640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17407ac00 0x7fd17407d0c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0058c0 tx=0x7fd17c00a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:00.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:00.851+0000 7fd17b7fe640 1 -- 172.21.15.46:0/3314263268 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd180089da0 con 0x7fd18c1033e0 2024-09-10T04:25:01.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.056+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fd18c062630 con 0x7fd17407ac00 2024-09-10T04:25:01.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.058+0000 7fd17b7fe640 1 -- 172.21.15.46:0/3314263268 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fd18c062630 con 0x7fd17407ac00 2024-09-10T04:25:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17407ac00 msgr2=0x7fd17407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17407ac00 0x7fd17407d0c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0058c0 tx=0x7fd17c00a250 comp rx=0 tx=0).stop 2024-09-10T04:25:01.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 msgr2=0x7fd18c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c100c60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd180031460 tx=0x7fd1800046c0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 shutdown_connections 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd17407ac00 0x7fd17407d0c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c1033e0 0x7fd18c100c60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 --2- 172.21.15.46:0/3314263268 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd18c102ac0 0x7fd18c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 >> 172.21.15.46:0/3314263268 conn(0x7fd18c0fc520 msgr2=0x7fd18c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 shutdown_connections 2024-09-10T04:25:01.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.060+0000 7fd190a52640 1 -- 172.21.15.46:0/3314263268 wait complete. 2024-09-10T04:25:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 -- 172.21.15.46:0/2242233635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 msgr2=0x7f26f0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 --2- 172.21.15.46:0/2242233635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f0101d60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009880 tx=0x7f26e402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 -- 172.21.15.46:0/2242233635 shutdown_connections 2024-09-10T04:25:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 --2- 172.21.15.46:0/2242233635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f0101d60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 --2- 172.21.15.46:0/2242233635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0107900 0x7f26f0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 -- 172.21.15.46:0/2242233635 >> 172.21.15.46:0/2242233635 conn(0x7f26f00fd600 msgr2=0x7f26f00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:01.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 -- 172.21.15.46:0/2242233635 shutdown_connections 2024-09-10T04:25:01.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 -- 172.21.15.46:0/2242233635 wait complete. 2024-09-10T04:25:01.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.145+0000 7f26f591e640 1 Processor -- start 2024-09-10T04:25:01.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26f591e640 1 -- start start 2024-09-10T04:25:01.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26f591e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26f591e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0107900 0x7f26f019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26f591e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010edb0 con 0x7f26f0101900 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26f591e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010ec30 con 0x7f26f0107900 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26eeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26eeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35800/0 (socket says 172.21.15.46:35800) 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26eeffd640 1 -- 172.21.15.46:0/1917391215 learned_addr learned my addr 172.21.15.46:0/1917391215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26ee7fc640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0107900 0x7f26f019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26eeffd640 1 -- 172.21.15.46:0/1917391215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0107900 msgr2=0x7f26f019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26eeffd640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0107900 0x7f26f019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.146+0000 7f26eeffd640 1 -- 172.21.15.46:0/1917391215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f01a0400 con 0x7f26f0101900 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.147+0000 7f26eeffd640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f019f810 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c8a0 tx=0x7f26e000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.147+0000 7f26f491c640 1 -- 172.21.15.46:0/1917391215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26e00079b0 con 0x7f26f0101900 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.147+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f0199af0 con 0x7f26f0101900 2024-09-10T04:25:01.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.147+0000 7f26f491c640 1 -- 172.21.15.46:0/1917391215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26e0007b50 con 0x7f26f0101900 2024-09-10T04:25:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.147+0000 7f26f491c640 1 -- 172.21.15.46:0/1917391215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26e0018700 con 0x7f26f0101900 2024-09-10T04:25:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.147+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f019a050 con 0x7f26f0101900 2024-09-10T04:25:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.148+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f0103040 con 0x7f26f0101900 2024-09-10T04:25:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.149+0000 7f26f491c640 1 -- 172.21.15.46:0/1917391215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f26e0007d00 con 0x7f26f0101900 2024-09-10T04:25:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.149+0000 7f26f491c640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26d807abf0 0x7f26d807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.149+0000 7f26f491c640 1 -- 172.21.15.46:0/1917391215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f26e009def0 con 0x7f26f0101900 2024-09-10T04:25:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.149+0000 7f26ee7fc640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26d807abf0 0x7f26d807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.150+0000 7f26ee7fc640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26d807abf0 0x7f26d807d0b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26f019b1c0 tx=0x7f26e4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.154+0000 7f26f491c640 1 -- 172.21.15.46:0/1917391215 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f26e0014030 con 0x7f26f0101900 2024-09-10T04:25:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.347+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f26f019a3a0 con 0x7f26d807abf0 2024-09-10T04:25:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.356+0000 7f26f491c640 1 -- 172.21.15.46:0/1917391215 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f26f019a3a0 con 0x7f26d807abf0 2024-09-10T04:25:01.358 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:25:01.358 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (16m) 11m ago 27m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:25:01.358 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (19m) 11m ago 28m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:25:01.358 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (18m) 10m ago 27m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:25:01.358 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (15m) 11m ago 28m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:25:01.358 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (14m) 10m ago 27m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (16m) 11m ago 27m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (11m) 11m ago 22m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (11m) 11m ago 22m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (10m) 10m ago 22m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (22m) 10m ago 22m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (18m) 11m ago 29m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (17m) 10m ago 27m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (15m) 11m ago 29m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (15m) 10m ago 27m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (17m) 11m ago 28m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (17m) 10m ago 27m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (14m) 11m ago 25m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (13m) 11m ago 25m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:25:01.359 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (13m) 11m ago 24m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:25:01.360 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (12m) 10m ago 24m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:25:01.360 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (12m) 10m ago 24m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:25:01.360 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (11m) 10m ago 23m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:25:01.360 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (16m) 11m ago 27m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26d807abf0 msgr2=0x7f26d807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26d807abf0 0x7f26d807d0b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f26f019b1c0 tx=0x7f26e4031400 comp rx=0 tx=0).stop 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 msgr2=0x7f26f019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f019f810 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c8a0 tx=0x7f26e000cd60 comp rx=0 tx=0).stop 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 shutdown_connections 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26d807abf0 0x7f26d807d0b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26f0107900 0x7f26f019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 --2- 172.21.15.46:0/1917391215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26f0101900 0x7f26f019f810 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 >> 172.21.15.46:0/1917391215 conn(0x7f26f00fd600 msgr2=0x7f26f0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 shutdown_connections 2024-09-10T04:25:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.359+0000 7f26f591e640 1 -- 172.21.15.46:0/1917391215 wait complete. 2024-09-10T04:25:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 -- 172.21.15.46:0/1472658276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 msgr2=0x7f1ed8100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/1472658276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8100630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009880 tx=0x7f1ec802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 -- 172.21.15.46:0/1472658276 shutdown_connections 2024-09-10T04:25:01.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/1472658276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed8100c00 0x7f1ed8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/1472658276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8100630 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 -- 172.21.15.46:0/1472658276 >> 172.21.15.46:0/1472658276 conn(0x7f1ed80fbc90 msgr2=0x7f1ed80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 -- 172.21.15.46:0/1472658276 shutdown_connections 2024-09-10T04:25:01.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.442+0000 7f1edd6ea640 1 -- 172.21.15.46:0/1472658276 wait complete. 2024-09-10T04:25:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.443+0000 7f1edd6ea640 1 Processor -- start 2024-09-10T04:25:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.443+0000 7f1edd6ea640 1 -- start start 2024-09-10T04:25:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.443+0000 7f1edd6ea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.443+0000 7f1edd6ea640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed8100c00 0x7f1ed8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.443+0000 7f1edd6ea640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed810d4d0 con 0x7f1ed8100250 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.443+0000 7f1edd6ea640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed810d350 con 0x7f1ed8100c00 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35818/0 (socket says 172.21.15.46:35818) 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed6ffd640 1 -- 172.21.15.46:0/625247073 learned_addr learned my addr 172.21.15.46:0/625247073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ece7fc640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed8100c00 0x7f1ed8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed6ffd640 1 -- 172.21.15.46:0/625247073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed8100c00 msgr2=0x7f1ed8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed6ffd640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed8100c00 0x7f1ed8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed6ffd640 1 -- 172.21.15.46:0/625247073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed8199400 con 0x7f1ed8100250 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed6ffd640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8198870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8031460 tx=0x7f1ec80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed4ff9640 1 -- 172.21.15.46:0/625247073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1ec803f070 con 0x7f1ed8100250 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed4ff9640 1 -- 172.21.15.46:0/625247073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1ec80035f0 con 0x7f1ed8100250 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed8199690 con 0x7f1ed8100250 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1ed4ff9640 1 -- 172.21.15.46:0/625247073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1ec8035030 con 0x7f1ed8100250 2024-09-10T04:25:01.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.444+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed81a1c40 con 0x7f1ed8100250 2024-09-10T04:25:01.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.445+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed8101730 con 0x7f1ed8100250 2024-09-10T04:25:01.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.446+0000 7f1ed4ff9640 1 -- 172.21.15.46:0/625247073 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1ec8042070 con 0x7f1ed8100250 2024-09-10T04:25:01.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.447+0000 7f1ed4ff9640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ec407abf0 0x7f1ec407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.447+0000 7f1ed4ff9640 1 -- 172.21.15.46:0/625247073 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f1ec80be720 con 0x7f1ed8100250 2024-09-10T04:25:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.447+0000 7f1ece7fc640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ec407abf0 0x7f1ec407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.447+0000 7f1ece7fc640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ec407abf0 0x7f1ec407d0b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0005fd0 tx=0x7f1ec0005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:01.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.452+0000 7f1ed4ff9640 1 -- 172.21.15.46:0/625247073 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1ec8089dc0 con 0x7f1ed8100250 2024-09-10T04:25:01.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.711+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1ed80fce20 con 0x7f1ed8100250 2024-09-10T04:25:01.714 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:01 smithi046 ceph-mon[109506]: pgmap v580: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:01.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.712+0000 7f1ed4ff9640 1 -- 172.21.15.46:0/625247073 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f1ec808ec70 con 0x7f1ed8100250 2024-09-10T04:25:01.714 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:25:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:25:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:25:01.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:25:01.715 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:25:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:25:01.716 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:25:01.716 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:25:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ec407abf0 msgr2=0x7f1ec407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ec407abf0 0x7f1ec407d0b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0005fd0 tx=0x7f1ec0005850 comp rx=0 tx=0).stop 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 msgr2=0x7f1ed8198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8198870 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8031460 tx=0x7f1ec80045e0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 shutdown_connections 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1ec407abf0 0x7f1ec407d0b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ed8100c00 0x7f1ed8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 --2- 172.21.15.46:0/625247073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ed8100250 0x7f1ed8198870 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 >> 172.21.15.46:0/625247073 conn(0x7f1ed80fbc90 msgr2=0x7f1ed8103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 shutdown_connections 2024-09-10T04:25:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.714+0000 7f1edd6ea640 1 -- 172.21.15.46:0/625247073 wait complete. 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 -- 172.21.15.46:0/2027127092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 msgr2=0x7efec010b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 --2- 172.21.15.46:0/2027127092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec010b8c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009880 tx=0x7efeb402a530 comp rx=0 tx=0).stop 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 -- 172.21.15.46:0/2027127092 shutdown_connections 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 --2- 172.21.15.46:0/2027127092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec010b8c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 --2- 172.21.15.46:0/2027127092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0102ab0 0x7efec0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 -- 172.21.15.46:0/2027127092 >> 172.21.15.46:0/2027127092 conn(0x7efec00fc550 msgr2=0x7efec00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 -- 172.21.15.46:0/2027127092 shutdown_connections 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.798+0000 7efec52e3640 1 -- 172.21.15.46:0/2027127092 wait complete. 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.799+0000 7efec52e3640 1 Processor -- start 2024-09-10T04:25:01.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.799+0000 7efec52e3640 1 -- start start 2024-09-10T04:25:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.799+0000 7efec52e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0102ab0 0x7efec019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.799+0000 7efec52e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.799+0000 7efec52e3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efec010ede0 con 0x7efec01033d0 2024-09-10T04:25:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.799+0000 7efec52e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efec010ec60 con 0x7efec0102ab0 2024-09-10T04:25:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebe7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebe7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35836/0 (socket says 172.21.15.46:35836) 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebe7fc640 1 -- 172.21.15.46:0/459829513 learned_addr learned my addr 172.21.15.46:0/459829513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebeffd640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0102ab0 0x7efec019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebe7fc640 1 -- 172.21.15.46:0/459829513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0102ab0 msgr2=0x7efec019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebe7fc640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0102ab0 0x7efec019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebe7fc640 1 -- 172.21.15.46:0/459829513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efec0197810 con 0x7efec01033d0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efebe7fc640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec019dbd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efeb4031460 tx=0x7efeb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efea7fff640 1 -- 172.21.15.46:0/459829513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efeb403f070 con 0x7efec01033d0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efea7fff640 1 -- 172.21.15.46:0/459829513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efeb4004ac0 con 0x7efec01033d0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efec0197aa0 con 0x7efec01033d0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efea7fff640 1 -- 172.21.15.46:0/459829513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efeb4035030 con 0x7efec01033d0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.800+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efec0197fd0 con 0x7efec01033d0 2024-09-10T04:25:01.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.802+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efec006a580 con 0x7efec01033d0 2024-09-10T04:25:01.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.802+0000 7efea7fff640 1 -- 172.21.15.46:0/459829513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efeb4042070 con 0x7efec01033d0 2024-09-10T04:25:01.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.803+0000 7efea7fff640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efea807abf0 0x7efea807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:01.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.803+0000 7efea7fff640 1 -- 172.21.15.46:0/459829513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7efeb40be6d0 con 0x7efec01033d0 2024-09-10T04:25:01.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.803+0000 7efebeffd640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efea807abf0 0x7efea807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:01.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.803+0000 7efebeffd640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efea807abf0 0x7efea807d0b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efeb00058c0 tx=0x7efeb000a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:01.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:01.807+0000 7efea7fff640 1 -- 172.21.15.46:0/459829513 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efeb4089d70 con 0x7efec01033d0 2024-09-10T04:25:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.032+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efec0105f50 con 0x7efec01033d0 2024-09-10T04:25:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.032+0000 7efea7fff640 1 -- 172.21.15.46:0/459829513 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7efeb408ec20 con 0x7efec01033d0 2024-09-10T04:25:02.034 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:25:02.034 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:25:02.035 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:25:02.036 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:02.037 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:02.038 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:02.038 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:02.038 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:25:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efea807abf0 msgr2=0x7efea807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efea807abf0 0x7efea807d0b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efeb00058c0 tx=0x7efeb000a250 comp rx=0 tx=0).stop 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 msgr2=0x7efec019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec019dbd0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efeb4031460 tx=0x7efeb40046c0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 shutdown_connections 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efea807abf0 0x7efea807d0b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efec01033d0 0x7efec019dbd0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 --2- 172.21.15.46:0/459829513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efec0102ab0 0x7efec019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 >> 172.21.15.46:0/459829513 conn(0x7efec00fc550 msgr2=0x7efec0108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 shutdown_connections 2024-09-10T04:25:02.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.035+0000 7efec52e3640 1 -- 172.21.15.46:0/459829513 wait complete. 2024-09-10T04:25:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:01 smithi070 ceph-mon[79774]: pgmap v580: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 -- 172.21.15.46:0/2515121499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf0102ac0 msgr2=0x7fbbf0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 --2- 172.21.15.46:0/2515121499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf0102ac0 0x7fbbf0102ea0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4009880 tx=0x7fbbe402a530 comp rx=0 tx=0).stop 2024-09-10T04:25:02.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 -- 172.21.15.46:0/2515121499 shutdown_connections 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 --2- 172.21.15.46:0/2515121499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf01033e0 0x7fbbf010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 --2- 172.21.15.46:0/2515121499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf0102ac0 0x7fbbf0102ea0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 -- 172.21.15.46:0/2515121499 >> 172.21.15.46:0/2515121499 conn(0x7fbbf00fc520 msgr2=0x7fbbf00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 -- 172.21.15.46:0/2515121499 shutdown_connections 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.120+0000 7fbbf700e640 1 -- 172.21.15.46:0/2515121499 wait complete. 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbf700e640 1 Processor -- start 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbf700e640 1 -- start start 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbf700e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf0102ac0 0x7fbbf019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbf700e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf01033e0 0x7fbbf019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbf700e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf010edf0 con 0x7fbbf01033e0 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbf700e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf010ec70 con 0x7fbbf0102ac0 2024-09-10T04:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbeffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf01033e0 0x7fbbf019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbeffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf01033e0 0x7fbbf019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35864/0 (socket says 172.21.15.46:35864) 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbeffff640 1 -- 172.21.15.46:0/692161223 learned_addr learned my addr 172.21.15.46:0/692161223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.121+0000 7fbbf4daa640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf0102ac0 0x7fbbf019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbeffff640 1 -- 172.21.15.46:0/692161223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf0102ac0 msgr2=0x7fbbf019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbeffff640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf0102ac0 0x7fbbf019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbeffff640 1 -- 172.21.15.46:0/692161223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf01977c0 con 0x7fbbf01033e0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbf4daa640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf0102ac0 0x7fbbf019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbeffff640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf01033e0 0x7fbbf019db80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00d6d0 tx=0x7fbbdc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbedffb640 1 -- 172.21.15.46:0/692161223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbbdc017070 con 0x7fbbf01033e0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf0197ab0 con 0x7fbbf01033e0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbedffb640 1 -- 172.21.15.46:0/692161223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbbdc0054e0 con 0x7fbbf01033e0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbedffb640 1 -- 172.21.15.46:0/692161223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbbdc004ae0 con 0x7fbbf01033e0 2024-09-10T04:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.122+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf0198040 con 0x7fbbf01033e0 2024-09-10T04:25:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.123+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf006aa90 con 0x7fbbf01033e0 2024-09-10T04:25:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.124+0000 7fbbedffb640 1 -- 172.21.15.46:0/692161223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbbdc004de0 con 0x7fbbf01033e0 2024-09-10T04:25:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.124+0000 7fbbedffb640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbbe007abf0 0x7fbbe007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.124+0000 7fbbedffb640 1 -- 172.21.15.46:0/692161223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fbbdc09d870 con 0x7fbbf01033e0 2024-09-10T04:25:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.124+0000 7fbbf4daa640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbbe007abf0 0x7fbbe007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:02.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.125+0000 7fbbf4daa640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbbe007abf0 0x7fbbe007d0b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4000c00 tx=0x7fbbe4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.129+0000 7fbbedffb640 1 -- 172.21.15.46:0/692161223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbbdc068fb0 con 0x7fbbf01033e0 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.333+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbbf01982f0 con 0x7fbbe007abf0 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.334+0000 7fbbedffb640 1 -- 172.21.15.46:0/692161223 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fbbf01982f0 con 0x7fbbe007abf0 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:25:02.337 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:25:02.338 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:25:02.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbbe007abf0 msgr2=0x7fbbe007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbbe007abf0 0x7fbbe007d0b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4000c00 tx=0x7fbbe4031400 comp rx=0 tx=0).stop 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf01033e0 msgr2=0x7fbbf019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf01033e0 0x7fbbf019db80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00d6d0 tx=0x7fbbdc00db90 comp rx=0 tx=0).stop 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 shutdown_connections 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbbe007abf0 0x7fbbe007d0b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf01033e0 0x7fbbf019db80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 --2- 172.21.15.46:0/692161223 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbbf0102ac0 0x7fbbf019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 >> 172.21.15.46:0/692161223 conn(0x7fbbf00fc520 msgr2=0x7fbbf010af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 shutdown_connections 2024-09-10T04:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.337+0000 7fbbf700e640 1 -- 172.21.15.46:0/692161223 wait complete. 2024-09-10T04:25:02.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 -- 172.21.15.46:0/2070918576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 msgr2=0x7fc6a8106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/2070918576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a8106be0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc698009880 tx=0x7fc69802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 -- 172.21.15.46:0/2070918576 shutdown_connections 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/2070918576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6a8100800 0x7fc6a8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/2070918576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a8106be0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 -- 172.21.15.46:0/2070918576 >> 172.21.15.46:0/2070918576 conn(0x7fc6a80fc520 msgr2=0x7fc6a80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 -- 172.21.15.46:0/2070918576 shutdown_connections 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.425+0000 7fc6ae34a640 1 -- 172.21.15.46:0/2070918576 wait complete. 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6ae34a640 1 Processor -- start 2024-09-10T04:25:02.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6ae34a640 1 -- start start 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6ae34a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6a8100800 0x7fc6a819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6ae34a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6ae34a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a810dcd0 con 0x7fc6a8106800 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6ae34a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a810db50 con 0x7fc6a8100800 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6a77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35892/0 (socket says 172.21.15.46:35892) 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.426+0000 7fc6a77fe640 1 -- 172.21.15.46:0/745066805 learned_addr learned my addr 172.21.15.46:0/745066805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a7fff640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6a8100800 0x7fc6a819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a77fe640 1 -- 172.21.15.46:0/745066805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6a8100800 msgr2=0x7fc6a819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a77fe640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6a8100800 0x7fc6a819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a77fe640 1 -- 172.21.15.46:0/745066805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81977c0 con 0x7fc6a8106800 2024-09-10T04:25:02.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a77fe640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a819db80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00d6f0 tx=0x7fc69c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a57fa640 1 -- 172.21.15.46:0/745066805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc69c017070 con 0x7fc6a8106800 2024-09-10T04:25:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a57fa640 1 -- 172.21.15.46:0/745066805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc69c00de00 con 0x7fc6a8106800 2024-09-10T04:25:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a8197ab0 con 0x7fc6a8106800 2024-09-10T04:25:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6a57fa640 1 -- 172.21.15.46:0/745066805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc69c004bb0 con 0x7fc6a8106800 2024-09-10T04:25:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.427+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a8198040 con 0x7fc6a8106800 2024-09-10T04:25:02.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.428+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a8101f40 con 0x7fc6a8106800 2024-09-10T04:25:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.429+0000 7fc6a57fa640 1 -- 172.21.15.46:0/745066805 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc69c004e60 con 0x7fc6a8106800 2024-09-10T04:25:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.429+0000 7fc6a57fa640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc69007abf0 0x7fc69007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.430+0000 7fc6a57fa640 1 -- 172.21.15.46:0/745066805 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc69c09d970 con 0x7fc6a8106800 2024-09-10T04:25:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.430+0000 7fc6a7fff640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc69007abf0 0x7fc69007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:02.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.430+0000 7fc6a7fff640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc69007abf0 0x7fc69007d0b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6980096c0 tx=0x7fc698031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:02.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.434+0000 7fc6a57fa640 1 -- 172.21.15.46:0/745066805 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc69c069010 con 0x7fc6a8106800 2024-09-10T04:25:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.697+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fc6a8109d80 con 0x7fc6a8106800 2024-09-10T04:25:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.698+0000 7fc6a57fa640 1 -- 172.21.15.46:0/745066805 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fc69c06dec0 con 0x7fc6a8106800 2024-09-10T04:25:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:25:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:25:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:25:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:25:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:25:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:25:02.700 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:25:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc69007abf0 msgr2=0x7fc69007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc69007abf0 0x7fc69007d0b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc6980096c0 tx=0x7fc698031400 comp rx=0 tx=0).stop 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 msgr2=0x7fc6a819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a819db80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00d6f0 tx=0x7fc69c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 shutdown_connections 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc69007abf0 0x7fc69007d0b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6a8106800 0x7fc6a819db80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 --2- 172.21.15.46:0/745066805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6a8100800 0x7fc6a819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 >> 172.21.15.46:0/745066805 conn(0x7fc6a80fc520 msgr2=0x7fc6a806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 shutdown_connections 2024-09-10T04:25:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:02.700+0000 7fc6ae34a640 1 -- 172.21.15.46:0/745066805 wait complete. 2024-09-10T04:25:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:02 smithi070 ceph-mon[79774]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:02 smithi070 ceph-mon[79774]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:02 smithi070 ceph-mon[79774]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:02 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/625247073' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:25:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:02 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/459829513' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:25:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:02 smithi046 ceph-mon[109506]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:03.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:02 smithi046 ceph-mon[109506]: from='client.34930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:03.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:02 smithi046 ceph-mon[109506]: from='client.34934 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:03.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:02 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/625247073' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:25:03.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:02 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/459829513' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:25:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:03 smithi046 ceph-mon[109506]: pgmap v581: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:03 smithi046 ceph-mon[109506]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:03 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/745066805' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:25:04.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:03 smithi070 ceph-mon[79774]: pgmap v581: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:04.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:03 smithi070 ceph-mon[79774]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:04.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:03 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/745066805' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:25:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:05 smithi070 ceph-mon[79774]: pgmap v582: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:06.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:05 smithi046 ceph-mon[109506]: pgmap v582: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:08.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:07 smithi070 ceph-mon[79774]: pgmap v583: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:08.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:07 smithi046 ceph-mon[109506]: pgmap v583: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:09 smithi070 ceph-mon[79774]: pgmap v584: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:10.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:09 smithi046 ceph-mon[109506]: pgmap v584: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:12.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:11 smithi070 ceph-mon[79774]: pgmap v585: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:11 smithi046 ceph-mon[109506]: pgmap v585: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:13 smithi046 ceph-mon[109506]: pgmap v586: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:13 smithi070 ceph-mon[79774]: pgmap v586: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:14 smithi070 ceph-mon[79774]: pgmap v587: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:14 smithi046 ceph-mon[109506]: pgmap v587: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:17.563 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:17 smithi046 ceph-mon[109506]: pgmap v588: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:17.788 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:17 smithi070 ceph-mon[79774]: pgmap v588: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:19 smithi070 ceph-mon[79774]: pgmap v589: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:19 smithi046 ceph-mon[109506]: pgmap v589: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:21 smithi070 ceph-mon[79774]: pgmap v590: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:21 smithi046 ceph-mon[109506]: pgmap v590: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:23 smithi046 ceph-mon[109506]: pgmap v591: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:23 smithi070 ceph-mon[79774]: pgmap v591: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:25 smithi070 ceph-mon[79774]: pgmap v592: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:25 smithi046 ceph-mon[109506]: pgmap v592: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:27.717 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:27 smithi046 ceph-mon[109506]: pgmap v593: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:27 smithi070 ceph-mon[79774]: pgmap v593: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:29 smithi046 ceph-mon[109506]: pgmap v594: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:29 smithi070 ceph-mon[79774]: pgmap v594: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:31 smithi070 ceph-mon[79774]: pgmap v595: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:31 smithi046 ceph-mon[109506]: pgmap v595: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 -- 172.21.15.46:0/3874492453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 msgr2=0x7fa1a0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/3874492453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a0101d60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa194009880 tx=0x7fa19402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 -- 172.21.15.46:0/3874492453 shutdown_connections 2024-09-10T04:25:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/3874492453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a0101d60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:32.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/3874492453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1a0107900 0x7fa1a0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 -- 172.21.15.46:0/3874492453 >> 172.21.15.46:0/3874492453 conn(0x7fa1a00fd600 msgr2=0x7fa1a00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 -- 172.21.15.46:0/3874492453 shutdown_connections 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.799+0000 7fa1a4a28640 1 -- 172.21.15.46:0/3874492453 wait complete. 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.800+0000 7fa1a4a28640 1 Processor -- start 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.800+0000 7fa1a4a28640 1 -- start start 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.800+0000 7fa1a4a28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.800+0000 7fa1a4a28640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1a0107900 0x7fa1a019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.800+0000 7fa1a4a28640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a010edd0 con 0x7fa1a0101900 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.800+0000 7fa1a4a28640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a010ec50 con 0x7fa1a0107900 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41828/0 (socket says 172.21.15.46:41828) 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19e59c640 1 -- 172.21.15.46:0/432942321 learned_addr learned my addr 172.21.15.46:0/432942321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:32.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19dd9b640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1a0107900 0x7fa1a019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:32.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19e59c640 1 -- 172.21.15.46:0/432942321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1a0107900 msgr2=0x7fa1a019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:32.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19e59c640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1a0107900 0x7fa1a019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:32.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19e59c640 1 -- 172.21.15.46:0/432942321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a0197800 con 0x7fa1a0101900 2024-09-10T04:25:32.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19dd9b640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1a0107900 0x7fa1a019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:25:32.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.801+0000 7fa19e59c640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a019d680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e970 tx=0x7fa19000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.802+0000 7fa18f7fe640 1 -- 172.21.15.46:0/432942321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa19000cac0 con 0x7fa1a0101900 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.802+0000 7fa18f7fe640 1 -- 172.21.15.46:0/432942321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1900054e0 con 0x7fa1a0101900 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.802+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a0197af0 con 0x7fa1a0101900 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.802+0000 7fa18f7fe640 1 -- 172.21.15.46:0/432942321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa190010430 con 0x7fa1a0101900 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.802+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a0198050 con 0x7fa1a0101900 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.803+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a0103040 con 0x7fa1a0101900 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.804+0000 7fa18f7fe640 1 -- 172.21.15.46:0/432942321 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa190004f00 con 0x7fa1a0101900 2024-09-10T04:25:32.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.804+0000 7fa18f7fe640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa18807abf0 0x7fa18807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:32.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.804+0000 7fa18f7fe640 1 -- 172.21.15.46:0/432942321 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fa19009e3f0 con 0x7fa1a0101900 2024-09-10T04:25:32.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.804+0000 7fa19dd9b640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa18807abf0 0x7fa18807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:32.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.805+0000 7fa19dd9b640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa18807abf0 0x7fa18807d0b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0198b50 tx=0x7fa194031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:32.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:32.809+0000 7fa18f7fe640 1 -- 172.21.15.46:0/432942321 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa190069b30 con 0x7fa1a0101900 2024-09-10T04:25:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.017+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa1a0198360 con 0x7fa18807abf0 2024-09-10T04:25:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.018+0000 7fa18f7fe640 1 -- 172.21.15.46:0/432942321 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fa1a0198360 con 0x7fa18807abf0 2024-09-10T04:25:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa18807abf0 msgr2=0x7fa18807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa18807abf0 0x7fa18807d0b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0198b50 tx=0x7fa194031400 comp rx=0 tx=0).stop 2024-09-10T04:25:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 msgr2=0x7fa1a019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a019d680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e970 tx=0x7fa19000ed70 comp rx=0 tx=0).stop 2024-09-10T04:25:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 shutdown_connections 2024-09-10T04:25:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa18807abf0 0x7fa18807d0b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1a0107900 0x7fa1a019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 --2- 172.21.15.46:0/432942321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a0101900 0x7fa1a019d680 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 >> 172.21.15.46:0/432942321 conn(0x7fa1a00fd600 msgr2=0x7fa1a00fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 shutdown_connections 2024-09-10T04:25:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.020+0000 7fa1a4a28640 1 -- 172.21.15.46:0/432942321 wait complete. 2024-09-10T04:25:33.030 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:25:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 -- 172.21.15.46:0/1340393093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24101900 msgr2=0x7f6e24101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/1340393093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24101900 0x7f6e24101d60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009880 tx=0x7f6e1802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 -- 172.21.15.46:0/1340393093 shutdown_connections 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/1340393093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24101900 0x7f6e24101d60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/1340393093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e24107900 0x7f6e24107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 -- 172.21.15.46:0/1340393093 >> 172.21.15.46:0/1340393093 conn(0x7f6e240fd600 msgr2=0x7f6e240ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 -- 172.21.15.46:0/1340393093 shutdown_connections 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.107+0000 7f6e2a56f640 1 -- 172.21.15.46:0/1340393093 wait complete. 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e2a56f640 1 Processor -- start 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e2a56f640 1 -- start start 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e2a56f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e24101900 0x7f6e2419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e2a56f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24107900 0x7f6e2419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e2a56f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410edd0 con 0x7f6e24107900 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e2a56f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2410ec50 con 0x7f6e24101900 2024-09-10T04:25:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24107900 0x7f6e2419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e237fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24107900 0x7f6e2419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41850/0 (socket says 172.21.15.46:41850) 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e237fe640 1 -- 172.21.15.46:0/2444756264 learned_addr learned my addr 172.21.15.46:0/2444756264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.108+0000 7f6e23fff640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e24101900 0x7f6e2419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e237fe640 1 -- 172.21.15.46:0/2444756264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e24101900 msgr2=0x7f6e2419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e237fe640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e24101900 0x7f6e2419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e237fe640 1 -- 172.21.15.46:0/2444756264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e241977a0 con 0x7f6e24107900 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e23fff640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e24101900 0x7f6e2419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e237fe640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24107900 0x7f6e2419db60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6e18031460 tx=0x7f6e180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e217fa640 1 -- 172.21.15.46:0/2444756264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6e1803f070 con 0x7f6e24107900 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e217fa640 1 -- 172.21.15.46:0/2444756264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6e180035f0 con 0x7f6e24107900 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e24197a30 con 0x7f6e24107900 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e217fa640 1 -- 172.21.15.46:0/2444756264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6e18035030 con 0x7f6e24107900 2024-09-10T04:25:33.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.109+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e24197f60 con 0x7f6e24107900 2024-09-10T04:25:33.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.110+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e24103040 con 0x7f6e24107900 2024-09-10T04:25:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.111+0000 7f6e217fa640 1 -- 172.21.15.46:0/2444756264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6e18042070 con 0x7f6e24107900 2024-09-10T04:25:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.111+0000 7f6e217fa640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6e0c07ac40 0x7f6e0c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.111+0000 7f6e217fa640 1 -- 172.21.15.46:0/2444756264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f6e180be5a0 con 0x7f6e24107900 2024-09-10T04:25:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.111+0000 7f6e23fff640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6e0c07ac40 0x7f6e0c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.112+0000 7f6e23fff640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6e0c07ac40 0x7f6e0c07d100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6e140058c0 tx=0x7f6e14005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:33.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.116+0000 7f6e217fa640 1 -- 172.21.15.46:0/2444756264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6e18089d60 con 0x7f6e24107900 2024-09-10T04:25:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.323+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6e24198210 con 0x7f6e0c07ac40 2024-09-10T04:25:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.325+0000 7f6e217fa640 1 -- 172.21.15.46:0/2444756264 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f6e24198210 con 0x7f6e0c07ac40 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6e0c07ac40 msgr2=0x7f6e0c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6e0c07ac40 0x7f6e0c07d100 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6e140058c0 tx=0x7f6e14005850 comp rx=0 tx=0).stop 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24107900 msgr2=0x7f6e2419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24107900 0x7f6e2419db60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6e18031460 tx=0x7f6e180045e0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 shutdown_connections 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6e0c07ac40 0x7f6e0c07d100 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e24107900 0x7f6e2419db60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 --2- 172.21.15.46:0/2444756264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e24101900 0x7f6e2419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 >> 172.21.15.46:0/2444756264 conn(0x7f6e240fd600 msgr2=0x7f6e240fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:33.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 shutdown_connections 2024-09-10T04:25:33.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.327+0000 7f6e2a56f640 1 -- 172.21.15.46:0/2444756264 wait complete. 2024-09-10T04:25:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 -- 172.21.15.46:0/107306872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768106800 msgr2=0x7f0768106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 --2- 172.21.15.46:0/107306872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768106800 0x7f0768106be0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0758009880 tx=0x7f075802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 -- 172.21.15.46:0/107306872 shutdown_connections 2024-09-10T04:25:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 --2- 172.21.15.46:0/107306872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0768100800 0x7f0768100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 --2- 172.21.15.46:0/107306872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768106800 0x7f0768106be0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 -- 172.21.15.46:0/107306872 >> 172.21.15.46:0/107306872 conn(0x7f07680fc520 msgr2=0x7f07680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 -- 172.21.15.46:0/107306872 shutdown_connections 2024-09-10T04:25:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.413+0000 7f076de81640 1 -- 172.21.15.46:0/107306872 wait complete. 2024-09-10T04:25:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f076de81640 1 Processor -- start 2024-09-10T04:25:33.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f076de81640 1 -- start start 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f076de81640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768100800 0x7f0768198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f076de81640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0768106800 0x7f0768198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f076de81640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f076810dd30 con 0x7f0768100800 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f076de81640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f076810dbb0 con 0x7f0768106800 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f07677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768100800 0x7f0768198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f07677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768100800 0x7f0768198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41874/0 (socket says 172.21.15.46:41874) 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f07677fe640 1 -- 172.21.15.46:0/123843795 learned_addr learned my addr 172.21.15.46:0/123843795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f07677fe640 1 -- 172.21.15.46:0/123843795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0768106800 msgr2=0x7f0768198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f0766ffd640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0768106800 0x7f0768198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f07677fe640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0768106800 0x7f0768198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.414+0000 7f07677fe640 1 -- 172.21.15.46:0/123843795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07681993c0 con 0x7f0768100800 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.415+0000 7f0766ffd640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0768106800 0x7f0768198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:25:33.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.415+0000 7f07677fe640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768100800 0x7f0768198880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0758031460 tx=0x7f07580048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.415+0000 7f0764ff9640 1 -- 172.21.15.46:0/123843795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f075803f070 con 0x7f0768100800 2024-09-10T04:25:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.415+0000 7f0764ff9640 1 -- 172.21.15.46:0/123843795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0758004cf0 con 0x7f0768100800 2024-09-10T04:25:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.415+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0768199650 con 0x7f0768100800 2024-09-10T04:25:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.415+0000 7f0764ff9640 1 -- 172.21.15.46:0/123843795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0758035030 con 0x7f0768100800 2024-09-10T04:25:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.415+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07681a1c00 con 0x7f0768100800 2024-09-10T04:25:33.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.416+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0768101f40 con 0x7f0768100800 2024-09-10T04:25:33.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.417+0000 7f0764ff9640 1 -- 172.21.15.46:0/123843795 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0758042070 con 0x7f0768100800 2024-09-10T04:25:33.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.417+0000 7f0764ff9640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f075007ac40 0x7f075007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.417+0000 7f0764ff9640 1 -- 172.21.15.46:0/123843795 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f07580be6d0 con 0x7f0768100800 2024-09-10T04:25:33.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.417+0000 7f0766ffd640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f075007ac40 0x7f075007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.418+0000 7f0766ffd640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f075007ac40 0x7f075007d100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f075c006fd0 tx=0x7f075c008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:33.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.422+0000 7f0764ff9640 1 -- 172.21.15.46:0/123843795 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0758089e90 con 0x7f0768100800 2024-09-10T04:25:33.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.617+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0768062630 con 0x7f075007ac40 2024-09-10T04:25:33.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.626+0000 7f0764ff9640 1 -- 172.21.15.46:0/123843795 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f0768062630 con 0x7f075007ac40 2024-09-10T04:25:33.627 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:33 smithi046 ceph-mon[109506]: pgmap v596: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (17m) 11m ago 28m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (19m) 11m ago 28m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (19m) 11m ago 27m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (15m) 11m ago 28m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (15m) 11m ago 27m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (16m) 11m ago 28m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (11m) 11m ago 23m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (11m) 11m ago 23m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (11m) 11m ago 23m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (23m) 11m ago 23m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (19m) 11m ago 29m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:25:33.628 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (18m) 11m ago 27m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (16m) 11m ago 30m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (15m) 11m ago 27m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (17m) 11m ago 28m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (17m) 11m ago 27m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (15m) 11m ago 26m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (14m) 11m ago 25m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (13m) 11m ago 25m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (13m) 11m ago 25m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (12m) 11m ago 24m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (12m) 11m ago 24m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:25:33.629 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (17m) 11m ago 28m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:25:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f075007ac40 msgr2=0x7f075007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f075007ac40 0x7f075007d100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f075c006fd0 tx=0x7f075c008040 comp rx=0 tx=0).stop 2024-09-10T04:25:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768100800 msgr2=0x7f0768198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768100800 0x7f0768198880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0758031460 tx=0x7f07580048f0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 shutdown_connections 2024-09-10T04:25:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f075007ac40 0x7f075007d100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0768106800 0x7f0768198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 --2- 172.21.15.46:0/123843795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0768100800 0x7f0768198880 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 >> 172.21.15.46:0/123843795 conn(0x7f07680fc520 msgr2=0x7f07680fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 shutdown_connections 2024-09-10T04:25:33.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.630+0000 7f076de81640 1 -- 172.21.15.46:0/123843795 wait complete. 2024-09-10T04:25:33.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 -- 172.21.15.46:0/2006463092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 msgr2=0x7ff560100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 --2- 172.21.15.46:0/2006463092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff560100c50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff54c009880 tx=0x7ff54c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 -- 172.21.15.46:0/2006463092 shutdown_connections 2024-09-10T04:25:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 --2- 172.21.15.46:0/2006463092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff560100c50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 --2- 172.21.15.46:0/2006463092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5601067f0 0x7ff560106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 -- 172.21.15.46:0/2006463092 >> 172.21.15.46:0/2006463092 conn(0x7ff5600fc550 msgr2=0x7ff5600fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 -- 172.21.15.46:0/2006463092 shutdown_connections 2024-09-10T04:25:33.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 -- 172.21.15.46:0/2006463092 wait complete. 2024-09-10T04:25:33.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 Processor -- start 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.715+0000 7ff566c8a640 1 -- start start 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff566c8a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff56019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff566c8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5601067f0 0x7ff56019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff566c8a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010dca0 con 0x7ff5601007f0 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff566c8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff56010db20 con 0x7ff5601067f0 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff564a26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff56019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff564a26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff56019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41884/0 (socket says 172.21.15.46:41884) 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff564a26640 1 -- 172.21.15.46:0/3928614043 learned_addr learned my addr 172.21.15.46:0/3928614043 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff55ffff640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5601067f0 0x7ff56019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff564a26640 1 -- 172.21.15.46:0/3928614043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5601067f0 msgr2=0x7ff56019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff564a26640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5601067f0 0x7ff56019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff564a26640 1 -- 172.21.15.46:0/3928614043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff560197780 con 0x7ff5601007f0 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff564a26640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff56019d600 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff55400d8d0 tx=0x7ff55400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff55dffb640 1 -- 172.21.15.46:0/3928614043 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff554017070 con 0x7ff5601007f0 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff55dffb640 1 -- 172.21.15.46:0/3928614043 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff554010460 con 0x7ff5601007f0 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff560197a70 con 0x7ff5601007f0 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.716+0000 7ff55dffb640 1 -- 172.21.15.46:0/3928614043 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff55400f5f0 con 0x7ff5601007f0 2024-09-10T04:25:33.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.717+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff560198000 con 0x7ff5601007f0 2024-09-10T04:25:33.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.718+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff560101f30 con 0x7ff5601007f0 2024-09-10T04:25:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.718+0000 7ff55dffb640 1 -- 172.21.15.46:0/3928614043 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff554009a30 con 0x7ff5601007f0 2024-09-10T04:25:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.719+0000 7ff55dffb640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff55007abf0 0x7ff55007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.719+0000 7ff55dffb640 1 -- 172.21.15.46:0/3928614043 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ff55409dc00 con 0x7ff5601007f0 2024-09-10T04:25:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.719+0000 7ff55ffff640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff55007abf0 0x7ff55007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:33.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.720+0000 7ff55ffff640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff55007abf0 0x7ff55007d0b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff54c031460 tx=0x7ff54c009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:33.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.723+0000 7ff55dffb640 1 -- 172.21.15.46:0/3928614043 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff5540692a0 con 0x7ff5601007f0 2024-09-10T04:25:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:33 smithi070 ceph-mon[79774]: pgmap v596: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:33.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.990+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff560109d90 con 0x7ff5601007f0 2024-09-10T04:25:33.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.990+0000 7ff55dffb640 1 -- 172.21.15.46:0/3928614043 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7ff55406e150 con 0x7ff5601007f0 2024-09-10T04:25:33.991 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:25:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:25:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:25:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:33.991 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:25:33.992 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:25:33.993 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff55007abf0 msgr2=0x7ff55007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff55007abf0 0x7ff55007d0b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff54c031460 tx=0x7ff54c009560 comp rx=0 tx=0).stop 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 msgr2=0x7ff56019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff56019d600 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff55400d8d0 tx=0x7ff55400dd90 comp rx=0 tx=0).stop 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 shutdown_connections 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff55007abf0 0x7ff55007d0b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5601067f0 0x7ff56019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 --2- 172.21.15.46:0/3928614043 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5601007f0 0x7ff56019d600 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 >> 172.21.15.46:0/3928614043 conn(0x7ff5600fc550 msgr2=0x7ff560069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 shutdown_connections 2024-09-10T04:25:33.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:33.994+0000 7ff566c8a640 1 -- 172.21.15.46:0/3928614043 wait complete. 2024-09-10T04:25:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/84405752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4101900 msgr2=0x7f5fb4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/84405752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4101900 0x7f5fb4101d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009c20 tx=0x7f5fa402a970 comp rx=0 tx=0).stop 2024-09-10T04:25:34.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/84405752 shutdown_connections 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/84405752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4101900 0x7f5fb4101d60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/84405752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb4107900 0x7f5fb4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/84405752 >> 172.21.15.46:0/84405752 conn(0x7f5fb40fd600 msgr2=0x7f5fb40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/84405752 shutdown_connections 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.079+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/84405752 wait complete. 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fbb1d4640 1 Processor -- start 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fbb1d4640 1 -- start start 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fbb1d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb4101900 0x7f5fb40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fbb1d4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4107900 0x7f5fb4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fbb1d4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410edd0 con 0x7f5fb4107900 2024-09-10T04:25:34.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fbb1d4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb410ec50 con 0x7f5fb4101900 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4107900 0x7f5fb4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4107900 0x7f5fb4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41902/0 (socket says 172.21.15.46:41902) 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb3fff640 1 -- 172.21.15.46:0/2401172890 learned_addr learned my addr 172.21.15.46:0/2401172890 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb8f70640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb4101900 0x7f5fb40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb3fff640 1 -- 172.21.15.46:0/2401172890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb4101900 msgr2=0x7f5fb40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb3fff640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb4101900 0x7f5fb40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb3fff640 1 -- 172.21.15.46:0/2401172890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb4074f70 con 0x7f5fb4107900 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.080+0000 7f5fb8f70640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb4101900 0x7f5fb40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.081+0000 7f5fb3fff640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4107900 0x7f5fb4074a30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402ae90 tx=0x7f5fa402f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.081+0000 7f5fb1ffb640 1 -- 172.21.15.46:0/2401172890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5fa4047070 con 0x7f5fb4107900 2024-09-10T04:25:34.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.081+0000 7f5fb1ffb640 1 -- 172.21.15.46:0/2401172890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5fa402f7e0 con 0x7f5fb4107900 2024-09-10T04:25:34.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.081+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb4075200 con 0x7f5fb4107900 2024-09-10T04:25:34.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.081+0000 7f5fb1ffb640 1 -- 172.21.15.46:0/2401172890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5fa4034070 con 0x7f5fb4107900 2024-09-10T04:25:34.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.081+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb41a3dc0 con 0x7f5fb4107900 2024-09-10T04:25:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.082+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb4103040 con 0x7f5fb4107900 2024-09-10T04:25:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.083+0000 7f5fb1ffb640 1 -- 172.21.15.46:0/2401172890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5fa404a070 con 0x7f5fb4107900 2024-09-10T04:25:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.083+0000 7f5fb1ffb640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fa007ac40 0x7f5fa007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.083+0000 7f5fb1ffb640 1 -- 172.21.15.46:0/2401172890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f5fa40be910 con 0x7f5fb4107900 2024-09-10T04:25:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.083+0000 7f5fb8f70640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fa007ac40 0x7f5fa007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.084+0000 7f5fb8f70640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fa007ac40 0x7f5fa007d100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009b40 tx=0x7f5f9c005e50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:34.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.088+0000 7f5fb1ffb640 1 -- 172.21.15.46:0/2401172890 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5fa403f070 con 0x7f5fb4107900 2024-09-10T04:25:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.311+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5fb4109e70 con 0x7f5fb4107900 2024-09-10T04:25:34.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.312+0000 7f5fb1ffb640 1 -- 172.21.15.46:0/2401172890 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f5fa408a0d0 con 0x7f5fb4107900 2024-09-10T04:25:34.313 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:25:34.313 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:25:34.313 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:25:34.313 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:25:34.314 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:25:34.315 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:25:34.316 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:25:34.317 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:34.317 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:34.317 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:34.317 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fa007ac40 msgr2=0x7f5fa007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fa007ac40 0x7f5fa007d100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009b40 tx=0x7f5f9c005e50 comp rx=0 tx=0).stop 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4107900 msgr2=0x7f5fb4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4107900 0x7f5fb4074a30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5fa402ae90 tx=0x7f5fa402f5f0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 shutdown_connections 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fa007ac40 0x7f5fa007d100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fb4107900 0x7f5fb4074a30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 --2- 172.21.15.46:0/2401172890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fb4101900 0x7f5fb40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.314+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 >> 172.21.15.46:0/2401172890 conn(0x7f5fb40fd600 msgr2=0x7f5fb40fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:34.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.315+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 shutdown_connections 2024-09-10T04:25:34.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.315+0000 7f5fbb1d4640 1 -- 172.21.15.46:0/2401172890 wait complete. 2024-09-10T04:25:34.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 -- 172.21.15.46:0/4279614271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 msgr2=0x7fdf8c100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 --2- 172.21.15.46:0/4279614271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100220 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009880 tx=0x7fdf8002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 -- 172.21.15.46:0/4279614271 shutdown_connections 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 --2- 172.21.15.46:0/4279614271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100220 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 --2- 172.21.15.46:0/4279614271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf8c1069a0 0x7fdf8c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 -- 172.21.15.46:0/4279614271 >> 172.21.15.46:0/4279614271 conn(0x7fdf8c0fbc90 msgr2=0x7fdf8c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 -- 172.21.15.46:0/4279614271 shutdown_connections 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 -- 172.21.15.46:0/4279614271 wait complete. 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 Processor -- start 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.402+0000 7fdf91578640 1 -- start start 2024-09-10T04:25:34.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf91578640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf91578640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf8c1069a0 0x7fdf8c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf91578640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c10ef20 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf91578640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c10eda0 con 0x7fdf8c1069a0 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41932/0 (socket says 172.21.15.46:41932) 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8affd640 1 -- 172.21.15.46:0/1809472472 learned_addr learned my addr 172.21.15.46:0/1809472472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8a7fc640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf8c1069a0 0x7fdf8c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8affd640 1 -- 172.21.15.46:0/1809472472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf8c1069a0 msgr2=0x7fdf8c1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8affd640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf8c1069a0 0x7fdf8c1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8affd640 1 -- 172.21.15.46:0/1809472472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c101a20 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.403+0000 7fdf8affd640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100fa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c810 tx=0x7fdf7c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:34.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.404+0000 7fdf73fff640 1 -- 172.21.15.46:0/1809472472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf7c004020 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.404+0000 7fdf73fff640 1 -- 172.21.15.46:0/1809472472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdf7c0041c0 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.404+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c101d10 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.404+0000 7fdf73fff640 1 -- 172.21.15.46:0/1809472472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdf7c0047a0 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.404+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8c1a3f30 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.405+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8c1030e0 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.405+0000 7fdf73fff640 1 -- 172.21.15.46:0/1809472472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdf7c0079b0 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.406+0000 7fdf73fff640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf7407ac40 0x7fdf7407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.406+0000 7fdf73fff640 1 -- 172.21.15.46:0/1809472472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fdf7c09daa0 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.406+0000 7fdf8a7fc640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf7407ac40 0x7fdf7407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.406+0000 7fdf8a7fc640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf7407ac40 0x7fdf7407d100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c102a30 tx=0x7fdf80031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:34.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.410+0000 7fdf73fff640 1 -- 172.21.15.46:0/1809472472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdf7c014030 con 0x7fdf8c0ffdc0 2024-09-10T04:25:34.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:34 smithi070 ceph-mon[79774]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:34.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:34 smithi070 ceph-mon[79774]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:34.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:34 smithi070 ceph-mon[79774]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:34.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:34 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3928614043' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:25:34.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:34 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2401172890' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:25:34.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:34 smithi046 ceph-mon[109506]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:34.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:34 smithi046 ceph-mon[109506]: from='client.34958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:34.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:34 smithi046 ceph-mon[109506]: from='client.34962 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:34.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:34 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3928614043' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:25:34.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:34 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2401172890' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:25:34.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.615+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdf8c062630 con 0x7fdf7407ac40 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.616+0000 7fdf73fff640 1 -- 172.21.15.46:0/1809472472 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fdf8c062630 con 0x7fdf7407ac40 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:25:34.617 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:25:34.618 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:25:34.618 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:25:34.618 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:25:34.618 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:25:34.618 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:25:34.618 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:25:34.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.618+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf7407ac40 msgr2=0x7fdf7407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf7407ac40 0x7fdf7407d100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c102a30 tx=0x7fdf80031420 comp rx=0 tx=0).stop 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 msgr2=0x7fdf8c100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100fa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c810 tx=0x7fdf7c00ccd0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 shutdown_connections 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdf7407ac40 0x7fdf7407d100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf8c1069a0 0x7fdf8c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 --2- 172.21.15.46:0/1809472472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf8c0ffdc0 0x7fdf8c100fa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 >> 172.21.15.46:0/1809472472 conn(0x7fdf8c0fbc90 msgr2=0x7fdf8c0fc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 shutdown_connections 2024-09-10T04:25:34.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.619+0000 7fdf91578640 1 -- 172.21.15.46:0/1809472472 wait complete. 2024-09-10T04:25:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 -- 172.21.15.46:0/1891918351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 msgr2=0x7f6288107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 --2- 172.21.15.46:0/1891918351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288107ce0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6284009880 tx=0x7f628402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 -- 172.21.15.46:0/1891918351 shutdown_connections 2024-09-10T04:25:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 --2- 172.21.15.46:0/1891918351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6288101900 0x7f6288101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 --2- 172.21.15.46:0/1891918351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288107ce0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 -- 172.21.15.46:0/1891918351 >> 172.21.15.46:0/1891918351 conn(0x7f62880fd600 msgr2=0x7f62880ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 -- 172.21.15.46:0/1891918351 shutdown_connections 2024-09-10T04:25:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 -- 172.21.15.46:0/1891918351 wait complete. 2024-09-10T04:25:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 Processor -- start 2024-09-10T04:25:34.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.703+0000 7f628f8b3640 1 -- start start 2024-09-10T04:25:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628f8b3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6288101900 0x7f62880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628f8b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628f8b3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f628810edd0 con 0x7f6288107900 2024-09-10T04:25:34.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628f8b3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f628810ec50 con 0x7f6288101900 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628ce4e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628ce4e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41948/0 (socket says 172.21.15.46:41948) 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628ce4e640 1 -- 172.21.15.46:0/4047606624 learned_addr learned my addr 172.21.15.46:0/4047606624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628d64f640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6288101900 0x7f62880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628ce4e640 1 -- 172.21.15.46:0/4047606624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6288101900 msgr2=0x7f62880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628ce4e640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6288101900 0x7f62880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628ce4e640 1 -- 172.21.15.46:0/4047606624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6288074f70 con 0x7f6288107900 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.704+0000 7f628ce4e640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288074a30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f627800d6f0 tx=0x7f627800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.705+0000 7f627e7fc640 1 -- 172.21.15.46:0/4047606624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6278017070 con 0x7f6288107900 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.705+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6288075260 con 0x7f6288107900 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.705+0000 7f627e7fc640 1 -- 172.21.15.46:0/4047606624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f627800de00 con 0x7f6288107900 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.705+0000 7f627e7fc640 1 -- 172.21.15.46:0/4047606624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6278004bb0 con 0x7f6288107900 2024-09-10T04:25:34.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.705+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62881a3e40 con 0x7f6288107900 2024-09-10T04:25:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.706+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6288103040 con 0x7f6288107900 2024-09-10T04:25:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.706+0000 7f627e7fc640 1 -- 172.21.15.46:0/4047606624 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6278004e60 con 0x7f6288107900 2024-09-10T04:25:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.707+0000 7f627e7fc640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f627407abf0 0x7f627407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:25:34.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.707+0000 7f627e7fc640 1 -- 172.21.15.46:0/4047606624 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f627809d8d0 con 0x7f6288107900 2024-09-10T04:25:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.707+0000 7f628d64f640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f627407abf0 0x7f627407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:25:34.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.707+0000 7f628d64f640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f627407abf0 0x7f627407d0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f62840096c0 tx=0x7f6284031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:25:34.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.711+0000 7f627e7fc640 1 -- 172.21.15.46:0/4047606624 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6278069010 con 0x7f6288107900 2024-09-10T04:25:34.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.971+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6288109e70 con 0x7f6288107900 2024-09-10T04:25:34.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.971+0000 7f627e7fc640 1 -- 172.21.15.46:0/4047606624 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f627806dec0 con 0x7f6288107900 2024-09-10T04:25:34.972 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:25:34.972 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:25:34.972 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:25:34.973 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:25:34.973 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:25:34.973 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:25:34.973 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:25:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f627407abf0 msgr2=0x7f627407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f627407abf0 0x7f627407d0b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f62840096c0 tx=0x7f6284031400 comp rx=0 tx=0).stop 2024-09-10T04:25:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 msgr2=0x7f6288074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:25:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288074a30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f627800d6f0 tx=0x7f627800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 shutdown_connections 2024-09-10T04:25:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f627407abf0 0x7f627407d0b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6288107900 0x7f6288074a30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 --2- 172.21.15.46:0/4047606624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6288101900 0x7f62880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:25:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 >> 172.21.15.46:0/4047606624 conn(0x7f62880fd600 msgr2=0x7f62880fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:25:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 shutdown_connections 2024-09-10T04:25:34.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:25:34.973+0000 7f628f8b3640 1 -- 172.21.15.46:0/4047606624 wait complete. 2024-09-10T04:25:35.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:35 smithi046 ceph-mon[109506]: pgmap v597: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:35 smithi046 ceph-mon[109506]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:35 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4047606624' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:25:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:35 smithi070 ceph-mon[79774]: pgmap v597: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:35 smithi070 ceph-mon[79774]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:25:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:35 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4047606624' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:25:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:37 smithi070 ceph-mon[79774]: pgmap v598: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:37.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:37 smithi046 ceph-mon[109506]: pgmap v598: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:39.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:39 smithi070 ceph-mon[79774]: pgmap v599: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:39 smithi046 ceph-mon[109506]: pgmap v599: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:41 smithi046 ceph-mon[109506]: pgmap v600: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:41 smithi070 ceph-mon[79774]: pgmap v600: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:43 smithi070 ceph-mon[79774]: pgmap v601: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:43 smithi046 ceph-mon[109506]: pgmap v601: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:45 smithi070 ceph-mon[79774]: pgmap v602: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:45 smithi046 ceph-mon[109506]: pgmap v602: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:47 smithi046 ceph-mon[109506]: pgmap v603: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:47 smithi070 ceph-mon[79774]: pgmap v603: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:49 smithi070 ceph-mon[79774]: pgmap v604: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:49.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:49 smithi046 ceph-mon[109506]: pgmap v604: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:51 smithi070 ceph-mon[79774]: pgmap v605: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:51 smithi046 ceph-mon[109506]: pgmap v605: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:53 smithi070 ceph-mon[79774]: pgmap v606: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:53 smithi046 ceph-mon[109506]: pgmap v606: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:25:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:55 smithi070 ceph-mon[79774]: pgmap v607: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:55 smithi046 ceph-mon[109506]: pgmap v607: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:57 smithi070 ceph-mon[79774]: pgmap v608: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:57 smithi046 ceph-mon[109506]: pgmap v608: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:59.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:25:59 smithi046 ceph-mon[109506]: pgmap v609: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:25:59.637 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:25:59 smithi070 ceph-mon[79774]: pgmap v609: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:01 smithi070 ceph-mon[79774]: pgmap v610: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:01 smithi046 ceph-mon[109506]: pgmap v610: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:03.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:03 smithi046 ceph-mon[109506]: pgmap v611: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:03 smithi070 ceph-mon[79774]: pgmap v611: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.070+0000 7f0fdc560640 1 -- 172.21.15.46:0/1851434088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 msgr2=0x7f0fd4100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.070+0000 7f0fdc560640 1 --2- 172.21.15.46:0/1851434088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd4100790 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009880 tx=0x7f0fc402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.070+0000 7f0fdc560640 1 -- 172.21.15.46:0/1851434088 shutdown_connections 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.070+0000 7f0fdc560640 1 --2- 172.21.15.46:0/1851434088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd4100790 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.070+0000 7f0fdc560640 1 --2- 172.21.15.46:0/1851434088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fd4069920 0x7f0fd4100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.070+0000 7f0fdc560640 1 -- 172.21.15.46:0/1851434088 >> 172.21.15.46:0/1851434088 conn(0x7f0fd40fbc90 msgr2=0x7f0fd40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 -- 172.21.15.46:0/1851434088 shutdown_connections 2024-09-10T04:26:05.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 -- 172.21.15.46:0/1851434088 wait complete. 2024-09-10T04:26:05.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 Processor -- start 2024-09-10T04:26:05.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 -- start start 2024-09-10T04:26:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fd4069920 0x7f0fd419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410dd50 con 0x7f0fd4069d90 2024-09-10T04:26:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.071+0000 7f0fdc560640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410dbd0 con 0x7f0fd4069920 2024-09-10T04:26:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fd9afb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fd9afb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51100/0 (socket says 172.21.15.46:51100) 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fd9afb640 1 -- 172.21.15.46:0/160839780 learned_addr learned my addr 172.21.15.46:0/160839780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fda2fc640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fd4069920 0x7f0fd419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fd9afb640 1 -- 172.21.15.46:0/160839780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fd4069920 msgr2=0x7f0fd419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fd9afb640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fd4069920 0x7f0fd419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fd9afb640 1 -- 172.21.15.46:0/160839780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd4197810 con 0x7f0fd4069d90 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fd9afb640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd419dbd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4031460 tx=0x7f0fc40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.072+0000 7f0fcb7fe640 1 -- 172.21.15.46:0/160839780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0fc403f070 con 0x7f0fd4069d90 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.073+0000 7f0fcb7fe640 1 -- 172.21.15.46:0/160839780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0fc40035f0 con 0x7f0fd4069d90 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.073+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd4197aa0 con 0x7f0fd4069d90 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.073+0000 7f0fcb7fe640 1 -- 172.21.15.46:0/160839780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0fc4035030 con 0x7f0fd4069d90 2024-09-10T04:26:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.073+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd4197fd0 con 0x7f0fd4069d90 2024-09-10T04:26:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.074+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd4101730 con 0x7f0fd4069d90 2024-09-10T04:26:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.075+0000 7f0fcb7fe640 1 -- 172.21.15.46:0/160839780 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0fc4042070 con 0x7f0fd4069d90 2024-09-10T04:26:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.075+0000 7f0fcb7fe640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fc007abf0 0x7f0fc007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.075+0000 7f0fcb7fe640 1 -- 172.21.15.46:0/160839780 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0fc40be670 con 0x7f0fd4069d90 2024-09-10T04:26:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.075+0000 7f0fda2fc640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fc007abf0 0x7f0fc007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.076+0000 7f0fda2fc640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fc007abf0 0x7f0fc007d0b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0005fd0 tx=0x7f0fd000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.080+0000 7f0fcb7fe640 1 -- 172.21.15.46:0/160839780 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0fc4089d10 con 0x7f0fd4069d90 2024-09-10T04:26:05.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.287+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0fd4198d40 con 0x7f0fc007abf0 2024-09-10T04:26:05.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.288+0000 7f0fcb7fe640 1 -- 172.21.15.46:0/160839780 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0fd4198d40 con 0x7f0fc007abf0 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fc007abf0 msgr2=0x7f0fc007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fc007abf0 0x7f0fc007d0b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0005fd0 tx=0x7f0fd000b390 comp rx=0 tx=0).stop 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 msgr2=0x7f0fd419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd419dbd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4031460 tx=0x7f0fc40045e0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 shutdown_connections 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0fc007abf0 0x7f0fc007d0b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fd4069d90 0x7f0fd419dbd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 --2- 172.21.15.46:0/160839780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fd4069920 0x7f0fd419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 >> 172.21.15.46:0/160839780 conn(0x7f0fd40fbc90 msgr2=0x7f0fd4101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 shutdown_connections 2024-09-10T04:26:05.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.291+0000 7f0fdc560640 1 -- 172.21.15.46:0/160839780 wait complete. 2024-09-10T04:26:05.301 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:26:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.377+0000 7fa18b3af640 1 -- 172.21.15.46:0/1473274624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184100800 msgr2=0x7fa184100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.377+0000 7fa18b3af640 1 --2- 172.21.15.46:0/1473274624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184100800 0x7fa184100c60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa174009880 tx=0x7fa17402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 -- 172.21.15.46:0/1473274624 shutdown_connections 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 --2- 172.21.15.46:0/1473274624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184100800 0x7fa184100c60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 --2- 172.21.15.46:0/1473274624 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa184106800 0x7fa184106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 -- 172.21.15.46:0/1473274624 >> 172.21.15.46:0/1473274624 conn(0x7fa1840fc520 msgr2=0x7fa1840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 -- 172.21.15.46:0/1473274624 shutdown_connections 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 -- 172.21.15.46:0/1473274624 wait complete. 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 Processor -- start 2024-09-10T04:26:05.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.378+0000 7fa18b3af640 1 -- start start 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18b3af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa184100800 0x7fa18419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18b3af640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184106800 0x7fa18419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18b3af640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410dcd0 con 0x7fa184106800 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18b3af640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410db50 con 0x7fa184100800 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18894a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184106800 0x7fa18419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18894a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184106800 0x7fa18419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51126/0 (socket says 172.21.15.46:51126) 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18894a640 1 -- 172.21.15.46:0/271478969 learned_addr learned my addr 172.21.15.46:0/271478969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18914b640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa184100800 0x7fa18419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18894a640 1 -- 172.21.15.46:0/271478969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa184100800 msgr2=0x7fa18419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18894a640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa184100800 0x7fa18419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18894a640 1 -- 172.21.15.46:0/271478969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1841999a0 con 0x7fa184106800 2024-09-10T04:26:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18894a640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184106800 0x7fa18419fd20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa174031460 tx=0x7fa174004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa17a7fc640 1 -- 172.21.15.46:0/271478969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa17403f070 con 0x7fa184106800 2024-09-10T04:26:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa17a7fc640 1 -- 172.21.15.46:0/271478969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1740035f0 con 0x7fa184106800 2024-09-10T04:26:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa184199c30 con 0x7fa184106800 2024-09-10T04:26:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.379+0000 7fa17a7fc640 1 -- 172.21.15.46:0/271478969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa174035030 con 0x7fa184106800 2024-09-10T04:26:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.380+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa18419a160 con 0x7fa184106800 2024-09-10T04:26:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.381+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa184101f40 con 0x7fa184106800 2024-09-10T04:26:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.381+0000 7fa17a7fc640 1 -- 172.21.15.46:0/271478969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa174042070 con 0x7fa184106800 2024-09-10T04:26:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.382+0000 7fa17a7fc640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa17007ac40 0x7fa17007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.382+0000 7fa17a7fc640 1 -- 172.21.15.46:0/271478969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fa1740be4a0 con 0x7fa184106800 2024-09-10T04:26:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.382+0000 7fa18914b640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa17007ac40 0x7fa17007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.382+0000 7fa18914b640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa17007ac40 0x7fa17007d100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa18419aae0 tx=0x7fa180005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.386+0000 7fa17a7fc640 1 -- 172.21.15.46:0/271478969 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa174089c60 con 0x7fa184106800 2024-09-10T04:26:05.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.594+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa18419a410 con 0x7fa17007ac40 2024-09-10T04:26:05.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.595+0000 7fa17a7fc640 1 -- 172.21.15.46:0/271478969 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fa18419a410 con 0x7fa17007ac40 2024-09-10T04:26:05.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa17007ac40 msgr2=0x7fa17007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa17007ac40 0x7fa17007d100 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa18419aae0 tx=0x7fa180005f90 comp rx=0 tx=0).stop 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184106800 msgr2=0x7fa18419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184106800 0x7fa18419fd20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa174031460 tx=0x7fa174004530 comp rx=0 tx=0).stop 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 shutdown_connections 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa17007ac40 0x7fa17007d100 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa184106800 0x7fa18419fd20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 --2- 172.21.15.46:0/271478969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa184100800 0x7fa18419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.597+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 >> 172.21.15.46:0/271478969 conn(0x7fa1840fc520 msgr2=0x7fa1840fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:05.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.598+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 shutdown_connections 2024-09-10T04:26:05.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.598+0000 7fa18b3af640 1 -- 172.21.15.46:0/271478969 wait complete. 2024-09-10T04:26:05.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.682+0000 7ff366754640 1 -- 172.21.15.46:0/2203292879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 msgr2=0x7ff360100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.682+0000 7ff366754640 1 --2- 172.21.15.46:0/2203292879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100220 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff354009880 tx=0x7ff35402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.682+0000 7ff366754640 1 -- 172.21.15.46:0/2203292879 shutdown_connections 2024-09-10T04:26:05.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.682+0000 7ff366754640 1 --2- 172.21.15.46:0/2203292879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100220 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.682+0000 7ff366754640 1 --2- 172.21.15.46:0/2203292879 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3601069a0 0x7ff360106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.682+0000 7ff366754640 1 -- 172.21.15.46:0/2203292879 >> 172.21.15.46:0/2203292879 conn(0x7ff3600fbc90 msgr2=0x7ff3600fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:05.685 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:05 smithi046 ceph-mon[109506]: pgmap v612: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:05.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.683+0000 7ff366754640 1 -- 172.21.15.46:0/2203292879 shutdown_connections 2024-09-10T04:26:05.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.683+0000 7ff366754640 1 -- 172.21.15.46:0/2203292879 wait complete. 2024-09-10T04:26:05.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.683+0000 7ff366754640 1 Processor -- start 2024-09-10T04:26:05.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.683+0000 7ff366754640 1 -- start start 2024-09-10T04:26:05.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff366754640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff366754640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3601069a0 0x7ff3601014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff366754640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff36010ef20 con 0x7ff3600ffdc0 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff366754640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff36010eda0 con 0x7ff3601069a0 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff35ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51154/0 (socket says 172.21.15.46:51154) 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff35ffff640 1 -- 172.21.15.46:0/2466047494 learned_addr learned my addr 172.21.15.46:0/2466047494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff35f7fe640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3601069a0 0x7ff3601014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff35ffff640 1 -- 172.21.15.46:0/2466047494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3601069a0 msgr2=0x7ff3601014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff35ffff640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3601069a0 0x7ff3601014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.684+0000 7ff35ffff640 1 -- 172.21.15.46:0/2466047494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff360101a20 con 0x7ff3600ffdc0 2024-09-10T04:26:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.685+0000 7ff35ffff640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100fa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff35000d8d0 tx=0x7ff35000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.685+0000 7ff35d7fa640 1 -- 172.21.15.46:0/2466047494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff350017030 con 0x7ff3600ffdc0 2024-09-10T04:26:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.685+0000 7ff35d7fa640 1 -- 172.21.15.46:0/2466047494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff350004020 con 0x7ff3600ffdc0 2024-09-10T04:26:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.685+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff360101d10 con 0x7ff3600ffdc0 2024-09-10T04:26:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.685+0000 7ff35d7fa640 1 -- 172.21.15.46:0/2466047494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff3500047a0 con 0x7ff3600ffdc0 2024-09-10T04:26:05.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.685+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3601a3f50 con 0x7ff3600ffdc0 2024-09-10T04:26:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.686+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3601030e0 con 0x7ff3600ffdc0 2024-09-10T04:26:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.687+0000 7ff35d7fa640 1 -- 172.21.15.46:0/2466047494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff3500041d0 con 0x7ff3600ffdc0 2024-09-10T04:26:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.687+0000 7ff35d7fa640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff34807ac40 0x7ff34807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.687+0000 7ff35d7fa640 1 -- 172.21.15.46:0/2466047494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ff350018070 con 0x7ff3600ffdc0 2024-09-10T04:26:05.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.687+0000 7ff35f7fe640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff34807ac40 0x7ff34807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.688+0000 7ff35f7fe640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff34807ac40 0x7ff34807d100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff360102a30 tx=0x7ff354031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.693+0000 7ff35d7fa640 1 -- 172.21.15.46:0/2466047494 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff3500691a0 con 0x7ff3600ffdc0 2024-09-10T04:26:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:05 smithi070 ceph-mon[79774]: pgmap v612: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.887+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff360062630 con 0x7ff34807ac40 2024-09-10T04:26:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.897+0000 7ff35d7fa640 1 -- 172.21.15.46:0/2466047494 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff360062630 con 0x7ff34807ac40 2024-09-10T04:26:05.899 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (17m) 12m ago 28m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (20m) 12m ago 29m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (20m) 11m ago 28m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (16m) 12m ago 29m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (16m) 11m ago 28m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (17m) 12m ago 28m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (12m) 12m ago 23m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (12m) 12m ago 23m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (11m) 11m ago 23m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:26:05.900 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (23m) 11m ago 23m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (19m) 12m ago 30m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (18m) 11m ago 28m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (16m) 12m ago 30m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (16m) 11m ago 28m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (18m) 12m ago 29m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (18m) 11m ago 28m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (15m) 12m ago 27m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (15m) 12m ago 26m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (14m) 12m ago 25m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (13m) 11m ago 25m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (13m) 11m ago 25m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (12m) 11m ago 24m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:26:05.901 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (18m) 12m ago 28m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:26:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff34807ac40 msgr2=0x7ff34807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff34807ac40 0x7ff34807d100 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff360102a30 tx=0x7ff354031400 comp rx=0 tx=0).stop 2024-09-10T04:26:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 msgr2=0x7ff360100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100fa0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff35000d8d0 tx=0x7ff35000dd90 comp rx=0 tx=0).stop 2024-09-10T04:26:05.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 shutdown_connections 2024-09-10T04:26:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff34807ac40 0x7ff34807d100 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3601069a0 0x7ff3601014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 --2- 172.21.15.46:0/2466047494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3600ffdc0 0x7ff360100fa0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 >> 172.21.15.46:0/2466047494 conn(0x7ff3600fbc90 msgr2=0x7ff3600fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 shutdown_connections 2024-09-10T04:26:05.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.900+0000 7ff366754640 1 -- 172.21.15.46:0/2466047494 wait complete. 2024-09-10T04:26:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 -- 172.21.15.46:0/1053539580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 msgr2=0x7fdac0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 --2- 172.21.15.46:0/1053539580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac0074a00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdab0009880 tx=0x7fdab002a530 comp rx=0 tx=0).stop 2024-09-10T04:26:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 -- 172.21.15.46:0/1053539580 shutdown_connections 2024-09-10T04:26:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 --2- 172.21.15.46:0/1053539580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdac0074f40 0x7fdac010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 --2- 172.21.15.46:0/1053539580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac0074a00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 -- 172.21.15.46:0/1053539580 >> 172.21.15.46:0/1053539580 conn(0x7fdac00fd600 msgr2=0x7fdac00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:05.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 -- 172.21.15.46:0/1053539580 shutdown_connections 2024-09-10T04:26:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.983+0000 7fdac697f640 1 -- 172.21.15.46:0/1053539580 wait complete. 2024-09-10T04:26:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.984+0000 7fdac697f640 1 Processor -- start 2024-09-10T04:26:05.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.984+0000 7fdac697f640 1 -- start start 2024-09-10T04:26:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.984+0000 7fdac697f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.984+0000 7fdac697f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdac0074f40 0x7fdac019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.984+0000 7fdac697f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0110fd0 con 0x7fdac0074620 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.984+0000 7fdac697f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0110e50 con 0x7fdac0074f40 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.984+0000 7fdabffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51172/0 (socket says 172.21.15.46:51172) 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabffff640 1 -- 172.21.15.46:0/601100743 learned_addr learned my addr 172.21.15.46:0/601100743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabf7fe640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdac0074f40 0x7fdac019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabffff640 1 -- 172.21.15.46:0/601100743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdac0074f40 msgr2=0x7fdac019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabffff640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdac0074f40 0x7fdac019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabffff640 1 -- 172.21.15.46:0/601100743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac01999e0 con 0x7fdac0074620 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabffff640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac019f820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdab0031460 tx=0x7fdab00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabd7fa640 1 -- 172.21.15.46:0/601100743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdab003f070 con 0x7fdac0074620 2024-09-10T04:26:05.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac0199c70 con 0x7fdac0074620 2024-09-10T04:26:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabd7fa640 1 -- 172.21.15.46:0/601100743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdab0004cf0 con 0x7fdac0074620 2024-09-10T04:26:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdabd7fa640 1 -- 172.21.15.46:0/601100743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdab0035030 con 0x7fdac0074620 2024-09-10T04:26:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.985+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac019a1a0 con 0x7fdac0074620 2024-09-10T04:26:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.987+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdac0075cd0 con 0x7fdac0074620 2024-09-10T04:26:05.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.988+0000 7fdabd7fa640 1 -- 172.21.15.46:0/601100743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdab0042070 con 0x7fdac0074620 2024-09-10T04:26:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.988+0000 7fdabd7fa640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdaa807abf0 0x7fdaa807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.988+0000 7fdabd7fa640 1 -- 172.21.15.46:0/601100743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fdab00be6a0 con 0x7fdac0074620 2024-09-10T04:26:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.988+0000 7fdabf7fe640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdaa807abf0 0x7fdaa807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:05.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.989+0000 7fdabf7fe640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdaa807abf0 0x7fdaa807d0b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdac010ac30 tx=0x7fdab400a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:05.993+0000 7fdabd7fa640 1 -- 172.21.15.46:0/601100743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdab0089de0 con 0x7fdac0074620 2024-09-10T04:26:06.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.253+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fdac019af10 con 0x7fdac0074620 2024-09-10T04:26:06.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.254+0000 7fdabd7fa640 1 -- 172.21.15.46:0/601100743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fdab008ec90 con 0x7fdac0074620 2024-09-10T04:26:06.256 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:26:06.256 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:26:06.256 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:26:06.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:06.256 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:26:06.256 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:26:06.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:26:06.257 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:26:06.258 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:26:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdaa807abf0 msgr2=0x7fdaa807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdaa807abf0 0x7fdaa807d0b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdac010ac30 tx=0x7fdab400a2d0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 msgr2=0x7fdac019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac019f820 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdab0031460 tx=0x7fdab00048f0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 shutdown_connections 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdaa807abf0 0x7fdaa807d0b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdac0074f40 0x7fdac019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 --2- 172.21.15.46:0/601100743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdac0074620 0x7fdac019f820 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 >> 172.21.15.46:0/601100743 conn(0x7fdac00fd600 msgr2=0x7fdac010b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 shutdown_connections 2024-09-10T04:26:06.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.256+0000 7fdac697f640 1 -- 172.21.15.46:0/601100743 wait complete. 2024-09-10T04:26:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 -- 172.21.15.46:0/1439883888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41067f0 msgr2=0x7fdce4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 --2- 172.21.15.46:0/1439883888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41067f0 0x7fdce4106bd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009880 tx=0x7fdce002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 -- 172.21.15.46:0/1439883888 shutdown_connections 2024-09-10T04:26:06.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 --2- 172.21.15.46:0/1439883888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce41007f0 0x7fdce4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 --2- 172.21.15.46:0/1439883888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41067f0 0x7fdce4106bd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 -- 172.21.15.46:0/1439883888 >> 172.21.15.46:0/1439883888 conn(0x7fdce40fc550 msgr2=0x7fdce40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 -- 172.21.15.46:0/1439883888 shutdown_connections 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 -- 172.21.15.46:0/1439883888 wait complete. 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 Processor -- start 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.341+0000 7fdceb279640 1 -- start start 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdceb279640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41007f0 0x7fdce40fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdceb279640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce41067f0 0x7fdce40fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdceb279640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdce410dcc0 con 0x7fdce41007f0 2024-09-10T04:26:06.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdceb279640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdce410db40 con 0x7fdce41067f0 2024-09-10T04:26:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce9015640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41007f0 0x7fdce40fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce9015640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41007f0 0x7fdce40fccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51188/0 (socket says 172.21.15.46:51188) 2024-09-10T04:26:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce9015640 1 -- 172.21.15.46:0/1774766138 learned_addr learned my addr 172.21.15.46:0/1774766138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce8814640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce41067f0 0x7fdce40fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce9015640 1 -- 172.21.15.46:0/1774766138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce41067f0 msgr2=0x7fdce40fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce9015640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce41067f0 0x7fdce40fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce9015640 1 -- 172.21.15.46:0/1774766138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce40fd770 con 0x7fdce41007f0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdce9015640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41007f0 0x7fdce40fccf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdce0031460 tx=0x7fdce0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdcd67fc640 1 -- 172.21.15.46:0/1774766138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdce003f070 con 0x7fdce41007f0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdcd67fc640 1 -- 172.21.15.46:0/1774766138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdce00035f0 con 0x7fdce41007f0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce40fda00 con 0x7fdce41007f0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.342+0000 7fdcd67fc640 1 -- 172.21.15.46:0/1774766138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdce0035030 con 0x7fdce41007f0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.343+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41a3d70 con 0x7fdce41007f0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.344+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdce4101f30 con 0x7fdce41007f0 2024-09-10T04:26:06.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.344+0000 7fdcd67fc640 1 -- 172.21.15.46:0/1774766138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdce0042070 con 0x7fdce41007f0 2024-09-10T04:26:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.345+0000 7fdcd67fc640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdcd007abf0 0x7fdcd007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.345+0000 7fdcd67fc640 1 -- 172.21.15.46:0/1774766138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fdce00be6c0 con 0x7fdce41007f0 2024-09-10T04:26:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.345+0000 7fdce8814640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdcd007abf0 0x7fdcd007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.345+0000 7fdce8814640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdcd007abf0 0x7fdcd007d0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdce40fe780 tx=0x7fdcd8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:06.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.349+0000 7fdcd67fc640 1 -- 172.21.15.46:0/1774766138 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdce0089da0 con 0x7fdce41007f0 2024-09-10T04:26:06.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.573+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fdce40fe220 con 0x7fdce41007f0 2024-09-10T04:26:06.576 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:06 smithi046 ceph-mon[109506]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:06.576 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:06 smithi046 ceph-mon[109506]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:06.576 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:06 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/601100743' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:26:06.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.574+0000 7fdcd67fc640 1 -- 172.21.15.46:0/1774766138 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7fdce008ec50 con 0x7fdce41007f0 2024-09-10T04:26:06.576 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:26:06.577 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:26:06.578 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:26:06.579 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:06.580 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:26:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdcd007abf0 msgr2=0x7fdcd007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdcd007abf0 0x7fdcd007d0b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdce40fe780 tx=0x7fdcd8005f90 comp rx=0 tx=0).stop 2024-09-10T04:26:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41007f0 msgr2=0x7fdce40fccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41007f0 0x7fdce40fccf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdce0031460 tx=0x7fdce0004530 comp rx=0 tx=0).stop 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 shutdown_connections 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdcd007abf0 0x7fdcd007d0b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdce41067f0 0x7fdce40fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 --2- 172.21.15.46:0/1774766138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdce41007f0 0x7fdce40fccf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 >> 172.21.15.46:0/1774766138 conn(0x7fdce40fc550 msgr2=0x7fdce4069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 shutdown_connections 2024-09-10T04:26:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.576+0000 7fdceb279640 1 -- 172.21.15.46:0/1774766138 wait complete. 2024-09-10T04:26:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.663+0000 7f65f868b640 1 -- 172.21.15.46:0/2370834953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 msgr2=0x7f65f0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.663+0000 7f65f868b640 1 --2- 172.21.15.46:0/2370834953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f0101d60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009880 tx=0x7f65e002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 -- 172.21.15.46:0/2370834953 shutdown_connections 2024-09-10T04:26:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 --2- 172.21.15.46:0/2370834953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f0101d60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 --2- 172.21.15.46:0/2370834953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f0107900 0x7f65f0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 -- 172.21.15.46:0/2370834953 >> 172.21.15.46:0/2370834953 conn(0x7f65f00fd600 msgr2=0x7f65f00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 -- 172.21.15.46:0/2370834953 shutdown_connections 2024-09-10T04:26:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 -- 172.21.15.46:0/2370834953 wait complete. 2024-09-10T04:26:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 Processor -- start 2024-09-10T04:26:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 -- start start 2024-09-10T04:26:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.664+0000 7f65f868b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f868b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f0107900 0x7f65f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f868b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65f010edd0 con 0x7f65f0101900 2024-09-10T04:26:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f868b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f65f010ec50 con 0x7f65f0107900 2024-09-10T04:26:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f6427640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f6427640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51202/0 (socket says 172.21.15.46:51202) 2024-09-10T04:26:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f6427640 1 -- 172.21.15.46:0/2622405924 learned_addr learned my addr 172.21.15.46:0/2622405924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f5c26640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f0107900 0x7f65f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f6427640 1 -- 172.21.15.46:0/2622405924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f0107900 msgr2=0x7f65f0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f6427640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f0107900 0x7f65f0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f6427640 1 -- 172.21.15.46:0/2622405924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f0074f70 con 0x7f65f0101900 2024-09-10T04:26:06.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f5c26640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f0107900 0x7f65f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:26:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f6427640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f00744f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00d6f0 tx=0x7f65ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65e77fe640 1 -- 172.21.15.46:0/2622405924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f65ec017070 con 0x7f65f0101900 2024-09-10T04:26:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f0075260 con 0x7f65f0101900 2024-09-10T04:26:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65e77fe640 1 -- 172.21.15.46:0/2622405924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f65ec00de00 con 0x7f65f0101900 2024-09-10T04:26:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65e77fe640 1 -- 172.21.15.46:0/2622405924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f65ec004bb0 con 0x7f65f0101900 2024-09-10T04:26:06.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.665+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f01a3e60 con 0x7f65f0101900 2024-09-10T04:26:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.666+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f0103040 con 0x7f65f0101900 2024-09-10T04:26:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.667+0000 7f65e77fe640 1 -- 172.21.15.46:0/2622405924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f65ec004e60 con 0x7f65f0101900 2024-09-10T04:26:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.668+0000 7f65e77fe640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65dc07abf0 0x7f65dc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.668+0000 7f65f5c26640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65dc07abf0 0x7f65dc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.668+0000 7f65e77fe640 1 -- 172.21.15.46:0/2622405924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f65ec09d8d0 con 0x7f65f0101900 2024-09-10T04:26:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.668+0000 7f65f5c26640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65dc07abf0 0x7f65dc07d0b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006010 tx=0x7f65e00095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:06.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.672+0000 7f65e77fe640 1 -- 172.21.15.46:0/2622405924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f65ec069010 con 0x7f65f0101900 2024-09-10T04:26:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:06 smithi070 ceph-mon[79774]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:06 smithi070 ceph-mon[79774]: from='client.34986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:06 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/601100743' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:26:06.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.877+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f65f0109ee0 con 0x7f65dc07abf0 2024-09-10T04:26:06.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.878+0000 7f65e77fe640 1 -- 172.21.15.46:0/2622405924 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f65e0004360 con 0x7f65dc07abf0 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:26:06.880 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:26:06.881 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:26:06.881 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:26:06.881 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:26:06.881 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:26:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65dc07abf0 msgr2=0x7f65dc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65dc07abf0 0x7f65dc07d0b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006010 tx=0x7f65e00095f0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 msgr2=0x7f65f00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f00744f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f65ec00d6f0 tx=0x7f65ec00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 shutdown_connections 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65dc07abf0 0x7f65dc07d0b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65f0107900 0x7f65f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 --2- 172.21.15.46:0/2622405924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f0101900 0x7f65f00744f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 >> 172.21.15.46:0/2622405924 conn(0x7f65f00fd600 msgr2=0x7f65f00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 shutdown_connections 2024-09-10T04:26:06.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.880+0000 7f65f868b640 1 -- 172.21.15.46:0/2622405924 wait complete. 2024-09-10T04:26:06.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 -- 172.21.15.46:0/1871946970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4102b50 msgr2=0x7f6db4102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/1871946970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4102b50 0x7f6db4102f30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009880 tx=0x7f6da402a530 comp rx=0 tx=0).stop 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 -- 172.21.15.46:0/1871946970 shutdown_connections 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/1871946970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db4103470 0x7f6db410b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/1871946970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4102b50 0x7f6db4102f30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 -- 172.21.15.46:0/1871946970 >> 172.21.15.46:0/1871946970 conn(0x7f6db40fc550 msgr2=0x7f6db40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 -- 172.21.15.46:0/1871946970 shutdown_connections 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 -- 172.21.15.46:0/1871946970 wait complete. 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 Processor -- start 2024-09-10T04:26:06.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.965+0000 7f6dbaf16640 1 -- start start 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6dbaf16640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db4102b50 0x7f6db419d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6dbaf16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4103470 0x7f6db419db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6dbaf16640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6db410ede0 con 0x7f6db4103470 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6dbaf16640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6db410ec60 con 0x7f6db4102b50 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4103470 0x7f6db419db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4103470 0x7f6db419db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51214/0 (socket says 172.21.15.46:51214) 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db3fff640 1 -- 172.21.15.46:0/3957410739 learned_addr learned my addr 172.21.15.46:0/3957410739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db8cb2640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db4102b50 0x7f6db419d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db3fff640 1 -- 172.21.15.46:0/3957410739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db4102b50 msgr2=0x7f6db419d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db3fff640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db4102b50 0x7f6db419d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db3fff640 1 -- 172.21.15.46:0/3957410739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db41977d0 con 0x7f6db4103470 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db3fff640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4103470 0x7f6db419db90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6da800d6f0 tx=0x7f6da800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:06.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db1ffb640 1 -- 172.21.15.46:0/3957410739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6da8017070 con 0x7f6db4103470 2024-09-10T04:26:06.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db1ffb640 1 -- 172.21.15.46:0/3957410739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6da800de00 con 0x7f6db4103470 2024-09-10T04:26:06.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db4197ac0 con 0x7f6db4103470 2024-09-10T04:26:06.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.966+0000 7f6db1ffb640 1 -- 172.21.15.46:0/3957410739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6da8004bb0 con 0x7f6db4103470 2024-09-10T04:26:06.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.967+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db4198050 con 0x7f6db4103470 2024-09-10T04:26:06.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.968+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db406a4e0 con 0x7f6db4103470 2024-09-10T04:26:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.968+0000 7f6db1ffb640 1 -- 172.21.15.46:0/3957410739 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6da8004e60 con 0x7f6db4103470 2024-09-10T04:26:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.969+0000 7f6db1ffb640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d9c07abf0 0x7f6d9c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.969+0000 7f6db1ffb640 1 -- 172.21.15.46:0/3957410739 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f6da809d8c0 con 0x7f6db4103470 2024-09-10T04:26:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.969+0000 7f6db8cb2640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d9c07abf0 0x7f6d9c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:06.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.969+0000 7f6db8cb2640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d9c07abf0 0x7f6d9c07d0b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6da40096c0 tx=0x7f6da4004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:06.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:06.973+0000 7f6db1ffb640 1 -- 172.21.15.46:0/3957410739 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6da8068f60 con 0x7f6db4103470 2024-09-10T04:26:07.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.235+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6db4105fb0 con 0x7f6db4103470 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.236+0000 7f6db1ffb640 1 -- 172.21.15.46:0/3957410739 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f6da806de10 con 0x7f6db4103470 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:26:07.238 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:26:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d9c07abf0 msgr2=0x7f6d9c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d9c07abf0 0x7f6d9c07d0b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6da40096c0 tx=0x7f6da4004f30 comp rx=0 tx=0).stop 2024-09-10T04:26:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4103470 msgr2=0x7f6db419db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4103470 0x7f6db419db90 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6da800d6f0 tx=0x7f6da800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:26:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 shutdown_connections 2024-09-10T04:26:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d9c07abf0 0x7f6d9c07d0b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6db4103470 0x7f6db419db90 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 --2- 172.21.15.46:0/3957410739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6db4102b50 0x7f6db419d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 >> 172.21.15.46:0/3957410739 conn(0x7f6db40fc550 msgr2=0x7f6db4108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 shutdown_connections 2024-09-10T04:26:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:07.238+0000 7f6dbaf16640 1 -- 172.21.15.46:0/3957410739 wait complete. 2024-09-10T04:26:07.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:07 smithi046 ceph-mon[109506]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:07.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:07 smithi046 ceph-mon[109506]: pgmap v613: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:07.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:07 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1774766138' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:26:07.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:07 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3957410739' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:26:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:07 smithi070 ceph-mon[79774]: from='client.34990 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:07 smithi070 ceph-mon[79774]: pgmap v613: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:07 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1774766138' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:26:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:07 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3957410739' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:26:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:08 smithi046 ceph-mon[109506]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:08 smithi070 ceph-mon[79774]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:09 smithi070 ceph-mon[79774]: pgmap v614: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:09.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:09 smithi046 ceph-mon[109506]: pgmap v614: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:11.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:11 smithi046 ceph-mon[109506]: pgmap v615: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:11 smithi070 ceph-mon[79774]: pgmap v615: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:13 smithi070 ceph-mon[79774]: pgmap v616: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:13 smithi046 ceph-mon[109506]: pgmap v616: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:15 smithi070 ceph-mon[79774]: pgmap v617: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:15 smithi046 ceph-mon[109506]: pgmap v617: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:17 smithi046 ceph-mon[109506]: pgmap v618: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:17 smithi070 ceph-mon[79774]: pgmap v618: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:19.809 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:19 smithi070 ceph-mon[79774]: pgmap v619: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:19.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:19 smithi046 ceph-mon[109506]: pgmap v619: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:21 smithi070 ceph-mon[79774]: pgmap v620: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:21 smithi046 ceph-mon[109506]: pgmap v620: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:23.563 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:23 smithi046 ceph-mon[109506]: pgmap v621: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:23.563 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:23 smithi070 ceph-mon[79774]: pgmap v621: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:25 smithi070 ceph-mon[79774]: pgmap v622: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:25 smithi046 ceph-mon[109506]: pgmap v622: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:27 smithi070 ceph-mon[79774]: pgmap v623: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:27 smithi046 ceph-mon[109506]: pgmap v623: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:29.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:29 smithi046 ceph-mon[109506]: pgmap v624: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:29 smithi070 ceph-mon[79774]: pgmap v624: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:31 smithi070 ceph-mon[79774]: pgmap v625: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:31 smithi046 ceph-mon[109506]: pgmap v625: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:33 smithi070 ceph-mon[79774]: pgmap v626: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:33 smithi046 ceph-mon[109506]: pgmap v626: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:35.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:35 smithi046 ceph-mon[109506]: pgmap v627: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:35 smithi070 ceph-mon[79774]: pgmap v627: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 -- 172.21.15.46:0/25123324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 msgr2=0x7f85b0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 --2- 172.21.15.46:0/25123324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b0101d60 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009880 tx=0x7f85a402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 -- 172.21.15.46:0/25123324 shutdown_connections 2024-09-10T04:26:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 --2- 172.21.15.46:0/25123324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b0101d60 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 --2- 172.21.15.46:0/25123324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b0107900 0x7f85b0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 -- 172.21.15.46:0/25123324 >> 172.21.15.46:0/25123324 conn(0x7f85b00fd600 msgr2=0x7f85b00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 -- 172.21.15.46:0/25123324 shutdown_connections 2024-09-10T04:26:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.337+0000 7f85b5715640 1 -- 172.21.15.46:0/25123324 wait complete. 2024-09-10T04:26:37.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85b5715640 1 Processor -- start 2024-09-10T04:26:37.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85b5715640 1 -- start start 2024-09-10T04:26:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85b5715640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85b5715640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b0107900 0x7f85b019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85b5715640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010edd0 con 0x7f85b0101900 2024-09-10T04:26:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85b5715640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f85b010ec50 con 0x7f85b0107900 2024-09-10T04:26:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85aeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85aeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35980/0 (socket says 172.21.15.46:35980) 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.338+0000 7f85aeffd640 1 -- 172.21.15.46:0/1079671087 learned_addr learned my addr 172.21.15.46:0/1079671087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f85ae7fc640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b0107900 0x7f85b019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f85aeffd640 1 -- 172.21.15.46:0/1079671087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b0107900 msgr2=0x7f85b019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f85aeffd640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b0107900 0x7f85b019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f85aeffd640 1 -- 172.21.15.46:0/1079671087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b01977a0 con 0x7f85b0101900 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f85aeffd640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b019d620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85a000d6f0 tx=0x7f85a000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f8597fff640 1 -- 172.21.15.46:0/1079671087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85a0017070 con 0x7f85b0101900 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b0197a90 con 0x7f85b0101900 2024-09-10T04:26:37.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f8597fff640 1 -- 172.21.15.46:0/1079671087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85a000de00 con 0x7f85b0101900 2024-09-10T04:26:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f8597fff640 1 -- 172.21.15.46:0/1079671087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f85a0004bb0 con 0x7f85b0101900 2024-09-10T04:26:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.339+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b0197ff0 con 0x7f85b0101900 2024-09-10T04:26:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.340+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b0103040 con 0x7f85b0101900 2024-09-10T04:26:37.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.341+0000 7f8597fff640 1 -- 172.21.15.46:0/1079671087 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f85a0004e60 con 0x7f85b0101900 2024-09-10T04:26:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.342+0000 7f8597fff640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f859807abf0 0x7f859807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.342+0000 7f8597fff640 1 -- 172.21.15.46:0/1079671087 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f85a009d820 con 0x7f85b0101900 2024-09-10T04:26:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.342+0000 7f85ae7fc640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f859807abf0 0x7f859807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.342+0000 7f85ae7fc640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f859807abf0 0x7f859807d0b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85b0198af0 tx=0x7f85a4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:37.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.346+0000 7f8597fff640 1 -- 172.21.15.46:0/1079671087 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f85a0068f60 con 0x7f85b0101900 2024-09-10T04:26:37.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.550+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f85b0198360 con 0x7f859807abf0 2024-09-10T04:26:37.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.551+0000 7f8597fff640 1 -- 172.21.15.46:0/1079671087 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f85b0198360 con 0x7f859807abf0 2024-09-10T04:26:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f859807abf0 msgr2=0x7f859807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f859807abf0 0x7f859807d0b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f85b0198af0 tx=0x7f85a4031400 comp rx=0 tx=0).stop 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 msgr2=0x7f85b019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b019d620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f85a000d6f0 tx=0x7f85a000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 shutdown_connections 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f859807abf0 0x7f859807d0b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85b0107900 0x7f85b019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 --2- 172.21.15.46:0/1079671087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85b0101900 0x7f85b019d620 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.553+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 >> 172.21.15.46:0/1079671087 conn(0x7f85b00fd600 msgr2=0x7f85b00fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.554+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 shutdown_connections 2024-09-10T04:26:37.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.554+0000 7f85b5715640 1 -- 172.21.15.46:0/1079671087 wait complete. 2024-09-10T04:26:37.564 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:26:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 -- 172.21.15.46:0/2842913380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454107900 msgr2=0x7f5454107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 --2- 172.21.15.46:0/2842913380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454107900 0x7f5454107ce0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5444009880 tx=0x7f544402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 -- 172.21.15.46:0/2842913380 shutdown_connections 2024-09-10T04:26:37.642 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:37 smithi046 ceph-mon[109506]: pgmap v628: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:37.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 --2- 172.21.15.46:0/2842913380 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5454101900 0x7f5454101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 --2- 172.21.15.46:0/2842913380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454107900 0x7f5454107ce0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 -- 172.21.15.46:0/2842913380 >> 172.21.15.46:0/2842913380 conn(0x7f54540fd600 msgr2=0x7f54540ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 -- 172.21.15.46:0/2842913380 shutdown_connections 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.639+0000 7f545a0f6640 1 -- 172.21.15.46:0/2842913380 wait complete. 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.640+0000 7f545a0f6640 1 Processor -- start 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.640+0000 7f545a0f6640 1 -- start start 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.640+0000 7f545a0f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454101900 0x7f54540744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f545a0f6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5454107900 0x7f5454074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f545a0f6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f545410edb0 con 0x7f5454101900 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f545a0f6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f545410ec30 con 0x7f5454107900 2024-09-10T04:26:37.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f54537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454101900 0x7f54540744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f54537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454101900 0x7f54540744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36002/0 (socket says 172.21.15.46:36002) 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f54537fe640 1 -- 172.21.15.46:0/4056588933 learned_addr learned my addr 172.21.15.46:0/4056588933 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f5452ffd640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5454107900 0x7f5454074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f54537fe640 1 -- 172.21.15.46:0/4056588933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5454107900 msgr2=0x7f5454074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f54537fe640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5454107900 0x7f5454074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f54537fe640 1 -- 172.21.15.46:0/4056588933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5454074f70 con 0x7f5454101900 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f54537fe640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454101900 0x7f54540744f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5444031460 tx=0x7f54440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f5450ff9640 1 -- 172.21.15.46:0/4056588933 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f544403f070 con 0x7f5454101900 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f5450ff9640 1 -- 172.21.15.46:0/4056588933 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5444004cf0 con 0x7f5454101900 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.642+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5454075200 con 0x7f5454101900 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.641+0000 7f5450ff9640 1 -- 172.21.15.46:0/4056588933 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5444035030 con 0x7f5454101900 2024-09-10T04:26:37.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.642+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54541a3d50 con 0x7f5454101900 2024-09-10T04:26:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.643+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5454103040 con 0x7f5454101900 2024-09-10T04:26:37.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.643+0000 7f5450ff9640 1 -- 172.21.15.46:0/4056588933 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5444042070 con 0x7f5454101900 2024-09-10T04:26:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.644+0000 7f5450ff9640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f543c07ac40 0x7f543c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.644+0000 7f5452ffd640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f543c07ac40 0x7f543c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.644+0000 7f5450ff9640 1 -- 172.21.15.46:0/4056588933 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f54440be680 con 0x7f5454101900 2024-09-10T04:26:37.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.645+0000 7f5452ffd640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f543c07ac40 0x7f543c07d100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5448005e10 tx=0x7f5448005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:37.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.649+0000 7f5450ff9640 1 -- 172.21.15.46:0/4056588933 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5444089e40 con 0x7f5454101900 2024-09-10T04:26:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:37 smithi070 ceph-mon[79774]: pgmap v628: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:37.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.852+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5454104b60 con 0x7f543c07ac40 2024-09-10T04:26:37.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.853+0000 7f5450ff9640 1 -- 172.21.15.46:0/4056588933 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f5454104b60 con 0x7f543c07ac40 2024-09-10T04:26:37.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f543c07ac40 msgr2=0x7f543c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f543c07ac40 0x7f543c07d100 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5448005e10 tx=0x7f5448005da0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454101900 msgr2=0x7f54540744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454101900 0x7f54540744f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5444031460 tx=0x7f54440048f0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 shutdown_connections 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f543c07ac40 0x7f543c07d100 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5454107900 0x7f5454074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 --2- 172.21.15.46:0/4056588933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5454101900 0x7f54540744f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 >> 172.21.15.46:0/4056588933 conn(0x7f54540fd600 msgr2=0x7f54540fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 shutdown_connections 2024-09-10T04:26:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.856+0000 7f545a0f6640 1 -- 172.21.15.46:0/4056588933 wait complete. 2024-09-10T04:26:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.940+0000 7f738a62b640 1 -- 172.21.15.46:0/2146161090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 msgr2=0x7f738410b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.940+0000 7f738a62b640 1 --2- 172.21.15.46:0/2146161090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738410b930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f73780098b0 tx=0x7f737802a560 comp rx=0 tx=0).stop 2024-09-10T04:26:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.940+0000 7f738a62b640 1 -- 172.21.15.46:0/2146161090 shutdown_connections 2024-09-10T04:26:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.940+0000 7f738a62b640 1 --2- 172.21.15.46:0/2146161090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738410b930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.940+0000 7f738a62b640 1 --2- 172.21.15.46:0/2146161090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7384102b20 0x7f7384102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.940+0000 7f738a62b640 1 -- 172.21.15.46:0/2146161090 >> 172.21.15.46:0/2146161090 conn(0x7f73840fc520 msgr2=0x7f73840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.941+0000 7f738a62b640 1 -- 172.21.15.46:0/2146161090 shutdown_connections 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.941+0000 7f738a62b640 1 -- 172.21.15.46:0/2146161090 wait complete. 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.941+0000 7f738a62b640 1 Processor -- start 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.941+0000 7f738a62b640 1 -- start start 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f738a62b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7384102b20 0x7f738419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f738a62b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f738a62b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f738410edf0 con 0x7f7384103440 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f738a62b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f738410ec70 con 0x7f7384102b20 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f73837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f73837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36008/0 (socket says 172.21.15.46:36008) 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f73837fe640 1 -- 172.21.15.46:0/817510659 learned_addr learned my addr 172.21.15.46:0/817510659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f7383fff640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7384102b20 0x7f738419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f73837fe640 1 -- 172.21.15.46:0/817510659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7384102b20 msgr2=0x7f738419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f73837fe640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7384102b20 0x7f738419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f73837fe640 1 -- 172.21.15.46:0/817510659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7384199a40 con 0x7f7384103440 2024-09-10T04:26:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.942+0000 7f73837fe640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738419fdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7378031460 tx=0x7f73780045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.943+0000 7f73817fa640 1 -- 172.21.15.46:0/817510659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f737803f070 con 0x7f7384103440 2024-09-10T04:26:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.943+0000 7f73817fa640 1 -- 172.21.15.46:0/817510659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f73780035f0 con 0x7f7384103440 2024-09-10T04:26:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.943+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7384199cd0 con 0x7f7384103440 2024-09-10T04:26:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.943+0000 7f73817fa640 1 -- 172.21.15.46:0/817510659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7378035030 con 0x7f7384103440 2024-09-10T04:26:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.943+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f738419a200 con 0x7f7384103440 2024-09-10T04:26:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.944+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f738406aa30 con 0x7f7384103440 2024-09-10T04:26:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.945+0000 7f73817fa640 1 -- 172.21.15.46:0/817510659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7378042070 con 0x7f7384103440 2024-09-10T04:26:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.945+0000 7f73817fa640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f736c07ac40 0x7f736c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.945+0000 7f73817fa640 1 -- 172.21.15.46:0/817510659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f73780be480 con 0x7f7384103440 2024-09-10T04:26:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.945+0000 7f7383fff640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f736c07ac40 0x7f736c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.946+0000 7f7383fff640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f736c07ac40 0x7f736c07d100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f738419ab80 tx=0x7f7374005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:37.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:37.950+0000 7f73817fa640 1 -- 172.21.15.46:0/817510659 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7378089c40 con 0x7f7384103440 2024-09-10T04:26:38.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.143+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f738419af70 con 0x7f736c07ac40 2024-09-10T04:26:38.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.152+0000 7f73817fa640 1 -- 172.21.15.46:0/817510659 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f738419af70 con 0x7f736c07ac40 2024-09-10T04:26:38.154 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:26:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (18m) 12m ago 29m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:26:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (20m) 12m ago 29m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:26:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (20m) 12m ago 28m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:26:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (16m) 12m ago 29m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:26:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (16m) 12m ago 28m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:26:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (17m) 12m ago 29m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:26:38.155 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (12m) 12m ago 24m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (13m) 12m ago 24m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (12m) 12m ago 24m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (24m) 12m ago 24m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (20m) 12m ago 30m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (19m) 12m ago 28m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (17m) 12m ago 31m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (16m) 12m ago 28m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (19m) 12m ago 29m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (18m) 12m ago 28m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (16m) 12m ago 27m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:26:38.156 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (15m) 12m ago 27m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:26:38.157 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (15m) 12m ago 26m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:26:38.157 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (14m) 12m ago 26m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:26:38.157 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (13m) 12m ago 25m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:26:38.157 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (13m) 12m ago 25m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:26:38.157 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (18m) 12m ago 29m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:26:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.154+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f736c07ac40 msgr2=0x7f736c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.154+0000 7f738a62b640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f736c07ac40 0x7f736c07d100 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f738419ab80 tx=0x7f7374005f90 comp rx=0 tx=0).stop 2024-09-10T04:26:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 msgr2=0x7f738419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738419fdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7378031460 tx=0x7f73780045e0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 shutdown_connections 2024-09-10T04:26:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f736c07ac40 0x7f736c07d100 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7384103440 0x7f738419fdc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 --2- 172.21.15.46:0/817510659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7384102b20 0x7f738419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 >> 172.21.15.46:0/817510659 conn(0x7f73840fc520 msgr2=0x7f7384100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 shutdown_connections 2024-09-10T04:26:38.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.155+0000 7f738a62b640 1 -- 172.21.15.46:0/817510659 wait complete. 2024-09-10T04:26:38.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 -- 172.21.15.46:0/1599838680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 msgr2=0x7f7308100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 --2- 172.21.15.46:0/1599838680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f7308100c50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009880 tx=0x7f72f802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 -- 172.21.15.46:0/1599838680 shutdown_connections 2024-09-10T04:26:38.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 --2- 172.21.15.46:0/1599838680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f7308100c50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 --2- 172.21.15.46:0/1599838680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081067f0 0x7f7308106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 -- 172.21.15.46:0/1599838680 >> 172.21.15.46:0/1599838680 conn(0x7f73080fc550 msgr2=0x7f73080fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:38.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 -- 172.21.15.46:0/1599838680 shutdown_connections 2024-09-10T04:26:38.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 -- 172.21.15.46:0/1599838680 wait complete. 2024-09-10T04:26:38.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.238+0000 7f730fe98640 1 Processor -- start 2024-09-10T04:26:38.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730fe98640 1 -- start start 2024-09-10T04:26:38.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730fe98640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f730819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730fe98640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081067f0 0x7f730819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730fe98640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f730810dcc0 con 0x7f73081007f0 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730fe98640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f730810db40 con 0x7f73081067f0 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730dc34640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f730819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730dc34640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f730819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36022/0 (socket says 172.21.15.46:36022) 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730dc34640 1 -- 172.21.15.46:0/2136017581 learned_addr learned my addr 172.21.15.46:0/2136017581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730d433640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081067f0 0x7f730819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730dc34640 1 -- 172.21.15.46:0/2136017581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081067f0 msgr2=0x7f730819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730dc34640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081067f0 0x7f730819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.239+0000 7f730dc34640 1 -- 172.21.15.46:0/2136017581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73081977e0 con 0x7f73081007f0 2024-09-10T04:26:38.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.240+0000 7f730dc34640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f730819d660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f730400d6f0 tx=0x7f730400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.240+0000 7f72feffd640 1 -- 172.21.15.46:0/2136017581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7304017070 con 0x7f73081007f0 2024-09-10T04:26:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.240+0000 7f72feffd640 1 -- 172.21.15.46:0/2136017581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f730400de00 con 0x7f73081007f0 2024-09-10T04:26:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.240+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7308197ad0 con 0x7f73081007f0 2024-09-10T04:26:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.240+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7308198030 con 0x7f73081007f0 2024-09-10T04:26:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.241+0000 7f72feffd640 1 -- 172.21.15.46:0/2136017581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7304004bb0 con 0x7f73081007f0 2024-09-10T04:26:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.241+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7308101f30 con 0x7f73081007f0 2024-09-10T04:26:38.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.242+0000 7f72feffd640 1 -- 172.21.15.46:0/2136017581 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7304009a30 con 0x7f73081007f0 2024-09-10T04:26:38.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.242+0000 7f72feffd640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f407a9f0 0x7f72f407ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.243+0000 7f72feffd640 1 -- 172.21.15.46:0/2136017581 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f730409d760 con 0x7f73081007f0 2024-09-10T04:26:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.243+0000 7f730d433640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f407a9f0 0x7f72f407ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.243+0000 7f730d433640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f407a9f0 0x7f72f407ceb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7308198b30 tx=0x7f72f8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:38.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.247+0000 7f72feffd640 1 -- 172.21.15.46:0/2136017581 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7304068f20 con 0x7f73081007f0 2024-09-10T04:26:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.508+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7308198d10 con 0x7f73081007f0 2024-09-10T04:26:38.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.509+0000 7f72feffd640 1 -- 172.21.15.46:0/2136017581 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f730406ddd0 con 0x7f73081007f0 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:26:38.511 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:26:38.512 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:26:38.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:26:38.512 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:26:38.512 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:26:38.512 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:26:38.512 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:26:38.512 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:26:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f407a9f0 msgr2=0x7f72f407ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f407a9f0 0x7f72f407ceb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7308198b30 tx=0x7f72f8031400 comp rx=0 tx=0).stop 2024-09-10T04:26:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 msgr2=0x7f730819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f730819d660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f730400d6f0 tx=0x7f730400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 shutdown_connections 2024-09-10T04:26:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72f407a9f0 0x7f72f407ceb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73081067f0 0x7f730819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 --2- 172.21.15.46:0/2136017581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73081007f0 0x7f730819d660 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 >> 172.21.15.46:0/2136017581 conn(0x7f73080fc550 msgr2=0x7f7308069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:38.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 shutdown_connections 2024-09-10T04:26:38.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.511+0000 7f730fe98640 1 -- 172.21.15.46:0/2136017581 wait complete. 2024-09-10T04:26:38.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 -- 172.21.15.46:0/2685612330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 msgr2=0x7f17b4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/2685612330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4106bd0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009880 tx=0x7f17b002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 -- 172.21.15.46:0/2685612330 shutdown_connections 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/2685612330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17b41007f0 0x7f17b4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/2685612330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4106bd0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 -- 172.21.15.46:0/2685612330 >> 172.21.15.46:0/2685612330 conn(0x7f17b40fc550 msgr2=0x7f17b40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 -- 172.21.15.46:0/2685612330 shutdown_connections 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 -- 172.21.15.46:0/2685612330 wait complete. 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.596+0000 7f17bc7e4640 1 Processor -- start 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17bc7e4640 1 -- start start 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17bc7e4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17b41007f0 0x7f17b4107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17bc7e4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17bc7e4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410dca0 con 0x7f17b41067f0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17bc7e4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410db20 con 0x7f17b41007f0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17b9d7f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17b9d7f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36036/0 (socket says 172.21.15.46:36036) 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17b9d7f640 1 -- 172.21.15.46:0/1766628261 learned_addr learned my addr 172.21.15.46:0/1766628261 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17ba580640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17b41007f0 0x7f17b4107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17b9d7f640 1 -- 172.21.15.46:0/1766628261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17b41007f0 msgr2=0x7f17b4107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17b9d7f640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17b41007f0 0x7f17b4107720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17b9d7f640 1 -- 172.21.15.46:0/1766628261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b41081a0 con 0x7f17b41067f0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.597+0000 7f17b9d7f640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4107c60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f17a400d8d0 tx=0x7f17a400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.598+0000 7f17ab7fe640 1 -- 172.21.15.46:0/1766628261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f17a4017070 con 0x7f17b41067f0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.598+0000 7f17ab7fe640 1 -- 172.21.15.46:0/1766628261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17a4010460 con 0x7f17b41067f0 2024-09-10T04:26:38.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.598+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b4108490 con 0x7f17b41067f0 2024-09-10T04:26:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.598+0000 7f17ab7fe640 1 -- 172.21.15.46:0/1766628261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f17a400f5f0 con 0x7f17b41067f0 2024-09-10T04:26:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.598+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b41a3dd0 con 0x7f17b41067f0 2024-09-10T04:26:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.599+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b4101f30 con 0x7f17b41067f0 2024-09-10T04:26:38.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.600+0000 7f17ab7fe640 1 -- 172.21.15.46:0/1766628261 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f17a4009a30 con 0x7f17b41067f0 2024-09-10T04:26:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.600+0000 7f17ab7fe640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17a007ac40 0x7f17a007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.600+0000 7f17ab7fe640 1 -- 172.21.15.46:0/1766628261 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f17a409da90 con 0x7f17b41067f0 2024-09-10T04:26:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.600+0000 7f17ba580640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17a007ac40 0x7f17a007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.601+0000 7f17ba580640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17a007ac40 0x7f17a007d100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17b00096c0 tx=0x7f17b0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:38.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.605+0000 7f17ab7fe640 1 -- 172.21.15.46:0/1766628261 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f17a4069250 con 0x7f17b41067f0 2024-09-10T04:26:38.794 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:38 smithi046 ceph-mon[109506]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:38.795 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:38 smithi070 ceph-mon[79774]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.855+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f17b4108c50 con 0x7f17b41067f0 2024-09-10T04:26:38.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.855+0000 7f17ab7fe640 1 -- 172.21.15.46:0/1766628261 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f17a406e100 con 0x7f17b41067f0 2024-09-10T04:26:38.857 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:26:38.857 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:26:38.857 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:26:38.857 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:26:38.858 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:26:38.859 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:38.860 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:26:38.861 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:26:38.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.857+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17a007ac40 msgr2=0x7f17a007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.857+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17a007ac40 0x7f17a007d100 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f17b00096c0 tx=0x7f17b0031400 comp rx=0 tx=0).stop 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.857+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 msgr2=0x7f17b4107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.857+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4107c60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f17a400d8d0 tx=0x7f17a400dd90 comp rx=0 tx=0).stop 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.858+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 shutdown_connections 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.858+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f17a007ac40 0x7f17a007d100 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.858+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17b41067f0 0x7f17b4107c60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.858+0000 7f17bc7e4640 1 --2- 172.21.15.46:0/1766628261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17b41007f0 0x7f17b4107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.858+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 >> 172.21.15.46:0/1766628261 conn(0x7f17b40fc550 msgr2=0x7f17b4069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.858+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 shutdown_connections 2024-09-10T04:26:38.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.858+0000 7f17bc7e4640 1 -- 172.21.15.46:0/1766628261 wait complete. 2024-09-10T04:26:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 -- 172.21.15.46:0/3368371174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb014102ac0 msgr2=0x7fb014102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 --2- 172.21.15.46:0/3368371174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb014102ac0 0x7fb014102ea0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb004009880 tx=0x7fb00402a530 comp rx=0 tx=0).stop 2024-09-10T04:26:38.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 -- 172.21.15.46:0/3368371174 shutdown_connections 2024-09-10T04:26:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 --2- 172.21.15.46:0/3368371174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0141033e0 0x7fb01410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 --2- 172.21.15.46:0/3368371174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb014102ac0 0x7fb014102ea0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 -- 172.21.15.46:0/3368371174 >> 172.21.15.46:0/3368371174 conn(0x7fb0140fc520 msgr2=0x7fb0140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 -- 172.21.15.46:0/3368371174 shutdown_connections 2024-09-10T04:26:38.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.946+0000 7fb01a205640 1 -- 172.21.15.46:0/3368371174 wait complete. 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.947+0000 7fb01a205640 1 Processor -- start 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.947+0000 7fb01a205640 1 -- start start 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.947+0000 7fb01a205640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb014102ac0 0x7fb01419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.947+0000 7fb01a205640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0141033e0 0x7fb01419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.947+0000 7fb01a205640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb01410edf0 con 0x7fb0141033e0 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.947+0000 7fb01a205640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb01410ec70 con 0x7fb014102ac0 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb012ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0141033e0 0x7fb01419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb012ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0141033e0 0x7fb01419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60986/0 (socket says 172.21.15.46:60986) 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb012ffd640 1 -- 172.21.15.46:0/4157348740 learned_addr learned my addr 172.21.15.46:0/4157348740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb0137fe640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb014102ac0 0x7fb01419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb012ffd640 1 -- 172.21.15.46:0/4157348740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb014102ac0 msgr2=0x7fb01419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb012ffd640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb014102ac0 0x7fb01419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb012ffd640 1 -- 172.21.15.46:0/4157348740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0141977c0 con 0x7fb0141033e0 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb012ffd640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0141033e0 0x7fb01419db80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb00800e970 tx=0x7fb00800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb010ff9640 1 -- 172.21.15.46:0/4157348740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb00800cac0 con 0x7fb0141033e0 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb010ff9640 1 -- 172.21.15.46:0/4157348740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb0080054e0 con 0x7fb0141033e0 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb014197ab0 con 0x7fb0141033e0 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb010ff9640 1 -- 172.21.15.46:0/4157348740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb008004dd0 con 0x7fb0141033e0 2024-09-10T04:26:38.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.948+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb014198040 con 0x7fb0141033e0 2024-09-10T04:26:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.949+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01406aa90 con 0x7fb0141033e0 2024-09-10T04:26:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.950+0000 7fb010ff9640 1 -- 172.21.15.46:0/4157348740 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb008005020 con 0x7fb0141033e0 2024-09-10T04:26:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.950+0000 7fb010ff9640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faffc07ab90 0x7faffc07d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.951+0000 7fb010ff9640 1 -- 172.21.15.46:0/4157348740 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fb00809e290 con 0x7fb0141033e0 2024-09-10T04:26:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.951+0000 7fb0137fe640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faffc07ab90 0x7faffc07d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:38.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.951+0000 7fb0137fe640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faffc07ab90 0x7faffc07d050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb0040096c0 tx=0x7fb004031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:38.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:38.955+0000 7fb010ff9640 1 -- 172.21.15.46:0/4157348740 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb0080699d0 con 0x7fb0141033e0 2024-09-10T04:26:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.160+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb014198350 con 0x7faffc07ab90 2024-09-10T04:26:39.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.162+0000 7fb010ff9640 1 -- 172.21.15.46:0/4157348740 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fb014198350 con 0x7faffc07ab90 2024-09-10T04:26:39.163 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:26:39.164 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:26:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faffc07ab90 msgr2=0x7faffc07d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faffc07ab90 0x7faffc07d050 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb0040096c0 tx=0x7fb004031400 comp rx=0 tx=0).stop 2024-09-10T04:26:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0141033e0 msgr2=0x7fb01419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0141033e0 0x7fb01419db80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb00800e970 tx=0x7fb00800ee30 comp rx=0 tx=0).stop 2024-09-10T04:26:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 shutdown_connections 2024-09-10T04:26:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faffc07ab90 0x7faffc07d050 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0141033e0 0x7fb01419db80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 --2- 172.21.15.46:0/4157348740 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb014102ac0 0x7fb01419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 >> 172.21.15.46:0/4157348740 conn(0x7fb0140fc520 msgr2=0x7fb01410af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:39.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 shutdown_connections 2024-09-10T04:26:39.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.164+0000 7fb01a205640 1 -- 172.21.15.46:0/4157348740 wait complete. 2024-09-10T04:26:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 -- 172.21.15.46:0/3983995487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 msgr2=0x7f5fcc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/3983995487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc100c50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009880 tx=0x7f5fc002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 -- 172.21.15.46:0/3983995487 shutdown_connections 2024-09-10T04:26:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/3983995487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc100c50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/3983995487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc1067f0 0x7f5fcc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 -- 172.21.15.46:0/3983995487 >> 172.21.15.46:0/3983995487 conn(0x7f5fcc0fc550 msgr2=0x7f5fcc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 -- 172.21.15.46:0/3983995487 shutdown_connections 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 -- 172.21.15.46:0/3983995487 wait complete. 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 Processor -- start 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.249+0000 7f5fd3b71640 1 -- start start 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd3b71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd3b71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc1067f0 0x7f5fcc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd3b71640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc10dca0 con 0x7f5fcc1007f0 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd3b71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc10db20 con 0x7f5fcc1067f0 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd190d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd190d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32772/0 (socket says 172.21.15.46:32772) 2024-09-10T04:26:39.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd190d640 1 -- 172.21.15.46:0/1049063286 learned_addr learned my addr 172.21.15.46:0/1049063286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:26:39.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd110c640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc1067f0 0x7f5fcc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:39.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd190d640 1 -- 172.21.15.46:0/1049063286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc1067f0 msgr2=0x7f5fcc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:39.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd190d640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc1067f0 0x7f5fcc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd190d640 1 -- 172.21.15.46:0/1049063286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc199930 con 0x7f5fcc1007f0 2024-09-10T04:26:39.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd110c640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc1067f0 0x7f5fcc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:26:39.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd190d640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc19f770 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e970 tx=0x7f5fc800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fbeffd640 1 -- 172.21.15.46:0/1049063286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5fc800cac0 con 0x7f5fcc1007f0 2024-09-10T04:26:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fbeffd640 1 -- 172.21.15.46:0/1049063286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5fc800cc60 con 0x7f5fcc1007f0 2024-09-10T04:26:39.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.250+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc199c20 con 0x7f5fcc1007f0 2024-09-10T04:26:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.251+0000 7f5fbeffd640 1 -- 172.21.15.46:0/1049063286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5fc8018720 con 0x7f5fcc1007f0 2024-09-10T04:26:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.251+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc19a180 con 0x7f5fcc1007f0 2024-09-10T04:26:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.252+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fcc101f30 con 0x7f5fcc1007f0 2024-09-10T04:26:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.252+0000 7f5fbeffd640 1 -- 172.21.15.46:0/1049063286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5fc8010430 con 0x7f5fcc1007f0 2024-09-10T04:26:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.253+0000 7f5fbeffd640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fb807ac40 0x7f5fb807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:26:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.253+0000 7f5fd110c640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fb807ac40 0x7f5fb807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:26:39.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.253+0000 7f5fbeffd640 1 -- 172.21.15.46:0/1049063286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f5fc809e280 con 0x7f5fcc1007f0 2024-09-10T04:26:39.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.254+0000 7f5fd110c640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fb807ac40 0x7f5fb807d100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc19b120 tx=0x7f5fc0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:26:39.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.258+0000 7f5fbeffd640 1 -- 172.21.15.46:0/1049063286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5fc8069a40 con 0x7f5fcc1007f0 2024-09-10T04:26:39.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.523+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f5fcc19a430 con 0x7f5fcc1007f0 2024-09-10T04:26:39.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.524+0000 7f5fbeffd640 1 -- 172.21.15.46:0/1049063286 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f5fc806e8f0 con 0x7f5fcc1007f0 2024-09-10T04:26:39.526 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:26:39.526 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:26:39.526 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:26:39.526 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:26:39.526 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:26:39.526 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:26:39.526 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:26:39.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.525+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fb807ac40 msgr2=0x7f5fb807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.525+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fb807ac40 0x7f5fb807d100 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc19b120 tx=0x7f5fc0031400 comp rx=0 tx=0).stop 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 msgr2=0x7f5fcc19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc19f770 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800e970 tx=0x7f5fc800ee30 comp rx=0 tx=0).stop 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 shutdown_connections 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5fb807ac40 0x7f5fb807d100 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5fcc1067f0 0x7f5fcc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 --2- 172.21.15.46:0/1049063286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fcc1007f0 0x7f5fcc19f770 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 >> 172.21.15.46:0/1049063286 conn(0x7f5fcc0fc550 msgr2=0x7f5fcc069c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 shutdown_connections 2024-09-10T04:26:39.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:26:39.526+0000 7f5fd3b71640 1 -- 172.21.15.46:0/1049063286 wait complete. 2024-09-10T04:26:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:39 smithi070 ceph-mon[79774]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:39 smithi070 ceph-mon[79774]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:39 smithi070 ceph-mon[79774]: pgmap v629: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:39 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2136017581' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:26:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:39 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1766628261' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:26:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:39 smithi046 ceph-mon[109506]: from='client.35014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:39 smithi046 ceph-mon[109506]: from='client.35018 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:39 smithi046 ceph-mon[109506]: pgmap v629: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:39 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2136017581' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:26:39.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:39 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1766628261' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:26:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:40 smithi070 ceph-mon[79774]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:40 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1049063286' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:26:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:40 smithi046 ceph-mon[109506]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:26:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:40 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1049063286' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:26:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:41 smithi046 ceph-mon[109506]: pgmap v630: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:41 smithi070 ceph-mon[79774]: pgmap v630: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:43 smithi070 ceph-mon[79774]: pgmap v631: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:43 smithi046 ceph-mon[109506]: pgmap v631: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:45 smithi070 ceph-mon[79774]: pgmap v632: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:45.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:45 smithi046 ceph-mon[109506]: pgmap v632: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:47.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:47 smithi046 ceph-mon[109506]: pgmap v633: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:47 smithi070 ceph-mon[79774]: pgmap v633: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:49 smithi070 ceph-mon[79774]: pgmap v634: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:49.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:49 smithi046 ceph-mon[109506]: pgmap v634: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:51 smithi070 ceph-mon[79774]: pgmap v635: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:51 smithi046 ceph-mon[109506]: pgmap v635: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:53 smithi046 ceph-mon[109506]: pgmap v636: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:53 smithi070 ceph-mon[79774]: pgmap v636: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:26:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:55 smithi070 ceph-mon[79774]: pgmap v637: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:55 smithi046 ceph-mon[109506]: pgmap v637: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:57 smithi070 ceph-mon[79774]: pgmap v638: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:57 smithi046 ceph-mon[109506]: pgmap v638: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:26:59 smithi070 ceph-mon[79774]: pgmap v639: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:26:59.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:26:59 smithi046 ceph-mon[109506]: pgmap v639: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:01 smithi070 ceph-mon[79774]: pgmap v640: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:01 smithi046 ceph-mon[109506]: pgmap v640: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:03 smithi070 ceph-mon[79774]: pgmap v641: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:03 smithi046 ceph-mon[109506]: pgmap v641: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:05.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:05 smithi046 ceph-mon[109506]: pgmap v642: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:05 smithi070 ceph-mon[79774]: pgmap v642: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:07.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:07 smithi070 ceph-mon[79774]: pgmap v643: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:07.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:07 smithi046 ceph-mon[109506]: pgmap v643: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:09.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 -- 172.21.15.46:0/237363155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 msgr2=0x7fc66c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 --2- 172.21.15.46:0/237363155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c101d60 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc660009880 tx=0x7fc66002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 -- 172.21.15.46:0/237363155 shutdown_connections 2024-09-10T04:27:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 --2- 172.21.15.46:0/237363155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c101d60 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 --2- 172.21.15.46:0/237363155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c107900 0x7fc66c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 -- 172.21.15.46:0/237363155 >> 172.21.15.46:0/237363155 conn(0x7fc66c0fd5e0 msgr2=0x7fc66c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 -- 172.21.15.46:0/237363155 shutdown_connections 2024-09-10T04:27:09.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.621+0000 7fc670a74640 1 -- 172.21.15.46:0/237363155 wait complete. 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.622+0000 7fc670a74640 1 Processor -- start 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.622+0000 7fc670a74640 1 -- start start 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.622+0000 7fc670a74640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.622+0000 7fc670a74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c107900 0x7fc66c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.622+0000 7fc670a74640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c10edb0 con 0x7fc66c101900 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.622+0000 7fc670a74640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c10ec30 con 0x7fc66c107900 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc66a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc66a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45418/0 (socket says 172.21.15.46:45418) 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc66a59c640 1 -- 172.21.15.46:0/4067431097 learned_addr learned my addr 172.21.15.46:0/4067431097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc669d9b640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c107900 0x7fc66c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc66a59c640 1 -- 172.21.15.46:0/4067431097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c107900 msgr2=0x7fc66c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc66a59c640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c107900 0x7fc66c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc66a59c640 1 -- 172.21.15.46:0/4067431097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c1a03e0 con 0x7fc66c101900 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc66a59c640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c19f7f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c8a0 tx=0x7fc65c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc65b7fe640 1 -- 172.21.15.46:0/4067431097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc65c0079b0 con 0x7fc66c101900 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc65b7fe640 1 -- 172.21.15.46:0/4067431097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc65c007b50 con 0x7fc66c101900 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c199ad0 con 0x7fc66c101900 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc65b7fe640 1 -- 172.21.15.46:0/4067431097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc65c018700 con 0x7fc66c101900 2024-09-10T04:27:09.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.623+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c19a030 con 0x7fc66c101900 2024-09-10T04:27:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.624+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c103040 con 0x7fc66c101900 2024-09-10T04:27:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.625+0000 7fc65b7fe640 1 -- 172.21.15.46:0/4067431097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc65c007d00 con 0x7fc66c101900 2024-09-10T04:27:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.626+0000 7fc65b7fe640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65407abf0 0x7fc65407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.626+0000 7fc65b7fe640 1 -- 172.21.15.46:0/4067431097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc65c09de90 con 0x7fc66c101900 2024-09-10T04:27:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.626+0000 7fc669d9b640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65407abf0 0x7fc65407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:09.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.626+0000 7fc669d9b640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65407abf0 0x7fc65407d0b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc66c19b1a0 tx=0x7fc660031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:09.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.630+0000 7fc65b7fe640 1 -- 172.21.15.46:0/4067431097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc65c014030 con 0x7fc66c101900 2024-09-10T04:27:09.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.833+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc66c19a370 con 0x7fc65407abf0 2024-09-10T04:27:09.834 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:09 smithi046 ceph-mon[109506]: pgmap v644: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:09.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.834+0000 7fc65b7fe640 1 -- 172.21.15.46:0/4067431097 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fc66c19a370 con 0x7fc65407abf0 2024-09-10T04:27:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:09 smithi070 ceph-mon[79774]: pgmap v644: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65407abf0 msgr2=0x7fc65407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65407abf0 0x7fc65407d0b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc66c19b1a0 tx=0x7fc660031400 comp rx=0 tx=0).stop 2024-09-10T04:27:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 msgr2=0x7fc66c19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c19f7f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00c8a0 tx=0x7fc65c00cd60 comp rx=0 tx=0).stop 2024-09-10T04:27:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 shutdown_connections 2024-09-10T04:27:09.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65407abf0 0x7fc65407d0b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c107900 0x7fc66c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 --2- 172.21.15.46:0/4067431097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c101900 0x7fc66c19f7f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 >> 172.21.15.46:0/4067431097 conn(0x7fc66c0fd5e0 msgr2=0x7fc66c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 shutdown_connections 2024-09-10T04:27:09.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.836+0000 7fc670a74640 1 -- 172.21.15.46:0/4067431097 wait complete. 2024-09-10T04:27:09.846 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:27:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 -- 172.21.15.46:0/2143040991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4101900 msgr2=0x7f32e4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2143040991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4101900 0x7f32e4101d60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f32cc009880 tx=0x7f32cc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 -- 172.21.15.46:0/2143040991 shutdown_connections 2024-09-10T04:27:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2143040991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4101900 0x7f32e4101d60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2143040991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4107900 0x7f32e4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 -- 172.21.15.46:0/2143040991 >> 172.21.15.46:0/2143040991 conn(0x7f32e40fd600 msgr2=0x7f32e40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 -- 172.21.15.46:0/2143040991 shutdown_connections 2024-09-10T04:27:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.923+0000 7f32e8fed640 1 -- 172.21.15.46:0/2143040991 wait complete. 2024-09-10T04:27:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.924+0000 7f32e8fed640 1 Processor -- start 2024-09-10T04:27:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.924+0000 7f32e8fed640 1 -- start start 2024-09-10T04:27:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.924+0000 7f32e8fed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4101900 0x7f32e419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.924+0000 7f32e8fed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4107900 0x7f32e419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.924+0000 7f32e8fed640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32e410edd0 con 0x7f32e4107900 2024-09-10T04:27:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.924+0000 7f32e8fed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f32e410ec50 con 0x7f32e4101900 2024-09-10T04:27:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4107900 0x7f32e419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32dbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4107900 0x7f32e419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45428/0 (socket says 172.21.15.46:45428) 2024-09-10T04:27:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32dbfff640 1 -- 172.21.15.46:0/2464534852 learned_addr learned my addr 172.21.15.46:0/2464534852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32e259c640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4101900 0x7f32e419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32dbfff640 1 -- 172.21.15.46:0/2464534852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4101900 msgr2=0x7f32e419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32dbfff640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4101900 0x7f32e419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32dbfff640 1 -- 172.21.15.46:0/2464534852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e41999d0 con 0x7f32e4107900 2024-09-10T04:27:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32dbfff640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4107900 0x7f32e419fd50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f32cc031460 tx=0x7f32cc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32db7fe640 1 -- 172.21.15.46:0/2464534852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f32cc03f070 con 0x7f32e4107900 2024-09-10T04:27:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32db7fe640 1 -- 172.21.15.46:0/2464534852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32cc0044d0 con 0x7f32e4107900 2024-09-10T04:27:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e4199c60 con 0x7f32e4107900 2024-09-10T04:27:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32db7fe640 1 -- 172.21.15.46:0/2464534852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f32cc035030 con 0x7f32e4107900 2024-09-10T04:27:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.925+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e419a190 con 0x7f32e4107900 2024-09-10T04:27:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.926+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e4103040 con 0x7f32e4107900 2024-09-10T04:27:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.927+0000 7f32db7fe640 1 -- 172.21.15.46:0/2464534852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f32cc042070 con 0x7f32e4107900 2024-09-10T04:27:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.927+0000 7f32db7fe640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f32d007ac50 0x7f32d007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.928+0000 7f32db7fe640 1 -- 172.21.15.46:0/2464534852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f32cc0be620 con 0x7f32e4107900 2024-09-10T04:27:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.928+0000 7f32e259c640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f32d007ac50 0x7f32d007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.928+0000 7f32e259c640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f32d007ac50 0x7f32d007d110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f32d40058c0 tx=0x7f32d400a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:09.932+0000 7f32db7fe640 1 -- 172.21.15.46:0/2464534852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f32cc004cf0 con 0x7f32e4107900 2024-09-10T04:27:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.136+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f32e419a440 con 0x7f32d007ac50 2024-09-10T04:27:10.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.137+0000 7f32db7fe640 1 -- 172.21.15.46:0/2464534852 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f32e419a440 con 0x7f32d007ac50 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f32d007ac50 msgr2=0x7f32d007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f32d007ac50 0x7f32d007d110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f32d40058c0 tx=0x7f32d400a250 comp rx=0 tx=0).stop 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4107900 msgr2=0x7f32e419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4107900 0x7f32e419fd50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f32cc031460 tx=0x7f32cc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 shutdown_connections 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f32d007ac50 0x7f32d007d110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32e4107900 0x7f32e419fd50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 --2- 172.21.15.46:0/2464534852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32e4101900 0x7f32e419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 >> 172.21.15.46:0/2464534852 conn(0x7f32e40fd600 msgr2=0x7f32e4074440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 shutdown_connections 2024-09-10T04:27:10.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.140+0000 7f32e8fed640 1 -- 172.21.15.46:0/2464534852 wait complete. 2024-09-10T04:27:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 -- 172.21.15.46:0/3659448695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10107900 msgr2=0x7f1b10107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 --2- 172.21.15.46:0/3659448695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10107900 0x7f1b10107ce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1b04009880 tx=0x7f1b0402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 -- 172.21.15.46:0/3659448695 shutdown_connections 2024-09-10T04:27:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 --2- 172.21.15.46:0/3659448695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b10101900 0x7f1b10101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 --2- 172.21.15.46:0/3659448695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10107900 0x7f1b10107ce0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 -- 172.21.15.46:0/3659448695 >> 172.21.15.46:0/3659448695 conn(0x7f1b100fd600 msgr2=0x7f1b100ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 -- 172.21.15.46:0/3659448695 shutdown_connections 2024-09-10T04:27:10.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.225+0000 7f1b16e21640 1 -- 172.21.15.46:0/3659448695 wait complete. 2024-09-10T04:27:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b16e21640 1 Processor -- start 2024-09-10T04:27:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b16e21640 1 -- start start 2024-09-10T04:27:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b16e21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10101900 0x7f1b100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b16e21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b10107900 0x7f1b10074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b16e21640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1010edb0 con 0x7f1b10101900 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b16e21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1010ec30 con 0x7f1b10107900 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b14bbd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10101900 0x7f1b100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b14bbd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10101900 0x7f1b100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45450/0 (socket says 172.21.15.46:45450) 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b14bbd640 1 -- 172.21.15.46:0/374531917 learned_addr learned my addr 172.21.15.46:0/374531917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.226+0000 7f1b0ffff640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b10107900 0x7f1b10074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b14bbd640 1 -- 172.21.15.46:0/374531917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b10107900 msgr2=0x7f1b10074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b14bbd640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b10107900 0x7f1b10074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b14bbd640 1 -- 172.21.15.46:0/374531917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b10074f70 con 0x7f1b10101900 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b0ffff640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b10107900 0x7f1b10074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b14bbd640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10101900 0x7f1b100744f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1b04031460 tx=0x7f1b040045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b0dffb640 1 -- 172.21.15.46:0/374531917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b0403f070 con 0x7f1b10101900 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b0dffb640 1 -- 172.21.15.46:0/374531917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b040035f0 con 0x7f1b10101900 2024-09-10T04:27:10.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b10075200 con 0x7f1b10101900 2024-09-10T04:27:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b0dffb640 1 -- 172.21.15.46:0/374531917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b04035030 con 0x7f1b10101900 2024-09-10T04:27:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.227+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b101a3d80 con 0x7f1b10101900 2024-09-10T04:27:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.228+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b10103040 con 0x7f1b10101900 2024-09-10T04:27:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.229+0000 7f1b0dffb640 1 -- 172.21.15.46:0/374531917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1b04042070 con 0x7f1b10101900 2024-09-10T04:27:10.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.229+0000 7f1b0dffb640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b0007ac40 0x7f1b0007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.230+0000 7f1b0dffb640 1 -- 172.21.15.46:0/374531917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f1b040be5a0 con 0x7f1b10101900 2024-09-10T04:27:10.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.230+0000 7f1b0ffff640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b0007ac40 0x7f1b0007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.230+0000 7f1b0ffff640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b0007ac40 0x7f1b0007d100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0058c0 tx=0x7f1afc005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:10.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.234+0000 7f1b0dffb640 1 -- 172.21.15.46:0/374531917 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1b04089d60 con 0x7f1b10101900 2024-09-10T04:27:10.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.432+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1b10104b80 con 0x7f1b0007ac40 2024-09-10T04:27:10.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.441+0000 7f1b0dffb640 1 -- 172.21.15.46:0/374531917 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f1afc00bca0 con 0x7f1b0007ac40 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (18m) 13m ago 30m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (21m) 13m ago 30m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (21m) 13m ago 29m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (17m) 13m ago 30m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (17m) 13m ago 29m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (18m) 13m ago 29m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (13m) 13m ago 24m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (13m) 13m ago 24m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (13m) 13m ago 24m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (24m) 13m ago 24m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:27:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (20m) 13m ago 31m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (19m) 13m ago 29m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (17m) 13m ago 31m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (17m) 13m ago 29m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (19m) 13m ago 30m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (19m) 13m ago 29m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (16m) 13m ago 28m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (16m) 13m ago 27m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (15m) 13m ago 27m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (14m) 13m ago 26m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (14m) 13m ago 26m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (13m) 13m ago 25m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:27:10.444 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (19m) 13m ago 29m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:27:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b0007ac40 msgr2=0x7f1b0007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b0007ac40 0x7f1b0007d100 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1afc0058c0 tx=0x7f1afc005850 comp rx=0 tx=0).stop 2024-09-10T04:27:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10101900 msgr2=0x7f1b100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10101900 0x7f1b100744f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1b04031460 tx=0x7f1b040045e0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 shutdown_connections 2024-09-10T04:27:10.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b0007ac40 0x7f1b0007d100 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b10107900 0x7f1b10074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 --2- 172.21.15.46:0/374531917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b10101900 0x7f1b100744f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 >> 172.21.15.46:0/374531917 conn(0x7f1b100fd600 msgr2=0x7f1b100ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 shutdown_connections 2024-09-10T04:27:10.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.444+0000 7f1b16e21640 1 -- 172.21.15.46:0/374531917 wait complete. 2024-09-10T04:27:10.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 -- 172.21.15.46:0/4220171103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 msgr2=0x7f474c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 --2- 172.21.15.46:0/4220171103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c107ce0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4740009880 tx=0x7f474002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 -- 172.21.15.46:0/4220171103 shutdown_connections 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 --2- 172.21.15.46:0/4220171103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f474c101900 0x7f474c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 --2- 172.21.15.46:0/4220171103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c107ce0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 -- 172.21.15.46:0/4220171103 >> 172.21.15.46:0/4220171103 conn(0x7f474c0fd600 msgr2=0x7f474c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 -- 172.21.15.46:0/4220171103 shutdown_connections 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.527+0000 7f4752fec640 1 -- 172.21.15.46:0/4220171103 wait complete. 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f4752fec640 1 Processor -- start 2024-09-10T04:27:10.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f4752fec640 1 -- start start 2024-09-10T04:27:10.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f4752fec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f474c101900 0x7f474c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f4752fec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f4752fec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10edd0 con 0x7f474c107900 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f4752fec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f474c10ec50 con 0x7f474c101900 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f474bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f474bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45466/0 (socket says 172.21.15.46:45466) 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f474bfff640 1 -- 172.21.15.46:0/1431091531 learned_addr learned my addr 172.21.15.46:0/1431091531 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f474bfff640 1 -- 172.21.15.46:0/1431091531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f474c101900 msgr2=0x7f474c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f474bfff640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f474c101900 0x7f474c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.528+0000 7f474bfff640 1 -- 172.21.15.46:0/1431091531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c074f70 con 0x7f474c107900 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.529+0000 7f474bfff640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c074a30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f473800e970 tx=0x7f473800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.529+0000 7f4749ffb640 1 -- 172.21.15.46:0/1431091531 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f473800cac0 con 0x7f474c107900 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.529+0000 7f4749ffb640 1 -- 172.21.15.46:0/1431091531 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47380054e0 con 0x7f474c107900 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.529+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c075260 con 0x7f474c107900 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.529+0000 7f4749ffb640 1 -- 172.21.15.46:0/1431091531 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4738010430 con 0x7f474c107900 2024-09-10T04:27:10.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.529+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1a3e20 con 0x7f474c107900 2024-09-10T04:27:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.530+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f474c103040 con 0x7f474c107900 2024-09-10T04:27:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.530+0000 7f4749ffb640 1 -- 172.21.15.46:0/1431091531 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4738004f00 con 0x7f474c107900 2024-09-10T04:27:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.531+0000 7f4749ffb640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f473c07ac40 0x7f473c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.531+0000 7f4749ffb640 1 -- 172.21.15.46:0/1431091531 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f473809e2f0 con 0x7f474c107900 2024-09-10T04:27:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.531+0000 7f4750d88640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f473c07ac40 0x7f473c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.532+0000 7f4750d88640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f473c07ac40 0x7f473c07d100 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47400096c0 tx=0x7f4740000bc0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.535+0000 7f4749ffb640 1 -- 172.21.15.46:0/1431091531 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4738069ab0 con 0x7f474c107900 2024-09-10T04:27:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.801+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f474c109e70 con 0x7f474c107900 2024-09-10T04:27:10.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.801+0000 7f4749ffb640 1 -- 172.21.15.46:0/1431091531 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f473806e960 con 0x7f474c107900 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:27:10.803 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:27:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:27:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:27:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:27:10.804 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:27:10.804 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f473c07ac40 msgr2=0x7f473c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f473c07ac40 0x7f473c07d100 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47400096c0 tx=0x7f4740000bc0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 msgr2=0x7f474c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c074a30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f473800e970 tx=0x7f473800ed70 comp rx=0 tx=0).stop 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 shutdown_connections 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f473c07ac40 0x7f473c07d100 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f474c107900 0x7f474c074a30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 --2- 172.21.15.46:0/1431091531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f474c101900 0x7f474c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 >> 172.21.15.46:0/1431091531 conn(0x7f474c0fd600 msgr2=0x7f474c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 shutdown_connections 2024-09-10T04:27:10.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.804+0000 7f4752fec640 1 -- 172.21.15.46:0/1431091531 wait complete. 2024-09-10T04:27:10.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 -- 172.21.15.46:0/861076581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 msgr2=0x7efd84102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 --2- 172.21.15.46:0/861076581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd84102f40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efd74009880 tx=0x7efd7402a530 comp rx=0 tx=0).stop 2024-09-10T04:27:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 -- 172.21.15.46:0/861076581 shutdown_connections 2024-09-10T04:27:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 --2- 172.21.15.46:0/861076581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd84103480 0x7efd8410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 --2- 172.21.15.46:0/861076581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd84102f40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 -- 172.21.15.46:0/861076581 >> 172.21.15.46:0/861076581 conn(0x7efd840fc520 msgr2=0x7efd840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 -- 172.21.15.46:0/861076581 shutdown_connections 2024-09-10T04:27:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.890+0000 7efd8996c640 1 -- 172.21.15.46:0/861076581 wait complete. 2024-09-10T04:27:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd8996c640 1 Processor -- start 2024-09-10T04:27:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd8996c640 1 -- start start 2024-09-10T04:27:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd8996c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd8419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd8996c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd84103480 0x7efd8419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd8996c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd8410ee50 con 0x7efd84102b60 2024-09-10T04:27:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd8996c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efd8410ecd0 con 0x7efd84103480 2024-09-10T04:27:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd82ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd8419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd82ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd8419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45478/0 (socket says 172.21.15.46:45478) 2024-09-10T04:27:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd82ffd640 1 -- 172.21.15.46:0/4053312451 learned_addr learned my addr 172.21.15.46:0/4053312451 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd7bfff640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd84103480 0x7efd8419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd82ffd640 1 -- 172.21.15.46:0/4053312451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd84103480 msgr2=0x7efd8419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd82ffd640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd84103480 0x7efd8419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.891+0000 7efd82ffd640 1 -- 172.21.15.46:0/4053312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd84199a50 con 0x7efd84102b60 2024-09-10T04:27:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.892+0000 7efd82ffd640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd8419f890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd74004f90 tx=0x7efd740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.892+0000 7efd8896a640 1 -- 172.21.15.46:0/4053312451 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efd7403f070 con 0x7efd84102b60 2024-09-10T04:27:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.892+0000 7efd8896a640 1 -- 172.21.15.46:0/4053312451 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd74004cf0 con 0x7efd84102b60 2024-09-10T04:27:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.892+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd84199ce0 con 0x7efd84102b60 2024-09-10T04:27:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.892+0000 7efd8896a640 1 -- 172.21.15.46:0/4053312451 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efd74035030 con 0x7efd84102b60 2024-09-10T04:27:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.892+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8419a210 con 0x7efd84102b60 2024-09-10T04:27:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.893+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8406a9f0 con 0x7efd84102b60 2024-09-10T04:27:10.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.894+0000 7efd8896a640 1 -- 172.21.15.46:0/4053312451 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efd74042070 con 0x7efd84102b60 2024-09-10T04:27:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.894+0000 7efd8896a640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd7007ac40 0x7efd7007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.895+0000 7efd8896a640 1 -- 172.21.15.46:0/4053312451 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7efd740be680 con 0x7efd84102b60 2024-09-10T04:27:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.895+0000 7efd7bfff640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd7007ac40 0x7efd7007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.895+0000 7efd7bfff640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd7007ac40 0x7efd7007d100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efd840feec0 tx=0x7efd6c009110 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:10.899+0000 7efd8896a640 1 -- 172.21.15.46:0/4053312451 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efd74089e40 con 0x7efd84102b60 2024-09-10T04:27:11.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.122+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efd8419af80 con 0x7efd84102b60 2024-09-10T04:27:11.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.123+0000 7efd8896a640 1 -- 172.21.15.46:0/4053312451 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7efd7408ecf0 con 0x7efd84102b60 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:27:11.124 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:27:11.125 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:27:11.126 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:11.127 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd7007ac40 msgr2=0x7efd7007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd7007ac40 0x7efd7007d100 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7efd840feec0 tx=0x7efd6c009110 comp rx=0 tx=0).stop 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 msgr2=0x7efd8419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd8419f890 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd74004f90 tx=0x7efd740048f0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 shutdown_connections 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd7007ac40 0x7efd7007d100 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd84103480 0x7efd8419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 --2- 172.21.15.46:0/4053312451 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd84102b60 0x7efd8419f890 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 >> 172.21.15.46:0/4053312451 conn(0x7efd840fc520 msgr2=0x7efd84100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 shutdown_connections 2024-09-10T04:27:11.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.125+0000 7efd8996c640 1 -- 172.21.15.46:0/4053312451 wait complete. 2024-09-10T04:27:11.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.210+0000 7f0f64606640 1 -- 172.21.15.46:0/2034479749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 msgr2=0x7f0f5c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.210+0000 7f0f64606640 1 --2- 172.21.15.46:0/2034479749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c106bd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f0f58009880 tx=0x7f0f5802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 -- 172.21.15.46:0/2034479749 shutdown_connections 2024-09-10T04:27:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 --2- 172.21.15.46:0/2034479749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c1007f0 0x7f0f5c100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 --2- 172.21.15.46:0/2034479749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c106bd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 -- 172.21.15.46:0/2034479749 >> 172.21.15.46:0/2034479749 conn(0x7f0f5c0fc550 msgr2=0x7f0f5c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 -- 172.21.15.46:0/2034479749 shutdown_connections 2024-09-10T04:27:11.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 -- 172.21.15.46:0/2034479749 wait complete. 2024-09-10T04:27:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 Processor -- start 2024-09-10T04:27:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.211+0000 7f0f64606640 1 -- start start 2024-09-10T04:27:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f64606640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c1007f0 0x7f0f5c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f64606640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f64606640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c10dca0 con 0x7f0f5c1067f0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f64606640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c10db20 con 0x7f0f5c1007f0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f61ba1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f61ba1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45502/0 (socket says 172.21.15.46:45502) 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f61ba1640 1 -- 172.21.15.46:0/2963954730 learned_addr learned my addr 172.21.15.46:0/2963954730 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f623a2640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c1007f0 0x7f0f5c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f61ba1640 1 -- 172.21.15.46:0/2963954730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c1007f0 msgr2=0x7f0f5c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f61ba1640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c1007f0 0x7f0f5c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f61ba1640 1 -- 172.21.15.46:0/2963954730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c199410 con 0x7f0f5c1067f0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f61ba1640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c198e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d8d0 tx=0x7f0f4c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.212+0000 7f0f537fe640 1 -- 172.21.15.46:0/2963954730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f4c017030 con 0x7f0f5c1067f0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.213+0000 7f0f537fe640 1 -- 172.21.15.46:0/2963954730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f4c004020 con 0x7f0f5c1067f0 2024-09-10T04:27:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.213+0000 7f0f537fe640 1 -- 172.21.15.46:0/2963954730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0f4c0047a0 con 0x7f0f5c1067f0 2024-09-10T04:27:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.213+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c199700 con 0x7f0f5c1067f0 2024-09-10T04:27:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.213+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c1a1d00 con 0x7f0f5c1067f0 2024-09-10T04:27:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.214+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c101f30 con 0x7f0f5c1067f0 2024-09-10T04:27:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.214+0000 7f0f537fe640 1 -- 172.21.15.46:0/2963954730 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0f4c0041d0 con 0x7f0f5c1067f0 2024-09-10T04:27:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.215+0000 7f0f537fe640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f4807abf0 0x7f0f4807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.215+0000 7f0f537fe640 1 -- 172.21.15.46:0/2963954730 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f0f4c018070 con 0x7f0f5c1067f0 2024-09-10T04:27:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.215+0000 7f0f623a2640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f4807abf0 0x7f0f4807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.216+0000 7f0f623a2640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f4807abf0 0x7f0f4807d0b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f580096c0 tx=0x7f0f58031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:11.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.220+0000 7f0f537fe640 1 -- 172.21.15.46:0/2963954730 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0f5c199700 con 0x7f0f5c1067f0 2024-09-10T04:27:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.424+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0f5c062630 con 0x7f0f4807abf0 2024-09-10T04:27:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.425+0000 7f0f537fe640 1 -- 172.21.15.46:0/2963954730 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f0f5c062630 con 0x7f0f4807abf0 2024-09-10T04:27:11.426 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:27:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:27:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:27:11.426 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:27:11.427 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:27:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.427+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f4807abf0 msgr2=0x7f0f4807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.427+0000 7f0f64606640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f4807abf0 0x7f0f4807d0b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f580096c0 tx=0x7f0f58031400 comp rx=0 tx=0).stop 2024-09-10T04:27:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.427+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 msgr2=0x7f0f5c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.427+0000 7f0f64606640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c198e10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c00d8d0 tx=0x7f0f4c00dd90 comp rx=0 tx=0).stop 2024-09-10T04:27:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.428+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 shutdown_connections 2024-09-10T04:27:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.428+0000 7f0f64606640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0f4807abf0 0x7f0f4807d0b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.428+0000 7f0f64606640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c1067f0 0x7f0f5c198e10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.428+0000 7f0f64606640 1 --2- 172.21.15.46:0/2963954730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f5c1007f0 0x7f0f5c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.428+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 >> 172.21.15.46:0/2963954730 conn(0x7f0f5c0fc550 msgr2=0x7f0f5c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.428+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 shutdown_connections 2024-09-10T04:27:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.428+0000 7f0f64606640 1 -- 172.21.15.46:0/2963954730 wait complete. 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.512+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/267907464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c105860 msgr2=0x7fe79c105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.512+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/267907464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c105860 0x7fe79c105c40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009c20 tx=0x7fe78c02a970 comp rx=0 tx=0).stop 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.513+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/267907464 shutdown_connections 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.513+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/267907464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe79c101010 0x7fe79c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.513+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/267907464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c105860 0x7fe79c105c40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.513+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/267907464 >> 172.21.15.46:0/267907464 conn(0x7fe79c0fbc90 msgr2=0x7fe79c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.513+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/267907464 shutdown_connections 2024-09-10T04:27:11.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.513+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/267907464 wait complete. 2024-09-10T04:27:11.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.513+0000 7fe7a1c1b640 1 Processor -- start 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe7a1c1b640 1 -- start start 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe7a1c1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c101010 0x7fe79c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe7a1c1b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe79c105860 0x7fe79c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe7a1c1b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c10de40 con 0x7fe79c101010 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe7a1c1b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c10dcc0 con 0x7fe79c105860 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c101010 0x7fe79c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c101010 0x7fe79c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45508/0 (socket says 172.21.15.46:45508) 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79b7fe640 1 -- 172.21.15.46:0/3932697709 learned_addr learned my addr 172.21.15.46:0/3932697709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79affd640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe79c105860 0x7fe79c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79b7fe640 1 -- 172.21.15.46:0/3932697709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe79c105860 msgr2=0x7fe79c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79b7fe640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe79c105860 0x7fe79c198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79b7fe640 1 -- 172.21.15.46:0/3932697709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c199470 con 0x7fe79c101010 2024-09-10T04:27:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.514+0000 7fe79b7fe640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c101010 0x7fe79c198930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c02fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.515+0000 7fe798ff9640 1 -- 172.21.15.46:0/3932697709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe78c047070 con 0x7fe79c101010 2024-09-10T04:27:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.515+0000 7fe798ff9640 1 -- 172.21.15.46:0/3932697709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe78c003e30 con 0x7fe79c101010 2024-09-10T04:27:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.515+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c199700 con 0x7fe79c101010 2024-09-10T04:27:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.515+0000 7fe798ff9640 1 -- 172.21.15.46:0/3932697709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe78c034070 con 0x7fe79c101010 2024-09-10T04:27:11.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.515+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c1a1cb0 con 0x7fe79c101010 2024-09-10T04:27:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.516+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c102020 con 0x7fe79c101010 2024-09-10T04:27:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.516+0000 7fe798ff9640 1 -- 172.21.15.46:0/3932697709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe78c04a070 con 0x7fe79c101010 2024-09-10T04:27:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.517+0000 7fe798ff9640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe78807ac50 0x7fe78807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.517+0000 7fe798ff9640 1 -- 172.21.15.46:0/3932697709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fe78c0be9a0 con 0x7fe79c101010 2024-09-10T04:27:11.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.517+0000 7fe79affd640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe78807ac50 0x7fe78807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:11.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.518+0000 7fe79affd640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe78807ac50 0x7fe78807d110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe79c100a50 tx=0x7fe784005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:11.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.521+0000 7fe798ff9640 1 -- 172.21.15.46:0/3932697709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe78c03f070 con 0x7fe79c101010 2024-09-10T04:27:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.779+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe79c108d80 con 0x7fe79c101010 2024-09-10T04:27:11.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:11 smithi046 ceph-mon[109506]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:11.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:11 smithi046 ceph-mon[109506]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:11.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:11 smithi046 ceph-mon[109506]: pgmap v645: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:11.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:11 smithi046 ceph-mon[109506]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:11.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:11 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1431091531' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:27:11.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:11 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4053312451' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:27:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.780+0000 7fe798ff9640 1 -- 172.21.15.46:0/3932697709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fe78c08a0c0 con 0x7fe79c101010 2024-09-10T04:27:11.782 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:27:11.782 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:27:11.782 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:27:11.782 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:27:11.782 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:27:11.782 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:27:11.782 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:27:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe78807ac50 msgr2=0x7fe78807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe78807ac50 0x7fe78807d110 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe79c100a50 tx=0x7fe784005f90 comp rx=0 tx=0).stop 2024-09-10T04:27:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c101010 msgr2=0x7fe79c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c101010 0x7fe79c198930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006010 tx=0x7fe78c02fd60 comp rx=0 tx=0).stop 2024-09-10T04:27:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 shutdown_connections 2024-09-10T04:27:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe78807ac50 0x7fe78807d110 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe79c105860 0x7fe79c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 --2- 172.21.15.46:0/3932697709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe79c101010 0x7fe79c198930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.782+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 >> 172.21.15.46:0/3932697709 conn(0x7fe79c0fbc90 msgr2=0x7fe79c0fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.783+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 shutdown_connections 2024-09-10T04:27:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:11.783+0000 7fe7a1c1b640 1 -- 172.21.15.46:0/3932697709 wait complete. 2024-09-10T04:27:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:11 smithi070 ceph-mon[79774]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:11 smithi070 ceph-mon[79774]: from='client.35042 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:11 smithi070 ceph-mon[79774]: pgmap v645: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:11 smithi070 ceph-mon[79774]: from='client.35046 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:11 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1431091531' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:27:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:11 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4053312451' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:27:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:12 smithi070 ceph-mon[79774]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:12 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3932697709' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:27:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:12 smithi046 ceph-mon[109506]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:12 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3932697709' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:27:13.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:13 smithi070 ceph-mon[79774]: pgmap v646: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:13 smithi046 ceph-mon[109506]: pgmap v646: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:15.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:15 smithi070 ceph-mon[79774]: pgmap v647: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:15.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:15 smithi046 ceph-mon[109506]: pgmap v647: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:17.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:17 smithi070 ceph-mon[79774]: pgmap v648: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:17.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:17 smithi046 ceph-mon[109506]: pgmap v648: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:19 smithi070 ceph-mon[79774]: pgmap v649: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:19.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:19 smithi046 ceph-mon[109506]: pgmap v649: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:21 smithi070 ceph-mon[79774]: pgmap v650: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:21.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:21 smithi046 ceph-mon[109506]: pgmap v650: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:23 smithi070 ceph-mon[79774]: pgmap v651: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:23 smithi046 ceph-mon[109506]: pgmap v651: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:25 smithi070 ceph-mon[79774]: pgmap v652: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:25.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:25 smithi046 ceph-mon[109506]: pgmap v652: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:27.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:27 smithi070 ceph-mon[79774]: pgmap v653: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:27.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:27 smithi046 ceph-mon[109506]: pgmap v653: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:29 smithi070 ceph-mon[79774]: pgmap v654: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:29.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:29 smithi046 ceph-mon[109506]: pgmap v654: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:31 smithi070 ceph-mon[79774]: pgmap v655: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:31.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:31 smithi046 ceph-mon[109506]: pgmap v655: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:33 smithi070 ceph-mon[79774]: pgmap v656: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:33 smithi046 ceph-mon[109506]: pgmap v656: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:35.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:35 smithi046 ceph-mon[109506]: pgmap v657: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:35.949 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:35 smithi070 ceph-mon[79774]: pgmap v657: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:38.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:37 smithi070 ceph-mon[79774]: pgmap v658: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:38.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:37 smithi046 ceph-mon[109506]: pgmap v658: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:40.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:39 smithi070 ceph-mon[79774]: pgmap v659: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:39 smithi046 ceph-mon[109506]: pgmap v659: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:41.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 -- 172.21.15.46:0/2052298522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100120 msgr2=0x7f699c100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:41.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 --2- 172.21.15.46:0/2052298522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100120 0x7f699c100500 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f698c009880 tx=0x7f698c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:41.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 -- 172.21.15.46:0/2052298522 shutdown_connections 2024-09-10T04:27:41.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 --2- 172.21.15.46:0/2052298522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f699c100ad0 0x7f699c106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:41.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 --2- 172.21.15.46:0/2052298522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100120 0x7f699c100500 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:41.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 -- 172.21.15.46:0/2052298522 >> 172.21.15.46:0/2052298522 conn(0x7f699c0fbc90 msgr2=0x7f699c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:41.883 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:41 smithi046 ceph-mon[109506]: pgmap v660: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 -- 172.21.15.46:0/2052298522 shutdown_connections 2024-09-10T04:27:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.880+0000 7f69a219d640 1 -- 172.21.15.46:0/2052298522 wait complete. 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f69a219d640 1 Processor -- start 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f69a219d640 1 -- start start 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f69a219d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f699c100120 0x7f699c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f69a219d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100ad0 0x7f699c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f69a219d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f699c10ee60 con 0x7f699c100ad0 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f69a219d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f699c10ece0 con 0x7f699c100120 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f699affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100ad0 0x7f699c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f699affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100ad0 0x7f699c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52478/0 (socket says 172.21.15.46:52478) 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.881+0000 7f699affd640 1 -- 172.21.15.46:0/3054942711 learned_addr learned my addr 172.21.15.46:0/3054942711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f699b7fe640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f699c100120 0x7f699c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f699affd640 1 -- 172.21.15.46:0/3054942711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f699c100120 msgr2=0x7f699c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f699affd640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f699c100120 0x7f699c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f699affd640 1 -- 172.21.15.46:0/3054942711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c19e2b0 con 0x7f699c100ad0 2024-09-10T04:27:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f699affd640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100ad0 0x7f699c19dc00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f699000c8f0 tx=0x7f699000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f6998ff9640 1 -- 172.21.15.46:0/3054942711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6990014070 con 0x7f699c100ad0 2024-09-10T04:27:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f6998ff9640 1 -- 172.21.15.46:0/3054942711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6990007c60 con 0x7f699c100ad0 2024-09-10T04:27:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1979a0 con 0x7f699c100ad0 2024-09-10T04:27:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f6998ff9640 1 -- 172.21.15.46:0/3054942711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6990003c30 con 0x7f699c100ad0 2024-09-10T04:27:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.882+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c197f00 con 0x7f699c100ad0 2024-09-10T04:27:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.883+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f699c100500 con 0x7f699c100ad0 2024-09-10T04:27:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.884+0000 7f6998ff9640 1 -- 172.21.15.46:0/3054942711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6990017070 con 0x7f699c100ad0 2024-09-10T04:27:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.885+0000 7f6998ff9640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f698407abf0 0x7f698407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.885+0000 7f6998ff9640 1 -- 172.21.15.46:0/3054942711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f699009da20 con 0x7f699c100ad0 2024-09-10T04:27:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.885+0000 7f699b7fe640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f698407abf0 0x7f698407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.885+0000 7f699b7fe640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f698407abf0 0x7f698407d0b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c004120 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:41.889+0000 7f6998ff9640 1 -- 172.21.15.46:0/3054942711 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6990069160 con 0x7f699c100ad0 2024-09-10T04:27:42.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:41 smithi070 ceph-mon[79774]: pgmap v660: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:42.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.094+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f699c198210 con 0x7f698407abf0 2024-09-10T04:27:42.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.095+0000 7f6998ff9640 1 -- 172.21.15.46:0/3054942711 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f699c198210 con 0x7f698407abf0 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f698407abf0 msgr2=0x7f698407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f698407abf0 0x7f698407d0b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f698c000c00 tx=0x7f698c004120 comp rx=0 tx=0).stop 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100ad0 msgr2=0x7f699c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100ad0 0x7f699c19dc00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f699000c8f0 tx=0x7f699000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 shutdown_connections 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f698407abf0 0x7f698407d0b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f699c100ad0 0x7f699c19dc00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 --2- 172.21.15.46:0/3054942711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f699c100120 0x7f699c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 >> 172.21.15.46:0/3054942711 conn(0x7f699c0fbc90 msgr2=0x7f699c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 shutdown_connections 2024-09-10T04:27:42.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.098+0000 7f69a219d640 1 -- 172.21.15.46:0/3054942711 wait complete. 2024-09-10T04:27:42.109 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:27:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 -- 172.21.15.46:0/319700185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 msgr2=0x7f548810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 --2- 172.21.15.46:0/319700185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f548810b8d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f547c009880 tx=0x7f547c02a530 comp rx=0 tx=0).stop 2024-09-10T04:27:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 -- 172.21.15.46:0/319700185 shutdown_connections 2024-09-10T04:27:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 --2- 172.21.15.46:0/319700185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f548810b8d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 --2- 172.21.15.46:0/319700185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488102ac0 0x7f5488102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 -- 172.21.15.46:0/319700185 >> 172.21.15.46:0/319700185 conn(0x7f54880fc520 msgr2=0x7f54880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 -- 172.21.15.46:0/319700185 shutdown_connections 2024-09-10T04:27:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.184+0000 7f548e4b7640 1 -- 172.21.15.46:0/319700185 wait complete. 2024-09-10T04:27:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f548e4b7640 1 Processor -- start 2024-09-10T04:27:42.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f548e4b7640 1 -- start start 2024-09-10T04:27:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f548e4b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488102ac0 0x7f5488100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f548e4b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f5488100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f548e4b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f548810edf0 con 0x7f54881033e0 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f548e4b7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f548810ec70 con 0x7f5488102ac0 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f54877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f5488100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f54877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f5488100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52508/0 (socket says 172.21.15.46:52508) 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f54877fe640 1 -- 172.21.15.46:0/1500604103 learned_addr learned my addr 172.21.15.46:0/1500604103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.185+0000 7f5487fff640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488102ac0 0x7f5488100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f54877fe640 1 -- 172.21.15.46:0/1500604103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488102ac0 msgr2=0x7f5488100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f54877fe640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488102ac0 0x7f5488100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f54877fe640 1 -- 172.21.15.46:0/1500604103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54881011a0 con 0x7f54881033e0 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f54877fe640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f5488100c60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f547c031460 tx=0x7f547c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:42.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f54857fa640 1 -- 172.21.15.46:0/1500604103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f547c03f070 con 0x7f54881033e0 2024-09-10T04:27:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f54857fa640 1 -- 172.21.15.46:0/1500604103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f547c004ac0 con 0x7f54881033e0 2024-09-10T04:27:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f54857fa640 1 -- 172.21.15.46:0/1500604103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f547c035030 con 0x7f54881033e0 2024-09-10T04:27:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5488101430 con 0x7f54881033e0 2024-09-10T04:27:42.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.186+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881a3de0 con 0x7f54881033e0 2024-09-10T04:27:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.188+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548806aa90 con 0x7f54881033e0 2024-09-10T04:27:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.188+0000 7f54857fa640 1 -- 172.21.15.46:0/1500604103 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f547c042070 con 0x7f54881033e0 2024-09-10T04:27:42.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.188+0000 7f54857fa640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f547007ac50 0x7f547007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.189+0000 7f54857fa640 1 -- 172.21.15.46:0/1500604103 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f547c0be640 con 0x7f54881033e0 2024-09-10T04:27:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.189+0000 7f5487fff640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f547007ac50 0x7f547007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.189+0000 7f5487fff640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f547007ac50 0x7f547007d110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5478005fd0 tx=0x7f5478005e20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:42.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.193+0000 7f54857fa640 1 -- 172.21.15.46:0/1500604103 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f547c089e00 con 0x7f54881033e0 2024-09-10T04:27:42.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.396+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5488062630 con 0x7f547007ac50 2024-09-10T04:27:42.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.397+0000 7f54857fa640 1 -- 172.21.15.46:0/1500604103 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f5488062630 con 0x7f547007ac50 2024-09-10T04:27:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.399+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f547007ac50 msgr2=0x7f547007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.399+0000 7f548e4b7640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f547007ac50 0x7f547007d110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5478005fd0 tx=0x7f5478005e20 comp rx=0 tx=0).stop 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.399+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 msgr2=0x7f5488100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.399+0000 7f548e4b7640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f5488100c60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f547c031460 tx=0x7f547c0046c0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.400+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 shutdown_connections 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.400+0000 7f548e4b7640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f547007ac50 0x7f547007d110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.400+0000 7f548e4b7640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54881033e0 0x7f5488100c60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.400+0000 7f548e4b7640 1 --2- 172.21.15.46:0/1500604103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5488102ac0 0x7f5488100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.400+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 >> 172.21.15.46:0/1500604103 conn(0x7f54880fc520 msgr2=0x7f5488108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.400+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 shutdown_connections 2024-09-10T04:27:42.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.400+0000 7f548e4b7640 1 -- 172.21.15.46:0/1500604103 wait complete. 2024-09-10T04:27:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 -- 172.21.15.46:0/3338260173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4901033e0 msgr2=0x7fe49010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 --2- 172.21.15.46:0/3338260173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4901033e0 0x7fe49010b8d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe478009880 tx=0x7fe47802a530 comp rx=0 tx=0).stop 2024-09-10T04:27:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 -- 172.21.15.46:0/3338260173 shutdown_connections 2024-09-10T04:27:42.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 --2- 172.21.15.46:0/3338260173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4901033e0 0x7fe49010b8d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 --2- 172.21.15.46:0/3338260173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe490102ac0 0x7fe490102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 -- 172.21.15.46:0/3338260173 >> 172.21.15.46:0/3338260173 conn(0x7fe4900fc520 msgr2=0x7fe4900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 -- 172.21.15.46:0/3338260173 shutdown_connections 2024-09-10T04:27:42.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.484+0000 7fe4954ca640 1 -- 172.21.15.46:0/3338260173 wait complete. 2024-09-10T04:27:42.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.485+0000 7fe4954ca640 1 Processor -- start 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.485+0000 7fe4954ca640 1 -- start start 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.485+0000 7fe4954ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe490102ac0 0x7fe49019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.485+0000 7fe4954ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4901033e0 0x7fe49019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.485+0000 7fe4954ca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe49010edf0 con 0x7fe490102ac0 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.485+0000 7fe4954ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe49010ec70 con 0x7fe4901033e0 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe490102ac0 0x7fe49019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe490102ac0 0x7fe49019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52518/0 (socket says 172.21.15.46:52518) 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48effd640 1 -- 172.21.15.46:0/1727881924 learned_addr learned my addr 172.21.15.46:0/1727881924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe4867fc640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4901033e0 0x7fe49019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48effd640 1 -- 172.21.15.46:0/1727881924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4901033e0 msgr2=0x7fe49019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48effd640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4901033e0 0x7fe49019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48effd640 1 -- 172.21.15.46:0/1727881924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe490197800 con 0x7fe490102ac0 2024-09-10T04:27:42.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48effd640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe490102ac0 0x7fe49019d680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c8f0 tx=0x7fe48000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:42.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48cff9640 1 -- 172.21.15.46:0/1727881924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe4800079b0 con 0x7fe490102ac0 2024-09-10T04:27:42.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48cff9640 1 -- 172.21.15.46:0/1727881924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe4800054e0 con 0x7fe490102ac0 2024-09-10T04:27:42.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe490197af0 con 0x7fe490102ac0 2024-09-10T04:27:42.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe48cff9640 1 -- 172.21.15.46:0/1727881924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe480004dd0 con 0x7fe490102ac0 2024-09-10T04:27:42.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.486+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe490198050 con 0x7fe490102ac0 2024-09-10T04:27:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.488+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49006aa90 con 0x7fe490102ac0 2024-09-10T04:27:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.488+0000 7fe48cff9640 1 -- 172.21.15.46:0/1727881924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe480005020 con 0x7fe490102ac0 2024-09-10T04:27:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.489+0000 7fe48cff9640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe47c07ab90 0x7fe47c07d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.489+0000 7fe4867fc640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe47c07ab90 0x7fe47c07d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.489+0000 7fe48cff9640 1 -- 172.21.15.46:0/1727881924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fe48009dbb0 con 0x7fe490102ac0 2024-09-10T04:27:42.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.489+0000 7fe4867fc640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe47c07ab90 0x7fe47c07d050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe490198b50 tx=0x7fe478031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:42.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.493+0000 7fe48cff9640 1 -- 172.21.15.46:0/1727881924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe4800a4080 con 0x7fe490102ac0 2024-09-10T04:27:42.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.687+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe490198360 con 0x7fe47c07ab90 2024-09-10T04:27:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.696+0000 7fe48cff9640 1 -- 172.21.15.46:0/1727881924 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fe490198360 con 0x7fe47c07ab90 2024-09-10T04:27:42.698 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:27:42.698 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (19m) 13m ago 30m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (21m) 13m ago 30m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (21m) 13m ago 29m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (17m) 13m ago 30m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (17m) 13m ago 29m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (18m) 13m ago 30m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (13m) 13m ago 25m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (14m) 13m ago 25m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (13m) 13m ago 25m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (25m) 13m ago 25m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (21m) 13m ago 31m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (20m) 13m ago 29m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (18m) 13m ago 32m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (17m) 13m ago 29m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:27:42.699 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (20m) 13m ago 30m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (20m) 13m ago 29m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (17m) 13m ago 28m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (16m) 13m ago 28m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (16m) 13m ago 27m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (15m) 13m ago 27m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (14m) 13m ago 26m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (14m) 13m ago 26m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:27:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (19m) 13m ago 30m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe47c07ab90 msgr2=0x7fe47c07d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe47c07ab90 0x7fe47c07d050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe490198b50 tx=0x7fe478031400 comp rx=0 tx=0).stop 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe490102ac0 msgr2=0x7fe49019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe490102ac0 0x7fe49019d680 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c8f0 tx=0x7fe48000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 shutdown_connections 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe47c07ab90 0x7fe47c07d050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4901033e0 0x7fe49019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 --2- 172.21.15.46:0/1727881924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe490102ac0 0x7fe49019d680 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 >> 172.21.15.46:0/1727881924 conn(0x7fe4900fc520 msgr2=0x7fe490108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:42.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 shutdown_connections 2024-09-10T04:27:42.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.699+0000 7fe4954ca640 1 -- 172.21.15.46:0/1727881924 wait complete. 2024-09-10T04:27:42.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 -- 172.21.15.46:0/1721007259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4101900 msgr2=0x7fabb4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 --2- 172.21.15.46:0/1721007259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4101900 0x7fabb4101d60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faba8009880 tx=0x7faba802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 -- 172.21.15.46:0/1721007259 shutdown_connections 2024-09-10T04:27:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 --2- 172.21.15.46:0/1721007259 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4101900 0x7fabb4101d60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 --2- 172.21.15.46:0/1721007259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabb4107900 0x7fabb4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 -- 172.21.15.46:0/1721007259 >> 172.21.15.46:0/1721007259 conn(0x7fabb40fd600 msgr2=0x7fabb40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 -- 172.21.15.46:0/1721007259 shutdown_connections 2024-09-10T04:27:42.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.782+0000 7fabbb976640 1 -- 172.21.15.46:0/1721007259 wait complete. 2024-09-10T04:27:42.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.783+0000 7fabbb976640 1 Processor -- start 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.783+0000 7fabbb976640 1 -- start start 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabbb976640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabb4101900 0x7fabb419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabbb976640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4107900 0x7fabb419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabbb976640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410edb0 con 0x7fabb4107900 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabbb976640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410ec30 con 0x7fabb4101900 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb8f11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4107900 0x7fabb419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb8f11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4107900 0x7fabb419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52526/0 (socket says 172.21.15.46:52526) 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb8f11640 1 -- 172.21.15.46:0/1842995140 learned_addr learned my addr 172.21.15.46:0/1842995140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb9712640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabb4101900 0x7fabb419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb8f11640 1 -- 172.21.15.46:0/1842995140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabb4101900 msgr2=0x7fabb419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb8f11640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabb4101900 0x7fabb419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:42.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb8f11640 1 -- 172.21.15.46:0/1842995140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb4197800 con 0x7fabb4107900 2024-09-10T04:27:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7fabb8f11640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4107900 0x7fabb419dbc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faba8031460 tx=0x7faba80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.784+0000 7faba67fc640 1 -- 172.21.15.46:0/1842995140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faba803f070 con 0x7fabb4107900 2024-09-10T04:27:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.785+0000 7faba67fc640 1 -- 172.21.15.46:0/1842995140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faba80035f0 con 0x7fabb4107900 2024-09-10T04:27:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.785+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb4197a90 con 0x7fabb4107900 2024-09-10T04:27:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.785+0000 7faba67fc640 1 -- 172.21.15.46:0/1842995140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faba8035030 con 0x7fabb4107900 2024-09-10T04:27:42.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.785+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb4197fc0 con 0x7fabb4107900 2024-09-10T04:27:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.786+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb4103040 con 0x7fabb4107900 2024-09-10T04:27:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.786+0000 7faba67fc640 1 -- 172.21.15.46:0/1842995140 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7faba8042070 con 0x7fabb4107900 2024-09-10T04:27:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.787+0000 7faba67fc640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faba007abf0 0x7faba007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.787+0000 7faba67fc640 1 -- 172.21.15.46:0/1842995140 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7faba80be150 con 0x7fabb4107900 2024-09-10T04:27:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.787+0000 7fabb9712640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faba007abf0 0x7faba007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:42.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.788+0000 7fabb9712640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faba007abf0 0x7faba007d0b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005d00 tx=0x7fabb000b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:42.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:42.791+0000 7faba67fc640 1 -- 172.21.15.46:0/1842995140 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7faba8089890 con 0x7fabb4107900 2024-09-10T04:27:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.049+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fabb4109e90 con 0x7fabb4107900 2024-09-10T04:27:43.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.049+0000 7faba67fc640 1 -- 172.21.15.46:0/1842995140 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7faba808e740 con 0x7fabb4107900 2024-09-10T04:27:43.051 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:27:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:27:43.051 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:27:43.052 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:27:43.053 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:27:43.053 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:27:43.053 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:27:43.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faba007abf0 msgr2=0x7faba007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faba007abf0 0x7faba007d0b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005d00 tx=0x7fabb000b410 comp rx=0 tx=0).stop 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4107900 msgr2=0x7fabb419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4107900 0x7fabb419dbc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7faba8031460 tx=0x7faba80045e0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 shutdown_connections 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faba007abf0 0x7faba007d0b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabb4107900 0x7fabb419dbc0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.051+0000 7fabbb976640 1 --2- 172.21.15.46:0/1842995140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fabb4101900 0x7fabb419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.052+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 >> 172.21.15.46:0/1842995140 conn(0x7fabb40fd600 msgr2=0x7fabb40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.052+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 shutdown_connections 2024-09-10T04:27:43.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.052+0000 7fabbb976640 1 -- 172.21.15.46:0/1842995140 wait complete. 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.135+0000 7f02e129e640 1 -- 172.21.15.46:0/2544915943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc101900 msgr2=0x7f02dc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.135+0000 7f02e129e640 1 --2- 172.21.15.46:0/2544915943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc101900 0x7f02dc101d60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009880 tx=0x7f02c402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 -- 172.21.15.46:0/2544915943 shutdown_connections 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 --2- 172.21.15.46:0/2544915943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc101900 0x7f02dc101d60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 --2- 172.21.15.46:0/2544915943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02dc107900 0x7f02dc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 -- 172.21.15.46:0/2544915943 >> 172.21.15.46:0/2544915943 conn(0x7f02dc0fd600 msgr2=0x7f02dc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 -- 172.21.15.46:0/2544915943 shutdown_connections 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 -- 172.21.15.46:0/2544915943 wait complete. 2024-09-10T04:27:43.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 Processor -- start 2024-09-10T04:27:43.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.136+0000 7f02e129e640 1 -- start start 2024-09-10T04:27:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02e129e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02dc101900 0x7f02dc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02e129e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc107900 0x7f02dc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02e129e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10edd0 con 0x7f02dc107900 2024-09-10T04:27:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02e129e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f02dc10ec50 con 0x7f02dc101900 2024-09-10T04:27:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02da7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc107900 0x7f02dc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02da7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc107900 0x7f02dc19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52558/0 (socket says 172.21.15.46:52558) 2024-09-10T04:27:43.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02da7fc640 1 -- 172.21.15.46:0/3775562456 learned_addr learned my addr 172.21.15.46:0/3775562456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02daffd640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02dc101900 0x7f02dc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02da7fc640 1 -- 172.21.15.46:0/3775562456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02dc101900 msgr2=0x7f02dc19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02da7fc640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02dc101900 0x7f02dc19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02da7fc640 1 -- 172.21.15.46:0/3775562456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02dc197800 con 0x7f02dc107900 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02da7fc640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc107900 0x7f02dc19dbc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02c4031460 tx=0x7f02c40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.137+0000 7f02d3fff640 1 -- 172.21.15.46:0/3775562456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02c403f070 con 0x7f02dc107900 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.138+0000 7f02d3fff640 1 -- 172.21.15.46:0/3775562456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02c40035f0 con 0x7f02dc107900 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.138+0000 7f02d3fff640 1 -- 172.21.15.46:0/3775562456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02c4035030 con 0x7f02dc107900 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.138+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02dc197a90 con 0x7f02dc107900 2024-09-10T04:27:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.138+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02dc197fc0 con 0x7f02dc107900 2024-09-10T04:27:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.139+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02dc103040 con 0x7f02dc107900 2024-09-10T04:27:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.139+0000 7f02d3fff640 1 -- 172.21.15.46:0/3775562456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f02c4042070 con 0x7f02dc107900 2024-09-10T04:27:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.140+0000 7f02d3fff640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807ac40 0x7f02c807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.140+0000 7f02daffd640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807ac40 0x7f02c807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.140+0000 7f02d3fff640 1 -- 172.21.15.46:0/3775562456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f02c40be4f0 con 0x7f02dc107900 2024-09-10T04:27:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.141+0000 7f02daffd640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807ac40 0x7f02c807d100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d4009680 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.145+0000 7f02d3fff640 1 -- 172.21.15.46:0/3775562456 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f02c4089cb0 con 0x7f02dc107900 2024-09-10T04:27:43.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.367+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f02dc109e70 con 0x7f02dc107900 2024-09-10T04:27:43.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.368+0000 7f02d3fff640 1 -- 172.21.15.46:0/3775562456 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f02c408eb60 con 0x7f02dc107900 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:27:43.370 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:27:43.371 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:27:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:43.373 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807ac40 msgr2=0x7f02c807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807ac40 0x7f02c807d100 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f02d4005fd0 tx=0x7f02d4009680 comp rx=0 tx=0).stop 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc107900 msgr2=0x7f02dc19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc107900 0x7f02dc19dbc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02c4031460 tx=0x7f02c40045e0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 shutdown_connections 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807ac40 0x7f02c807d100 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02dc107900 0x7f02dc19dbc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 --2- 172.21.15.46:0/3775562456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02dc101900 0x7f02dc19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 >> 172.21.15.46:0/3775562456 conn(0x7f02dc0fd600 msgr2=0x7f02dc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:43.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 shutdown_connections 2024-09-10T04:27:43.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.370+0000 7f02e129e640 1 -- 172.21.15.46:0/3775562456 wait complete. 2024-09-10T04:27:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.455+0000 7f01daaaf640 1 -- 172.21.15.46:0/2229236059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4102ab0 msgr2=0x7f01d4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.455+0000 7f01daaaf640 1 --2- 172.21.15.46:0/2229236059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4102ab0 0x7f01d4102e90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009880 tx=0x7f01bc02a530 comp rx=0 tx=0).stop 2024-09-10T04:27:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 -- 172.21.15.46:0/2229236059 shutdown_connections 2024-09-10T04:27:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 --2- 172.21.15.46:0/2229236059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01d41033d0 0x7f01d410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 --2- 172.21.15.46:0/2229236059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d4102ab0 0x7f01d4102e90 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 -- 172.21.15.46:0/2229236059 >> 172.21.15.46:0/2229236059 conn(0x7f01d40fc550 msgr2=0x7f01d40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 -- 172.21.15.46:0/2229236059 shutdown_connections 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 -- 172.21.15.46:0/2229236059 wait complete. 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 Processor -- start 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 -- start start 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01d4102ab0 0x7f01d4100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d41033d0 0x7f01d4100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01d410edc0 con 0x7f01d41033d0 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.456+0000 7f01daaaf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f01d410ec40 con 0x7f01d4102ab0 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d41033d0 0x7f01d4100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d41033d0 0x7f01d4100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52570/0 (socket says 172.21.15.46:52570) 2024-09-10T04:27:43.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d3fff640 1 -- 172.21.15.46:0/114735502 learned_addr learned my addr 172.21.15.46:0/114735502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:43.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d884b640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01d4102ab0 0x7f01d4100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d3fff640 1 -- 172.21.15.46:0/114735502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01d4102ab0 msgr2=0x7f01d4100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d3fff640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01d4102ab0 0x7f01d4100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d3fff640 1 -- 172.21.15.46:0/114735502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d4101190 con 0x7f01d41033d0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d3fff640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d41033d0 0x7f01d4100c50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f01c400da60 tx=0x7f01c400df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d1ffb640 1 -- 172.21.15.46:0/114735502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01c4017070 con 0x7f01d41033d0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d4101480 con 0x7f01d41033d0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d1ffb640 1 -- 172.21.15.46:0/114735502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01c4003e30 con 0x7f01d41033d0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.457+0000 7f01d1ffb640 1 -- 172.21.15.46:0/114735502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01c4010460 con 0x7f01d41033d0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.458+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d41a3e30 con 0x7f01d41033d0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.458+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d406a580 con 0x7f01d41033d0 2024-09-10T04:27:43.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.459+0000 7f01d1ffb640 1 -- 172.21.15.46:0/114735502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f01c400bbd0 con 0x7f01d41033d0 2024-09-10T04:27:43.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.459+0000 7f01d1ffb640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01c007ac00 0x7f01c007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.460+0000 7f01d1ffb640 1 -- 172.21.15.46:0/114735502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f01c409da40 con 0x7f01d41033d0 2024-09-10T04:27:43.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.460+0000 7f01d884b640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01c007ac00 0x7f01c007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.460+0000 7f01d884b640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01c007ac00 0x7f01c007d0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0096c0 tx=0x7f01bc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:43.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.464+0000 7f01d1ffb640 1 -- 172.21.15.46:0/114735502 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f01c4020070 con 0x7f01d41033d0 2024-09-10T04:27:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.674+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f01d4062630 con 0x7f01c007ac00 2024-09-10T04:27:43.676 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:43 smithi046 ceph-mon[109506]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:43.676 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:43 smithi046 ceph-mon[109506]: pgmap v661: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:43.676 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:43 smithi046 ceph-mon[109506]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:43.676 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:43 smithi046 ceph-mon[109506]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:43.676 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:43 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1842995140' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:27:43.676 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:43 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3775562456' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.675+0000 7f01d1ffb640 1 -- 172.21.15.46:0/114735502 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f01d4062630 con 0x7f01c007ac00 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:27:43.677 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:27:43.678 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:27:43.678 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:27:43.678 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:27:43.678 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:27:43.678 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01c007ac00 msgr2=0x7f01c007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01c007ac00 0x7f01c007d0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f01bc0096c0 tx=0x7f01bc031400 comp rx=0 tx=0).stop 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d41033d0 msgr2=0x7f01d4100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d41033d0 0x7f01d4100c50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f01c400da60 tx=0x7f01c400df20 comp rx=0 tx=0).stop 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 shutdown_connections 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01c007ac00 0x7f01c007d0c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01d41033d0 0x7f01d4100c50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 --2- 172.21.15.46:0/114735502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01d4102ab0 0x7f01d4100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 >> 172.21.15.46:0/114735502 conn(0x7f01d40fc550 msgr2=0x7f01d4108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:43.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 shutdown_connections 2024-09-10T04:27:43.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.677+0000 7f01daaaf640 1 -- 172.21.15.46:0/114735502 wait complete. 2024-09-10T04:27:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 -- 172.21.15.46:0/4272991593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 msgr2=0x7ffb9c10b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 --2- 172.21.15.46:0/4272991593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c10b930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ffb880098b0 tx=0x7ffb8802a560 comp rx=0 tx=0).stop 2024-09-10T04:27:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 -- 172.21.15.46:0/4272991593 shutdown_connections 2024-09-10T04:27:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 --2- 172.21.15.46:0/4272991593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c10b930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 --2- 172.21.15.46:0/4272991593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9c102b20 0x7ffb9c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 -- 172.21.15.46:0/4272991593 >> 172.21.15.46:0/4272991593 conn(0x7ffb9c0fc520 msgr2=0x7ffb9c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 -- 172.21.15.46:0/4272991593 shutdown_connections 2024-09-10T04:27:43.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.761+0000 7ffba3189640 1 -- 172.21.15.46:0/4272991593 wait complete. 2024-09-10T04:27:43.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffba3189640 1 Processor -- start 2024-09-10T04:27:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffba3189640 1 -- start start 2024-09-10T04:27:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffba3189640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9c102b20 0x7ffb9c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffba3189640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffba3189640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c10edd0 con 0x7ffb9c103440 2024-09-10T04:27:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffba3189640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c10ec50 con 0x7ffb9c102b20 2024-09-10T04:27:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52594/0 (socket says 172.21.15.46:52594) 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.762+0000 7ffb9bfff640 1 -- 172.21.15.46:0/4058169526 learned_addr learned my addr 172.21.15.46:0/4058169526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffba0f25640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9c102b20 0x7ffb9c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffb9bfff640 1 -- 172.21.15.46:0/4058169526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9c102b20 msgr2=0x7ffb9c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffb9bfff640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9c102b20 0x7ffb9c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffb9bfff640 1 -- 172.21.15.46:0/4058169526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c197880 con 0x7ffb9c103440 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffb9bfff640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c19dc00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ffb88031460 tx=0x7ffb880045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffb99ffb640 1 -- 172.21.15.46:0/4058169526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffb8803f070 con 0x7ffb9c103440 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c197b10 con 0x7ffb9c103440 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffb99ffb640 1 -- 172.21.15.46:0/4058169526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb880035f0 con 0x7ffb9c103440 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffb99ffb640 1 -- 172.21.15.46:0/4058169526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffb88035030 con 0x7ffb9c103440 2024-09-10T04:27:43.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.763+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c198040 con 0x7ffb9c103440 2024-09-10T04:27:43.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.764+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb9c06aa30 con 0x7ffb9c103440 2024-09-10T04:27:43.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.765+0000 7ffb99ffb640 1 -- 172.21.15.46:0/4058169526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ffb88042070 con 0x7ffb9c103440 2024-09-10T04:27:43.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.766+0000 7ffb99ffb640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffb8c07abf0 0x7ffb8c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:27:43.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.766+0000 7ffb99ffb640 1 -- 172.21.15.46:0/4058169526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7ffb880be5f0 con 0x7ffb9c103440 2024-09-10T04:27:43.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.766+0000 7ffba0f25640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffb8c07abf0 0x7ffb8c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:27:43.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.766+0000 7ffba0f25640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffb8c07abf0 0x7ffb8c07d0b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c1989c0 tx=0x7ffb9000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:27:43.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:43.770+0000 7ffb99ffb640 1 -- 172.21.15.46:0/4058169526 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ffb88089c90 con 0x7ffb9c103440 2024-09-10T04:27:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.027+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7ffb9c198db0 con 0x7ffb9c103440 2024-09-10T04:27:44.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.028+0000 7ffb99ffb640 1 -- 172.21.15.46:0/4058169526 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7ffb8808eb40 con 0x7ffb9c103440 2024-09-10T04:27:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:27:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:27:44.030 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:27:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:27:44.030 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:27:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:27:44.030 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:27:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffb8c07abf0 msgr2=0x7ffb8c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffb8c07abf0 0x7ffb8c07d0b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c1989c0 tx=0x7ffb9000b390 comp rx=0 tx=0).stop 2024-09-10T04:27:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 msgr2=0x7ffb9c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:27:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c19dc00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ffb88031460 tx=0x7ffb880045c0 comp rx=0 tx=0).stop 2024-09-10T04:27:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 shutdown_connections 2024-09-10T04:27:44.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffb8c07abf0 0x7ffb8c07d0b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9c103440 0x7ffb9c19dc00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 --2- 172.21.15.46:0/4058169526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb9c102b20 0x7ffb9c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:27:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 >> 172.21.15.46:0/4058169526 conn(0x7ffb9c0fc520 msgr2=0x7ffb9c103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:27:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 shutdown_connections 2024-09-10T04:27:44.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:27:44.030+0000 7ffba3189640 1 -- 172.21.15.46:0/4058169526 wait complete. 2024-09-10T04:27:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:43 smithi070 ceph-mon[79774]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:43 smithi070 ceph-mon[79774]: pgmap v661: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:43 smithi070 ceph-mon[79774]: from='client.35070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:43 smithi070 ceph-mon[79774]: from='client.35074 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:44.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:43 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1842995140' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:27:44.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:43 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3775562456' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:27:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:44 smithi070 ceph-mon[79774]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:44 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4058169526' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:27:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:44 smithi046 ceph-mon[109506]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:27:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:44 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4058169526' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:27:46.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:45 smithi070 ceph-mon[79774]: pgmap v662: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:46.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:45 smithi046 ceph-mon[109506]: pgmap v662: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:47 smithi070 ceph-mon[79774]: pgmap v663: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:47 smithi046 ceph-mon[109506]: pgmap v663: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:49.898 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:49 smithi046 ceph-mon[109506]: pgmap v664: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:50.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:49 smithi070 ceph-mon[79774]: pgmap v664: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:51 smithi070 ceph-mon[79774]: pgmap v665: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:52.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:51 smithi046 ceph-mon[109506]: pgmap v665: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:53 smithi070 ceph-mon[79774]: pgmap v666: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:53 smithi046 ceph-mon[109506]: pgmap v666: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:54.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:27:56.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:55 smithi070 ceph-mon[79774]: pgmap v667: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:55 smithi046 ceph-mon[109506]: pgmap v667: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:58.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:57 smithi070 ceph-mon[79774]: pgmap v668: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:27:58.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:57 smithi046 ceph-mon[109506]: pgmap v668: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:00.053 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:27:59 smithi046 ceph-mon[109506]: pgmap v669: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:00.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:27:59 smithi070 ceph-mon[79774]: pgmap v669: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:01 smithi070 ceph-mon[79774]: pgmap v670: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:01 smithi046 ceph-mon[109506]: pgmap v670: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:03.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:03 smithi046 ceph-mon[109506]: pgmap v671: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:04.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:03 smithi070 ceph-mon[79774]: pgmap v671: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:05 smithi070 ceph-mon[79774]: pgmap v672: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:06.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:05 smithi046 ceph-mon[109506]: pgmap v672: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:08.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:07 smithi070 ceph-mon[79774]: pgmap v673: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:08.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:07 smithi046 ceph-mon[109506]: pgmap v673: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:09 smithi070 ceph-mon[79774]: pgmap v674: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:10.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:09 smithi046 ceph-mon[109506]: pgmap v674: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:11 smithi070 ceph-mon[79774]: pgmap v675: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:11 smithi046 ceph-mon[109506]: pgmap v675: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:13 smithi046 ceph-mon[109506]: pgmap v676: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:14.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:13 smithi070 ceph-mon[79774]: pgmap v676: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:14.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.128+0000 7f6051cf2640 1 -- 172.21.15.46:0/3373363795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c1033e0 msgr2=0x7f604c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.128+0000 7f6051cf2640 1 --2- 172.21.15.46:0/3373363795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c1033e0 0x7f604c10b8d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6040009880 tx=0x7f604002a530 comp rx=0 tx=0).stop 2024-09-10T04:28:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.128+0000 7f6051cf2640 1 -- 172.21.15.46:0/3373363795 shutdown_connections 2024-09-10T04:28:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.128+0000 7f6051cf2640 1 --2- 172.21.15.46:0/3373363795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c1033e0 0x7f604c10b8d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.128+0000 7f6051cf2640 1 --2- 172.21.15.46:0/3373363795 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604c102ac0 0x7f604c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.128+0000 7f6051cf2640 1 -- 172.21.15.46:0/3373363795 >> 172.21.15.46:0/3373363795 conn(0x7f604c0fc520 msgr2=0x7f604c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.129+0000 7f6051cf2640 1 -- 172.21.15.46:0/3373363795 shutdown_connections 2024-09-10T04:28:14.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.129+0000 7f6051cf2640 1 -- 172.21.15.46:0/3373363795 wait complete. 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.129+0000 7f6051cf2640 1 Processor -- start 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.129+0000 7f6051cf2640 1 -- start start 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f6051cf2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c102ac0 0x7f604c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f6051cf2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604c1033e0 0x7f604c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f6051cf2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f604c10edf0 con 0x7f604c102ac0 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f6051cf2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f604c10ec70 con 0x7f604c1033e0 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c102ac0 0x7f604c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c102ac0 0x7f604c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60636/0 (socket says 172.21.15.46:60636) 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604b7fe640 1 -- 172.21.15.46:0/4059552566 learned_addr learned my addr 172.21.15.46:0/4059552566 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604affd640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604c1033e0 0x7f604c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604b7fe640 1 -- 172.21.15.46:0/4059552566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604c1033e0 msgr2=0x7f604c100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604b7fe640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604c1033e0 0x7f604c100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604b7fe640 1 -- 172.21.15.46:0/4059552566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1011a0 con 0x7f604c102ac0 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.130+0000 7f604affd640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604c1033e0 0x7f604c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.131+0000 7f604b7fe640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c102ac0 0x7f604c100720 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c8f0 tx=0x7f603c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.131+0000 7f6048ff9640 1 -- 172.21.15.46:0/4059552566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f603c0079b0 con 0x7f604c102ac0 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.131+0000 7f6048ff9640 1 -- 172.21.15.46:0/4059552566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f603c0054e0 con 0x7f604c102ac0 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.131+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c101490 con 0x7f604c102ac0 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.131+0000 7f6048ff9640 1 -- 172.21.15.46:0/4059552566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f603c004dd0 con 0x7f604c102ac0 2024-09-10T04:28:14.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.131+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c1a3ea0 con 0x7f604c102ac0 2024-09-10T04:28:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.132+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604c06aa90 con 0x7f604c102ac0 2024-09-10T04:28:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.133+0000 7f6048ff9640 1 -- 172.21.15.46:0/4059552566 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f603c005020 con 0x7f604c102ac0 2024-09-10T04:28:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.133+0000 7f6048ff9640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f603407abf0 0x7f603407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.133+0000 7f6048ff9640 1 -- 172.21.15.46:0/4059552566 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f603c09dbb0 con 0x7f604c102ac0 2024-09-10T04:28:14.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.134+0000 7f604affd640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f603407abf0 0x7f603407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.134+0000 7f604affd640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f603407abf0 0x7f603407d0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f604c1021b0 tx=0x7f6040009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:14.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.138+0000 7f6048ff9640 1 -- 172.21.15.46:0/4059552566 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f603c0a4080 con 0x7f604c102ac0 2024-09-10T04:28:14.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.345+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f604c062630 con 0x7f603407abf0 2024-09-10T04:28:14.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.346+0000 7f6048ff9640 1 -- 172.21.15.46:0/4059552566 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f604c062630 con 0x7f603407abf0 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f603407abf0 msgr2=0x7f603407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f603407abf0 0x7f603407d0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f604c1021b0 tx=0x7f6040009560 comp rx=0 tx=0).stop 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c102ac0 msgr2=0x7f604c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c102ac0 0x7f604c100720 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f603c00c8f0 tx=0x7f603c00cdb0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 shutdown_connections 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f603407abf0 0x7f603407d0b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f604c1033e0 0x7f604c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 --2- 172.21.15.46:0/4059552566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f604c102ac0 0x7f604c100720 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 >> 172.21.15.46:0/4059552566 conn(0x7f604c0fc520 msgr2=0x7f604c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:14.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 shutdown_connections 2024-09-10T04:28:14.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.349+0000 7f6051cf2640 1 -- 172.21.15.46:0/4059552566 wait complete. 2024-09-10T04:28:14.360 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:28:14.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.435+0000 7efce5c4f640 1 -- 172.21.15.46:0/4246359096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0100800 msgr2=0x7efce0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.435+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4246359096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0100800 0x7efce0100c60 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009880 tx=0x7efcd402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.435+0000 7efce5c4f640 1 -- 172.21.15.46:0/4246359096 shutdown_connections 2024-09-10T04:28:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.435+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4246359096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0100800 0x7efce0100c60 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.435+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4246359096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efce0106800 0x7efce0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.435+0000 7efce5c4f640 1 -- 172.21.15.46:0/4246359096 >> 172.21.15.46:0/4246359096 conn(0x7efce00fc520 msgr2=0x7efce00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.436+0000 7efce5c4f640 1 -- 172.21.15.46:0/4246359096 shutdown_connections 2024-09-10T04:28:14.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.436+0000 7efce5c4f640 1 -- 172.21.15.46:0/4246359096 wait complete. 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.436+0000 7efce5c4f640 1 Processor -- start 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.436+0000 7efce5c4f640 1 -- start start 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efce5c4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efce0100800 0x7efce019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efce5c4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0106800 0x7efce019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efce5c4f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efce010dcb0 con 0x7efce0106800 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efce5c4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efce010db30 con 0x7efce0100800 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0106800 0x7efce019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0106800 0x7efce019fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60646/0 (socket says 172.21.15.46:60646) 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdeffd640 1 -- 172.21.15.46:0/4008211104 learned_addr learned my addr 172.21.15.46:0/4008211104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdf7fe640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efce0100800 0x7efce019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdeffd640 1 -- 172.21.15.46:0/4008211104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efce0100800 msgr2=0x7efce019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdeffd640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efce0100800 0x7efce019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdeffd640 1 -- 172.21.15.46:0/4008211104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce01999c0 con 0x7efce0106800 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.437+0000 7efcdeffd640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0106800 0x7efce019fd40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efcd4031460 tx=0x7efcd4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.438+0000 7efcdcff9640 1 -- 172.21.15.46:0/4008211104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efcd403f070 con 0x7efce0106800 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.438+0000 7efcdcff9640 1 -- 172.21.15.46:0/4008211104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efcd40035f0 con 0x7efce0106800 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.438+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce0199c50 con 0x7efce0106800 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.438+0000 7efcdcff9640 1 -- 172.21.15.46:0/4008211104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efcd4035030 con 0x7efce0106800 2024-09-10T04:28:14.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.438+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce019a180 con 0x7efce0106800 2024-09-10T04:28:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.440+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce0101f40 con 0x7efce0106800 2024-09-10T04:28:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.440+0000 7efcdcff9640 1 -- 172.21.15.46:0/4008211104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efcd4042070 con 0x7efce0106800 2024-09-10T04:28:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.440+0000 7efcdcff9640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efcc807abf0 0x7efcc807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.441+0000 7efcdcff9640 1 -- 172.21.15.46:0/4008211104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7efcd40be6a0 con 0x7efce0106800 2024-09-10T04:28:14.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.441+0000 7efcdf7fe640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efcc807abf0 0x7efcc807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.441+0000 7efcdf7fe640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efcc807abf0 0x7efcc807d0b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efce019ab00 tx=0x7efcd0005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.445+0000 7efcdcff9640 1 -- 172.21.15.46:0/4008211104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efcd4089de0 con 0x7efce0106800 2024-09-10T04:28:14.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.649+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7efce019a430 con 0x7efcc807abf0 2024-09-10T04:28:14.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.650+0000 7efcdcff9640 1 -- 172.21.15.46:0/4008211104 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7efce019a430 con 0x7efcc807abf0 2024-09-10T04:28:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efcc807abf0 msgr2=0x7efcc807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efcc807abf0 0x7efcc807d0b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efce019ab00 tx=0x7efcd0005f90 comp rx=0 tx=0).stop 2024-09-10T04:28:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0106800 msgr2=0x7efce019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0106800 0x7efce019fd40 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efcd4031460 tx=0x7efcd4004530 comp rx=0 tx=0).stop 2024-09-10T04:28:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 shutdown_connections 2024-09-10T04:28:14.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efcc807abf0 0x7efcc807d0b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efce0106800 0x7efce019fd40 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 --2- 172.21.15.46:0/4008211104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efce0100800 0x7efce019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 >> 172.21.15.46:0/4008211104 conn(0x7efce00fc520 msgr2=0x7efce006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 shutdown_connections 2024-09-10T04:28:14.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.653+0000 7efce5c4f640 1 -- 172.21.15.46:0/4008211104 wait complete. 2024-09-10T04:28:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 -- 172.21.15.46:0/3219839986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 msgr2=0x7f2398100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 --2- 172.21.15.46:0/3219839986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398100c60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f238c009880 tx=0x7f238c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 -- 172.21.15.46:0/3219839986 shutdown_connections 2024-09-10T04:28:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 --2- 172.21.15.46:0/3219839986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398100c60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 --2- 172.21.15.46:0/3219839986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2398106800 0x7f2398106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 -- 172.21.15.46:0/3219839986 >> 172.21.15.46:0/3219839986 conn(0x7f23980fc520 msgr2=0x7f23980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 -- 172.21.15.46:0/3219839986 shutdown_connections 2024-09-10T04:28:14.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.738+0000 7f23a0464640 1 -- 172.21.15.46:0/3219839986 wait complete. 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.739+0000 7f23a0464640 1 Processor -- start 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.739+0000 7f23a0464640 1 -- start start 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.739+0000 7f23a0464640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.739+0000 7f23a0464640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2398106800 0x7f2398198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.739+0000 7f23a0464640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f239810dcd0 con 0x7f2398100800 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.739+0000 7f23a0464640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f239810db50 con 0x7f2398106800 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60664/0 (socket says 172.21.15.46:60664) 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239e200640 1 -- 172.21.15.46:0/986783272 learned_addr learned my addr 172.21.15.46:0/986783272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239d9ff640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2398106800 0x7f2398198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239e200640 1 -- 172.21.15.46:0/986783272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2398106800 msgr2=0x7f2398198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239e200640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2398106800 0x7f2398198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239e200640 1 -- 172.21.15.46:0/986783272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2398199360 con 0x7f2398100800 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f239e200640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398198820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f239400c8f0 tx=0x7f239400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f238b7fe640 1 -- 172.21.15.46:0/986783272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f23940079b0 con 0x7f2398100800 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f238b7fe640 1 -- 172.21.15.46:0/986783272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f23940054e0 con 0x7f2398100800 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f238b7fe640 1 -- 172.21.15.46:0/986783272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2394004dd0 con 0x7f2398100800 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.740+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2398199650 con 0x7f2398100800 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.741+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23981a1c50 con 0x7f2398100800 2024-09-10T04:28:14.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.742+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2398101f40 con 0x7f2398100800 2024-09-10T04:28:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.742+0000 7f238b7fe640 1 -- 172.21.15.46:0/986783272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2394005020 con 0x7f2398100800 2024-09-10T04:28:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.743+0000 7f238b7fe640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f238407ab90 0x7f238407d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.743+0000 7f238b7fe640 1 -- 172.21.15.46:0/986783272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f239409dc10 con 0x7f2398100800 2024-09-10T04:28:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.743+0000 7f239d9ff640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f238407ab90 0x7f238407d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:14.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.743+0000 7f239d9ff640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f238407ab90 0x7f238407d050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f239806ad00 tx=0x7f238c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:14.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.747+0000 7f238b7fe640 1 -- 172.21.15.46:0/986783272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2398199650 con 0x7f2398100800 2024-09-10T04:28:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.940+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2398062630 con 0x7f238407ab90 2024-09-10T04:28:14.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.950+0000 7f238b7fe640 1 -- 172.21.15.46:0/986783272 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f2398062630 con 0x7f238407ab90 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (19m) 14m ago 31m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (22m) 14m ago 31m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (22m) 14m ago 30m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (18m) 14m ago 31m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (18m) 14m ago 30m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (19m) 14m ago 30m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (14m) 14m ago 25m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (14m) 14m ago 25m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (14m) 14m ago 25m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:28:14.952 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (25m) 14m ago 25m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (21m) 14m ago 32m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (20m) 14m ago 30m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (18m) 14m ago 32m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (18m) 14m ago 30m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (20m) 14m ago 31m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (20m) 14m ago 30m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (17m) 14m ago 29m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (17m) 14m ago 28m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (16m) 14m ago 28m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (16m) 14m ago 27m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (15m) 14m ago 27m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (14m) 14m ago 26m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:28:14.953 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (20m) 14m ago 30m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:28:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f238407ab90 msgr2=0x7f238407d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f238407ab90 0x7f238407d050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f239806ad00 tx=0x7f238c031400 comp rx=0 tx=0).stop 2024-09-10T04:28:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 msgr2=0x7f2398198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:14.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398198820 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f239400c8f0 tx=0x7f239400cdb0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 shutdown_connections 2024-09-10T04:28:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f238407ab90 0x7f238407d050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2398106800 0x7f2398198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 --2- 172.21.15.46:0/986783272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2398100800 0x7f2398198820 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 >> 172.21.15.46:0/986783272 conn(0x7f23980fc520 msgr2=0x7f23980fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 shutdown_connections 2024-09-10T04:28:14.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:14.952+0000 7f23a0464640 1 -- 172.21.15.46:0/986783272 wait complete. 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 -- 172.21.15.46:0/1500730021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220103470 msgr2=0x7fb22010b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 --2- 172.21.15.46:0/1500730021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220103470 0x7fb22010b960 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb214009880 tx=0x7fb21402a530 comp rx=0 tx=0).stop 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 -- 172.21.15.46:0/1500730021 shutdown_connections 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 --2- 172.21.15.46:0/1500730021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220103470 0x7fb22010b960 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 --2- 172.21.15.46:0/1500730021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb220102b50 0x7fb220102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 -- 172.21.15.46:0/1500730021 >> 172.21.15.46:0/1500730021 conn(0x7fb2200fc550 msgr2=0x7fb2200fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 -- 172.21.15.46:0/1500730021 shutdown_connections 2024-09-10T04:28:15.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 -- 172.21.15.46:0/1500730021 wait complete. 2024-09-10T04:28:15.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.036+0000 7fb22556b640 1 Processor -- start 2024-09-10T04:28:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb22556b640 1 -- start start 2024-09-10T04:28:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb22556b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220102b50 0x7fb22019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb22556b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb220103470 0x7fb22019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb22556b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb22010ede0 con 0x7fb220102b50 2024-09-10T04:28:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb22556b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb22010ec60 con 0x7fb220103470 2024-09-10T04:28:15.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220102b50 0x7fb22019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220102b50 0x7fb22019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60678/0 (socket says 172.21.15.46:60678) 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21effd640 1 -- 172.21.15.46:0/1099150956 learned_addr learned my addr 172.21.15.46:0/1099150956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21e7fc640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb220103470 0x7fb22019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21effd640 1 -- 172.21.15.46:0/1099150956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb220103470 msgr2=0x7fb22019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21effd640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb220103470 0x7fb22019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21effd640 1 -- 172.21.15.46:0/1099150956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2201977b0 con 0x7fb220102b50 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21e7fc640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb220103470 0x7fb22019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.037+0000 7fb21effd640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220102b50 0x7fb22019d630 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb21000ea00 tx=0x7fb21000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.038+0000 7fb207fff640 1 -- 172.21.15.46:0/1099150956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb21000cb80 con 0x7fb220102b50 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.038+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb220197aa0 con 0x7fb220102b50 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.038+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb220198000 con 0x7fb220102b50 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.038+0000 7fb207fff640 1 -- 172.21.15.46:0/1099150956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb21000cd20 con 0x7fb220102b50 2024-09-10T04:28:15.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.038+0000 7fb207fff640 1 -- 172.21.15.46:0/1099150956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb210011470 con 0x7fb220102b50 2024-09-10T04:28:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.039+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c005290 con 0x7fb220102b50 2024-09-10T04:28:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.040+0000 7fb207fff640 1 -- 172.21.15.46:0/1099150956 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb210004a40 con 0x7fb220102b50 2024-09-10T04:28:15.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.041+0000 7fb207fff640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb20807ac40 0x7fb20807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.041+0000 7fb207fff640 1 -- 172.21.15.46:0/1099150956 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fb21009dff0 con 0x7fb220102b50 2024-09-10T04:28:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.041+0000 7fb21e7fc640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb20807ac40 0x7fb20807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.041+0000 7fb21e7fc640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb20807ac40 0x7fb20807d100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb220198b00 tx=0x7fb214004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.045+0000 7fb207fff640 1 -- 172.21.15.46:0/1099150956 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb210016030 con 0x7fb220102b50 2024-09-10T04:28:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.306+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fb20c005d90 con 0x7fb220102b50 2024-09-10T04:28:15.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.306+0000 7fb207fff640 1 -- 172.21.15.46:0/1099150956 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7fb210069750 con 0x7fb220102b50 2024-09-10T04:28:15.308 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:28:15.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:28:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:28:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:28:15.310 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:28:15.310 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:28:15.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb20807ac40 msgr2=0x7fb20807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb20807ac40 0x7fb20807d100 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb220198b00 tx=0x7fb214004f30 comp rx=0 tx=0).stop 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220102b50 msgr2=0x7fb22019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220102b50 0x7fb22019d630 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb21000ea00 tx=0x7fb21000eec0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 shutdown_connections 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb20807ac40 0x7fb20807d100 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb220103470 0x7fb22019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 --2- 172.21.15.46:0/1099150956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb220102b50 0x7fb22019d630 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 >> 172.21.15.46:0/1099150956 conn(0x7fb2200fc550 msgr2=0x7fb220103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 shutdown_connections 2024-09-10T04:28:15.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.309+0000 7fb22556b640 1 -- 172.21.15.46:0/1099150956 wait complete. 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/3760954517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1007f0 msgr2=0x7fcfbc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/3760954517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1007f0 0x7fcfbc100c50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009880 tx=0x7fcfa402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/3760954517 shutdown_connections 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/3760954517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1007f0 0x7fcfbc100c50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/3760954517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfbc1067f0 0x7fcfbc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/3760954517 >> 172.21.15.46:0/3760954517 conn(0x7fcfbc0fc550 msgr2=0x7fcfbc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/3760954517 shutdown_connections 2024-09-10T04:28:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.412+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/3760954517 wait complete. 2024-09-10T04:28:15.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfc1a3d640 1 Processor -- start 2024-09-10T04:28:15.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfc1a3d640 1 -- start start 2024-09-10T04:28:15.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfc1a3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfbc1007f0 0x7fcfbc19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfc1a3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1067f0 0x7fcfbc19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfc1a3d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc10dca0 con 0x7fcfbc1067f0 2024-09-10T04:28:15.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfc1a3d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc10db20 con 0x7fcfbc1007f0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfba7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1067f0 0x7fcfbc19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfba7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1067f0 0x7fcfbc19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60694/0 (socket says 172.21.15.46:60694) 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfba7fc640 1 -- 172.21.15.46:0/1702463455 learned_addr learned my addr 172.21.15.46:0/1702463455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfbaffd640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfbc1007f0 0x7fcfbc19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfba7fc640 1 -- 172.21.15.46:0/1702463455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfbc1007f0 msgr2=0x7fcfbc19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.413+0000 7fcfba7fc640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfbc1007f0 0x7fcfbc19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.414+0000 7fcfba7fc640 1 -- 172.21.15.46:0/1702463455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc197780 con 0x7fcfbc1067f0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.414+0000 7fcfba7fc640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1067f0 0x7fcfbc19db40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4031460 tx=0x7fcfa40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.414+0000 7fcfc0a3b640 1 -- 172.21.15.46:0/1702463455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcfa403f070 con 0x7fcfbc1067f0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.414+0000 7fcfc0a3b640 1 -- 172.21.15.46:0/1702463455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcfa4004cf0 con 0x7fcfbc1067f0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.414+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc197a10 con 0x7fcfbc1067f0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.414+0000 7fcfc0a3b640 1 -- 172.21.15.46:0/1702463455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcfa4035030 con 0x7fcfbc1067f0 2024-09-10T04:28:15.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.414+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc197f40 con 0x7fcfbc1067f0 2024-09-10T04:28:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.415+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc101f30 con 0x7fcfbc1067f0 2024-09-10T04:28:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.416+0000 7fcfc0a3b640 1 -- 172.21.15.46:0/1702463455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcfa4042070 con 0x7fcfbc1067f0 2024-09-10T04:28:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.416+0000 7fcfc0a3b640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfa807ac40 0x7fcfa807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.416+0000 7fcfc0a3b640 1 -- 172.21.15.46:0/1702463455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fcfa40be6b0 con 0x7fcfbc1067f0 2024-09-10T04:28:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.416+0000 7fcfbaffd640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfa807ac40 0x7fcfa807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.417+0000 7fcfbaffd640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfa807ac40 0x7fcfa807d100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcfac007350 tx=0x7fcfac0072e0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:15.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.421+0000 7fcfc0a3b640 1 -- 172.21.15.46:0/1702463455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fcfa4089e70 con 0x7fcfbc1067f0 2024-09-10T04:28:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.648+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fcfbc198cb0 con 0x7fcfbc1067f0 2024-09-10T04:28:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.648+0000 7fcfc0a3b640 1 -- 172.21.15.46:0/1702463455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7fcfa408ed20 con 0x7fcfbc1067f0 2024-09-10T04:28:15.650 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:28:15.650 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:28:15.651 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:28:15.651 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:28:15.651 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:28:15.651 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:15.651 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:28:15.651 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:28:15.651 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:28:15.652 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:28:15.653 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:28:15.653 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:28:15.653 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:28:15.653 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:28:15.653 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:28:15.653 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:28:15.653 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:28:15.654 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:15.655 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:15.656 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:15.656 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:28:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfa807ac40 msgr2=0x7fcfa807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfa807ac40 0x7fcfa807d100 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcfac007350 tx=0x7fcfac0072e0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1067f0 msgr2=0x7fcfbc19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1067f0 0x7fcfbc19db40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4031460 tx=0x7fcfa40048f0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 shutdown_connections 2024-09-10T04:28:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfa807ac40 0x7fcfa807d100 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfbc1067f0 0x7fcfbc19db40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 --2- 172.21.15.46:0/1702463455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfbc1007f0 0x7fcfbc19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.650+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 >> 172.21.15.46:0/1702463455 conn(0x7fcfbc0fc550 msgr2=0x7fcfbc069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.651+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 shutdown_connections 2024-09-10T04:28:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.651+0000 7fcfc1a3d640 1 -- 172.21.15.46:0/1702463455 wait complete. 2024-09-10T04:28:15.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 -- 172.21.15.46:0/2955774648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 msgr2=0x7fbfc8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 --2- 172.21.15.46:0/2955774648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc8100c50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009880 tx=0x7fbfbc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 -- 172.21.15.46:0/2955774648 shutdown_connections 2024-09-10T04:28:15.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 --2- 172.21.15.46:0/2955774648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc8100c50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 --2- 172.21.15.46:0/2955774648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc81067f0 0x7fbfc8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 -- 172.21.15.46:0/2955774648 >> 172.21.15.46:0/2955774648 conn(0x7fbfc80fc550 msgr2=0x7fbfc80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 -- 172.21.15.46:0/2955774648 shutdown_connections 2024-09-10T04:28:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 -- 172.21.15.46:0/2955774648 wait complete. 2024-09-10T04:28:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 Processor -- start 2024-09-10T04:28:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.737+0000 7fbfce89b640 1 -- start start 2024-09-10T04:28:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfce89b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfce89b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc81067f0 0x7fbfc819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfce89b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc810dcc0 con 0x7fbfc81007f0 2024-09-10T04:28:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfce89b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc810db40 con 0x7fbfc81067f0 2024-09-10T04:28:15.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfc7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60718/0 (socket says 172.21.15.46:60718) 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfc7fff640 1 -- 172.21.15.46:0/1941048399 learned_addr learned my addr 172.21.15.46:0/1941048399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfc77fe640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc81067f0 0x7fbfc819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfc7fff640 1 -- 172.21.15.46:0/1941048399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc81067f0 msgr2=0x7fbfc819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfc7fff640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc81067f0 0x7fbfc819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.738+0000 7fbfc7fff640 1 -- 172.21.15.46:0/1941048399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc81977e0 con 0x7fbfc81007f0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.739+0000 7fbfc7fff640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc819d660 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800d6f0 tx=0x7fbfb800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.739+0000 7fbfc57fa640 1 -- 172.21.15.46:0/1941048399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbfb8017070 con 0x7fbfc81007f0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.739+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc8197ad0 con 0x7fbfc81007f0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.739+0000 7fbfc57fa640 1 -- 172.21.15.46:0/1941048399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbfb800de00 con 0x7fbfc81007f0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.739+0000 7fbfc57fa640 1 -- 172.21.15.46:0/1941048399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbfb8004bb0 con 0x7fbfc81007f0 2024-09-10T04:28:15.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.739+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc8198030 con 0x7fbfc81007f0 2024-09-10T04:28:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.740+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc8101f30 con 0x7fbfc81007f0 2024-09-10T04:28:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.741+0000 7fbfc57fa640 1 -- 172.21.15.46:0/1941048399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbfb8004e60 con 0x7fbfc81007f0 2024-09-10T04:28:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.741+0000 7fbfc57fa640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbfb007abf0 0x7fbfb007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.741+0000 7fbfc57fa640 1 -- 172.21.15.46:0/1941048399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fbfb809d8d0 con 0x7fbfc81007f0 2024-09-10T04:28:15.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.741+0000 7fbfc77fe640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbfb007abf0 0x7fbfb007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:15.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.742+0000 7fbfc77fe640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbfb007abf0 0x7fbfb007d0b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8198b30 tx=0x7fbfbc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:15.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.745+0000 7fbfc57fa640 1 -- 172.21.15.46:0/1941048399 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbfb8069010 con 0x7fbfc81007f0 2024-09-10T04:28:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.949+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbfc81982e0 con 0x7fbfb007abf0 2024-09-10T04:28:15.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:15 smithi046 ceph-mon[109506]: pgmap v677: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:15.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:15 smithi046 ceph-mon[109506]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:15.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:15 smithi046 ceph-mon[109506]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:15.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:15 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1099150956' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:28:15.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:15 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1702463455' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.950+0000 7fbfc57fa640 1 -- 172.21.15.46:0/1941048399 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fbfc81982e0 con 0x7fbfb007abf0 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:28:15.952 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:28:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:28:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:28:15.953 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:28:15.953 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:28:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbfb007abf0 msgr2=0x7fbfb007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbfb007abf0 0x7fbfb007d0b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8198b30 tx=0x7fbfbc031400 comp rx=0 tx=0).stop 2024-09-10T04:28:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 msgr2=0x7fbfc819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc819d660 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800d6f0 tx=0x7fbfb800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 shutdown_connections 2024-09-10T04:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbfb007abf0 0x7fbfb007d0b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc81067f0 0x7fbfc819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 --2- 172.21.15.46:0/1941048399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbfc81007f0 0x7fbfc819d660 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 >> 172.21.15.46:0/1941048399 conn(0x7fbfc80fc550 msgr2=0x7fbfc8069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.952+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 shutdown_connections 2024-09-10T04:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:15.953+0000 7fbfce89b640 1 -- 172.21.15.46:0/1941048399 wait complete. 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 -- 172.21.15.46:0/1464860854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 msgr2=0x7f8f90107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 --2- 172.21.15.46:0/1464860854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90107ce0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009880 tx=0x7f8f8002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 -- 172.21.15.46:0/1464860854 shutdown_connections 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 --2- 172.21.15.46:0/1464860854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f90101900 0x7f8f90101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 --2- 172.21.15.46:0/1464860854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90107ce0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 -- 172.21.15.46:0/1464860854 >> 172.21.15.46:0/1464860854 conn(0x7f8f900fd600 msgr2=0x7f8f900ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 -- 172.21.15.46:0/1464860854 shutdown_connections 2024-09-10T04:28:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.037+0000 7f8f95180640 1 -- 172.21.15.46:0/1464860854 wait complete. 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f95180640 1 Processor -- start 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f95180640 1 -- start start 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f95180640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f90101900 0x7f8f900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f95180640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f95180640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9010edb0 con 0x7f8f90107900 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f95180640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9010ec30 con 0x7f8f90101900 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f8e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f8e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60744/0 (socket says 172.21.15.46:60744) 2024-09-10T04:28:16.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f8e59c640 1 -- 172.21.15.46:0/1975834492 learned_addr learned my addr 172.21.15.46:0/1975834492 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.038+0000 7f8f8ed9d640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f90101900 0x7f8f900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f8e59c640 1 -- 172.21.15.46:0/1975834492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f90101900 msgr2=0x7f8f900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f8e59c640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f90101900 0x7f8f900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f8e59c640 1 -- 172.21.15.46:0/1975834492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f90074f70 con 0x7f8f90107900 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f8e59c640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90074a30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400cc60 tx=0x7f8f8400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f77fff640 1 -- 172.21.15.46:0/1975834492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f8400ce10 con 0x7f8f90107900 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f90075260 con 0x7f8f90107900 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f77fff640 1 -- 172.21.15.46:0/1975834492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f8400f910 con 0x7f8f90107900 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f77fff640 1 -- 172.21.15.46:0/1975834492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f84018ae0 con 0x7f8f90107900 2024-09-10T04:28:16.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.039+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f901a3de0 con 0x7f8f90107900 2024-09-10T04:28:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.040+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f90103040 con 0x7f8f90107900 2024-09-10T04:28:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.041+0000 7f8f77fff640 1 -- 172.21.15.46:0/1975834492 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8f84018ca0 con 0x7f8f90107900 2024-09-10T04:28:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.041+0000 7f8f77fff640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f7807abf0 0x7f8f7807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.041+0000 7f8f8ed9d640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f7807abf0 0x7f8f7807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:16.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.042+0000 7f8f77fff640 1 -- 172.21.15.46:0/1975834492 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f8f8409e380 con 0x7f8f90107900 2024-09-10T04:28:16.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.042+0000 7f8f8ed9d640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f7807abf0 0x7f8f7807d0b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f80031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:16.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.046+0000 7f8f77fff640 1 -- 172.21.15.46:0/1975834492 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8f84014030 con 0x7f8f90107900 2024-09-10T04:28:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:15 smithi070 ceph-mon[79774]: pgmap v677: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:15 smithi070 ceph-mon[79774]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:15 smithi070 ceph-mon[79774]: from='client.35098 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:15 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1099150956' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:28:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:15 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1702463455' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.307+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f8f9002d7c0 con 0x7f8f90107900 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.307+0000 7f8f77fff640 1 -- 172.21.15.46:0/1975834492 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7f8f84069a20 con 0x7f8f90107900 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:28:16.309 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:28:16.310 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f7807abf0 msgr2=0x7f8f7807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f7807abf0 0x7f8f7807d0b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f80031400 comp rx=0 tx=0).stop 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 msgr2=0x7f8f90074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90074a30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400cc60 tx=0x7f8f8400a620 comp rx=0 tx=0).stop 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 shutdown_connections 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f7807abf0 0x7f8f7807d0b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f90107900 0x7f8f90074a30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 --2- 172.21.15.46:0/1975834492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f90101900 0x7f8f900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 >> 172.21.15.46:0/1975834492 conn(0x7f8f900fd600 msgr2=0x7f8f900ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 shutdown_connections 2024-09-10T04:28:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:16.310+0000 7f8f95180640 1 -- 172.21.15.46:0/1975834492 wait complete. 2024-09-10T04:28:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:16 smithi070 ceph-mon[79774]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:16 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1975834492' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:28:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:16 smithi046 ceph-mon[109506]: from='client.35102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:16 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1975834492' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:28:18.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:17 smithi070 ceph-mon[79774]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:18.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:17 smithi070 ceph-mon[79774]: pgmap v678: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:18.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:17 smithi046 ceph-mon[109506]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:18.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:17 smithi046 ceph-mon[109506]: pgmap v678: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:19 smithi070 ceph-mon[79774]: pgmap v679: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:19 smithi046 ceph-mon[109506]: pgmap v679: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:21 smithi070 ceph-mon[79774]: pgmap v680: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:21 smithi046 ceph-mon[109506]: pgmap v680: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:23 smithi070 ceph-mon[79774]: pgmap v681: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:23 smithi046 ceph-mon[109506]: pgmap v681: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:24.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:25 smithi046 ceph-mon[109506]: pgmap v682: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:26.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:25 smithi070 ceph-mon[79774]: pgmap v682: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:28.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:27 smithi070 ceph-mon[79774]: pgmap v683: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:28.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:27 smithi046 ceph-mon[109506]: pgmap v683: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:29 smithi070 ceph-mon[79774]: pgmap v684: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:30.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:29 smithi046 ceph-mon[109506]: pgmap v684: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:32.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:31 smithi070 ceph-mon[79774]: pgmap v685: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:32.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:31 smithi046 ceph-mon[109506]: pgmap v685: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:33 smithi046 ceph-mon[109506]: pgmap v686: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:33 smithi070 ceph-mon[79774]: pgmap v686: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:36.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:35 smithi070 ceph-mon[79774]: pgmap v687: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:36.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:35 smithi046 ceph-mon[109506]: pgmap v687: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:37.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:36 smithi070 ceph-mon[79774]: pgmap v688: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:37.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:36 smithi046 ceph-mon[109506]: pgmap v688: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:38.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:38.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:39 smithi070 ceph-mon[79774]: pgmap v689: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:39.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:39 smithi046 ceph-mon[109506]: pgmap v689: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:41.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:41 smithi046 ceph-mon[109506]: pgmap v690: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:41 smithi070 ceph-mon[79774]: pgmap v690: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:43 smithi070 ceph-mon[79774]: pgmap v691: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:43.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:43 smithi046 ceph-mon[109506]: pgmap v691: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:45.771 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:45 smithi046 ceph-mon[109506]: pgmap v692: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:45 smithi070 ceph-mon[79774]: pgmap v692: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:46.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.409+0000 7fbb27a3a640 1 -- 172.21.15.46:0/4081625841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 msgr2=0x7fbb20074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.409+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/4081625841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb20074970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c009880 tx=0x7fbb1c02a530 comp rx=0 tx=0).stop 2024-09-10T04:28:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.409+0000 7fbb27a3a640 1 -- 172.21.15.46:0/4081625841 shutdown_connections 2024-09-10T04:28:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.409+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/4081625841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb20074eb0 0x7fbb201123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.409+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/4081625841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb20074970 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.409+0000 7fbb27a3a640 1 -- 172.21.15.46:0/4081625841 >> 172.21.15.46:0/4081625841 conn(0x7fbb200fd600 msgr2=0x7fbb200ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.410+0000 7fbb27a3a640 1 -- 172.21.15.46:0/4081625841 shutdown_connections 2024-09-10T04:28:46.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.410+0000 7fbb27a3a640 1 -- 172.21.15.46:0/4081625841 wait complete. 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.410+0000 7fbb27a3a640 1 Processor -- start 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.410+0000 7fbb27a3a640 1 -- start start 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb27a3a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb2019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb27a3a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb20074eb0 0x7fbb2019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb27a3a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb201153d0 con 0x7fbb20074590 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb27a3a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbb20115250 con 0x7fbb20074eb0 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb257d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb2019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb257d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb2019dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39346/0 (socket says 172.21.15.46:39346) 2024-09-10T04:28:46.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb257d6640 1 -- 172.21.15.46:0/814199543 learned_addr learned my addr 172.21.15.46:0/814199543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb24fd5640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb20074eb0 0x7fbb2019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb257d6640 1 -- 172.21.15.46:0/814199543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb20074eb0 msgr2=0x7fbb2019e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb257d6640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb20074eb0 0x7fbb2019e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb257d6640 1 -- 172.21.15.46:0/814199543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb201a28c0 con 0x7fbb20074590 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.411+0000 7fbb257d6640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb2019dff0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c031460 tx=0x7fbb1c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.412+0000 7fbb167fc640 1 -- 172.21.15.46:0/814199543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb1c03f070 con 0x7fbb20074590 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.412+0000 7fbb167fc640 1 -- 172.21.15.46:0/814199543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbb1c0035f0 con 0x7fbb20074590 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.412+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb201a2b50 con 0x7fbb20074590 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.412+0000 7fbb167fc640 1 -- 172.21.15.46:0/814199543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbb1c035030 con 0x7fbb20074590 2024-09-10T04:28:46.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.412+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb201a3080 con 0x7fbb20074590 2024-09-10T04:28:46.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.413+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb20075cd0 con 0x7fbb20074590 2024-09-10T04:28:46.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.413+0000 7fbb167fc640 1 -- 172.21.15.46:0/814199543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbb1c042070 con 0x7fbb20074590 2024-09-10T04:28:46.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.414+0000 7fbb167fc640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb0c07abf0 0x7fbb0c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.414+0000 7fbb167fc640 1 -- 172.21.15.46:0/814199543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fbb1c0be670 con 0x7fbb20074590 2024-09-10T04:28:46.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.414+0000 7fbb24fd5640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb0c07abf0 0x7fbb0c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:46.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.415+0000 7fbb24fd5640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb0c07abf0 0x7fbb0c07d0b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb20104860 tx=0x7fbb1000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:46.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.418+0000 7fbb167fc640 1 -- 172.21.15.46:0/814199543 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbb1c089d10 con 0x7fbb20074590 2024-09-10T04:28:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.625+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbb201a33f0 con 0x7fbb0c07abf0 2024-09-10T04:28:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.627+0000 7fbb167fc640 1 -- 172.21.15.46:0/814199543 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fbb201a33f0 con 0x7fbb0c07abf0 2024-09-10T04:28:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb0c07abf0 msgr2=0x7fbb0c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb0c07abf0 0x7fbb0c07d0b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbb20104860 tx=0x7fbb1000b390 comp rx=0 tx=0).stop 2024-09-10T04:28:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 msgr2=0x7fbb2019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb2019dff0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c031460 tx=0x7fbb1c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 shutdown_connections 2024-09-10T04:28:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbb0c07abf0 0x7fbb0c07d0b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbb20074eb0 0x7fbb2019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 --2- 172.21.15.46:0/814199543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb20074590 0x7fbb2019dff0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 >> 172.21.15.46:0/814199543 conn(0x7fbb200fd600 msgr2=0x7fbb20104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.629+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 shutdown_connections 2024-09-10T04:28:46.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.630+0000 7fbb27a3a640 1 -- 172.21.15.46:0/814199543 wait complete. 2024-09-10T04:28:46.640 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:28:46.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 -- 172.21.15.46:0/4069701222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 msgr2=0x7fc89c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 --2- 172.21.15.46:0/4069701222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c107ce0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc898009880 tx=0x7fc89802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 -- 172.21.15.46:0/4069701222 shutdown_connections 2024-09-10T04:28:46.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 --2- 172.21.15.46:0/4069701222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc89c101900 0x7fc89c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 --2- 172.21.15.46:0/4069701222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c107ce0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 -- 172.21.15.46:0/4069701222 >> 172.21.15.46:0/4069701222 conn(0x7fc89c0fd600 msgr2=0x7fc89c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:46.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 -- 172.21.15.46:0/4069701222 shutdown_connections 2024-09-10T04:28:46.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.715+0000 7fc8a4110640 1 -- 172.21.15.46:0/4069701222 wait complete. 2024-09-10T04:28:46.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a4110640 1 Processor -- start 2024-09-10T04:28:46.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a4110640 1 -- start start 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a4110640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc89c101900 0x7fc89c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a4110640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a4110640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c10edd0 con 0x7fc89c107900 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a4110640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c10ec50 con 0x7fc89c101900 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a16ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a16ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39366/0 (socket says 172.21.15.46:39366) 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a16ab640 1 -- 172.21.15.46:0/90168276 learned_addr learned my addr 172.21.15.46:0/90168276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a1eac640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc89c101900 0x7fc89c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a16ab640 1 -- 172.21.15.46:0/90168276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc89c101900 msgr2=0x7fc89c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a16ab640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc89c101900 0x7fc89c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a16ab640 1 -- 172.21.15.46:0/90168276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c074f70 con 0x7fc89c107900 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.716+0000 7fc8a1eac640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc89c101900 0x7fc89c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.717+0000 7fc8a16ab640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c074a30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00e970 tx=0x7fc88c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.717+0000 7fc892ffd640 1 -- 172.21.15.46:0/90168276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc88c00cac0 con 0x7fc89c107900 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.717+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c075260 con 0x7fc89c107900 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.717+0000 7fc892ffd640 1 -- 172.21.15.46:0/90168276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc88c0054e0 con 0x7fc89c107900 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.717+0000 7fc892ffd640 1 -- 172.21.15.46:0/90168276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc88c010430 con 0x7fc89c107900 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.717+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1a3e40 con 0x7fc89c107900 2024-09-10T04:28:46.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.718+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c103040 con 0x7fc89c107900 2024-09-10T04:28:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.719+0000 7fc892ffd640 1 -- 172.21.15.46:0/90168276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc88c004f00 con 0x7fc89c107900 2024-09-10T04:28:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.719+0000 7fc892ffd640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc88807ac50 0x7fc88807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.719+0000 7fc892ffd640 1 -- 172.21.15.46:0/90168276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fc88c09e2f0 con 0x7fc89c107900 2024-09-10T04:28:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.719+0000 7fc8a1eac640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc88807ac50 0x7fc88807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:46.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.720+0000 7fc8a1eac640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc88807ac50 0x7fc88807d110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc898031460 tx=0x7fc8980041a0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:46.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.724+0000 7fc892ffd640 1 -- 172.21.15.46:0/90168276 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc88c069ab0 con 0x7fc89c107900 2024-09-10T04:28:46.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.927+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc89c062630 con 0x7fc88807ac50 2024-09-10T04:28:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.928+0000 7fc892ffd640 1 -- 172.21.15.46:0/90168276 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7fc89c062630 con 0x7fc88807ac50 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc88807ac50 msgr2=0x7fc88807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc88807ac50 0x7fc88807d110 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc898031460 tx=0x7fc8980041a0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 msgr2=0x7fc89c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c074a30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00e970 tx=0x7fc88c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 shutdown_connections 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc88807ac50 0x7fc88807d110 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc89c107900 0x7fc89c074a30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 --2- 172.21.15.46:0/90168276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc89c101900 0x7fc89c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 >> 172.21.15.46:0/90168276 conn(0x7fc89c0fd600 msgr2=0x7fc89c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 shutdown_connections 2024-09-10T04:28:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:46.931+0000 7fc8a4110640 1 -- 172.21.15.46:0/90168276 wait complete. 2024-09-10T04:28:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 -- 172.21.15.46:0/2621048230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 msgr2=0x7f010c106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 --2- 172.21.15.46:0/2621048230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c106bf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0108009880 tx=0x7f010802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 -- 172.21.15.46:0/2621048230 shutdown_connections 2024-09-10T04:28:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 --2- 172.21.15.46:0/2621048230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c100810 0x7f010c100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 --2- 172.21.15.46:0/2621048230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c106bf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 -- 172.21.15.46:0/2621048230 >> 172.21.15.46:0/2621048230 conn(0x7f010c0fc510 msgr2=0x7f010c0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:47.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:46 smithi046 ceph-mon[109506]: pgmap v693: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:47.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:46 smithi046 ceph-mon[109506]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 -- 172.21.15.46:0/2621048230 shutdown_connections 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.018+0000 7f0114010640 1 -- 172.21.15.46:0/2621048230 wait complete. 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f0114010640 1 Processor -- start 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f0114010640 1 -- start start 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f0114010640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c100810 0x7f010c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f0114010640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f0114010640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f010c10dce0 con 0x7f010c106810 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f0114010640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f010c10db60 con 0x7f010c100810 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f01115ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f01115ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c198d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39398/0 (socket says 172.21.15.46:39398) 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f01115ab640 1 -- 172.21.15.46:0/3735326712 learned_addr learned my addr 172.21.15.46:0/3735326712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.019+0000 7f0111dac640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c100810 0x7f010c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f01115ab640 1 -- 172.21.15.46:0/3735326712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c100810 msgr2=0x7f010c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f01115ab640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c100810 0x7f010c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f01115ab640 1 -- 172.21.15.46:0/3735326712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c199390 con 0x7f010c106810 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f01115ab640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c198d90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00d6f0 tx=0x7f00fc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f0102ffd640 1 -- 172.21.15.46:0/3735326712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00fc017070 con 0x7f010c106810 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f0102ffd640 1 -- 172.21.15.46:0/3735326712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00fc00de00 con 0x7f010c106810 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f0102ffd640 1 -- 172.21.15.46:0/3735326712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00fc004bb0 con 0x7f010c106810 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c199680 con 0x7f010c106810 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.020+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c1a1c80 con 0x7f010c106810 2024-09-10T04:28:47.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.021+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c101f50 con 0x7f010c106810 2024-09-10T04:28:47.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.022+0000 7f0102ffd640 1 -- 172.21.15.46:0/3735326712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f00fc004e60 con 0x7f010c106810 2024-09-10T04:28:47.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.023+0000 7f0102ffd640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00f807ac00 0x7f00f807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.023+0000 7f0111dac640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00f807ac00 0x7f00f807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.023+0000 7f0102ffd640 1 -- 172.21.15.46:0/3735326712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f00fc09dcb0 con 0x7f010c106810 2024-09-10T04:28:47.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.024+0000 7f0111dac640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00f807ac00 0x7f00f807d0c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f01080096c0 tx=0x7f0108031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:47.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.027+0000 7f0102ffd640 1 -- 172.21.15.46:0/3735326712 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f010c199680 con 0x7f010c106810 2024-09-10T04:28:47.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.219+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f010c062630 con 0x7f00f807ac00 2024-09-10T04:28:47.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.228+0000 7f0102ffd640 1 -- 172.21.15.46:0/3735326712 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f010c062630 con 0x7f00f807ac00 2024-09-10T04:28:47.230 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:28:47.230 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (20m) 14m ago 31m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (22m) 14m ago 31m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (22m) 14m ago 31m 9084k - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (18m) 14m ago 31m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (18m) 14m ago 31m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (19m) 14m ago 31m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (14m) 14m ago 26m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (15m) 14m ago 26m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (14m) 14m ago 26m 11.8M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (26m) 14m ago 26m 12.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 83c82eacb8a0 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (22m) 14m ago 32m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (21m) 14m ago 30m 457M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (19m) 14m ago 33m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:28:47.231 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (19m) 14m ago 30m 40.1M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (21m) 14m ago 31m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (21m) 14m ago 30m 11.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (18m) 14m ago 29m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (17m) 14m ago 29m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (17m) 14m ago 28m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (16m) 14m ago 28m 91.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (16m) 14m ago 27m 85.5M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (15m) 14m ago 27m 78.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:28:47.232 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (20m) 14m ago 31m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00f807ac00 msgr2=0x7f00f807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00f807ac00 0x7f00f807d0c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f01080096c0 tx=0x7f0108031400 comp rx=0 tx=0).stop 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 msgr2=0x7f010c198d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c198d90 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00d6f0 tx=0x7f00fc00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 shutdown_connections 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00f807ac00 0x7f00f807d0c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f010c106810 0x7f010c198d90 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 --2- 172.21.15.46:0/3735326712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f010c100810 0x7f010c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 >> 172.21.15.46:0/3735326712 conn(0x7f010c0fc510 msgr2=0x7f010c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:47.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 shutdown_connections 2024-09-10T04:28:47.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.231+0000 7f0114010640 1 -- 172.21.15.46:0/3735326712 wait complete. 2024-09-10T04:28:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 -- 172.21.15.46:0/4071906501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268106800 msgr2=0x7f9268106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 --2- 172.21.15.46:0/4071906501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268106800 0x7f9268106be0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9258009880 tx=0x7f925802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 -- 172.21.15.46:0/4071906501 shutdown_connections 2024-09-10T04:28:47.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 --2- 172.21.15.46:0/4071906501 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9268100800 0x7f9268100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 --2- 172.21.15.46:0/4071906501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268106800 0x7f9268106be0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 -- 172.21.15.46:0/4071906501 >> 172.21.15.46:0/4071906501 conn(0x7f92680fc520 msgr2=0x7f92680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:47.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 -- 172.21.15.46:0/4071906501 shutdown_connections 2024-09-10T04:28:47.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 -- 172.21.15.46:0/4071906501 wait complete. 2024-09-10T04:28:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.316+0000 7f926da6c640 1 Processor -- start 2024-09-10T04:28:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f926da6c640 1 -- start start 2024-09-10T04:28:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f926da6c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268100800 0x7f926819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f926da6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9268106800 0x7f926819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f926da6c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f926810dd30 con 0x7f9268100800 2024-09-10T04:28:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f926da6c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f926810dbb0 con 0x7f9268106800 2024-09-10T04:28:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f92677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268100800 0x7f926819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f92677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268100800 0x7f926819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39414/0 (socket says 172.21.15.46:39414) 2024-09-10T04:28:47.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f92677fe640 1 -- 172.21.15.46:0/3369168898 learned_addr learned my addr 172.21.15.46:0/3369168898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:47.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f9266ffd640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9268106800 0x7f926819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f92677fe640 1 -- 172.21.15.46:0/3369168898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9268106800 msgr2=0x7f926819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f92677fe640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9268106800 0x7f926819dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.317+0000 7f92677fe640 1 -- 172.21.15.46:0/3369168898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9268197880 con 0x7f9268100800 2024-09-10T04:28:47.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.318+0000 7f92677fe640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268100800 0x7f926819d6c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9258031460 tx=0x7f92580048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.318+0000 7f9264ff9640 1 -- 172.21.15.46:0/3369168898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f925803f070 con 0x7f9268100800 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.318+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9268197b10 con 0x7f9268100800 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.318+0000 7f9264ff9640 1 -- 172.21.15.46:0/3369168898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9258004cf0 con 0x7f9268100800 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.318+0000 7f9264ff9640 1 -- 172.21.15.46:0/3369168898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9258035030 con 0x7f9268100800 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.318+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9268198040 con 0x7f9268100800 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.319+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9268101f40 con 0x7f9268100800 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.319+0000 7f9264ff9640 1 -- 172.21.15.46:0/3369168898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9258042070 con 0x7f9268100800 2024-09-10T04:28:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.320+0000 7f9264ff9640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f925007abf0 0x7f925007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.320+0000 7f9266ffd640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f925007abf0 0x7f925007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.320+0000 7f9264ff9640 1 -- 172.21.15.46:0/3369168898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f92580be350 con 0x7f9268100800 2024-09-10T04:28:47.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.321+0000 7f9266ffd640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f925007abf0 0x7f925007d0b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9268104f50 tx=0x7f925c008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:47.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.325+0000 7f9264ff9640 1 -- 172.21.15.46:0/3369168898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9258089a90 con 0x7f9268100800 2024-09-10T04:28:47.334 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:46 smithi070 ceph-mon[79774]: pgmap v693: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:47.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:46 smithi070 ceph-mon[79774]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.589+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f9268100c60 con 0x7f9268100800 2024-09-10T04:28:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.590+0000 7f9264ff9640 1 -- 172.21.15.46:0/3369168898 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+835 (secure 0 0 0) 0x7f925808e940 con 0x7f9268100800 2024-09-10T04:28:47.592 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:28:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:28:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:28:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:28:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:28:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:28:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:28:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:28:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:28:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 3 2024-09-10T04:28:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:28:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:28:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)": 1, 2024-09-10T04:28:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 13 2024-09-10T04:28:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:28:47.596 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:28:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f925007abf0 msgr2=0x7f925007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f925007abf0 0x7f925007d0b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9268104f50 tx=0x7f925c008040 comp rx=0 tx=0).stop 2024-09-10T04:28:47.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268100800 msgr2=0x7f926819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268100800 0x7f926819d6c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9258031460 tx=0x7f92580048f0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 shutdown_connections 2024-09-10T04:28:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f925007abf0 0x7f925007d0b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9268106800 0x7f926819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 --2- 172.21.15.46:0/3369168898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9268100800 0x7f926819d6c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 >> 172.21.15.46:0/3369168898 conn(0x7f92680fc520 msgr2=0x7f926806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 shutdown_connections 2024-09-10T04:28:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.592+0000 7f926da6c640 1 -- 172.21.15.46:0/3369168898 wait complete. 2024-09-10T04:28:47.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 -- 172.21.15.46:0/2971989114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 msgr2=0x7f7e40106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 --2- 172.21.15.46:0/2971989114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40106be0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009880 tx=0x7f7e3c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 -- 172.21.15.46:0/2971989114 shutdown_connections 2024-09-10T04:28:47.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 --2- 172.21.15.46:0/2971989114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e40100800 0x7f7e40100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 --2- 172.21.15.46:0/2971989114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40106be0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 -- 172.21.15.46:0/2971989114 >> 172.21.15.46:0/2971989114 conn(0x7f7e400fc520 msgr2=0x7f7e400fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:47.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 -- 172.21.15.46:0/2971989114 shutdown_connections 2024-09-10T04:28:47.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.676+0000 7f7e46acb640 1 -- 172.21.15.46:0/2971989114 wait complete. 2024-09-10T04:28:47.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e46acb640 1 Processor -- start 2024-09-10T04:28:47.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e46acb640 1 -- start start 2024-09-10T04:28:47.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e46acb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e40100800 0x7f7e40198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e46acb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e46acb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4010dcd0 con 0x7f7e40106800 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e46acb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7e4010db50 con 0x7f7e40100800 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39430/0 (socket says 172.21.15.46:39430) 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e37fff640 1 -- 172.21.15.46:0/1524759574 learned_addr learned my addr 172.21.15.46:0/1524759574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.677+0000 7f7e44867640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e40100800 0x7f7e40198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e37fff640 1 -- 172.21.15.46:0/1524759574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e40100800 msgr2=0x7f7e40198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e37fff640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e40100800 0x7f7e40198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e37fff640 1 -- 172.21.15.46:0/1524759574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e40199350 con 0x7f7e40106800 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e44867640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e40100800 0x7f7e40198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e37fff640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40198d50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000d6f0 tx=0x7f7e3000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:47.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e35ffb640 1 -- 172.21.15.46:0/1524759574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e30017070 con 0x7f7e40106800 2024-09-10T04:28:47.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e40199640 con 0x7f7e40106800 2024-09-10T04:28:47.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e35ffb640 1 -- 172.21.15.46:0/1524759574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e3000de00 con 0x7f7e40106800 2024-09-10T04:28:47.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e35ffb640 1 -- 172.21.15.46:0/1524759574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e30004bb0 con 0x7f7e40106800 2024-09-10T04:28:47.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.678+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e401a1c40 con 0x7f7e40106800 2024-09-10T04:28:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.679+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e40101f40 con 0x7f7e40106800 2024-09-10T04:28:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.680+0000 7f7e35ffb640 1 -- 172.21.15.46:0/1524759574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7e30004e60 con 0x7f7e40106800 2024-09-10T04:28:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.680+0000 7f7e35ffb640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e2807abf0 0x7f7e2807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.680+0000 7f7e35ffb640 1 -- 172.21.15.46:0/1524759574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f7e3009d930 con 0x7f7e40106800 2024-09-10T04:28:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.681+0000 7f7e44867640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e2807abf0 0x7f7e2807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.681+0000 7f7e44867640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e2807abf0 0x7f7e2807d0b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c000c00 tx=0x7f7e3c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:47.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.685+0000 7f7e35ffb640 1 -- 172.21.15.46:0/1524759574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7e40199640 con 0x7f7e40106800 2024-09-10T04:28:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.907+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7e40108d70 con 0x7f7e40106800 2024-09-10T04:28:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.908+0000 7f7e35ffb640 1 -- 172.21.15.46:0/1524759574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+1994 (secure 0 0 0) 0x7f7e30069010 con 0x7f7e40106800 2024-09-10T04:28:47.910 INFO:teuthology.orchestra.run.smithi046.stdout:e25 2024-09-10T04:28:47.910 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:14:07:776361+0000 2024-09-10T04:28:47.910 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:28:47.910 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:28:47.910 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:28:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:28:47.911 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:28:47.912 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44295} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/470769983,v1:172.21.15.70:6827/470769983] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:47.913 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e2807abf0 msgr2=0x7f7e2807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e2807abf0 0x7f7e2807d0b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c000c00 tx=0x7f7e3c031400 comp rx=0 tx=0).stop 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 msgr2=0x7f7e40198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40198d50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000d6f0 tx=0x7f7e3000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 shutdown_connections 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e2807abf0 0x7f7e2807d0b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e40106800 0x7f7e40198d50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 --2- 172.21.15.46:0/1524759574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e40100800 0x7f7e40198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 >> 172.21.15.46:0/1524759574 conn(0x7f7e400fc520 msgr2=0x7f7e400fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 shutdown_connections 2024-09-10T04:28:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.911+0000 7f7e46acb640 1 -- 172.21.15.46:0/1524759574 wait complete. 2024-09-10T04:28:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.996+0000 7f23c3166640 1 -- 172.21.15.46:0/3909148544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 msgr2=0x7f23bc100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.996+0000 7f23c3166640 1 --2- 172.21.15.46:0/3909148544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc100c50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009880 tx=0x7f23b002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 -- 172.21.15.46:0/3909148544 shutdown_connections 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 --2- 172.21.15.46:0/3909148544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc100c50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 --2- 172.21.15.46:0/3909148544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23bc1067f0 0x7f23bc106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 -- 172.21.15.46:0/3909148544 >> 172.21.15.46:0/3909148544 conn(0x7f23bc0fc550 msgr2=0x7f23bc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 -- 172.21.15.46:0/3909148544 shutdown_connections 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 -- 172.21.15.46:0/3909148544 wait complete. 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 Processor -- start 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 -- start start 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:48.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23bc1067f0 0x7f23bc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc10dcc0 con 0x7f23bc1007f0 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.997+0000 7f23c3166640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f23bc10db40 con 0x7f23bc1067f0 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c0f02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c0f02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39438/0 (socket says 172.21.15.46:39438) 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c0f02640 1 -- 172.21.15.46:0/621942810 learned_addr learned my addr 172.21.15.46:0/621942810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23bbfff640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23bc1067f0 0x7f23bc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c0f02640 1 -- 172.21.15.46:0/621942810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23bc1067f0 msgr2=0x7f23bc19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c0f02640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23bc1067f0 0x7f23bc19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c0f02640 1 -- 172.21.15.46:0/621942810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23bc1999b0 con 0x7f23bc1007f0 2024-09-10T04:28:48.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c0f02640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc19f7f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00d6f0 tx=0x7f23ac00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23b9ffb640 1 -- 172.21.15.46:0/621942810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f23ac017070 con 0x7f23bc1007f0 2024-09-10T04:28:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23b9ffb640 1 -- 172.21.15.46:0/621942810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f23ac00de00 con 0x7f23bc1007f0 2024-09-10T04:28:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23bc199ca0 con 0x7f23bc1007f0 2024-09-10T04:28:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.998+0000 7f23b9ffb640 1 -- 172.21.15.46:0/621942810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f23ac004bb0 con 0x7f23bc1007f0 2024-09-10T04:28:48.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:47.999+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23bc19a200 con 0x7f23bc1007f0 2024-09-10T04:28:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.000+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23bc101f30 con 0x7f23bc1007f0 2024-09-10T04:28:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.000+0000 7f23b9ffb640 1 -- 172.21.15.46:0/621942810 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f23ac009a30 con 0x7f23bc1007f0 2024-09-10T04:28:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.001+0000 7f23b9ffb640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f23a407ac40 0x7f23a407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:48.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.001+0000 7f23b9ffb640 1 -- 172.21.15.46:0/621942810 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7f23ac09d740 con 0x7f23bc1007f0 2024-09-10T04:28:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.001+0000 7f23bbfff640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f23a407ac40 0x7f23a407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:48.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.001+0000 7f23bbfff640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f23a407ac40 0x7f23a407d100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23bc19b1a0 tx=0x7f23b0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:48.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.005+0000 7f23b9ffb640 1 -- 172.21.15.46:0/621942810 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f23ac068f00 con 0x7f23bc1007f0 2024-09-10T04:28:48.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.208+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f23bc19a4b0 con 0x7f23a407ac40 2024-09-10T04:28:48.211 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:47 smithi046 ceph-mon[109506]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:48.211 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:47 smithi046 ceph-mon[109506]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:48.211 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:47 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3369168898' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:28:48.211 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:47 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1524759574' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.210+0000 7f23b9ffb640 1 -- 172.21.15.46:0/621942810 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+416 (secure 0 0 0) 0x7f23bc19a4b0 con 0x7f23a407ac40 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr" 2024-09-10T04:28:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:28:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T04:28:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:28:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:28:48.213 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:28:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f23a407ac40 msgr2=0x7f23a407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f23a407ac40 0x7f23a407d100 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f23bc19b1a0 tx=0x7f23b0031400 comp rx=0 tx=0).stop 2024-09-10T04:28:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 msgr2=0x7f23bc19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc19f7f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00d6f0 tx=0x7f23ac00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 shutdown_connections 2024-09-10T04:28:48.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f23a407ac40 0x7f23a407d100 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23bc1067f0 0x7f23bc19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 --2- 172.21.15.46:0/621942810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23bc1007f0 0x7f23bc19f7f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 >> 172.21.15.46:0/621942810 conn(0x7f23bc0fc550 msgr2=0x7f23bc069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 shutdown_connections 2024-09-10T04:28:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.212+0000 7f23c3166640 1 -- 172.21.15.46:0/621942810 wait complete. 2024-09-10T04:28:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.296+0000 7fa9d0095640 1 -- 172.21.15.46:0/2129978570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8102b70 msgr2=0x7fa9c8102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.296+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2129978570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8102b70 0x7fa9c8102f50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009880 tx=0x7fa9c002a530 comp rx=0 tx=0).stop 2024-09-10T04:28:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 -- 172.21.15.46:0/2129978570 shutdown_connections 2024-09-10T04:28:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2129978570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c8103490 0x7fa9c810b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2129978570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8102b70 0x7fa9c8102f50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 -- 172.21.15.46:0/2129978570 >> 172.21.15.46:0/2129978570 conn(0x7fa9c80fc550 msgr2=0x7fa9c80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 -- 172.21.15.46:0/2129978570 shutdown_connections 2024-09-10T04:28:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 -- 172.21.15.46:0/2129978570 wait complete. 2024-09-10T04:28:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 Processor -- start 2024-09-10T04:28:48.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.297+0000 7fa9d0095640 1 -- start start 2024-09-10T04:28:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9d0095640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c8102b70 0x7fa9c819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9d0095640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8103490 0x7fa9c819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:48.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9d0095640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810ede0 con 0x7fa9c8103490 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9d0095640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c810ec60 con 0x7fa9c8102b70 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cd630640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8103490 0x7fa9c819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cd630640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8103490 0x7fa9c819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39446/0 (socket says 172.21.15.46:39446) 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cd630640 1 -- 172.21.15.46:0/2356461546 learned_addr learned my addr 172.21.15.46:0/2356461546 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cde31640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c8102b70 0x7fa9c819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cd630640 1 -- 172.21.15.46:0/2356461546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c8102b70 msgr2=0x7fa9c819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cd630640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c8102b70 0x7fa9c819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cd630640 1 -- 172.21.15.46:0/2356461546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c81977b0 con 0x7fa9c8103490 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9cd630640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8103490 0x7fa9c819db70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00d6f0 tx=0x7fa9bc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.298+0000 7fa9c6ffd640 1 -- 172.21.15.46:0/2356461546 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc017070 con 0x7fa9c8103490 2024-09-10T04:28:48.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.299+0000 7fa9c6ffd640 1 -- 172.21.15.46:0/2356461546 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9bc00de00 con 0x7fa9c8103490 2024-09-10T04:28:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.299+0000 7fa9c6ffd640 1 -- 172.21.15.46:0/2356461546 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc004bb0 con 0x7fa9c8103490 2024-09-10T04:28:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.299+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c8197aa0 con 0x7fa9c8103490 2024-09-10T04:28:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.299+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c8198030 con 0x7fa9c8103490 2024-09-10T04:28:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.300+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c806a4c0 con 0x7fa9c8103490 2024-09-10T04:28:48.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.300+0000 7fa9c6ffd640 1 -- 172.21.15.46:0/2356461546 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa9bc004e60 con 0x7fa9c8103490 2024-09-10T04:28:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.301+0000 7fa9c6ffd640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa9b007ac00 0x7fa9b007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:28:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.301+0000 7fa9c6ffd640 1 -- 172.21.15.46:0/2356461546 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6242+0+0 (secure 0 0 0) 0x7fa9bc09d820 con 0x7fa9c8103490 2024-09-10T04:28:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.301+0000 7fa9cde31640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa9b007ac00 0x7fa9b007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:28:48.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.301+0000 7fa9cde31640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa9b007ac00 0x7fa9b007d0c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00096c0 tx=0x7fa9c0004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:28:48.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.305+0000 7fa9c6ffd640 1 -- 172.21.15.46:0/2356461546 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa9bc068f60 con 0x7fa9c8103490 2024-09-10T04:28:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:47 smithi070 ceph-mon[79774]: from='client.35126 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:47 smithi070 ceph-mon[79774]: from='client.35130 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:47 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3369168898' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:28:48.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:47 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1524759574' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:28:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.563+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa9c8105fd0 con 0x7fa9c8103490 2024-09-10T04:28:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.563+0000 7fa9c6ffd640 1 -- 172.21.15.46:0/2356461546 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+423 (secure 0 0 0) 0x7fa9bc06de10 con 0x7fa9c8103490 2024-09-10T04:28:48.565 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:28:48.565 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:28:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:28:48.566 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:28:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:28:48.566 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:28:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa9b007ac00 msgr2=0x7fa9b007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa9b007ac00 0x7fa9b007d0c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00096c0 tx=0x7fa9c0004f30 comp rx=0 tx=0).stop 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8103490 msgr2=0x7fa9c819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8103490 0x7fa9c819db70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00d6f0 tx=0x7fa9bc00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 shutdown_connections 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa9b007ac00 0x7fa9b007d0c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c8103490 0x7fa9c819db70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 --2- 172.21.15.46:0/2356461546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9c8102b70 0x7fa9c819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 >> 172.21.15.46:0/2356461546 conn(0x7fa9c80fc550 msgr2=0x7fa9c8108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 shutdown_connections 2024-09-10T04:28:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:28:48.566+0000 7fa9d0095640 1 -- 172.21.15.46:0/2356461546 wait complete. 2024-09-10T04:28:49.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:48 smithi070 ceph-mon[79774]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:49.236 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:48 smithi070 ceph-mon[79774]: pgmap v694: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:49.237 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:48 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2356461546' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:28:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:48 smithi046 ceph-mon[109506]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:28:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:48 smithi046 ceph-mon[109506]: pgmap v694: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:49.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:48 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2356461546' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:28:51.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:51 smithi070 ceph-mon[79774]: pgmap v695: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:51.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:51 smithi046 ceph-mon[109506]: pgmap v695: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:53.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:53 smithi046 ceph-mon[109506]: pgmap v696: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:53.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:53 smithi070 ceph-mon[79774]: pgmap v696: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:28:55.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:55 smithi070 ceph-mon[79774]: pgmap v697: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:55.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:55 smithi046 ceph-mon[109506]: pgmap v697: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:57 smithi070 ceph-mon[79774]: pgmap v698: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:57.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:57 smithi046 ceph-mon[109506]: pgmap v698: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:59.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:28:59 smithi046 ceph-mon[109506]: pgmap v699: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:28:59.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:28:59 smithi070 ceph-mon[79774]: pgmap v699: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:01 smithi070 ceph-mon[79774]: pgmap v700: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:01 smithi046 ceph-mon[109506]: pgmap v700: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:03 smithi070 ceph-mon[79774]: pgmap v701: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:03.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:03 smithi046 ceph-mon[109506]: pgmap v701: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:05.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:05 smithi046 ceph-mon[109506]: pgmap v702: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:05.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:05 smithi070 ceph-mon[79774]: pgmap v702: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:07.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:07 smithi070 ceph-mon[79774]: pgmap v703: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:07.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:07 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:29:07.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:07 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:29:07.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:07 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:07.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:07 smithi046 ceph-mon[109506]: pgmap v703: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:07.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:07 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:29:07.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:07 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:29:07.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:07 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: pgmap v704: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:08.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: pgmap v704: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-10T04:29:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:29:08.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:08.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:08.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:10.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:10 smithi070 ceph-mon[79774]: Upgrade: Updating mds.cephfs.smithi070.myztiz 2024-09-10T04:29:10.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:10 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:10.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:10 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.myztiz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:29:10.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:10 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:29:10.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:10 smithi070 ceph-mon[79774]: Deploying daemon mds.cephfs.smithi070.myztiz on smithi070 2024-09-10T04:29:10.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:10 smithi070 ceph-mon[79774]: pgmap v705: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:10.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:10 smithi046 ceph-mon[109506]: Upgrade: Updating mds.cephfs.smithi070.myztiz 2024-09-10T04:29:10.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:10 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:10.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:10 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi070.myztiz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T04:29:10.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:10 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:29:10.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:10 smithi046 ceph-mon[109506]: Deploying daemon mds.cephfs.smithi070.myztiz on smithi070 2024-09-10T04:29:10.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:10 smithi046 ceph-mon[109506]: pgmap v705: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:12.381 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:12 smithi070 ceph-mon[79774]: osdmap e79: 6 total, 6 up, 6 in 2024-09-10T04:29:12.382 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:12 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:29:12.382 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:12 smithi070 ceph-mon[79774]: pgmap v707: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:12.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:12 smithi046 ceph-mon[109506]: osdmap e79: 6 total, 6 up, 6 in 2024-09-10T04:29:12.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:12 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T04:29:12.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:12 smithi046 ceph-mon[109506]: pgmap v707: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:14.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:14 smithi046 ceph-mon[109506]: pgmap v708: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:14.826 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:14 smithi070 ceph-mon[79774]: pgmap v708: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: pgmap v709: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:29:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.588 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: pgmap v709: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:16.741 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:17.585 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:17 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] up:boot 2024-09-10T04:29:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:17 smithi046 ceph-mon[109506]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:29:17.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"} : dispatch 2024-09-10T04:29:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:17 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] up:boot 2024-09-10T04:29:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:17 smithi070 ceph-mon[79774]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T04:29:17.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "mds metadata", "who": "cephfs.smithi070.myztiz"} : dispatch 2024-09-10T04:29:18.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:18 smithi070 ceph-mon[79774]: pgmap v710: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 -- 172.21.15.46:0/3816257114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 msgr2=0x7fac4c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 --2- 172.21.15.46:0/3816257114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c101d60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fac40009880 tx=0x7fac4002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 -- 172.21.15.46:0/3816257114 shutdown_connections 2024-09-10T04:29:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 --2- 172.21.15.46:0/3816257114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c101d60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 --2- 172.21.15.46:0/3816257114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c107900 0x7fac4c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:18.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 -- 172.21.15.46:0/3816257114 >> 172.21.15.46:0/3816257114 conn(0x7fac4c0fd600 msgr2=0x7fac4c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:18.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 -- 172.21.15.46:0/3816257114 shutdown_connections 2024-09-10T04:29:18.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.656+0000 7fac5101d640 1 -- 172.21.15.46:0/3816257114 wait complete. 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.657+0000 7fac5101d640 1 Processor -- start 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.657+0000 7fac5101d640 1 -- start start 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.657+0000 7fac5101d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.657+0000 7fac5101d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c107900 0x7fac4c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.657+0000 7fac5101d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c10edb0 con 0x7fac4c101900 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.657+0000 7fac5101d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c10ec30 con 0x7fac4c107900 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41502/0 (socket says 172.21.15.46:41502) 2024-09-10T04:29:18.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4ad9d640 1 -- 172.21.15.46:0/387913643 learned_addr learned my addr 172.21.15.46:0/387913643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c107900 0x7fac4c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4ad9d640 1 -- 172.21.15.46:0/387913643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c107900 msgr2=0x7fac4c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4ad9d640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c107900 0x7fac4c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4ad9d640 1 -- 172.21.15.46:0/387913643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c074f70 con 0x7fac4c101900 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac4ad9d640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c0744f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00e970 tx=0x7fac3c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac33fff640 1 -- 172.21.15.46:0/387913643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac3c00cac0 con 0x7fac4c101900 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c075260 con 0x7fac4c101900 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac33fff640 1 -- 172.21.15.46:0/387913643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fac3c00cc60 con 0x7fac4c101900 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac33fff640 1 -- 172.21.15.46:0/387913643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fac3c018720 con 0x7fac4c101900 2024-09-10T04:29:18.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.658+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c1a3e50 con 0x7fac4c101900 2024-09-10T04:29:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.659+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4c103040 con 0x7fac4c101900 2024-09-10T04:29:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.660+0000 7fac33fff640 1 -- 172.21.15.46:0/387913643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fac3c010430 con 0x7fac4c101900 2024-09-10T04:29:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.661+0000 7fac33fff640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fac3407abf0 0x7fac3407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.661+0000 7fac33fff640 1 -- 172.21.15.46:0/387913643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fac3c09e310 con 0x7fac4c101900 2024-09-10T04:29:18.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.661+0000 7fac4a59c640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fac3407abf0 0x7fac3407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:18.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.661+0000 7fac4a59c640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fac3407abf0 0x7fac3407d0b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fac4c075f80 tx=0x7fac40031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:18.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.665+0000 7fac33fff640 1 -- 172.21.15.46:0/387913643 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fac3c0699b0 con 0x7fac4c101900 2024-09-10T04:29:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:18 smithi046 ceph-mon[109506]: pgmap v710: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:18.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.907+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fac4c062630 con 0x7fac3407abf0 2024-09-10T04:29:18.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.909+0000 7fac33fff640 1 -- 172.21.15.46:0/387913643 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fac4c062630 con 0x7fac3407abf0 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fac3407abf0 msgr2=0x7fac3407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fac3407abf0 0x7fac3407d0b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fac4c075f80 tx=0x7fac40031400 comp rx=0 tx=0).stop 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 msgr2=0x7fac4c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c0744f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00e970 tx=0x7fac3c00ee30 comp rx=0 tx=0).stop 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 shutdown_connections 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fac3407abf0 0x7fac3407d0b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac4c107900 0x7fac4c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 --2- 172.21.15.46:0/387913643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac4c101900 0x7fac4c0744f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 >> 172.21.15.46:0/387913643 conn(0x7fac4c0fd600 msgr2=0x7fac4c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:18.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 shutdown_connections 2024-09-10T04:29:18.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.911+0000 7fac5101d640 1 -- 172.21.15.46:0/387913643 wait complete. 2024-09-10T04:29:18.922 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.997+0000 7fe78a162640 1 -- 172.21.15.46:0/251412220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7841033d0 msgr2=0x7fe78410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.997+0000 7fe78a162640 1 --2- 172.21.15.46:0/251412220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7841033d0 0x7fe78410b8c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe778009880 tx=0x7fe77802a530 comp rx=0 tx=0).stop 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.998+0000 7fe78a162640 1 -- 172.21.15.46:0/251412220 shutdown_connections 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.998+0000 7fe78a162640 1 --2- 172.21.15.46:0/251412220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7841033d0 0x7fe78410b8c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.998+0000 7fe78a162640 1 --2- 172.21.15.46:0/251412220 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe784102ab0 0x7fe784102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.998+0000 7fe78a162640 1 -- 172.21.15.46:0/251412220 >> 172.21.15.46:0/251412220 conn(0x7fe7840fc550 msgr2=0x7fe7840fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.998+0000 7fe78a162640 1 -- 172.21.15.46:0/251412220 shutdown_connections 2024-09-10T04:29:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.998+0000 7fe78a162640 1 -- 172.21.15.46:0/251412220 wait complete. 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.998+0000 7fe78a162640 1 Processor -- start 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe78a162640 1 -- start start 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe78a162640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe784102ab0 0x7fe78419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe78a162640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7841033d0 0x7fe78419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe78a162640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe78410ede0 con 0x7fe784102ab0 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe78a162640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe78410ec60 con 0x7fe7841033d0 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe7837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe784102ab0 0x7fe78419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe7837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe784102ab0 0x7fe78419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44248/0 (socket says 172.21.15.46:44248) 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe7837fe640 1 -- 172.21.15.46:0/2375450600 learned_addr learned my addr 172.21.15.46:0/2375450600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe782ffd640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7841033d0 0x7fe78419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe7837fe640 1 -- 172.21.15.46:0/2375450600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7841033d0 msgr2=0x7fe78419dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe7837fe640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7841033d0 0x7fe78419dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:18.999+0000 7fe7837fe640 1 -- 172.21.15.46:0/2375450600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784197810 con 0x7fe784102ab0 2024-09-10T04:29:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.000+0000 7fe7837fe640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe784102ab0 0x7fe78419d690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe77400ea00 tx=0x7fe77400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.000+0000 7fe780ff9640 1 -- 172.21.15.46:0/2375450600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe77400cb80 con 0x7fe784102ab0 2024-09-10T04:29:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.000+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe784197b00 con 0x7fe784102ab0 2024-09-10T04:29:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.000+0000 7fe780ff9640 1 -- 172.21.15.46:0/2375450600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe77400cd20 con 0x7fe784102ab0 2024-09-10T04:29:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.000+0000 7fe780ff9640 1 -- 172.21.15.46:0/2375450600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe774011470 con 0x7fe784102ab0 2024-09-10T04:29:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.000+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe784198060 con 0x7fe784102ab0 2024-09-10T04:29:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.002+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78406a580 con 0x7fe784102ab0 2024-09-10T04:29:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.002+0000 7fe780ff9640 1 -- 172.21.15.46:0/2375450600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe774005020 con 0x7fe784102ab0 2024-09-10T04:29:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.002+0000 7fe780ff9640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe76c07abf0 0x7fe76c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.003+0000 7fe780ff9640 1 -- 172.21.15.46:0/2375450600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe77409e070 con 0x7fe784102ab0 2024-09-10T04:29:19.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.003+0000 7fe782ffd640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe76c07abf0 0x7fe76c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.003+0000 7fe782ffd640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe76c07abf0 0x7fe76c07d0b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe784198b60 tx=0x7fe778031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.007+0000 7fe780ff9640 1 -- 172.21.15.46:0/2375450600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe774016030 con 0x7fe784102ab0 2024-09-10T04:29:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.219+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe784198370 con 0x7fe76c07abf0 2024-09-10T04:29:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.220+0000 7fe780ff9640 1 -- 172.21.15.46:0/2375450600 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fe784198370 con 0x7fe76c07abf0 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe76c07abf0 msgr2=0x7fe76c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe76c07abf0 0x7fe76c07d0b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe784198b60 tx=0x7fe778031400 comp rx=0 tx=0).stop 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe784102ab0 msgr2=0x7fe78419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe784102ab0 0x7fe78419d690 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe77400ea00 tx=0x7fe77400eec0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 shutdown_connections 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe76c07abf0 0x7fe76c07d0b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7841033d0 0x7fe78419dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 --2- 172.21.15.46:0/2375450600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe784102ab0 0x7fe78419d690 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 >> 172.21.15.46:0/2375450600 conn(0x7fe7840fc550 msgr2=0x7fe784108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 shutdown_connections 2024-09-10T04:29:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.223+0000 7fe78a162640 1 -- 172.21.15.46:0/2375450600 wait complete. 2024-09-10T04:29:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.308+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/2738107951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc101900 msgr2=0x7f2cdc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.308+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/2738107951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc101900 0x7f2cdc101d60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009880 tx=0x7f2cd002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/2738107951 shutdown_connections 2024-09-10T04:29:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/2738107951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc101900 0x7f2cdc101d60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/2738107951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cdc107900 0x7f2cdc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/2738107951 >> 172.21.15.46:0/2738107951 conn(0x7f2cdc0fd600 msgr2=0x7f2cdc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/2738107951 shutdown_connections 2024-09-10T04:29:19.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/2738107951 wait complete. 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 Processor -- start 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.309+0000 7f2ce0c8d640 1 -- start start 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2ce0c8d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cdc101900 0x7f2cdc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2ce0c8d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc107900 0x7f2cdc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2ce0c8d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc10edd0 con 0x7f2cdc107900 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2ce0c8d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2cdc10ec50 con 0x7f2cdc101900 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cd9d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc107900 0x7f2cdc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cd9d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc107900 0x7f2cdc19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44266/0 (socket says 172.21.15.46:44266) 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cd9d9b640 1 -- 172.21.15.46:0/3423006873 learned_addr learned my addr 172.21.15.46:0/3423006873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cda59c640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cdc101900 0x7f2cdc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cd9d9b640 1 -- 172.21.15.46:0/3423006873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cdc101900 msgr2=0x7f2cdc19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cd9d9b640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cdc101900 0x7f2cdc19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cd9d9b640 1 -- 172.21.15.46:0/3423006873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cdc197800 con 0x7f2cdc107900 2024-09-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2cd9d9b640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc107900 0x7f2cdc19dbc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0031460 tx=0x7f2cd00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2ccb7fe640 1 -- 172.21.15.46:0/3423006873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2cd003f070 con 0x7f2cdc107900 2024-09-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cdc197a90 con 0x7f2cdc107900 2024-09-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.310+0000 7f2ccb7fe640 1 -- 172.21.15.46:0/3423006873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2cd00035f0 con 0x7f2cdc107900 2024-09-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.311+0000 7f2ccb7fe640 1 -- 172.21.15.46:0/3423006873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2cd0035030 con 0x7f2cdc107900 2024-09-10T04:29:19.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.311+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cdc197fc0 con 0x7f2cdc107900 2024-09-10T04:29:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.312+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cdc103040 con 0x7f2cdc107900 2024-09-10T04:29:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.312+0000 7f2ccb7fe640 1 -- 172.21.15.46:0/3423006873 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2cd0042070 con 0x7f2cdc107900 2024-09-10T04:29:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.313+0000 7f2ccb7fe640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2cc407abf0 0x7f2cc407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.313+0000 7f2ccb7fe640 1 -- 172.21.15.46:0/3423006873 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2cd00be5d0 con 0x7f2cdc107900 2024-09-10T04:29:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.313+0000 7f2cda59c640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2cc407abf0 0x7f2cc407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.313+0000 7f2cda59c640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2cc407abf0 0x7f2cc407d0b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc005fd0 tx=0x7f2ccc00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.317+0000 7f2ccb7fe640 1 -- 172.21.15.46:0/3423006873 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2cd0089d10 con 0x7f2cdc107900 2024-09-10T04:29:19.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.516+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2cdc198270 con 0x7f2cc407abf0 2024-09-10T04:29:19.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.529+0000 7f2ccb7fe640 1 -- 172.21.15.46:0/3423006873 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f2cdc198270 con 0x7f2cc407abf0 2024-09-10T04:29:19.531 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:29:19.531 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (20m) 15m ago 32m 20.2M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:29:19.531 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (23m) 15m ago 32m 9198k - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (23m) 0s ago 31m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (19m) 15m ago 32m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (19m) 0s ago 31m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (20m) 15m ago 32m 77.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (15m) 15m ago 26m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (15m) 15m ago 26m 13.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (15m) 0s ago 26m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (4s) 0s ago 26m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (22m) 15m ago 33m 531M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (21m) 0s ago 31m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (19m) 15m ago 33m 50.4M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (19m) 0s ago 31m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (21m) 15m ago 32m 12.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:29:19.532 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (21m) 0s ago 31m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:29:19.533 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (18m) 15m ago 30m 86.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:29:19.533 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (18m) 15m ago 29m 90.2M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:29:19.533 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (17m) 15m ago 29m 83.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:29:19.533 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (17m) 0s ago 28m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:29:19.533 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (16m) 0s ago 28m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:29:19.533 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (16m) 0s ago 27m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:29:19.533 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (21m) 15m ago 31m 58.1M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2cc407abf0 msgr2=0x7f2cc407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2cc407abf0 0x7f2cc407d0b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc005fd0 tx=0x7f2ccc00b390 comp rx=0 tx=0).stop 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc107900 msgr2=0x7f2cdc19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc107900 0x7f2cdc19dbc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0031460 tx=0x7f2cd00045e0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 shutdown_connections 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2cc407abf0 0x7f2cc407d0b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cdc107900 0x7f2cdc19dbc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 --2- 172.21.15.46:0/3423006873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cdc101900 0x7f2cdc19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 >> 172.21.15.46:0/3423006873 conn(0x7f2cdc0fd600 msgr2=0x7f2cdc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.531+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 shutdown_connections 2024-09-10T04:29:19.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.532+0000 7f2ce0c8d640 1 -- 172.21.15.46:0/3423006873 wait complete. 2024-09-10T04:29:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 -- 172.21.15.46:0/2262099829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 msgr2=0x7ff7a010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 --2- 172.21.15.46:0/2262099829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a010b8d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009880 tx=0x7ff78c02a530 comp rx=0 tx=0).stop 2024-09-10T04:29:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 -- 172.21.15.46:0/2262099829 shutdown_connections 2024-09-10T04:29:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 --2- 172.21.15.46:0/2262099829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a010b8d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 --2- 172.21.15.46:0/2262099829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a0102ac0 0x7ff7a0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 -- 172.21.15.46:0/2262099829 >> 172.21.15.46:0/2262099829 conn(0x7ff7a00fc520 msgr2=0x7ff7a00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 -- 172.21.15.46:0/2262099829 shutdown_connections 2024-09-10T04:29:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.616+0000 7ff7a721d640 1 -- 172.21.15.46:0/2262099829 wait complete. 2024-09-10T04:29:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.617+0000 7ff7a721d640 1 Processor -- start 2024-09-10T04:29:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.617+0000 7ff7a721d640 1 -- start start 2024-09-10T04:29:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.617+0000 7ff7a721d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a0102ac0 0x7ff7a019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.617+0000 7ff7a721d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.617+0000 7ff7a721d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a010edf0 con 0x7ff7a01033e0 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.617+0000 7ff7a721d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a010ec70 con 0x7ff7a0102ac0 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.617+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44278/0 (socket says 172.21.15.46:44278) 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79ffff640 1 -- 172.21.15.46:0/3114424614 learned_addr learned my addr 172.21.15.46:0/3114424614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff7a4fb9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a0102ac0 0x7ff7a019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79ffff640 1 -- 172.21.15.46:0/3114424614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a0102ac0 msgr2=0x7ff7a019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79ffff640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a0102ac0 0x7ff7a019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79ffff640 1 -- 172.21.15.46:0/3114424614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a0197800 con 0x7ff7a01033e0 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79ffff640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a019dbc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff78c031460 tx=0x7ff78c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79dffb640 1 -- 172.21.15.46:0/3114424614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff78c03f070 con 0x7ff7a01033e0 2024-09-10T04:29:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79dffb640 1 -- 172.21.15.46:0/3114424614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff78c0035f0 con 0x7ff7a01033e0 2024-09-10T04:29:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a0197a90 con 0x7ff7a01033e0 2024-09-10T04:29:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff79dffb640 1 -- 172.21.15.46:0/3114424614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff78c035030 con 0x7ff7a01033e0 2024-09-10T04:29:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.618+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a0197fc0 con 0x7ff7a01033e0 2024-09-10T04:29:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.619+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a006aa90 con 0x7ff7a01033e0 2024-09-10T04:29:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.620+0000 7ff79dffb640 1 -- 172.21.15.46:0/3114424614 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff78c042070 con 0x7ff7a01033e0 2024-09-10T04:29:19.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.621+0000 7ff79dffb640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff79007ac00 0x7ff79007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.621+0000 7ff79dffb640 1 -- 172.21.15.46:0/3114424614 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff78c0be580 con 0x7ff7a01033e0 2024-09-10T04:29:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.621+0000 7ff7a4fb9640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff79007ac00 0x7ff79007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.621+0000 7ff7a4fb9640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff79007ac00 0x7ff79007d0c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff794009bd0 tx=0x7ff794005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.625+0000 7ff79dffb640 1 -- 172.21.15.46:0/3114424614 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff78c089cc0 con 0x7ff7a01033e0 2024-09-10T04:29:19.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.893+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7ff7a0105f60 con 0x7ff7a01033e0 2024-09-10T04:29:19.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.894+0000 7ff79dffb640 1 -- 172.21.15.46:0/3114424614 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7ff78c08eb70 con 0x7ff7a01033e0 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:19.896 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:29:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:29:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:29:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:29:19.897 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:29:19.897 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:29:19.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff79007ac00 msgr2=0x7ff79007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff79007ac00 0x7ff79007d0c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff794009bd0 tx=0x7ff794005f90 comp rx=0 tx=0).stop 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 msgr2=0x7ff7a019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a019dbc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff78c031460 tx=0x7ff78c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 shutdown_connections 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff79007ac00 0x7ff79007d0c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7a01033e0 0x7ff7a019dbc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 --2- 172.21.15.46:0/3114424614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7a0102ac0 0x7ff7a019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 >> 172.21.15.46:0/3114424614 conn(0x7ff7a00fc520 msgr2=0x7ff7a0108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 shutdown_connections 2024-09-10T04:29:19.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.896+0000 7ff7a721d640 1 -- 172.21.15.46:0/3114424614 wait complete. 2024-09-10T04:29:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/3379822094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 msgr2=0x7fb1e4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/3379822094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e4106bd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009c20 tx=0x7fb1e002a970 comp rx=0 tx=0).stop 2024-09-10T04:29:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/3379822094 shutdown_connections 2024-09-10T04:29:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/3379822094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1e41007f0 0x7fb1e4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/3379822094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e4106bd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/3379822094 >> 172.21.15.46:0/3379822094 conn(0x7fb1e40fc550 msgr2=0x7fb1e40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/3379822094 shutdown_connections 2024-09-10T04:29:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/3379822094 wait complete. 2024-09-10T04:29:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 Processor -- start 2024-09-10T04:29:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.983+0000 7fb1eb2a0640 1 -- start start 2024-09-10T04:29:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1eb2a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1e41007f0 0x7fb1e419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1eb2a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1eb2a0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410dca0 con 0x7fb1e41067f0 2024-09-10T04:29:19.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1eb2a0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e410db20 con 0x7fb1e41007f0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e883b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e883b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e419dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44292/0 (socket says 172.21.15.46:44292) 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e883b640 1 -- 172.21.15.46:0/593436903 learned_addr learned my addr 172.21.15.46:0/593436903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e903c640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1e41007f0 0x7fb1e419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e883b640 1 -- 172.21.15.46:0/593436903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1e41007f0 msgr2=0x7fb1e419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e883b640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1e41007f0 0x7fb1e419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e883b640 1 -- 172.21.15.46:0/593436903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e4197750 con 0x7fb1e41067f0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1e883b640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e419dbc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c810 tx=0x7fb1d400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.984+0000 7fb1da7fc640 1 -- 172.21.15.46:0/593436903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb1d4004020 con 0x7fb1e41067f0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.985+0000 7fb1da7fc640 1 -- 172.21.15.46:0/593436903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb1d40041c0 con 0x7fb1e41067f0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.985+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e4197a40 con 0x7fb1e41067f0 2024-09-10T04:29:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.985+0000 7fb1da7fc640 1 -- 172.21.15.46:0/593436903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb1d40047a0 con 0x7fb1e41067f0 2024-09-10T04:29:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.985+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e4197fa0 con 0x7fb1e41067f0 2024-09-10T04:29:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.986+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e4101f30 con 0x7fb1e41067f0 2024-09-10T04:29:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.986+0000 7fb1da7fc640 1 -- 172.21.15.46:0/593436903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb1d40079b0 con 0x7fb1e41067f0 2024-09-10T04:29:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.987+0000 7fb1da7fc640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1d007abf0 0x7fb1d007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.987+0000 7fb1e903c640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1d007abf0 0x7fb1d007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.987+0000 7fb1da7fc640 1 -- 172.21.15.46:0/593436903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb1d409dad0 con 0x7fb1e41067f0 2024-09-10T04:29:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.988+0000 7fb1e903c640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1d007abf0 0x7fb1d007d0b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009430 tx=0x7fb1e003c040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:19.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:19.992+0000 7fb1da7fc640 1 -- 172.21.15.46:0/593436903 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb1d4014030 con 0x7fb1e41067f0 2024-09-10T04:29:20.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:20.036 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:20.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: pgmap v711: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:20.037 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:19 smithi070 ceph-mon[79774]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: from='client.35154 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: pgmap v711: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:19 smithi046 ceph-mon[109506]: from='client.35158 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.221+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb1e4198250 con 0x7fb1e41067f0 2024-09-10T04:29:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.221+0000 7fb1da7fc640 1 -- 172.21.15.46:0/593436903 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fb1d4069210 con 0x7fb1e41067f0 2024-09-10T04:29:20.223 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:29:20.223 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:29:20.223 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:29:20.224 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:29:20.225 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:20.226 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:29:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.224+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1d007abf0 msgr2=0x7fb1d007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.224+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1d007abf0 0x7fb1d007d0b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009430 tx=0x7fb1e003c040 comp rx=0 tx=0).stop 2024-09-10T04:29:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.224+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 msgr2=0x7fb1e419dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.224+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e419dbc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c810 tx=0x7fb1d400ccd0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.225+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 shutdown_connections 2024-09-10T04:29:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.225+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1d007abf0 0x7fb1d007d0b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.225+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1e41067f0 0x7fb1e419dbc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.225+0000 7fb1eb2a0640 1 --2- 172.21.15.46:0/593436903 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1e41007f0 0x7fb1e419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.225+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 >> 172.21.15.46:0/593436903 conn(0x7fb1e40fc550 msgr2=0x7fb1e4069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.225+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 shutdown_connections 2024-09-10T04:29:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.225+0000 7fb1eb2a0640 1 -- 172.21.15.46:0/593436903 wait complete. 2024-09-10T04:29:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 -- 172.21.15.46:0/1359259755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81067f0 msgr2=0x7effe8106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 --2- 172.21.15.46:0/1359259755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81067f0 0x7effe8106bd0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7effd8009880 tx=0x7effd802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 -- 172.21.15.46:0/1359259755 shutdown_connections 2024-09-10T04:29:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 --2- 172.21.15.46:0/1359259755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe81007f0 0x7effe8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 --2- 172.21.15.46:0/1359259755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81067f0 0x7effe8106bd0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 -- 172.21.15.46:0/1359259755 >> 172.21.15.46:0/1359259755 conn(0x7effe80fc550 msgr2=0x7effe80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 -- 172.21.15.46:0/1359259755 shutdown_connections 2024-09-10T04:29:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.325+0000 7effeecac640 1 -- 172.21.15.46:0/1359259755 wait complete. 2024-09-10T04:29:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeecac640 1 Processor -- start 2024-09-10T04:29:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeecac640 1 -- start start 2024-09-10T04:29:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeecac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81007f0 0x7effe819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:20.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeecac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe81067f0 0x7effe819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeecac640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7effe810dcc0 con 0x7effe81007f0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeecac640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7effe810db40 con 0x7effe81067f0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeca48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81007f0 0x7effe819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeca48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81007f0 0x7effe819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44306/0 (socket says 172.21.15.46:44306) 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeca48640 1 -- 172.21.15.46:0/1266768383 learned_addr learned my addr 172.21.15.46:0/1266768383 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effe7fff640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe81067f0 0x7effe819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeca48640 1 -- 172.21.15.46:0/1266768383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe81067f0 msgr2=0x7effe819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeca48640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe81067f0 0x7effe819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeca48640 1 -- 172.21.15.46:0/1266768383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe81977b0 con 0x7effe81007f0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.326+0000 7effeca48640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81007f0 0x7effe819d630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7effd8031460 tx=0x7effd8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.327+0000 7effe5ffb640 1 -- 172.21.15.46:0/1266768383 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7effd803f070 con 0x7effe81007f0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.327+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe8197a40 con 0x7effe81007f0 2024-09-10T04:29:20.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.327+0000 7effe5ffb640 1 -- 172.21.15.46:0/1266768383 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7effd80035f0 con 0x7effe81007f0 2024-09-10T04:29:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.327+0000 7effe5ffb640 1 -- 172.21.15.46:0/1266768383 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7effd8035030 con 0x7effe81007f0 2024-09-10T04:29:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.327+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe8197f70 con 0x7effe81007f0 2024-09-10T04:29:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.328+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe8101f30 con 0x7effe81007f0 2024-09-10T04:29:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.329+0000 7effe5ffb640 1 -- 172.21.15.46:0/1266768383 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7effd8042070 con 0x7effe81007f0 2024-09-10T04:29:20.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.329+0000 7effe5ffb640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7effd007abf0 0x7effd007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.329+0000 7effe5ffb640 1 -- 172.21.15.46:0/1266768383 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7effd80be660 con 0x7effe81007f0 2024-09-10T04:29:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.329+0000 7effe7fff640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7effd007abf0 0x7effd007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:20.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.330+0000 7effe7fff640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7effd007abf0 0x7effd007d0b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7effe8104fd0 tx=0x7effdc005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:20.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.334+0000 7effe5ffb640 1 -- 172.21.15.46:0/1266768383 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7effd8089da0 con 0x7effe81007f0 2024-09-10T04:29:20.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.542+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7effe8198220 con 0x7effd007abf0 2024-09-10T04:29:20.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.544+0000 7effe5ffb640 1 -- 172.21.15.46:0/1266768383 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7effe8198220 con 0x7effd007abf0 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:29:20.546 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:29:20.547 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:29:20.547 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:29:20.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.546+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7effd007abf0 msgr2=0x7effd007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7effd007abf0 0x7effd007d0b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7effe8104fd0 tx=0x7effdc005f90 comp rx=0 tx=0).stop 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81007f0 msgr2=0x7effe819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81007f0 0x7effe819d630 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7effd8031460 tx=0x7effd8004530 comp rx=0 tx=0).stop 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 shutdown_connections 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7effd007abf0 0x7effd007d0b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effe81067f0 0x7effe819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 --2- 172.21.15.46:0/1266768383 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7effe81007f0 0x7effe819d630 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 >> 172.21.15.46:0/1266768383 conn(0x7effe80fc550 msgr2=0x7effe8069ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 shutdown_connections 2024-09-10T04:29:20.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.547+0000 7effeecac640 1 -- 172.21.15.46:0/1266768383 wait complete. 2024-09-10T04:29:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 -- 172.21.15.46:0/561552881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 msgr2=0x7f0200100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 --2- 172.21.15.46:0/561552881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f0200100c60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009880 tx=0x7f01f402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 -- 172.21.15.46:0/561552881 shutdown_connections 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 --2- 172.21.15.46:0/561552881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f0200100c60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 --2- 172.21.15.46:0/561552881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200106800 0x7f0200106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 -- 172.21.15.46:0/561552881 >> 172.21.15.46:0/561552881 conn(0x7f02000fc520 msgr2=0x7f02000fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 -- 172.21.15.46:0/561552881 shutdown_connections 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.641+0000 7f0204bd8640 1 -- 172.21.15.46:0/561552881 wait complete. 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f0204bd8640 1 Processor -- start 2024-09-10T04:29:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f0204bd8640 1 -- start start 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f0204bd8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f020010b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f0204bd8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200106800 0x7f020010bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f0204bd8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f020010edb0 con 0x7f0200100800 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f0204bd8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f020010ec30 con 0x7f0200106800 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f01fe59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f020010b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f01fe59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f020010b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44336/0 (socket says 172.21.15.46:44336) 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f01fe59c640 1 -- 172.21.15.46:0/2814788826 learned_addr learned my addr 172.21.15.46:0/2814788826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.642+0000 7f01fdd9b640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200106800 0x7f020010bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f01fe59c640 1 -- 172.21.15.46:0/2814788826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200106800 msgr2=0x7f020010bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f01fe59c640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200106800 0x7f020010bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f01fe59c640 1 -- 172.21.15.46:0/2814788826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020010c210 con 0x7f0200100800 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f01fe59c640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f020010b790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f01f000c8a0 tx=0x7f01f000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f01ef7fe640 1 -- 172.21.15.46:0/2814788826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01f00079b0 con 0x7f0200100800 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f01ef7fe640 1 -- 172.21.15.46:0/2814788826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01f0007b50 con 0x7f0200100800 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020010c500 con 0x7f0200100800 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f01ef7fe640 1 -- 172.21.15.46:0/2814788826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01f0018700 con 0x7f0200100800 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.643+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02001a3de0 con 0x7f0200100800 2024-09-10T04:29:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.644+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0200101f40 con 0x7f0200100800 2024-09-10T04:29:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.645+0000 7f01ef7fe640 1 -- 172.21.15.46:0/2814788826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f01f0007d00 con 0x7f0200100800 2024-09-10T04:29:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.645+0000 7f01ef7fe640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01e807ac40 0x7f01e807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.645+0000 7f01ef7fe640 1 -- 172.21.15.46:0/2814788826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f01f009de20 con 0x7f0200100800 2024-09-10T04:29:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.645+0000 7f01fdd9b640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01e807ac40 0x7f01e807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.646+0000 7f01fdd9b640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01e807ac40 0x7f01e807d100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f020010d220 tx=0x7f01f4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.650+0000 7f01ef7fe640 1 -- 172.21.15.46:0/2814788826 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f01f0014030 con 0x7f0200100800 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.914+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f0200100c60 con 0x7f0200100800 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.914+0000 7f01ef7fe640 1 -- 172.21.15.46:0/2814788826 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f01f00695e0 con 0x7f0200100800 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:29:20.916 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:29:20.917 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:29:20.918 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:29:20.918 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:29:20.918 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:29:20.918 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:29:20.918 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:29:20.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.916+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01e807ac40 msgr2=0x7f01e807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.916+0000 7f0204bd8640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01e807ac40 0x7f01e807d100 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f020010d220 tx=0x7f01f4031400 comp rx=0 tx=0).stop 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.916+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 msgr2=0x7f020010b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.916+0000 7f0204bd8640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f020010b790 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f01f000c8a0 tx=0x7f01f000cd60 comp rx=0 tx=0).stop 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.917+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 shutdown_connections 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.917+0000 7f0204bd8640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f01e807ac40 0x7f01e807d100 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.917+0000 7f0204bd8640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200106800 0x7f020010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.917+0000 7f0204bd8640 1 --2- 172.21.15.46:0/2814788826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0200100800 0x7f020010b790 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.917+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 >> 172.21.15.46:0/2814788826 conn(0x7f02000fc520 msgr2=0x7f02000fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.917+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 shutdown_connections 2024-09-10T04:29:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:20.917+0000 7f0204bd8640 1 -- 172.21.15.46:0/2814788826 wait complete. 2024-09-10T04:29:21.068 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:20 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3114424614' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:21.068 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:20 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/593436903' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:29:21.068 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:20 smithi070 ceph-mon[79774]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:20 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3114424614' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:20 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/593436903' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:29:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:20 smithi046 ceph-mon[109506]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:21 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2814788826' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:29:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:21 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:21 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:21 smithi070 ceph-mon[79774]: pgmap v712: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:21 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2814788826' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:29:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:21 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:21 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:29:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:21 smithi046 ceph-mon[109506]: pgmap v712: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:23.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:24.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:24 smithi070 ceph-mon[79774]: pgmap v713: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:24.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:24 smithi046 ceph-mon[109506]: pgmap v713: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:26.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:26 smithi046 ceph-mon[109506]: pgmap v714: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:26 smithi070 ceph-mon[79774]: pgmap v714: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:28 smithi070 ceph-mon[79774]: pgmap v715: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:28.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:28 smithi046 ceph-mon[109506]: pgmap v715: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:30 smithi070 ceph-mon[79774]: pgmap v716: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:30 smithi046 ceph-mon[109506]: pgmap v716: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:32.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:32 smithi046 ceph-mon[109506]: pgmap v717: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:32 smithi070 ceph-mon[79774]: pgmap v717: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:34 smithi070 ceph-mon[79774]: pgmap v718: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:34 smithi046 ceph-mon[109506]: pgmap v718: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:36 smithi070 ceph-mon[79774]: pgmap v719: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:36.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:36 smithi046 ceph-mon[109506]: pgmap v719: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:38 smithi046 ceph-mon[109506]: pgmap v720: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:38 smithi070 ceph-mon[79774]: pgmap v720: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:40 smithi070 ceph-mon[79774]: pgmap v721: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:40 smithi046 ceph-mon[109506]: pgmap v721: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:42.831 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:42 smithi070 ceph-mon[79774]: pgmap v722: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:42.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:42 smithi046 ceph-mon[109506]: pgmap v722: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:44 smithi046 ceph-mon[109506]: pgmap v723: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:44 smithi070 ceph-mon[79774]: pgmap v723: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:46 smithi070 ceph-mon[79774]: pgmap v724: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:46 smithi046 ceph-mon[109506]: pgmap v724: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:48 smithi070 ceph-mon[79774]: pgmap v725: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:48.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:48 smithi046 ceph-mon[109506]: pgmap v725: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:50 smithi046 ceph-mon[109506]: pgmap v726: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:50 smithi070 ceph-mon[79774]: pgmap v726: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:51.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 -- 172.21.15.46:0/413571577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924107900 msgr2=0x7fc924107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 --2- 172.21.15.46:0/413571577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924107900 0x7fc924107ce0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc914009880 tx=0x7fc91402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 -- 172.21.15.46:0/413571577 shutdown_connections 2024-09-10T04:29:51.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 --2- 172.21.15.46:0/413571577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924101900 0x7fc924101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 --2- 172.21.15.46:0/413571577 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924107900 0x7fc924107ce0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 -- 172.21.15.46:0/413571577 >> 172.21.15.46:0/413571577 conn(0x7fc9240fd600 msgr2=0x7fc9240ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:51.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 -- 172.21.15.46:0/413571577 shutdown_connections 2024-09-10T04:29:51.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.017+0000 7fc928aba640 1 -- 172.21.15.46:0/413571577 wait complete. 2024-09-10T04:29:51.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc928aba640 1 Processor -- start 2024-09-10T04:29:51.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc928aba640 1 -- start start 2024-09-10T04:29:51.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc928aba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924101900 0x7fc92419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc928aba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924107900 0x7fc92419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc928aba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410edd0 con 0x7fc924107900 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc928aba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc92410ec50 con 0x7fc924101900 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc91bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924107900 0x7fc92419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc91bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924107900 0x7fc92419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44948/0 (socket says 172.21.15.46:44948) 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc92259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924101900 0x7fc92419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc91bfff640 1 -- 172.21.15.46:0/4153636405 learned_addr learned my addr 172.21.15.46:0/4153636405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc92259c640 1 -- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924107900 msgr2=0x7fc92419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.018+0000 7fc92259c640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924107900 0x7fc92419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc92259c640 1 -- 172.21.15.46:0/4153636405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc924197820 con 0x7fc924101900 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc91bfff640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924107900 0x7fc92419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:29:51.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc92259c640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924101900 0x7fc92419d6a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc91402a9c0 tx=0x7fc9140045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc91b7fe640 1 -- 172.21.15.46:0/4153636405 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc91403f070 con 0x7fc924101900 2024-09-10T04:29:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc91b7fe640 1 -- 172.21.15.46:0/4153636405 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9140035f0 con 0x7fc924101900 2024-09-10T04:29:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc924197ab0 con 0x7fc924101900 2024-09-10T04:29:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc91b7fe640 1 -- 172.21.15.46:0/4153636405 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc914035030 con 0x7fc924101900 2024-09-10T04:29:51.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.019+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc924197fe0 con 0x7fc924101900 2024-09-10T04:29:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.020+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc924103040 con 0x7fc924101900 2024-09-10T04:29:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.021+0000 7fc91b7fe640 1 -- 172.21.15.46:0/4153636405 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc914042070 con 0x7fc924101900 2024-09-10T04:29:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.021+0000 7fc91b7fe640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc91007ac50 0x7fc91007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.021+0000 7fc91bfff640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc91007ac50 0x7fc91007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.021+0000 7fc91b7fe640 1 -- 172.21.15.46:0/4153636405 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc9140be410 con 0x7fc924101900 2024-09-10T04:29:51.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.022+0000 7fc91bfff640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc91007ac50 0x7fc91007d110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9240ffc40 tx=0x7fc90c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.026+0000 7fc91b7fe640 1 -- 172.21.15.46:0/4153636405 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc914089bd0 con 0x7fc924101900 2024-09-10T04:29:51.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.232+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc924198290 con 0x7fc91007ac50 2024-09-10T04:29:51.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.233+0000 7fc91b7fe640 1 -- 172.21.15.46:0/4153636405 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc924198290 con 0x7fc91007ac50 2024-09-10T04:29:51.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc91007ac50 msgr2=0x7fc91007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc91007ac50 0x7fc91007d110 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9240ffc40 tx=0x7fc90c005f90 comp rx=0 tx=0).stop 2024-09-10T04:29:51.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924101900 msgr2=0x7fc92419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924101900 0x7fc92419d6a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc91402a9c0 tx=0x7fc9140045e0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 shutdown_connections 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc91007ac50 0x7fc91007d110 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc924107900 0x7fc92419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 --2- 172.21.15.46:0/4153636405 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc924101900 0x7fc92419d6a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.235+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 >> 172.21.15.46:0/4153636405 conn(0x7fc9240fd600 msgr2=0x7fc9240fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.236+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 shutdown_connections 2024-09-10T04:29:51.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.236+0000 7fc928aba640 1 -- 172.21.15.46:0/4153636405 wait complete. 2024-09-10T04:29:51.245 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:29:51.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 -- 172.21.15.46:0/1000078092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10100800 msgr2=0x7fde10100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 --2- 172.21.15.46:0/1000078092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10100800 0x7fde10100c60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fde00009880 tx=0x7fde0002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 -- 172.21.15.46:0/1000078092 shutdown_connections 2024-09-10T04:29:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 --2- 172.21.15.46:0/1000078092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10100800 0x7fde10100c60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 --2- 172.21.15.46:0/1000078092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde10106800 0x7fde10106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 -- 172.21.15.46:0/1000078092 >> 172.21.15.46:0/1000078092 conn(0x7fde100fc520 msgr2=0x7fde100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 -- 172.21.15.46:0/1000078092 shutdown_connections 2024-09-10T04:29:51.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.321+0000 7fde17911640 1 -- 172.21.15.46:0/1000078092 wait complete. 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.322+0000 7fde17911640 1 Processor -- start 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.322+0000 7fde17911640 1 -- start start 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde17911640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde10100800 0x7fde101965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde17911640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10106800 0x7fde10196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde17911640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde1010dcd0 con 0x7fde10106800 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde17911640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fde1010db50 con 0x7fde10100800 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde14eac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10106800 0x7fde10196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde14eac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10106800 0x7fde10196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44962/0 (socket says 172.21.15.46:44962) 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde14eac640 1 -- 172.21.15.46:0/441820586 learned_addr learned my addr 172.21.15.46:0/441820586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde156ad640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde10100800 0x7fde101965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde156ad640 1 -- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10106800 msgr2=0x7fde10196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde156ad640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10106800 0x7fde10196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde156ad640 1 -- 172.21.15.46:0/441820586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde10197050 con 0x7fde10100800 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.323+0000 7fde156ad640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde10100800 0x7fde101965d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00e970 tx=0x7fde0c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.324+0000 7fde067fc640 1 -- 172.21.15.46:0/441820586 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fde0c00cac0 con 0x7fde10100800 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.324+0000 7fde067fc640 1 -- 172.21.15.46:0/441820586 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde0c00cc60 con 0x7fde10100800 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.324+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde10197340 con 0x7fde10100800 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.324+0000 7fde067fc640 1 -- 172.21.15.46:0/441820586 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fde0c0047a0 con 0x7fde10100800 2024-09-10T04:29:51.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.324+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101a3e10 con 0x7fde10100800 2024-09-10T04:29:51.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.325+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde10101f40 con 0x7fde10100800 2024-09-10T04:29:51.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.325+0000 7fde067fc640 1 -- 172.21.15.46:0/441820586 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fde0c004a60 con 0x7fde10100800 2024-09-10T04:29:51.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.326+0000 7fde067fc640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddfc07ac40 0x7fddfc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.326+0000 7fde067fc640 1 -- 172.21.15.46:0/441820586 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fde0c09e060 con 0x7fde10100800 2024-09-10T04:29:51.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.326+0000 7fde14eac640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddfc07ac40 0x7fddfc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.326+0000 7fde14eac640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddfc07ac40 0x7fddfc07d100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde10198730 tx=0x7fde000058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.330+0000 7fde067fc640 1 -- 172.21.15.46:0/441820586 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fde0c0697c0 con 0x7fde10100800 2024-09-10T04:29:51.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.537+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fde10062630 con 0x7fddfc07ac40 2024-09-10T04:29:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.538+0000 7fde067fc640 1 -- 172.21.15.46:0/441820586 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fde10062630 con 0x7fddfc07ac40 2024-09-10T04:29:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddfc07ac40 msgr2=0x7fddfc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddfc07ac40 0x7fddfc07d100 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fde10198730 tx=0x7fde000058d0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde10100800 msgr2=0x7fde101965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde10100800 0x7fde101965d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00e970 tx=0x7fde0c00ee30 comp rx=0 tx=0).stop 2024-09-10T04:29:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 shutdown_connections 2024-09-10T04:29:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fddfc07ac40 0x7fddfc07d100 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde10106800 0x7fde10196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 --2- 172.21.15.46:0/441820586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde10100800 0x7fde101965d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 >> 172.21.15.46:0/441820586 conn(0x7fde100fc520 msgr2=0x7fde100fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 shutdown_connections 2024-09-10T04:29:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.540+0000 7fde17911640 1 -- 172.21.15.46:0/441820586 wait complete. 2024-09-10T04:29:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 -- 172.21.15.46:0/1533271907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901007f0 msgr2=0x7f4390100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 --2- 172.21.15.46:0/1533271907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901007f0 0x7f4390100c50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4384009880 tx=0x7f438402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 -- 172.21.15.46:0/1533271907 shutdown_connections 2024-09-10T04:29:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 --2- 172.21.15.46:0/1533271907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901007f0 0x7f4390100c50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 --2- 172.21.15.46:0/1533271907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43901067f0 0x7f4390106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 -- 172.21.15.46:0/1533271907 >> 172.21.15.46:0/1533271907 conn(0x7f43900fc550 msgr2=0x7f43900fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:51.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 -- 172.21.15.46:0/1533271907 shutdown_connections 2024-09-10T04:29:51.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 -- 172.21.15.46:0/1533271907 wait complete. 2024-09-10T04:29:51.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 Processor -- start 2024-09-10T04:29:51.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.625+0000 7f43969bb640 1 -- start start 2024-09-10T04:29:51.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f43969bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43901007f0 0x7f439019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f43969bb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901067f0 0x7f439019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f43969bb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f439010dca0 con 0x7f43901067f0 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f43969bb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f439010db20 con 0x7f43901007f0 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901067f0 0x7f439019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901067f0 0x7f439019db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44972/0 (socket says 172.21.15.46:44972) 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438f7fe640 1 -- 172.21.15.46:0/2077149679 learned_addr learned my addr 172.21.15.46:0/2077149679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438ffff640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43901007f0 0x7f439019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438f7fe640 1 -- 172.21.15.46:0/2077149679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43901007f0 msgr2=0x7f439019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438f7fe640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43901007f0 0x7f439019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438f7fe640 1 -- 172.21.15.46:0/2077149679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4390197780 con 0x7f43901067f0 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438ffff640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43901007f0 0x7f439019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.626+0000 7f438f7fe640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901067f0 0x7f439019db40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4384031460 tx=0x7f4384004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.627+0000 7f438d7fa640 1 -- 172.21.15.46:0/2077149679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f438403f070 con 0x7f43901067f0 2024-09-10T04:29:51.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.627+0000 7f438d7fa640 1 -- 172.21.15.46:0/2077149679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f43840035f0 con 0x7f43901067f0 2024-09-10T04:29:51.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.627+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4390197a10 con 0x7f43901067f0 2024-09-10T04:29:51.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.627+0000 7f438d7fa640 1 -- 172.21.15.46:0/2077149679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4384035030 con 0x7f43901067f0 2024-09-10T04:29:51.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.627+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4390197f40 con 0x7f43901067f0 2024-09-10T04:29:51.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.628+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4390101f30 con 0x7f43901067f0 2024-09-10T04:29:51.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.629+0000 7f438d7fa640 1 -- 172.21.15.46:0/2077149679 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4384042070 con 0x7f43901067f0 2024-09-10T04:29:51.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.629+0000 7f438d7fa640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f437807abf0 0x7f437807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.629+0000 7f438ffff640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f437807abf0 0x7f437807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.629+0000 7f438d7fa640 1 -- 172.21.15.46:0/2077149679 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f43840be5b0 con 0x7f43901067f0 2024-09-10T04:29:51.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.630+0000 7f438ffff640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f437807abf0 0x7f437807d0b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4380009fc0 tx=0x7f4380005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.634+0000 7f438d7fa640 1 -- 172.21.15.46:0/2077149679 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4384089cf0 con 0x7f43901067f0 2024-09-10T04:29:51.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.849+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f43901981f0 con 0x7f437807abf0 2024-09-10T04:29:51.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.859+0000 7f438d7fa640 1 -- 172.21.15.46:0/2077149679 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f43901981f0 con 0x7f437807abf0 2024-09-10T04:29:51.860 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:29:51.860 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (21m) 30s ago 32m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:29:51.860 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (24m) 30s ago 33m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (23m) 33s ago 32m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (19m) 30s ago 32m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (19m) 33s ago 32m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (21m) 30s ago 32m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (16m) 30s ago 27m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (16m) 30s ago 27m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (15m) 33s ago 27m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (36s) 33s ago 27m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (23m) 30s ago 34m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (22m) 33s ago 31m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (20m) 30s ago 34m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (20m) 33s ago 31m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:29:51.861 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (22m) 30s ago 32m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (22m) 33s ago 32m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (19m) 30s ago 30m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (18m) 30s ago 30m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (18m) 30s ago 29m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (17m) 33s ago 29m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (17m) 33s ago 28m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (16m) 33s ago 28m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:29:51.862 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (21m) 30s ago 32m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f437807abf0 msgr2=0x7f437807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f437807abf0 0x7f437807d0b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4380009fc0 tx=0x7f4380005f90 comp rx=0 tx=0).stop 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901067f0 msgr2=0x7f439019db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901067f0 0x7f439019db40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f4384031460 tx=0x7f4384004530 comp rx=0 tx=0).stop 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 shutdown_connections 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f437807abf0 0x7f437807d0b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43901067f0 0x7f439019db40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.861+0000 7f43969bb640 1 --2- 172.21.15.46:0/2077149679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43901007f0 0x7f439019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.862+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 >> 172.21.15.46:0/2077149679 conn(0x7f43900fc550 msgr2=0x7f4390069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:51.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.862+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 shutdown_connections 2024-09-10T04:29:51.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.862+0000 7f43969bb640 1 -- 172.21.15.46:0/2077149679 wait complete. 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.946+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/1631257171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a881033e0 msgr2=0x7f7a8810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.946+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/1631257171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a881033e0 0x7f7a8810b8d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7a70009880 tx=0x7f7a7002a530 comp rx=0 tx=0).stop 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.946+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/1631257171 shutdown_connections 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.946+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/1631257171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a881033e0 0x7f7a8810b8d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.946+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/1631257171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a88102ac0 0x7f7a88102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.946+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/1631257171 >> 172.21.15.46:0/1631257171 conn(0x7f7a880fc520 msgr2=0x7f7a880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.947+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/1631257171 shutdown_connections 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.947+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/1631257171 wait complete. 2024-09-10T04:29:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.947+0000 7f7a8e1a0640 1 Processor -- start 2024-09-10T04:29:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a8e1a0640 1 -- start start 2024-09-10T04:29:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a8e1a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a88102ac0 0x7f7a8819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a8e1a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a881033e0 0x7f7a8819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a8e1a0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810edd0 con 0x7f7a88102ac0 2024-09-10T04:29:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a8e1a0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8810ec50 con 0x7f7a881033e0 2024-09-10T04:29:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a88102ac0 0x7f7a8819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a88102ac0 0x7f7a8819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44998/0 (socket says 172.21.15.46:44998) 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a877fe640 1 -- 172.21.15.46:0/656823132 learned_addr learned my addr 172.21.15.46:0/656823132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a86ffd640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a881033e0 0x7f7a8819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a877fe640 1 -- 172.21.15.46:0/656823132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a881033e0 msgr2=0x7f7a8819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a877fe640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a881033e0 0x7f7a8819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a877fe640 1 -- 172.21.15.46:0/656823132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a881977e0 con 0x7f7a88102ac0 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.948+0000 7f7a877fe640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a88102ac0 0x7f7a8819d660 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d8d0 tx=0x7f7a7800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.949+0000 7f7a84ff9640 1 -- 172.21.15.46:0/656823132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a78017030 con 0x7f7a88102ac0 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.949+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a88197ad0 con 0x7f7a88102ac0 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.949+0000 7f7a84ff9640 1 -- 172.21.15.46:0/656823132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a78004020 con 0x7f7a88102ac0 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.949+0000 7f7a84ff9640 1 -- 172.21.15.46:0/656823132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a780047a0 con 0x7f7a88102ac0 2024-09-10T04:29:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.949+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a88198030 con 0x7f7a88102ac0 2024-09-10T04:29:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.950+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8806aa90 con 0x7f7a88102ac0 2024-09-10T04:29:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.950+0000 7f7a84ff9640 1 -- 172.21.15.46:0/656823132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7a78009a30 con 0x7f7a88102ac0 2024-09-10T04:29:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.951+0000 7f7a84ff9640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7407ac40 0x7f7a7407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.951+0000 7f7a84ff9640 1 -- 172.21.15.46:0/656823132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7a78018070 con 0x7f7a88102ac0 2024-09-10T04:29:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.951+0000 7f7a86ffd640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7407ac40 0x7f7a7407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:51.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.952+0000 7f7a86ffd640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7407ac40 0x7f7a7407d100 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7a88198b30 tx=0x7f7a70031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:51.956+0000 7f7a84ff9640 1 -- 172.21.15.46:0/656823132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7a78069190 con 0x7f7a88102ac0 2024-09-10T04:29:52.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.216+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7a88105f60 con 0x7f7a88102ac0 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.217+0000 7f7a84ff9640 1 -- 172.21.15.46:0/656823132 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f7a7806e040 con 0x7f7a88102ac0 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:29:52.218 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:29:52.219 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:29:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7407ac40 msgr2=0x7f7a7407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7407ac40 0x7f7a7407d100 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7a88198b30 tx=0x7f7a70031400 comp rx=0 tx=0).stop 2024-09-10T04:29:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a88102ac0 msgr2=0x7f7a8819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a88102ac0 0x7f7a8819d660 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800d8d0 tx=0x7f7a7800dd90 comp rx=0 tx=0).stop 2024-09-10T04:29:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 shutdown_connections 2024-09-10T04:29:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a7407ac40 0x7f7a7407d100 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a881033e0 0x7f7a8819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 --2- 172.21.15.46:0/656823132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a88102ac0 0x7f7a8819d660 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 >> 172.21.15.46:0/656823132 conn(0x7f7a880fc520 msgr2=0x7f7a88108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 shutdown_connections 2024-09-10T04:29:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.219+0000 7f7a8e1a0640 1 -- 172.21.15.46:0/656823132 wait complete. 2024-09-10T04:29:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 -- 172.21.15.46:0/1625156912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50101900 msgr2=0x7f1b50101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 --2- 172.21.15.46:0/1625156912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50101900 0x7f1b50101d60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009880 tx=0x7f1b4402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 -- 172.21.15.46:0/1625156912 shutdown_connections 2024-09-10T04:29:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 --2- 172.21.15.46:0/1625156912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50101900 0x7f1b50101d60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 --2- 172.21.15.46:0/1625156912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b50107900 0x7f1b50107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 -- 172.21.15.46:0/1625156912 >> 172.21.15.46:0/1625156912 conn(0x7f1b500fd600 msgr2=0x7f1b500ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 -- 172.21.15.46:0/1625156912 shutdown_connections 2024-09-10T04:29:52.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.304+0000 7f1b54b71640 1 -- 172.21.15.46:0/1625156912 wait complete. 2024-09-10T04:29:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b54b71640 1 Processor -- start 2024-09-10T04:29:52.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b54b71640 1 -- start start 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b54b71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b50101900 0x7f1b5019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b54b71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50107900 0x7f1b5019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b54b71640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5010edd0 con 0x7f1b50107900 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b54b71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5010ec50 con 0x7f1b50101900 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b4dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50107900 0x7f1b5019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b4dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50107900 0x7f1b5019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45012/0 (socket says 172.21.15.46:45012) 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b4dd9b640 1 -- 172.21.15.46:0/3335454513 learned_addr learned my addr 172.21.15.46:0/3335454513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.305+0000 7f1b4e59c640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b50101900 0x7f1b5019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b4dd9b640 1 -- 172.21.15.46:0/3335454513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b50101900 msgr2=0x7f1b5019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b4dd9b640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b50101900 0x7f1b5019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b4dd9b640 1 -- 172.21.15.46:0/3335454513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b50197800 con 0x7f1b50107900 2024-09-10T04:29:52.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b4e59c640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b50101900 0x7f1b5019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:29:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b4dd9b640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50107900 0x7f1b5019dbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1b44031460 tx=0x7f1b44004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b3f7fe640 1 -- 172.21.15.46:0/3335454513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b4403f070 con 0x7f1b50107900 2024-09-10T04:29:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b50197a90 con 0x7f1b50107900 2024-09-10T04:29:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b3f7fe640 1 -- 172.21.15.46:0/3335454513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b440035f0 con 0x7f1b50107900 2024-09-10T04:29:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b3f7fe640 1 -- 172.21.15.46:0/3335454513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b44035030 con 0x7f1b50107900 2024-09-10T04:29:52.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.306+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b50197fc0 con 0x7f1b50107900 2024-09-10T04:29:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.307+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b50103040 con 0x7f1b50107900 2024-09-10T04:29:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.308+0000 7f1b3f7fe640 1 -- 172.21.15.46:0/3335454513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1b44042070 con 0x7f1b50107900 2024-09-10T04:29:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.308+0000 7f1b3f7fe640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b3807abf0 0x7f1b3807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.309+0000 7f1b3f7fe640 1 -- 172.21.15.46:0/3335454513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1b440be5b0 con 0x7f1b50107900 2024-09-10T04:29:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.309+0000 7f1b4e59c640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b3807abf0 0x7f1b3807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.309+0000 7f1b4e59c640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b3807abf0 0x7f1b3807d0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009fc0 tx=0x7f1b40005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.313+0000 7f1b3f7fe640 1 -- 172.21.15.46:0/3335454513 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1b44089cf0 con 0x7f1b50107900 2024-09-10T04:29:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.536+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1b50109e70 con 0x7f1b50107900 2024-09-10T04:29:52.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.536+0000 7f1b3f7fe640 1 -- 172.21.15.46:0/3335454513 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f1b4408eba0 con 0x7f1b50107900 2024-09-10T04:29:52.537 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:29:52.537 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:29:52.537 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:29:52.538 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:29:52.539 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:52.540 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:52.541 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:29:52.541 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:29:52.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.538+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b3807abf0 msgr2=0x7f1b3807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.538+0000 7f1b54b71640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b3807abf0 0x7f1b3807d0b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009fc0 tx=0x7f1b40005f90 comp rx=0 tx=0).stop 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.538+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50107900 msgr2=0x7f1b5019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.538+0000 7f1b54b71640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50107900 0x7f1b5019dbc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1b44031460 tx=0x7f1b44004530 comp rx=0 tx=0).stop 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.539+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 shutdown_connections 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.539+0000 7f1b54b71640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b3807abf0 0x7f1b3807d0b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.539+0000 7f1b54b71640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b50107900 0x7f1b5019dbc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.539+0000 7f1b54b71640 1 --2- 172.21.15.46:0/3335454513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b50101900 0x7f1b5019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.539+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 >> 172.21.15.46:0/3335454513 conn(0x7f1b500fd600 msgr2=0x7f1b500fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.539+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 shutdown_connections 2024-09-10T04:29:52.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.539+0000 7f1b54b71640 1 -- 172.21.15.46:0/3335454513 wait complete. 2024-09-10T04:29:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3044531873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8101900 msgr2=0x7f91e8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3044531873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8101900 0x7f91e8101d60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f91dc009880 tx=0x7f91dc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3044531873 shutdown_connections 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3044531873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8101900 0x7f91e8101d60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3044531873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e8107900 0x7f91e8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3044531873 >> 172.21.15.46:0/3044531873 conn(0x7f91e80fd600 msgr2=0x7f91e80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3044531873 shutdown_connections 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.624+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3044531873 wait complete. 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91ef0ca640 1 Processor -- start 2024-09-10T04:29:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91ef0ca640 1 -- start start 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91ef0ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e8101900 0x7f91e819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91ef0ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8107900 0x7f91e819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91ef0ca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810edd0 con 0x7f91e8107900 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91ef0ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810ec50 con 0x7f91e8101900 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8107900 0x7f91e819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8107900 0x7f91e819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45042/0 (socket says 172.21.15.46:45042) 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91e7fff640 1 -- 172.21.15.46:0/3215068970 learned_addr learned my addr 172.21.15.46:0/3215068970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.625+0000 7f91ece66640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e8101900 0x7f91e819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91e7fff640 1 -- 172.21.15.46:0/3215068970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e8101900 msgr2=0x7f91e819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91e7fff640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e8101900 0x7f91e819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91e7fff640 1 -- 172.21.15.46:0/3215068970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e81977a0 con 0x7f91e8107900 2024-09-10T04:29:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91e7fff640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8107900 0x7f91e819db60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f91dc031460 tx=0x7f91dc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91e5ffb640 1 -- 172.21.15.46:0/3215068970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f91dc03f070 con 0x7f91e8107900 2024-09-10T04:29:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91e5ffb640 1 -- 172.21.15.46:0/3215068970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f91dc0035f0 con 0x7f91e8107900 2024-09-10T04:29:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e8197a30 con 0x7f91e8107900 2024-09-10T04:29:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91e5ffb640 1 -- 172.21.15.46:0/3215068970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f91dc035030 con 0x7f91e8107900 2024-09-10T04:29:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.626+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e8197f60 con 0x7f91e8107900 2024-09-10T04:29:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.627+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e8103040 con 0x7f91e8107900 2024-09-10T04:29:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.628+0000 7f91e5ffb640 1 -- 172.21.15.46:0/3215068970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f91dc042070 con 0x7f91e8107900 2024-09-10T04:29:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.629+0000 7f91e5ffb640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f91d007ac40 0x7f91d007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.629+0000 7f91e5ffb640 1 -- 172.21.15.46:0/3215068970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f91dc0be4a0 con 0x7f91e8107900 2024-09-10T04:29:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.629+0000 7f91ece66640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f91d007ac40 0x7f91d007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.629+0000 7f91ece66640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f91d007ac40 0x7f91d007d100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009bd0 tx=0x7f91d8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.633+0000 7f91e5ffb640 1 -- 172.21.15.46:0/3215068970 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f91dc089c60 con 0x7f91e8107900 2024-09-10T04:29:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:52 smithi070 ceph-mon[79774]: from='client.44995 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:52 smithi070 ceph-mon[79774]: pgmap v727: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:52 smithi070 ceph-mon[79774]: from='client.44999 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:52 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/656823132' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:52.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.835+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f91e81982d0 con 0x7f91d007ac40 2024-09-10T04:29:52.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:52 smithi046 ceph-mon[109506]: from='client.44995 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:52 smithi046 ceph-mon[109506]: pgmap v727: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:52 smithi046 ceph-mon[109506]: from='client.44999 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:52 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/656823132' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:29:52.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.837+0000 7f91e5ffb640 1 -- 172.21.15.46:0/3215068970 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f91e81982d0 con 0x7f91d007ac40 2024-09-10T04:29:52.837 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:29:52.838 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:29:52.839 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:29:52.839 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:29:52.839 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f91d007ac40 msgr2=0x7f91d007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f91d007ac40 0x7f91d007d100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009bd0 tx=0x7f91d8005f90 comp rx=0 tx=0).stop 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8107900 msgr2=0x7f91e819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8107900 0x7f91e819db60 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f91dc031460 tx=0x7f91dc004530 comp rx=0 tx=0).stop 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 shutdown_connections 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f91d007ac40 0x7f91d007d100 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91e8107900 0x7f91e819db60 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 --2- 172.21.15.46:0/3215068970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91e8101900 0x7f91e819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 >> 172.21.15.46:0/3215068970 conn(0x7f91e80fd600 msgr2=0x7f91e80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 shutdown_connections 2024-09-10T04:29:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.839+0000 7f91ef0ca640 1 -- 172.21.15.46:0/3215068970 wait complete. 2024-09-10T04:29:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 -- 172.21.15.46:0/2894655241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074590 msgr2=0x7f523c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 --2- 172.21.15.46:0/2894655241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074590 0x7f523c074970 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5238009880 tx=0x7f523802a530 comp rx=0 tx=0).stop 2024-09-10T04:29:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 -- 172.21.15.46:0/2894655241 shutdown_connections 2024-09-10T04:29:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 --2- 172.21.15.46:0/2894655241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f523c074eb0 0x7f523c1123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 --2- 172.21.15.46:0/2894655241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074590 0x7f523c074970 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 -- 172.21.15.46:0/2894655241 >> 172.21.15.46:0/2894655241 conn(0x7f523c0fd600 msgr2=0x7f523c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:52.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 -- 172.21.15.46:0/2894655241 shutdown_connections 2024-09-10T04:29:52.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.923+0000 7f52438b1640 1 -- 172.21.15.46:0/2894655241 wait complete. 2024-09-10T04:29:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f52438b1640 1 Processor -- start 2024-09-10T04:29:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f52438b1640 1 -- start start 2024-09-10T04:29:52.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f52438b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f523c074590 0x7f523c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f52438b1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074eb0 0x7f523c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f52438b1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1153d0 con 0x7f523c074eb0 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f52438b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f523c115250 con 0x7f523c074590 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f5240e4c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074eb0 0x7f523c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f5240e4c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074eb0 0x7f523c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45066/0 (socket says 172.21.15.46:45066) 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.924+0000 7f5240e4c640 1 -- 172.21.15.46:0/1616047140 learned_addr learned my addr 172.21.15.46:0/1616047140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f524164d640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f523c074590 0x7f523c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f5240e4c640 1 -- 172.21.15.46:0/1616047140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f523c074590 msgr2=0x7f523c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f5240e4c640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f523c074590 0x7f523c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f5240e4c640 1 -- 172.21.15.46:0/1616047140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c1a28a0 con 0x7f523c074eb0 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f5240e4c640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074eb0 0x7f523c19e510 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c810 tx=0x7f522c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f52327fc640 1 -- 172.21.15.46:0/1616047140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f522c004020 con 0x7f523c074eb0 2024-09-10T04:29:52.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f52327fc640 1 -- 172.21.15.46:0/1616047140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f522c0041c0 con 0x7f523c074eb0 2024-09-10T04:29:52.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c1a2b90 con 0x7f523c074eb0 2024-09-10T04:29:52.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f52327fc640 1 -- 172.21.15.46:0/1616047140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f522c0047a0 con 0x7f523c074eb0 2024-09-10T04:29:52.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.925+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1a3120 con 0x7f523c074eb0 2024-09-10T04:29:52.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.926+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c075cd0 con 0x7f523c074eb0 2024-09-10T04:29:52.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.927+0000 7f52327fc640 1 -- 172.21.15.46:0/1616047140 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f522c0079b0 con 0x7f523c074eb0 2024-09-10T04:29:52.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.928+0000 7f52327fc640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f522807ac40 0x7f522807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:29:52.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.928+0000 7f524164d640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f522807ac40 0x7f522807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:29:52.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.928+0000 7f52327fc640 1 -- 172.21.15.46:0/1616047140 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f522c09dba0 con 0x7f523c074eb0 2024-09-10T04:29:52.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.928+0000 7f524164d640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f522807ac40 0x7f522807d100 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5238031460 tx=0x7f5238005bd0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:29:52.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:52.932+0000 7f52327fc640 1 -- 172.21.15.46:0/1616047140 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f522c014030 con 0x7f523c074eb0 2024-09-10T04:29:53.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.191+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f523c10fac0 con 0x7f523c074eb0 2024-09-10T04:29:53.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.191+0000 7f52327fc640 1 -- 172.21.15.46:0/1616047140 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f522c0692c0 con 0x7f523c074eb0 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:29:53.193 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:29:53.194 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:29:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:29:53.195 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:29:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f522807ac40 msgr2=0x7f522807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f522807ac40 0x7f522807d100 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5238031460 tx=0x7f5238005bd0 comp rx=0 tx=0).stop 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074eb0 msgr2=0x7f523c19e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074eb0 0x7f523c19e510 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c810 tx=0x7f522c00ccd0 comp rx=0 tx=0).stop 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 shutdown_connections 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f522807ac40 0x7f522807d100 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c074eb0 0x7f523c19e510 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 --2- 172.21.15.46:0/1616047140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f523c074590 0x7f523c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.194+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 >> 172.21.15.46:0/1616047140 conn(0x7f523c0fd600 msgr2=0x7f523c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.195+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 shutdown_connections 2024-09-10T04:29:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:29:53.195+0000 7f52438b1640 1 -- 172.21.15.46:0/1616047140 wait complete. 2024-09-10T04:29:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:53 smithi070 ceph-mon[79774]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:53 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3335454513' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:29:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:53 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1616047140' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:29:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:53 smithi046 ceph-mon[109506]: from='client.35186 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:53 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3335454513' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:29:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:29:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:53 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1616047140' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:29:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:54 smithi070 ceph-mon[79774]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:54 smithi070 ceph-mon[79774]: pgmap v728: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:54 smithi046 ceph-mon[109506]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:29:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:54 smithi046 ceph-mon[109506]: pgmap v728: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:56 smithi070 ceph-mon[79774]: pgmap v729: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:56.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:56 smithi046 ceph-mon[109506]: pgmap v729: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:29:58 smithi070 ceph-mon[79774]: pgmap v730: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:29:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:29:58 smithi046 ceph-mon[109506]: pgmap v730: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:30:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:30:00.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:29:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:30:00.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:30:00.339 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:30:00.000+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-10T04:30:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: pgmap v731: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: /usr/bin/podman: timeout after 895 seconds 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: Traceback (most recent call last): 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: return _run_code(code, main_globals, None, 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: exec(code, run_globals) 2024-09-10T04:30:00.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: fs cephfs is degraded 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:30:00.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:30:00.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:00 smithi070 ceph-mon[79774]: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:30:00.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: pgmap v731: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:00.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:30:00.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:30:00.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:30:00.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: /usr/bin/podman: timeout after 895 seconds 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: Traceback (most recent call last): 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: return _run_code(code, main_globals, None, 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: exec(code, run_globals) 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:30:00.839 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: fs cephfs is degraded 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:30:00.840 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:00 smithi046 ceph-mon[109506]: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:30:02.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:02 smithi070 ceph-mon[79774]: pgmap v732: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:02.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:02 smithi046 ceph-mon[109506]: pgmap v732: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:04 smithi070 ceph-mon[79774]: pgmap v733: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:04.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:04 smithi046 ceph-mon[109506]: pgmap v733: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:06.801 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:06 smithi070 ceph-mon[79774]: pgmap v734: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:06 smithi046 ceph-mon[109506]: pgmap v734: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:08 smithi070 ceph-mon[79774]: pgmap v735: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:08 smithi046 ceph-mon[109506]: pgmap v735: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:08.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:10 smithi070 ceph-mon[79774]: pgmap v736: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:10.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:10 smithi046 ceph-mon[109506]: pgmap v736: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:12 smithi070 ceph-mon[79774]: pgmap v737: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:12 smithi046 ceph-mon[109506]: pgmap v737: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:14 smithi070 ceph-mon[79774]: pgmap v738: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:14 smithi046 ceph-mon[109506]: pgmap v738: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:16 smithi070 ceph-mon[79774]: pgmap v739: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:16 smithi046 ceph-mon[109506]: pgmap v739: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:18 smithi070 ceph-mon[79774]: pgmap v740: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:18 smithi046 ceph-mon[109506]: pgmap v740: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:20 smithi070 ceph-mon[79774]: pgmap v741: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:20.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:20 smithi046 ceph-mon[109506]: pgmap v741: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:22 smithi070 ceph-mon[79774]: pgmap v742: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:22 smithi046 ceph-mon[109506]: pgmap v742: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 -- 172.21.15.46:0/3498809976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 msgr2=0x7f1474102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 --2- 172.21.15.46:0/3498809976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f1474102e90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1464009880 tx=0x7f146402a530 comp rx=0 tx=0).stop 2024-09-10T04:30:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 -- 172.21.15.46:0/3498809976 shutdown_connections 2024-09-10T04:30:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 --2- 172.21.15.46:0/3498809976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14741033d0 0x7f147410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 --2- 172.21.15.46:0/3498809976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f1474102e90 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 -- 172.21.15.46:0/3498809976 >> 172.21.15.46:0/3498809976 conn(0x7f14740fc550 msgr2=0x7f14740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 -- 172.21.15.46:0/3498809976 shutdown_connections 2024-09-10T04:30:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.294+0000 7f1479e63640 1 -- 172.21.15.46:0/3498809976 wait complete. 2024-09-10T04:30:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.295+0000 7f1479e63640 1 Processor -- start 2024-09-10T04:30:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.295+0000 7f1479e63640 1 -- start start 2024-09-10T04:30:23.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f1479e63640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f147419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f1479e63640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14741033d0 0x7f147419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f1479e63640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f147410ede0 con 0x7f1474102ab0 2024-09-10T04:30:23.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f1479e63640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f147410ec60 con 0x7f14741033d0 2024-09-10T04:30:23.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f14737fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f147419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f14737fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f147419f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57956/0 (socket says 172.21.15.46:57956) 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f14737fe640 1 -- 172.21.15.46:0/412864865 learned_addr learned my addr 172.21.15.46:0/412864865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f146affd640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14741033d0 0x7f147419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f14737fe640 1 -- 172.21.15.46:0/412864865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14741033d0 msgr2=0x7f147419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f14737fe640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14741033d0 0x7f147419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f14737fe640 1 -- 172.21.15.46:0/412864865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14741999b0 con 0x7f1474102ab0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.296+0000 7f14737fe640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f147419f7f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1464031460 tx=0x7f14640045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.297+0000 7f14717fa640 1 -- 172.21.15.46:0/412864865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f146403f070 con 0x7f1474102ab0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.297+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1474199c40 con 0x7f1474102ab0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.297+0000 7f14717fa640 1 -- 172.21.15.46:0/412864865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f14640035f0 con 0x7f1474102ab0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.297+0000 7f14717fa640 1 -- 172.21.15.46:0/412864865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1464035030 con 0x7f1474102ab0 2024-09-10T04:30:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.297+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f147419a170 con 0x7f1474102ab0 2024-09-10T04:30:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.298+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147406a580 con 0x7f1474102ab0 2024-09-10T04:30:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.299+0000 7f14717fa640 1 -- 172.21.15.46:0/412864865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1464042070 con 0x7f1474102ab0 2024-09-10T04:30:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.300+0000 7f14717fa640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f146007abf0 0x7f146007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.300+0000 7f14717fa640 1 -- 172.21.15.46:0/412864865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f14640be630 con 0x7f1474102ab0 2024-09-10T04:30:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.300+0000 7f146affd640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f146007abf0 0x7f146007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.301+0000 7f146affd640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f146007abf0 0x7f146007d0b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f14741026a0 tx=0x7f145c00b2e0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.305+0000 7f14717fa640 1 -- 172.21.15.46:0/412864865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1464089d10 con 0x7f1474102ab0 2024-09-10T04:30:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.511+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f147419aee0 con 0x7f146007abf0 2024-09-10T04:30:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.512+0000 7f14717fa640 1 -- 172.21.15.46:0/412864865 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f147419aee0 con 0x7f146007abf0 2024-09-10T04:30:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f146007abf0 msgr2=0x7f146007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f146007abf0 0x7f146007d0b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f14741026a0 tx=0x7f145c00b2e0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 msgr2=0x7f147419f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f147419f7f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1464031460 tx=0x7f14640045e0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 shutdown_connections 2024-09-10T04:30:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f146007abf0 0x7f146007d0b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14741033d0 0x7f147419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.514+0000 7f1479e63640 1 --2- 172.21.15.46:0/412864865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1474102ab0 0x7f147419f7f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.515+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 >> 172.21.15.46:0/412864865 conn(0x7f14740fc550 msgr2=0x7f1474100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.515+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 shutdown_connections 2024-09-10T04:30:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.515+0000 7f1479e63640 1 -- 172.21.15.46:0/412864865 wait complete. 2024-09-10T04:30:23.526 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 -- 172.21.15.46:0/2534734413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814101900 msgr2=0x7fc814101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 --2- 172.21.15.46:0/2534734413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814101900 0x7fc814101d60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009880 tx=0x7fc7fc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 -- 172.21.15.46:0/2534734413 shutdown_connections 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 --2- 172.21.15.46:0/2534734413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814101900 0x7fc814101d60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 --2- 172.21.15.46:0/2534734413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc814107900 0x7fc814107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 -- 172.21.15.46:0/2534734413 >> 172.21.15.46:0/2534734413 conn(0x7fc8140fd600 msgr2=0x7fc8140ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 -- 172.21.15.46:0/2534734413 shutdown_connections 2024-09-10T04:30:23.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.601+0000 7fc819120640 1 -- 172.21.15.46:0/2534734413 wait complete. 2024-09-10T04:30:23.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc819120640 1 Processor -- start 2024-09-10T04:30:23.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc819120640 1 -- start start 2024-09-10T04:30:23.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc819120640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc814101900 0x7fc8140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc819120640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814107900 0x7fc814074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc819120640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc81410edd0 con 0x7fc814107900 2024-09-10T04:30:23.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc819120640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc81410ec50 con 0x7fc814101900 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc81259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814107900 0x7fc814074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc81259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814107900 0x7fc814074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57972/0 (socket says 172.21.15.46:57972) 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc81259c640 1 -- 172.21.15.46:0/1029983816 learned_addr learned my addr 172.21.15.46:0/1029983816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.602+0000 7fc812d9d640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc814101900 0x7fc8140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc81259c640 1 -- 172.21.15.46:0/1029983816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc814101900 msgr2=0x7fc8140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc81259c640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc814101900 0x7fc8140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc81259c640 1 -- 172.21.15.46:0/1029983816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc814074f70 con 0x7fc814107900 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc81259c640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814107900 0x7fc814074a30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc031460 tx=0x7fc7fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc80bfff640 1 -- 172.21.15.46:0/1029983816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc7fc03f070 con 0x7fc814107900 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc814075200 con 0x7fc814107900 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc80bfff640 1 -- 172.21.15.46:0/1029983816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc7fc0035f0 con 0x7fc814107900 2024-09-10T04:30:23.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc80bfff640 1 -- 172.21.15.46:0/1029983816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc7fc035030 con 0x7fc814107900 2024-09-10T04:30:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.603+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8141a3d60 con 0x7fc814107900 2024-09-10T04:30:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.604+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc814103040 con 0x7fc814107900 2024-09-10T04:30:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.605+0000 7fc80bfff640 1 -- 172.21.15.46:0/1029983816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc7fc042070 con 0x7fc814107900 2024-09-10T04:30:23.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.606+0000 7fc80bfff640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc80007abf0 0x7fc80007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.606+0000 7fc80bfff640 1 -- 172.21.15.46:0/1029983816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc7fc0be670 con 0x7fc814107900 2024-09-10T04:30:23.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.606+0000 7fc812d9d640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc80007abf0 0x7fc80007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.606+0000 7fc812d9d640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc80007abf0 0x7fc80007d0b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc804005fd0 tx=0x7fc80400b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:23.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.610+0000 7fc80bfff640 1 -- 172.21.15.46:0/1029983816 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc7fc089d10 con 0x7fc814107900 2024-09-10T04:30:23.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.817+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc814062630 con 0x7fc80007abf0 2024-09-10T04:30:23.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.818+0000 7fc80bfff640 1 -- 172.21.15.46:0/1029983816 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc814062630 con 0x7fc80007abf0 2024-09-10T04:30:23.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc80007abf0 msgr2=0x7fc80007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc80007abf0 0x7fc80007d0b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc804005fd0 tx=0x7fc80400b390 comp rx=0 tx=0).stop 2024-09-10T04:30:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814107900 msgr2=0x7fc814074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814107900 0x7fc814074a30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc031460 tx=0x7fc7fc0045e0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 shutdown_connections 2024-09-10T04:30:23.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc80007abf0 0x7fc80007d0b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc814107900 0x7fc814074a30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 --2- 172.21.15.46:0/1029983816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc814101900 0x7fc8140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 >> 172.21.15.46:0/1029983816 conn(0x7fc8140fd600 msgr2=0x7fc8140fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:23.824 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 shutdown_connections 2024-09-10T04:30:23.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.820+0000 7fc819120640 1 -- 172.21.15.46:0/1029983816 wait complete. 2024-09-10T04:30:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 -- 172.21.15.46:0/3077656857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1007f0 msgr2=0x7fb62c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 --2- 172.21.15.46:0/3077656857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1007f0 0x7fb62c100c50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb618009880 tx=0x7fb61802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 -- 172.21.15.46:0/3077656857 shutdown_connections 2024-09-10T04:30:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 --2- 172.21.15.46:0/3077656857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1007f0 0x7fb62c100c50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 --2- 172.21.15.46:0/3077656857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c1067f0 0x7fb62c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 -- 172.21.15.46:0/3077656857 >> 172.21.15.46:0/3077656857 conn(0x7fb62c0fc550 msgr2=0x7fb62c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 -- 172.21.15.46:0/3077656857 shutdown_connections 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.905+0000 7fb632bca640 1 -- 172.21.15.46:0/3077656857 wait complete. 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb632bca640 1 Processor -- start 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb632bca640 1 -- start start 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb632bca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c1007f0 0x7fb62c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb632bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1067f0 0x7fb62c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb632bca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10dca0 con 0x7fb62c1067f0 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb632bca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10db20 con 0x7fb62c1007f0 2024-09-10T04:30:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb62bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1067f0 0x7fb62c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb62bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1067f0 0x7fb62c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57996/0 (socket says 172.21.15.46:57996) 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb62bfff640 1 -- 172.21.15.46:0/4282932222 learned_addr learned my addr 172.21.15.46:0/4282932222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb630966640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c1007f0 0x7fb62c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb62bfff640 1 -- 172.21.15.46:0/4282932222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c1007f0 msgr2=0x7fb62c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb62bfff640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c1007f0 0x7fb62c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.906+0000 7fb62bfff640 1 -- 172.21.15.46:0/4282932222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c0fe970 con 0x7fb62c1067f0 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.907+0000 7fb62bfff640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1067f0 0x7fb62c0fd1c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb618031460 tx=0x7fb6180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.907+0000 7fb629ffb640 1 -- 172.21.15.46:0/4282932222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb61803f070 con 0x7fb62c1067f0 2024-09-10T04:30:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.907+0000 7fb629ffb640 1 -- 172.21.15.46:0/4282932222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb618004cf0 con 0x7fb62c1067f0 2024-09-10T04:30:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.907+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c0fd700 con 0x7fb62c1067f0 2024-09-10T04:30:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.907+0000 7fb629ffb640 1 -- 172.21.15.46:0/4282932222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb618035030 con 0x7fb62c1067f0 2024-09-10T04:30:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.907+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1a82f0 con 0x7fb62c1067f0 2024-09-10T04:30:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.908+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62c101f30 con 0x7fb62c1067f0 2024-09-10T04:30:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.909+0000 7fb629ffb640 1 -- 172.21.15.46:0/4282932222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb618042070 con 0x7fb62c1067f0 2024-09-10T04:30:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.909+0000 7fb629ffb640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb61c07ac40 0x7fb61c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.909+0000 7fb629ffb640 1 -- 172.21.15.46:0/4282932222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb6180be680 con 0x7fb62c1067f0 2024-09-10T04:30:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.909+0000 7fb630966640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb61c07ac40 0x7fb61c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.910+0000 7fb630966640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb61c07ac40 0x7fb61c07d100 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb620005e10 tx=0x7fb620005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:23.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:23.914+0000 7fb629ffb640 1 -- 172.21.15.46:0/4282932222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb618089e40 con 0x7fb62c1067f0 2024-09-10T04:30:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.108+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb62c062630 con 0x7fb61c07ac40 2024-09-10T04:30:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.118+0000 7fb629ffb640 1 -- 172.21.15.46:0/4282932222 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fb62c062630 con 0x7fb61c07ac40 2024-09-10T04:30:24.120 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:30:24.120 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (21m) 62s ago 33m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:30:24.120 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (24m) 62s ago 33m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:30:24.120 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (24m) 65s ago 32m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:30:24.120 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (20m) 62s ago 33m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:30:24.120 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (20m) 65s ago 32m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (21m) 62s ago 33m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (16m) 62s ago 27m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (16m) 62s ago 27m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (16m) 65s ago 27m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (68s) 65s ago 27m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (23m) 62s ago 34m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (23m) 65s ago 32m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (20m) 62s ago 34m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (20m) 65s ago 32m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (22m) 62s ago 33m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (22m) 65s ago 32m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:30:24.121 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (19m) 62s ago 31m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:30:24.122 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (19m) 62s ago 30m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:30:24.122 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (18m) 62s ago 30m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:30:24.122 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (18m) 65s ago 29m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:30:24.122 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (17m) 65s ago 29m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:30:24.122 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (17m) 65s ago 28m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:30:24.122 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (22m) 62s ago 32m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb61c07ac40 msgr2=0x7fb61c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb61c07ac40 0x7fb61c07d100 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb620005e10 tx=0x7fb620005da0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1067f0 msgr2=0x7fb62c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1067f0 0x7fb62c0fd1c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb618031460 tx=0x7fb6180048f0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 shutdown_connections 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb61c07ac40 0x7fb61c07d100 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb62c1067f0 0x7fb62c0fd1c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 --2- 172.21.15.46:0/4282932222 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb62c1007f0 0x7fb62c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 >> 172.21.15.46:0/4282932222 conn(0x7fb62c0fc550 msgr2=0x7fb62c069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 shutdown_connections 2024-09-10T04:30:24.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.120+0000 7fb632bca640 1 -- 172.21.15.46:0/4282932222 wait complete. 2024-09-10T04:30:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 -- 172.21.15.46:0/1884424964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7508102ac0 msgr2=0x7f7508102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 --2- 172.21.15.46:0/1884424964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7508102ac0 0x7f7508102ea0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009c20 tx=0x7f74f802aa00 comp rx=0 tx=0).stop 2024-09-10T04:30:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 -- 172.21.15.46:0/1884424964 shutdown_connections 2024-09-10T04:30:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 --2- 172.21.15.46:0/1884424964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75081033e0 0x7f750810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 --2- 172.21.15.46:0/1884424964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7508102ac0 0x7f7508102ea0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 -- 172.21.15.46:0/1884424964 >> 172.21.15.46:0/1884424964 conn(0x7f75080fc520 msgr2=0x7f75080fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 -- 172.21.15.46:0/1884424964 shutdown_connections 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 -- 172.21.15.46:0/1884424964 wait complete. 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 Processor -- start 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.204+0000 7f751037d640 1 -- start start 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f751037d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7508102ac0 0x7f750819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f751037d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75081033e0 0x7f750819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f751037d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f750810edf0 con 0x7f75081033e0 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f751037d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f750810ec70 con 0x7f7508102ac0 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750d918640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75081033e0 0x7f750819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750d918640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75081033e0 0x7f750819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58014/0 (socket says 172.21.15.46:58014) 2024-09-10T04:30:24.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750d918640 1 -- 172.21.15.46:0/3884299545 learned_addr learned my addr 172.21.15.46:0/3884299545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750e119640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7508102ac0 0x7f750819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750d918640 1 -- 172.21.15.46:0/3884299545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7508102ac0 msgr2=0x7f750819d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750d918640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7508102ac0 0x7f750819d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750d918640 1 -- 172.21.15.46:0/3884299545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7508197800 con 0x7f75081033e0 2024-09-10T04:30:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.205+0000 7f750d918640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75081033e0 0x7f750819dbc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00e970 tx=0x7f74fc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.206+0000 7f75077fe640 1 -- 172.21.15.46:0/3884299545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74fc00cac0 con 0x7f75081033e0 2024-09-10T04:30:24.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.206+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7508197af0 con 0x7f75081033e0 2024-09-10T04:30:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.206+0000 7f75077fe640 1 -- 172.21.15.46:0/3884299545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f74fc0054e0 con 0x7f75081033e0 2024-09-10T04:30:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.206+0000 7f75077fe640 1 -- 172.21.15.46:0/3884299545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f74fc004dd0 con 0x7f75081033e0 2024-09-10T04:30:24.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.206+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7508198080 con 0x7f75081033e0 2024-09-10T04:30:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.207+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750806aa90 con 0x7f75081033e0 2024-09-10T04:30:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.207+0000 7f75077fe640 1 -- 172.21.15.46:0/3884299545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f74fc005020 con 0x7f75081033e0 2024-09-10T04:30:24.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.208+0000 7f75077fe640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f74f007ac00 0x7f74f007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.208+0000 7f750e119640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f74f007ac00 0x7f74f007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.208+0000 7f75077fe640 1 -- 172.21.15.46:0/3884299545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f74fc09e1e0 con 0x7f75081033e0 2024-09-10T04:30:24.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.209+0000 7f750e119640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f74f007ac00 0x7f74f007d0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009430 tx=0x7f74f803a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:24.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.212+0000 7f75077fe640 1 -- 172.21.15.46:0/3884299545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f74fc0699a0 con 0x7f75081033e0 2024-09-10T04:30:24.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.477+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7508105f90 con 0x7f75081033e0 2024-09-10T04:30:24.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.478+0000 7f75077fe640 1 -- 172.21.15.46:0/3884299545 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f74fc06e850 con 0x7f75081033e0 2024-09-10T04:30:24.480 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:30:24.480 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:30:24.480 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:30:24.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:24.480 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:30:24.481 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:30:24.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.480+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f74f007ac00 msgr2=0x7f74f007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.480+0000 7f751037d640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f74f007ac00 0x7f74f007d0c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f74f8009430 tx=0x7f74f803a040 comp rx=0 tx=0).stop 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.480+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75081033e0 msgr2=0x7f750819dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.480+0000 7f751037d640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75081033e0 0x7f750819dbc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00e970 tx=0x7f74fc00ee30 comp rx=0 tx=0).stop 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.481+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 shutdown_connections 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.481+0000 7f751037d640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f74f007ac00 0x7f74f007d0c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.481+0000 7f751037d640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75081033e0 0x7f750819dbc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.481+0000 7f751037d640 1 --2- 172.21.15.46:0/3884299545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7508102ac0 0x7f750819d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.481+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 >> 172.21.15.46:0/3884299545 conn(0x7f75080fc520 msgr2=0x7f7508108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.481+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 shutdown_connections 2024-09-10T04:30:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.481+0000 7f751037d640 1 -- 172.21.15.46:0/3884299545 wait complete. 2024-09-10T04:30:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 -- 172.21.15.46:0/2398487580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 msgr2=0x7f9954106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 --2- 172.21.15.46:0/2398487580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954106be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f9944009880 tx=0x7f994402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 -- 172.21.15.46:0/2398487580 shutdown_connections 2024-09-10T04:30:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 --2- 172.21.15.46:0/2398487580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9954100800 0x7f9954100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 --2- 172.21.15.46:0/2398487580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954106be0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 -- 172.21.15.46:0/2398487580 >> 172.21.15.46:0/2398487580 conn(0x7f99540fc520 msgr2=0x7f99540fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 -- 172.21.15.46:0/2398487580 shutdown_connections 2024-09-10T04:30:24.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.565+0000 7f9958dad640 1 -- 172.21.15.46:0/2398487580 wait complete. 2024-09-10T04:30:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9958dad640 1 Processor -- start 2024-09-10T04:30:24.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9958dad640 1 -- start start 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9958dad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9954100800 0x7f9954198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9958dad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9958dad640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f995410dcd0 con 0x7f9954106800 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9958dad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f995410db50 con 0x7f9954100800 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9949d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58030/0 (socket says 172.21.15.46:58030) 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f9949d9b640 1 -- 172.21.15.46:0/2385064728 learned_addr learned my addr 172.21.15.46:0/2385064728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.566+0000 7f995259c640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9954100800 0x7f9954198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f9949d9b640 1 -- 172.21.15.46:0/2385064728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9954100800 msgr2=0x7f9954198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f9949d9b640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9954100800 0x7f9954198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f9949d9b640 1 -- 172.21.15.46:0/2385064728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9954199350 con 0x7f9954106800 2024-09-10T04:30:24.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f9949d9b640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954198d50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e970 tx=0x7f993c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f994bfff640 1 -- 172.21.15.46:0/2385064728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f993c00cac0 con 0x7f9954106800 2024-09-10T04:30:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f994bfff640 1 -- 172.21.15.46:0/2385064728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f993c0054e0 con 0x7f9954106800 2024-09-10T04:30:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9954199640 con 0x7f9954106800 2024-09-10T04:30:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f994bfff640 1 -- 172.21.15.46:0/2385064728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f993c010430 con 0x7f9954106800 2024-09-10T04:30:24.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.567+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99541a1c40 con 0x7f9954106800 2024-09-10T04:30:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.568+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9954101f40 con 0x7f9954106800 2024-09-10T04:30:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.569+0000 7f994bfff640 1 -- 172.21.15.46:0/2385064728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f993c004e80 con 0x7f9954106800 2024-09-10T04:30:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.569+0000 7f994bfff640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994007abf0 0x7f994007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.569+0000 7f994bfff640 1 -- 172.21.15.46:0/2385064728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f993c09e2b0 con 0x7f9954106800 2024-09-10T04:30:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.569+0000 7f995259c640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994007abf0 0x7f994007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.570+0000 7f995259c640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994007abf0 0x7f994007d0b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f99440096c0 tx=0x7f9944031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:24.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.574+0000 7f994bfff640 1 -- 172.21.15.46:0/2385064728 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9954199640 con 0x7f9954106800 2024-09-10T04:30:24.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.801+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9954108d70 con 0x7f9954106800 2024-09-10T04:30:24.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:24 smithi046 ceph-mon[109506]: pgmap v743: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:24.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:24 smithi046 ceph-mon[109506]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:24.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:24 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3884299545' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.801+0000 7f994bfff640 1 -- 172.21.15.46:0/2385064728 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f993c0699f0 con 0x7f9954106800 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:30:24.804 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:30:24.805 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:30:24.806 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:24.807 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994007abf0 msgr2=0x7f994007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994007abf0 0x7f994007d0b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f99440096c0 tx=0x7f9944031400 comp rx=0 tx=0).stop 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 msgr2=0x7f9954198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954198d50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f993c00e970 tx=0x7f993c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 shutdown_connections 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994007abf0 0x7f994007d0b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9954106800 0x7f9954198d50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 --2- 172.21.15.46:0/2385064728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9954100800 0x7f9954198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 >> 172.21.15.46:0/2385064728 conn(0x7f99540fc520 msgr2=0x7f99540fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 shutdown_connections 2024-09-10T04:30:24.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.804+0000 7f9958dad640 1 -- 172.21.15.46:0/2385064728 wait complete. 2024-09-10T04:30:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:24 smithi070 ceph-mon[79774]: pgmap v743: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:24 smithi070 ceph-mon[79774]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:24 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3884299545' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:30:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 -- 172.21.15.46:0/2395777245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5718102ac0 msgr2=0x7f5718102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 --2- 172.21.15.46:0/2395777245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5718102ac0 0x7f5718102ea0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5714009880 tx=0x7f571402a530 comp rx=0 tx=0).stop 2024-09-10T04:30:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 -- 172.21.15.46:0/2395777245 shutdown_connections 2024-09-10T04:30:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 --2- 172.21.15.46:0/2395777245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57181033e0 0x7f571810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 --2- 172.21.15.46:0/2395777245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5718102ac0 0x7f5718102ea0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 -- 172.21.15.46:0/2395777245 >> 172.21.15.46:0/2395777245 conn(0x7f57180fc520 msgr2=0x7f57180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 -- 172.21.15.46:0/2395777245 shutdown_connections 2024-09-10T04:30:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.890+0000 7f571f48b640 1 -- 172.21.15.46:0/2395777245 wait complete. 2024-09-10T04:30:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571f48b640 1 Processor -- start 2024-09-10T04:30:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571f48b640 1 -- start start 2024-09-10T04:30:24.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571f48b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5718102ac0 0x7f571819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571f48b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57181033e0 0x7f571819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571f48b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f571810edf0 con 0x7f57181033e0 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571f48b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f571810ec70 con 0x7f5718102ac0 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571ca26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57181033e0 0x7f571819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571ca26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57181033e0 0x7f571819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58048/0 (socket says 172.21.15.46:58048) 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571ca26640 1 -- 172.21.15.46:0/1201948402 learned_addr learned my addr 172.21.15.46:0/1201948402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571d227640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5718102ac0 0x7f571819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571ca26640 1 -- 172.21.15.46:0/1201948402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5718102ac0 msgr2=0x7f571819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571ca26640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5718102ac0 0x7f571819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.891+0000 7f571ca26640 1 -- 172.21.15.46:0/1201948402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5718199a10 con 0x7f57181033e0 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.892+0000 7f571ca26640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57181033e0 0x7f571819fd40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f570800d6d0 tx=0x7f570800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.892+0000 7f570e7fc640 1 -- 172.21.15.46:0/1201948402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5708017070 con 0x7f57181033e0 2024-09-10T04:30:24.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.892+0000 7f570e7fc640 1 -- 172.21.15.46:0/1201948402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57080054e0 con 0x7f57181033e0 2024-09-10T04:30:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.892+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5718199d00 con 0x7f57181033e0 2024-09-10T04:30:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.892+0000 7f570e7fc640 1 -- 172.21.15.46:0/1201948402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5708004ae0 con 0x7f57181033e0 2024-09-10T04:30:24.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.892+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571819a290 con 0x7f57181033e0 2024-09-10T04:30:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.893+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571806aa90 con 0x7f57181033e0 2024-09-10T04:30:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.894+0000 7f570e7fc640 1 -- 172.21.15.46:0/1201948402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5708004d90 con 0x7f57181033e0 2024-09-10T04:30:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.894+0000 7f570e7fc640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f570407ac40 0x7f570407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.894+0000 7f570e7fc640 1 -- 172.21.15.46:0/1201948402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f570809d720 con 0x7f57181033e0 2024-09-10T04:30:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.894+0000 7f571d227640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f570407ac40 0x7f570407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:24.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.895+0000 7f571d227640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f570407ac40 0x7f570407d100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5714000c00 tx=0x7f5714031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:24.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:24.899+0000 7f570e7fc640 1 -- 172.21.15.46:0/1201948402 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5708068ee0 con 0x7f57181033e0 2024-09-10T04:30:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.101+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f571819aef0 con 0x7f570407ac40 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.102+0000 7f570e7fc640 1 -- 172.21.15.46:0/1201948402 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f571819aef0 con 0x7f570407ac40 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:30:25.104 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:30:25.105 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:30:25.105 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:30:25.105 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:30:25.105 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:30:25.105 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:30:25.105 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:30:25.105 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:30:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f570407ac40 msgr2=0x7f570407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f570407ac40 0x7f570407d100 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5714000c00 tx=0x7f5714031400 comp rx=0 tx=0).stop 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57181033e0 msgr2=0x7f571819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57181033e0 0x7f571819fd40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f570800d6d0 tx=0x7f570800db90 comp rx=0 tx=0).stop 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 shutdown_connections 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f570407ac40 0x7f570407d100 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57181033e0 0x7f571819fd40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 --2- 172.21.15.46:0/1201948402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5718102ac0 0x7f571819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.104+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 >> 172.21.15.46:0/1201948402 conn(0x7f57180fc520 msgr2=0x7f5718100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.105+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 shutdown_connections 2024-09-10T04:30:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.105+0000 7f571f48b640 1 -- 172.21.15.46:0/1201948402 wait complete. 2024-09-10T04:30:25.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/650675553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 msgr2=0x7fe3f8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/650675553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f8102ea0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009880 tx=0x7fe3e802a530 comp rx=0 tx=0).stop 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/650675553 shutdown_connections 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/650675553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f81033e0 0x7fe3f810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/650675553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f8102ea0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/650675553 >> 172.21.15.46:0/650675553 conn(0x7fe3f80fc520 msgr2=0x7fe3f80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/650675553 shutdown_connections 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.188+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/650675553 wait complete. 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3fd3b2640 1 Processor -- start 2024-09-10T04:30:25.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3fd3b2640 1 -- start start 2024-09-10T04:30:25.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3fd3b2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3fd3b2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f81033e0 0x7fe3f819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3fd3b2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f810edf0 con 0x7fe3f8102ac0 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3fd3b2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f810ec70 con 0x7fe3f81033e0 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3f6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3f6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58064/0 (socket says 172.21.15.46:58064) 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.189+0000 7fe3f6ffd640 1 -- 172.21.15.46:0/2867673399 learned_addr learned my addr 172.21.15.46:0/2867673399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3f67fc640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f81033e0 0x7fe3f819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3f6ffd640 1 -- 172.21.15.46:0/2867673399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f81033e0 msgr2=0x7fe3f819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3f6ffd640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f81033e0 0x7fe3f819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3f6ffd640 1 -- 172.21.15.46:0/2867673399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f81977e0 con 0x7fe3f8102ac0 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3f6ffd640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f819d660 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8031460 tx=0x7fe3e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:25.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3dffff640 1 -- 172.21.15.46:0/2867673399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3e803f070 con 0x7fe3f8102ac0 2024-09-10T04:30:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f8197a70 con 0x7fe3f8102ac0 2024-09-10T04:30:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3dffff640 1 -- 172.21.15.46:0/2867673399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3e80035f0 con 0x7fe3f8102ac0 2024-09-10T04:30:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3dffff640 1 -- 172.21.15.46:0/2867673399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3e8035030 con 0x7fe3f8102ac0 2024-09-10T04:30:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.190+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f8197fa0 con 0x7fe3f8102ac0 2024-09-10T04:30:25.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.191+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f806aa90 con 0x7fe3f8102ac0 2024-09-10T04:30:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.192+0000 7fe3dffff640 1 -- 172.21.15.46:0/2867673399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe3e8042070 con 0x7fe3f8102ac0 2024-09-10T04:30:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.193+0000 7fe3dffff640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3e007abf0 0x7fe3e007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.193+0000 7fe3f67fc640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3e007abf0 0x7fe3e007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.193+0000 7fe3dffff640 1 -- 172.21.15.46:0/2867673399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe3e80be670 con 0x7fe3f8102ac0 2024-09-10T04:30:25.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.193+0000 7fe3f67fc640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3e007abf0 0x7fe3e007d0b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8108900 tx=0x7fe3ec00b2e0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:25.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.197+0000 7fe3dffff640 1 -- 172.21.15.46:0/2867673399 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe3e8089d10 con 0x7fe3f8102ac0 2024-09-10T04:30:25.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.459+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe3f8105f90 con 0x7fe3f8102ac0 2024-09-10T04:30:25.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.459+0000 7fe3dffff640 1 -- 172.21.15.46:0/2867673399 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fe3e808ebc0 con 0x7fe3f8102ac0 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:30:25.462 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:30:25.463 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:25.464 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:30:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:30:25.464 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:30:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:30:25.464 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:30:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.462+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3e007abf0 msgr2=0x7fe3e007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.462+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3e007abf0 0x7fe3e007d0b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8108900 tx=0x7fe3ec00b2e0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.462+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 msgr2=0x7fe3f819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.462+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f819d660 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8031460 tx=0x7fe3e80045e0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.463+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 shutdown_connections 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.463+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3e007abf0 0x7fe3e007d0b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.463+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f81033e0 0x7fe3f819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.463+0000 7fe3fd3b2640 1 --2- 172.21.15.46:0/2867673399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8102ac0 0x7fe3f819d660 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.463+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 >> 172.21.15.46:0/2867673399 conn(0x7fe3f80fc520 msgr2=0x7fe3f8108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:25.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.463+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 shutdown_connections 2024-09-10T04:30:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:25.463+0000 7fe3fd3b2640 1 -- 172.21.15.46:0/2867673399 wait complete. 2024-09-10T04:30:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:25 smithi070 ceph-mon[79774]: from='client.35210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:25 smithi070 ceph-mon[79774]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:25 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2385064728' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:30:25.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:25 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2867673399' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:30:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:25 smithi046 ceph-mon[109506]: from='client.35210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:25 smithi046 ceph-mon[109506]: from='client.35214 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:25 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2385064728' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:30:25.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:25 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2867673399' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:30:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:26 smithi070 ceph-mon[79774]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:26 smithi070 ceph-mon[79774]: pgmap v744: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:26.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:26 smithi046 ceph-mon[109506]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:26 smithi046 ceph-mon[109506]: pgmap v744: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:28 smithi070 ceph-mon[79774]: pgmap v745: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:28.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:28 smithi046 ceph-mon[109506]: pgmap v745: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:30 smithi070 ceph-mon[79774]: pgmap v746: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:31.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:30 smithi046 ceph-mon[109506]: pgmap v746: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:32 smithi070 ceph-mon[79774]: pgmap v747: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:32 smithi046 ceph-mon[109506]: pgmap v747: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:34 smithi070 ceph-mon[79774]: pgmap v748: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:34 smithi046 ceph-mon[109506]: pgmap v748: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:36 smithi070 ceph-mon[79774]: pgmap v749: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:36 smithi046 ceph-mon[109506]: pgmap v749: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:38 smithi070 ceph-mon[79774]: pgmap v750: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:38 smithi046 ceph-mon[109506]: pgmap v750: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:40 smithi070 ceph-mon[79774]: pgmap v751: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:40 smithi046 ceph-mon[109506]: pgmap v751: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:42 smithi046 ceph-mon[109506]: pgmap v752: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:42 smithi070 ceph-mon[79774]: pgmap v752: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:44 smithi070 ceph-mon[79774]: pgmap v753: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:44 smithi046 ceph-mon[109506]: pgmap v753: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:46 smithi070 ceph-mon[79774]: pgmap v754: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:46 smithi046 ceph-mon[109506]: pgmap v754: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:48 smithi070 ceph-mon[79774]: pgmap v755: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:48 smithi046 ceph-mon[109506]: pgmap v755: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:50.887 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:50 smithi070 ceph-mon[79774]: pgmap v756: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:50 smithi046 ceph-mon[109506]: pgmap v756: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:52.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:51 smithi070 ceph-mon[79774]: pgmap v757: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:52.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:51 smithi046 ceph-mon[109506]: pgmap v757: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:53.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:53.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:30:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:54 smithi070 ceph-mon[79774]: pgmap v758: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:54.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:54 smithi046 ceph-mon[109506]: pgmap v758: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:55.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 -- 172.21.15.46:0/1765411225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda81033e0 msgr2=0x7fdda810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:55.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 --2- 172.21.15.46:0/1765411225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda81033e0 0x7fdda810b8d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009880 tx=0x7fdd9c02a530 comp rx=0 tx=0).stop 2024-09-10T04:30:55.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 -- 172.21.15.46:0/1765411225 shutdown_connections 2024-09-10T04:30:55.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 --2- 172.21.15.46:0/1765411225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda81033e0 0x7fdda810b8d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 --2- 172.21.15.46:0/1765411225 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdda8102ac0 0x7fdda8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 -- 172.21.15.46:0/1765411225 >> 172.21.15.46:0/1765411225 conn(0x7fdda80fc520 msgr2=0x7fdda80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:55.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 -- 172.21.15.46:0/1765411225 shutdown_connections 2024-09-10T04:30:55.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.563+0000 7fddad915640 1 -- 172.21.15.46:0/1765411225 wait complete. 2024-09-10T04:30:55.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.564+0000 7fddad915640 1 Processor -- start 2024-09-10T04:30:55.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.564+0000 7fddad915640 1 -- start start 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.564+0000 7fddad915640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8102ac0 0x7fdda819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.564+0000 7fddad915640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdda81033e0 0x7fdda819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.564+0000 7fddad915640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdda810edf0 con 0x7fdda8102ac0 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.564+0000 7fddad915640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdda810ec70 con 0x7fdda81033e0 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8102ac0 0x7fdda819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8102ac0 0x7fdda819f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55806/0 (socket says 172.21.15.46:55806) 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda6ffd640 1 -- 172.21.15.46:0/1395830256 learned_addr learned my addr 172.21.15.46:0/1395830256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda67fc640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdda81033e0 0x7fdda819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda6ffd640 1 -- 172.21.15.46:0/1395830256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdda81033e0 msgr2=0x7fdda819fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda6ffd640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdda81033e0 0x7fdda819fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda6ffd640 1 -- 172.21.15.46:0/1395830256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda8199ab0 con 0x7fdda8102ac0 2024-09-10T04:30:55.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fdda6ffd640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8102ac0 0x7fdda819f8f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800e970 tx=0x7fdd9800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:55.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fddac913640 1 -- 172.21.15.46:0/1395830256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdd9800cac0 con 0x7fdda8102ac0 2024-09-10T04:30:55.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda8199da0 con 0x7fdda8102ac0 2024-09-10T04:30:55.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fddac913640 1 -- 172.21.15.46:0/1395830256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd980054e0 con 0x7fdda8102ac0 2024-09-10T04:30:55.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fddac913640 1 -- 172.21.15.46:0/1395830256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdd98010430 con 0x7fdda8102ac0 2024-09-10T04:30:55.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.565+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda819a300 con 0x7fdda8102ac0 2024-09-10T04:30:55.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.567+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda806aa90 con 0x7fdda8102ac0 2024-09-10T04:30:55.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.567+0000 7fddac913640 1 -- 172.21.15.46:0/1395830256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdd980106e0 con 0x7fdda8102ac0 2024-09-10T04:30:55.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.568+0000 7fddac913640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd9007abf0 0x7fdd9007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:55.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.568+0000 7fddac913640 1 -- 172.21.15.46:0/1395830256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdd9809e3a0 con 0x7fdda8102ac0 2024-09-10T04:30:55.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.568+0000 7fdda67fc640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd9007abf0 0x7fdd9007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:55.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.568+0000 7fdda67fc640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd9007abf0 0x7fdd9007d0b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdda819b2a0 tx=0x7fdd9c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:55.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.572+0000 7fddac913640 1 -- 172.21.15.46:0/1395830256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdd98069ae0 con 0x7fdda8102ac0 2024-09-10T04:30:55.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.778+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdda819afe0 con 0x7fdd9007abf0 2024-09-10T04:30:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.779+0000 7fddac913640 1 -- 172.21.15.46:0/1395830256 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fdda819afe0 con 0x7fdd9007abf0 2024-09-10T04:30:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd9007abf0 msgr2=0x7fdd9007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd9007abf0 0x7fdd9007d0b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdda819b2a0 tx=0x7fdd9c031400 comp rx=0 tx=0).stop 2024-09-10T04:30:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8102ac0 msgr2=0x7fdda819f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8102ac0 0x7fdda819f8f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800e970 tx=0x7fdd9800ed70 comp rx=0 tx=0).stop 2024-09-10T04:30:55.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 shutdown_connections 2024-09-10T04:30:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd9007abf0 0x7fdd9007d0b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdda81033e0 0x7fdda819fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 --2- 172.21.15.46:0/1395830256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdda8102ac0 0x7fdda819f8f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 >> 172.21.15.46:0/1395830256 conn(0x7fdda80fc520 msgr2=0x7fdda8100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 shutdown_connections 2024-09-10T04:30:55.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.781+0000 7fddad915640 1 -- 172.21.15.46:0/1395830256 wait complete. 2024-09-10T04:30:55.792 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.867+0000 7fa338076640 1 -- 172.21.15.46:0/376674301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 msgr2=0x7fa330100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.867+0000 7fa338076640 1 --2- 172.21.15.46:0/376674301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100220 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa320009880 tx=0x7fa32002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 -- 172.21.15.46:0/376674301 shutdown_connections 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 --2- 172.21.15.46:0/376674301 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100220 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 --2- 172.21.15.46:0/376674301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3301069a0 0x7fa330106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 -- 172.21.15.46:0/376674301 >> 172.21.15.46:0/376674301 conn(0x7fa3300fbc90 msgr2=0x7fa3300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 -- 172.21.15.46:0/376674301 shutdown_connections 2024-09-10T04:30:55.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 -- 172.21.15.46:0/376674301 wait complete. 2024-09-10T04:30:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 Processor -- start 2024-09-10T04:30:55.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.868+0000 7fa338076640 1 -- start start 2024-09-10T04:30:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa338076640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa338076640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3301069a0 0x7fa3301014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:55.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa338076640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa33010ef40 con 0x7fa3300ffdc0 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa338076640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa33010edc0 con 0x7fa3301069a0 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335e12640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55826/0 (socket says 172.21.15.46:55826) 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335e12640 1 -- 172.21.15.46:0/29293199 learned_addr learned my addr 172.21.15.46:0/29293199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335611640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3301069a0 0x7fa3301014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335e12640 1 -- 172.21.15.46:0/29293199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3301069a0 msgr2=0x7fa3301014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335e12640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3301069a0 0x7fa3301014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335e12640 1 -- 172.21.15.46:0/29293199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa330101a20 con 0x7fa3300ffdc0 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa335e12640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e970 tx=0x7fa32c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:55.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa326ffd640 1 -- 172.21.15.46:0/29293199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa32c00cac0 con 0x7fa3300ffdc0 2024-09-10T04:30:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa326ffd640 1 -- 172.21.15.46:0/29293199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa32c0054e0 con 0x7fa3300ffdc0 2024-09-10T04:30:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa330101d10 con 0x7fa3300ffdc0 2024-09-10T04:30:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa326ffd640 1 -- 172.21.15.46:0/29293199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa32c010430 con 0x7fa3300ffdc0 2024-09-10T04:30:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.869+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301a3f70 con 0x7fa3300ffdc0 2024-09-10T04:30:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.870+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3301030e0 con 0x7fa3300ffdc0 2024-09-10T04:30:55.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.871+0000 7fa326ffd640 1 -- 172.21.15.46:0/29293199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa32c004e80 con 0x7fa3300ffdc0 2024-09-10T04:30:55.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.872+0000 7fa326ffd640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa31c07abf0 0x7fa31c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.872+0000 7fa326ffd640 1 -- 172.21.15.46:0/29293199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa32c09e350 con 0x7fa3300ffdc0 2024-09-10T04:30:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.872+0000 7fa335611640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa31c07abf0 0x7fa31c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:55.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.872+0000 7fa335611640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa31c07abf0 0x7fa31c07d0b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa330102a30 tx=0x7fa320031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:55.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:55.876+0000 7fa326ffd640 1 -- 172.21.15.46:0/29293199 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa32c0699f0 con 0x7fa3300ffdc0 2024-09-10T04:30:56.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.081+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa330062630 con 0x7fa31c07abf0 2024-09-10T04:30:56.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.082+0000 7fa326ffd640 1 -- 172.21.15.46:0/29293199 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fa330062630 con 0x7fa31c07abf0 2024-09-10T04:30:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa31c07abf0 msgr2=0x7fa31c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa31c07abf0 0x7fa31c07d0b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa330102a30 tx=0x7fa320031400 comp rx=0 tx=0).stop 2024-09-10T04:30:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 msgr2=0x7fa330100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100fa0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e970 tx=0x7fa32c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:30:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 shutdown_connections 2024-09-10T04:30:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa31c07abf0 0x7fa31c07d0b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3301069a0 0x7fa3301014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 --2- 172.21.15.46:0/29293199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3300ffdc0 0x7fa330100fa0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 >> 172.21.15.46:0/29293199 conn(0x7fa3300fbc90 msgr2=0x7fa3300fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 shutdown_connections 2024-09-10T04:30:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.084+0000 7fa338076640 1 -- 172.21.15.46:0/29293199 wait complete. 2024-09-10T04:30:56.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.168+0000 7fd5a683a640 1 -- 172.21.15.46:0/1789829433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 msgr2=0x7fd5a0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.168+0000 7fd5a683a640 1 --2- 172.21.15.46:0/1789829433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a0101d60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd594009880 tx=0x7fd59402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.169+0000 7fd5a683a640 1 -- 172.21.15.46:0/1789829433 shutdown_connections 2024-09-10T04:30:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.169+0000 7fd5a683a640 1 --2- 172.21.15.46:0/1789829433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a0101d60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.169+0000 7fd5a683a640 1 --2- 172.21.15.46:0/1789829433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a0107900 0x7fd5a0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.169+0000 7fd5a683a640 1 -- 172.21.15.46:0/1789829433 >> 172.21.15.46:0/1789829433 conn(0x7fd5a00fd600 msgr2=0x7fd5a00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.169+0000 7fd5a683a640 1 -- 172.21.15.46:0/1789829433 shutdown_connections 2024-09-10T04:30:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.169+0000 7fd5a683a640 1 -- 172.21.15.46:0/1789829433 wait complete. 2024-09-10T04:30:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd5a683a640 1 Processor -- start 2024-09-10T04:30:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd5a683a640 1 -- start start 2024-09-10T04:30:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd5a683a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd5a683a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a0107900 0x7fd5a019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd5a683a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a010edd0 con 0x7fd5a0101900 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd5a683a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a010ec50 con 0x7fd5a0107900 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd59ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55834/0 (socket says 172.21.15.46:55834) 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.170+0000 7fd59ffff640 1 -- 172.21.15.46:0/3951510401 learned_addr learned my addr 172.21.15.46:0/3951510401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59f7fe640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a0107900 0x7fd5a019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59ffff640 1 -- 172.21.15.46:0/3951510401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a0107900 msgr2=0x7fd5a019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59ffff640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a0107900 0x7fd5a019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59ffff640 1 -- 172.21.15.46:0/3951510401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a0197800 con 0x7fd5a0101900 2024-09-10T04:30:56.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59ffff640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a019d680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c8f0 tx=0x7fd59000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59d7fa640 1 -- 172.21.15.46:0/3951510401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd5900079b0 con 0x7fd5a0101900 2024-09-10T04:30:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a0197af0 con 0x7fd5a0101900 2024-09-10T04:30:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59d7fa640 1 -- 172.21.15.46:0/3951510401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5900054e0 con 0x7fd5a0101900 2024-09-10T04:30:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd59d7fa640 1 -- 172.21.15.46:0/3951510401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd590004dd0 con 0x7fd5a0101900 2024-09-10T04:30:56.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.171+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a0198050 con 0x7fd5a0101900 2024-09-10T04:30:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.172+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a0103040 con 0x7fd5a0101900 2024-09-10T04:30:56.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.173+0000 7fd59d7fa640 1 -- 172.21.15.46:0/3951510401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd590005020 con 0x7fd5a0101900 2024-09-10T04:30:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.174+0000 7fd59d7fa640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd58807abf0 0x7fd58807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.174+0000 7fd59d7fa640 1 -- 172.21.15.46:0/3951510401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd59009db40 con 0x7fd5a0101900 2024-09-10T04:30:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.174+0000 7fd59f7fe640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd58807abf0 0x7fd58807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.174+0000 7fd59f7fe640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd58807abf0 0x7fd58807d0b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0198b50 tx=0x7fd594031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:56.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.179+0000 7fd59d7fa640 1 -- 172.21.15.46:0/3951510401 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd590014030 con 0x7fd5a0101900 2024-09-10T04:30:56.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.376+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd5a0198360 con 0x7fd58807abf0 2024-09-10T04:30:56.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.386+0000 7fd59d7fa640 1 -- 172.21.15.46:0/3951510401 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7fd5a0198360 con 0x7fd58807abf0 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (22m) 95s ago 33m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (25m) 95s ago 34m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (24m) 97s ago 33m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (20m) 95s ago 34m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (20m) 97s ago 33m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (22m) 95s ago 33m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (17m) 95s ago 28m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (17m) 95s ago 28m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:30:56.388 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (16m) 97s ago 28m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (101s) 97s ago 28m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (24m) 95s ago 35m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (23m) 97s ago 33m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (21m) 95s ago 35m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (21m) 97s ago 32m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (23m) 95s ago 33m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (23m) 97s ago 33m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (20m) 95s ago 31m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (19m) 95s ago 31m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (19m) 95s ago 30m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (18m) 97s ago 30m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (18m) 97s ago 29m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (17m) 97s ago 29m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:30:56.389 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (22m) 95s ago 33m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:30:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd58807abf0 msgr2=0x7fd58807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd58807abf0 0x7fd58807d0b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0198b50 tx=0x7fd594031420 comp rx=0 tx=0).stop 2024-09-10T04:30:56.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 msgr2=0x7fd5a019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a019d680 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c8f0 tx=0x7fd59000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 shutdown_connections 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd58807abf0 0x7fd58807d0b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5a0107900 0x7fd5a019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 --2- 172.21.15.46:0/3951510401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5a0101900 0x7fd5a019d680 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 >> 172.21.15.46:0/3951510401 conn(0x7fd5a00fd600 msgr2=0x7fd5a00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 shutdown_connections 2024-09-10T04:30:56.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.388+0000 7fd5a683a640 1 -- 172.21.15.46:0/3951510401 wait complete. 2024-09-10T04:30:56.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.471+0000 7f42d3924640 1 -- 172.21.15.46:0/1382568360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc1033e0 msgr2=0x7f42cc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.471+0000 7f42d3924640 1 --2- 172.21.15.46:0/1382568360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc1033e0 0x7f42cc10b8d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009880 tx=0x7f42c002a530 comp rx=0 tx=0).stop 2024-09-10T04:30:56.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.471+0000 7f42d3924640 1 -- 172.21.15.46:0/1382568360 shutdown_connections 2024-09-10T04:30:56.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.471+0000 7f42d3924640 1 --2- 172.21.15.46:0/1382568360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc1033e0 0x7f42cc10b8d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.471+0000 7f42d3924640 1 --2- 172.21.15.46:0/1382568360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42cc102ac0 0x7f42cc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.471+0000 7f42d3924640 1 -- 172.21.15.46:0/1382568360 >> 172.21.15.46:0/1382568360 conn(0x7f42cc0fc520 msgr2=0x7f42cc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:56.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.472+0000 7f42d3924640 1 -- 172.21.15.46:0/1382568360 shutdown_connections 2024-09-10T04:30:56.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.472+0000 7f42d3924640 1 -- 172.21.15.46:0/1382568360 wait complete. 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.472+0000 7f42d3924640 1 Processor -- start 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.472+0000 7f42d3924640 1 -- start start 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.472+0000 7f42d3924640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc102ac0 0x7f42cc19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d3924640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42cc1033e0 0x7f42cc19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d3924640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc10edd0 con 0x7f42cc102ac0 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d3924640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc10ec50 con 0x7f42cc1033e0 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d16c0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc102ac0 0x7f42cc19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d16c0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc102ac0 0x7f42cc19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55858/0 (socket says 172.21.15.46:55858) 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d16c0640 1 -- 172.21.15.46:0/3871483842 learned_addr learned my addr 172.21.15.46:0/3871483842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d0ebf640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42cc1033e0 0x7f42cc19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d16c0640 1 -- 172.21.15.46:0/3871483842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42cc1033e0 msgr2=0x7f42cc19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d16c0640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42cc1033e0 0x7f42cc19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d16c0640 1 -- 172.21.15.46:0/3871483842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc1999f0 con 0x7f42cc102ac0 2024-09-10T04:30:56.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42d16c0640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc102ac0 0x7f42cc19f830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f42c800d8d0 tx=0x7f42c800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:56.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42be7fc640 1 -- 172.21.15.46:0/3871483842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42c8017070 con 0x7f42cc102ac0 2024-09-10T04:30:56.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.473+0000 7f42be7fc640 1 -- 172.21.15.46:0/3871483842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42c8010460 con 0x7f42cc102ac0 2024-09-10T04:30:56.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.474+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42cc199ce0 con 0x7f42cc102ac0 2024-09-10T04:30:56.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.474+0000 7f42be7fc640 1 -- 172.21.15.46:0/3871483842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42c800f5f0 con 0x7f42cc102ac0 2024-09-10T04:30:56.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.474+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42cc19a240 con 0x7f42cc102ac0 2024-09-10T04:30:56.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.475+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42cc06aa90 con 0x7f42cc102ac0 2024-09-10T04:30:56.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.475+0000 7f42be7fc640 1 -- 172.21.15.46:0/3871483842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f42c8009a30 con 0x7f42cc102ac0 2024-09-10T04:30:56.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.476+0000 7f42be7fc640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42b807abf0 0x7f42b807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.476+0000 7f42be7fc640 1 -- 172.21.15.46:0/3871483842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f42c809dab0 con 0x7f42cc102ac0 2024-09-10T04:30:56.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.476+0000 7f42d0ebf640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42b807abf0 0x7f42b807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.477+0000 7f42d0ebf640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42b807abf0 0x7f42b807d0b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f42cc19b1e0 tx=0x7f42c0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.481+0000 7f42be7fc640 1 -- 172.21.15.46:0/3871483842 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f42c80691f0 con 0x7f42cc102ac0 2024-09-10T04:30:56.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.743+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f42cc10b370 con 0x7f42cc102ac0 2024-09-10T04:30:56.745 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:56 smithi046 ceph-mon[109506]: pgmap v759: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:56.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.743+0000 7f42be7fc640 1 -- 172.21.15.46:0/3871483842 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f42c806e0a0 con 0x7f42cc102ac0 2024-09-10T04:30:56.745 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:30:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:30:56.747 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:30:56.747 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:30:56.747 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:30:56.747 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:30:56.747 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42b807abf0 msgr2=0x7f42b807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42b807abf0 0x7f42b807d0b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f42cc19b1e0 tx=0x7f42c0031400 comp rx=0 tx=0).stop 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc102ac0 msgr2=0x7f42cc19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc102ac0 0x7f42cc19f830 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f42c800d8d0 tx=0x7f42c800dd90 comp rx=0 tx=0).stop 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 shutdown_connections 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42b807abf0 0x7f42b807d0b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42cc1033e0 0x7f42cc19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 --2- 172.21.15.46:0/3871483842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42cc102ac0 0x7f42cc19f830 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 >> 172.21.15.46:0/3871483842 conn(0x7f42cc0fc520 msgr2=0x7f42cc100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:56.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 shutdown_connections 2024-09-10T04:30:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.745+0000 7f42d3924640 1 -- 172.21.15.46:0/3871483842 wait complete. 2024-09-10T04:30:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 -- 172.21.15.46:0/4236333337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41067f0 msgr2=0x7efee4106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 --2- 172.21.15.46:0/4236333337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41067f0 0x7efee4106bd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efed4009880 tx=0x7efed402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 -- 172.21.15.46:0/4236333337 shutdown_connections 2024-09-10T04:30:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 --2- 172.21.15.46:0/4236333337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efee41007f0 0x7efee4100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 --2- 172.21.15.46:0/4236333337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41067f0 0x7efee4106bd0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 -- 172.21.15.46:0/4236333337 >> 172.21.15.46:0/4236333337 conn(0x7efee40fc550 msgr2=0x7efee40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 -- 172.21.15.46:0/4236333337 shutdown_connections 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 -- 172.21.15.46:0/4236333337 wait complete. 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 Processor -- start 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.830+0000 7efeea9d2640 1 -- start start 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efeea9d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41007f0 0x7efee4107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efeea9d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efee41067f0 0x7efee4107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efeea9d2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efee410dca0 con 0x7efee41007f0 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efeea9d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efee410db20 con 0x7efee41067f0 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efee3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41007f0 0x7efee4107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efee3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41007f0 0x7efee4107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55872/0 (socket says 172.21.15.46:55872) 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efee3fff640 1 -- 172.21.15.46:0/2162567548 learned_addr learned my addr 172.21.15.46:0/2162567548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efee37fe640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efee41067f0 0x7efee4107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efee3fff640 1 -- 172.21.15.46:0/2162567548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efee41067f0 msgr2=0x7efee4107c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efee3fff640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efee41067f0 0x7efee4107c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.831+0000 7efee3fff640 1 -- 172.21.15.46:0/2162567548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee41081a0 con 0x7efee41007f0 2024-09-10T04:30:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.832+0000 7efee3fff640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41007f0 0x7efee4107720 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efed4031460 tx=0x7efed40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.832+0000 7efee17fa640 1 -- 172.21.15.46:0/2162567548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efed403f070 con 0x7efee41007f0 2024-09-10T04:30:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.832+0000 7efee17fa640 1 -- 172.21.15.46:0/2162567548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efed4004cf0 con 0x7efee41007f0 2024-09-10T04:30:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.832+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee4108430 con 0x7efee41007f0 2024-09-10T04:30:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.832+0000 7efee17fa640 1 -- 172.21.15.46:0/2162567548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efed4035030 con 0x7efee41007f0 2024-09-10T04:30:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.832+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a3cc0 con 0x7efee41007f0 2024-09-10T04:30:56.838 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:56 smithi070 ceph-mon[79774]: pgmap v759: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.833+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee4101f30 con 0x7efee41007f0 2024-09-10T04:30:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.834+0000 7efee17fa640 1 -- 172.21.15.46:0/2162567548 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efed4042070 con 0x7efee41007f0 2024-09-10T04:30:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.834+0000 7efee17fa640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efecc07abf0 0x7efecc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.834+0000 7efee17fa640 1 -- 172.21.15.46:0/2162567548 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efed40be6a0 con 0x7efee41007f0 2024-09-10T04:30:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.834+0000 7efee37fe640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efecc07abf0 0x7efecc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.835+0000 7efee37fe640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efecc07abf0 0x7efecc07d0b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efee41091b0 tx=0x7efed800a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:56.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:56.839+0000 7efee17fa640 1 -- 172.21.15.46:0/2162567548 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efed4089de0 con 0x7efee41007f0 2024-09-10T04:30:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.062+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efee41a3fe0 con 0x7efee41007f0 2024-09-10T04:30:57.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.062+0000 7efee17fa640 1 -- 172.21.15.46:0/2162567548 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7efed408ec90 con 0x7efee41007f0 2024-09-10T04:30:57.064 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:30:57.065 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:30:57.066 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:30:57.067 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:57.068 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:57.068 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:57.068 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:30:57.068 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.065+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efecc07abf0 msgr2=0x7efecc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.065+0000 7efeea9d2640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efecc07abf0 0x7efecc07d0b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efee41091b0 tx=0x7efed800a2d0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.065+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41007f0 msgr2=0x7efee4107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.065+0000 7efeea9d2640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41007f0 0x7efee4107720 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7efed4031460 tx=0x7efed40048f0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.065+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 shutdown_connections 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.065+0000 7efeea9d2640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efecc07abf0 0x7efecc07d0b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.066+0000 7efeea9d2640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efee41067f0 0x7efee4107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.066+0000 7efeea9d2640 1 --2- 172.21.15.46:0/2162567548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efee41007f0 0x7efee4107720 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.066+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 >> 172.21.15.46:0/2162567548 conn(0x7efee40fc550 msgr2=0x7efee4069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.066+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 shutdown_connections 2024-09-10T04:30:57.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.066+0000 7efeea9d2640 1 -- 172.21.15.46:0/2162567548 wait complete. 2024-09-10T04:30:57.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.151+0000 7f874c736640 1 -- 172.21.15.46:0/2597254632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 msgr2=0x7f8744101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.151+0000 7f874c736640 1 --2- 172.21.15.46:0/2597254632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f8744101d60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8738009880 tx=0x7f873802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.151+0000 7f874c736640 1 -- 172.21.15.46:0/2597254632 shutdown_connections 2024-09-10T04:30:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.151+0000 7f874c736640 1 --2- 172.21.15.46:0/2597254632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f8744101d60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.151+0000 7f874c736640 1 --2- 172.21.15.46:0/2597254632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8744107900 0x7f8744107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.151+0000 7f874c736640 1 -- 172.21.15.46:0/2597254632 >> 172.21.15.46:0/2597254632 conn(0x7f87440fd600 msgr2=0x7f87440ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 -- 172.21.15.46:0/2597254632 shutdown_connections 2024-09-10T04:30:57.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 -- 172.21.15.46:0/2597254632 wait complete. 2024-09-10T04:30:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 Processor -- start 2024-09-10T04:30:57.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 -- start start 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f87440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8744107900 0x7f8744074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f874410edb0 con 0x7f8744101900 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.152+0000 7f874c736640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f874410ec30 con 0x7f8744107900 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874a4d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f87440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874a4d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f87440744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55886/0 (socket says 172.21.15.46:55886) 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874a4d2640 1 -- 172.21.15.46:0/1505005126 learned_addr learned my addr 172.21.15.46:0/1505005126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f8749cd1640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8744107900 0x7f8744074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874a4d2640 1 -- 172.21.15.46:0/1505005126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8744107900 msgr2=0x7f8744074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874a4d2640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8744107900 0x7f8744074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874a4d2640 1 -- 172.21.15.46:0/1505005126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8744074f70 con 0x7f8744101900 2024-09-10T04:30:57.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874a4d2640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f87440744f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f873400db80 tx=0x7f873400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f87437fe640 1 -- 172.21.15.46:0/1505005126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8734017070 con 0x7f8744101900 2024-09-10T04:30:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f87437fe640 1 -- 172.21.15.46:0/1505005126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8734003e30 con 0x7f8744101900 2024-09-10T04:30:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8744075260 con 0x7f8744101900 2024-09-10T04:30:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f87437fe640 1 -- 172.21.15.46:0/1505005126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8734010460 con 0x7f8744101900 2024-09-10T04:30:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.153+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87441a3e50 con 0x7f8744101900 2024-09-10T04:30:57.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.155+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8744103040 con 0x7f8744101900 2024-09-10T04:30:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.155+0000 7f87437fe640 1 -- 172.21.15.46:0/1505005126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f873400dc70 con 0x7f8744101900 2024-09-10T04:30:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.156+0000 7f87437fe640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f872c07abf0 0x7f872c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.156+0000 7f87437fe640 1 -- 172.21.15.46:0/1505005126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f873409e130 con 0x7f8744101900 2024-09-10T04:30:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.156+0000 7f8749cd1640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f872c07abf0 0x7f872c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:57.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.157+0000 7f8749cd1640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f872c07abf0 0x7f872c07d0b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8744075f80 tx=0x7f8738031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:57.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.161+0000 7f87437fe640 1 -- 172.21.15.46:0/1505005126 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f87340697d0 con 0x7f8744101900 2024-09-10T04:30:57.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.366+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8744109f00 con 0x7f872c07abf0 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.367+0000 7f87437fe640 1 -- 172.21.15.46:0/1505005126 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8738004260 con 0x7f872c07abf0 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:30:57.369 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:30:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:30:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:30:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:30:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:30:57.370 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:30:57.370 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:30:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f872c07abf0 msgr2=0x7f872c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f872c07abf0 0x7f872c07d0b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8744075f80 tx=0x7f8738031400 comp rx=0 tx=0).stop 2024-09-10T04:30:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 msgr2=0x7f87440744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f87440744f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f873400db80 tx=0x7f873400b630 comp rx=0 tx=0).stop 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 shutdown_connections 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f872c07abf0 0x7f872c07d0b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8744107900 0x7f8744074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 --2- 172.21.15.46:0/1505005126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8744101900 0x7f87440744f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 >> 172.21.15.46:0/1505005126 conn(0x7f87440fd600 msgr2=0x7f87440fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 shutdown_connections 2024-09-10T04:30:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.369+0000 7f874c736640 1 -- 172.21.15.46:0/1505005126 wait complete. 2024-09-10T04:30:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 -- 172.21.15.46:0/1187161742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a881033d0 msgr2=0x7f4a8810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 --2- 172.21.15.46:0/1187161742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a881033d0 0x7f4a8810b8c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009880 tx=0x7f4a7002a530 comp rx=0 tx=0).stop 2024-09-10T04:30:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 -- 172.21.15.46:0/1187161742 shutdown_connections 2024-09-10T04:30:57.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 --2- 172.21.15.46:0/1187161742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a881033d0 0x7f4a8810b8c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 --2- 172.21.15.46:0/1187161742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a88102ab0 0x7f4a88102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 -- 172.21.15.46:0/1187161742 >> 172.21.15.46:0/1187161742 conn(0x7f4a880fc550 msgr2=0x7f4a880fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 -- 172.21.15.46:0/1187161742 shutdown_connections 2024-09-10T04:30:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 -- 172.21.15.46:0/1187161742 wait complete. 2024-09-10T04:30:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 Processor -- start 2024-09-10T04:30:57.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.454+0000 7f4a8df48640 1 -- start start 2024-09-10T04:30:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a8df48640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a88102ab0 0x7f4a88100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:57.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a8df48640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a881033d0 0x7f4a88100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a8df48640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8810ede0 con 0x7f4a88102ab0 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a8df48640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8810ec60 con 0x7f4a881033d0 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a88102ab0 0x7f4a88100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a877fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a88102ab0 0x7f4a88100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55908/0 (socket says 172.21.15.46:55908) 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a877fe640 1 -- 172.21.15.46:0/536588243 learned_addr learned my addr 172.21.15.46:0/536588243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a7effd640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a881033d0 0x7f4a88100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a877fe640 1 -- 172.21.15.46:0/536588243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a881033d0 msgr2=0x7f4a88100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a877fe640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a881033d0 0x7f4a88100c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a877fe640 1 -- 172.21.15.46:0/536588243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a88101190 con 0x7f4a88102ab0 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a877fe640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a88102ab0 0x7f4a88100710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800e970 tx=0x7f4a7800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.455+0000 7f4a857fa640 1 -- 172.21.15.46:0/536588243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a7800cac0 con 0x7f4a88102ab0 2024-09-10T04:30:57.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.456+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a88101480 con 0x7f4a88102ab0 2024-09-10T04:30:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.456+0000 7f4a857fa640 1 -- 172.21.15.46:0/536588243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a780054e0 con 0x7f4a88102ab0 2024-09-10T04:30:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.456+0000 7f4a857fa640 1 -- 172.21.15.46:0/536588243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a78010430 con 0x7f4a88102ab0 2024-09-10T04:30:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.456+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a881a3e90 con 0x7f4a88102ab0 2024-09-10T04:30:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.457+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8806a580 con 0x7f4a88102ab0 2024-09-10T04:30:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.457+0000 7f4a857fa640 1 -- 172.21.15.46:0/536588243 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4a78004e80 con 0x7f4a88102ab0 2024-09-10T04:30:57.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.458+0000 7f4a857fa640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a7407ac00 0x7f4a7407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:30:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.458+0000 7f4a857fa640 1 -- 172.21.15.46:0/536588243 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4a7809e2d0 con 0x7f4a88102ab0 2024-09-10T04:30:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.458+0000 7f4a7effd640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a7407ac00 0x7f4a7407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:30:57.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.458+0000 7f4a7effd640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a7407ac00 0x7f4a7407d0c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4a881021a0 tx=0x7f4a70031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:30:57.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.462+0000 7f4a857fa640 1 -- 172.21.15.46:0/536588243 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4a78069a10 con 0x7f4a88102ab0 2024-09-10T04:30:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.721+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f4a88102e90 con 0x7f4a88102ab0 2024-09-10T04:30:57.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:57 smithi046 ceph-mon[109506]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:57.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:57 smithi046 ceph-mon[109506]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:57.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:57 smithi046 ceph-mon[109506]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:57.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:57 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3871483842' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:30:57.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:57 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2162567548' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:30:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.721+0000 7f4a857fa640 1 -- 172.21.15.46:0/536588243 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f4a7806e8c0 con 0x7f4a88102ab0 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:30:57.726 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:30:57.727 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:30:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a7407ac00 msgr2=0x7f4a7407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a7407ac00 0x7f4a7407d0c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4a881021a0 tx=0x7f4a70031400 comp rx=0 tx=0).stop 2024-09-10T04:30:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a88102ab0 msgr2=0x7f4a88100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:30:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a88102ab0 0x7f4a88100710 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800e970 tx=0x7f4a7800ed70 comp rx=0 tx=0).stop 2024-09-10T04:30:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 shutdown_connections 2024-09-10T04:30:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a7407ac00 0x7f4a7407d0c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a881033d0 0x7f4a88100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 --2- 172.21.15.46:0/536588243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a88102ab0 0x7f4a88100710 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:30:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 >> 172.21.15.46:0/536588243 conn(0x7f4a880fc550 msgr2=0x7f4a88108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:30:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 shutdown_connections 2024-09-10T04:30:57.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:30:57.723+0000 7f4a8df48640 1 -- 172.21.15.46:0/536588243 wait complete. 2024-09-10T04:30:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:57 smithi070 ceph-mon[79774]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:57 smithi070 ceph-mon[79774]: from='client.35238 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:57 smithi070 ceph-mon[79774]: from='client.35242 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:57 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3871483842' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:30:57.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:57 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2162567548' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:30:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:58 smithi070 ceph-mon[79774]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:58 smithi070 ceph-mon[79774]: pgmap v760: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:30:58 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/536588243' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:30:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:58 smithi046 ceph-mon[109506]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:30:58.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:58 smithi046 ceph-mon[109506]: pgmap v760: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:30:58.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:30:58 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/536588243' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:31:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:00 smithi070 ceph-mon[79774]: pgmap v761: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:00 smithi046 ceph-mon[109506]: pgmap v761: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:02.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:02 smithi046 ceph-mon[109506]: pgmap v762: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:02.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:02 smithi070 ceph-mon[79774]: pgmap v762: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:04 smithi070 ceph-mon[79774]: pgmap v763: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:04.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:04 smithi046 ceph-mon[109506]: pgmap v763: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:06 smithi070 ceph-mon[79774]: pgmap v764: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:06.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:06 smithi046 ceph-mon[109506]: pgmap v764: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:08 smithi046 ceph-mon[109506]: pgmap v765: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:08 smithi070 ceph-mon[79774]: pgmap v765: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:10 smithi070 ceph-mon[79774]: pgmap v766: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:10.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:10 smithi046 ceph-mon[109506]: pgmap v766: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:12 smithi070 ceph-mon[79774]: pgmap v767: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:12 smithi046 ceph-mon[109506]: pgmap v767: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:14 smithi070 ceph-mon[79774]: pgmap v768: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:14 smithi046 ceph-mon[109506]: pgmap v768: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:16 smithi070 ceph-mon[79774]: pgmap v769: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:16 smithi046 ceph-mon[109506]: pgmap v769: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:18 smithi070 ceph-mon[79774]: pgmap v770: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:18 smithi046 ceph-mon[109506]: pgmap v770: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:20 smithi070 ceph-mon[79774]: pgmap v771: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:20.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:20 smithi046 ceph-mon[109506]: pgmap v771: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:22 smithi070 ceph-mon[79774]: pgmap v772: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:22 smithi046 ceph-mon[109506]: pgmap v772: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:24 smithi070 ceph-mon[79774]: pgmap v773: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:24.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:24 smithi046 ceph-mon[109506]: pgmap v773: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:26 smithi070 ceph-mon[79774]: pgmap v774: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:26.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:26 smithi046 ceph-mon[109506]: pgmap v774: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.823+0000 7f3564547640 1 -- 172.21.15.46:0/2628004365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c100fd0 msgr2=0x7f355c101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:27.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.823+0000 7f3564547640 1 --2- 172.21.15.46:0/2628004365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c100fd0 0x7f355c101430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f354c009880 tx=0x7f354c02a530 comp rx=0 tx=0).stop 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.824+0000 7f3564547640 1 -- 172.21.15.46:0/2628004365 shutdown_connections 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.824+0000 7f3564547640 1 --2- 172.21.15.46:0/2628004365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c100fd0 0x7f355c101430 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.824+0000 7f3564547640 1 --2- 172.21.15.46:0/2628004365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f355c1068b0 0x7f355c106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.824+0000 7f3564547640 1 -- 172.21.15.46:0/2628004365 >> 172.21.15.46:0/2628004365 conn(0x7f355c0fbc90 msgr2=0x7f355c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.824+0000 7f3564547640 1 -- 172.21.15.46:0/2628004365 shutdown_connections 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.824+0000 7f3564547640 1 -- 172.21.15.46:0/2628004365 wait complete. 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.824+0000 7f3564547640 1 Processor -- start 2024-09-10T04:31:27.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3564547640 1 -- start start 2024-09-10T04:31:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3564547640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f355c100fd0 0x7f355c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3564547640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c1068b0 0x7f355c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3564547640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f355c10ee80 con 0x7f355c1068b0 2024-09-10T04:31:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3564547640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f355c10ed00 con 0x7f355c100fd0 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3561ae2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c1068b0 0x7f355c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3561ae2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c1068b0 0x7f355c19dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38050/0 (socket says 172.21.15.46:38050) 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3561ae2640 1 -- 172.21.15.46:0/1458061379 learned_addr learned my addr 172.21.15.46:0/1458061379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f35622e3640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f355c100fd0 0x7f355c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3561ae2640 1 -- 172.21.15.46:0/1458061379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f355c100fd0 msgr2=0x7f355c19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3561ae2640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f355c100fd0 0x7f355c19d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.825+0000 7f3561ae2640 1 -- 172.21.15.46:0/1458061379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c197870 con 0x7f355c1068b0 2024-09-10T04:31:27.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.826+0000 7f3561ae2640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c1068b0 0x7f355c19dbf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f354c031460 tx=0x7f354c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.826+0000 7f35537fe640 1 -- 172.21.15.46:0/1458061379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f354c03f070 con 0x7f355c1068b0 2024-09-10T04:31:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.826+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c197b00 con 0x7f355c1068b0 2024-09-10T04:31:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.826+0000 7f35537fe640 1 -- 172.21.15.46:0/1458061379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f354c004cf0 con 0x7f355c1068b0 2024-09-10T04:31:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.826+0000 7f35537fe640 1 -- 172.21.15.46:0/1458061379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f354c035030 con 0x7f355c1068b0 2024-09-10T04:31:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.826+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c198030 con 0x7f355c1068b0 2024-09-10T04:31:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.827+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c101fe0 con 0x7f355c1068b0 2024-09-10T04:31:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.828+0000 7f35537fe640 1 -- 172.21.15.46:0/1458061379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f354c042070 con 0x7f355c1068b0 2024-09-10T04:31:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.828+0000 7f35537fe640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f354807abf0 0x7f354807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.828+0000 7f35537fe640 1 -- 172.21.15.46:0/1458061379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f354c0be6a0 con 0x7f355c1068b0 2024-09-10T04:31:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.828+0000 7f35622e3640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f354807abf0 0x7f354807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:27.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.829+0000 7f35622e3640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f354807abf0 0x7f354807d0b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f35580058c0 tx=0x7f355800a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:27.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:27.833+0000 7f35537fe640 1 -- 172.21.15.46:0/1458061379 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f354c089de0 con 0x7f355c1068b0 2024-09-10T04:31:28.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.037+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f355c198da0 con 0x7f354807abf0 2024-09-10T04:31:28.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.038+0000 7f35537fe640 1 -- 172.21.15.46:0/1458061379 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f355c198da0 con 0x7f354807abf0 2024-09-10T04:31:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.040+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f354807abf0 msgr2=0x7f354807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.040+0000 7f3564547640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f354807abf0 0x7f354807d0b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f35580058c0 tx=0x7f355800a250 comp rx=0 tx=0).stop 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.040+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c1068b0 msgr2=0x7f355c19dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.040+0000 7f3564547640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c1068b0 0x7f355c19dbf0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f354c031460 tx=0x7f354c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.041+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 shutdown_connections 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.041+0000 7f3564547640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f354807abf0 0x7f354807d0b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.041+0000 7f3564547640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f355c1068b0 0x7f355c19dbf0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.041+0000 7f3564547640 1 --2- 172.21.15.46:0/1458061379 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f355c100fd0 0x7f355c19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.041+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 >> 172.21.15.46:0/1458061379 conn(0x7f355c0fbc90 msgr2=0x7f355c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.041+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 shutdown_connections 2024-09-10T04:31:28.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.041+0000 7f3564547640 1 -- 172.21.15.46:0/1458061379 wait complete. 2024-09-10T04:31:28.051 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:31:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.126+0000 7fad5e28f640 1 -- 172.21.15.46:0/1440428248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 msgr2=0x7fad58100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.126+0000 7fad5e28f640 1 --2- 172.21.15.46:0/1440428248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58100710 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fad48009880 tx=0x7fad4802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.126+0000 7fad5e28f640 1 -- 172.21.15.46:0/1440428248 shutdown_connections 2024-09-10T04:31:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.126+0000 7fad5e28f640 1 --2- 172.21.15.46:0/1440428248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58100ce0 0x7fad58104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.127+0000 7fad5e28f640 1 --2- 172.21.15.46:0/1440428248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58100710 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.127+0000 7fad5e28f640 1 -- 172.21.15.46:0/1440428248 >> 172.21.15.46:0/1440428248 conn(0x7fad58077270 msgr2=0x7fad58077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.127+0000 7fad5e28f640 1 -- 172.21.15.46:0/1440428248 shutdown_connections 2024-09-10T04:31:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.127+0000 7fad5e28f640 1 -- 172.21.15.46:0/1440428248 wait complete. 2024-09-10T04:31:28.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.127+0000 7fad5e28f640 1 Processor -- start 2024-09-10T04:31:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.127+0000 7fad5e28f640 1 -- start start 2024-09-10T04:31:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad5e28f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad5e28f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58100ce0 0x7fad58198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad5e28f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad5810d4a0 con 0x7fad58100330 2024-09-10T04:31:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad5e28f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fad5810d320 con 0x7fad58100ce0 2024-09-10T04:31:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad57fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad57fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38058/0 (socket says 172.21.15.46:38058) 2024-09-10T04:31:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad57fff640 1 -- 172.21.15.46:0/3257671183 learned_addr learned my addr 172.21.15.46:0/3257671183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:31:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad577fe640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58100ce0 0x7fad58198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad57fff640 1 -- 172.21.15.46:0/3257671183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58100ce0 msgr2=0x7fad58198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad57fff640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58100ce0 0x7fad58198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad57fff640 1 -- 172.21.15.46:0/3257671183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad581993c0 con 0x7fad58100330 2024-09-10T04:31:28.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.128+0000 7fad57fff640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58198880 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fad48031460 tx=0x7fad480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.129+0000 7fad557fa640 1 -- 172.21.15.46:0/3257671183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fad4803f070 con 0x7fad58100330 2024-09-10T04:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.129+0000 7fad557fa640 1 -- 172.21.15.46:0/3257671183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad480035f0 con 0x7fad58100330 2024-09-10T04:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.129+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad58199650 con 0x7fad58100330 2024-09-10T04:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.129+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad581a1c00 con 0x7fad58100330 2024-09-10T04:31:28.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.129+0000 7fad557fa640 1 -- 172.21.15.46:0/3257671183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fad48035030 con 0x7fad58100330 2024-09-10T04:31:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.130+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad58101700 con 0x7fad58100330 2024-09-10T04:31:28.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.130+0000 7fad557fa640 1 -- 172.21.15.46:0/3257671183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fad48042070 con 0x7fad58100330 2024-09-10T04:31:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.131+0000 7fad557fa640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fad4007abf0 0x7fad4007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.131+0000 7fad557fa640 1 -- 172.21.15.46:0/3257671183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fad480be6e0 con 0x7fad58100330 2024-09-10T04:31:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.131+0000 7fad577fe640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fad4007abf0 0x7fad4007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.131+0000 7fad577fe640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fad4007abf0 0x7fad4007d0b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005fd0 tx=0x7fad4c005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.135+0000 7fad557fa640 1 -- 172.21.15.46:0/3257671183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fad48089d80 con 0x7fad58100330 2024-09-10T04:31:28.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.343+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fad58104310 con 0x7fad4007abf0 2024-09-10T04:31:28.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.344+0000 7fad557fa640 1 -- 172.21.15.46:0/3257671183 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fad58104310 con 0x7fad4007abf0 2024-09-10T04:31:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fad4007abf0 msgr2=0x7fad4007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fad4007abf0 0x7fad4007d0b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005fd0 tx=0x7fad4c005850 comp rx=0 tx=0).stop 2024-09-10T04:31:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 msgr2=0x7fad58198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58198880 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fad48031460 tx=0x7fad480045e0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 shutdown_connections 2024-09-10T04:31:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fad4007abf0 0x7fad4007d0b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad58100ce0 0x7fad58198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 --2- 172.21.15.46:0/3257671183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad58100330 0x7fad58198880 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 >> 172.21.15.46:0/3257671183 conn(0x7fad58077270 msgr2=0x7fad58103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 shutdown_connections 2024-09-10T04:31:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.347+0000 7fad5e28f640 1 -- 172.21.15.46:0/3257671183 wait complete. 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 -- 172.21.15.46:0/2060609308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107900 msgr2=0x7fbed0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 --2- 172.21.15.46:0/2060609308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107900 0x7fbed0107ce0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009880 tx=0x7fbec002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 -- 172.21.15.46:0/2060609308 shutdown_connections 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 --2- 172.21.15.46:0/2060609308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbed0101900 0x7fbed0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 --2- 172.21.15.46:0/2060609308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107900 0x7fbed0107ce0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 -- 172.21.15.46:0/2060609308 >> 172.21.15.46:0/2060609308 conn(0x7fbed00fd600 msgr2=0x7fbed00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 -- 172.21.15.46:0/2060609308 shutdown_connections 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.431+0000 7fbed5586640 1 -- 172.21.15.46:0/2060609308 wait complete. 2024-09-10T04:31:28.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.432+0000 7fbed5586640 1 Processor -- start 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.432+0000 7fbed5586640 1 -- start start 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.432+0000 7fbed5586640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0101900 0x7fbed019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.432+0000 7fbed5586640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbed0107900 0x7fbed019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.432+0000 7fbed5586640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010edd0 con 0x7fbed0101900 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.432+0000 7fbed5586640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010ec50 con 0x7fbed0107900 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0101900 0x7fbed019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeceffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0101900 0x7fbed019f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38076/0 (socket says 172.21.15.46:38076) 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeceffd640 1 -- 172.21.15.46:0/1740879141 learned_addr learned my addr 172.21.15.46:0/1740879141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbece7fc640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbed0107900 0x7fbed019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeceffd640 1 -- 172.21.15.46:0/1740879141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbed0107900 msgr2=0x7fbed019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeceffd640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbed0107900 0x7fbed019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeceffd640 1 -- 172.21.15.46:0/1740879141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed01999d0 con 0x7fbed0101900 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeceffd640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0101900 0x7fbed019f810 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbec0031460 tx=0x7fbec0004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeb7fff640 1 -- 172.21.15.46:0/1740879141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbec003f070 con 0x7fbed0101900 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbeb7fff640 1 -- 172.21.15.46:0/1740879141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbec00035f0 con 0x7fbed0101900 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.433+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed0199c60 con 0x7fbed0101900 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.434+0000 7fbeb7fff640 1 -- 172.21.15.46:0/1740879141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbec0035030 con 0x7fbed0101900 2024-09-10T04:31:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.434+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed019a190 con 0x7fbed0101900 2024-09-10T04:31:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.435+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed0103040 con 0x7fbed0101900 2024-09-10T04:31:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.435+0000 7fbeb7fff640 1 -- 172.21.15.46:0/1740879141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbec0042070 con 0x7fbed0101900 2024-09-10T04:31:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.436+0000 7fbeb7fff640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbeb807abf0 0x7fbeb807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.436+0000 7fbeb7fff640 1 -- 172.21.15.46:0/1740879141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fbec00be5b0 con 0x7fbed0101900 2024-09-10T04:31:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.436+0000 7fbece7fc640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbeb807abf0 0x7fbeb807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.437+0000 7fbece7fc640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbeb807abf0 0x7fbeb807d0b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbed00757e0 tx=0x7fbec4005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.440+0000 7fbeb7fff640 1 -- 172.21.15.46:0/1740879141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbec0089cf0 con 0x7fbed0101900 2024-09-10T04:31:28.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.634+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fbed019afa0 con 0x7fbeb807abf0 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.644+0000 7fbeb7fff640 1 -- 172.21.15.46:0/1740879141 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fbed019afa0 con 0x7fbeb807abf0 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (23m) 2m ago 34m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (25m) 2m ago 34m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (25m) 2m ago 33m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (21m) 2m ago 34m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (21m) 2m ago 33m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (22m) 2m ago 34m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (17m) 2m ago 28m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (17m) 2m ago 29m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:31:28.645 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (17m) 2m ago 29m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (2m) 2m ago 28m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (25m) 2m ago 35m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (24m) 2m ago 33m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (21m) 2m ago 35m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (21m) 2m ago 33m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (23m) 2m ago 34m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (23m) 2m ago 33m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (21m) 2m ago 32m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (20m) 2m ago 31m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (19m) 2m ago 31m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (19m) 2m ago 30m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (18m) 2m ago 30m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (18m) 2m ago 30m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:31:28.646 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (23m) 2m ago 34m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:31:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbeb807abf0 msgr2=0x7fbeb807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbeb807abf0 0x7fbeb807d0b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbed00757e0 tx=0x7fbec4005f90 comp rx=0 tx=0).stop 2024-09-10T04:31:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0101900 msgr2=0x7fbed019f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0101900 0x7fbed019f810 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbec0031460 tx=0x7fbec0004530 comp rx=0 tx=0).stop 2024-09-10T04:31:28.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 shutdown_connections 2024-09-10T04:31:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbeb807abf0 0x7fbeb807d0b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbed0107900 0x7fbed019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 --2- 172.21.15.46:0/1740879141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0101900 0x7fbed019f810 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 >> 172.21.15.46:0/1740879141 conn(0x7fbed00fd600 msgr2=0x7fbed0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 shutdown_connections 2024-09-10T04:31:28.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.646+0000 7fbed5586640 1 -- 172.21.15.46:0/1740879141 wait complete. 2024-09-10T04:31:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 -- 172.21.15.46:0/3231523155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4103470 msgr2=0x7f32b410b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/3231523155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4103470 0x7f32b410b960 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009880 tx=0x7f32a802a530 comp rx=0 tx=0).stop 2024-09-10T04:31:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 -- 172.21.15.46:0/3231523155 shutdown_connections 2024-09-10T04:31:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/3231523155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4103470 0x7f32b410b960 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/3231523155 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b4102b50 0x7f32b4102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 -- 172.21.15.46:0/3231523155 >> 172.21.15.46:0/3231523155 conn(0x7f32b40fc550 msgr2=0x7f32b40fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:28.731 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:28 smithi046 ceph-mon[109506]: pgmap v775: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 -- 172.21.15.46:0/3231523155 shutdown_connections 2024-09-10T04:31:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 -- 172.21.15.46:0/3231523155 wait complete. 2024-09-10T04:31:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 Processor -- start 2024-09-10T04:31:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.730+0000 7f32bb1e1640 1 -- start start 2024-09-10T04:31:28.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32bb1e1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4102b50 0x7f32b41007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32bb1e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b4103470 0x7f32b4100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32bb1e1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410ede0 con 0x7f32b4102b50 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32bb1e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410ec60 con 0x7f32b4103470 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b8f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4102b50 0x7f32b41007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b8f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4102b50 0x7f32b41007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38104/0 (socket says 172.21.15.46:38104) 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b8f7d640 1 -- 172.21.15.46:0/2584012659 learned_addr learned my addr 172.21.15.46:0/2584012659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b3fff640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b4103470 0x7f32b4100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b8f7d640 1 -- 172.21.15.46:0/2584012659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b4103470 msgr2=0x7f32b4100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b8f7d640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b4103470 0x7f32b4100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b8f7d640 1 -- 172.21.15.46:0/2584012659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b4102530 con 0x7f32b4102b50 2024-09-10T04:31:28.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.731+0000 7f32b8f7d640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4102b50 0x7f32b41007b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e970 tx=0x7f32a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.732+0000 7f32b1ffb640 1 -- 172.21.15.46:0/2584012659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f32a400cac0 con 0x7f32b4102b50 2024-09-10T04:31:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.732+0000 7f32b1ffb640 1 -- 172.21.15.46:0/2584012659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f32a40054e0 con 0x7f32b4102b50 2024-09-10T04:31:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.732+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b4101230 con 0x7f32b4102b50 2024-09-10T04:31:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.732+0000 7f32b1ffb640 1 -- 172.21.15.46:0/2584012659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f32a4010430 con 0x7f32b4102b50 2024-09-10T04:31:28.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.732+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b41a8320 con 0x7f32b4102b50 2024-09-10T04:31:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.733+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b406a4e0 con 0x7f32b4102b50 2024-09-10T04:31:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.734+0000 7f32b1ffb640 1 -- 172.21.15.46:0/2584012659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f32a4004f00 con 0x7f32b4102b50 2024-09-10T04:31:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.734+0000 7f32b1ffb640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f329c07ac40 0x7f329c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.734+0000 7f32b1ffb640 1 -- 172.21.15.46:0/2584012659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f32a409e330 con 0x7f32b4102b50 2024-09-10T04:31:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.734+0000 7f32b3fff640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f329c07ac40 0x7f329c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:28.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.735+0000 7f32b3fff640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f329c07ac40 0x7f329c07d100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000c00 tx=0x7f32a8004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:28.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:28.739+0000 7f32b1ffb640 1 -- 172.21.15.46:0/2584012659 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f32a4069a90 con 0x7f32b4102b50 2024-09-10T04:31:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:28 smithi070 ceph-mon[79774]: pgmap v775: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:29.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.001+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f32b4105fe0 con 0x7f32b4102b50 2024-09-10T04:31:29.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.001+0000 7f32b1ffb640 1 -- 172.21.15.46:0/2584012659 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f32a406e940 con 0x7f32b4102b50 2024-09-10T04:31:29.002 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:31:29.002 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:31:29.003 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:31:29.004 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:31:29.004 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:31:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.003+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f329c07ac40 msgr2=0x7f329c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.003+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f329c07ac40 0x7f329c07d100 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f32a8000c00 tx=0x7f32a8004f30 comp rx=0 tx=0).stop 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.003+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4102b50 msgr2=0x7f32b41007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.003+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4102b50 0x7f32b41007b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f32a400e970 tx=0x7f32a400ed70 comp rx=0 tx=0).stop 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.004+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 shutdown_connections 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.004+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f329c07ac40 0x7f329c07d100 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.004+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f32b4103470 0x7f32b4100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.004+0000 7f32bb1e1640 1 --2- 172.21.15.46:0/2584012659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32b4102b50 0x7f32b41007b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.004+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 >> 172.21.15.46:0/2584012659 conn(0x7f32b40fc550 msgr2=0x7f32b4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.004+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 shutdown_connections 2024-09-10T04:31:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.004+0000 7f32bb1e1640 1 -- 172.21.15.46:0/2584012659 wait complete. 2024-09-10T04:31:29.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 -- 172.21.15.46:0/186139717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 msgr2=0x7f9578100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 --2- 172.21.15.46:0/186139717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f9578100c50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9564009880 tx=0x7f956402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 -- 172.21.15.46:0/186139717 shutdown_connections 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 --2- 172.21.15.46:0/186139717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f9578100c50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 --2- 172.21.15.46:0/186139717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95781067f0 0x7f9578106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 -- 172.21.15.46:0/186139717 >> 172.21.15.46:0/186139717 conn(0x7f95780fc550 msgr2=0x7f95780fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 -- 172.21.15.46:0/186139717 shutdown_connections 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 -- 172.21.15.46:0/186139717 wait complete. 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 Processor -- start 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.088+0000 7f957ee34640 1 -- start start 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957ee34640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f95780fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957ee34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95781067f0 0x7f95780fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957ee34640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f957810dca0 con 0x7f95781007f0 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957ee34640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f957810db20 con 0x7f95781067f0 2024-09-10T04:31:29.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957cbd0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f95780fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957cbd0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f95780fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42542/0 (socket says 172.21.15.46:42542) 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957cbd0640 1 -- 172.21.15.46:0/126810410 learned_addr learned my addr 172.21.15.46:0/126810410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f9577fff640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95781067f0 0x7f95780fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957cbd0640 1 -- 172.21.15.46:0/126810410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95781067f0 msgr2=0x7f95780fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957cbd0640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95781067f0 0x7f95780fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957cbd0640 1 -- 172.21.15.46:0/126810410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95780fd700 con 0x7f95781007f0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.089+0000 7f957cbd0640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f95780fcc80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c810 tx=0x7f956c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.090+0000 7f9575ffb640 1 -- 172.21.15.46:0/126810410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f956c004020 con 0x7f95781007f0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.090+0000 7f9575ffb640 1 -- 172.21.15.46:0/126810410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f956c0041c0 con 0x7f95781007f0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.090+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95780fd9f0 con 0x7f95781007f0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.090+0000 7f9575ffb640 1 -- 172.21.15.46:0/126810410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f956c0047a0 con 0x7f95781007f0 2024-09-10T04:31:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.090+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95781a3e00 con 0x7f95781007f0 2024-09-10T04:31:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.091+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9578101f30 con 0x7f95781007f0 2024-09-10T04:31:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.091+0000 7f9575ffb640 1 -- 172.21.15.46:0/126810410 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f956c0079b0 con 0x7f95781007f0 2024-09-10T04:31:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.092+0000 7f9575ffb640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f956807ac40 0x7f956807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.092+0000 7f9575ffb640 1 -- 172.21.15.46:0/126810410 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f956c09dae0 con 0x7f95781007f0 2024-09-10T04:31:29.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.092+0000 7f9577fff640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f956807ac40 0x7f956807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.092+0000 7f9577fff640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f956807ac40 0x7f956807d100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f95780fe710 tx=0x7f9564031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:29.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.096+0000 7f9575ffb640 1 -- 172.21.15.46:0/126810410 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f956c014030 con 0x7f95781007f0 2024-09-10T04:31:29.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.333+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f9578109d90 con 0x7f95781007f0 2024-09-10T04:31:29.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.333+0000 7f9575ffb640 1 -- 172.21.15.46:0/126810410 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f956c0692a0 con 0x7f95781007f0 2024-09-10T04:31:29.334 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:31:29.334 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:31:29.334 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:31:29.334 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:31:29.335 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:31:29.336 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:31:29.337 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:31:29.338 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:31:29.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f956807ac40 msgr2=0x7f956807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f956807ac40 0x7f956807d100 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f95780fe710 tx=0x7f9564031420 comp rx=0 tx=0).stop 2024-09-10T04:31:29.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 msgr2=0x7f95780fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f95780fcc80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c810 tx=0x7f956c00ccd0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 shutdown_connections 2024-09-10T04:31:29.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f956807ac40 0x7f956807d100 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95781067f0 0x7f95780fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 --2- 172.21.15.46:0/126810410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95781007f0 0x7f95780fcc80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 >> 172.21.15.46:0/126810410 conn(0x7f95780fc550 msgr2=0x7f9578069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:29.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 shutdown_connections 2024-09-10T04:31:29.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.335+0000 7f957ee34640 1 -- 172.21.15.46:0/126810410 wait complete. 2024-09-10T04:31:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 -- 172.21.15.46:0/1298428576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c106800 msgr2=0x7fb71c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 --2- 172.21.15.46:0/1298428576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c106800 0x7fb71c106be0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009880 tx=0x7fb70c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 -- 172.21.15.46:0/1298428576 shutdown_connections 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 --2- 172.21.15.46:0/1298428576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb71c100800 0x7fb71c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 --2- 172.21.15.46:0/1298428576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c106800 0x7fb71c106be0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 -- 172.21.15.46:0/1298428576 >> 172.21.15.46:0/1298428576 conn(0x7fb71c0fc520 msgr2=0x7fb71c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 -- 172.21.15.46:0/1298428576 shutdown_connections 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.421+0000 7fb72241d640 1 -- 172.21.15.46:0/1298428576 wait complete. 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb72241d640 1 Processor -- start 2024-09-10T04:31:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb72241d640 1 -- start start 2024-09-10T04:31:29.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb72241d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c100800 0x7fb71c107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb72241d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb71c106800 0x7fb71c107c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb72241d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10dcb0 con 0x7fb71c100800 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb72241d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c10db30 con 0x7fb71c106800 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c100800 0x7fb71c107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb71bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c100800 0x7fb71c107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42562/0 (socket says 172.21.15.46:42562) 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb71bfff640 1 -- 172.21.15.46:0/438214749 learned_addr learned my addr 172.21.15.46:0/438214749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.422+0000 7fb713fff640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb71c106800 0x7fb71c107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb71bfff640 1 -- 172.21.15.46:0/438214749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb71c106800 msgr2=0x7fb71c107c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb71bfff640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb71c106800 0x7fb71c107c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb71bfff640 1 -- 172.21.15.46:0/438214749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1081b0 con 0x7fb71c100800 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb71bfff640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c100800 0x7fb71c107730 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb70c031460 tx=0x7fb70c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb719ffb640 1 -- 172.21.15.46:0/438214749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb70c03f070 con 0x7fb71c100800 2024-09-10T04:31:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb719ffb640 1 -- 172.21.15.46:0/438214749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb70c004a90 con 0x7fb71c100800 2024-09-10T04:31:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb719ffb640 1 -- 172.21.15.46:0/438214749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb70c035030 con 0x7fb71c100800 2024-09-10T04:31:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c108440 con 0x7fb71c100800 2024-09-10T04:31:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.423+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c1a3d50 con 0x7fb71c100800 2024-09-10T04:31:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.424+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c101f40 con 0x7fb71c100800 2024-09-10T04:31:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.425+0000 7fb719ffb640 1 -- 172.21.15.46:0/438214749 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb70c042070 con 0x7fb71c100800 2024-09-10T04:31:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.425+0000 7fb719ffb640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb70807ac00 0x7fb70807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.426+0000 7fb719ffb640 1 -- 172.21.15.46:0/438214749 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb70c0be6e0 con 0x7fb71c100800 2024-09-10T04:31:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.426+0000 7fb713fff640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb70807ac00 0x7fb70807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.426+0000 7fb713fff640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb70807ac00 0x7fb70807d0c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb71c1091c0 tx=0x7fb704005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:29.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.430+0000 7fb719ffb640 1 -- 172.21.15.46:0/438214749 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb70c089e20 con 0x7fb71c100800 2024-09-10T04:31:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.635+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb71c06a9c0 con 0x7fb70807ac00 2024-09-10T04:31:29.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.636+0000 7fb719ffb640 1 -- 172.21.15.46:0/438214749 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fb71c06a9c0 con 0x7fb70807ac00 2024-09-10T04:31:29.637 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:31:29.637 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:31:29.638 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:31:29.638 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:31:29.638 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:31:29.638 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:31:29.638 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:31:29.638 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:31:29.638 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:31:29.639 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:31:29.639 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:31:29.639 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:31:29.639 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:31:29.639 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:31:29.639 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:31:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.638+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb70807ac00 msgr2=0x7fb70807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.638+0000 7fb72241d640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb70807ac00 0x7fb70807d0c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb71c1091c0 tx=0x7fb704005f90 comp rx=0 tx=0).stop 2024-09-10T04:31:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.638+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c100800 msgr2=0x7fb71c107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.638+0000 7fb72241d640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c100800 0x7fb71c107730 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb70c031460 tx=0x7fb70c004530 comp rx=0 tx=0).stop 2024-09-10T04:31:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.639+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 shutdown_connections 2024-09-10T04:31:29.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.639+0000 7fb72241d640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb70807ac00 0x7fb70807d0c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.639+0000 7fb72241d640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb71c106800 0x7fb71c107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.639+0000 7fb72241d640 1 --2- 172.21.15.46:0/438214749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb71c100800 0x7fb71c107730 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.639+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 >> 172.21.15.46:0/438214749 conn(0x7fb71c0fc520 msgr2=0x7fb71c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.639+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 shutdown_connections 2024-09-10T04:31:29.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.639+0000 7fb72241d640 1 -- 172.21.15.46:0/438214749 wait complete. 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 -- 172.21.15.46:0/653861551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 msgr2=0x7f69541123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 --2- 172.21.15.46:0/653861551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f69541123d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6944009880 tx=0x7f694402a530 comp rx=0 tx=0).stop 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 -- 172.21.15.46:0/653861551 shutdown_connections 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 --2- 172.21.15.46:0/653861551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f69541123d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 --2- 172.21.15.46:0/653861551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6954074590 0x7f6954074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 -- 172.21.15.46:0/653861551 >> 172.21.15.46:0/653861551 conn(0x7f69540fd600 msgr2=0x7f69540ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 -- 172.21.15.46:0/653861551 shutdown_connections 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.723+0000 7f695bbed640 1 -- 172.21.15.46:0/653861551 wait complete. 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f695bbed640 1 Processor -- start 2024-09-10T04:31:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f695bbed640 1 -- start start 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f695bbed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6954074590 0x7f695419e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f695bbed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f695419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f695bbed640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6954115440 con 0x7f6954074eb0 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f695bbed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f69541152c0 con 0x7f6954074590 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f6959188640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f695419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f6959188640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f695419e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42574/0 (socket says 172.21.15.46:42574) 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f6959188640 1 -- 172.21.15.46:0/3688442477 learned_addr learned my addr 172.21.15.46:0/3688442477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f6959989640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6954074590 0x7f695419e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.724+0000 7f6959188640 1 -- 172.21.15.46:0/3688442477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6954074590 msgr2=0x7f695419e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f6959188640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6954074590 0x7f695419e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f6959188640 1 -- 172.21.15.46:0/3688442477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69541a28d0 con 0x7f6954074eb0 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f6959989640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6954074590 0x7f695419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f6959188640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f695419e540 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6944031460 tx=0x7f69440048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:29.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f694affd640 1 -- 172.21.15.46:0/3688442477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f694403f070 con 0x7f6954074eb0 2024-09-10T04:31:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f694affd640 1 -- 172.21.15.46:0/3688442477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6944004cf0 con 0x7f6954074eb0 2024-09-10T04:31:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69541a2b60 con 0x7f6954074eb0 2024-09-10T04:31:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f694affd640 1 -- 172.21.15.46:0/3688442477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6944035030 con 0x7f6954074eb0 2024-09-10T04:31:29.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.725+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69541a3090 con 0x7f6954074eb0 2024-09-10T04:31:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.726+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6954075cd0 con 0x7f6954074eb0 2024-09-10T04:31:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.727+0000 7f694affd640 1 -- 172.21.15.46:0/3688442477 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6944042070 con 0x7f6954074eb0 2024-09-10T04:31:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.727+0000 7f694affd640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f694007abf0 0x7f694007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:31:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.727+0000 7f694affd640 1 -- 172.21.15.46:0/3688442477 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f69440be850 con 0x7f6954074eb0 2024-09-10T04:31:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.727+0000 7f6959989640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f694007abf0 0x7f694007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:31:29.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.728+0000 7f6959989640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f694007abf0 0x7f694007d0b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6950009930 tx=0x7f6950008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:31:29.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.732+0000 7f694affd640 1 -- 172.21.15.46:0/3688442477 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6944089ef0 con 0x7f6954074eb0 2024-09-10T04:31:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:29 smithi070 ceph-mon[79774]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:29 smithi070 ceph-mon[79774]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:29 smithi070 ceph-mon[79774]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:29 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2584012659' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:31:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:29 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/126810410' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:31:29.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:29 smithi046 ceph-mon[109506]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:29.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:29 smithi046 ceph-mon[109506]: from='client.35266 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:29 smithi046 ceph-mon[109506]: from='client.35270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:29 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2584012659' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:31:29.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:29 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/126810410' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:31:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.989+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f695410fac0 con 0x7f6954074eb0 2024-09-10T04:31:29.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.989+0000 7f694affd640 1 -- 172.21.15.46:0/3688442477 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f694408eda0 con 0x7f6954074eb0 2024-09-10T04:31:29.990 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:31:29.990 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:31:29.990 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:31:29.990 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:31:29.990 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:31:29.991 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:31:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:31:29.992 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:31:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:31:29.992 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:31:29.992 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:31:29.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f694007abf0 msgr2=0x7f694007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f694007abf0 0x7f694007d0b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6950009930 tx=0x7f6950008040 comp rx=0 tx=0).stop 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 msgr2=0x7f695419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f695419e540 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6944031460 tx=0x7f69440048f0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 shutdown_connections 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f694007abf0 0x7f694007d0b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6954074eb0 0x7f695419e540 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 --2- 172.21.15.46:0/3688442477 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6954074590 0x7f695419e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 >> 172.21.15.46:0/3688442477 conn(0x7f69540fd600 msgr2=0x7f6954104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 shutdown_connections 2024-09-10T04:31:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:31:29.991+0000 7f695bbed640 1 -- 172.21.15.46:0/3688442477 wait complete. 2024-09-10T04:31:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:30 smithi070 ceph-mon[79774]: pgmap v776: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:30 smithi070 ceph-mon[79774]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:30 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3688442477' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:31:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:30 smithi046 ceph-mon[109506]: pgmap v776: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:30 smithi046 ceph-mon[109506]: from='client.35282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:31:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:30 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3688442477' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:31:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:32 smithi070 ceph-mon[79774]: pgmap v777: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:32.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:32 smithi046 ceph-mon[109506]: pgmap v777: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:34 smithi070 ceph-mon[79774]: pgmap v778: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:34 smithi046 ceph-mon[109506]: pgmap v778: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:36 smithi070 ceph-mon[79774]: pgmap v779: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:36.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:36 smithi046 ceph-mon[109506]: pgmap v779: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:38 smithi070 ceph-mon[79774]: pgmap v780: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:38 smithi046 ceph-mon[109506]: pgmap v780: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:40 smithi070 ceph-mon[79774]: pgmap v781: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:40 smithi046 ceph-mon[109506]: pgmap v781: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:42.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:41 smithi070 ceph-mon[79774]: pgmap v782: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:42.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:41 smithi046 ceph-mon[109506]: pgmap v782: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:44 smithi046 ceph-mon[109506]: pgmap v783: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:44 smithi070 ceph-mon[79774]: pgmap v783: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:46 smithi070 ceph-mon[79774]: pgmap v784: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:46 smithi046 ceph-mon[109506]: pgmap v784: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:48 smithi070 ceph-mon[79774]: pgmap v785: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:48.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:48 smithi046 ceph-mon[109506]: pgmap v785: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:50 smithi046 ceph-mon[109506]: pgmap v786: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:50 smithi070 ceph-mon[79774]: pgmap v786: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:52 smithi070 ceph-mon[79774]: pgmap v787: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:52.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:52 smithi046 ceph-mon[109506]: pgmap v787: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:31:54.787 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:54 smithi070 ceph-mon[79774]: pgmap v788: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:54 smithi046 ceph-mon[109506]: pgmap v788: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:56 smithi070 ceph-mon[79774]: pgmap v789: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:56.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:56 smithi046 ceph-mon[109506]: pgmap v789: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:31:58 smithi070 ceph-mon[79774]: pgmap v790: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:31:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:31:58 smithi046 ceph-mon[109506]: pgmap v790: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 -- 172.21.15.46:0/1695915877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c107900 msgr2=0x7f8c5c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 --2- 172.21.15.46:0/1695915877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c107900 0x7f8c5c107ce0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009880 tx=0x7f8c5802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 -- 172.21.15.46:0/1695915877 shutdown_connections 2024-09-10T04:32:00.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 --2- 172.21.15.46:0/1695915877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5c101900 0x7f8c5c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 --2- 172.21.15.46:0/1695915877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c107900 0x7f8c5c107ce0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 -- 172.21.15.46:0/1695915877 >> 172.21.15.46:0/1695915877 conn(0x7f8c5c0fd600 msgr2=0x7f8c5c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 -- 172.21.15.46:0/1695915877 shutdown_connections 2024-09-10T04:32:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 -- 172.21.15.46:0/1695915877 wait complete. 2024-09-10T04:32:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 Processor -- start 2024-09-10T04:32:00.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.088+0000 7f8c64739640 1 -- start start 2024-09-10T04:32:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c64739640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c101900 0x7f8c5c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c64739640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5c107900 0x7f8c5c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c64739640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c10edb0 con 0x7f8c5c101900 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c64739640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c10ec30 con 0x7f8c5c107900 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c624d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c101900 0x7f8c5c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c624d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c101900 0x7f8c5c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46230/0 (socket says 172.21.15.46:46230) 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c624d5640 1 -- 172.21.15.46:0/215186707 learned_addr learned my addr 172.21.15.46:0/215186707 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c61cd4640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5c107900 0x7f8c5c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c624d5640 1 -- 172.21.15.46:0/215186707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5c107900 msgr2=0x7f8c5c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c624d5640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5c107900 0x7f8c5c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c624d5640 1 -- 172.21.15.46:0/215186707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c1999d0 con 0x7f8c5c101900 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.089+0000 7f8c624d5640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c101900 0x7f8c5c19f810 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8c58031460 tx=0x7f8c580045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.090+0000 7f8c537fe640 1 -- 172.21.15.46:0/215186707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8c5803f070 con 0x7f8c5c101900 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.090+0000 7f8c537fe640 1 -- 172.21.15.46:0/215186707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c580035f0 con 0x7f8c5c101900 2024-09-10T04:32:00.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.090+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c199c60 con 0x7f8c5c101900 2024-09-10T04:32:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.090+0000 7f8c537fe640 1 -- 172.21.15.46:0/215186707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8c58035030 con 0x7f8c5c101900 2024-09-10T04:32:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.090+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c19a190 con 0x7f8c5c101900 2024-09-10T04:32:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.091+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5c103040 con 0x7f8c5c101900 2024-09-10T04:32:00.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.092+0000 7f8c537fe640 1 -- 172.21.15.46:0/215186707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8c58042070 con 0x7f8c5c101900 2024-09-10T04:32:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.092+0000 7f8c537fe640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8c4807abf0 0x7f8c4807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.093+0000 7f8c61cd4640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8c4807abf0 0x7f8c4807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.093+0000 7f8c537fe640 1 -- 172.21.15.46:0/215186707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8c580be5d0 con 0x7f8c5c101900 2024-09-10T04:32:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.093+0000 7f8c61cd4640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8c4807abf0 0x7f8c4807d0b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c076480 tx=0x7f8c4c00b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.097+0000 7f8c537fe640 1 -- 172.21.15.46:0/215186707 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8c58089d10 con 0x7f8c5c101900 2024-09-10T04:32:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.302+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8c5c19a4a0 con 0x7f8c4807abf0 2024-09-10T04:32:00.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.303+0000 7f8c537fe640 1 -- 172.21.15.46:0/215186707 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8c5c19a4a0 con 0x7f8c4807abf0 2024-09-10T04:32:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8c4807abf0 msgr2=0x7f8c4807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8c4807abf0 0x7f8c4807d0b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c076480 tx=0x7f8c4c00b410 comp rx=0 tx=0).stop 2024-09-10T04:32:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c101900 msgr2=0x7f8c5c19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c101900 0x7f8c5c19f810 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8c58031460 tx=0x7f8c580045e0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 shutdown_connections 2024-09-10T04:32:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8c4807abf0 0x7f8c4807d0b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8c5c107900 0x7f8c5c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 --2- 172.21.15.46:0/215186707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8c5c101900 0x7f8c5c19f810 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 >> 172.21.15.46:0/215186707 conn(0x7f8c5c0fd600 msgr2=0x7f8c5c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 shutdown_connections 2024-09-10T04:32:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.305+0000 7f8c64739640 1 -- 172.21.15.46:0/215186707 wait complete. 2024-09-10T04:32:00.316 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 -- 172.21.15.46:0/3865978191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301007f0 msgr2=0x7f8e30100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3865978191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301007f0 0x7f8e30100c50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009880 tx=0x7f8e2002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 -- 172.21.15.46:0/3865978191 shutdown_connections 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3865978191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301007f0 0x7f8e30100c50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3865978191 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e301067f0 0x7f8e30106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 -- 172.21.15.46:0/3865978191 >> 172.21.15.46:0/3865978191 conn(0x7f8e300fc550 msgr2=0x7f8e300fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 -- 172.21.15.46:0/3865978191 shutdown_connections 2024-09-10T04:32:00.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.392+0000 7f8e37f17640 1 -- 172.21.15.46:0/3865978191 wait complete. 2024-09-10T04:32:00.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e37f17640 1 Processor -- start 2024-09-10T04:32:00.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e37f17640 1 -- start start 2024-09-10T04:32:00.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e37f17640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e301007f0 0x7f8e3019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e37f17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301067f0 0x7f8e3019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e37f17640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3010dca0 con 0x7f8e301067f0 2024-09-10T04:32:00.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e37f17640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3010db20 con 0x7f8e301007f0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e354b2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301067f0 0x7f8e3019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e354b2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301067f0 0x7f8e3019dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46252/0 (socket says 172.21.15.46:46252) 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e354b2640 1 -- 172.21.15.46:0/3982134497 learned_addr learned my addr 172.21.15.46:0/3982134497 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e35cb3640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e301007f0 0x7f8e3019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e354b2640 1 -- 172.21.15.46:0/3982134497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e301007f0 msgr2=0x7f8e3019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e354b2640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e301007f0 0x7f8e3019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.393+0000 7f8e354b2640 1 -- 172.21.15.46:0/3982134497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e30197770 con 0x7f8e301067f0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.394+0000 7f8e354b2640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301067f0 0x7f8e3019dbe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e20031460 tx=0x7f8e200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.394+0000 7f8e26ffd640 1 -- 172.21.15.46:0/3982134497 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8e2003f070 con 0x7f8e301067f0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.394+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e30197a00 con 0x7f8e301067f0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.394+0000 7f8e26ffd640 1 -- 172.21.15.46:0/3982134497 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8e20004cf0 con 0x7f8e301067f0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.394+0000 7f8e26ffd640 1 -- 172.21.15.46:0/3982134497 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8e20035030 con 0x7f8e301067f0 2024-09-10T04:32:00.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.394+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e30197f00 con 0x7f8e301067f0 2024-09-10T04:32:00.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.395+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e30101f30 con 0x7f8e301067f0 2024-09-10T04:32:00.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.396+0000 7f8e26ffd640 1 -- 172.21.15.46:0/3982134497 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8e20042070 con 0x7f8e301067f0 2024-09-10T04:32:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.397+0000 7f8e26ffd640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e1c07abf0 0x7f8e1c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.397+0000 7f8e26ffd640 1 -- 172.21.15.46:0/3982134497 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8e200be750 con 0x7f8e301067f0 2024-09-10T04:32:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.397+0000 7f8e35cb3640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e1c07abf0 0x7f8e1c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.398+0000 7f8e35cb3640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e1c07abf0 0x7f8e1c07d0b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0058c0 tx=0x7f8e2c00a4c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:00.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.401+0000 7f8e26ffd640 1 -- 172.21.15.46:0/3982134497 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8e20089e90 con 0x7f8e301067f0 2024-09-10T04:32:00.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.610+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8e301981f0 con 0x7f8e1c07abf0 2024-09-10T04:32:00.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.611+0000 7f8e26ffd640 1 -- 172.21.15.46:0/3982134497 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8e301981f0 con 0x7f8e1c07abf0 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e1c07abf0 msgr2=0x7f8e1c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e1c07abf0 0x7f8e1c07d0b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0058c0 tx=0x7f8e2c00a4c0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301067f0 msgr2=0x7f8e3019dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301067f0 0x7f8e3019dbe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e20031460 tx=0x7f8e200048f0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 shutdown_connections 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8e1c07abf0 0x7f8e1c07d0b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e301067f0 0x7f8e3019dbe0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 --2- 172.21.15.46:0/3982134497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e301007f0 0x7f8e3019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 >> 172.21.15.46:0/3982134497 conn(0x7f8e300fc550 msgr2=0x7f8e30104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:00.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 shutdown_connections 2024-09-10T04:32:00.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.614+0000 7f8e37f17640 1 -- 172.21.15.46:0/3982134497 wait complete. 2024-09-10T04:32:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.698+0000 7efd741fb640 1 -- 172.21.15.46:0/4204215540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 msgr2=0x7efd6c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.698+0000 7efd741fb640 1 --2- 172.21.15.46:0/4204215540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c102ea0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efd68009880 tx=0x7efd6802a530 comp rx=0 tx=0).stop 2024-09-10T04:32:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.698+0000 7efd741fb640 1 -- 172.21.15.46:0/4204215540 shutdown_connections 2024-09-10T04:32:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.698+0000 7efd741fb640 1 --2- 172.21.15.46:0/4204215540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c1033e0 0x7efd6c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.698+0000 7efd741fb640 1 --2- 172.21.15.46:0/4204215540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c102ea0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.698+0000 7efd741fb640 1 -- 172.21.15.46:0/4204215540 >> 172.21.15.46:0/4204215540 conn(0x7efd6c0fc520 msgr2=0x7efd6c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:00.701 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:00 smithi046 ceph-mon[109506]: pgmap v791: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.699+0000 7efd741fb640 1 -- 172.21.15.46:0/4204215540 shutdown_connections 2024-09-10T04:32:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.699+0000 7efd741fb640 1 -- 172.21.15.46:0/4204215540 wait complete. 2024-09-10T04:32:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.699+0000 7efd741fb640 1 Processor -- start 2024-09-10T04:32:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.699+0000 7efd741fb640 1 -- start start 2024-09-10T04:32:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd741fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd741fb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c1033e0 0x7efd6c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd741fb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c10edd0 con 0x7efd6c102ac0 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd741fb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c10ec50 con 0x7efd6c1033e0 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71f97640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71f97640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46272/0 (socket says 172.21.15.46:46272) 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71f97640 1 -- 172.21.15.46:0/3032783019 learned_addr learned my addr 172.21.15.46:0/3032783019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71796640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c1033e0 0x7efd6c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71f97640 1 -- 172.21.15.46:0/3032783019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c1033e0 msgr2=0x7efd6c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71f97640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c1033e0 0x7efd6c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71f97640 1 -- 172.21.15.46:0/3032783019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c1977c0 con 0x7efd6c102ac0 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd71f97640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c19d640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efd68031460 tx=0x7efd680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd5effd640 1 -- 172.21.15.46:0/3032783019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efd6803f070 con 0x7efd6c102ac0 2024-09-10T04:32:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.700+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c197a50 con 0x7efd6c102ac0 2024-09-10T04:32:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.701+0000 7efd5effd640 1 -- 172.21.15.46:0/3032783019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd68004cf0 con 0x7efd6c102ac0 2024-09-10T04:32:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.701+0000 7efd5effd640 1 -- 172.21.15.46:0/3032783019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efd68035030 con 0x7efd6c102ac0 2024-09-10T04:32:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.701+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c197f80 con 0x7efd6c102ac0 2024-09-10T04:32:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.702+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c06aa90 con 0x7efd6c102ac0 2024-09-10T04:32:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.702+0000 7efd5effd640 1 -- 172.21.15.46:0/3032783019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efd68042070 con 0x7efd6c102ac0 2024-09-10T04:32:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.703+0000 7efd5effd640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd5807ac40 0x7efd5807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.703+0000 7efd5effd640 1 -- 172.21.15.46:0/3032783019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efd680be6a0 con 0x7efd6c102ac0 2024-09-10T04:32:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.703+0000 7efd71796640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd5807ac40 0x7efd5807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.703+0000 7efd71796640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd5807ac40 0x7efd5807d100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efd60005fd0 tx=0x7efd60005f60 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:00.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.707+0000 7efd5effd640 1 -- 172.21.15.46:0/3032783019 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efd68089e60 con 0x7efd6c102ac0 2024-09-10T04:32:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:00 smithi070 ceph-mon[79774]: pgmap v791: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.901+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7efd6c198cf0 con 0x7efd5807ac40 2024-09-10T04:32:00.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.911+0000 7efd5effd640 1 -- 172.21.15.46:0/3032783019 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7efd6c198cf0 con 0x7efd5807ac40 2024-09-10T04:32:00.913 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:32:00.913 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (23m) 2m ago 34m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:32:00.913 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (26m) 2m ago 35m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:32:00.913 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (25m) 2m ago 34m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (22m) 2m ago 35m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (21m) 2m ago 34m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (23m) 2m ago 34m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (18m) 2m ago 29m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (18m) 2m ago 29m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (17m) 2m ago 29m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (2m) 2m ago 29m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (25m) 2m ago 36m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (24m) 2m ago 34m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:32:00.914 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (22m) 2m ago 36m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (22m) 2m ago 34m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (24m) 2m ago 35m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (24m) 2m ago 34m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (21m) 2m ago 32m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (20m) 2m ago 32m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (20m) 2m ago 31m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (19m) 2m ago 31m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (19m) 2m ago 31m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (18m) 2m ago 30m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:32:00.915 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (23m) 2m ago 34m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:32:00.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd5807ac40 msgr2=0x7efd5807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd5807ac40 0x7efd5807d100 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efd60005fd0 tx=0x7efd60005f60 comp rx=0 tx=0).stop 2024-09-10T04:32:00.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 msgr2=0x7efd6c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c19d640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efd68031460 tx=0x7efd680048f0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 shutdown_connections 2024-09-10T04:32:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd5807ac40 0x7efd5807d100 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd6c1033e0 0x7efd6c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 --2- 172.21.15.46:0/3032783019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd6c102ac0 0x7efd6c19d640 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 >> 172.21.15.46:0/3032783019 conn(0x7efd6c0fc520 msgr2=0x7efd6c108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 shutdown_connections 2024-09-10T04:32:00.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.913+0000 7efd741fb640 1 -- 172.21.15.46:0/3032783019 wait complete. 2024-09-10T04:32:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 -- 172.21.15.46:0/2457536531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101250 msgr2=0x7f6c5c101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:00.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2457536531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101250 0x7f6c5c101630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009880 tx=0x7f6c5802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 -- 172.21.15.46:0/2457536531 shutdown_connections 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2457536531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c5c101c00 0x7f6c5c105520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2457536531 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101250 0x7f6c5c101630 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 -- 172.21.15.46:0/2457536531 >> 172.21.15.46:0/2457536531 conn(0x7f6c5c0fbc90 msgr2=0x7f6c5c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 -- 172.21.15.46:0/2457536531 shutdown_connections 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.997+0000 7f6c641e8640 1 -- 172.21.15.46:0/2457536531 wait complete. 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c641e8640 1 Processor -- start 2024-09-10T04:32:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c641e8640 1 -- start start 2024-09-10T04:32:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c641e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c5c101250 0x7f6c5c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c641e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101c00 0x7f6c5c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c641e8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10dde0 con 0x7f6c5c101c00 2024-09-10T04:32:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c641e8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5c10dc60 con 0x7f6c5c101250 2024-09-10T04:32:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c61783640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101c00 0x7f6c5c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c61783640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101c00 0x7f6c5c198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46286/0 (socket says 172.21.15.46:46286) 2024-09-10T04:32:01.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c61783640 1 -- 172.21.15.46:0/2197051894 learned_addr learned my addr 172.21.15.46:0/2197051894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c61f84640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c5c101250 0x7f6c5c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c61783640 1 -- 172.21.15.46:0/2197051894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c5c101250 msgr2=0x7f6c5c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c61783640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c5c101250 0x7f6c5c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.998+0000 7f6c61783640 1 -- 172.21.15.46:0/2197051894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c5c199460 con 0x7f6c5c101c00 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.999+0000 7f6c61783640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101c00 0x7f6c5c198e10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c8f0 tx=0x7f6c4c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.999+0000 7f6c52ffd640 1 -- 172.21.15.46:0/2197051894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6c4c0079b0 con 0x7f6c5c101c00 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.999+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5c1a1940 con 0x7f6c5c101c00 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.999+0000 7f6c52ffd640 1 -- 172.21.15.46:0/2197051894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6c4c0054e0 con 0x7f6c5c101c00 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.999+0000 7f6c52ffd640 1 -- 172.21.15.46:0/2197051894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6c4c004dd0 con 0x7f6c5c101c00 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:00.999+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c5c1a1d40 con 0x7f6c5c101c00 2024-09-10T04:32:01.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.000+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5c102000 con 0x7f6c5c101c00 2024-09-10T04:32:01.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.001+0000 7f6c52ffd640 1 -- 172.21.15.46:0/2197051894 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6c4c005020 con 0x7f6c5c101c00 2024-09-10T04:32:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.001+0000 7f6c52ffd640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c4807ab90 0x7f6c4807d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.002+0000 7f6c52ffd640 1 -- 172.21.15.46:0/2197051894 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6c4c09dba0 con 0x7f6c5c101c00 2024-09-10T04:32:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.002+0000 7f6c61f84640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c4807ab90 0x7f6c4807d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.002+0000 7f6c61f84640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c4807ab90 0x7f6c4807d050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c58000c00 tx=0x7f6c58031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.006+0000 7f6c52ffd640 1 -- 172.21.15.46:0/2197051894 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6c4c014030 con 0x7f6c5c101c00 2024-09-10T04:32:01.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.264+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f6c5c108e00 con 0x7f6c5c101c00 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.265+0000 7f6c52ffd640 1 -- 172.21.15.46:0/2197051894 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f6c4c0692e0 con 0x7f6c5c101c00 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:32:01.267 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:32:01.268 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c4807ab90 msgr2=0x7f6c4807d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c4807ab90 0x7f6c4807d050 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c58000c00 tx=0x7f6c58031400 comp rx=0 tx=0).stop 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101c00 msgr2=0x7f6c5c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101c00 0x7f6c5c198e10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c8f0 tx=0x7f6c4c00cdb0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 shutdown_connections 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c4807ab90 0x7f6c4807d050 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c5c101c00 0x7f6c5c198e10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 --2- 172.21.15.46:0/2197051894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c5c101250 0x7f6c5c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 >> 172.21.15.46:0/2197051894 conn(0x7f6c5c0fbc90 msgr2=0x7f6c5c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:01.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 shutdown_connections 2024-09-10T04:32:01.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.267+0000 7f6c641e8640 1 -- 172.21.15.46:0/2197051894 wait complete. 2024-09-10T04:32:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 -- 172.21.15.46:0/1566383008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8107900 msgr2=0x7f97e8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1566383008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8107900 0x7f97e8107ce0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009880 tx=0x7f97e402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 -- 172.21.15.46:0/1566383008 shutdown_connections 2024-09-10T04:32:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1566383008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97e8101900 0x7f97e8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1566383008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8107900 0x7f97e8107ce0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 -- 172.21.15.46:0/1566383008 >> 172.21.15.46:0/1566383008 conn(0x7f97e80fd600 msgr2=0x7f97e80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 -- 172.21.15.46:0/1566383008 shutdown_connections 2024-09-10T04:32:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.351+0000 7f97ef94c640 1 -- 172.21.15.46:0/1566383008 wait complete. 2024-09-10T04:32:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ef94c640 1 Processor -- start 2024-09-10T04:32:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ef94c640 1 -- start start 2024-09-10T04:32:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ef94c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8101900 0x7f97e819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ef94c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97e8107900 0x7f97e819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ef94c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810edd0 con 0x7f97e8101900 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ef94c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f97e810ec50 con 0x7f97e8107900 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ed6e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8101900 0x7f97e819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ed6e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8101900 0x7f97e819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46300/0 (socket says 172.21.15.46:46300) 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ed6e8640 1 -- 172.21.15.46:0/1423005783 learned_addr learned my addr 172.21.15.46:0/1423005783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ecee7640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97e8107900 0x7f97e819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ed6e8640 1 -- 172.21.15.46:0/1423005783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97e8107900 msgr2=0x7f97e819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.352+0000 7f97ed6e8640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97e8107900 0x7f97e819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.353+0000 7f97ed6e8640 1 -- 172.21.15.46:0/1423005783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e81977a0 con 0x7f97e8101900 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.353+0000 7f97ed6e8640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8101900 0x7f97e819d620 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f97e4031460 tx=0x7f97e4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.353+0000 7f97de7fc640 1 -- 172.21.15.46:0/1423005783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97e403f070 con 0x7f97e8101900 2024-09-10T04:32:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.353+0000 7f97de7fc640 1 -- 172.21.15.46:0/1423005783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97e40035f0 con 0x7f97e8101900 2024-09-10T04:32:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.353+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e8197a30 con 0x7f97e8101900 2024-09-10T04:32:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.353+0000 7f97de7fc640 1 -- 172.21.15.46:0/1423005783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97e4035030 con 0x7f97e8101900 2024-09-10T04:32:01.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.353+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e8197f60 con 0x7f97e8101900 2024-09-10T04:32:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.354+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e8103040 con 0x7f97e8101900 2024-09-10T04:32:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.355+0000 7f97de7fc640 1 -- 172.21.15.46:0/1423005783 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f97e4042070 con 0x7f97e8101900 2024-09-10T04:32:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.355+0000 7f97de7fc640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97d407abf0 0x7f97d407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.355+0000 7f97de7fc640 1 -- 172.21.15.46:0/1423005783 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f97e40be5b0 con 0x7f97e8101900 2024-09-10T04:32:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.355+0000 7f97ecee7640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97d407abf0 0x7f97d407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.356+0000 7f97ecee7640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97d407abf0 0x7f97d407d0b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f97e80ffc20 tx=0x7f97d8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.360+0000 7f97de7fc640 1 -- 172.21.15.46:0/1423005783 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f97e4089cf0 con 0x7f97e8101900 2024-09-10T04:32:01.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.584+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f97e8109e70 con 0x7f97e8101900 2024-09-10T04:32:01.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.584+0000 7f97de7fc640 1 -- 172.21.15.46:0/1423005783 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f97e408eba0 con 0x7f97e8101900 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:32:01.587 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:32:01.588 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:32:01.589 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:01.590 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:01.590 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:32:01.590 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:01.590 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:01.590 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:01.590 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:01.590 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:32:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97d407abf0 msgr2=0x7f97d407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97d407abf0 0x7f97d407d0b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f97e80ffc20 tx=0x7f97d8005f90 comp rx=0 tx=0).stop 2024-09-10T04:32:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8101900 msgr2=0x7f97e819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8101900 0x7f97e819d620 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f97e4031460 tx=0x7f97e4004530 comp rx=0 tx=0).stop 2024-09-10T04:32:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 shutdown_connections 2024-09-10T04:32:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97d407abf0 0x7f97d407d0b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97e8107900 0x7f97e819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 --2- 172.21.15.46:0/1423005783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e8101900 0x7f97e819d620 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 >> 172.21.15.46:0/1423005783 conn(0x7f97e80fd600 msgr2=0x7f97e80fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 shutdown_connections 2024-09-10T04:32:01.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.587+0000 7f97ef94c640 1 -- 172.21.15.46:0/1423005783 wait complete. 2024-09-10T04:32:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 -- 172.21.15.46:0/681317030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 msgr2=0x7f0d68107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 --2- 172.21.15.46:0/681317030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68107ce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009880 tx=0x7f0d5802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 -- 172.21.15.46:0/681317030 shutdown_connections 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 --2- 172.21.15.46:0/681317030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d68101900 0x7f0d68101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 --2- 172.21.15.46:0/681317030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68107ce0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 -- 172.21.15.46:0/681317030 >> 172.21.15.46:0/681317030 conn(0x7f0d680fd600 msgr2=0x7f0d680ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 -- 172.21.15.46:0/681317030 shutdown_connections 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.672+0000 7f0d6d587640 1 -- 172.21.15.46:0/681317030 wait complete. 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d6d587640 1 Processor -- start 2024-09-10T04:32:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d6d587640 1 -- start start 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d6d587640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d68101900 0x7f0d680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d6d587640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d6d587640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6810edd0 con 0x7f0d68107900 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d6d587640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0d6810ec50 con 0x7f0d68101900 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d667fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d667fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46318/0 (socket says 172.21.15.46:46318) 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d667fc640 1 -- 172.21.15.46:0/2608135635 learned_addr learned my addr 172.21.15.46:0/2608135635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.673+0000 7f0d66ffd640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d68101900 0x7f0d680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d667fc640 1 -- 172.21.15.46:0/2608135635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d68101900 msgr2=0x7f0d680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d667fc640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d68101900 0x7f0d680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d667fc640 1 -- 172.21.15.46:0/2608135635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68074f70 con 0x7f0d68107900 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d66ffd640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d68101900 0x7f0d680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d667fc640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68074a30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00d6f0 tx=0x7f0d5c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d4ffff640 1 -- 172.21.15.46:0/2608135635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d5c017070 con 0x7f0d68107900 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d4ffff640 1 -- 172.21.15.46:0/2608135635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d5c00de00 con 0x7f0d68107900 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d68075260 con 0x7f0d68107900 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d4ffff640 1 -- 172.21.15.46:0/2608135635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0d5c004bb0 con 0x7f0d68107900 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.674+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d681a3e40 con 0x7f0d68107900 2024-09-10T04:32:01.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.675+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d68103040 con 0x7f0d68107900 2024-09-10T04:32:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.676+0000 7f0d4ffff640 1 -- 172.21.15.46:0/2608135635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0d5c004e60 con 0x7f0d68107900 2024-09-10T04:32:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.676+0000 7f0d4ffff640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d5007abf0 0x7f0d5007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.676+0000 7f0d4ffff640 1 -- 172.21.15.46:0/2608135635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0d5c09d970 con 0x7f0d68107900 2024-09-10T04:32:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.676+0000 7f0d66ffd640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d5007abf0 0x7f0d5007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.677+0000 7f0d66ffd640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d5007abf0 0x7f0d5007d0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0d58000c00 tx=0x7f0d58031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.681+0000 7f0d4ffff640 1 -- 172.21.15.46:0/2608135635 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0d5c069010 con 0x7f0d68107900 2024-09-10T04:32:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:01 smithi070 ceph-mon[79774]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:01 smithi070 ceph-mon[79774]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:01.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:01 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2197051894' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:32:01.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:01 smithi046 ceph-mon[109506]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:01 smithi046 ceph-mon[109506]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:01.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:01 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2197051894' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:32:01.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.883+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0d68062630 con 0x7f0d5007abf0 2024-09-10T04:32:01.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.885+0000 7f0d4ffff640 1 -- 172.21.15.46:0/2608135635 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f0d68062630 con 0x7f0d5007abf0 2024-09-10T04:32:01.886 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:32:01.886 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:32:01.887 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:32:01.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d5007abf0 msgr2=0x7f0d5007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d5007abf0 0x7f0d5007d0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0d58000c00 tx=0x7f0d58031400 comp rx=0 tx=0).stop 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 msgr2=0x7f0d68074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68074a30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00d6f0 tx=0x7f0d5c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 shutdown_connections 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0d5007abf0 0x7f0d5007d0b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d68107900 0x7f0d68074a30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 --2- 172.21.15.46:0/2608135635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d68101900 0x7f0d680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 >> 172.21.15.46:0/2608135635 conn(0x7f0d680fd600 msgr2=0x7f0d680fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 shutdown_connections 2024-09-10T04:32:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.887+0000 7f0d6d587640 1 -- 172.21.15.46:0/2608135635 wait complete. 2024-09-10T04:32:01.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 -- 172.21.15.46:0/1375590982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 msgr2=0x7f4c40107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 --2- 172.21.15.46:0/1375590982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c40107ce0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009880 tx=0x7f4c3002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 -- 172.21.15.46:0/1375590982 shutdown_connections 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 --2- 172.21.15.46:0/1375590982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c40101900 0x7f4c40101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 --2- 172.21.15.46:0/1375590982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c40107ce0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 -- 172.21.15.46:0/1375590982 >> 172.21.15.46:0/1375590982 conn(0x7f4c400fd600 msgr2=0x7f4c400ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 -- 172.21.15.46:0/1375590982 shutdown_connections 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.971+0000 7f4c4584e640 1 -- 172.21.15.46:0/1375590982 wait complete. 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.972+0000 7f4c4584e640 1 Processor -- start 2024-09-10T04:32:01.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.972+0000 7f4c4584e640 1 -- start start 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.972+0000 7f4c4584e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c40101900 0x7f4c4019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.972+0000 7f4c4584e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c4019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.972+0000 7f4c4584e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010edb0 con 0x7f4c40107900 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.972+0000 7f4c4584e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4010ec30 con 0x7f4c40101900 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c4019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c367fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c4019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46338/0 (socket says 172.21.15.46:46338) 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c367fc640 1 -- 172.21.15.46:0/3450230845 learned_addr learned my addr 172.21.15.46:0/3450230845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c3effd640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c40101900 0x7f4c4019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c367fc640 1 -- 172.21.15.46:0/3450230845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c40101900 msgr2=0x7f4c4019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:01.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c367fc640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c40101900 0x7f4c4019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c367fc640 1 -- 172.21.15.46:0/3450230845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c401977c0 con 0x7f4c40107900 2024-09-10T04:32:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c367fc640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c4019db80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800d8d0 tx=0x7f4c2800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c4484c640 1 -- 172.21.15.46:0/3450230845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4c28017030 con 0x7f4c40107900 2024-09-10T04:32:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c40197ab0 con 0x7f4c40107900 2024-09-10T04:32:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c4484c640 1 -- 172.21.15.46:0/3450230845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4c28004020 con 0x7f4c40107900 2024-09-10T04:32:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c4484c640 1 -- 172.21.15.46:0/3450230845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4c280047a0 con 0x7f4c40107900 2024-09-10T04:32:01.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.973+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c40198040 con 0x7f4c40107900 2024-09-10T04:32:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.974+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c40103040 con 0x7f4c40107900 2024-09-10T04:32:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.975+0000 7f4c4484c640 1 -- 172.21.15.46:0/3450230845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4c280041d0 con 0x7f4c40107900 2024-09-10T04:32:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.976+0000 7f4c4484c640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4c2c07ac90 0x7f4c2c07d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.976+0000 7f4c4484c640 1 -- 172.21.15.46:0/3450230845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4c28018070 con 0x7f4c40107900 2024-09-10T04:32:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.976+0000 7f4c3effd640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4c2c07ac90 0x7f4c2c07d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:01.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.976+0000 7f4c3effd640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4c2c07ac90 0x7f4c2c07d150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c300096c0 tx=0x7f4c30031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:01.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:01.980+0000 7f4c4484c640 1 -- 172.21.15.46:0/3450230845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4c28069190 con 0x7f4c40107900 2024-09-10T04:32:02.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.240+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f4c40104b10 con 0x7f4c40107900 2024-09-10T04:32:02.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.240+0000 7f4c4484c640 1 -- 172.21.15.46:0/3450230845 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f4c2806e040 con 0x7f4c40107900 2024-09-10T04:32:02.242 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:32:02.242 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:32:02.243 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:32:02.244 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:32:02.245 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:32:02.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.242+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4c2c07ac90 msgr2=0x7f4c2c07d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.242+0000 7f4c4584e640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4c2c07ac90 0x7f4c2c07d150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4c300096c0 tx=0x7f4c30031400 comp rx=0 tx=0).stop 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.242+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 msgr2=0x7f4c4019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.242+0000 7f4c4584e640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c4019db80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800d8d0 tx=0x7f4c2800dd90 comp rx=0 tx=0).stop 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.243+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 shutdown_connections 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.243+0000 7f4c4584e640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4c2c07ac90 0x7f4c2c07d150 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.243+0000 7f4c4584e640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c40107900 0x7f4c4019db80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.243+0000 7f4c4584e640 1 --2- 172.21.15.46:0/3450230845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c40101900 0x7f4c4019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.243+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 >> 172.21.15.46:0/3450230845 conn(0x7f4c400fd600 msgr2=0x7f4c400ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.243+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 shutdown_connections 2024-09-10T04:32:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:02.243+0000 7f4c4584e640 1 -- 172.21.15.46:0/3450230845 wait complete. 2024-09-10T04:32:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:02 smithi070 ceph-mon[79774]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:02 smithi070 ceph-mon[79774]: pgmap v792: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:02 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1423005783' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:32:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:02 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3450230845' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:32:02.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:02 smithi046 ceph-mon[109506]: from='client.35298 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:02.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:02 smithi046 ceph-mon[109506]: pgmap v792: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:02.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:02 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1423005783' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:32:02.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:02 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3450230845' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:32:03.801 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:03 smithi046 ceph-mon[109506]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:03.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:03 smithi070 ceph-mon[79774]: from='client.35310 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:04 smithi070 ceph-mon[79774]: pgmap v793: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:04.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:04 smithi046 ceph-mon[109506]: pgmap v793: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:06 smithi070 ceph-mon[79774]: pgmap v794: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:06.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:06 smithi046 ceph-mon[109506]: pgmap v794: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:08 smithi070 ceph-mon[79774]: pgmap v795: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:08 smithi046 ceph-mon[109506]: pgmap v795: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:10 smithi070 ceph-mon[79774]: pgmap v796: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:10.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:10 smithi046 ceph-mon[109506]: pgmap v796: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:12 smithi070 ceph-mon[79774]: pgmap v797: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:12 smithi046 ceph-mon[109506]: pgmap v797: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:14 smithi070 ceph-mon[79774]: pgmap v798: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:14 smithi046 ceph-mon[109506]: pgmap v798: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:16 smithi070 ceph-mon[79774]: pgmap v799: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:16 smithi046 ceph-mon[109506]: pgmap v799: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:18 smithi070 ceph-mon[79774]: pgmap v800: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:18 smithi046 ceph-mon[109506]: pgmap v800: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:20.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:20 smithi046 ceph-mon[109506]: pgmap v801: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:20 smithi070 ceph-mon[79774]: pgmap v801: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:22 smithi070 ceph-mon[79774]: pgmap v802: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:22 smithi046 ceph-mon[109506]: pgmap v802: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:24 smithi070 ceph-mon[79774]: pgmap v803: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:24 smithi046 ceph-mon[109506]: pgmap v803: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:26 smithi070 ceph-mon[79774]: pgmap v804: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:26 smithi046 ceph-mon[109506]: pgmap v804: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:28 smithi070 ceph-mon[79774]: pgmap v805: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:28 smithi046 ceph-mon[109506]: pgmap v805: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:30 smithi070 ceph-mon[79774]: pgmap v806: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:30 smithi046 ceph-mon[109506]: pgmap v806: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.342+0000 7efedc197640 1 -- 172.21.15.46:0/2650337684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 msgr2=0x7efed4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.342+0000 7efedc197640 1 --2- 172.21.15.46:0/2650337684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4107ce0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efed0009880 tx=0x7efed002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.342+0000 7efedc197640 1 -- 172.21.15.46:0/2650337684 shutdown_connections 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.342+0000 7efedc197640 1 --2- 172.21.15.46:0/2650337684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4101900 0x7efed4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.342+0000 7efedc197640 1 --2- 172.21.15.46:0/2650337684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4107ce0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.342+0000 7efedc197640 1 -- 172.21.15.46:0/2650337684 >> 172.21.15.46:0/2650337684 conn(0x7efed40fd600 msgr2=0x7efed40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.343+0000 7efedc197640 1 -- 172.21.15.46:0/2650337684 shutdown_connections 2024-09-10T04:32:32.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.343+0000 7efedc197640 1 -- 172.21.15.46:0/2650337684 wait complete. 2024-09-10T04:32:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.343+0000 7efedc197640 1 Processor -- start 2024-09-10T04:32:32.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.343+0000 7efedc197640 1 -- start start 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efedc197640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4101900 0x7efed40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efedc197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efedc197640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efed410edb0 con 0x7efed4107900 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efedc197640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efed410ec30 con 0x7efed4101900 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9732640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9732640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39776/0 (socket says 172.21.15.46:39776) 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9732640 1 -- 172.21.15.46:0/2236467133 learned_addr learned my addr 172.21.15.46:0/2236467133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9f33640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4101900 0x7efed40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9732640 1 -- 172.21.15.46:0/2236467133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4101900 msgr2=0x7efed40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9732640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4101900 0x7efed40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9732640 1 -- 172.21.15.46:0/2236467133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed4074f70 con 0x7efed4107900 2024-09-10T04:32:32.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efed9732640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4074a30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efec400c8a0 tx=0x7efec400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efecaffd640 1 -- 172.21.15.46:0/2236467133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efec40079b0 con 0x7efed4107900 2024-09-10T04:32:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.344+0000 7efecaffd640 1 -- 172.21.15.46:0/2236467133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efec4007b50 con 0x7efed4107900 2024-09-10T04:32:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.345+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed4075260 con 0x7efed4107900 2024-09-10T04:32:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.345+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed41a3e00 con 0x7efed4107900 2024-09-10T04:32:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.345+0000 7efecaffd640 1 -- 172.21.15.46:0/2236467133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efec4018700 con 0x7efed4107900 2024-09-10T04:32:32.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.346+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed4103040 con 0x7efed4107900 2024-09-10T04:32:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.346+0000 7efecaffd640 1 -- 172.21.15.46:0/2236467133 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efec400f450 con 0x7efed4107900 2024-09-10T04:32:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.347+0000 7efecaffd640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efec007abf0 0x7efec007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.347+0000 7efecaffd640 1 -- 172.21.15.46:0/2236467133 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efec409deb0 con 0x7efed4107900 2024-09-10T04:32:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.347+0000 7efed9f33640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efec007abf0 0x7efec007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.347+0000 7efed9f33640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efec007abf0 0x7efec007d0b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efed0000c00 tx=0x7efed0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:32.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.352+0000 7efecaffd640 1 -- 172.21.15.46:0/2236467133 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efec4014030 con 0x7efed4107900 2024-09-10T04:32:32.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.558+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7efed4062630 con 0x7efec007abf0 2024-09-10T04:32:32.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.559+0000 7efecaffd640 1 -- 172.21.15.46:0/2236467133 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7efed4062630 con 0x7efec007abf0 2024-09-10T04:32:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efec007abf0 msgr2=0x7efec007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efec007abf0 0x7efec007d0b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efed0000c00 tx=0x7efed0031400 comp rx=0 tx=0).stop 2024-09-10T04:32:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 msgr2=0x7efed4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4074a30 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efec400c8a0 tx=0x7efec400cd60 comp rx=0 tx=0).stop 2024-09-10T04:32:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 shutdown_connections 2024-09-10T04:32:32.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efec007abf0 0x7efec007d0b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efed4107900 0x7efed4074a30 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 --2- 172.21.15.46:0/2236467133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efed4101900 0x7efed40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 >> 172.21.15.46:0/2236467133 conn(0x7efed40fd600 msgr2=0x7efed40fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 shutdown_connections 2024-09-10T04:32:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.561+0000 7efedc197640 1 -- 172.21.15.46:0/2236467133 wait complete. 2024-09-10T04:32:32.572 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 -- 172.21.15.46:0/1135724605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 msgr2=0x7fee98107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 --2- 172.21.15.46:0/1135724605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98107cf0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fee94009880 tx=0x7fee9402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 -- 172.21.15.46:0/1135724605 shutdown_connections 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 --2- 172.21.15.46:0/1135724605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee98101910 0x7fee98101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 --2- 172.21.15.46:0/1135724605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98107cf0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 -- 172.21.15.46:0/1135724605 >> 172.21.15.46:0/1135724605 conn(0x7fee980fd630 msgr2=0x7fee980ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 -- 172.21.15.46:0/1135724605 shutdown_connections 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.647+0000 7fee9fc60640 1 -- 172.21.15.46:0/1135724605 wait complete. 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9fc60640 1 Processor -- start 2024-09-10T04:32:32.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9fc60640 1 -- start start 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9fc60640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee98101910 0x7fee980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9fc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9fc60640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810edc0 con 0x7fee98107910 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9fc60640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810ec40 con 0x7fee98101910 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9d1fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9d1fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39794/0 (socket says 172.21.15.46:39794) 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.648+0000 7fee9d1fb640 1 -- 172.21.15.46:0/661049662 learned_addr learned my addr 172.21.15.46:0/661049662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee9d9fc640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee98101910 0x7fee980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee9d1fb640 1 -- 172.21.15.46:0/661049662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee98101910 msgr2=0x7fee980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee9d1fb640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee98101910 0x7fee980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee9d1fb640 1 -- 172.21.15.46:0/661049662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee98074f70 con 0x7fee98107910 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee9d1fb640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98074a30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c810 tx=0x7fee8800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee8effd640 1 -- 172.21.15.46:0/661049662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee88004020 con 0x7fee98107910 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee8effd640 1 -- 172.21.15.46:0/661049662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee880041c0 con 0x7fee98107910 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee98075260 con 0x7fee98107910 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee8effd640 1 -- 172.21.15.46:0/661049662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fee880047a0 con 0x7fee98107910 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.649+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a3df0 con 0x7fee98107910 2024-09-10T04:32:32.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.650+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee98103050 con 0x7fee98107910 2024-09-10T04:32:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.651+0000 7fee8effd640 1 -- 172.21.15.46:0/661049662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fee880079b0 con 0x7fee98107910 2024-09-10T04:32:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.651+0000 7fee8effd640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407ac40 0x7fee8407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.651+0000 7fee8effd640 1 -- 172.21.15.46:0/661049662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fee8809dc00 con 0x7fee98107910 2024-09-10T04:32:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.651+0000 7fee9d9fc640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407ac40 0x7fee8407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.652+0000 7fee9d9fc640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407ac40 0x7fee8407d100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fee94000c00 tx=0x7fee94031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.656+0000 7fee8effd640 1 -- 172.21.15.46:0/661049662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fee88014030 con 0x7fee98107910 2024-09-10T04:32:32.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:32 smithi046 ceph-mon[109506]: pgmap v807: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:32.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.860+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fee98062630 con 0x7fee8407ac40 2024-09-10T04:32:32.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.861+0000 7fee8effd640 1 -- 172.21.15.46:0/661049662 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fee98062630 con 0x7fee8407ac40 2024-09-10T04:32:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407ac40 msgr2=0x7fee8407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407ac40 0x7fee8407d100 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fee94000c00 tx=0x7fee94031400 comp rx=0 tx=0).stop 2024-09-10T04:32:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 msgr2=0x7fee98074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98074a30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c810 tx=0x7fee8800ccd0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 shutdown_connections 2024-09-10T04:32:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fee8407ac40 0x7fee8407d100 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee98107910 0x7fee98074a30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 --2- 172.21.15.46:0/661049662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fee98101910 0x7fee980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 >> 172.21.15.46:0/661049662 conn(0x7fee980fd630 msgr2=0x7fee980fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 shutdown_connections 2024-09-10T04:32:32.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.863+0000 7fee9fc60640 1 -- 172.21.15.46:0/661049662 wait complete. 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 -- 172.21.15.46:0/1001776227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 msgr2=0x7f01b0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 --2- 172.21.15.46:0/1001776227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b0107ce0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f01ac009880 tx=0x7f01ac02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 -- 172.21.15.46:0/1001776227 shutdown_connections 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 --2- 172.21.15.46:0/1001776227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01b0101900 0x7f01b0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 --2- 172.21.15.46:0/1001776227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b0107ce0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 -- 172.21.15.46:0/1001776227 >> 172.21.15.46:0/1001776227 conn(0x7f01b00fd600 msgr2=0x7f01b00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 -- 172.21.15.46:0/1001776227 shutdown_connections 2024-09-10T04:32:32.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 -- 172.21.15.46:0/1001776227 wait complete. 2024-09-10T04:32:32.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 Processor -- start 2024-09-10T04:32:32.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.948+0000 7f01b80d2640 1 -- start start 2024-09-10T04:32:32.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b80d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01b0101900 0x7f01b019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b80d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b80d2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010edb0 con 0x7f01b0107900 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b80d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f01b010ec30 con 0x7f01b0101900 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b566d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b566d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39802/0 (socket says 172.21.15.46:39802) 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b566d640 1 -- 172.21.15.46:0/935400744 learned_addr learned my addr 172.21.15.46:0/935400744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b5e6e640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01b0101900 0x7f01b019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b566d640 1 -- 172.21.15.46:0/935400744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01b0101900 msgr2=0x7f01b019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b566d640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01b0101900 0x7f01b019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b566d640 1 -- 172.21.15.46:0/935400744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b01977c0 con 0x7f01b0107900 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b5e6e640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01b0101900 0x7f01b019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01b566d640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b019db80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f01a000d8d0 tx=0x7f01a000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:32.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.949+0000 7f01a6ffd640 1 -- 172.21.15.46:0/935400744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01a0017070 con 0x7f01b0107900 2024-09-10T04:32:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.950+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b0197ab0 con 0x7f01b0107900 2024-09-10T04:32:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.950+0000 7f01a6ffd640 1 -- 172.21.15.46:0/935400744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01a0010460 con 0x7f01b0107900 2024-09-10T04:32:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.950+0000 7f01a6ffd640 1 -- 172.21.15.46:0/935400744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f01a000f5f0 con 0x7f01b0107900 2024-09-10T04:32:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.950+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b0198040 con 0x7f01b0107900 2024-09-10T04:32:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.951+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b0103040 con 0x7f01b0107900 2024-09-10T04:32:32.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.951+0000 7f01a6ffd640 1 -- 172.21.15.46:0/935400744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f01a0009a30 con 0x7f01b0107900 2024-09-10T04:32:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.952+0000 7f01a6ffd640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f019c07abf0 0x7f019c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.952+0000 7f01a6ffd640 1 -- 172.21.15.46:0/935400744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f01a009db60 con 0x7f01b0107900 2024-09-10T04:32:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.952+0000 7f01b5e6e640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f019c07abf0 0x7f019c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:32.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.952+0000 7f01b5e6e640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f019c07abf0 0x7f019c07d0b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006010 tx=0x7f01ac0095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:32.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:32.956+0000 7f01a6ffd640 1 -- 172.21.15.46:0/935400744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f01a00692a0 con 0x7f01b0107900 2024-09-10T04:32:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:32 smithi070 ceph-mon[79774]: pgmap v807: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:33.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.148+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f01b01982f0 con 0x7f019c07abf0 2024-09-10T04:32:33.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.158+0000 7f01a6ffd640 1 -- 172.21.15.46:0/935400744 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f01b01982f0 con 0x7f019c07abf0 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (24m) 3m ago 35m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (26m) 3m ago 35m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (26m) 3m ago 34m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (22m) 3m ago 35m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (22m) 3m ago 34m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (23m) 3m ago 35m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (18m) 3m ago 30m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (18m) 3m ago 30m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (18m) 3m ago 30m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:32:33.160 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (3m) 3m ago 30m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (26m) 3m ago 36m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (25m) 3m ago 34m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (23m) 3m ago 37m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (22m) 3m ago 34m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (24m) 3m ago 35m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (24m) 3m ago 34m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (22m) 3m ago 33m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (21m) 3m ago 32m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (20m) 3m ago 32m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (20m) 3m ago 32m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (19m) 3m ago 31m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (19m) 3m ago 31m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:32:33.161 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (24m) 3m ago 35m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:32:33.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f019c07abf0 msgr2=0x7f019c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f019c07abf0 0x7f019c07d0b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006010 tx=0x7f01ac0095f0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 msgr2=0x7f01b019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b019db80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f01a000d8d0 tx=0x7f01a000dd90 comp rx=0 tx=0).stop 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 shutdown_connections 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f019c07abf0 0x7f019c07d0b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01b0107900 0x7f01b019db80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 --2- 172.21.15.46:0/935400744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01b0101900 0x7f01b019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 >> 172.21.15.46:0/935400744 conn(0x7f01b00fd600 msgr2=0x7f01b00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 shutdown_connections 2024-09-10T04:32:33.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.160+0000 7f01b80d2640 1 -- 172.21.15.46:0/935400744 wait complete. 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 -- 172.21.15.46:0/1911104131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 msgr2=0x7f0070107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 --2- 172.21.15.46:0/1911104131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070107ce0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0060009880 tx=0x7f006002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 -- 172.21.15.46:0/1911104131 shutdown_connections 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 --2- 172.21.15.46:0/1911104131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0070101900 0x7f0070101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 --2- 172.21.15.46:0/1911104131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070107ce0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 -- 172.21.15.46:0/1911104131 >> 172.21.15.46:0/1911104131 conn(0x7f00700fd600 msgr2=0x7f00700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 -- 172.21.15.46:0/1911104131 shutdown_connections 2024-09-10T04:32:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.243+0000 7f0074e41640 1 -- 172.21.15.46:0/1911104131 wait complete. 2024-09-10T04:32:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.244+0000 7f0074e41640 1 Processor -- start 2024-09-10T04:32:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.244+0000 7f0074e41640 1 -- start start 2024-09-10T04:32:33.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.244+0000 7f0074e41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0070101900 0x7f00700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f0074e41640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f0074e41640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f007010edd0 con 0x7f0070107900 2024-09-10T04:32:33.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f0074e41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f007010ec50 con 0x7f0070101900 2024-09-10T04:32:33.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39812/0 (socket says 172.21.15.46:39812) 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006dd9b640 1 -- 172.21.15.46:0/255297271 learned_addr learned my addr 172.21.15.46:0/255297271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006e59c640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0070101900 0x7f00700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006dd9b640 1 -- 172.21.15.46:0/255297271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0070101900 msgr2=0x7f00700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006dd9b640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0070101900 0x7f00700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006dd9b640 1 -- 172.21.15.46:0/255297271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0070074f70 con 0x7f0070107900 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006e59c640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0070101900 0x7f00700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f006dd9b640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070074a30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f006400d6f0 tx=0x7f006400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f005f7fe640 1 -- 172.21.15.46:0/255297271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0064017070 con 0x7f0070107900 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0070075260 con 0x7f0070107900 2024-09-10T04:32:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f005f7fe640 1 -- 172.21.15.46:0/255297271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f006400de00 con 0x7f0070107900 2024-09-10T04:32:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f005f7fe640 1 -- 172.21.15.46:0/255297271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0064004bb0 con 0x7f0070107900 2024-09-10T04:32:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.245+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701a3e20 con 0x7f0070107900 2024-09-10T04:32:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.247+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0070103040 con 0x7f0070107900 2024-09-10T04:32:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.247+0000 7f005f7fe640 1 -- 172.21.15.46:0/255297271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0064004e60 con 0x7f0070107900 2024-09-10T04:32:33.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.248+0000 7f005f7fe640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f005807abf0 0x7f005807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.248+0000 7f005f7fe640 1 -- 172.21.15.46:0/255297271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f006409d8b0 con 0x7f0070107900 2024-09-10T04:32:33.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.248+0000 7f006e59c640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f005807abf0 0x7f005807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.249+0000 7f006e59c640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f005807abf0 0x7f005807d0b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:33.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.253+0000 7f005f7fe640 1 -- 172.21.15.46:0/255297271 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0064068ff0 con 0x7f0070107900 2024-09-10T04:32:33.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.515+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f0070109e70 con 0x7f0070107900 2024-09-10T04:32:33.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.515+0000 7f005f7fe640 1 -- 172.21.15.46:0/255297271 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f006406dea0 con 0x7f0070107900 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:33.518 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:32:33.519 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:32:33.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:32:33.519 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:32:33.519 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:32:33.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:32:33.519 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f005807abf0 msgr2=0x7f005807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f005807abf0 0x7f005807d0b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0060000c00 tx=0x7f0060031400 comp rx=0 tx=0).stop 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 msgr2=0x7f0070074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070074a30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f006400d6f0 tx=0x7f006400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 shutdown_connections 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f005807abf0 0x7f005807d0b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0070107900 0x7f0070074a30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 --2- 172.21.15.46:0/255297271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0070101900 0x7f00700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 >> 172.21.15.46:0/255297271 conn(0x7f00700fd600 msgr2=0x7f00700fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:33.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 shutdown_connections 2024-09-10T04:32:33.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.518+0000 7f0074e41640 1 -- 172.21.15.46:0/255297271 wait complete. 2024-09-10T04:32:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 -- 172.21.15.46:0/4286251082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 msgr2=0x7f2bcc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/4286251082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc10b8d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009880 tx=0x7f2bb802a530 comp rx=0 tx=0).stop 2024-09-10T04:32:33.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 -- 172.21.15.46:0/4286251082 shutdown_connections 2024-09-10T04:32:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/4286251082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc10b8d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/4286251082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 -- 172.21.15.46:0/4286251082 >> 172.21.15.46:0/4286251082 conn(0x7f2bcc0fc520 msgr2=0x7f2bcc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 -- 172.21.15.46:0/4286251082 shutdown_connections 2024-09-10T04:32:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.602+0000 7f2bd2b41640 1 -- 172.21.15.46:0/4286251082 wait complete. 2024-09-10T04:32:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bd2b41640 1 Processor -- start 2024-09-10T04:32:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bd2b41640 1 -- start start 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bd2b41640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bd2b41640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bd2b41640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10edf0 con 0x7f2bcc1033e0 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bd2b41640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10ec70 con 0x7f2bcc102ac0 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bcbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bcbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39820/0 (socket says 172.21.15.46:39820) 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.603+0000 7f2bcbfff640 1 -- 172.21.15.46:0/1195297608 learned_addr learned my addr 172.21.15.46:0/1195297608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bd08dd640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bcbfff640 1 -- 172.21.15.46:0/1195297608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bcc102ac0 msgr2=0x7f2bcc19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bcbfff640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bcbfff640 1 -- 172.21.15.46:0/1195297608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc197800 con 0x7f2bcc1033e0 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bcbfff640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc19dbc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8031460 tx=0x7f2bb8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bc9ffb640 1 -- 172.21.15.46:0/1195297608 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bb803f070 con 0x7f2bcc1033e0 2024-09-10T04:32:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc197a90 con 0x7f2bcc1033e0 2024-09-10T04:32:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bc9ffb640 1 -- 172.21.15.46:0/1195297608 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bb8004a90 con 0x7f2bcc1033e0 2024-09-10T04:32:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bc9ffb640 1 -- 172.21.15.46:0/1195297608 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bb8035030 con 0x7f2bcc1033e0 2024-09-10T04:32:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.604+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc197fc0 con 0x7f2bcc1033e0 2024-09-10T04:32:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.605+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bcc06aa90 con 0x7f2bcc1033e0 2024-09-10T04:32:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.606+0000 7f2bc9ffb640 1 -- 172.21.15.46:0/1195297608 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2bb8042070 con 0x7f2bcc1033e0 2024-09-10T04:32:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.607+0000 7f2bc9ffb640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bbc07ac00 0x7f2bbc07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.607+0000 7f2bc9ffb640 1 -- 172.21.15.46:0/1195297608 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2bb80be830 con 0x7f2bcc1033e0 2024-09-10T04:32:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.607+0000 7f2bd08dd640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bbc07ac00 0x7f2bbc07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.607+0000 7f2bd08dd640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bbc07ac00 0x7f2bbc07d0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0005fd0 tx=0x7f2bc0005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:33.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.612+0000 7f2bc9ffb640 1 -- 172.21.15.46:0/1195297608 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2bb8089ed0 con 0x7f2bcc1033e0 2024-09-10T04:32:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:33 smithi046 ceph-mon[109506]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:33.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:33 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/255297271' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:32:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.840+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2bcc105f60 con 0x7f2bcc1033e0 2024-09-10T04:32:33.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.840+0000 7f2bc9ffb640 1 -- 172.21.15.46:0/1195297608 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f2bb808ed80 con 0x7f2bcc1033e0 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:32:33.843 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:32:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:33.845 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:32:33.846 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:32:33.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:33.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:33.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:33.846 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:32:33.846 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bbc07ac00 msgr2=0x7f2bbc07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bbc07ac00 0x7f2bbc07d0c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0005fd0 tx=0x7f2bc0005850 comp rx=0 tx=0).stop 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 msgr2=0x7f2bcc19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc19dbc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8031460 tx=0x7f2bb8004530 comp rx=0 tx=0).stop 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 shutdown_connections 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2bbc07ac00 0x7f2bbc07d0c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bcc1033e0 0x7f2bcc19dbc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 --2- 172.21.15.46:0/1195297608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bcc102ac0 0x7f2bcc19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 >> 172.21.15.46:0/1195297608 conn(0x7f2bcc0fc520 msgr2=0x7f2bcc108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 shutdown_connections 2024-09-10T04:32:33.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.843+0000 7f2bd2b41640 1 -- 172.21.15.46:0/1195297608 wait complete. 2024-09-10T04:32:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 -- 172.21.15.46:0/1981147244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c106800 msgr2=0x7eff6c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 --2- 172.21.15.46:0/1981147244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c106800 0x7eff6c106be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7eff5c009880 tx=0x7eff5c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 -- 172.21.15.46:0/1981147244 shutdown_connections 2024-09-10T04:32:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 --2- 172.21.15.46:0/1981147244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff6c100800 0x7eff6c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 --2- 172.21.15.46:0/1981147244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c106800 0x7eff6c106be0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 -- 172.21.15.46:0/1981147244 >> 172.21.15.46:0/1981147244 conn(0x7eff6c0fc520 msgr2=0x7eff6c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 -- 172.21.15.46:0/1981147244 shutdown_connections 2024-09-10T04:32:33.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.928+0000 7eff70df9640 1 -- 172.21.15.46:0/1981147244 wait complete. 2024-09-10T04:32:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff70df9640 1 Processor -- start 2024-09-10T04:32:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff70df9640 1 -- start start 2024-09-10T04:32:33.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff70df9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c100800 0x7eff6c107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff70df9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff6c106800 0x7eff6c107c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff70df9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c10dcb0 con 0x7eff6c100800 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff70df9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c10db30 con 0x7eff6c106800 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff6a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c100800 0x7eff6c107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff6a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c100800 0x7eff6c107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39830/0 (socket says 172.21.15.46:39830) 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff6a59c640 1 -- 172.21.15.46:0/2233270471 learned_addr learned my addr 172.21.15.46:0/2233270471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.929+0000 7eff63fff640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff6c106800 0x7eff6c107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff6a59c640 1 -- 172.21.15.46:0/2233270471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff6c106800 msgr2=0x7eff6c107c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff6a59c640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff6c106800 0x7eff6c107c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff6a59c640 1 -- 172.21.15.46:0/2233270471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c1081b0 con 0x7eff6c100800 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff63fff640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff6c106800 0x7eff6c107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff6a59c640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c100800 0x7eff6c107730 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7eff5c031460 tx=0x7eff5c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:33.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff637fe640 1 -- 172.21.15.46:0/2233270471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7eff5c03f070 con 0x7eff6c100800 2024-09-10T04:32:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff637fe640 1 -- 172.21.15.46:0/2233270471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7eff5c0035f0 con 0x7eff6c100800 2024-09-10T04:32:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c108440 con 0x7eff6c100800 2024-09-10T04:32:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff637fe640 1 -- 172.21.15.46:0/2233270471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7eff5c035030 con 0x7eff6c100800 2024-09-10T04:32:33.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.930+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1a3d50 con 0x7eff6c100800 2024-09-10T04:32:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.931+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c101f40 con 0x7eff6c100800 2024-09-10T04:32:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.932+0000 7eff637fe640 1 -- 172.21.15.46:0/2233270471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7eff5c042070 con 0x7eff6c100800 2024-09-10T04:32:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.933+0000 7eff637fe640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7eff5807abf0 0x7eff5807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.933+0000 7eff637fe640 1 -- 172.21.15.46:0/2233270471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7eff5c0becc0 con 0x7eff6c100800 2024-09-10T04:32:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.933+0000 7eff63fff640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7eff5807abf0 0x7eff5807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:33.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.933+0000 7eff63fff640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7eff5807abf0 0x7eff5807d0b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7eff6c1091c0 tx=0x7eff54005e50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:33.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:33.936+0000 7eff637fe640 1 -- 172.21.15.46:0/2233270471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7eff5c08a480 con 0x7eff6c100800 2024-09-10T04:32:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:33 smithi070 ceph-mon[79774]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:33 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/255297271' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:32:34.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.139+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7eff6c06aa00 con 0x7eff5807abf0 2024-09-10T04:32:34.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.140+0000 7eff637fe640 1 -- 172.21.15.46:0/2233270471 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7eff6c06aa00 con 0x7eff5807abf0 2024-09-10T04:32:34.142 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:32:34.142 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:32:34.142 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:32:34.142 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:32:34.143 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:32:34.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7eff5807abf0 msgr2=0x7eff5807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:34.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7eff5807abf0 0x7eff5807d0b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7eff6c1091c0 tx=0x7eff54005e50 comp rx=0 tx=0).stop 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c100800 msgr2=0x7eff6c107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c100800 0x7eff6c107730 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7eff5c031460 tx=0x7eff5c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 shutdown_connections 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7eff5807abf0 0x7eff5807d0b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff6c106800 0x7eff6c107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 --2- 172.21.15.46:0/2233270471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff6c100800 0x7eff6c107730 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.142+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 >> 172.21.15.46:0/2233270471 conn(0x7eff6c0fc520 msgr2=0x7eff6c06a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:34.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.143+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 shutdown_connections 2024-09-10T04:32:34.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.143+0000 7eff70df9640 1 -- 172.21.15.46:0/2233270471 wait complete. 2024-09-10T04:32:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 -- 172.21.15.46:0/3513639118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100250 msgr2=0x7f6c7c100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 --2- 172.21.15.46:0/3513639118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100250 0x7f6c7c100630 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009880 tx=0x7f6c7802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 -- 172.21.15.46:0/3513639118 shutdown_connections 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 --2- 172.21.15.46:0/3513639118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7c100c00 0x7f6c7c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 --2- 172.21.15.46:0/3513639118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100250 0x7f6c7c100630 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 -- 172.21.15.46:0/3513639118 >> 172.21.15.46:0/3513639118 conn(0x7f6c7c0fbc90 msgr2=0x7f6c7c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 -- 172.21.15.46:0/3513639118 shutdown_connections 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.227+0000 7f6c83643640 1 -- 172.21.15.46:0/3513639118 wait complete. 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.228+0000 7f6c83643640 1 Processor -- start 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.228+0000 7f6c83643640 1 -- start start 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.228+0000 7f6c83643640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7c100250 0x7f6c7c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.228+0000 7f6c83643640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100c00 0x7f6c7c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.228+0000 7f6c83643640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10ee70 con 0x7f6c7c100c00 2024-09-10T04:32:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.228+0000 7f6c83643640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c10ecf0 con 0x7f6c7c100250 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c80bde640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100c00 0x7f6c7c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c80bde640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100c00 0x7f6c7c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39856/0 (socket says 172.21.15.46:39856) 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c80bde640 1 -- 172.21.15.46:0/1072415851 learned_addr learned my addr 172.21.15.46:0/1072415851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c813df640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7c100250 0x7f6c7c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c80bde640 1 -- 172.21.15.46:0/1072415851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7c100250 msgr2=0x7f6c7c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c80bde640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7c100250 0x7f6c7c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c80bde640 1 -- 172.21.15.46:0/1072415851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c197880 con 0x7f6c7c100c00 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c80bde640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100c00 0x7f6c7c19dc00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e970 tx=0x7f6c6c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c727fc640 1 -- 172.21.15.46:0/1072415851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6c6c00cac0 con 0x7f6c7c100c00 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c197b70 con 0x7f6c7c100c00 2024-09-10T04:32:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c727fc640 1 -- 172.21.15.46:0/1072415851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6c6c0054e0 con 0x7f6c7c100c00 2024-09-10T04:32:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c727fc640 1 -- 172.21.15.46:0/1072415851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6c6c010430 con 0x7f6c7c100c00 2024-09-10T04:32:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.229+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c198100 con 0x7f6c7c100c00 2024-09-10T04:32:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.230+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c101730 con 0x7f6c7c100c00 2024-09-10T04:32:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.231+0000 7f6c727fc640 1 -- 172.21.15.46:0/1072415851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6c6c004ea0 con 0x7f6c7c100c00 2024-09-10T04:32:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.232+0000 7f6c727fc640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c6807abf0 0x7f6c6807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:32:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.232+0000 7f6c727fc640 1 -- 172.21.15.46:0/1072415851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6c6c09e5b0 con 0x7f6c7c100c00 2024-09-10T04:32:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.232+0000 7f6c813df640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c6807abf0 0x7f6c6807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:32:34.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.232+0000 7f6c813df640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c6807abf0 0x7f6c6807d0b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:32:34.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.236+0000 7f6c727fc640 1 -- 172.21.15.46:0/1072415851 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6c6c069cd0 con 0x7f6c7c100c00 2024-09-10T04:32:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.496+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f6c7c198db0 con 0x7f6c7c100c00 2024-09-10T04:32:34.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.497+0000 7f6c727fc640 1 -- 172.21.15.46:0/1072415851 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f6c6c06eb80 con 0x7f6c7c100c00 2024-09-10T04:32:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:32:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:32:34.499 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:32:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:32:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:32:34.499 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:32:34.500 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:32:34.501 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:32:34.501 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:32:34.501 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:32:34.501 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:32:34.501 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:32:34.501 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:32:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c6807abf0 msgr2=0x7f6c6807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c6807abf0 0x7f6c6807d0b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6c78000c00 tx=0x7f6c78031400 comp rx=0 tx=0).stop 2024-09-10T04:32:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100c00 msgr2=0x7f6c7c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:32:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100c00 0x7f6c7c19dc00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00e970 tx=0x7f6c6c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:32:34.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 shutdown_connections 2024-09-10T04:32:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6c6807abf0 0x7f6c6807d0b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c7c100c00 0x7f6c7c19dc00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 --2- 172.21.15.46:0/1072415851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c7c100250 0x7f6c7c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:32:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.500+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 >> 172.21.15.46:0/1072415851 conn(0x7f6c7c0fbc90 msgr2=0x7f6c7c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:32:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.501+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 shutdown_connections 2024-09-10T04:32:34.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:32:34.501+0000 7f6c83643640 1 -- 172.21.15.46:0/1072415851 wait complete. 2024-09-10T04:32:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:34 smithi046 ceph-mon[109506]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:34 smithi046 ceph-mon[109506]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:34 smithi046 ceph-mon[109506]: pgmap v808: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:34 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1195297608' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:32:34.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:34 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1072415851' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:32:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:34 smithi070 ceph-mon[79774]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:34 smithi070 ceph-mon[79774]: from='client.35326 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:34 smithi070 ceph-mon[79774]: pgmap v808: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:34 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1195297608' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:32:35.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:34 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1072415851' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:32:35.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:35 smithi070 ceph-mon[79774]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:36.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:35 smithi046 ceph-mon[109506]: from='client.35338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:32:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:36 smithi070 ceph-mon[79774]: pgmap v809: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:36 smithi046 ceph-mon[109506]: pgmap v809: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:38 smithi070 ceph-mon[79774]: pgmap v810: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:38 smithi046 ceph-mon[109506]: pgmap v810: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:40 smithi070 ceph-mon[79774]: pgmap v811: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:40 smithi046 ceph-mon[109506]: pgmap v811: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:42 smithi070 ceph-mon[79774]: pgmap v812: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:42 smithi046 ceph-mon[109506]: pgmap v812: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:44 smithi070 ceph-mon[79774]: pgmap v813: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:44 smithi046 ceph-mon[109506]: pgmap v813: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:46 smithi070 ceph-mon[79774]: pgmap v814: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:46 smithi046 ceph-mon[109506]: pgmap v814: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:48 smithi070 ceph-mon[79774]: pgmap v815: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:48 smithi046 ceph-mon[109506]: pgmap v815: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:50.887 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:50 smithi070 ceph-mon[79774]: pgmap v816: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:50 smithi046 ceph-mon[109506]: pgmap v816: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:52 smithi070 ceph-mon[79774]: pgmap v817: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:52 smithi046 ceph-mon[109506]: pgmap v817: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:54.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:32:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:54 smithi046 ceph-mon[109506]: pgmap v818: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:54 smithi070 ceph-mon[79774]: pgmap v818: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:56 smithi070 ceph-mon[79774]: pgmap v819: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:56 smithi046 ceph-mon[109506]: pgmap v819: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:32:58 smithi070 ceph-mon[79774]: pgmap v820: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:32:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:32:58 smithi046 ceph-mon[109506]: pgmap v820: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:00 smithi070 ceph-mon[79774]: pgmap v821: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:00 smithi046 ceph-mon[109506]: pgmap v821: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:02 smithi070 ceph-mon[79774]: pgmap v822: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:02 smithi046 ceph-mon[109506]: pgmap v822: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.600+0000 7f18616ef640 1 -- 172.21.15.46:0/854262635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 msgr2=0x7f185c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.600+0000 7f18616ef640 1 --2- 172.21.15.46:0/854262635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c1015d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f184c009880 tx=0x7f184c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.600+0000 7f18616ef640 1 -- 172.21.15.46:0/854262635 shutdown_connections 2024-09-10T04:33:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.600+0000 7f18616ef640 1 --2- 172.21.15.46:0/854262635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c101ba0 0x7f185c105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.600+0000 7f18616ef640 1 --2- 172.21.15.46:0/854262635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c1015d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.600+0000 7f18616ef640 1 -- 172.21.15.46:0/854262635 >> 172.21.15.46:0/854262635 conn(0x7f185c0fbc90 msgr2=0x7f185c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.601+0000 7f18616ef640 1 -- 172.21.15.46:0/854262635 shutdown_connections 2024-09-10T04:33:04.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.601+0000 7f18616ef640 1 -- 172.21.15.46:0/854262635 wait complete. 2024-09-10T04:33:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.601+0000 7f18616ef640 1 Processor -- start 2024-09-10T04:33:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.601+0000 7f18616ef640 1 -- start start 2024-09-10T04:33:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f18616ef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f18616ef640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c101ba0 0x7f185c19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f18616ef640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f185c10dd60 con 0x7f185c1011f0 2024-09-10T04:33:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f18616ef640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f185c10dbe0 con 0x7f185c101ba0 2024-09-10T04:33:04.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f185affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f185affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c19fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51966/0 (socket says 172.21.15.46:51966) 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f185affd640 1 -- 172.21.15.46:0/1723887664 learned_addr learned my addr 172.21.15.46:0/1723887664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f18527fc640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c101ba0 0x7f185c19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f185affd640 1 -- 172.21.15.46:0/1723887664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c101ba0 msgr2=0x7f185c19ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f185affd640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c101ba0 0x7f185c19ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f185affd640 1 -- 172.21.15.46:0/1723887664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c199b10 con 0x7f185c1011f0 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.602+0000 7f185affd640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c19fa40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f184c031460 tx=0x7f184c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.603+0000 7f1858ff9640 1 -- 172.21.15.46:0/1723887664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f184c03f070 con 0x7f185c1011f0 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.603+0000 7f1858ff9640 1 -- 172.21.15.46:0/1723887664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f184c0035f0 con 0x7f185c1011f0 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.603+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c199da0 con 0x7f185c1011f0 2024-09-10T04:33:04.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.603+0000 7f1858ff9640 1 -- 172.21.15.46:0/1723887664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f184c035030 con 0x7f185c1011f0 2024-09-10T04:33:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.603+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c19a2a0 con 0x7f185c1011f0 2024-09-10T04:33:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.604+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c1021b0 con 0x7f185c1011f0 2024-09-10T04:33:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.605+0000 7f1858ff9640 1 -- 172.21.15.46:0/1723887664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f184c042070 con 0x7f185c1011f0 2024-09-10T04:33:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.605+0000 7f1858ff9640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f184807abf0 0x7f184807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:04.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.605+0000 7f1858ff9640 1 -- 172.21.15.46:0/1723887664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f184c0be720 con 0x7f185c1011f0 2024-09-10T04:33:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.605+0000 7f18527fc640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f184807abf0 0x7f184807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:04.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.606+0000 7f18527fc640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f184807abf0 0x7f184807d0b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1844005fd0 tx=0x7f1844005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:04.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.610+0000 7f1858ff9640 1 -- 172.21.15.46:0/1723887664 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f184c089dc0 con 0x7f185c1011f0 2024-09-10T04:33:04.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.827+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f185c19a610 con 0x7f184807abf0 2024-09-10T04:33:04.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.828+0000 7f1858ff9640 1 -- 172.21.15.46:0/1723887664 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f185c19a610 con 0x7f184807abf0 2024-09-10T04:33:04.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f184807abf0 msgr2=0x7f184807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f184807abf0 0x7f184807d0b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1844005fd0 tx=0x7f1844005850 comp rx=0 tx=0).stop 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 msgr2=0x7f185c19fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c19fa40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f184c031460 tx=0x7f184c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 shutdown_connections 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f184807abf0 0x7f184807d0b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c101ba0 0x7f185c19ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 --2- 172.21.15.46:0/1723887664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f185c1011f0 0x7f185c19fa40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 >> 172.21.15.46:0/1723887664 conn(0x7f185c0fbc90 msgr2=0x7f185c1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 shutdown_connections 2024-09-10T04:33:04.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.831+0000 7f18616ef640 1 -- 172.21.15.46:0/1723887664 wait complete. 2024-09-10T04:33:04.840 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:33:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.916+0000 7fb8a26f8640 1 -- 172.21.15.46:0/535183710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c106800 msgr2=0x7fb89c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.916+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/535183710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c106800 0x7fb89c106be0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009880 tx=0x7fb88c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.916+0000 7fb8a26f8640 1 -- 172.21.15.46:0/535183710 shutdown_connections 2024-09-10T04:33:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.916+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/535183710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb89c100800 0x7fb89c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.916+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/535183710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c106800 0x7fb89c106be0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.916+0000 7fb8a26f8640 1 -- 172.21.15.46:0/535183710 >> 172.21.15.46:0/535183710 conn(0x7fb89c0fc520 msgr2=0x7fb89c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:04.918 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:04 smithi046 ceph-mon[109506]: pgmap v823: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.916+0000 7fb8a26f8640 1 -- 172.21.15.46:0/535183710 shutdown_connections 2024-09-10T04:33:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.917+0000 7fb8a26f8640 1 -- 172.21.15.46:0/535183710 wait complete. 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.917+0000 7fb8a26f8640 1 Processor -- start 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.917+0000 7fb8a26f8640 1 -- start start 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.917+0000 7fb8a26f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c100800 0x7fb89c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.917+0000 7fb8a26f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb89c106800 0x7fb89c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.917+0000 7fb8a26f8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10dcd0 con 0x7fb89c100800 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.917+0000 7fb8a26f8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb89c10db50 con 0x7fb89c106800 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb89bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c100800 0x7fb89c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb89bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c100800 0x7fb89c19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51978/0 (socket says 172.21.15.46:51978) 2024-09-10T04:33:04.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb89bfff640 1 -- 172.21.15.46:0/2914185034 learned_addr learned my addr 172.21.15.46:0/2914185034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb893fff640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb89c106800 0x7fb89c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb89bfff640 1 -- 172.21.15.46:0/2914185034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb89c106800 msgr2=0x7fb89c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb89bfff640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb89c106800 0x7fb89c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb89bfff640 1 -- 172.21.15.46:0/2914185034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89c1977b0 con 0x7fb89c100800 2024-09-10T04:33:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb89bfff640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c100800 0x7fb89c19d630 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb88c031460 tx=0x7fb88c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:04.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb899ffb640 1 -- 172.21.15.46:0/2914185034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb88c03f070 con 0x7fb89c100800 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb899ffb640 1 -- 172.21.15.46:0/2914185034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb88c0035f0 con 0x7fb89c100800 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89c197a40 con 0x7fb89c100800 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.918+0000 7fb899ffb640 1 -- 172.21.15.46:0/2914185034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb88c035030 con 0x7fb89c100800 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.919+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89c197f70 con 0x7fb89c100800 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.920+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89c101f40 con 0x7fb89c100800 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.920+0000 7fb899ffb640 1 -- 172.21.15.46:0/2914185034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb88c042070 con 0x7fb89c100800 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.921+0000 7fb899ffb640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb88807abf0 0x7fb88807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.921+0000 7fb899ffb640 1 -- 172.21.15.46:0/2914185034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb88c0be5d0 con 0x7fb89c100800 2024-09-10T04:33:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.921+0000 7fb893fff640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb88807abf0 0x7fb88807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:04.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.921+0000 7fb893fff640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb88807abf0 0x7fb88807d0b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb89c104f70 tx=0x7fb88400b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:04.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:04.925+0000 7fb899ffb640 1 -- 172.21.15.46:0/2914185034 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb88c089d10 con 0x7fb89c100800 2024-09-10T04:33:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:04 smithi070 ceph-mon[79774]: pgmap v823: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.132+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb89c1982e0 con 0x7fb88807abf0 2024-09-10T04:33:05.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.133+0000 7fb899ffb640 1 -- 172.21.15.46:0/2914185034 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fb89c1982e0 con 0x7fb88807abf0 2024-09-10T04:33:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb88807abf0 msgr2=0x7fb88807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb88807abf0 0x7fb88807d0b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb89c104f70 tx=0x7fb88400b390 comp rx=0 tx=0).stop 2024-09-10T04:33:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c100800 msgr2=0x7fb89c19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c100800 0x7fb89c19d630 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fb88c031460 tx=0x7fb88c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 shutdown_connections 2024-09-10T04:33:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb88807abf0 0x7fb88807d0b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb89c106800 0x7fb89c19db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 --2- 172.21.15.46:0/2914185034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb89c100800 0x7fb89c19d630 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 >> 172.21.15.46:0/2914185034 conn(0x7fb89c0fc520 msgr2=0x7fb89c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 shutdown_connections 2024-09-10T04:33:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.135+0000 7fb8a26f8640 1 -- 172.21.15.46:0/2914185034 wait complete. 2024-09-10T04:33:05.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 -- 172.21.15.46:0/4258897193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 msgr2=0x7f4640104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 --2- 172.21.15.46:0/4258897193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f4640104c50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4630009880 tx=0x7f463002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 -- 172.21.15.46:0/4258897193 shutdown_connections 2024-09-10T04:33:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 --2- 172.21.15.46:0/4258897193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f4640104c50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 --2- 172.21.15.46:0/4258897193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4640100250 0x7f4640100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 -- 172.21.15.46:0/4258897193 >> 172.21.15.46:0/4258897193 conn(0x7f46400fbc90 msgr2=0x7f46400fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 -- 172.21.15.46:0/4258897193 shutdown_connections 2024-09-10T04:33:05.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.220+0000 7f4647fdc640 1 -- 172.21.15.46:0/4258897193 wait complete. 2024-09-10T04:33:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4647fdc640 1 Processor -- start 2024-09-10T04:33:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4647fdc640 1 -- start start 2024-09-10T04:33:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4647fdc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4640100250 0x7f464019d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4647fdc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f464019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4647fdc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f464010d4b0 con 0x7f4640100c00 2024-09-10T04:33:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4647fdc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f464010d330 con 0x7f4640100250 2024-09-10T04:33:05.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4645577640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f464019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4645577640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f464019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52002/0 (socket says 172.21.15.46:52002) 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4645577640 1 -- 172.21.15.46:0/1337028596 learned_addr learned my addr 172.21.15.46:0/1337028596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.221+0000 7f4645d78640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4640100250 0x7f464019d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4645577640 1 -- 172.21.15.46:0/1337028596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4640100250 msgr2=0x7f464019d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4645577640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4640100250 0x7f464019d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4645577640 1 -- 172.21.15.46:0/1337028596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4640197900 con 0x7f4640100c00 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4645d78640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4640100250 0x7f464019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4645577640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f464019dc80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4630031460 tx=0x7f46300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4636ffd640 1 -- 172.21.15.46:0/1337028596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f463003f070 con 0x7f4640100c00 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4636ffd640 1 -- 172.21.15.46:0/1337028596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4630004cf0 con 0x7f4640100c00 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4636ffd640 1 -- 172.21.15.46:0/1337028596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4630035030 con 0x7f4640100c00 2024-09-10T04:33:05.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4640197b90 con 0x7f4640100c00 2024-09-10T04:33:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.222+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46401980c0 con 0x7f4640100c00 2024-09-10T04:33:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.223+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4640101730 con 0x7f4640100c00 2024-09-10T04:33:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.224+0000 7f4636ffd640 1 -- 172.21.15.46:0/1337028596 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4630042070 con 0x7f4640100c00 2024-09-10T04:33:05.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.225+0000 7f4636ffd640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f462c07abf0 0x7f462c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.225+0000 7f4645d78640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f462c07abf0 0x7f462c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.225+0000 7f4636ffd640 1 -- 172.21.15.46:0/1337028596 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f46300be750 con 0x7f4640100c00 2024-09-10T04:33:05.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.225+0000 7f4645d78640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f462c07abf0 0x7f462c07d0b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f463c005c40 tx=0x7f463c00a430 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:05.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.229+0000 7f4636ffd640 1 -- 172.21.15.46:0/1337028596 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4630089e90 con 0x7f4640100c00 2024-09-10T04:33:05.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.429+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4640109d50 con 0x7f462c07abf0 2024-09-10T04:33:05.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.438+0000 7f4636ffd640 1 -- 172.21.15.46:0/1337028596 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f463c00a600 con 0x7f462c07abf0 2024-09-10T04:33:05.439 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:33:05.439 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (24m) 3m ago 35m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:33:05.439 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (27m) 3m ago 36m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (27m) 3m ago 35m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (23m) 3m ago 36m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (23m) 3m ago 35m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (24m) 3m ago 35m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (19m) 3m ago 30m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (19m) 3m ago 30m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (18m) 3m ago 30m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (3m) 3m ago 30m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (26m) 3m ago 37m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (25m) 3m ago 35m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (23m) 3m ago 37m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (23m) 3m ago 35m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:33:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (25m) 3m ago 36m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (25m) 3m ago 35m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (22m) 3m ago 34m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (22m) 3m ago 33m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (21m) 3m ago 32m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (20m) 3m ago 32m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (20m) 3m ago 32m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (19m) 3m ago 31m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:33:05.441 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (25m) 3m ago 35m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f462c07abf0 msgr2=0x7f462c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f462c07abf0 0x7f462c07d0b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f463c005c40 tx=0x7f463c00a430 comp rx=0 tx=0).stop 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 msgr2=0x7f464019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f464019dc80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4630031460 tx=0x7f46300048f0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 shutdown_connections 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f462c07abf0 0x7f462c07d0b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4640100c00 0x7f464019dc80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 --2- 172.21.15.46:0/1337028596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4640100250 0x7f464019d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.440+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 >> 172.21.15.46:0/1337028596 conn(0x7f46400fbc90 msgr2=0x7f4640106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.441+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 shutdown_connections 2024-09-10T04:33:05.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.441+0000 7f4647fdc640 1 -- 172.21.15.46:0/1337028596 wait complete. 2024-09-10T04:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 -- 172.21.15.46:0/3073310205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 msgr2=0x7f0730102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 --2- 172.21.15.46:0/3073310205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f0730102f70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0720009880 tx=0x7f072002a530 comp rx=0 tx=0).stop 2024-09-10T04:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 -- 172.21.15.46:0/3073310205 shutdown_connections 2024-09-10T04:33:05.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 --2- 172.21.15.46:0/3073310205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07301034b0 0x7f073010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 --2- 172.21.15.46:0/3073310205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f0730102f70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 -- 172.21.15.46:0/3073310205 >> 172.21.15.46:0/3073310205 conn(0x7f07300fc510 msgr2=0x7f07300fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 -- 172.21.15.46:0/3073310205 shutdown_connections 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.524+0000 7f0734d0c640 1 -- 172.21.15.46:0/3073310205 wait complete. 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f0734d0c640 1 Processor -- start 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f0734d0c640 1 -- start start 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f0734d0c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f07301007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f0734d0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07301034b0 0x7f0730100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f0734d0c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f073010ee00 con 0x7f0730102b90 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f0734d0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f073010ec80 con 0x7f07301034b0 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f072e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f07301007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f072e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f07301007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52008/0 (socket says 172.21.15.46:52008) 2024-09-10T04:33:05.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f072e59c640 1 -- 172.21.15.46:0/2835169739 learned_addr learned my addr 172.21.15.46:0/2835169739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:05.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.525+0000 7f072dd9b640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07301034b0 0x7f0730100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f072dd9b640 1 -- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 msgr2=0x7f07301007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f072dd9b640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f07301007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f072dd9b640 1 -- 172.21.15.46:0/2835169739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0730101270 con 0x7f07301034b0 2024-09-10T04:33:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f072e59c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f07301007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:33:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f072dd9b640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07301034b0 0x7f0730100d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f072400e970 tx=0x7f072400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f071f7fe640 1 -- 172.21.15.46:0/2835169739 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f072400cac0 con 0x7f07301034b0 2024-09-10T04:33:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0730101560 con 0x7f07301034b0 2024-09-10T04:33:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f071f7fe640 1 -- 172.21.15.46:0/2835169739 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f072400cc60 con 0x7f07301034b0 2024-09-10T04:33:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f071f7fe640 1 -- 172.21.15.46:0/2835169739 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f07240047a0 con 0x7f07301034b0 2024-09-10T04:33:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.526+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07301a3e70 con 0x7f07301034b0 2024-09-10T04:33:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.527+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f073006ace0 con 0x7f07301034b0 2024-09-10T04:33:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.528+0000 7f071f7fe640 1 -- 172.21.15.46:0/2835169739 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0724004a60 con 0x7f07301034b0 2024-09-10T04:33:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.529+0000 7f071f7fe640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f071807ac40 0x7f071807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.529+0000 7f072e59c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f071807ac40 0x7f071807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.529+0000 7f071f7fe640 1 -- 172.21.15.46:0/2835169739 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f072409e0f0 con 0x7f07301034b0 2024-09-10T04:33:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.529+0000 7f072e59c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f071807ac40 0x7f071807d100 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0720009230 tx=0x7f07200058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.533+0000 7f071f7fe640 1 -- 172.21.15.46:0/2835169739 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0724069810 con 0x7f07301034b0 2024-09-10T04:33:05.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.798+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f0730104fe0 con 0x7f07301034b0 2024-09-10T04:33:05.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.799+0000 7f071f7fe640 1 -- 172.21.15.46:0/2835169739 <== mon.1 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f072406e6c0 con 0x7f07301034b0 2024-09-10T04:33:05.800 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:33:05.800 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:33:05.801 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:33:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:33:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.801+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f071807ac40 msgr2=0x7f071807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.801+0000 7f0734d0c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f071807ac40 0x7f071807d100 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0720009230 tx=0x7f07200058d0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07301034b0 msgr2=0x7f0730100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07301034b0 0x7f0730100d30 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f072400e970 tx=0x7f072400ee30 comp rx=0 tx=0).stop 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 shutdown_connections 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f071807ac40 0x7f071807d100 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07301034b0 0x7f0730100d30 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 --2- 172.21.15.46:0/2835169739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0730102b90 0x7f07301007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 >> 172.21.15.46:0/2835169739 conn(0x7f07300fc510 msgr2=0x7f0730107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 shutdown_connections 2024-09-10T04:33:05.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.802+0000 7f0734d0c640 1 -- 172.21.15.46:0/2835169739 wait complete. 2024-09-10T04:33:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 -- 172.21.15.46:0/2664192944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 msgr2=0x7f2f24100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 --2- 172.21.15.46:0/2664192944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f24100c50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009880 tx=0x7f2f1802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 -- 172.21.15.46:0/2664192944 shutdown_connections 2024-09-10T04:33:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 --2- 172.21.15.46:0/2664192944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f24100c50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 --2- 172.21.15.46:0/2664192944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f241067f0 0x7f2f24106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 -- 172.21.15.46:0/2664192944 >> 172.21.15.46:0/2664192944 conn(0x7f2f240fc550 msgr2=0x7f2f240fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 -- 172.21.15.46:0/2664192944 shutdown_connections 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 -- 172.21.15.46:0/2664192944 wait complete. 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.887+0000 7f2f297be640 1 Processor -- start 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f297be640 1 -- start start 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f297be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f241987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f297be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f241067f0 0x7f2f24198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f297be640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2410dcc0 con 0x7f2f241007f0 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f297be640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2410db40 con 0x7f2f241067f0 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f22ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f241987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f22ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f241987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52024/0 (socket says 172.21.15.46:52024) 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f22ffd640 1 -- 172.21.15.46:0/3380221197 learned_addr learned my addr 172.21.15.46:0/3380221197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f227fc640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f241067f0 0x7f2f24198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f22ffd640 1 -- 172.21.15.46:0/3380221197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f241067f0 msgr2=0x7f2f24198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f22ffd640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f241067f0 0x7f2f24198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f22ffd640 1 -- 172.21.15.46:0/3380221197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f24199330 con 0x7f2f241007f0 2024-09-10T04:33:05.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.888+0000 7f2f22ffd640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f241987f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400eb10 tx=0x7f2f1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.889+0000 7f2f0bfff640 1 -- 172.21.15.46:0/3380221197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f1400cce0 con 0x7f2f241007f0 2024-09-10T04:33:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.889+0000 7f2f0bfff640 1 -- 172.21.15.46:0/3380221197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2f14005020 con 0x7f2f241007f0 2024-09-10T04:33:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.889+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f24199620 con 0x7f2f241007f0 2024-09-10T04:33:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.889+0000 7f2f0bfff640 1 -- 172.21.15.46:0/3380221197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f14010660 con 0x7f2f241007f0 2024-09-10T04:33:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.889+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f241a1c20 con 0x7f2f241007f0 2024-09-10T04:33:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.890+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f24101f30 con 0x7f2f241007f0 2024-09-10T04:33:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.890+0000 7f2f0bfff640 1 -- 172.21.15.46:0/3380221197 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2f140051d0 con 0x7f2f241007f0 2024-09-10T04:33:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.891+0000 7f2f0bfff640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2f0c07ac40 0x7f2f0c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.891+0000 7f2f0bfff640 1 -- 172.21.15.46:0/3380221197 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2f1409e3d0 con 0x7f2f241007f0 2024-09-10T04:33:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.891+0000 7f2f227fc640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2f0c07ac40 0x7f2f0c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.892+0000 7f2f227fc640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2f0c07ac40 0x7f2f0c07d100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f2406a7f0 tx=0x7f2f18031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:05.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:05.895+0000 7f2f0bfff640 1 -- 172.21.15.46:0/3380221197 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2f24199620 con 0x7f2f241007f0 2024-09-10T04:33:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.118+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2f24108d60 con 0x7f2f241007f0 2024-09-10T04:33:06.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.119+0000 7f2f0bfff640 1 -- 172.21.15.46:0/3380221197 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f2f14069b90 con 0x7f2f241007f0 2024-09-10T04:33:06.119 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:33:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:33:06.121 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:06.122 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:33:06.123 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:06.123 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:06.123 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:06.123 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:06.123 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:33:06.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2f0c07ac40 msgr2=0x7f2f0c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2f0c07ac40 0x7f2f0c07d100 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f2406a7f0 tx=0x7f2f18031400 comp rx=0 tx=0).stop 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 msgr2=0x7f2f241987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f241987f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400eb10 tx=0x7f2f1400c2d0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 shutdown_connections 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2f0c07ac40 0x7f2f0c07d100 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f241067f0 0x7f2f24198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 --2- 172.21.15.46:0/3380221197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f241007f0 0x7f2f241987f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 >> 172.21.15.46:0/3380221197 conn(0x7f2f240fc550 msgr2=0x7f2f240fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 shutdown_connections 2024-09-10T04:33:06.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.121+0000 7f2f297be640 1 -- 172.21.15.46:0/3380221197 wait complete. 2024-09-10T04:33:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.206+0000 7f423066b640 1 -- 172.21.15.46:0/4082771557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42281033e0 msgr2=0x7f422810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.206+0000 7f423066b640 1 --2- 172.21.15.46:0/4082771557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42281033e0 0x7f422810b8d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4218009880 tx=0x7f421802a530 comp rx=0 tx=0).stop 2024-09-10T04:33:06.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 -- 172.21.15.46:0/4082771557 shutdown_connections 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 --2- 172.21.15.46:0/4082771557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42281033e0 0x7f422810b8d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 --2- 172.21.15.46:0/4082771557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4228102ac0 0x7f4228102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 -- 172.21.15.46:0/4082771557 >> 172.21.15.46:0/4082771557 conn(0x7f42280fc520 msgr2=0x7f42280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 -- 172.21.15.46:0/4082771557 shutdown_connections 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 -- 172.21.15.46:0/4082771557 wait complete. 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 Processor -- start 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 -- start start 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.207+0000 7f423066b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4228102ac0 0x7f4228100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f423066b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42281033e0 0x7f4228100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f423066b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f422810edd0 con 0x7f4228102ac0 2024-09-10T04:33:06.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f423066b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f422810ec50 con 0x7f42281033e0 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422e407640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4228102ac0 0x7f4228100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422e407640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4228102ac0 0x7f4228100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52036/0 (socket says 172.21.15.46:52036) 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422e407640 1 -- 172.21.15.46:0/3765142669 learned_addr learned my addr 172.21.15.46:0/3765142669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422dc06640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42281033e0 0x7f4228100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422e407640 1 -- 172.21.15.46:0/3765142669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42281033e0 msgr2=0x7f4228100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422e407640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42281033e0 0x7f4228100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422e407640 1 -- 172.21.15.46:0/3765142669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42281011a0 con 0x7f4228102ac0 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422dc06640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42281033e0 0x7f4228100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f422e407640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4228102ac0 0x7f4228100720 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f422400e970 tx=0x7f422400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:06.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f421f7fe640 1 -- 172.21.15.46:0/3765142669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f422400cac0 con 0x7f4228102ac0 2024-09-10T04:33:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f421f7fe640 1 -- 172.21.15.46:0/3765142669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f422400cc60 con 0x7f4228102ac0 2024-09-10T04:33:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4228101490 con 0x7f4228102ac0 2024-09-10T04:33:06.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f421f7fe640 1 -- 172.21.15.46:0/3765142669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4224018720 con 0x7f4228102ac0 2024-09-10T04:33:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.208+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281a3e60 con 0x7f4228102ac0 2024-09-10T04:33:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.209+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422806aa90 con 0x7f4228102ac0 2024-09-10T04:33:06.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.210+0000 7f421f7fe640 1 -- 172.21.15.46:0/3765142669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4224010430 con 0x7f4228102ac0 2024-09-10T04:33:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.211+0000 7f421f7fe640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f421407ac00 0x7f421407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.211+0000 7f421f7fe640 1 -- 172.21.15.46:0/3765142669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f422409e340 con 0x7f4228102ac0 2024-09-10T04:33:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.211+0000 7f422dc06640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f421407ac00 0x7f421407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:06.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.211+0000 7f422dc06640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f421407ac00 0x7f421407d0c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4218031460 tx=0x7f42180041a0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:06.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.215+0000 7f421f7fe640 1 -- 172.21.15.46:0/3765142669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4224069a80 con 0x7f4228102ac0 2024-09-10T04:33:06.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.420+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4228105fd0 con 0x7f421407ac00 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.421+0000 7f421f7fe640 1 -- 172.21.15.46:0/3765142669 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f4218004330 con 0x7f421407ac00 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:33:06.422 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:33:06.423 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:33:06.423 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:33:06.423 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:33:06.423 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:33:06.423 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:33:06.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.423+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f421407ac00 msgr2=0x7f421407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.423+0000 7f423066b640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f421407ac00 0x7f421407d0c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4218031460 tx=0x7f42180041a0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.423+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4228102ac0 msgr2=0x7f4228100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.423+0000 7f423066b640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4228102ac0 0x7f4228100720 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f422400e970 tx=0x7f422400ee30 comp rx=0 tx=0).stop 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.424+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 shutdown_connections 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.424+0000 7f423066b640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f421407ac00 0x7f421407d0c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.424+0000 7f423066b640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42281033e0 0x7f4228100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.424+0000 7f423066b640 1 --2- 172.21.15.46:0/3765142669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4228102ac0 0x7f4228100720 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.424+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 >> 172.21.15.46:0/3765142669 conn(0x7f42280fc520 msgr2=0x7f4228108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.424+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 shutdown_connections 2024-09-10T04:33:06.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.424+0000 7f423066b640 1 -- 172.21.15.46:0/3765142669 wait complete. 2024-09-10T04:33:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 -- 172.21.15.46:0/398977750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4981033d0 msgr2=0x7fa49810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 --2- 172.21.15.46:0/398977750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4981033d0 0x7fa49810b8c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009880 tx=0x7fa48c02a530 comp rx=0 tx=0).stop 2024-09-10T04:33:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 -- 172.21.15.46:0/398977750 shutdown_connections 2024-09-10T04:33:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 --2- 172.21.15.46:0/398977750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4981033d0 0x7fa49810b8c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 --2- 172.21.15.46:0/398977750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa498102ab0 0x7fa498102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 -- 172.21.15.46:0/398977750 >> 172.21.15.46:0/398977750 conn(0x7fa4980fc550 msgr2=0x7fa4980fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:06.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 -- 172.21.15.46:0/398977750 shutdown_connections 2024-09-10T04:33:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 -- 172.21.15.46:0/398977750 wait complete. 2024-09-10T04:33:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 Processor -- start 2024-09-10T04:33:06.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.508+0000 7fa49e89f640 1 -- start start 2024-09-10T04:33:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa49e89f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa498102ab0 0x7fa498100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa49e89f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4981033d0 0x7fa498100c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa49e89f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa49810ede0 con 0x7fa498102ab0 2024-09-10T04:33:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa49e89f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa49810ec60 con 0x7fa4981033d0 2024-09-10T04:33:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa497fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa498102ab0 0x7fa498100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:06.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa497fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa498102ab0 0x7fa498100710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52058/0 (socket says 172.21.15.46:52058) 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa497fff640 1 -- 172.21.15.46:0/1677928192 learned_addr learned my addr 172.21.15.46:0/1677928192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa497fff640 1 -- 172.21.15.46:0/1677928192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4981033d0 msgr2=0x7fa498100c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa4977fe640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4981033d0 0x7fa498100c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa497fff640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4981033d0 0x7fa498100c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa497fff640 1 -- 172.21.15.46:0/1677928192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa498101190 con 0x7fa498102ab0 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa497fff640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa498102ab0 0x7fa498100710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa48800d6f0 tx=0x7fa48800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.509+0000 7fa4957fa640 1 -- 172.21.15.46:0/1677928192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa488017070 con 0x7fa498102ab0 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.510+0000 7fa4957fa640 1 -- 172.21.15.46:0/1677928192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa48800de00 con 0x7fa498102ab0 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.510+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa498101480 con 0x7fa498102ab0 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.510+0000 7fa4957fa640 1 -- 172.21.15.46:0/1677928192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa488004bb0 con 0x7fa498102ab0 2024-09-10T04:33:06.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.510+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4981a3e90 con 0x7fa498102ab0 2024-09-10T04:33:06.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.511+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49806a580 con 0x7fa498102ab0 2024-09-10T04:33:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.511+0000 7fa4957fa640 1 -- 172.21.15.46:0/1677928192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa488009a30 con 0x7fa498102ab0 2024-09-10T04:33:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.512+0000 7fa4957fa640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa48007ac40 0x7fa48007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.512+0000 7fa4957fa640 1 -- 172.21.15.46:0/1677928192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa48809d7f0 con 0x7fa498102ab0 2024-09-10T04:33:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.512+0000 7fa4977fe640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa48007ac40 0x7fa48007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:06.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.512+0000 7fa4977fe640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa48007ac40 0x7fa48007d100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa48c031460 tx=0x7fa48c009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:06.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.516+0000 7fa4957fa640 1 -- 172.21.15.46:0/1677928192 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa488068fb0 con 0x7fa498102ab0 2024-09-10T04:33:06.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.780+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa498102e90 con 0x7fa498102ab0 2024-09-10T04:33:06.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.780+0000 7fa4957fa640 1 -- 172.21.15.46:0/1677928192 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fa48806de60 con 0x7fa498102ab0 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:33:06.782 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:33:06.783 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.782+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa48007ac40 msgr2=0x7fa48007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.782+0000 7fa49e89f640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa48007ac40 0x7fa48007d100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa48c031460 tx=0x7fa48c009560 comp rx=0 tx=0).stop 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.782+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa498102ab0 msgr2=0x7fa498100710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.782+0000 7fa49e89f640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa498102ab0 0x7fa498100710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fa48800d6f0 tx=0x7fa48800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.783+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 shutdown_connections 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.783+0000 7fa49e89f640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa48007ac40 0x7fa48007d100 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.783+0000 7fa49e89f640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4981033d0 0x7fa498100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.783+0000 7fa49e89f640 1 --2- 172.21.15.46:0/1677928192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa498102ab0 0x7fa498100710 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.783+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 >> 172.21.15.46:0/1677928192 conn(0x7fa4980fc550 msgr2=0x7fa498108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.783+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 shutdown_connections 2024-09-10T04:33:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:06.783+0000 7fa49e89f640 1 -- 172.21.15.46:0/1677928192 wait complete. 2024-09-10T04:33:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:06 smithi070 ceph-mon[79774]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:06 smithi070 ceph-mon[79774]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:06 smithi070 ceph-mon[79774]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:06 smithi070 ceph-mon[79774]: pgmap v824: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:06 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2835169739' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:33:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:06 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3380221197' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:33:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:06 smithi046 ceph-mon[109506]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:06 smithi046 ceph-mon[109506]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:06 smithi046 ceph-mon[109506]: from='client.35354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:06 smithi046 ceph-mon[109506]: pgmap v824: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:06 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2835169739' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:33:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:06 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3380221197' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:33:08.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:07 smithi070 ceph-mon[79774]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:08.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:07 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1677928192' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:33:08.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:07 smithi046 ceph-mon[109506]: from='client.35366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:08.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:07 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1677928192' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:33:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:08 smithi070 ceph-mon[79774]: pgmap v825: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:08 smithi046 ceph-mon[109506]: pgmap v825: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:10 smithi070 ceph-mon[79774]: pgmap v826: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:10 smithi046 ceph-mon[109506]: pgmap v826: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:12 smithi070 ceph-mon[79774]: pgmap v827: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:12 smithi046 ceph-mon[109506]: pgmap v827: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:14 smithi070 ceph-mon[79774]: pgmap v828: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:14 smithi046 ceph-mon[109506]: pgmap v828: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:16 smithi070 ceph-mon[79774]: pgmap v829: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:16 smithi046 ceph-mon[109506]: pgmap v829: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:18 smithi070 ceph-mon[79774]: pgmap v830: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:19.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:18 smithi046 ceph-mon[109506]: pgmap v830: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:21.067 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:20 smithi070 ceph-mon[79774]: pgmap v831: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:20 smithi046 ceph-mon[109506]: pgmap v831: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:22 smithi070 ceph-mon[79774]: pgmap v832: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:22 smithi046 ceph-mon[109506]: pgmap v832: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:25.032 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:24 smithi046 ceph-mon[109506]: pgmap v833: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:24 smithi070 ceph-mon[79774]: pgmap v833: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:26 smithi070 ceph-mon[79774]: pgmap v834: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:26 smithi046 ceph-mon[109506]: pgmap v834: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:28 smithi070 ceph-mon[79774]: pgmap v835: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:28 smithi046 ceph-mon[109506]: pgmap v835: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:30 smithi070 ceph-mon[79774]: pgmap v836: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:30 smithi046 ceph-mon[109506]: pgmap v836: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:32 smithi070 ceph-mon[79774]: pgmap v837: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:32 smithi046 ceph-mon[109506]: pgmap v837: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:34 smithi070 ceph-mon[79774]: pgmap v838: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:34 smithi046 ceph-mon[109506]: pgmap v838: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.879+0000 7f363b7d3640 1 -- 172.21.15.46:0/2759369306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 msgr2=0x7f363410b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.879+0000 7f363b7d3640 1 --2- 172.21.15.46:0/2759369306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363410b980 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3624009880 tx=0x7f362402a530 comp rx=0 tx=0).stop 2024-09-10T04:33:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.879+0000 7f363b7d3640 1 -- 172.21.15.46:0/2759369306 shutdown_connections 2024-09-10T04:33:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.879+0000 7f363b7d3640 1 --2- 172.21.15.46:0/2759369306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363410b980 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.879+0000 7f363b7d3640 1 --2- 172.21.15.46:0/2759369306 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3634102b70 0x7f3634102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.879+0000 7f363b7d3640 1 -- 172.21.15.46:0/2759369306 >> 172.21.15.46:0/2759369306 conn(0x7f36340fc550 msgr2=0x7f36340fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.880+0000 7f363b7d3640 1 -- 172.21.15.46:0/2759369306 shutdown_connections 2024-09-10T04:33:36.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.880+0000 7f363b7d3640 1 -- 172.21.15.46:0/2759369306 wait complete. 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.880+0000 7f363b7d3640 1 Processor -- start 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.880+0000 7f363b7d3640 1 -- start start 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f363b7d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3634102b70 0x7f363419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f363b7d3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f363b7d3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f363410ede0 con 0x7f3634103490 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f363b7d3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f363410ec60 con 0x7f3634102b70 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f3638d6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f3638d6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363419db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57978/0 (socket says 172.21.15.46:57978) 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f3638d6e640 1 -- 172.21.15.46:0/388985311 learned_addr learned my addr 172.21.15.46:0/388985311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f363956f640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3634102b70 0x7f363419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f3638d6e640 1 -- 172.21.15.46:0/388985311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3634102b70 msgr2=0x7f363419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:36.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f3638d6e640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3634102b70 0x7f363419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f3638d6e640 1 -- 172.21.15.46:0/388985311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3634197790 con 0x7f3634103490 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.881+0000 7f3638d6e640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363419db50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3624004f90 tx=0x7f36240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.882+0000 7f362a7fc640 1 -- 172.21.15.46:0/388985311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f362403f070 con 0x7f3634103490 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.882+0000 7f362a7fc640 1 -- 172.21.15.46:0/388985311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3624004c70 con 0x7f3634103490 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.882+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3634197a20 con 0x7f3634103490 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.882+0000 7f362a7fc640 1 -- 172.21.15.46:0/388985311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3624035030 con 0x7f3634103490 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.882+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3634197f50 con 0x7f3634103490 2024-09-10T04:33:36.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.883+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363406a4c0 con 0x7f3634103490 2024-09-10T04:33:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.884+0000 7f362a7fc640 1 -- 172.21.15.46:0/388985311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3624042070 con 0x7f3634103490 2024-09-10T04:33:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.884+0000 7f362a7fc640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f362007abf0 0x7f362007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.884+0000 7f362a7fc640 1 -- 172.21.15.46:0/388985311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f36240be5a0 con 0x7f3634103490 2024-09-10T04:33:36.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.884+0000 7f363956f640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f362007abf0 0x7f362007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:36.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.885+0000 7f363956f640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f362007abf0 0x7f362007d0b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3630005fd0 tx=0x7f363000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:36.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:36.889+0000 7f362a7fc640 1 -- 172.21.15.46:0/388985311 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3624089ce0 con 0x7f3634103490 2024-09-10T04:33:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:36 smithi070 ceph-mon[79774]: pgmap v839: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:36 smithi046 ceph-mon[109506]: pgmap v839: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:37.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.096+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3634198200 con 0x7f362007abf0 2024-09-10T04:33:37.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.097+0000 7f362a7fc640 1 -- 172.21.15.46:0/388985311 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3634198200 con 0x7f362007abf0 2024-09-10T04:33:37.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f362007abf0 msgr2=0x7f362007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f362007abf0 0x7f362007d0b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3630005fd0 tx=0x7f363000b390 comp rx=0 tx=0).stop 2024-09-10T04:33:37.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 msgr2=0x7f363419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363419db50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3624004f90 tx=0x7f36240048f0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 shutdown_connections 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f362007abf0 0x7f362007d0b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3634103490 0x7f363419db50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 --2- 172.21.15.46:0/388985311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3634102b70 0x7f363419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.099+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 >> 172.21.15.46:0/388985311 conn(0x7f36340fc550 msgr2=0x7f3634103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.100+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 shutdown_connections 2024-09-10T04:33:37.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.100+0000 7f363b7d3640 1 -- 172.21.15.46:0/388985311 wait complete. 2024-09-10T04:33:37.110 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:33:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 -- 172.21.15.46:0/916962695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f44102ac0 msgr2=0x7f1f44102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 --2- 172.21.15.46:0/916962695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f44102ac0 0x7f1f44102ea0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009880 tx=0x7f1f3402a530 comp rx=0 tx=0).stop 2024-09-10T04:33:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 -- 172.21.15.46:0/916962695 shutdown_connections 2024-09-10T04:33:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 --2- 172.21.15.46:0/916962695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f441033e0 0x7f1f4410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 --2- 172.21.15.46:0/916962695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f44102ac0 0x7f1f44102ea0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 -- 172.21.15.46:0/916962695 >> 172.21.15.46:0/916962695 conn(0x7f1f440fc520 msgr2=0x7f1f440fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:37.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 -- 172.21.15.46:0/916962695 shutdown_connections 2024-09-10T04:33:37.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.185+0000 7f1f4add0640 1 -- 172.21.15.46:0/916962695 wait complete. 2024-09-10T04:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f4add0640 1 Processor -- start 2024-09-10T04:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f4add0640 1 -- start start 2024-09-10T04:33:37.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f4add0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f44102ac0 0x7f1f4419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f4add0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f441033e0 0x7f1f4419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f4add0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410edf0 con 0x7f1f441033e0 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f4add0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4410ec70 con 0x7f1f44102ac0 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f441033e0 0x7f1f4419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f43fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f441033e0 0x7f1f4419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57998/0 (socket says 172.21.15.46:57998) 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f43fff640 1 -- 172.21.15.46:0/3856565146 learned_addr learned my addr 172.21.15.46:0/3856565146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f48b6c640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f44102ac0 0x7f1f4419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f43fff640 1 -- 172.21.15.46:0/3856565146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f44102ac0 msgr2=0x7f1f4419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f43fff640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f44102ac0 0x7f1f4419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.186+0000 7f1f43fff640 1 -- 172.21.15.46:0/3856565146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f441977c0 con 0x7f1f441033e0 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.187+0000 7f1f43fff640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f441033e0 0x7f1f4419db80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c8f0 tx=0x7f1f2c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:37.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.187+0000 7f1f41ffb640 1 -- 172.21.15.46:0/3856565146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c00ae50 con 0x7f1f441033e0 2024-09-10T04:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.187+0000 7f1f41ffb640 1 -- 172.21.15.46:0/3856565146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1f2c005020 con 0x7f1f441033e0 2024-09-10T04:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.187+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f44197ab0 con 0x7f1f441033e0 2024-09-10T04:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.187+0000 7f1f41ffb640 1 -- 172.21.15.46:0/3856565146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1f2c004d70 con 0x7f1f441033e0 2024-09-10T04:33:37.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.187+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f44198040 con 0x7f1f441033e0 2024-09-10T04:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.188+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f4406aa90 con 0x7f1f441033e0 2024-09-10T04:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.189+0000 7f1f41ffb640 1 -- 172.21.15.46:0/3856565146 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1f2c0051d0 con 0x7f1f441033e0 2024-09-10T04:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.189+0000 7f1f41ffb640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1f3007adf0 0x7f1f3007d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.189+0000 7f1f41ffb640 1 -- 172.21.15.46:0/3856565146 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1f2c09da10 con 0x7f1f441033e0 2024-09-10T04:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.189+0000 7f1f48b6c640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1f3007adf0 0x7f1f3007d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.190+0000 7f1f48b6c640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1f3007adf0 0x7f1f3007d2b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f34031460 tx=0x7f1f34005bd0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:37.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.194+0000 7f1f41ffb640 1 -- 172.21.15.46:0/3856565146 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1f2c014030 con 0x7f1f441033e0 2024-09-10T04:33:37.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.396+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1f44198350 con 0x7f1f3007adf0 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.397+0000 7f1f41ffb640 1 -- 172.21.15.46:0/3856565146 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f1f44198350 con 0x7f1f3007adf0 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1f3007adf0 msgr2=0x7f1f3007d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1f3007adf0 0x7f1f3007d2b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1f34031460 tx=0x7f1f34005bd0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f441033e0 msgr2=0x7f1f4419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f441033e0 0x7f1f4419db80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00c8f0 tx=0x7f1f2c00cdb0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 shutdown_connections 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1f3007adf0 0x7f1f3007d2b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f441033e0 0x7f1f4419db80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 --2- 172.21.15.46:0/3856565146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1f44102ac0 0x7f1f4419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.400+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 >> 172.21.15.46:0/3856565146 conn(0x7f1f440fc520 msgr2=0x7f1f4410af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.401+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 shutdown_connections 2024-09-10T04:33:37.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.401+0000 7f1f4add0640 1 -- 172.21.15.46:0/3856565146 wait complete. 2024-09-10T04:33:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1774375133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81067f0 msgr2=0x7f6cd8106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1774375133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81067f0 0x7f6cd8106bd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009880 tx=0x7f6cd402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1774375133 shutdown_connections 2024-09-10T04:33:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1774375133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cd81007f0 0x7f6cd8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1774375133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81067f0 0x7f6cd8106bd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1774375133 >> 172.21.15.46:0/1774375133 conn(0x7f6cd80fc550 msgr2=0x7f6cd80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1774375133 shutdown_connections 2024-09-10T04:33:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.485+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1774375133 wait complete. 2024-09-10T04:33:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdf13a640 1 Processor -- start 2024-09-10T04:33:37.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdf13a640 1 -- start start 2024-09-10T04:33:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdf13a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81007f0 0x7f6cd80fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdf13a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cd81067f0 0x7f6cd80fd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdf13a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd810dcc0 con 0x7f6cd81007f0 2024-09-10T04:33:37.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdf13a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd810db40 con 0x7f6cd81067f0 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdced6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81007f0 0x7f6cd80fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdced6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81007f0 0x7f6cd80fcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58018/0 (socket says 172.21.15.46:58018) 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdced6640 1 -- 172.21.15.46:0/1045492589 learned_addr learned my addr 172.21.15.46:0/1045492589 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6ccffff640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cd81067f0 0x7f6cd80fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdced6640 1 -- 172.21.15.46:0/1045492589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cd81067f0 msgr2=0x7f6cd80fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdced6640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cd81067f0 0x7f6cd80fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6cdced6640 1 -- 172.21.15.46:0/1045492589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd80fd790 con 0x7f6cd81007f0 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.486+0000 7f6ccffff640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cd81067f0 0x7f6cd80fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.487+0000 7f6cdced6640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81007f0 0x7f6cd80fcd10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4031460 tx=0x7f6cd4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.487+0000 7f6ccdffb640 1 -- 172.21.15.46:0/1045492589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6cd403f070 con 0x7f6cd81007f0 2024-09-10T04:33:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.487+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd80fda20 con 0x7f6cd81007f0 2024-09-10T04:33:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.487+0000 7f6ccdffb640 1 -- 172.21.15.46:0/1045492589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6cd40035f0 con 0x7f6cd81007f0 2024-09-10T04:33:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.487+0000 7f6ccdffb640 1 -- 172.21.15.46:0/1045492589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6cd4035030 con 0x7f6cd81007f0 2024-09-10T04:33:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.487+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd81a3da0 con 0x7f6cd81007f0 2024-09-10T04:33:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.488+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd8101f30 con 0x7f6cd81007f0 2024-09-10T04:33:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.489+0000 7f6ccdffb640 1 -- 172.21.15.46:0/1045492589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6cd4042070 con 0x7f6cd81007f0 2024-09-10T04:33:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.489+0000 7f6ccdffb640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6cc007abf0 0x7f6cc007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.489+0000 7f6ccdffb640 1 -- 172.21.15.46:0/1045492589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6cd40be660 con 0x7f6cd81007f0 2024-09-10T04:33:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.489+0000 7f6ccffff640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6cc007abf0 0x7f6cc007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.490+0000 7f6ccffff640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6cc007abf0 0x7f6cc007d0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80fe7a0 tx=0x7f6cc8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:37.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.494+0000 7f6ccdffb640 1 -- 172.21.15.46:0/1045492589 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6cd4089da0 con 0x7f6cd81007f0 2024-09-10T04:33:37.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.687+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f6cd8062630 con 0x7f6cc007abf0 2024-09-10T04:33:37.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.698+0000 7f6ccdffb640 1 -- 172.21.15.46:0/1045492589 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f6cd8062630 con 0x7f6cc007abf0 2024-09-10T04:33:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (25m) 4m ago 36m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (27m) 4m ago 36m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (27m) 4m ago 35m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (23m) 4m ago 36m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (23m) 4m ago 35m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (24m) 4m ago 36m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (19m) 4m ago 31m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (20m) 4m ago 31m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (19m) 4m ago 31m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (4m) 4m ago 31m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (27m) 4m ago 37m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (26m) 4m ago 35m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:33:37.701 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (24m) 4m ago 38m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (23m) 4m ago 35m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (26m) 4m ago 36m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (25m) 4m ago 35m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (23m) 4m ago 34m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (22m) 4m ago 34m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (22m) 4m ago 33m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (21m) 4m ago 33m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (20m) 4m ago 32m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (20m) 4m ago 32m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:33:37.702 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (25m) 4m ago 36m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:33:37.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6cc007abf0 msgr2=0x7f6cc007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6cc007abf0 0x7f6cc007d0b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6cd80fe7a0 tx=0x7f6cc8005f90 comp rx=0 tx=0).stop 2024-09-10T04:33:37.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81007f0 msgr2=0x7f6cd80fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81007f0 0x7f6cd80fcd10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4031460 tx=0x7f6cd4004530 comp rx=0 tx=0).stop 2024-09-10T04:33:37.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 shutdown_connections 2024-09-10T04:33:37.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6cc007abf0 0x7f6cc007d0b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6cd81067f0 0x7f6cd80fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 --2- 172.21.15.46:0/1045492589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cd81007f0 0x7f6cd80fcd10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 >> 172.21.15.46:0/1045492589 conn(0x7f6cd80fc550 msgr2=0x7f6cd8069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:37.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 shutdown_connections 2024-09-10T04:33:37.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.701+0000 7f6cdf13a640 1 -- 172.21.15.46:0/1045492589 wait complete. 2024-09-10T04:33:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2461387377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 msgr2=0x7fe0e8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2461387377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8102ea0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8009880 tx=0x7fe0d802a530 comp rx=0 tx=0).stop 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2461387377 shutdown_connections 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2461387377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0e81033e0 0x7fe0e810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2461387377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8102ea0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2461387377 >> 172.21.15.46:0/2461387377 conn(0x7fe0e80fc520 msgr2=0x7fe0e80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2461387377 shutdown_connections 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.786+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2461387377 wait complete. 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0ee89c640 1 Processor -- start 2024-09-10T04:33:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0ee89c640 1 -- start start 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0ee89c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0ee89c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0e81033e0 0x7fe0e8100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0ee89c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810ee50 con 0x7fe0e8102ac0 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0ee89c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e810ecd0 con 0x7fe0e81033e0 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0e7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58050/0 (socket says 172.21.15.46:58050) 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0e7fff640 1 -- 172.21.15.46:0/2123120237 learned_addr learned my addr 172.21.15.46:0/2123120237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.787+0000 7fe0e77fe640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0e81033e0 0x7fe0e8100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0e7fff640 1 -- 172.21.15.46:0/2123120237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0e81033e0 msgr2=0x7fe0e8100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0e7fff640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0e81033e0 0x7fe0e8100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0e7fff640 1 -- 172.21.15.46:0/2123120237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e81011a0 con 0x7fe0e8102ac0 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0e7fff640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8100720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8031460 tx=0x7fe0d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0e57fa640 1 -- 172.21.15.46:0/2123120237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe0d803f070 con 0x7fe0e8102ac0 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0e57fa640 1 -- 172.21.15.46:0/2123120237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe0d8004cf0 con 0x7fe0e8102ac0 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e8101430 con 0x7fe0e8102ac0 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0e57fa640 1 -- 172.21.15.46:0/2123120237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe0d8035030 con 0x7fe0e8102ac0 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.788+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e81a3e20 con 0x7fe0e8102ac0 2024-09-10T04:33:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.789+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e806aa90 con 0x7fe0e8102ac0 2024-09-10T04:33:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.790+0000 7fe0e57fa640 1 -- 172.21.15.46:0/2123120237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe0d8042070 con 0x7fe0e8102ac0 2024-09-10T04:33:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.790+0000 7fe0e57fa640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe0d007abf0 0x7fe0d007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.790+0000 7fe0e57fa640 1 -- 172.21.15.46:0/2123120237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe0d80be700 con 0x7fe0e8102ac0 2024-09-10T04:33:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.790+0000 7fe0e77fe640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe0d007abf0 0x7fe0d007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.791+0000 7fe0e77fe640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe0d007abf0 0x7fe0d007d0b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe0e81021b0 tx=0x7fe0dc009060 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:37.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:37.795+0000 7fe0e57fa640 1 -- 172.21.15.46:0/2123120237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe0d8089e40 con 0x7fe0e8102ac0 2024-09-10T04:33:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.054+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fe0e81a4190 con 0x7fe0e8102ac0 2024-09-10T04:33:38.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.055+0000 7fe0e57fa640 1 -- 172.21.15.46:0/2123120237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fe0d808ecf0 con 0x7fe0e8102ac0 2024-09-10T04:33:38.057 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:33:38.057 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:33:38.058 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:33:38.059 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:33:38.059 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:33:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe0d007abf0 msgr2=0x7fe0d007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe0d007abf0 0x7fe0d007d0b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe0e81021b0 tx=0x7fe0dc009060 comp rx=0 tx=0).stop 2024-09-10T04:33:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 msgr2=0x7fe0e8100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8100720 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe0d8031460 tx=0x7fe0d80048f0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 shutdown_connections 2024-09-10T04:33:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe0d007abf0 0x7fe0d007d0b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0e81033e0 0x7fe0e8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 --2- 172.21.15.46:0/2123120237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0e8102ac0 0x7fe0e8100720 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 >> 172.21.15.46:0/2123120237 conn(0x7fe0e80fc520 msgr2=0x7fe0e810af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 shutdown_connections 2024-09-10T04:33:38.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.057+0000 7fe0ee89c640 1 -- 172.21.15.46:0/2123120237 wait complete. 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 -- 172.21.15.46:0/4203721619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 msgr2=0x7f559010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 --2- 172.21.15.46:0/4203721619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559010b8d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5584009880 tx=0x7f558402a530 comp rx=0 tx=0).stop 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 -- 172.21.15.46:0/4203721619 shutdown_connections 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 --2- 172.21.15.46:0/4203721619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559010b8d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 --2- 172.21.15.46:0/4203721619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5590102ac0 0x7f5590102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 -- 172.21.15.46:0/4203721619 >> 172.21.15.46:0/4203721619 conn(0x7f55900fc520 msgr2=0x7f55900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 -- 172.21.15.46:0/4203721619 shutdown_connections 2024-09-10T04:33:38.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.142+0000 7f5595ee8640 1 -- 172.21.15.46:0/4203721619 wait complete. 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f5595ee8640 1 Processor -- start 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f5595ee8640 1 -- start start 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f5595ee8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5590102ac0 0x7f559019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f5595ee8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f5595ee8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f559010edf0 con 0x7f55901033e0 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f5595ee8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f559010ec70 con 0x7f5590102ac0 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f558effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f558effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559019dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58078/0 (socket says 172.21.15.46:58078) 2024-09-10T04:33:38.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f558effd640 1 -- 172.21.15.46:0/883119411 learned_addr learned my addr 172.21.15.46:0/883119411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f558f7fe640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5590102ac0 0x7f559019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f558effd640 1 -- 172.21.15.46:0/883119411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5590102ac0 msgr2=0x7f559019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f558effd640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5590102ac0 0x7f559019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.143+0000 7f558effd640 1 -- 172.21.15.46:0/883119411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5590197800 con 0x7f55901033e0 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.144+0000 7f558effd640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559019dbc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5584031460 tx=0x7f5584004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.144+0000 7f558cff9640 1 -- 172.21.15.46:0/883119411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f558403f070 con 0x7f55901033e0 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.144+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5590197a90 con 0x7f55901033e0 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.144+0000 7f558cff9640 1 -- 172.21.15.46:0/883119411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5584004a90 con 0x7f55901033e0 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.144+0000 7f558cff9640 1 -- 172.21.15.46:0/883119411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5584035030 con 0x7f55901033e0 2024-09-10T04:33:38.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.144+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5590197fc0 con 0x7f55901033e0 2024-09-10T04:33:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.145+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559006aa90 con 0x7f55901033e0 2024-09-10T04:33:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.146+0000 7f558cff9640 1 -- 172.21.15.46:0/883119411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5584042070 con 0x7f55901033e0 2024-09-10T04:33:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.146+0000 7f558cff9640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f557807ac00 0x7f557807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.146+0000 7f558cff9640 1 -- 172.21.15.46:0/883119411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f55840be6e0 con 0x7f55901033e0 2024-09-10T04:33:38.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.146+0000 7f558f7fe640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f557807ac00 0x7f557807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.147+0000 7f558f7fe640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f557807ac00 0x7f557807d0c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5580005fd0 tx=0x7f558000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:38.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.151+0000 7f558cff9640 1 -- 172.21.15.46:0/883119411 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5584089e20 con 0x7f55901033e0 2024-09-10T04:33:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.377+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f5590105f60 con 0x7f55901033e0 2024-09-10T04:33:38.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.377+0000 7f558cff9640 1 -- 172.21.15.46:0/883119411 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f558408ecd0 con 0x7f55901033e0 2024-09-10T04:33:38.379 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:33:38.379 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:33:38.380 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:33:38.381 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:38.382 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:33:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:33:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f557807ac00 msgr2=0x7f557807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f557807ac00 0x7f557807d0c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5580005fd0 tx=0x7f558000b390 comp rx=0 tx=0).stop 2024-09-10T04:33:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 msgr2=0x7f559019dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559019dbc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5584031460 tx=0x7f5584004530 comp rx=0 tx=0).stop 2024-09-10T04:33:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 shutdown_connections 2024-09-10T04:33:38.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f557807ac00 0x7f557807d0c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55901033e0 0x7f559019dbc0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 --2- 172.21.15.46:0/883119411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5590102ac0 0x7f559019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 >> 172.21.15.46:0/883119411 conn(0x7f55900fc520 msgr2=0x7f5590108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 shutdown_connections 2024-09-10T04:33:38.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.379+0000 7f5595ee8640 1 -- 172.21.15.46:0/883119411 wait complete. 2024-09-10T04:33:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.464+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/3522320572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc106800 msgr2=0x7fbccc106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.464+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/3522320572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc106800 0x7fbccc106be0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009880 tx=0x7fbcbc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/3522320572 shutdown_connections 2024-09-10T04:33:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/3522320572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbccc100800 0x7fbccc100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/3522320572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc106800 0x7fbccc106be0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/3522320572 >> 172.21.15.46:0/3522320572 conn(0x7fbccc0fc520 msgr2=0x7fbccc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:38.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/3522320572 shutdown_connections 2024-09-10T04:33:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/3522320572 wait complete. 2024-09-10T04:33:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 Processor -- start 2024-09-10T04:33:38.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.465+0000 7fbcd0e6e640 1 -- start start 2024-09-10T04:33:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcd0e6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc100800 0x7fbccc19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcd0e6e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbccc106800 0x7fbccc19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcd0e6e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10dcd0 con 0x7fbccc100800 2024-09-10T04:33:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcd0e6e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10db50 con 0x7fbccc106800 2024-09-10T04:33:38.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcca59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc100800 0x7fbccc19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcca59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc100800 0x7fbccc19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58084/0 (socket says 172.21.15.46:58084) 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcca59c640 1 -- 172.21.15.46:0/2510500925 learned_addr learned my addr 172.21.15.46:0/2510500925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcc9d9b640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbccc106800 0x7fbccc19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcca59c640 1 -- 172.21.15.46:0/2510500925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbccc106800 msgr2=0x7fbccc19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcca59c640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbccc106800 0x7fbccc19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcca59c640 1 -- 172.21.15.46:0/2510500925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc1977e0 con 0x7fbccc100800 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcca59c640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc100800 0x7fbccc19d660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031460 tx=0x7fbcbc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcbb7fe640 1 -- 172.21.15.46:0/2510500925 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbcbc03f070 con 0x7fbccc100800 2024-09-10T04:33:38.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc197a70 con 0x7fbccc100800 2024-09-10T04:33:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.466+0000 7fbcbb7fe640 1 -- 172.21.15.46:0/2510500925 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcbc0035f0 con 0x7fbccc100800 2024-09-10T04:33:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.467+0000 7fbcbb7fe640 1 -- 172.21.15.46:0/2510500925 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbcbc035030 con 0x7fbccc100800 2024-09-10T04:33:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.467+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc197fa0 con 0x7fbccc100800 2024-09-10T04:33:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.468+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc101f40 con 0x7fbccc100800 2024-09-10T04:33:38.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.468+0000 7fbcbb7fe640 1 -- 172.21.15.46:0/2510500925 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbcbc042070 con 0x7fbccc100800 2024-09-10T04:33:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.469+0000 7fbcbb7fe640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcb407abf0 0x7fbcb407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.469+0000 7fbcbb7fe640 1 -- 172.21.15.46:0/2510500925 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fbcbc0be5d0 con 0x7fbccc100800 2024-09-10T04:33:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.469+0000 7fbcc9d9b640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcb407abf0 0x7fbcb407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.469+0000 7fbcc9d9b640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcb407abf0 0x7fbcb407d0b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbccc104f70 tx=0x7fbcc000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.473+0000 7fbcbb7fe640 1 -- 172.21.15.46:0/2510500925 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbcbc089d10 con 0x7fbccc100800 2024-09-10T04:33:38.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.676+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fbccc198250 con 0x7fbcb407abf0 2024-09-10T04:33:38.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.677+0000 7fbcbb7fe640 1 -- 172.21.15.46:0/2510500925 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fbccc198250 con 0x7fbcb407abf0 2024-09-10T04:33:38.679 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:33:38.680 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcb407abf0 msgr2=0x7fbcb407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcb407abf0 0x7fbcb407d0b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbccc104f70 tx=0x7fbcc000b390 comp rx=0 tx=0).stop 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc100800 msgr2=0x7fbccc19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc100800 0x7fbccc19d660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031460 tx=0x7fbcbc0045e0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 shutdown_connections 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcb407abf0 0x7fbcb407d0b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbccc106800 0x7fbccc19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 --2- 172.21.15.46:0/2510500925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbccc100800 0x7fbccc19d660 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 >> 172.21.15.46:0/2510500925 conn(0x7fbccc0fc520 msgr2=0x7fbccc06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 shutdown_connections 2024-09-10T04:33:38.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.680+0000 7fbcd0e6e640 1 -- 172.21.15.46:0/2510500925 wait complete. 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 -- 172.21.15.46:0/1399788067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8100800 msgr2=0x7fc6c8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 --2- 172.21.15.46:0/1399788067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8100800 0x7fc6c8100c60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009880 tx=0x7fc6bc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 -- 172.21.15.46:0/1399788067 shutdown_connections 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 --2- 172.21.15.46:0/1399788067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8100800 0x7fc6c8100c60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 --2- 172.21.15.46:0/1399788067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c8106800 0x7fc6c8106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 -- 172.21.15.46:0/1399788067 >> 172.21.15.46:0/1399788067 conn(0x7fc6c80fc520 msgr2=0x7fc6c80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 -- 172.21.15.46:0/1399788067 shutdown_connections 2024-09-10T04:33:38.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 -- 172.21.15.46:0/1399788067 wait complete. 2024-09-10T04:33:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.765+0000 7fc6cd220640 1 Processor -- start 2024-09-10T04:33:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6cd220640 1 -- start start 2024-09-10T04:33:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6cd220640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c8100800 0x7fc6c819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6cd220640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8106800 0x7fc6c819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6cd220640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c810dcd0 con 0x7fc6c8106800 2024-09-10T04:33:38.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6cd220640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c810db50 con 0x7fc6c8100800 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6c659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8106800 0x7fc6c819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6c659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8106800 0x7fc6c819dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58104/0 (socket says 172.21.15.46:58104) 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6c659c640 1 -- 172.21.15.46:0/3992359182 learned_addr learned my addr 172.21.15.46:0/3992359182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6c6d9d640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c8100800 0x7fc6c819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6c659c640 1 -- 172.21.15.46:0/3992359182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c8100800 msgr2=0x7fc6c819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6c659c640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c8100800 0x7fc6c819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.766+0000 7fc6c659c640 1 -- 172.21.15.46:0/3992359182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c81977f0 con 0x7fc6c8106800 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.767+0000 7fc6c659c640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8106800 0x7fc6c819dbb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc031460 tx=0x7fc6bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.767+0000 7fc6affff640 1 -- 172.21.15.46:0/3992359182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc6bc03f070 con 0x7fc6c8106800 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.767+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c8197a80 con 0x7fc6c8106800 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.767+0000 7fc6affff640 1 -- 172.21.15.46:0/3992359182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6bc0035f0 con 0x7fc6c8106800 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.767+0000 7fc6affff640 1 -- 172.21.15.46:0/3992359182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc6bc035030 con 0x7fc6c8106800 2024-09-10T04:33:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.767+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c8197fb0 con 0x7fc6c8106800 2024-09-10T04:33:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.768+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c8101f40 con 0x7fc6c8106800 2024-09-10T04:33:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.769+0000 7fc6affff640 1 -- 172.21.15.46:0/3992359182 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc6bc042070 con 0x7fc6c8106800 2024-09-10T04:33:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.769+0000 7fc6affff640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6b007abf0 0x7fc6b007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:33:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.770+0000 7fc6c6d9d640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6b007abf0 0x7fc6b007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:33:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.770+0000 7fc6affff640 1 -- 172.21.15.46:0/3992359182 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc6bc0be630 con 0x7fc6c8106800 2024-09-10T04:33:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.770+0000 7fc6c6d9d640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6b007abf0 0x7fc6b007d0b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005fd0 tx=0x7fc6b800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:33:38.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:38.774+0000 7fc6affff640 1 -- 172.21.15.46:0/3992359182 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc6bc089d10 con 0x7fc6c8106800 2024-09-10T04:33:39.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.031+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fc6c8109d80 con 0x7fc6c8106800 2024-09-10T04:33:39.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:38 smithi046 ceph-mon[109506]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:39.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:38 smithi046 ceph-mon[109506]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:39.033 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:38 smithi046 ceph-mon[109506]: pgmap v840: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:39.034 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:38 smithi046 ceph-mon[109506]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:39.034 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:39.034 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:38 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2123120237' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:33:39.034 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:38 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/883119411' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:33:39.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.031+0000 7fc6affff640 1 -- 172.21.15.46:0/3992359182 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fc6bc08ebc0 con 0x7fc6c8106800 2024-09-10T04:33:39.034 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:33:39.034 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:33:39.034 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:33:39.035 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:33:39.036 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:33:39.036 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:33:39.036 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:33:39.036 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:33:39.036 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:33:39.036 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:33:39.036 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6b007abf0 msgr2=0x7fc6b007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6b007abf0 0x7fc6b007d0b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005fd0 tx=0x7fc6b800b390 comp rx=0 tx=0).stop 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8106800 msgr2=0x7fc6c819dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8106800 0x7fc6c819dbb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc031460 tx=0x7fc6bc0045e0 comp rx=0 tx=0).stop 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 shutdown_connections 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6b007abf0 0x7fc6b007d0b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6c8106800 0x7fc6c819dbb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 --2- 172.21.15.46:0/3992359182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6c8100800 0x7fc6c819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 >> 172.21.15.46:0/3992359182 conn(0x7fc6c80fc520 msgr2=0x7fc6c806a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 shutdown_connections 2024-09-10T04:33:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:33:39.034+0000 7fc6cd220640 1 -- 172.21.15.46:0/3992359182 wait complete. 2024-09-10T04:33:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:38 smithi070 ceph-mon[79774]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:38 smithi070 ceph-mon[79774]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:38 smithi070 ceph-mon[79774]: pgmap v840: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:38 smithi070 ceph-mon[79774]: from='client.35382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:38 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2123120237' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:33:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:38 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/883119411' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:33:40.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:39 smithi070 ceph-mon[79774]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:40.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:39 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3992359182' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:33:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:39 smithi046 ceph-mon[109506]: from='client.35394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:33:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:39 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3992359182' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:33:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:40 smithi070 ceph-mon[79774]: pgmap v841: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:40 smithi046 ceph-mon[109506]: pgmap v841: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:42 smithi070 ceph-mon[79774]: pgmap v842: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:42 smithi046 ceph-mon[109506]: pgmap v842: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:44 smithi070 ceph-mon[79774]: pgmap v843: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:44 smithi046 ceph-mon[109506]: pgmap v843: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:46 smithi070 ceph-mon[79774]: pgmap v844: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:46 smithi046 ceph-mon[109506]: pgmap v844: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:48 smithi070 ceph-mon[79774]: pgmap v845: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:48 smithi046 ceph-mon[109506]: pgmap v845: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:50 smithi070 ceph-mon[79774]: pgmap v846: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:50 smithi046 ceph-mon[109506]: pgmap v846: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:52 smithi070 ceph-mon[79774]: pgmap v847: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:52 smithi046 ceph-mon[109506]: pgmap v847: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:33:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:54 smithi070 ceph-mon[79774]: pgmap v848: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:54 smithi046 ceph-mon[109506]: pgmap v848: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:56 smithi070 ceph-mon[79774]: pgmap v849: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:56 smithi046 ceph-mon[109506]: pgmap v849: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:33:58 smithi070 ceph-mon[79774]: pgmap v850: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:33:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:33:58 smithi046 ceph-mon[109506]: pgmap v850: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:00 smithi046 ceph-mon[109506]: pgmap v851: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:00 smithi070 ceph-mon[79774]: pgmap v851: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:02 smithi070 ceph-mon[79774]: pgmap v852: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:02 smithi046 ceph-mon[109506]: pgmap v852: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:04 smithi070 ceph-mon[79774]: pgmap v853: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:04 smithi046 ceph-mon[109506]: pgmap v853: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:06 smithi070 ceph-mon[79774]: pgmap v854: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:06 smithi046 ceph-mon[109506]: pgmap v854: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:08 smithi070 ceph-mon[79774]: pgmap v855: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:08 smithi046 ceph-mon[109506]: pgmap v855: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:09.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 -- 172.21.15.46:0/3116611935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78201033e0 msgr2=0x7f782010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 --2- 172.21.15.46:0/3116611935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78201033e0 0x7f782010b8d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7810009880 tx=0x7f781002a530 comp rx=0 tx=0).stop 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 -- 172.21.15.46:0/3116611935 shutdown_connections 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 --2- 172.21.15.46:0/3116611935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78201033e0 0x7f782010b8d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 --2- 172.21.15.46:0/3116611935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7820102ac0 0x7f7820102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 -- 172.21.15.46:0/3116611935 >> 172.21.15.46:0/3116611935 conn(0x7f78200fc520 msgr2=0x7f78200fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 -- 172.21.15.46:0/3116611935 shutdown_connections 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.132+0000 7f78281d9640 1 -- 172.21.15.46:0/3116611935 wait complete. 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f78281d9640 1 Processor -- start 2024-09-10T04:34:09.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f78281d9640 1 -- start start 2024-09-10T04:34:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f78281d9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7820102ac0 0x7f7820100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f78281d9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78201033e0 0x7f7820100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f78281d9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f782010edf0 con 0x7f7820102ac0 2024-09-10T04:34:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f78281d9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f782010ec70 con 0x7f78201033e0 2024-09-10T04:34:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f7825f75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7820102ac0 0x7f7820100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f7825f75640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7820102ac0 0x7f7820100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50920/0 (socket says 172.21.15.46:50920) 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f7825f75640 1 -- 172.21.15.46:0/368227549 learned_addr learned my addr 172.21.15.46:0/368227549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.133+0000 7f7825774640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78201033e0 0x7f7820100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f7825f75640 1 -- 172.21.15.46:0/368227549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78201033e0 msgr2=0x7f7820100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f7825f75640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78201033e0 0x7f7820100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f7825f75640 1 -- 172.21.15.46:0/368227549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78201011a0 con 0x7f7820102ac0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f7825f75640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7820102ac0 0x7f7820100720 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f781c00d6f0 tx=0x7f781c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f7816ffd640 1 -- 172.21.15.46:0/368227549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f781c017070 con 0x7f7820102ac0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f7816ffd640 1 -- 172.21.15.46:0/368227549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f781c00de00 con 0x7f7820102ac0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f7816ffd640 1 -- 172.21.15.46:0/368227549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f781c004bb0 con 0x7f7820102ac0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7820101490 con 0x7f7820102ac0 2024-09-10T04:34:09.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.134+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78201a3ea0 con 0x7f7820102ac0 2024-09-10T04:34:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.135+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f782006aa90 con 0x7f7820102ac0 2024-09-10T04:34:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.136+0000 7f7816ffd640 1 -- 172.21.15.46:0/368227549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f781c009a30 con 0x7f7820102ac0 2024-09-10T04:34:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.137+0000 7f7816ffd640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f780c07ac40 0x7f780c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.137+0000 7f7816ffd640 1 -- 172.21.15.46:0/368227549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f781c09d760 con 0x7f7820102ac0 2024-09-10T04:34:09.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.137+0000 7f7825774640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f780c07ac40 0x7f780c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.137+0000 7f7825774640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f780c07ac40 0x7f780c07d100 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78201021b0 tx=0x7f7810031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:09.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.141+0000 7f7816ffd640 1 -- 172.21.15.46:0/368227549 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f781c068f20 con 0x7f7820102ac0 2024-09-10T04:34:09.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.347+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7820062630 con 0x7f780c07ac40 2024-09-10T04:34:09.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.348+0000 7f7816ffd640 1 -- 172.21.15.46:0/368227549 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7820062630 con 0x7f780c07ac40 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.350+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f780c07ac40 msgr2=0x7f780c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.350+0000 7f78281d9640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f780c07ac40 0x7f780c07d100 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78201021b0 tx=0x7f7810031400 comp rx=0 tx=0).stop 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.350+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7820102ac0 msgr2=0x7f7820100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.350+0000 7f78281d9640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7820102ac0 0x7f7820100720 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f781c00d6f0 tx=0x7f781c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.350+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 shutdown_connections 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.350+0000 7f78281d9640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f780c07ac40 0x7f780c07d100 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.351+0000 7f78281d9640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78201033e0 0x7f7820100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.351+0000 7f78281d9640 1 --2- 172.21.15.46:0/368227549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7820102ac0 0x7f7820100720 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.351+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 >> 172.21.15.46:0/368227549 conn(0x7f78200fc520 msgr2=0x7f7820108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.351+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 shutdown_connections 2024-09-10T04:34:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.351+0000 7f78281d9640 1 -- 172.21.15.46:0/368227549 wait complete. 2024-09-10T04:34:09.361 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:34:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 -- 172.21.15.46:0/1189720769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 msgr2=0x7f8cc0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/1189720769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc0100c60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009880 tx=0x7f8cb402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 -- 172.21.15.46:0/1189720769 shutdown_connections 2024-09-10T04:34:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/1189720769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc0100c60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/1189720769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc0106800 0x7f8cc0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 -- 172.21.15.46:0/1189720769 >> 172.21.15.46:0/1189720769 conn(0x7f8cc00fc520 msgr2=0x7f8cc00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 -- 172.21.15.46:0/1189720769 shutdown_connections 2024-09-10T04:34:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.436+0000 7f8cc58ae640 1 -- 172.21.15.46:0/1189720769 wait complete. 2024-09-10T04:34:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.437+0000 7f8cc58ae640 1 Processor -- start 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.437+0000 7f8cc58ae640 1 -- start start 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.437+0000 7f8cc58ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc00fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cc58ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc0106800 0x7f8cc00fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cc58ae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010dcb0 con 0x7f8cc0100800 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cc58ae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc010db30 con 0x7f8cc0106800 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc00fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc00fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50936/0 (socket says 172.21.15.46:50936) 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbeffd640 1 -- 172.21.15.46:0/963652721 learned_addr learned my addr 172.21.15.46:0/963652721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbe7fc640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc0106800 0x7f8cc00fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbeffd640 1 -- 172.21.15.46:0/963652721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc0106800 msgr2=0x7f8cc00fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbeffd640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc0106800 0x7f8cc00fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbeffd640 1 -- 172.21.15.46:0/963652721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc00fd650 con 0x7f8cc0100800 2024-09-10T04:34:09.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cbeffd640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc00fcbd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d8d0 tx=0x7f8cb000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.438+0000 7f8cc48ac640 1 -- 172.21.15.46:0/963652721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cb0017070 con 0x7f8cc0100800 2024-09-10T04:34:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.439+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc00fd940 con 0x7f8cc0100800 2024-09-10T04:34:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.439+0000 7f8cc48ac640 1 -- 172.21.15.46:0/963652721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cb0010460 con 0x7f8cc0100800 2024-09-10T04:34:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.439+0000 7f8cc48ac640 1 -- 172.21.15.46:0/963652721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cb000f5f0 con 0x7f8cc0100800 2024-09-10T04:34:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.439+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc01a3e20 con 0x7f8cc0100800 2024-09-10T04:34:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.440+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cc0101f40 con 0x7f8cc0100800 2024-09-10T04:34:09.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.440+0000 7f8cc48ac640 1 -- 172.21.15.46:0/963652721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8cb0009a30 con 0x7f8cc0100800 2024-09-10T04:34:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.441+0000 7f8cc48ac640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca807ac00 0x7f8ca807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.441+0000 7f8cc48ac640 1 -- 172.21.15.46:0/963652721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8cb009dac0 con 0x7f8cc0100800 2024-09-10T04:34:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.441+0000 7f8cbe7fc640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca807ac00 0x7f8ca807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.442+0000 7f8cbe7fc640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca807ac00 0x7f8ca807d0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00fe660 tx=0x7f8cb4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.445+0000 7f8cc48ac640 1 -- 172.21.15.46:0/963652721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8cb0069200 con 0x7f8cc0100800 2024-09-10T04:34:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.664+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8cc0109e10 con 0x7f8ca807ac00 2024-09-10T04:34:09.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.665+0000 7f8cc48ac640 1 -- 172.21.15.46:0/963652721 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8cb4004260 con 0x7f8ca807ac00 2024-09-10T04:34:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca807ac00 msgr2=0x7f8ca807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca807ac00 0x7f8ca807d0c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8cc00fe660 tx=0x7f8cb4031400 comp rx=0 tx=0).stop 2024-09-10T04:34:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 msgr2=0x7f8cc00fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc00fcbd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d8d0 tx=0x7f8cb000dd90 comp rx=0 tx=0).stop 2024-09-10T04:34:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 shutdown_connections 2024-09-10T04:34:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca807ac00 0x7f8ca807d0c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cc0106800 0x7f8cc00fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 --2- 172.21.15.46:0/963652721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cc0100800 0x7f8cc00fcbd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 >> 172.21.15.46:0/963652721 conn(0x7f8cc00fc520 msgr2=0x7f8cc006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 shutdown_connections 2024-09-10T04:34:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.667+0000 7f8cc58ae640 1 -- 172.21.15.46:0/963652721 wait complete. 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.751+0000 7f2c64b46640 1 -- 172.21.15.46:0/2514052137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 msgr2=0x7f2c60107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.752+0000 7f2c64b46640 1 --2- 172.21.15.46:0/2514052137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c60107ce0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009880 tx=0x7f2c5002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.752+0000 7f2c64b46640 1 -- 172.21.15.46:0/2514052137 shutdown_connections 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.752+0000 7f2c64b46640 1 --2- 172.21.15.46:0/2514052137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c60101900 0x7f2c60101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.752+0000 7f2c64b46640 1 --2- 172.21.15.46:0/2514052137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c60107ce0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.752+0000 7f2c64b46640 1 -- 172.21.15.46:0/2514052137 >> 172.21.15.46:0/2514052137 conn(0x7f2c600fd600 msgr2=0x7f2c600ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.752+0000 7f2c64b46640 1 -- 172.21.15.46:0/2514052137 shutdown_connections 2024-09-10T04:34:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.752+0000 7f2c64b46640 1 -- 172.21.15.46:0/2514052137 wait complete. 2024-09-10T04:34:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c64b46640 1 Processor -- start 2024-09-10T04:34:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c64b46640 1 -- start start 2024-09-10T04:34:09.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c64b46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c60101900 0x7f2c6019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c64b46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c6019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c64b46640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010edd0 con 0x7f2c60107900 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c64b46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6010ec50 con 0x7f2c60101900 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c57fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c6019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c57fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c6019db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50960/0 (socket says 172.21.15.46:50960) 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.753+0000 7f2c57fff640 1 -- 172.21.15.46:0/938329236 learned_addr learned my addr 172.21.15.46:0/938329236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c5e59c640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c60101900 0x7f2c6019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c57fff640 1 -- 172.21.15.46:0/938329236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c60101900 msgr2=0x7f2c6019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c57fff640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c60101900 0x7f2c6019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c57fff640 1 -- 172.21.15.46:0/938329236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c601977a0 con 0x7f2c60107900 2024-09-10T04:34:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c5e59c640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c60101900 0x7f2c6019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:34:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c57fff640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c6019db60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e970 tx=0x7f2c4800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c577fe640 1 -- 172.21.15.46:0/938329236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2c4800cac0 con 0x7f2c60107900 2024-09-10T04:34:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c577fe640 1 -- 172.21.15.46:0/938329236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c480054e0 con 0x7f2c60107900 2024-09-10T04:34:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c577fe640 1 -- 172.21.15.46:0/938329236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2c48010430 con 0x7f2c60107900 2024-09-10T04:34:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c60197a90 con 0x7f2c60107900 2024-09-10T04:34:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.754+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c60198020 con 0x7f2c60107900 2024-09-10T04:34:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.755+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c60103040 con 0x7f2c60107900 2024-09-10T04:34:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.756+0000 7f2c577fe640 1 -- 172.21.15.46:0/938329236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2c48004e80 con 0x7f2c60107900 2024-09-10T04:34:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.757+0000 7f2c577fe640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2c4c07abf0 0x7f2c4c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.757+0000 7f2c5e59c640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2c4c07abf0 0x7f2c4c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.757+0000 7f2c577fe640 1 -- 172.21.15.46:0/938329236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2c4809e380 con 0x7f2c60107900 2024-09-10T04:34:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.758+0000 7f2c5e59c640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2c4c07abf0 0x7f2c4c07d0b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000c00 tx=0x7f2c50031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.762+0000 7f2c577fe640 1 -- 172.21.15.46:0/938329236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2c48069ac0 con 0x7f2c60107900 2024-09-10T04:34:09.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.956+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2c60198390 con 0x7f2c4c07abf0 2024-09-10T04:34:09.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.966+0000 7f2c577fe640 1 -- 172.21.15.46:0/938329236 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f2c60198390 con 0x7f2c4c07abf0 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (25m) 4m ago 37m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (28m) 4m ago 37m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (28m) 4m ago 36m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (24m) 4m ago 37m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (24m) 4m ago 36m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (25m) 4m ago 36m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (20m) 4m ago 31m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (20m) 4m ago 31m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:34:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (20m) 4m ago 31m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (4m) 4m ago 31m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (27m) 4m ago 38m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (26m) 4m ago 36m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (24m) 4m ago 38m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (24m) 4m ago 36m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (26m) 4m ago 37m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (26m) 4m ago 36m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (23m) 4m ago 35m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (23m) 4m ago 34m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (22m) 4m ago 34m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (21m) 4m ago 33m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:34:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (21m) 4m ago 33m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:34:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (20m) 4m ago 32m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:34:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (26m) 4m ago 36m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:34:09.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2c4c07abf0 msgr2=0x7f2c4c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2c4c07abf0 0x7f2c4c07d0b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2c50000c00 tx=0x7f2c50031400 comp rx=0 tx=0).stop 2024-09-10T04:34:09.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 msgr2=0x7f2c6019db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c6019db60 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800e970 tx=0x7f2c4800ed70 comp rx=0 tx=0).stop 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 shutdown_connections 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2c4c07abf0 0x7f2c4c07d0b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c60107900 0x7f2c6019db60 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 --2- 172.21.15.46:0/938329236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c60101900 0x7f2c6019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 >> 172.21.15.46:0/938329236 conn(0x7f2c600fd600 msgr2=0x7f2c600fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 shutdown_connections 2024-09-10T04:34:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:09.968+0000 7f2c64b46640 1 -- 172.21.15.46:0/938329236 wait complete. 2024-09-10T04:34:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/1141069728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64107900 msgr2=0x7f5c64107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/1141069728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64107900 0x7f5c64107ce0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009880 tx=0x7f5c6002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/1141069728 shutdown_connections 2024-09-10T04:34:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/1141069728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c64101900 0x7f5c64101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/1141069728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64107900 0x7f5c64107ce0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/1141069728 >> 172.21.15.46:0/1141069728 conn(0x7f5c640fd600 msgr2=0x7f5c640ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:10.054 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:09 smithi046 ceph-mon[109506]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:10.054 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:09 smithi046 ceph-mon[109506]: pgmap v856: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:10.054 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:09 smithi046 ceph-mon[109506]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/1141069728 shutdown_connections 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.051+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/1141069728 wait complete. 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c6bfb3640 1 Processor -- start 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c6bfb3640 1 -- start start 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c6bfb3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64101900 0x7f5c6419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c6bfb3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c64107900 0x7f5c6419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c6bfb3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6410edd0 con 0x7f5c64101900 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c6bfb3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6410ec50 con 0x7f5c64107900 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c69d4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64101900 0x7f5c6419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c69d4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64101900 0x7f5c6419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50972/0 (socket says 172.21.15.46:50972) 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c69d4f640 1 -- 172.21.15.46:0/178055701 learned_addr learned my addr 172.21.15.46:0/178055701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:10.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.052+0000 7f5c6954e640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c64107900 0x7f5c6419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c69d4f640 1 -- 172.21.15.46:0/178055701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c64107900 msgr2=0x7f5c6419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c69d4f640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c64107900 0x7f5c6419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c69d4f640 1 -- 172.21.15.46:0/178055701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c641977a0 con 0x7f5c64101900 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c69d4f640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64101900 0x7f5c6419d620 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5c60031460 tx=0x7f5c600045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c5affd640 1 -- 172.21.15.46:0/178055701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5c6003f070 con 0x7f5c64101900 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c5affd640 1 -- 172.21.15.46:0/178055701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c600035f0 con 0x7f5c64101900 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c5affd640 1 -- 172.21.15.46:0/178055701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5c60035030 con 0x7f5c64101900 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c64197a30 con 0x7f5c64101900 2024-09-10T04:34:10.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.053+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c64197f60 con 0x7f5c64101900 2024-09-10T04:34:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.054+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c64103040 con 0x7f5c64101900 2024-09-10T04:34:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.055+0000 7f5c5affd640 1 -- 172.21.15.46:0/178055701 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5c60042070 con 0x7f5c64101900 2024-09-10T04:34:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.056+0000 7f5c5affd640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5c5007abf0 0x7f5c5007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.056+0000 7f5c6954e640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5c5007abf0 0x7f5c5007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.056+0000 7f5c5affd640 1 -- 172.21.15.46:0/178055701 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5c600be150 con 0x7f5c64101900 2024-09-10T04:34:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.057+0000 7f5c6954e640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5c5007abf0 0x7f5c5007d0b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5c640ffc20 tx=0x7f5c5400b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.060+0000 7f5c5affd640 1 -- 172.21.15.46:0/178055701 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5c60089890 con 0x7f5c64101900 2024-09-10T04:34:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:09 smithi070 ceph-mon[79774]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:09 smithi070 ceph-mon[79774]: pgmap v856: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:09 smithi070 ceph-mon[79774]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.327+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f5c64109e70 con 0x7f5c64101900 2024-09-10T04:34:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.327+0000 7f5c5affd640 1 -- 172.21.15.46:0/178055701 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f5c6008e740 con 0x7f5c64101900 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:10.330 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:34:10.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:34:10.331 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:34:10.331 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5c5007abf0 msgr2=0x7f5c5007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5c5007abf0 0x7f5c5007d0b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5c640ffc20 tx=0x7f5c5400b390 comp rx=0 tx=0).stop 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64101900 msgr2=0x7f5c6419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64101900 0x7f5c6419d620 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5c60031460 tx=0x7f5c600045e0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 shutdown_connections 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5c5007abf0 0x7f5c5007d0b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c64107900 0x7f5c6419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 --2- 172.21.15.46:0/178055701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c64101900 0x7f5c6419d620 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 >> 172.21.15.46:0/178055701 conn(0x7f5c640fd600 msgr2=0x7f5c640fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 shutdown_connections 2024-09-10T04:34:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.330+0000 7f5c6bfb3640 1 -- 172.21.15.46:0/178055701 wait complete. 2024-09-10T04:34:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 -- 172.21.15.46:0/1482205171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68103440 msgr2=0x7f1b6810b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 --2- 172.21.15.46:0/1482205171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68103440 0x7f1b6810b930 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0098b0 tx=0x7f1b5c02a560 comp rx=0 tx=0).stop 2024-09-10T04:34:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 -- 172.21.15.46:0/1482205171 shutdown_connections 2024-09-10T04:34:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 --2- 172.21.15.46:0/1482205171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68103440 0x7f1b6810b930 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 --2- 172.21.15.46:0/1482205171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b68102b20 0x7f1b68102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 -- 172.21.15.46:0/1482205171 >> 172.21.15.46:0/1482205171 conn(0x7f1b680fc520 msgr2=0x7f1b680fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:10.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 -- 172.21.15.46:0/1482205171 shutdown_connections 2024-09-10T04:34:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 -- 172.21.15.46:0/1482205171 wait complete. 2024-09-10T04:34:10.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.414+0000 7f1b6de66640 1 Processor -- start 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b6de66640 1 -- start start 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b6de66640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68102b20 0x7f1b6819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b6de66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b68103440 0x7f1b6819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b6de66640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6810edf0 con 0x7f1b68102b20 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b6de66640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6810ec70 con 0x7f1b68103440 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68102b20 0x7f1b6819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b677fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68102b20 0x7f1b6819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50988/0 (socket says 172.21.15.46:50988) 2024-09-10T04:34:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b677fe640 1 -- 172.21.15.46:0/2655462530 learned_addr learned my addr 172.21.15.46:0/2655462530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b66ffd640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b68103440 0x7f1b6819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b677fe640 1 -- 172.21.15.46:0/2655462530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b68103440 msgr2=0x7f1b6819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b677fe640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b68103440 0x7f1b6819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.415+0000 7f1b677fe640 1 -- 172.21.15.46:0/2655462530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b68199a60 con 0x7f1b68102b20 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.416+0000 7f1b677fe640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68102b20 0x7f1b6819f8a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c8f0 tx=0x7f1b5800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.416+0000 7f1b64ff9640 1 -- 172.21.15.46:0/2655462530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b580079b0 con 0x7f1b68102b20 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.416+0000 7f1b64ff9640 1 -- 172.21.15.46:0/2655462530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1b580054e0 con 0x7f1b68102b20 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.416+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b68199d50 con 0x7f1b68102b20 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.416+0000 7f1b64ff9640 1 -- 172.21.15.46:0/2655462530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1b58004dd0 con 0x7f1b68102b20 2024-09-10T04:34:10.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.416+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6819a2b0 con 0x7f1b68102b20 2024-09-10T04:34:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.417+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6806aa30 con 0x7f1b68102b20 2024-09-10T04:34:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.417+0000 7f1b64ff9640 1 -- 172.21.15.46:0/2655462530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1b58005020 con 0x7f1b68102b20 2024-09-10T04:34:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.418+0000 7f1b64ff9640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b5007ab90 0x7f1b5007d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.418+0000 7f1b64ff9640 1 -- 172.21.15.46:0/2655462530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1b5809dbb0 con 0x7f1b68102b20 2024-09-10T04:34:10.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.418+0000 7f1b66ffd640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b5007ab90 0x7f1b5007d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.419+0000 7f1b66ffd640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b5007ab90 0x7f1b5007d050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000c00 tx=0x7f1b5c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:10.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.423+0000 7f1b64ff9640 1 -- 172.21.15.46:0/2655462530 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1b580a4080 con 0x7f1b68102b20 2024-09-10T04:34:10.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.651+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1b6819af90 con 0x7f1b68102b20 2024-09-10T04:34:10.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.651+0000 7f1b64ff9640 1 -- 172.21.15.46:0/2655462530 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f1b58014030 con 0x7f1b68102b20 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:34:10.654 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:34:10.655 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:34:10.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:10.657 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:10.657 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:10.657 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:10.657 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b5007ab90 msgr2=0x7f1b5007d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b5007ab90 0x7f1b5007d050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c000c00 tx=0x7f1b5c031400 comp rx=0 tx=0).stop 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68102b20 msgr2=0x7f1b6819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68102b20 0x7f1b6819f8a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800c8f0 tx=0x7f1b5800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 shutdown_connections 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1b5007ab90 0x7f1b5007d050 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b68103440 0x7f1b6819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 --2- 172.21.15.46:0/2655462530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b68102b20 0x7f1b6819f8a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 >> 172.21.15.46:0/2655462530 conn(0x7f1b680fc520 msgr2=0x7f1b68100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:10.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 shutdown_connections 2024-09-10T04:34:10.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.655+0000 7f1b6de66640 1 -- 172.21.15.46:0/2655462530 wait complete. 2024-09-10T04:34:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 -- 172.21.15.46:0/2608880618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100640 msgr2=0x7f43a0100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 --2- 172.21.15.46:0/2608880618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100640 0x7f43a0100a20 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f439c009880 tx=0x7f439c02a490 comp rx=0 tx=0).stop 2024-09-10T04:34:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 -- 172.21.15.46:0/2608880618 shutdown_connections 2024-09-10T04:34:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 --2- 172.21.15.46:0/2608880618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43a0100ff0 0x7f43a010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 --2- 172.21.15.46:0/2608880618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100640 0x7f43a0100a20 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 -- 172.21.15.46:0/2608880618 >> 172.21.15.46:0/2608880618 conn(0x7f43a00fc550 msgr2=0x7f43a00fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 -- 172.21.15.46:0/2608880618 shutdown_connections 2024-09-10T04:34:10.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.741+0000 7f43a7282640 1 -- 172.21.15.46:0/2608880618 wait complete. 2024-09-10T04:34:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a7282640 1 Processor -- start 2024-09-10T04:34:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a7282640 1 -- start start 2024-09-10T04:34:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a7282640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43a0100640 0x7f43a019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a7282640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100ff0 0x7f43a019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a7282640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0110f80 con 0x7f43a0100ff0 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a7282640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f43a0110e00 con 0x7f43a0100640 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a481d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100ff0 0x7f43a019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a481d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100ff0 0x7f43a019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51008/0 (socket says 172.21.15.46:51008) 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a481d640 1 -- 172.21.15.46:0/1809431599 learned_addr learned my addr 172.21.15.46:0/1809431599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.742+0000 7f43a501e640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43a0100640 0x7f43a019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43a481d640 1 -- 172.21.15.46:0/1809431599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43a0100640 msgr2=0x7f43a019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43a481d640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43a0100640 0x7f43a019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43a481d640 1 -- 172.21.15.46:0/1809431599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a0199970 con 0x7f43a0100ff0 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43a481d640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100ff0 0x7f43a019fcf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f439000d6f0 tx=0x7f439000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43967fc640 1 -- 172.21.15.46:0/1809431599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4390017070 con 0x7f43a0100ff0 2024-09-10T04:34:10.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43967fc640 1 -- 172.21.15.46:0/1809431599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f439000de00 con 0x7f43a0100ff0 2024-09-10T04:34:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a0199c60 con 0x7f43a0100ff0 2024-09-10T04:34:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43967fc640 1 -- 172.21.15.46:0/1809431599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4390004bb0 con 0x7f43a0100ff0 2024-09-10T04:34:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.743+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a019a1f0 con 0x7f43a0100ff0 2024-09-10T04:34:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.744+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a006a580 con 0x7f43a0100ff0 2024-09-10T04:34:10.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.746+0000 7f43967fc640 1 -- 172.21.15.46:0/1809431599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4390004e60 con 0x7f43a0100ff0 2024-09-10T04:34:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.746+0000 7f43967fc640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f438c07abf0 0x7f438c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.746+0000 7f43967fc640 1 -- 172.21.15.46:0/1809431599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f439009d970 con 0x7f43a0100ff0 2024-09-10T04:34:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.746+0000 7f43a501e640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f438c07abf0 0x7f438c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:10.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.747+0000 7f43a501e640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f438c07abf0 0x7f438c07d0b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c00 tx=0x7f439c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:10.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.751+0000 7f43967fc640 1 -- 172.21.15.46:0/1809431599 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4390069010 con 0x7f43a0100ff0 2024-09-10T04:34:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.954+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f43a019aea0 con 0x7f438c07abf0 2024-09-10T04:34:10.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:10 smithi046 ceph-mon[109506]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:10.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:10 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/178055701' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:34:10.956 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:10 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2655462530' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.955+0000 7f43967fc640 1 -- 172.21.15.46:0/1809431599 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f43a019aea0 con 0x7f438c07abf0 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:34:10.957 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:34:10.958 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:34:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f438c07abf0 msgr2=0x7f438c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f438c07abf0 0x7f438c07d0b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c00 tx=0x7f439c031400 comp rx=0 tx=0).stop 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100ff0 msgr2=0x7f43a019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100ff0 0x7f43a019fcf0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f439000d6f0 tx=0x7f439000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 shutdown_connections 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f438c07abf0 0x7f438c07d0b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43a0100ff0 0x7f43a019fcf0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 --2- 172.21.15.46:0/1809431599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43a0100640 0x7f43a019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 >> 172.21.15.46:0/1809431599 conn(0x7f43a00fc550 msgr2=0x7f43a00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 shutdown_connections 2024-09-10T04:34:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:10.958+0000 7f43a7282640 1 -- 172.21.15.46:0/1809431599 wait complete. 2024-09-10T04:34:11.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 -- 172.21.15.46:0/2147017163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710107900 msgr2=0x7f3710107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 --2- 172.21.15.46:0/2147017163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710107900 0x7f3710107ce0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3700009880 tx=0x7f370002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 -- 172.21.15.46:0/2147017163 shutdown_connections 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 --2- 172.21.15.46:0/2147017163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3710101900 0x7f3710101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 --2- 172.21.15.46:0/2147017163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710107900 0x7f3710107ce0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 -- 172.21.15.46:0/2147017163 >> 172.21.15.46:0/2147017163 conn(0x7f37100fd600 msgr2=0x7f37100ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 -- 172.21.15.46:0/2147017163 shutdown_connections 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.042+0000 7f3714f61640 1 -- 172.21.15.46:0/2147017163 wait complete. 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f3714f61640 1 Processor -- start 2024-09-10T04:34:11.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f3714f61640 1 -- start start 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f3714f61640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710101900 0x7f37100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f3714f61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3710107900 0x7f3710074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f3714f61640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f371010edb0 con 0x7f3710101900 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f3714f61640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f371010ec30 con 0x7f3710107900 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f370e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710101900 0x7f37100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f370e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710101900 0x7f37100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51024/0 (socket says 172.21.15.46:51024) 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f370e59c640 1 -- 172.21.15.46:0/2018774671 learned_addr learned my addr 172.21.15.46:0/2018774671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.043+0000 7f3707fff640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3710107900 0x7f3710074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f370e59c640 1 -- 172.21.15.46:0/2018774671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3710107900 msgr2=0x7f3710074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:11.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f370e59c640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3710107900 0x7f3710074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f370e59c640 1 -- 172.21.15.46:0/2018774671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3710074f70 con 0x7f3710101900 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f3707fff640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3710107900 0x7f3710074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f370e59c640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710101900 0x7f37100744f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3700031460 tx=0x7f3700004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f37077fe640 1 -- 172.21.15.46:0/2018774671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f370003f070 con 0x7f3710101900 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f37077fe640 1 -- 172.21.15.46:0/2018774671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f37000035f0 con 0x7f3710101900 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3710075200 con 0x7f3710101900 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f37077fe640 1 -- 172.21.15.46:0/2018774671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3700035030 con 0x7f3710101900 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.044+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37101a3d80 con 0x7f3710101900 2024-09-10T04:34:11.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.045+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3710103040 con 0x7f3710101900 2024-09-10T04:34:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.046+0000 7f37077fe640 1 -- 172.21.15.46:0/2018774671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3700042070 con 0x7f3710101900 2024-09-10T04:34:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.046+0000 7f37077fe640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36fc07ac40 0x7f36fc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.046+0000 7f37077fe640 1 -- 172.21.15.46:0/2018774671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f37000be4a0 con 0x7f3710101900 2024-09-10T04:34:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.046+0000 7f3707fff640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36fc07ac40 0x7f36fc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:11.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.047+0000 7f3707fff640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36fc07ac40 0x7f36fc07d100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3710075f80 tx=0x7f36f8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:11.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.051+0000 7f37077fe640 1 -- 172.21.15.46:0/2018774671 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3700089c60 con 0x7f3710101900 2024-09-10T04:34:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:10 smithi070 ceph-mon[79774]: from='client.35410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:10 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/178055701' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:34:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:10 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2655462530' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:34:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.309+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f371002d7c0 con 0x7f3710101900 2024-09-10T04:34:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.309+0000 7f37077fe640 1 -- 172.21.15.46:0/2018774671 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f370008eb10 con 0x7f3710101900 2024-09-10T04:34:11.313 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:34:11.313 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:34:11.313 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:34:11.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:34:11.314 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:34:11.315 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36fc07ac40 msgr2=0x7f36fc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36fc07ac40 0x7f36fc07d100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3710075f80 tx=0x7f36f8005f90 comp rx=0 tx=0).stop 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710101900 msgr2=0x7f37100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710101900 0x7f37100744f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3700031460 tx=0x7f3700004530 comp rx=0 tx=0).stop 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 shutdown_connections 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f36fc07ac40 0x7f36fc07d100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3710107900 0x7f3710074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 --2- 172.21.15.46:0/2018774671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3710101900 0x7f37100744f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 >> 172.21.15.46:0/2018774671 conn(0x7f37100fd600 msgr2=0x7f37100ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:11.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 shutdown_connections 2024-09-10T04:34:11.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:11.312+0000 7f3714f61640 1 -- 172.21.15.46:0/2018774671 wait complete. 2024-09-10T04:34:12.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:11 smithi070 ceph-mon[79774]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:12.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:11 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2018774671' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:34:12.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:11 smithi070 ceph-mon[79774]: pgmap v857: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:11 smithi046 ceph-mon[109506]: from='client.35422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:11 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2018774671' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:34:12.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:11 smithi046 ceph-mon[109506]: pgmap v857: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:14 smithi070 ceph-mon[79774]: pgmap v858: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:14 smithi046 ceph-mon[109506]: pgmap v858: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:16.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:16 smithi070 ceph-mon[79774]: pgmap v859: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:16.838 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:16 smithi046 ceph-mon[109506]: pgmap v859: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:18 smithi070 ceph-mon[79774]: pgmap v860: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:18 smithi046 ceph-mon[109506]: pgmap v860: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:20 smithi070 ceph-mon[79774]: pgmap v861: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:20.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:20 smithi046 ceph-mon[109506]: pgmap v861: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:22 smithi070 ceph-mon[79774]: pgmap v862: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:22 smithi046 ceph-mon[109506]: pgmap v862: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:24 smithi070 ceph-mon[79774]: pgmap v863: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:24.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:24 smithi046 ceph-mon[109506]: pgmap v863: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:26 smithi070 ceph-mon[79774]: pgmap v864: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:26.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:26 smithi046 ceph-mon[109506]: pgmap v864: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:28 smithi070 ceph-mon[79774]: pgmap v865: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:28.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:28 smithi046 ceph-mon[109506]: pgmap v865: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:31.030 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:30 smithi046 ceph-mon[109506]: pgmap v866: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:30 smithi070 ceph-mon[79774]: pgmap v866: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:32 smithi070 ceph-mon[79774]: pgmap v867: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:32 smithi046 ceph-mon[109506]: pgmap v867: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:34 smithi070 ceph-mon[79774]: pgmap v868: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:34 smithi046 ceph-mon[109506]: pgmap v868: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:36 smithi070 ceph-mon[79774]: pgmap v869: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:36 smithi046 ceph-mon[109506]: pgmap v869: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:38 smithi070 ceph-mon[79774]: pgmap v870: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:38 smithi046 ceph-mon[109506]: pgmap v870: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:40 smithi070 ceph-mon[79774]: pgmap v871: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:40 smithi046 ceph-mon[109506]: pgmap v871: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 -- 172.21.15.46:0/3113780969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501067f0 msgr2=0x7f9050106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 --2- 172.21.15.46:0/3113780969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501067f0 0x7f9050106bd0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9040009880 tx=0x7f904002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 -- 172.21.15.46:0/3113780969 shutdown_connections 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 --2- 172.21.15.46:0/3113780969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90501007f0 0x7f9050100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 --2- 172.21.15.46:0/3113780969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501067f0 0x7f9050106bd0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 -- 172.21.15.46:0/3113780969 >> 172.21.15.46:0/3113780969 conn(0x7f90500fc550 msgr2=0x7f90500fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 -- 172.21.15.46:0/3113780969 shutdown_connections 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.403+0000 7f9054c8f640 1 -- 172.21.15.46:0/3113780969 wait complete. 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f9054c8f640 1 Processor -- start 2024-09-10T04:34:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f9054c8f640 1 -- start start 2024-09-10T04:34:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f9054c8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501007f0 0x7f905019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f9054c8f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90501067f0 0x7f905019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f9054c8f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f905010dcc0 con 0x7f90501007f0 2024-09-10T04:34:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f9054c8f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f905010db40 con 0x7f90501067f0 2024-09-10T04:34:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f904e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501007f0 0x7f905019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f904e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501007f0 0x7f905019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46010/0 (socket says 172.21.15.46:46010) 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f904e59c640 1 -- 172.21.15.46:0/2459051203 learned_addr learned my addr 172.21.15.46:0/2459051203 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.404+0000 7f904dd9b640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90501067f0 0x7f905019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f904e59c640 1 -- 172.21.15.46:0/2459051203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90501067f0 msgr2=0x7f905019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f904e59c640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90501067f0 0x7f905019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f904e59c640 1 -- 172.21.15.46:0/2459051203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90501977f0 con 0x7f90501007f0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f904e59c640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501007f0 0x7f905019d670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9040031460 tx=0x7f90400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f903f7fe640 1 -- 172.21.15.46:0/2459051203 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f904003f070 con 0x7f90501007f0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f903f7fe640 1 -- 172.21.15.46:0/2459051203 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f90400035f0 con 0x7f90501007f0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9050197a80 con 0x7f90501007f0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f903f7fe640 1 -- 172.21.15.46:0/2459051203 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9040035030 con 0x7f90501007f0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.405+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9050197fb0 con 0x7f90501007f0 2024-09-10T04:34:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.406+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9050101f30 con 0x7f90501007f0 2024-09-10T04:34:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.407+0000 7f903f7fe640 1 -- 172.21.15.46:0/2459051203 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9040042070 con 0x7f90501007f0 2024-09-10T04:34:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.407+0000 7f903f7fe640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f903807ac40 0x7f903807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.407+0000 7f903f7fe640 1 -- 172.21.15.46:0/2459051203 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f90400be640 con 0x7f90501007f0 2024-09-10T04:34:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.407+0000 7f904dd9b640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f903807ac40 0x7f903807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.408+0000 7f904dd9b640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f903807ac40 0x7f903807d100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9044005fd0 tx=0x7f9044005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:41.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.412+0000 7f903f7fe640 1 -- 172.21.15.46:0/2459051203 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9040089d60 con 0x7f90501007f0 2024-09-10T04:34:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.618+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9050198320 con 0x7f903807ac40 2024-09-10T04:34:41.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.619+0000 7f903f7fe640 1 -- 172.21.15.46:0/2459051203 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f9050198320 con 0x7f903807ac40 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f903807ac40 msgr2=0x7f903807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f903807ac40 0x7f903807d100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9044005fd0 tx=0x7f9044005850 comp rx=0 tx=0).stop 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501007f0 msgr2=0x7f905019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501007f0 0x7f905019d670 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9040031460 tx=0x7f90400045e0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 shutdown_connections 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f903807ac40 0x7f903807d100 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f90501067f0 0x7f905019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 --2- 172.21.15.46:0/2459051203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90501007f0 0x7f905019d670 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 >> 172.21.15.46:0/2459051203 conn(0x7f90500fc550 msgr2=0x7f9050069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 shutdown_connections 2024-09-10T04:34:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.622+0000 7f9054c8f640 1 -- 172.21.15.46:0/2459051203 wait complete. 2024-09-10T04:34:41.632 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:34:41.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 -- 172.21.15.46:0/1729719244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 msgr2=0x7f6018100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 --2- 172.21.15.46:0/1729719244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f6018100c60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6000009880 tx=0x7f600002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 -- 172.21.15.46:0/1729719244 shutdown_connections 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 --2- 172.21.15.46:0/1729719244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f6018100c60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 --2- 172.21.15.46:0/1729719244 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6018106800 0x7f6018106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 -- 172.21.15.46:0/1729719244 >> 172.21.15.46:0/1729719244 conn(0x7f60180fc520 msgr2=0x7f60180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 -- 172.21.15.46:0/1729719244 shutdown_connections 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.707+0000 7f601c89a640 1 -- 172.21.15.46:0/1729719244 wait complete. 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601c89a640 1 Processor -- start 2024-09-10T04:34:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601c89a640 1 -- start start 2024-09-10T04:34:41.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601c89a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f60181987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601c89a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6018106800 0x7f6018198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601c89a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f601810dcd0 con 0x7f6018100800 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601c89a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f601810db50 con 0x7f6018106800 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f60181987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f60181987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46022/0 (socket says 172.21.15.46:46022) 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601659c640 1 -- 172.21.15.46:0/243773915 learned_addr learned my addr 172.21.15.46:0/243773915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f600dd9b640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6018106800 0x7f6018198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601659c640 1 -- 172.21.15.46:0/243773915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6018106800 msgr2=0x7f6018198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.708+0000 7f601659c640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6018106800 0x7f6018198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.709+0000 7f601659c640 1 -- 172.21.15.46:0/243773915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6018199320 con 0x7f6018100800 2024-09-10T04:34:41.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.709+0000 7f601659c640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f60181987e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f600800c8f0 tx=0x7f600800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.709+0000 7f600ffff640 1 -- 172.21.15.46:0/243773915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f60080079b0 con 0x7f6018100800 2024-09-10T04:34:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.709+0000 7f600ffff640 1 -- 172.21.15.46:0/243773915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f60080054e0 con 0x7f6018100800 2024-09-10T04:34:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.709+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6018199610 con 0x7f6018100800 2024-09-10T04:34:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.709+0000 7f600ffff640 1 -- 172.21.15.46:0/243773915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6008004dd0 con 0x7f6018100800 2024-09-10T04:34:41.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.709+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60181a1c10 con 0x7f6018100800 2024-09-10T04:34:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.710+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6018101f40 con 0x7f6018100800 2024-09-10T04:34:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.711+0000 7f600ffff640 1 -- 172.21.15.46:0/243773915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6008005020 con 0x7f6018100800 2024-09-10T04:34:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.711+0000 7f600ffff640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f600407ab90 0x7f600407d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.712+0000 7f600ffff640 1 -- 172.21.15.46:0/243773915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f600809dc50 con 0x7f6018100800 2024-09-10T04:34:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.712+0000 7f600dd9b640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f600407ab90 0x7f600407d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:41.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.712+0000 7f600dd9b640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f600407ab90 0x7f600407d050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f601806ad00 tx=0x7f6000031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:41.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.716+0000 7f600ffff640 1 -- 172.21.15.46:0/243773915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6018199610 con 0x7f6018100800 2024-09-10T04:34:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.922+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6018062630 con 0x7f600407ab90 2024-09-10T04:34:41.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.923+0000 7f600ffff640 1 -- 172.21.15.46:0/243773915 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f6018062630 con 0x7f600407ab90 2024-09-10T04:34:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f600407ab90 msgr2=0x7f600407d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f600407ab90 0x7f600407d050 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f601806ad00 tx=0x7f6000031420 comp rx=0 tx=0).stop 2024-09-10T04:34:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 msgr2=0x7f60181987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f60181987e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f600800c8f0 tx=0x7f600800cdb0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 shutdown_connections 2024-09-10T04:34:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f600407ab90 0x7f600407d050 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6018106800 0x7f6018198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 --2- 172.21.15.46:0/243773915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6018100800 0x7f60181987e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 >> 172.21.15.46:0/243773915 conn(0x7f60180fc520 msgr2=0x7f60180fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 shutdown_connections 2024-09-10T04:34:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:41.926+0000 7f601c89a640 1 -- 172.21.15.46:0/243773915 wait complete. 2024-09-10T04:34:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 -- 172.21.15.46:0/2308672503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074590 msgr2=0x7fa98c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 --2- 172.21.15.46:0/2308672503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074590 0x7fa98c074970 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009880 tx=0x7fa97c02a530 comp rx=0 tx=0).stop 2024-09-10T04:34:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 -- 172.21.15.46:0/2308672503 shutdown_connections 2024-09-10T04:34:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 --2- 172.21.15.46:0/2308672503 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa98c074eb0 0x7fa98c1123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 --2- 172.21.15.46:0/2308672503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074590 0x7fa98c074970 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 -- 172.21.15.46:0/2308672503 >> 172.21.15.46:0/2308672503 conn(0x7fa98c0fd600 msgr2=0x7fa98c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 -- 172.21.15.46:0/2308672503 shutdown_connections 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.011+0000 7fa990d90640 1 -- 172.21.15.46:0/2308672503 wait complete. 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa990d90640 1 Processor -- start 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa990d90640 1 -- start start 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa990d90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa98c074590 0x7fa98c19dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa990d90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074eb0 0x7fa98c19e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa990d90640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1153d0 con 0x7fa98c074eb0 2024-09-10T04:34:42.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa990d90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c115250 con 0x7fa98c074590 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa989d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074eb0 0x7fa98c19e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa989d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074eb0 0x7fa98c19e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46036/0 (socket says 172.21.15.46:46036) 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.012+0000 7fa989d9b640 1 -- 172.21.15.46:0/1562161685 learned_addr learned my addr 172.21.15.46:0/1562161685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa98a59c640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa98c074590 0x7fa98c19dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa989d9b640 1 -- 172.21.15.46:0/1562161685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa98c074590 msgr2=0x7fa98c19dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa989d9b640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa98c074590 0x7fa98c19dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa989d9b640 1 -- 172.21.15.46:0/1562161685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c1a28a0 con 0x7fa98c074eb0 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa98a59c640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa98c074590 0x7fa98c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa989d9b640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074eb0 0x7fa98c19e510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa98000c810 tx=0x7fa98000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa97b7fe640 1 -- 172.21.15.46:0/1562161685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa980004020 con 0x7fa98c074eb0 2024-09-10T04:34:42.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa97b7fe640 1 -- 172.21.15.46:0/1562161685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa9800041c0 con 0x7fa98c074eb0 2024-09-10T04:34:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa97b7fe640 1 -- 172.21.15.46:0/1562161685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa9800047a0 con 0x7fa98c074eb0 2024-09-10T04:34:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c1a2b90 con 0x7fa98c074eb0 2024-09-10T04:34:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.013+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c1a3120 con 0x7fa98c074eb0 2024-09-10T04:34:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.014+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c075cd0 con 0x7fa98c074eb0 2024-09-10T04:34:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.015+0000 7fa97b7fe640 1 -- 172.21.15.46:0/1562161685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa9800079b0 con 0x7fa98c074eb0 2024-09-10T04:34:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.015+0000 7fa97b7fe640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa97407ac00 0x7fa97407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.016+0000 7fa97b7fe640 1 -- 172.21.15.46:0/1562161685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa98009db80 con 0x7fa98c074eb0 2024-09-10T04:34:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.016+0000 7fa98a59c640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa97407ac00 0x7fa97407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.016+0000 7fa98a59c640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa97407ac00 0x7fa97407d0c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa97c031460 tx=0x7fa97c009580 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.020+0000 7fa97b7fe640 1 -- 172.21.15.46:0/1562161685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa980014030 con 0x7fa98c074eb0 2024-09-10T04:34:42.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.213+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa98c19f390 con 0x7fa97407ac00 2024-09-10T04:34:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.224+0000 7fa97b7fe640 1 -- 172.21.15.46:0/1562161685 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fa98c19f390 con 0x7fa97407ac00 2024-09-10T04:34:42.225 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:34:42.225 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (26m) 5m ago 37m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (28m) 5m ago 37m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (28m) 5m ago 36m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (24m) 5m ago 37m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (24m) 5m ago 36m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (25m) 5m ago 37m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (20m) 5m ago 32m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (21m) 5m ago 32m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (20m) 5m ago 32m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (5m) 5m ago 32m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (28m) 5m ago 38m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (27m) 5m ago 36m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (25m) 5m ago 39m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:34:42.226 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (24m) 5m ago 36m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (27m) 5m ago 37m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (27m) 5m ago 36m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (24m) 5m ago 35m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (23m) 5m ago 35m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (23m) 5m ago 34m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (22m) 5m ago 34m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (21m) 5m ago 33m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (21m) 5m ago 33m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:34:42.227 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (26m) 5m ago 37m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:34:42.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa97407ac00 msgr2=0x7fa97407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa97407ac00 0x7fa97407d0c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa97c031460 tx=0x7fa97c009580 comp rx=0 tx=0).stop 2024-09-10T04:34:42.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074eb0 msgr2=0x7fa98c19e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074eb0 0x7fa98c19e510 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa98000c810 tx=0x7fa98000ccd0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 shutdown_connections 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa97407ac00 0x7fa97407d0c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa98c074eb0 0x7fa98c19e510 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 --2- 172.21.15.46:0/1562161685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa98c074590 0x7fa98c19dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 >> 172.21.15.46:0/1562161685 conn(0x7fa98c0fd600 msgr2=0x7fa98c104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 shutdown_connections 2024-09-10T04:34:42.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.226+0000 7fa990d90640 1 -- 172.21.15.46:0/1562161685 wait complete. 2024-09-10T04:34:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1000035676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 msgr2=0x7fb198102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1000035676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb198102f30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009880 tx=0x7fb18c02a530 comp rx=0 tx=0).stop 2024-09-10T04:34:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1000035676 shutdown_connections 2024-09-10T04:34:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1000035676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198103470 0x7fb19810b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1000035676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb198102f30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1000035676 >> 172.21.15.46:0/1000035676 conn(0x7fb1980fc550 msgr2=0x7fb1980fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1000035676 shutdown_connections 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.309+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1000035676 wait complete. 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19f0aa640 1 Processor -- start 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19f0aa640 1 -- start start 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19f0aa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb19819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19f0aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198103470 0x7fb19819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19f0aa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb19810edc0 con 0x7fb198102b50 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19f0aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb19810ec40 con 0x7fb198103470 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19ce46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb19819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19ce46640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb19819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46064/0 (socket says 172.21.15.46:46064) 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19ce46640 1 -- 172.21.15.46:0/1474806900 learned_addr learned my addr 172.21.15.46:0/1474806900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb197fff640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198103470 0x7fb19819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19ce46640 1 -- 172.21.15.46:0/1474806900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198103470 msgr2=0x7fb19819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.310+0000 7fb19ce46640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198103470 0x7fb19819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.311+0000 7fb19ce46640 1 -- 172.21.15.46:0/1474806900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1981977b0 con 0x7fb198102b50 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.311+0000 7fb19ce46640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb19819d630 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb18c004f90 tx=0x7fb18c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.311+0000 7fb195ffb640 1 -- 172.21.15.46:0/1474806900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb18c03f070 con 0x7fb198102b50 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.311+0000 7fb195ffb640 1 -- 172.21.15.46:0/1474806900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb18c004c70 con 0x7fb198102b50 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.311+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb198197a40 con 0x7fb198102b50 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.311+0000 7fb195ffb640 1 -- 172.21.15.46:0/1474806900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb18c035030 con 0x7fb198102b50 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.311+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb198197f70 con 0x7fb198102b50 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.312+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb19806a4e0 con 0x7fb198102b50 2024-09-10T04:34:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.313+0000 7fb195ffb640 1 -- 172.21.15.46:0/1474806900 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb18c042070 con 0x7fb198102b50 2024-09-10T04:34:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.314+0000 7fb195ffb640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18807abf0 0x7fb18807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.314+0000 7fb197fff640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18807abf0 0x7fb18807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.314+0000 7fb195ffb640 1 -- 172.21.15.46:0/1474806900 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb18c0be5a0 con 0x7fb198102b50 2024-09-10T04:34:42.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.314+0000 7fb197fff640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18807abf0 0x7fb18807d0b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb198108920 tx=0x7fb18400b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.318+0000 7fb195ffb640 1 -- 172.21.15.46:0/1474806900 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb18c089ce0 con 0x7fb198102b50 2024-09-10T04:34:42.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.580+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fb198105fb0 con 0x7fb198102b50 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.581+0000 7fb195ffb640 1 -- 172.21.15.46:0/1474806900 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fb18c08eb90 con 0x7fb198102b50 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:34:42.583 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:34:42.584 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.583+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18807abf0 msgr2=0x7fb18807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.583+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18807abf0 0x7fb18807d0b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb198108920 tx=0x7fb18400b410 comp rx=0 tx=0).stop 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.583+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 msgr2=0x7fb19819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.583+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb19819d630 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb18c004f90 tx=0x7fb18c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.584+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 shutdown_connections 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.584+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb18807abf0 0x7fb18807d0b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.584+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb198103470 0x7fb19819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.584+0000 7fb19f0aa640 1 --2- 172.21.15.46:0/1474806900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb198102b50 0x7fb19819d630 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.584+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 >> 172.21.15.46:0/1474806900 conn(0x7fb1980fc550 msgr2=0x7fb198108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:42.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.584+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 shutdown_connections 2024-09-10T04:34:42.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.584+0000 7fb19f0aa640 1 -- 172.21.15.46:0/1474806900 wait complete. 2024-09-10T04:34:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 -- 172.21.15.46:0/3730458079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 msgr2=0x7f2aac107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3730458079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac107ce0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009880 tx=0x7f2aa802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 -- 172.21.15.46:0/3730458079 shutdown_connections 2024-09-10T04:34:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3730458079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac101900 0x7f2aac101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3730458079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac107ce0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 -- 172.21.15.46:0/3730458079 >> 172.21.15.46:0/3730458079 conn(0x7f2aac0fd600 msgr2=0x7f2aac0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:42.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 -- 172.21.15.46:0/3730458079 shutdown_connections 2024-09-10T04:34:42.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.669+0000 7f2ab377f640 1 -- 172.21.15.46:0/3730458079 wait complete. 2024-09-10T04:34:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab377f640 1 Processor -- start 2024-09-10T04:34:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab377f640 1 -- start start 2024-09-10T04:34:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab377f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac101900 0x7f2aac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab377f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab377f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac10edd0 con 0x7f2aac107900 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab377f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2aac10ec50 con 0x7f2aac101900 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab0d1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab0d1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46092/0 (socket says 172.21.15.46:46092) 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab0d1a640 1 -- 172.21.15.46:0/3067943378 learned_addr learned my addr 172.21.15.46:0/3067943378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab151b640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac101900 0x7f2aac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab0d1a640 1 -- 172.21.15.46:0/3067943378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac101900 msgr2=0x7f2aac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab0d1a640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac101900 0x7f2aac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.670+0000 7f2ab0d1a640 1 -- 172.21.15.46:0/3067943378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aac074f70 con 0x7f2aac107900 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.671+0000 7f2ab0d1a640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac074a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c8f0 tx=0x7f2a9c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.671+0000 7f2aa27fc640 1 -- 172.21.15.46:0/3067943378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a9c0079b0 con 0x7f2aac107900 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.671+0000 7f2aa27fc640 1 -- 172.21.15.46:0/3067943378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a9c0054e0 con 0x7f2aac107900 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.671+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aac075260 con 0x7f2aac107900 2024-09-10T04:34:42.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.671+0000 7f2aa27fc640 1 -- 172.21.15.46:0/3067943378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2a9c004dd0 con 0x7f2aac107900 2024-09-10T04:34:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.671+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aac1a3e20 con 0x7f2aac107900 2024-09-10T04:34:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.672+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aac103040 con 0x7f2aac107900 2024-09-10T04:34:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.673+0000 7f2aa27fc640 1 -- 172.21.15.46:0/3067943378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2a9c005020 con 0x7f2aac107900 2024-09-10T04:34:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.673+0000 7f2aa27fc640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9807ac00 0x7f2a9807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.674+0000 7f2ab151b640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9807ac00 0x7f2a9807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.674+0000 7f2aa27fc640 1 -- 172.21.15.46:0/3067943378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2a9c09dbb0 con 0x7f2aac107900 2024-09-10T04:34:42.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.674+0000 7f2ab151b640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9807ac00 0x7f2a9807d0c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8031460 tx=0x7f2aa8009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.678+0000 7f2aa27fc640 1 -- 172.21.15.46:0/3067943378 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2a9c0a4080 con 0x7f2aac107900 2024-09-10T04:34:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:42 smithi046 ceph-mon[109506]: pgmap v872: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:42 smithi046 ceph-mon[109506]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:42 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1474806900' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:34:42.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.901+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2aac109e70 con 0x7f2aac107900 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.902+0000 7f2aa27fc640 1 -- 172.21.15.46:0/3067943378 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f2a9c014030 con 0x7f2aac107900 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:34:42.904 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:34:42.905 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:34:42.906 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:42.907 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:34:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:34:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9807ac00 msgr2=0x7f2a9807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9807ac00 0x7f2a9807d0c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8031460 tx=0x7f2aa8009560 comp rx=0 tx=0).stop 2024-09-10T04:34:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 msgr2=0x7f2aac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac074a30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00c8f0 tx=0x7f2a9c00cdb0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 shutdown_connections 2024-09-10T04:34:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2a9807ac00 0x7f2a9807d0c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aac107900 0x7f2aac074a30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 --2- 172.21.15.46:0/3067943378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aac101900 0x7f2aac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 >> 172.21.15.46:0/3067943378 conn(0x7f2aac0fd600 msgr2=0x7f2aac0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 shutdown_connections 2024-09-10T04:34:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.904+0000 7f2ab377f640 1 -- 172.21.15.46:0/3067943378 wait complete. 2024-09-10T04:34:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 -- 172.21.15.46:0/3502088810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 msgr2=0x7fdcb410b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3502088810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb410b930 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdca80098b0 tx=0x7fdca802a560 comp rx=0 tx=0).stop 2024-09-10T04:34:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 -- 172.21.15.46:0/3502088810 shutdown_connections 2024-09-10T04:34:42.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3502088810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb410b930 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3502088810 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb4102b20 0x7fdcb4102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 -- 172.21.15.46:0/3502088810 >> 172.21.15.46:0/3502088810 conn(0x7fdcb40fc520 msgr2=0x7fdcb40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 -- 172.21.15.46:0/3502088810 shutdown_connections 2024-09-10T04:34:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 -- 172.21.15.46:0/3502088810 wait complete. 2024-09-10T04:34:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.989+0000 7fdcb956f640 1 Processor -- start 2024-09-10T04:34:42.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb956f640 1 -- start start 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb956f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb4102b20 0x7fdcb419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb956f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb419dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb956f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410edf0 con 0x7fdcb4103440 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb956f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410ec70 con 0x7fdcb4102b20 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb419dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb419dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46116/0 (socket says 172.21.15.46:46116) 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb27fc640 1 -- 172.21.15.46:0/3464426308 learned_addr learned my addr 172.21.15.46:0/3464426308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb2ffd640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb4102b20 0x7fdcb419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb27fc640 1 -- 172.21.15.46:0/3464426308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb4102b20 msgr2=0x7fdcb419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb27fc640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb4102b20 0x7fdcb419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.990+0000 7fdcb27fc640 1 -- 172.21.15.46:0/3464426308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb41978a0 con 0x7fdcb4103440 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.991+0000 7fdcb27fc640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb419dc20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdca8031460 tx=0x7fdca80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.991+0000 7fdc9bfff640 1 -- 172.21.15.46:0/3464426308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdca803f070 con 0x7fdcb4103440 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.991+0000 7fdc9bfff640 1 -- 172.21.15.46:0/3464426308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdca80035f0 con 0x7fdcb4103440 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.991+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb4197b30 con 0x7fdcb4103440 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.991+0000 7fdc9bfff640 1 -- 172.21.15.46:0/3464426308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdca8035030 con 0x7fdcb4103440 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.991+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb4198060 con 0x7fdcb4103440 2024-09-10T04:34:42.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.992+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb406aa30 con 0x7fdcb4103440 2024-09-10T04:34:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.993+0000 7fdc9bfff640 1 -- 172.21.15.46:0/3464426308 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdca8042070 con 0x7fdcb4103440 2024-09-10T04:34:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.993+0000 7fdc9bfff640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc9c07ac40 0x7fdc9c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.993+0000 7fdc9bfff640 1 -- 172.21.15.46:0/3464426308 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdca80be480 con 0x7fdcb4103440 2024-09-10T04:34:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.993+0000 7fdcb2ffd640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc9c07ac40 0x7fdc9c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:42.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.994+0000 7fdcb2ffd640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc9c07ac40 0x7fdc9c07d100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdcb41989e0 tx=0x7fdca4005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:42.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:42.997+0000 7fdc9bfff640 1 -- 172.21.15.46:0/3464426308 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdca8089c40 con 0x7fdcb4103440 2024-09-10T04:34:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:42 smithi070 ceph-mon[79774]: pgmap v872: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:42 smithi070 ceph-mon[79774]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:42 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1474806900' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:34:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.202+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdcb4106020 con 0x7fdc9c07ac40 2024-09-10T04:34:43.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.203+0000 7fdc9bfff640 1 -- 172.21.15.46:0/3464426308 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fdca401b3f0 con 0x7fdc9c07ac40 2024-09-10T04:34:43.205 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:34:43.205 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:34:43.205 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:34:43.205 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:34:43.205 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:34:43.206 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:34:43.207 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:34:43.207 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.205+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc9c07ac40 msgr2=0x7fdc9c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.205+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc9c07ac40 0x7fdc9c07d100 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdcb41989e0 tx=0x7fdca4005f90 comp rx=0 tx=0).stop 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.205+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 msgr2=0x7fdcb419dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.205+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb419dc20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fdca8031460 tx=0x7fdca80045e0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.205+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 shutdown_connections 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.205+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdc9c07ac40 0x7fdc9c07d100 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.206+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcb4103440 0x7fdcb419dc20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.206+0000 7fdcb956f640 1 --2- 172.21.15.46:0/3464426308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcb4102b20 0x7fdcb419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.206+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 >> 172.21.15.46:0/3464426308 conn(0x7fdcb40fc520 msgr2=0x7fdcb4103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.206+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 shutdown_connections 2024-09-10T04:34:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.206+0000 7fdcb956f640 1 -- 172.21.15.46:0/3464426308 wait complete. 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 -- 172.21.15.46:0/3209489107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08681033d0 msgr2=0x7f086810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 --2- 172.21.15.46:0/3209489107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08681033d0 0x7f086810b8c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f085c009880 tx=0x7f085c02a530 comp rx=0 tx=0).stop 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 -- 172.21.15.46:0/3209489107 shutdown_connections 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 --2- 172.21.15.46:0/3209489107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08681033d0 0x7f086810b8c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 --2- 172.21.15.46:0/3209489107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0868102ab0 0x7f0868102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 -- 172.21.15.46:0/3209489107 >> 172.21.15.46:0/3209489107 conn(0x7f08680fc550 msgr2=0x7f08680fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 -- 172.21.15.46:0/3209489107 shutdown_connections 2024-09-10T04:34:43.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 -- 172.21.15.46:0/3209489107 wait complete. 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.290+0000 7f087010d640 1 Processor -- start 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f087010d640 1 -- start start 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f087010d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0868102ab0 0x7f086819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f087010d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08681033d0 0x7f086819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f087010d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f086810ede0 con 0x7f0868102ab0 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f087010d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f086810ec60 con 0x7f08681033d0 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086dea9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0868102ab0 0x7f086819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:43.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086dea9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0868102ab0 0x7f086819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46134/0 (socket says 172.21.15.46:46134) 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086dea9640 1 -- 172.21.15.46:0/1522395411 learned_addr learned my addr 172.21.15.46:0/1522395411 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086d6a8640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08681033d0 0x7f086819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086dea9640 1 -- 172.21.15.46:0/1522395411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08681033d0 msgr2=0x7f086819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086dea9640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08681033d0 0x7f086819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086dea9640 1 -- 172.21.15.46:0/1522395411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08681999e0 con 0x7f0868102ab0 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086d6a8640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08681033d0 0x7f086819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.291+0000 7f086dea9640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0868102ab0 0x7f086819f820 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f085800ea00 tx=0x7f085800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.292+0000 7f0866ffd640 1 -- 172.21.15.46:0/1522395411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f085800cb80 con 0x7f0868102ab0 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.292+0000 7f0866ffd640 1 -- 172.21.15.46:0/1522395411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f085800cd20 con 0x7f0868102ab0 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.292+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0868199cd0 con 0x7f0868102ab0 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.292+0000 7f0866ffd640 1 -- 172.21.15.46:0/1522395411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0858011470 con 0x7f0868102ab0 2024-09-10T04:34:43.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.292+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f086819a260 con 0x7f0868102ab0 2024-09-10T04:34:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.293+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f086806a580 con 0x7f0868102ab0 2024-09-10T04:34:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.293+0000 7f0866ffd640 1 -- 172.21.15.46:0/1522395411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0858005020 con 0x7f0868102ab0 2024-09-10T04:34:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.294+0000 7f0866ffd640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f085007ac50 0x7f085007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:34:43.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.294+0000 7f086d6a8640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f085007ac50 0x7f085007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:34:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.294+0000 7f0866ffd640 1 -- 172.21.15.46:0/1522395411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f085809e0b0 con 0x7f0868102ab0 2024-09-10T04:34:43.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.295+0000 7f086d6a8640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f085007ac50 0x7f085007d110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f085c031460 tx=0x7f085c0041a0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:34:43.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.298+0000 7f0866ffd640 1 -- 172.21.15.46:0/1522395411 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0858016030 con 0x7f0868102ab0 2024-09-10T04:34:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.563+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f086819af10 con 0x7f0868102ab0 2024-09-10T04:34:43.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.563+0000 7f0866ffd640 1 -- 172.21.15.46:0/1522395411 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f0858069810 con 0x7f0868102ab0 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:34:43.566 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:34:43.567 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:34:43.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f085007ac50 msgr2=0x7f085007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:43.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f085007ac50 0x7f085007d110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f085c031460 tx=0x7f085c0041a0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0868102ab0 msgr2=0x7f086819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:34:43.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0868102ab0 0x7f086819f820 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f085800ea00 tx=0x7f085800eec0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 shutdown_connections 2024-09-10T04:34:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f085007ac50 0x7f085007d110 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08681033d0 0x7f086819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 --2- 172.21.15.46:0/1522395411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0868102ab0 0x7f086819f820 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:34:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 >> 172.21.15.46:0/1522395411 conn(0x7f08680fc550 msgr2=0x7f0868100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:34:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 shutdown_connections 2024-09-10T04:34:43.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:34:43.566+0000 7f087010d640 1 -- 172.21.15.46:0/1522395411 wait complete. 2024-09-10T04:34:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:43 smithi046 ceph-mon[109506]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:43 smithi046 ceph-mon[109506]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:43 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3067943378' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:34:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:43 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1522395411' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:34:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:43 smithi070 ceph-mon[79774]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:43 smithi070 ceph-mon[79774]: from='client.35438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:43 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3067943378' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:34:44.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:43 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1522395411' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:34:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:44 smithi070 ceph-mon[79774]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:44 smithi070 ceph-mon[79774]: pgmap v873: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:44 smithi046 ceph-mon[109506]: from='client.35450 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:34:45.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:44 smithi046 ceph-mon[109506]: pgmap v873: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:46 smithi070 ceph-mon[79774]: pgmap v874: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:46 smithi046 ceph-mon[109506]: pgmap v874: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:48 smithi070 ceph-mon[79774]: pgmap v875: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:48 smithi046 ceph-mon[109506]: pgmap v875: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:50.887 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:50 smithi070 ceph-mon[79774]: pgmap v876: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:51.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:50 smithi046 ceph-mon[109506]: pgmap v876: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:52 smithi070 ceph-mon[79774]: pgmap v877: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:53.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:52 smithi046 ceph-mon[109506]: pgmap v877: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:34:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:54 smithi070 ceph-mon[79774]: pgmap v878: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:54 smithi046 ceph-mon[109506]: pgmap v878: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:56 smithi070 ceph-mon[79774]: pgmap v879: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:56 smithi046 ceph-mon[109506]: pgmap v879: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:34:58 smithi070 ceph-mon[79774]: pgmap v880: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:34:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:34:58 smithi046 ceph-mon[109506]: pgmap v880: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:00 smithi070 ceph-mon[79774]: pgmap v881: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:01.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:00 smithi046 ceph-mon[109506]: pgmap v881: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:02 smithi070 ceph-mon[79774]: pgmap v882: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:03.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:02 smithi046 ceph-mon[109506]: pgmap v882: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:05.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:04 smithi070 ceph-mon[79774]: pgmap v883: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:05.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:04 smithi046 ceph-mon[109506]: pgmap v883: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:06.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:06 smithi046 ceph-mon[109506]: pgmap v884: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:06 smithi070 ceph-mon[79774]: pgmap v884: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:08 smithi070 ceph-mon[79774]: pgmap v885: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:08 smithi046 ceph-mon[109506]: pgmap v885: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:10 smithi070 ceph-mon[79774]: pgmap v886: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:10 smithi046 ceph-mon[109506]: pgmap v886: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:12 smithi070 ceph-mon[79774]: pgmap v887: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:12 smithi046 ceph-mon[109506]: pgmap v887: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 -- 172.21.15.46:0/1495066949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8107900 msgr2=0x7f78f8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/1495066949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8107900 0x7f78f8107ce0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009880 tx=0x7f78e802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 -- 172.21.15.46:0/1495066949 shutdown_connections 2024-09-10T04:35:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/1495066949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78f8101900 0x7f78f8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/1495066949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8107900 0x7f78f8107ce0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 -- 172.21.15.46:0/1495066949 >> 172.21.15.46:0/1495066949 conn(0x7f78f80fd600 msgr2=0x7f78f80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 -- 172.21.15.46:0/1495066949 shutdown_connections 2024-09-10T04:35:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 -- 172.21.15.46:0/1495066949 wait complete. 2024-09-10T04:35:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.665+0000 7f78fe3b8640 1 Processor -- start 2024-09-10T04:35:13.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78fe3b8640 1 -- start start 2024-09-10T04:35:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78fe3b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8101900 0x7f78f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78fe3b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78f8107900 0x7f78f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:13.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78fe3b8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f78f810ee30 con 0x7f78f8101900 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78fe3b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f78f810ecb0 con 0x7f78f8107900 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78f7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8101900 0x7f78f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78f7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8101900 0x7f78f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38834/0 (socket says 172.21.15.46:38834) 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78f7fff640 1 -- 172.21.15.46:0/314041769 learned_addr learned my addr 172.21.15.46:0/314041769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.666+0000 7f78f77fe640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78f8107900 0x7f78f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78f7fff640 1 -- 172.21.15.46:0/314041769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78f8107900 msgr2=0x7f78f8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78f7fff640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78f8107900 0x7f78f8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78f7fff640 1 -- 172.21.15.46:0/314041769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f8074f70 con 0x7f78f8101900 2024-09-10T04:35:13.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78f7fff640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8101900 0x7f78f80744f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f78e8031460 tx=0x7f78e80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78f57fa640 1 -- 172.21.15.46:0/314041769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f78e803f070 con 0x7f78f8101900 2024-09-10T04:35:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78f57fa640 1 -- 172.21.15.46:0/314041769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f78e80044d0 con 0x7f78f8101900 2024-09-10T04:35:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f8075200 con 0x7f78f8101900 2024-09-10T04:35:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78f57fa640 1 -- 172.21.15.46:0/314041769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f78e8035030 con 0x7f78f8101900 2024-09-10T04:35:13.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.667+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f81a3e40 con 0x7f78f8101900 2024-09-10T04:35:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.668+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f8103040 con 0x7f78f8101900 2024-09-10T04:35:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.669+0000 7f78f57fa640 1 -- 172.21.15.46:0/314041769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f78e8042070 con 0x7f78f8101900 2024-09-10T04:35:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.669+0000 7f78f57fa640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78e407ac00 0x7f78e407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.670+0000 7f78f57fa640 1 -- 172.21.15.46:0/314041769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f78e80be890 con 0x7f78f8101900 2024-09-10T04:35:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.670+0000 7f78f77fe640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78e407ac00 0x7f78e407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:13.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.670+0000 7f78f77fe640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78e407ac00 0x7f78e407d0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f78f8075f80 tx=0x7f78e0008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:13.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.674+0000 7f78f57fa640 1 -- 172.21.15.46:0/314041769 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f78e8004cf0 con 0x7f78f8101900 2024-09-10T04:35:13.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.878+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f78f8062630 con 0x7f78e407ac00 2024-09-10T04:35:13.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.879+0000 7f78f57fa640 1 -- 172.21.15.46:0/314041769 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f78f8062630 con 0x7f78e407ac00 2024-09-10T04:35:13.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78e407ac00 msgr2=0x7f78e407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78e407ac00 0x7f78e407d0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f78f8075f80 tx=0x7f78e0008040 comp rx=0 tx=0).stop 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8101900 msgr2=0x7f78f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8101900 0x7f78f80744f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f78e8031460 tx=0x7f78e80048f0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 shutdown_connections 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f78e407ac00 0x7f78e407d0c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78f8107900 0x7f78f8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.881+0000 7f78fe3b8640 1 --2- 172.21.15.46:0/314041769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f8101900 0x7f78f80744f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.882+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 >> 172.21.15.46:0/314041769 conn(0x7f78f80fd600 msgr2=0x7f78f80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.882+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 shutdown_connections 2024-09-10T04:35:13.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.882+0000 7f78fe3b8640 1 -- 172.21.15.46:0/314041769 wait complete. 2024-09-10T04:35:13.892 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.966+0000 7ff682b97640 1 -- 172.21.15.46:0/1676464596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 msgr2=0x7ff67c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.966+0000 7ff682b97640 1 --2- 172.21.15.46:0/1676464596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c100c50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff670009880 tx=0x7ff67002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 -- 172.21.15.46:0/1676464596 shutdown_connections 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 --2- 172.21.15.46:0/1676464596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c100c50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 --2- 172.21.15.46:0/1676464596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff67c1067f0 0x7ff67c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 -- 172.21.15.46:0/1676464596 >> 172.21.15.46:0/1676464596 conn(0x7ff67c0fc550 msgr2=0x7ff67c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 -- 172.21.15.46:0/1676464596 shutdown_connections 2024-09-10T04:35:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 -- 172.21.15.46:0/1676464596 wait complete. 2024-09-10T04:35:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 Processor -- start 2024-09-10T04:35:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 -- start start 2024-09-10T04:35:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff67c1067f0 0x7ff67c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c10dd20 con 0x7ff67c1007f0 2024-09-10T04:35:13.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.967+0000 7ff682b97640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff67c10dba0 con 0x7ff67c1067f0 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff680933640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff680933640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c19d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38856/0 (socket says 172.21.15.46:38856) 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff680933640 1 -- 172.21.15.46:0/1498805224 learned_addr learned my addr 172.21.15.46:0/1498805224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff67bfff640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff67c1067f0 0x7ff67c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff680933640 1 -- 172.21.15.46:0/1498805224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff67c1067f0 msgr2=0x7ff67c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff680933640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff67c1067f0 0x7ff67c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff680933640 1 -- 172.21.15.46:0/1498805224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff67c197850 con 0x7ff67c1007f0 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff67bfff640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff67c1067f0 0x7ff67c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff680933640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c19d6d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b420 tx=0x7ff66c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff679ffb640 1 -- 172.21.15.46:0/1498805224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff66c011630 con 0x7ff67c1007f0 2024-09-10T04:35:13.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff679ffb640 1 -- 172.21.15.46:0/1498805224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff66c016070 con 0x7ff67c1007f0 2024-09-10T04:35:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff67c197b40 con 0x7ff67c1007f0 2024-09-10T04:35:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.968+0000 7ff679ffb640 1 -- 172.21.15.46:0/1498805224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff66c010d40 con 0x7ff67c1007f0 2024-09-10T04:35:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.969+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67c1980a0 con 0x7ff67c1007f0 2024-09-10T04:35:13.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.970+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff67c101f30 con 0x7ff67c1007f0 2024-09-10T04:35:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.970+0000 7ff679ffb640 1 -- 172.21.15.46:0/1498805224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff66c01b460 con 0x7ff67c1007f0 2024-09-10T04:35:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.971+0000 7ff679ffb640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff66407ac40 0x7ff66407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.971+0000 7ff679ffb640 1 -- 172.21.15.46:0/1498805224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff66c09d370 con 0x7ff67c1007f0 2024-09-10T04:35:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.971+0000 7ff67bfff640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff66407ac40 0x7ff66407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:13.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.971+0000 7ff67bfff640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff66407ac40 0x7ff66407d100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff67c198ba0 tx=0x7ff670031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:13.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:13.975+0000 7ff679ffb640 1 -- 172.21.15.46:0/1498805224 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff66c021250 con 0x7ff67c1007f0 2024-09-10T04:35:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.180+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff67c1983b0 con 0x7ff66407ac40 2024-09-10T04:35:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.182+0000 7ff679ffb640 1 -- 172.21.15.46:0/1498805224 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ff67c1983b0 con 0x7ff66407ac40 2024-09-10T04:35:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff66407ac40 msgr2=0x7ff66407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff66407ac40 0x7ff66407d100 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff67c198ba0 tx=0x7ff670031400 comp rx=0 tx=0).stop 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 msgr2=0x7ff67c19d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c19d6d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00b420 tx=0x7ff66c00b8e0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 shutdown_connections 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff66407ac40 0x7ff66407d100 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff67c1067f0 0x7ff67c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 --2- 172.21.15.46:0/1498805224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff67c1007f0 0x7ff67c19d6d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 >> 172.21.15.46:0/1498805224 conn(0x7ff67c0fc550 msgr2=0x7ff67c069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 shutdown_connections 2024-09-10T04:35:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.185+0000 7ff682b97640 1 -- 172.21.15.46:0/1498805224 wait complete. 2024-09-10T04:35:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.270+0000 7f7aa97b1640 1 -- 172.21.15.46:0/1742544616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4107900 msgr2=0x7f7aa4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.270+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/1742544616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4107900 0x7f7aa4107ce0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009880 tx=0x7f7a9402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 -- 172.21.15.46:0/1742544616 shutdown_connections 2024-09-10T04:35:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/1742544616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4101900 0x7f7aa4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/1742544616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4107900 0x7f7aa4107ce0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 -- 172.21.15.46:0/1742544616 >> 172.21.15.46:0/1742544616 conn(0x7f7aa40fd600 msgr2=0x7f7aa40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 -- 172.21.15.46:0/1742544616 shutdown_connections 2024-09-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 -- 172.21.15.46:0/1742544616 wait complete. 2024-09-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 Processor -- start 2024-09-10T04:35:14.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.271+0000 7f7aa97b1640 1 -- start start 2024-09-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa97b1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4101900 0x7f7aa419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa97b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4107900 0x7f7aa419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa97b1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa410edb0 con 0x7f7aa4101900 2024-09-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa97b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa410ec30 con 0x7f7aa4107900 2024-09-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4101900 0x7f7aa419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4101900 0x7f7aa419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38880/0 (socket says 172.21.15.46:38880) 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa2ffd640 1 -- 172.21.15.46:0/3699391051 learned_addr learned my addr 172.21.15.46:0/3699391051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa27fc640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4107900 0x7f7aa419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa2ffd640 1 -- 172.21.15.46:0/3699391051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4107900 msgr2=0x7f7aa419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa2ffd640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4107900 0x7f7aa419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa2ffd640 1 -- 172.21.15.46:0/3699391051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa41977c0 con 0x7f7aa4101900 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.272+0000 7f7aa2ffd640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4101900 0x7f7aa419d640 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7a94031460 tx=0x7f7a940045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.273+0000 7f7a8bfff640 1 -- 172.21.15.46:0/3699391051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a9403f070 con 0x7f7aa4101900 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.273+0000 7f7a8bfff640 1 -- 172.21.15.46:0/3699391051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7a940035f0 con 0x7f7aa4101900 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.273+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa4197a50 con 0x7f7aa4101900 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.273+0000 7f7a8bfff640 1 -- 172.21.15.46:0/3699391051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7a94035030 con 0x7f7aa4101900 2024-09-10T04:35:14.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.273+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa4197f80 con 0x7f7aa4101900 2024-09-10T04:35:14.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.274+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa4103040 con 0x7f7aa4101900 2024-09-10T04:35:14.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.274+0000 7f7a8bfff640 1 -- 172.21.15.46:0/3699391051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7a94042070 con 0x7f7aa4101900 2024-09-10T04:35:14.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.275+0000 7f7a8bfff640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a8c07abf0 0x7f7a8c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.275+0000 7f7a8bfff640 1 -- 172.21.15.46:0/3699391051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7a940be6c0 con 0x7f7aa4101900 2024-09-10T04:35:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.275+0000 7f7aa27fc640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a8c07abf0 0x7f7a8c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.276+0000 7f7aa27fc640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a8c07abf0 0x7f7a8c07d0b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005fd0 tx=0x7f7a98005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:14.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.280+0000 7f7a8bfff640 1 -- 172.21.15.46:0/3699391051 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7a94089e00 con 0x7f7aa4101900 2024-09-10T04:35:14.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.473+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7aa41982f0 con 0x7f7a8c07abf0 2024-09-10T04:35:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.483+0000 7f7a8bfff640 1 -- 172.21.15.46:0/3699391051 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f7aa41982f0 con 0x7f7a8c07abf0 2024-09-10T04:35:14.485 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:35:14.485 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (26m) 5m ago 38m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (29m) 5m ago 38m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (29m) 5m ago 37m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (25m) 5m ago 38m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (25m) 5m ago 37m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (26m) 5m ago 37m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (21m) 5m ago 32m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (21m) 5m ago 32m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (21m) 5m ago 32m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (5m) 5m ago 32m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (28m) 5m ago 39m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (27m) 5m ago 37m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (25m) 5m ago 39m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (25m) 5m ago 37m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:35:14.486 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (27m) 5m ago 38m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (27m) 5m ago 37m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (24m) 5m ago 36m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (24m) 5m ago 35m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (23m) 5m ago 35m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (23m) 5m ago 34m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (22m) 5m ago 34m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (21m) 5m ago 33m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:35:14.487 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (27m) 5m ago 37m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:35:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a8c07abf0 msgr2=0x7f7a8c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a8c07abf0 0x7f7a8c07d0b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005fd0 tx=0x7f7a98005850 comp rx=0 tx=0).stop 2024-09-10T04:35:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4101900 msgr2=0x7f7aa419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4101900 0x7f7aa419d640 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7a94031460 tx=0x7f7a940045e0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 shutdown_connections 2024-09-10T04:35:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7a8c07abf0 0x7f7a8c07d0b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7aa4107900 0x7f7aa419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 --2- 172.21.15.46:0/3699391051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7aa4101900 0x7f7aa419d640 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 >> 172.21.15.46:0/3699391051 conn(0x7f7aa40fd600 msgr2=0x7f7aa40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 shutdown_connections 2024-09-10T04:35:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.486+0000 7f7aa97b1640 1 -- 172.21.15.46:0/3699391051 wait complete. 2024-09-10T04:35:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 -- 172.21.15.46:0/224863667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c103490 msgr2=0x7fca8c10b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 --2- 172.21.15.46:0/224863667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c103490 0x7fca8c10b980 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fca74009880 tx=0x7fca7402a530 comp rx=0 tx=0).stop 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 -- 172.21.15.46:0/224863667 shutdown_connections 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 --2- 172.21.15.46:0/224863667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c103490 0x7fca8c10b980 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 --2- 172.21.15.46:0/224863667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c102b70 0x7fca8c102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 -- 172.21.15.46:0/224863667 >> 172.21.15.46:0/224863667 conn(0x7fca8c0fc550 msgr2=0x7fca8c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 -- 172.21.15.46:0/224863667 shutdown_connections 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.569+0000 7fca908f4640 1 -- 172.21.15.46:0/224863667 wait complete. 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca908f4640 1 Processor -- start 2024-09-10T04:35:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca908f4640 1 -- start start 2024-09-10T04:35:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca908f4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c102b70 0x7fca8c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca908f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c103490 0x7fca8c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca908f4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10ede0 con 0x7fca8c102b70 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca908f4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fca8c10ec60 con 0x7fca8c103490 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca8a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c102b70 0x7fca8c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca8a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c102b70 0x7fca8c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38898/0 (socket says 172.21.15.46:38898) 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca8a59c640 1 -- 172.21.15.46:0/1665298514 learned_addr learned my addr 172.21.15.46:0/1665298514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca8a59c640 1 -- 172.21.15.46:0/1665298514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c103490 msgr2=0x7fca8c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca89d9b640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c103490 0x7fca8c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca8a59c640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c103490 0x7fca8c19fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.570+0000 7fca8a59c640 1 -- 172.21.15.46:0/1665298514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8c1999f0 con 0x7fca8c102b70 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.571+0000 7fca8a59c640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c102b70 0x7fca8c19f830 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d6f0 tx=0x7fca7c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.571+0000 7fca837fe640 1 -- 172.21.15.46:0/1665298514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fca7c017070 con 0x7fca8c102b70 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.571+0000 7fca837fe640 1 -- 172.21.15.46:0/1665298514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca7c00de00 con 0x7fca8c102b70 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.571+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8c199ce0 con 0x7fca8c102b70 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.571+0000 7fca837fe640 1 -- 172.21.15.46:0/1665298514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fca7c004bb0 con 0x7fca8c102b70 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.571+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca8c19a270 con 0x7fca8c102b70 2024-09-10T04:35:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.572+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca8c06a4c0 con 0x7fca8c102b70 2024-09-10T04:35:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.573+0000 7fca837fe640 1 -- 172.21.15.46:0/1665298514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fca7c009a30 con 0x7fca8c102b70 2024-09-10T04:35:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.573+0000 7fca837fe640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fca7807ac40 0x7fca7807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.573+0000 7fca837fe640 1 -- 172.21.15.46:0/1665298514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fca7c09d7f0 con 0x7fca8c102b70 2024-09-10T04:35:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.573+0000 7fca89d9b640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fca7807ac40 0x7fca7807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.574+0000 7fca89d9b640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fca7807ac40 0x7fca7807d100 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fca74004f90 tx=0x7fca74009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.578+0000 7fca837fe640 1 -- 172.21.15.46:0/1665298514 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fca7c068fb0 con 0x7fca8c102b70 2024-09-10T04:35:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:14 smithi046 ceph-mon[109506]: pgmap v888: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:14.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.841+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fca8c10b3a0 con 0x7fca8c102b70 2024-09-10T04:35:14.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.842+0000 7fca837fe640 1 -- 172.21.15.46:0/1665298514 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fca7c06de60 con 0x7fca8c102b70 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:35:14.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:35:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:35:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:35:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:35:14.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:35:14.845 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:35:14.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fca7807ac40 msgr2=0x7fca7807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fca7807ac40 0x7fca7807d100 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fca74004f90 tx=0x7fca74009560 comp rx=0 tx=0).stop 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c102b70 msgr2=0x7fca8c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c102b70 0x7fca8c19f830 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d6f0 tx=0x7fca7c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 shutdown_connections 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fca7807ac40 0x7fca7807d100 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c103490 0x7fca8c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.844+0000 7fca908f4640 1 --2- 172.21.15.46:0/1665298514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8c102b70 0x7fca8c19f830 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.845+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 >> 172.21.15.46:0/1665298514 conn(0x7fca8c0fc550 msgr2=0x7fca8c100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.845+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 shutdown_connections 2024-09-10T04:35:14.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.845+0000 7fca908f4640 1 -- 172.21.15.46:0/1665298514 wait complete. 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 -- 172.21.15.46:0/3746425492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f10102ac0 msgr2=0x7f4f10102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 --2- 172.21.15.46:0/3746425492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f10102ac0 0x7f4f10102ea0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009c20 tx=0x7f4f0c02aa00 comp rx=0 tx=0).stop 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 -- 172.21.15.46:0/3746425492 shutdown_connections 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 --2- 172.21.15.46:0/3746425492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f101033e0 0x7f4f1010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 --2- 172.21.15.46:0/3746425492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f10102ac0 0x7f4f10102ea0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 -- 172.21.15.46:0/3746425492 >> 172.21.15.46:0/3746425492 conn(0x7f4f100fc520 msgr2=0x7f4f100fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 -- 172.21.15.46:0/3746425492 shutdown_connections 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.928+0000 7f4f1855c640 1 -- 172.21.15.46:0/3746425492 wait complete. 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.929+0000 7f4f1855c640 1 Processor -- start 2024-09-10T04:35:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.929+0000 7f4f1855c640 1 -- start start 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.929+0000 7f4f1855c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f10102ac0 0x7f4f1019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.929+0000 7f4f1855c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f101033e0 0x7f4f1019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.929+0000 7f4f1855c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1010edd0 con 0x7f4f101033e0 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.929+0000 7f4f1855c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1010ec50 con 0x7f4f10102ac0 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f15af7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f101033e0 0x7f4f1019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f15af7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f101033e0 0x7f4f1019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38922/0 (socket says 172.21.15.46:38922) 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f15af7640 1 -- 172.21.15.46:0/4165956340 learned_addr learned my addr 172.21.15.46:0/4165956340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f162f8640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f10102ac0 0x7f4f1019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f15af7640 1 -- 172.21.15.46:0/4165956340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f10102ac0 msgr2=0x7f4f1019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f15af7640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f10102ac0 0x7f4f1019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f15af7640 1 -- 172.21.15.46:0/4165956340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f101977c0 con 0x7f4f101033e0 2024-09-10T04:35:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f15af7640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f101033e0 0x7f4f1019db80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000c8c0 tx=0x7f4f0000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f077fe640 1 -- 172.21.15.46:0/4165956340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f00004020 con 0x7f4f101033e0 2024-09-10T04:35:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f077fe640 1 -- 172.21.15.46:0/4165956340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f000041c0 con 0x7f4f101033e0 2024-09-10T04:35:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f10197ab0 con 0x7f4f101033e0 2024-09-10T04:35:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f077fe640 1 -- 172.21.15.46:0/4165956340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f000047a0 con 0x7f4f101033e0 2024-09-10T04:35:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.930+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f10198040 con 0x7f4f101033e0 2024-09-10T04:35:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.931+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1006aa90 con 0x7f4f101033e0 2024-09-10T04:35:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.932+0000 7f4f077fe640 1 -- 172.21.15.46:0/4165956340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4f000079b0 con 0x7f4f101033e0 2024-09-10T04:35:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.933+0000 7f4f077fe640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4efc07ac50 0x7f4efc07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.933+0000 7f4f077fe640 1 -- 172.21.15.46:0/4165956340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4f0009d9a0 con 0x7f4f101033e0 2024-09-10T04:35:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.933+0000 7f4f162f8640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4efc07ac50 0x7f4efc07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.933+0000 7f4f162f8640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4efc07ac50 0x7f4efc07d110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009430 tx=0x7f4f0c03a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:14.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:14.937+0000 7f4f077fe640 1 -- 172.21.15.46:0/4165956340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4f00014030 con 0x7f4f101033e0 2024-09-10T04:35:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:14 smithi070 ceph-mon[79774]: pgmap v888: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:15.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.162+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f4f10105f90 con 0x7f4f101033e0 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.163+0000 7f4f077fe640 1 -- 172.21.15.46:0/4165956340 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f4f00069160 con 0x7f4f101033e0 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:15.165 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:35:15.166 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:35:15.167 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:15.168 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4efc07ac50 msgr2=0x7f4efc07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4efc07ac50 0x7f4efc07d110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009430 tx=0x7f4f0c03a040 comp rx=0 tx=0).stop 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f101033e0 msgr2=0x7f4f1019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f101033e0 0x7f4f1019db80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000c8c0 tx=0x7f4f0000cd80 comp rx=0 tx=0).stop 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 shutdown_connections 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4efc07ac50 0x7f4efc07d110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f101033e0 0x7f4f1019db80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 --2- 172.21.15.46:0/4165956340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f10102ac0 0x7f4f1019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.165+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 >> 172.21.15.46:0/4165956340 conn(0x7f4f100fc520 msgr2=0x7f4f10108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:15.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.166+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 shutdown_connections 2024-09-10T04:35:15.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.166+0000 7f4f1855c640 1 -- 172.21.15.46:0/4165956340 wait complete. 2024-09-10T04:35:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/2970545673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 msgr2=0x7fc8e8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/2970545673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e8100c50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009880 tx=0x7fc8d002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/2970545673 shutdown_connections 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/2970545673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e8100c50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/2970545673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e81067f0 0x7fc8e8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/2970545673 >> 172.21.15.46:0/2970545673 conn(0x7fc8e80fc550 msgr2=0x7fc8e80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/2970545673 shutdown_connections 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/2970545673 wait complete. 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.251+0000 7fc8ec8a5640 1 Processor -- start 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8ec8a5640 1 -- start start 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8ec8a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8ec8a5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e81067f0 0x7fc8e819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8ec8a5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810dcc0 con 0x7fc8e81007f0 2024-09-10T04:35:15.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8ec8a5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810db40 con 0x7fc8e81067f0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8e659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8e659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38926/0 (socket says 172.21.15.46:38926) 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8e659c640 1 -- 172.21.15.46:0/3932077292 learned_addr learned my addr 172.21.15.46:0/3932077292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8dffff640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e81067f0 0x7fc8e819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8e659c640 1 -- 172.21.15.46:0/3932077292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e81067f0 msgr2=0x7fc8e819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8e659c640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e81067f0 0x7fc8e819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.252+0000 7fc8e659c640 1 -- 172.21.15.46:0/3932077292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e81977e0 con 0x7fc8e81007f0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.253+0000 7fc8e659c640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e819d660 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800d6f0 tx=0x7fc8d800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.253+0000 7fc8df7fe640 1 -- 172.21.15.46:0/3932077292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc8d8017070 con 0x7fc8e81007f0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.253+0000 7fc8df7fe640 1 -- 172.21.15.46:0/3932077292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc8d800de00 con 0x7fc8e81007f0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.253+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e8197ad0 con 0x7fc8e81007f0 2024-09-10T04:35:15.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.253+0000 7fc8df7fe640 1 -- 172.21.15.46:0/3932077292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc8d8004bb0 con 0x7fc8e81007f0 2024-09-10T04:35:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.253+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e8198030 con 0x7fc8e81007f0 2024-09-10T04:35:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.254+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e8101f30 con 0x7fc8e81007f0 2024-09-10T04:35:15.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.255+0000 7fc8df7fe640 1 -- 172.21.15.46:0/3932077292 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc8d8009a30 con 0x7fc8e81007f0 2024-09-10T04:35:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.255+0000 7fc8df7fe640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc8d407ac40 0x7fc8d407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.255+0000 7fc8df7fe640 1 -- 172.21.15.46:0/3932077292 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc8d809d740 con 0x7fc8e81007f0 2024-09-10T04:35:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.255+0000 7fc8dffff640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc8d407ac40 0x7fc8d407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:15.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.256+0000 7fc8dffff640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc8d407ac40 0x7fc8d407d100 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8198b30 tx=0x7fc8d0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:15.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.260+0000 7fc8df7fe640 1 -- 172.21.15.46:0/3932077292 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc8d8068f00 con 0x7fc8e81007f0 2024-09-10T04:35:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.466+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc8e81982e0 con 0x7fc8d407ac40 2024-09-10T04:35:15.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.467+0000 7fc8df7fe640 1 -- 172.21.15.46:0/3932077292 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc8e81982e0 con 0x7fc8d407ac40 2024-09-10T04:35:15.469 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:35:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:35:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:35:15.469 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:35:15.470 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:35:15.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.469+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc8d407ac40 msgr2=0x7fc8d407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.469+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc8d407ac40 0x7fc8d407d100 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8198b30 tx=0x7fc8d0031400 comp rx=0 tx=0).stop 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.469+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 msgr2=0x7fc8e819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.469+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e819d660 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800d6f0 tx=0x7fc8d800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.470+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 shutdown_connections 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.470+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc8d407ac40 0x7fc8d407d100 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.470+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e81067f0 0x7fc8e819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.470+0000 7fc8ec8a5640 1 --2- 172.21.15.46:0/3932077292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8e81007f0 0x7fc8e819d660 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.470+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 >> 172.21.15.46:0/3932077292 conn(0x7fc8e80fc550 msgr2=0x7fc8e8069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.470+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 shutdown_connections 2024-09-10T04:35:15.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.470+0000 7fc8ec8a5640 1 -- 172.21.15.46:0/3932077292 wait complete. 2024-09-10T04:35:15.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 -- 172.21.15.46:0/797454697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c106800 msgr2=0x7f293c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 --2- 172.21.15.46:0/797454697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c106800 0x7f293c106be0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f292c009880 tx=0x7f292c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 -- 172.21.15.46:0/797454697 shutdown_connections 2024-09-10T04:35:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 --2- 172.21.15.46:0/797454697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f293c100800 0x7f293c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 --2- 172.21.15.46:0/797454697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c106800 0x7f293c106be0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 -- 172.21.15.46:0/797454697 >> 172.21.15.46:0/797454697 conn(0x7f293c0fc520 msgr2=0x7f293c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 -- 172.21.15.46:0/797454697 shutdown_connections 2024-09-10T04:35:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.553+0000 7f2940c6d640 1 -- 172.21.15.46:0/797454697 wait complete. 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.554+0000 7f2940c6d640 1 Processor -- start 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.554+0000 7f2940c6d640 1 -- start start 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f2940c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c100800 0x7f293c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f2940c6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f293c106800 0x7f293c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f2940c6d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f293c10dcd0 con 0x7f293c100800 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f2940c6d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f293c10db50 con 0x7f293c106800 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f293a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c100800 0x7f293c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f293a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c100800 0x7f293c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38956/0 (socket says 172.21.15.46:38956) 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f293a59c640 1 -- 172.21.15.46:0/1088822055 learned_addr learned my addr 172.21.15.46:0/1088822055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:15.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f2939d9b640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f293c106800 0x7f293c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f293a59c640 1 -- 172.21.15.46:0/1088822055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f293c106800 msgr2=0x7f293c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f293a59c640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f293c106800 0x7f293c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f293a59c640 1 -- 172.21.15.46:0/1088822055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f293c1977c0 con 0x7f293c100800 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f293a59c640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c100800 0x7f293c19d640 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f292c031460 tx=0x7f292c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f292b7fe640 1 -- 172.21.15.46:0/1088822055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f292c03f070 con 0x7f293c100800 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f293c197a50 con 0x7f293c100800 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f292b7fe640 1 -- 172.21.15.46:0/1088822055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f292c0044d0 con 0x7f293c100800 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.555+0000 7f292b7fe640 1 -- 172.21.15.46:0/1088822055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f292c035030 con 0x7f293c100800 2024-09-10T04:35:15.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.556+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293c197f80 con 0x7f293c100800 2024-09-10T04:35:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.556+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293c101f40 con 0x7f293c100800 2024-09-10T04:35:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.557+0000 7f292b7fe640 1 -- 172.21.15.46:0/1088822055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f292c042070 con 0x7f293c100800 2024-09-10T04:35:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.558+0000 7f292b7fe640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f292407ac00 0x7f292407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.558+0000 7f292b7fe640 1 -- 172.21.15.46:0/1088822055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f292c0be7e0 con 0x7f293c100800 2024-09-10T04:35:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.558+0000 7f2939d9b640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f292407ac00 0x7f292407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.558+0000 7f2939d9b640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f292407ac00 0x7f292407d0c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f2930005e20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.562+0000 7f292b7fe640 1 -- 172.21.15.46:0/1088822055 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f292c004cf0 con 0x7f293c100800 2024-09-10T04:35:15.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.820+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f293c109d80 con 0x7f293c100800 2024-09-10T04:35:15.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:15 smithi046 ceph-mon[109506]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:15.822 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:15 smithi046 ceph-mon[109506]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:15.823 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:15 smithi046 ceph-mon[109506]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:15.823 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:15 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1665298514' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:35:15.823 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:15 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4165956340' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:35:15.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.820+0000 7f292b7fe640 1 -- 172.21.15.46:0/1088822055 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f292c089f20 con 0x7f293c100800 2024-09-10T04:35:15.823 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:35:15.823 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:35:15.823 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:35:15.823 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:35:15.823 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:35:15.824 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:35:15.825 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:35:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:35:15.825 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:35:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:35:15.825 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:35:15.825 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:35:15.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f292407ac00 msgr2=0x7f292407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f292407ac00 0x7f292407d0c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2930005fd0 tx=0x7f2930005e20 comp rx=0 tx=0).stop 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c100800 msgr2=0x7f293c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c100800 0x7f293c19d640 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f292c031460 tx=0x7f292c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 shutdown_connections 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f292407ac00 0x7f292407d0c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f293c106800 0x7f293c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 --2- 172.21.15.46:0/1088822055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f293c100800 0x7f293c19d640 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 >> 172.21.15.46:0/1088822055 conn(0x7f293c0fc520 msgr2=0x7f293c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 shutdown_connections 2024-09-10T04:35:15.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:15.823+0000 7f2940c6d640 1 -- 172.21.15.46:0/1088822055 wait complete. 2024-09-10T04:35:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:15 smithi070 ceph-mon[79774]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:15 smithi070 ceph-mon[79774]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:15 smithi070 ceph-mon[79774]: from='client.35466 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:15 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1665298514' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:35:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:15 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4165956340' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:35:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:16 smithi070 ceph-mon[79774]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:16 smithi070 ceph-mon[79774]: pgmap v889: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:16 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1088822055' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:35:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:16 smithi046 ceph-mon[109506]: from='client.35478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:16 smithi046 ceph-mon[109506]: pgmap v889: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:17.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:16 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1088822055' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:35:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:18 smithi070 ceph-mon[79774]: pgmap v890: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:18 smithi046 ceph-mon[109506]: pgmap v890: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:20 smithi070 ceph-mon[79774]: pgmap v891: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:20 smithi046 ceph-mon[109506]: pgmap v891: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:22 smithi070 ceph-mon[79774]: pgmap v892: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:22 smithi046 ceph-mon[109506]: pgmap v892: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:24 smithi070 ceph-mon[79774]: pgmap v893: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:24 smithi046 ceph-mon[109506]: pgmap v893: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:26 smithi070 ceph-mon[79774]: pgmap v894: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:26 smithi046 ceph-mon[109506]: pgmap v894: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:28 smithi070 ceph-mon[79774]: pgmap v895: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:28 smithi046 ceph-mon[109506]: pgmap v895: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:30 smithi070 ceph-mon[79774]: pgmap v896: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:31.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:30 smithi046 ceph-mon[109506]: pgmap v896: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:33.027 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:32 smithi070 ceph-mon[79774]: pgmap v897: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:32 smithi046 ceph-mon[109506]: pgmap v897: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:34 smithi070 ceph-mon[79774]: pgmap v898: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:34 smithi046 ceph-mon[109506]: pgmap v898: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:37.026 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:36 smithi046 ceph-mon[109506]: pgmap v899: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:36 smithi070 ceph-mon[79774]: pgmap v899: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:38 smithi070 ceph-mon[79774]: pgmap v900: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:38 smithi046 ceph-mon[109506]: pgmap v900: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:40 smithi070 ceph-mon[79774]: pgmap v901: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:40 smithi046 ceph-mon[109506]: pgmap v901: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:42.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:42 smithi070 ceph-mon[79774]: pgmap v902: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:42.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:42 smithi046 ceph-mon[109506]: pgmap v902: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:44.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:44 smithi046 ceph-mon[109506]: pgmap v903: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:44 smithi070 ceph-mon[79774]: pgmap v903: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:45.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 -- 172.21.15.46:0/2019199014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 msgr2=0x7f38f410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 --2- 172.21.15.46:0/2019199014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f410b8d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009880 tx=0x7f38e802a530 comp rx=0 tx=0).stop 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 -- 172.21.15.46:0/2019199014 shutdown_connections 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 --2- 172.21.15.46:0/2019199014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f410b8d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 --2- 172.21.15.46:0/2019199014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f4102ac0 0x7f38f4102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 -- 172.21.15.46:0/2019199014 >> 172.21.15.46:0/2019199014 conn(0x7f38f40fc520 msgr2=0x7f38f40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 -- 172.21.15.46:0/2019199014 shutdown_connections 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.922+0000 7f38f96c2640 1 -- 172.21.15.46:0/2019199014 wait complete. 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f96c2640 1 Processor -- start 2024-09-10T04:35:45.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f96c2640 1 -- start start 2024-09-10T04:35:45.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f96c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f4102ac0 0x7f38f4100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f96c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f4100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f96c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f38f410edf0 con 0x7f38f41033e0 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f96c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f38f410ec70 con 0x7f38f4102ac0 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f4100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f27fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f4100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58522/0 (socket says 172.21.15.46:58522) 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f27fc640 1 -- 172.21.15.46:0/81932446 learned_addr learned my addr 172.21.15.46:0/81932446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.923+0000 7f38f2ffd640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f4102ac0 0x7f38f4100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38f27fc640 1 -- 172.21.15.46:0/81932446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f4102ac0 msgr2=0x7f38f4100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38f27fc640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f4102ac0 0x7f38f4100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38f27fc640 1 -- 172.21.15.46:0/81932446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f41011a0 con 0x7f38f41033e0 2024-09-10T04:35:45.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38f27fc640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f4100c60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f38e8031460 tx=0x7f38e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38dbfff640 1 -- 172.21.15.46:0/81932446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f38e803f070 con 0x7f38f41033e0 2024-09-10T04:35:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38dbfff640 1 -- 172.21.15.46:0/81932446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38e80035f0 con 0x7f38f41033e0 2024-09-10T04:35:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f4101430 con 0x7f38f41033e0 2024-09-10T04:35:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38dbfff640 1 -- 172.21.15.46:0/81932446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f38e8035030 con 0x7f38f41033e0 2024-09-10T04:35:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.924+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f41a3de0 con 0x7f38f41033e0 2024-09-10T04:35:45.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.925+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f406aa90 con 0x7f38f41033e0 2024-09-10T04:35:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.926+0000 7f38dbfff640 1 -- 172.21.15.46:0/81932446 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f38e8042070 con 0x7f38f41033e0 2024-09-10T04:35:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.926+0000 7f38dbfff640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38dc07ac00 0x7f38dc07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.927+0000 7f38f2ffd640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38dc07ac00 0x7f38dc07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.927+0000 7f38dbfff640 1 -- 172.21.15.46:0/81932446 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f38e80be580 con 0x7f38f41033e0 2024-09-10T04:35:45.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.927+0000 7f38f2ffd640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38dc07ac00 0x7f38dc07d0c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009bd0 tx=0x7f38e4005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:45.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:45.931+0000 7f38dbfff640 1 -- 172.21.15.46:0/81932446 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f38e8089cc0 con 0x7f38f41033e0 2024-09-10T04:35:46.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.136+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f38f4062630 con 0x7f38dc07ac00 2024-09-10T04:35:46.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.137+0000 7f38dbfff640 1 -- 172.21.15.46:0/81932446 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f38f4062630 con 0x7f38dc07ac00 2024-09-10T04:35:46.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38dc07ac00 msgr2=0x7f38dc07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38dc07ac00 0x7f38dc07d0c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009bd0 tx=0x7f38e4005f90 comp rx=0 tx=0).stop 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 msgr2=0x7f38f4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f4100c60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f38e8031460 tx=0x7f38e80045e0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 shutdown_connections 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f38dc07ac00 0x7f38dc07d0c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f41033e0 0x7f38f4100c60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 --2- 172.21.15.46:0/81932446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38f4102ac0 0x7f38f4100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.139+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 >> 172.21.15.46:0/81932446 conn(0x7f38f40fc520 msgr2=0x7f38f4108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.140+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 shutdown_connections 2024-09-10T04:35:46.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.140+0000 7f38f96c2640 1 -- 172.21.15.46:0/81932446 wait complete. 2024-09-10T04:35:46.150 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:35:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 -- 172.21.15.46:0/1628742104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70101000 msgr2=0x7f7e7010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 --2- 172.21.15.46:0/1628742104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70101000 0x7f7e7010dcc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009880 tx=0x7f7e6002a490 comp rx=0 tx=0).stop 2024-09-10T04:35:46.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 -- 172.21.15.46:0/1628742104 shutdown_connections 2024-09-10T04:35:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 --2- 172.21.15.46:0/1628742104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70101000 0x7f7e7010dcc0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 --2- 172.21.15.46:0/1628742104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e70100650 0x7f7e70100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 -- 172.21.15.46:0/1628742104 >> 172.21.15.46:0/1628742104 conn(0x7f7e700fc520 msgr2=0x7f7e700fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 -- 172.21.15.46:0/1628742104 shutdown_connections 2024-09-10T04:35:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.225+0000 7f7e777c0640 1 -- 172.21.15.46:0/1628742104 wait complete. 2024-09-10T04:35:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e777c0640 1 Processor -- start 2024-09-10T04:35:46.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e777c0640 1 -- start start 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e777c0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70100650 0x7f7e7019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e777c0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e70101000 0x7f7e7019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e777c0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70110f70 con 0x7f7e70100650 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e777c0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7e70110df0 con 0x7f7e70101000 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e7555c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70100650 0x7f7e7019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e7555c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70100650 0x7f7e7019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58544/0 (socket says 172.21.15.46:58544) 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e7555c640 1 -- 172.21.15.46:0/17954233 learned_addr learned my addr 172.21.15.46:0/17954233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e74d5b640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e70101000 0x7f7e7019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e7555c640 1 -- 172.21.15.46:0/17954233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e70101000 msgr2=0x7f7e7019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e7555c640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e70101000 0x7f7e7019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e7555c640 1 -- 172.21.15.46:0/17954233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e701999e0 con 0x7f7e70100650 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.226+0000 7f7e74d5b640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e70101000 0x7f7e7019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.227+0000 7f7e7555c640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70100650 0x7f7e7019f820 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00d8d0 tx=0x7f7e6c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.227+0000 7f7e667fc640 1 -- 172.21.15.46:0/17954233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e6c017070 con 0x7f7e70100650 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.227+0000 7f7e667fc640 1 -- 172.21.15.46:0/17954233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e6c010460 con 0x7f7e70100650 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.227+0000 7f7e667fc640 1 -- 172.21.15.46:0/17954233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7e6c00f5f0 con 0x7f7e70100650 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.227+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e70199cd0 con 0x7f7e70100650 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.227+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7019a230 con 0x7f7e70100650 2024-09-10T04:35:46.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.228+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7006aa90 con 0x7f7e70100650 2024-09-10T04:35:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.229+0000 7f7e667fc640 1 -- 172.21.15.46:0/17954233 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7e6c009a30 con 0x7f7e70100650 2024-09-10T04:35:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.229+0000 7f7e667fc640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e5c07ac40 0x7f7e5c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.230+0000 7f7e667fc640 1 -- 172.21.15.46:0/17954233 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7e6c09db30 con 0x7f7e70100650 2024-09-10T04:35:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.230+0000 7f7e74d5b640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e5c07ac40 0x7f7e5c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.230+0000 7f7e74d5b640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e5c07ac40 0x7f7e5c07d100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7e7019b1d0 tx=0x7f7e60031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:46.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.234+0000 7f7e667fc640 1 -- 172.21.15.46:0/17954233 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7e6c069250 con 0x7f7e70100650 2024-09-10T04:35:46.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.440+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7e7019af10 con 0x7f7e5c07ac40 2024-09-10T04:35:46.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.442+0000 7f7e667fc640 1 -- 172.21.15.46:0/17954233 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7e7019af10 con 0x7f7e5c07ac40 2024-09-10T04:35:46.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e5c07ac40 msgr2=0x7f7e5c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e5c07ac40 0x7f7e5c07d100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7e7019b1d0 tx=0x7f7e60031400 comp rx=0 tx=0).stop 2024-09-10T04:35:46.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70100650 msgr2=0x7f7e7019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70100650 0x7f7e7019f820 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00d8d0 tx=0x7f7e6c00dd90 comp rx=0 tx=0).stop 2024-09-10T04:35:46.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 shutdown_connections 2024-09-10T04:35:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7e5c07ac40 0x7f7e5c07d100 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e70101000 0x7f7e7019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 --2- 172.21.15.46:0/17954233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e70100650 0x7f7e7019f820 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 >> 172.21.15.46:0/17954233 conn(0x7f7e700fc520 msgr2=0x7f7e700fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 shutdown_connections 2024-09-10T04:35:46.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.444+0000 7f7e777c0640 1 -- 172.21.15.46:0/17954233 wait complete. 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 -- 172.21.15.46:0/3291309216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100650 msgr2=0x7f49fc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 --2- 172.21.15.46:0/3291309216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100650 0x7f49fc100a30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009880 tx=0x7f49f802a490 comp rx=0 tx=0).stop 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 -- 172.21.15.46:0/3291309216 shutdown_connections 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 --2- 172.21.15.46:0/3291309216 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc101000 0x7f49fc10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 --2- 172.21.15.46:0/3291309216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100650 0x7f49fc100a30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 -- 172.21.15.46:0/3291309216 >> 172.21.15.46:0/3291309216 conn(0x7f49fc0fc520 msgr2=0x7f49fc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 -- 172.21.15.46:0/3291309216 shutdown_connections 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.528+0000 7f4a04599640 1 -- 172.21.15.46:0/3291309216 wait complete. 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.529+0000 7f4a04599640 1 Processor -- start 2024-09-10T04:35:46.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.529+0000 7f4a04599640 1 -- start start 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.529+0000 7f4a04599640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc100650 0x7f49fc19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.529+0000 7f4a04599640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc101000 0x7f49fc19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.529+0000 7f4a04599640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc110f90 con 0x7f49fc101000 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.529+0000 7f4a04599640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc110e10 con 0x7f49fc100650 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a01b34640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc101000 0x7f49fc19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a01b34640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc101000 0x7f49fc19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58570/0 (socket says 172.21.15.46:58570) 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a02335640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc100650 0x7f49fc19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a01b34640 1 -- 172.21.15.46:0/1238541710 learned_addr learned my addr 172.21.15.46:0/1238541710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a01b34640 1 -- 172.21.15.46:0/1238541710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc100650 msgr2=0x7f49fc19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a01b34640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc100650 0x7f49fc19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a01b34640 1 -- 172.21.15.46:0/1238541710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc1a03b0 con 0x7f49fc101000 2024-09-10T04:35:46.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a01b34640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc101000 0x7f49fc19fd00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c8a0 tx=0x7f49ec00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f49f37fe640 1 -- 172.21.15.46:0/1238541710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49ec0079b0 con 0x7f49fc101000 2024-09-10T04:35:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc199aa0 con 0x7f49fc101000 2024-09-10T04:35:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f49f37fe640 1 -- 172.21.15.46:0/1238541710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49ec007b50 con 0x7f49fc101000 2024-09-10T04:35:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f49f37fe640 1 -- 172.21.15.46:0/1238541710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49ec018700 con 0x7f49fc101000 2024-09-10T04:35:46.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.530+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc19a000 con 0x7f49fc101000 2024-09-10T04:35:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.531+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc06aa90 con 0x7f49fc101000 2024-09-10T04:35:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.532+0000 7f49f37fe640 1 -- 172.21.15.46:0/1238541710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f49ec007d00 con 0x7f49fc101000 2024-09-10T04:35:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.533+0000 7f49f37fe640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807abf0 0x7f49e807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.533+0000 7f49f37fe640 1 -- 172.21.15.46:0/1238541710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f49ec09dff0 con 0x7f49fc101000 2024-09-10T04:35:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.533+0000 7f4a02335640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807abf0 0x7f49e807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.534+0000 7f4a02335640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807abf0 0x7f49e807d0b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000c00 tx=0x7f49f8031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:46.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.536+0000 7f49f37fe640 1 -- 172.21.15.46:0/1238541710 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f49ec014030 con 0x7f49fc101000 2024-09-10T04:35:46.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.733+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f49fc19af50 con 0x7f49e807abf0 2024-09-10T04:35:46.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.743+0000 7f49f37fe640 1 -- 172.21.15.46:0/1238541710 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f49fc19af50 con 0x7f49e807abf0 2024-09-10T04:35:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:35:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (27m) 6m ago 38m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:35:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (29m) 6m ago 38m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (29m) 6m ago 38m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (25m) 6m ago 38m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (25m) 6m ago 38m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (26m) 6m ago 38m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (21m) 6m ago 33m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (22m) 6m ago 33m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (21m) 6m ago 33m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (6m) 6m ago 33m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (29m) 6m ago 39m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:35:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (28m) 6m ago 37m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (26m) 6m ago 40m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (25m) 6m ago 37m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (28m) 6m ago 38m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (28m) 6m ago 37m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (25m) 6m ago 36m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (24m) 6m ago 36m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (24m) 6m ago 35m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (23m) 6m ago 35m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (23m) 6m ago 34m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (22m) 6m ago 34m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:35:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (27m) 6m ago 38m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.745+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807abf0 msgr2=0x7f49e807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.745+0000 7f4a04599640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807abf0 0x7f49e807d0b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f49f8000c00 tx=0x7f49f8031400 comp rx=0 tx=0).stop 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.745+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc101000 msgr2=0x7f49fc19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.745+0000 7f4a04599640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc101000 0x7f49fc19fd00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00c8a0 tx=0x7f49ec00cd60 comp rx=0 tx=0).stop 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.746+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 shutdown_connections 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.746+0000 7f4a04599640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807abf0 0x7f49e807d0b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.746+0000 7f4a04599640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc101000 0x7f49fc19fd00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.746+0000 7f4a04599640 1 --2- 172.21.15.46:0/1238541710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc100650 0x7f49fc19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.746+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 >> 172.21.15.46:0/1238541710 conn(0x7f49fc0fc520 msgr2=0x7f49fc0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.746+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 shutdown_connections 2024-09-10T04:35:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.746+0000 7f4a04599640 1 -- 172.21.15.46:0/1238541710 wait complete. 2024-09-10T04:35:46.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 -- 172.21.15.46:0/1285370790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 msgr2=0x7f9958100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 --2- 172.21.15.46:0/1285370790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f9958100c50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9940009880 tx=0x7f994002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.830 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:46 smithi046 ceph-mon[109506]: pgmap v904: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 -- 172.21.15.46:0/1285370790 shutdown_connections 2024-09-10T04:35:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 --2- 172.21.15.46:0/1285370790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f9958100c50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 --2- 172.21.15.46:0/1285370790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99581067f0 0x7f9958106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 -- 172.21.15.46:0/1285370790 >> 172.21.15.46:0/1285370790 conn(0x7f99580fc550 msgr2=0x7f99580fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 -- 172.21.15.46:0/1285370790 shutdown_connections 2024-09-10T04:35:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.829+0000 7f995cdaf640 1 -- 172.21.15.46:0/1285370790 wait complete. 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995cdaf640 1 Processor -- start 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995cdaf640 1 -- start start 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995cdaf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f99581987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995cdaf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99581067f0 0x7f9958198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995cdaf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f995810dcc0 con 0x7f99581007f0 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995cdaf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f995810db40 con 0x7f99581067f0 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f99581987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f99581987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58586/0 (socket says 172.21.15.46:58586) 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995659c640 1 -- 172.21.15.46:0/815734947 learned_addr learned my addr 172.21.15.46:0/815734947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f9955d9b640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99581067f0 0x7f9958198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995659c640 1 -- 172.21.15.46:0/815734947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99581067f0 msgr2=0x7f9958198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995659c640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99581067f0 0x7f9958198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f995659c640 1 -- 172.21.15.46:0/815734947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9958199330 con 0x7f99581007f0 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.830+0000 7f9955d9b640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99581067f0 0x7f9958198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.831+0000 7f995659c640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f99581987f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f994800d6f0 tx=0x7f994800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.831+0000 7f994f7fe640 1 -- 172.21.15.46:0/815734947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9948017070 con 0x7f99581007f0 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.831+0000 7f994f7fe640 1 -- 172.21.15.46:0/815734947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f994800de00 con 0x7f99581007f0 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.831+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9958199620 con 0x7f99581007f0 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.831+0000 7f994f7fe640 1 -- 172.21.15.46:0/815734947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9948004bb0 con 0x7f99581007f0 2024-09-10T04:35:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.831+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99581a1c20 con 0x7f99581007f0 2024-09-10T04:35:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.832+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9958101f30 con 0x7f99581007f0 2024-09-10T04:35:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.833+0000 7f994f7fe640 1 -- 172.21.15.46:0/815734947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9948009a30 con 0x7f99581007f0 2024-09-10T04:35:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.834+0000 7f994f7fe640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994407abf0 0x7f994407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.834+0000 7f994f7fe640 1 -- 172.21.15.46:0/815734947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f994809d930 con 0x7f99581007f0 2024-09-10T04:35:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.834+0000 7f9955d9b640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994407abf0 0x7f994407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.835+0000 7f9955d9b640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994407abf0 0x7f994407d0b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f995806a7f0 tx=0x7f9940031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:46.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:46.838+0000 7f994f7fe640 1 -- 172.21.15.46:0/815734947 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9958199620 con 0x7f99581007f0 2024-09-10T04:35:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:46 smithi070 ceph-mon[79774]: pgmap v904: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.096+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f9958108d60 con 0x7f99581007f0 2024-09-10T04:35:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.097+0000 7f994f7fe640 1 -- 172.21.15.46:0/815734947 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f9948068fd0 con 0x7f99581007f0 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:47.098 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:35:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:35:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:35:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:35:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:35:47.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:35:47.099 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994407abf0 msgr2=0x7f994407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994407abf0 0x7f994407d0b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f995806a7f0 tx=0x7f9940031400 comp rx=0 tx=0).stop 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 msgr2=0x7f99581987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f99581987f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f994800d6f0 tx=0x7f994800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 shutdown_connections 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f994407abf0 0x7f994407d0b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99581067f0 0x7f9958198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 --2- 172.21.15.46:0/815734947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99581007f0 0x7f99581987f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 >> 172.21.15.46:0/815734947 conn(0x7f99580fc550 msgr2=0x7f99580fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:47.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 shutdown_connections 2024-09-10T04:35:47.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.099+0000 7f995cdaf640 1 -- 172.21.15.46:0/815734947 wait complete. 2024-09-10T04:35:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 -- 172.21.15.46:0/950303868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100ff0 msgr2=0x7ff72410dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/950303868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100ff0 0x7ff72410dcb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff714009880 tx=0x7ff71402a490 comp rx=0 tx=0).stop 2024-09-10T04:35:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 -- 172.21.15.46:0/950303868 shutdown_connections 2024-09-10T04:35:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/950303868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100ff0 0x7ff72410dcb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/950303868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100640 0x7ff724100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 -- 172.21.15.46:0/950303868 >> 172.21.15.46:0/950303868 conn(0x7ff7240fc550 msgr2=0x7ff7240fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:47.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 -- 172.21.15.46:0/950303868 shutdown_connections 2024-09-10T04:35:47.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 -- 172.21.15.46:0/950303868 wait complete. 2024-09-10T04:35:47.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 Processor -- start 2024-09-10T04:35:47.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.185+0000 7ff72bcf5640 1 -- start start 2024-09-10T04:35:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff72bcf5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100640 0x7ff72419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff72bcf5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100ff0 0x7ff72419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff72bcf5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff724110f90 con 0x7ff724100640 2024-09-10T04:35:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff72bcf5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff724110e10 con 0x7ff724100ff0 2024-09-10T04:35:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729a91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100640 0x7ff72419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729a91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100640 0x7ff72419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58604/0 (socket says 172.21.15.46:58604) 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729a91640 1 -- 172.21.15.46:0/3977388146 learned_addr learned my addr 172.21.15.46:0/3977388146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729290640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100ff0 0x7ff72419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729a91640 1 -- 172.21.15.46:0/3977388146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100ff0 msgr2=0x7ff72419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729a91640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100ff0 0x7ff72419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729a91640 1 -- 172.21.15.46:0/3977388146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241999c0 con 0x7ff724100640 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.186+0000 7ff729a91640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100640 0x7ff72419f800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e970 tx=0x7ff72000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.187+0000 7ff71affd640 1 -- 172.21.15.46:0/3977388146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff72000cac0 con 0x7ff724100640 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.187+0000 7ff71affd640 1 -- 172.21.15.46:0/3977388146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff7200054e0 con 0x7ff724100640 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.187+0000 7ff71affd640 1 -- 172.21.15.46:0/3977388146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff720010430 con 0x7ff724100640 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.187+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff724199cb0 con 0x7ff724100640 2024-09-10T04:35:47.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.187+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72419a210 con 0x7ff724100640 2024-09-10T04:35:47.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.188+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72406a580 con 0x7ff724100640 2024-09-10T04:35:47.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.189+0000 7ff71affd640 1 -- 172.21.15.46:0/3977388146 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff720004e80 con 0x7ff724100640 2024-09-10T04:35:47.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.189+0000 7ff71affd640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff71007abf0 0x7ff71007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.189+0000 7ff71affd640 1 -- 172.21.15.46:0/3977388146 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff72009e2d0 con 0x7ff724100640 2024-09-10T04:35:47.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.189+0000 7ff729290640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff71007abf0 0x7ff71007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.190+0000 7ff729290640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff71007abf0 0x7ff71007d0b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff72419b1b0 tx=0x7ff714031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:47.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.194+0000 7ff71affd640 1 -- 172.21.15.46:0/3977388146 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff720069a10 con 0x7ff724100640 2024-09-10T04:35:47.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.419+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff72419aef0 con 0x7ff724100640 2024-09-10T04:35:47.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.419+0000 7ff71affd640 1 -- 172.21.15.46:0/3977388146 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7ff72006e8c0 con 0x7ff724100640 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:35:47.421 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:35:47.422 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:47.424 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:35:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:35:47.424 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:47.424 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:47.424 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:47.424 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:35:47.424 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.421+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff71007abf0 msgr2=0x7ff71007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.421+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff71007abf0 0x7ff71007d0b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff72419b1b0 tx=0x7ff714031400 comp rx=0 tx=0).stop 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100640 msgr2=0x7ff72419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100640 0x7ff72419f800 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e970 tx=0x7ff72000ed70 comp rx=0 tx=0).stop 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 shutdown_connections 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff71007abf0 0x7ff71007d0b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff724100ff0 0x7ff72419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 --2- 172.21.15.46:0/3977388146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff724100640 0x7ff72419f800 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 >> 172.21.15.46:0/3977388146 conn(0x7ff7240fc550 msgr2=0x7ff7240fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 shutdown_connections 2024-09-10T04:35:47.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.422+0000 7ff72bcf5640 1 -- 172.21.15.46:0/3977388146 wait complete. 2024-09-10T04:35:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 -- 172.21.15.46:0/2732084235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 msgr2=0x7fcbc8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/2732084235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc8102ea0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009880 tx=0x7fcbb802a530 comp rx=0 tx=0).stop 2024-09-10T04:35:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 -- 172.21.15.46:0/2732084235 shutdown_connections 2024-09-10T04:35:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/2732084235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc81033e0 0x7fcbc810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/2732084235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc8102ea0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 -- 172.21.15.46:0/2732084235 >> 172.21.15.46:0/2732084235 conn(0x7fcbc80fc520 msgr2=0x7fcbc80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 -- 172.21.15.46:0/2732084235 shutdown_connections 2024-09-10T04:35:47.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.507+0000 7fcbcd1da640 1 -- 172.21.15.46:0/2732084235 wait complete. 2024-09-10T04:35:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.508+0000 7fcbcd1da640 1 Processor -- start 2024-09-10T04:35:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.508+0000 7fcbcd1da640 1 -- start start 2024-09-10T04:35:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.508+0000 7fcbcd1da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.508+0000 7fcbcd1da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc81033e0 0x7fcbc819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.508+0000 7fcbcd1da640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc810edf0 con 0x7fcbc8102ac0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.508+0000 7fcbcd1da640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc810ec70 con 0x7fcbc81033e0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58626/0 (socket says 172.21.15.46:58626) 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc6d9d640 1 -- 172.21.15.46:0/777192848 learned_addr learned my addr 172.21.15.46:0/777192848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc659c640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc81033e0 0x7fcbc819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc6d9d640 1 -- 172.21.15.46:0/777192848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc81033e0 msgr2=0x7fcbc819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc6d9d640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc81033e0 0x7fcbc819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc6d9d640 1 -- 172.21.15.46:0/777192848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc81977c0 con 0x7fcbc8102ac0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbc6d9d640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc819d640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8031460 tx=0x7fcbb80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbbffff640 1 -- 172.21.15.46:0/777192848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcbb803f070 con 0x7fcbc8102ac0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc8197a50 con 0x7fcbc8102ac0 2024-09-10T04:35:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbbffff640 1 -- 172.21.15.46:0/777192848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcbb8004cf0 con 0x7fcbc8102ac0 2024-09-10T04:35:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbbffff640 1 -- 172.21.15.46:0/777192848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcbb8035030 con 0x7fcbc8102ac0 2024-09-10T04:35:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.509+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc8197f80 con 0x7fcbc8102ac0 2024-09-10T04:35:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.510+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc806aa90 con 0x7fcbc8102ac0 2024-09-10T04:35:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.511+0000 7fcbbffff640 1 -- 172.21.15.46:0/777192848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcbb8042070 con 0x7fcbc8102ac0 2024-09-10T04:35:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.512+0000 7fcbbffff640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbb407abf0 0x7fcbb407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.512+0000 7fcbbffff640 1 -- 172.21.15.46:0/777192848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcbb80be6a0 con 0x7fcbc8102ac0 2024-09-10T04:35:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.512+0000 7fcbc659c640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbb407abf0 0x7fcbb407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.512+0000 7fcbc659c640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbb407abf0 0x7fcbb407d0b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8108900 tx=0x7fcbb0005f50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.516+0000 7fcbbffff640 1 -- 172.21.15.46:0/777192848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fcbb8089de0 con 0x7fcbc8102ac0 2024-09-10T04:35:47.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.724+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fcbc8198290 con 0x7fcbb407abf0 2024-09-10T04:35:47.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:47 smithi046 ceph-mon[109506]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:47.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:47 smithi046 ceph-mon[109506]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:47.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:47 smithi046 ceph-mon[109506]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:47.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:47 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/815734947' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:35:47.725 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:47 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3977388146' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.725+0000 7fcbbffff640 1 -- 172.21.15.46:0/777192848 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fcbc8198290 con 0x7fcbb407abf0 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:35:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:35:47.727 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:35:47.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbb407abf0 msgr2=0x7fcbb407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbb407abf0 0x7fcbb407d0b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8108900 tx=0x7fcbb0005f50 comp rx=0 tx=0).stop 2024-09-10T04:35:47.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 msgr2=0x7fcbc819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc819d640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8031460 tx=0x7fcbb80048f0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 shutdown_connections 2024-09-10T04:35:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbb407abf0 0x7fcbb407d0b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbc81033e0 0x7fcbc819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 --2- 172.21.15.46:0/777192848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbc8102ac0 0x7fcbc819d640 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.727+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 >> 172.21.15.46:0/777192848 conn(0x7fcbc80fc520 msgr2=0x7fcbc810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.728+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 shutdown_connections 2024-09-10T04:35:47.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.728+0000 7fcbcd1da640 1 -- 172.21.15.46:0/777192848 wait complete. 2024-09-10T04:35:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 -- 172.21.15.46:0/3911536401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 msgr2=0x7f3c5010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 --2- 172.21.15.46:0/3911536401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5010b930 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3c400098b0 tx=0x7f3c4002a560 comp rx=0 tx=0).stop 2024-09-10T04:35:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 -- 172.21.15.46:0/3911536401 shutdown_connections 2024-09-10T04:35:47.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 --2- 172.21.15.46:0/3911536401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5010b930 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 --2- 172.21.15.46:0/3911536401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c50102b20 0x7f3c50102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 -- 172.21.15.46:0/3911536401 >> 172.21.15.46:0/3911536401 conn(0x7f3c500fc520 msgr2=0x7f3c500fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 -- 172.21.15.46:0/3911536401 shutdown_connections 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.812+0000 7f3c58021640 1 -- 172.21.15.46:0/3911536401 wait complete. 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c58021640 1 Processor -- start 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c58021640 1 -- start start 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c58021640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c50102b20 0x7f3c5019d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c58021640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5019dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c58021640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5010edf0 con 0x7f3c50103440 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c58021640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5010ec70 con 0x7f3c50102b20 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c555bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5019dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c555bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5019dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58646/0 (socket says 172.21.15.46:58646) 2024-09-10T04:35:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c555bc640 1 -- 172.21.15.46:0/1144145802 learned_addr learned my addr 172.21.15.46:0/1144145802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:35:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c55dbd640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c50102b20 0x7f3c5019d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c555bc640 1 -- 172.21.15.46:0/1144145802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c50102b20 msgr2=0x7f3c5019d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c555bc640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c50102b20 0x7f3c5019d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.813+0000 7f3c555bc640 1 -- 172.21.15.46:0/1144145802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c501978a0 con 0x7f3c50103440 2024-09-10T04:35:47.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.814+0000 7f3c555bc640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5019dc20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3c40031460 tx=0x7f3c400045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.814+0000 7f3c46ffd640 1 -- 172.21.15.46:0/1144145802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3c4003f070 con 0x7f3c50103440 2024-09-10T04:35:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.814+0000 7f3c46ffd640 1 -- 172.21.15.46:0/1144145802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3c400035f0 con 0x7f3c50103440 2024-09-10T04:35:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.814+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c50197b30 con 0x7f3c50103440 2024-09-10T04:35:47.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.814+0000 7f3c46ffd640 1 -- 172.21.15.46:0/1144145802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3c40035030 con 0x7f3c50103440 2024-09-10T04:35:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.814+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c50198060 con 0x7f3c50103440 2024-09-10T04:35:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.815+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c5006aa30 con 0x7f3c50103440 2024-09-10T04:35:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.816+0000 7f3c46ffd640 1 -- 172.21.15.46:0/1144145802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3c40042070 con 0x7f3c50103440 2024-09-10T04:35:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.816+0000 7f3c46ffd640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c3c07abf0 0x7f3c3c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:35:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.817+0000 7f3c55dbd640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c3c07abf0 0x7f3c3c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:35:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.817+0000 7f3c46ffd640 1 -- 172.21.15.46:0/1144145802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3c400be550 con 0x7f3c50103440 2024-09-10T04:35:47.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.817+0000 7f3c55dbd640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c3c07abf0 0x7f3c3c07d0b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3c501989e0 tx=0x7f3c4c00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:35:47.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:47.821+0000 7f3c46ffd640 1 -- 172.21.15.46:0/1144145802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3c40089c90 con 0x7f3c50103440 2024-09-10T04:35:48.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.080+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f3c50105fe0 con 0x7f3c50103440 2024-09-10T04:35:48.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.081+0000 7f3c46ffd640 1 -- 172.21.15.46:0/1144145802 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f3c4008eb40 con 0x7f3c50103440 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:35:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:35:48.083 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:35:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:35:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c3c07abf0 msgr2=0x7f3c3c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c3c07abf0 0x7f3c3c07d0b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3c501989e0 tx=0x7f3c4c00b390 comp rx=0 tx=0).stop 2024-09-10T04:35:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 msgr2=0x7f3c5019dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:35:48.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5019dc20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3c40031460 tx=0x7f3c400045c0 comp rx=0 tx=0).stop 2024-09-10T04:35:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 shutdown_connections 2024-09-10T04:35:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3c3c07abf0 0x7f3c3c07d0b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c50103440 0x7f3c5019dc20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 --2- 172.21.15.46:0/1144145802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c50102b20 0x7f3c5019d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:35:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 >> 172.21.15.46:0/1144145802 conn(0x7f3c500fc520 msgr2=0x7f3c50103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:35:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 shutdown_connections 2024-09-10T04:35:48.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:35:48.083+0000 7f3c58021640 1 -- 172.21.15.46:0/1144145802 wait complete. 2024-09-10T04:35:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:47 smithi070 ceph-mon[79774]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:48.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:47 smithi070 ceph-mon[79774]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:48.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:47 smithi070 ceph-mon[79774]: from='client.35494 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:48.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:47 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/815734947' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:35:48.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:47 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3977388146' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:35:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:48 smithi070 ceph-mon[79774]: pgmap v905: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:48 smithi070 ceph-mon[79774]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:48 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1144145802' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:35:49.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:48 smithi046 ceph-mon[109506]: pgmap v905: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:49.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:48 smithi046 ceph-mon[109506]: from='client.35506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:35:49.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:48 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1144145802' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:35:50.887 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:50 smithi070 ceph-mon[79774]: pgmap v906: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:51.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:50 smithi046 ceph-mon[109506]: pgmap v906: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:52 smithi070 ceph-mon[79774]: pgmap v907: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:52 smithi046 ceph-mon[109506]: pgmap v907: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:35:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:54 smithi070 ceph-mon[79774]: pgmap v908: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:54 smithi046 ceph-mon[109506]: pgmap v908: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:56 smithi070 ceph-mon[79774]: pgmap v909: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:56 smithi046 ceph-mon[109506]: pgmap v909: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:35:58 smithi070 ceph-mon[79774]: pgmap v910: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:35:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:35:58 smithi046 ceph-mon[109506]: pgmap v910: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:00 smithi070 ceph-mon[79774]: pgmap v911: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:00 smithi046 ceph-mon[109506]: pgmap v911: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:02 smithi070 ceph-mon[79774]: pgmap v912: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:02 smithi046 ceph-mon[109506]: pgmap v912: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:04 smithi070 ceph-mon[79774]: pgmap v913: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:04 smithi046 ceph-mon[109506]: pgmap v913: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:06 smithi070 ceph-mon[79774]: pgmap v914: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:06 smithi046 ceph-mon[109506]: pgmap v914: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:08 smithi070 ceph-mon[79774]: pgmap v915: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:08 smithi046 ceph-mon[109506]: pgmap v915: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:10 smithi070 ceph-mon[79774]: pgmap v916: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:10 smithi046 ceph-mon[109506]: pgmap v916: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:12 smithi046 ceph-mon[109506]: pgmap v917: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:12 smithi070 ceph-mon[79774]: pgmap v917: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:14 smithi070 ceph-mon[79774]: pgmap v918: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:14 smithi046 ceph-mon[109506]: pgmap v918: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:16 smithi070 ceph-mon[79774]: pgmap v919: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:16 smithi046 ceph-mon[109506]: pgmap v919: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:18.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.179+0000 7f2c0046f640 1 -- 172.21.15.46:0/33005263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 msgr2=0x7f2bf8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.179+0000 7f2c0046f640 1 --2- 172.21.15.46:0/33005263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf8102ea0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009880 tx=0x7f2bf402a530 comp rx=0 tx=0).stop 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 -- 172.21.15.46:0/33005263 shutdown_connections 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 --2- 172.21.15.46:0/33005263 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bf81033e0 0x7f2bf810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 --2- 172.21.15.46:0/33005263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf8102ea0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 -- 172.21.15.46:0/33005263 >> 172.21.15.46:0/33005263 conn(0x7f2bf80fc520 msgr2=0x7f2bf80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 -- 172.21.15.46:0/33005263 shutdown_connections 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 -- 172.21.15.46:0/33005263 wait complete. 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 Processor -- start 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.180+0000 7f2c0046f640 1 -- start start 2024-09-10T04:36:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2c0046f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2c0046f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bf81033e0 0x7f2bf819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2c0046f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf810edf0 con 0x7f2bf8102ac0 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2c0046f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf810ec70 con 0x7f2bf81033e0 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfe20b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfe20b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58886/0 (socket says 172.21.15.46:58886) 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfe20b640 1 -- 172.21.15.46:0/1824534055 learned_addr learned my addr 172.21.15.46:0/1824534055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfda0a640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bf81033e0 0x7f2bf819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfe20b640 1 -- 172.21.15.46:0/1824534055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bf81033e0 msgr2=0x7f2bf819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfe20b640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bf81033e0 0x7f2bf819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfe20b640 1 -- 172.21.15.46:0/1824534055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf81977c0 con 0x7f2bf8102ac0 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.181+0000 7f2bfe20b640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf819d640 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4031460 tx=0x7f2bf4004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:18.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.182+0000 7f2bef7fe640 1 -- 172.21.15.46:0/1824534055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bf403f070 con 0x7f2bf8102ac0 2024-09-10T04:36:18.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.182+0000 7f2bef7fe640 1 -- 172.21.15.46:0/1824534055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bf40035f0 con 0x7f2bf8102ac0 2024-09-10T04:36:18.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.182+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf8197a50 con 0x7f2bf8102ac0 2024-09-10T04:36:18.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.182+0000 7f2bef7fe640 1 -- 172.21.15.46:0/1824534055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2bf4035030 con 0x7f2bf8102ac0 2024-09-10T04:36:18.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.182+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf8197f80 con 0x7f2bf8102ac0 2024-09-10T04:36:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.183+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf806aa90 con 0x7f2bf8102ac0 2024-09-10T04:36:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.184+0000 7f2bef7fe640 1 -- 172.21.15.46:0/1824534055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2bf4042070 con 0x7f2bf8102ac0 2024-09-10T04:36:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.184+0000 7f2bef7fe640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2be407ac40 0x7f2be407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.184+0000 7f2bef7fe640 1 -- 172.21.15.46:0/1824534055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2bf40be550 con 0x7f2bf8102ac0 2024-09-10T04:36:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.184+0000 7f2bfda0a640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2be407ac40 0x7f2be407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.185+0000 7f2bfda0a640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2be407ac40 0x7f2be407d100 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8108900 tx=0x7f2be8005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:18.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.189+0000 7f2bef7fe640 1 -- 172.21.15.46:0/1824534055 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2bf4089d10 con 0x7f2bf8102ac0 2024-09-10T04:36:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.396+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2bf81982f0 con 0x7f2be407ac40 2024-09-10T04:36:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.397+0000 7f2bef7fe640 1 -- 172.21.15.46:0/1824534055 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f2bf81982f0 con 0x7f2be407ac40 2024-09-10T04:36:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2be407ac40 msgr2=0x7f2be407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2be407ac40 0x7f2be407d100 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8108900 tx=0x7f2be8005f90 comp rx=0 tx=0).stop 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 msgr2=0x7f2bf819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf819d640 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4031460 tx=0x7f2bf4004530 comp rx=0 tx=0).stop 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 shutdown_connections 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2be407ac40 0x7f2be407d100 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2bf81033e0 0x7f2bf819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 --2- 172.21.15.46:0/1824534055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2bf8102ac0 0x7f2bf819d640 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.399+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 >> 172.21.15.46:0/1824534055 conn(0x7f2bf80fc520 msgr2=0x7f2bf810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.400+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 shutdown_connections 2024-09-10T04:36:18.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.400+0000 7f2c0046f640 1 -- 172.21.15.46:0/1824534055 wait complete. 2024-09-10T04:36:18.410 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:36:18.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.484+0000 7f4369826640 1 -- 172.21.15.46:0/1211614110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 msgr2=0x7f436410b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.484+0000 7f4369826640 1 --2- 172.21.15.46:0/1211614110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436410b960 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4358009880 tx=0x7f435802a530 comp rx=0 tx=0).stop 2024-09-10T04:36:18.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 -- 172.21.15.46:0/1211614110 shutdown_connections 2024-09-10T04:36:18.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 --2- 172.21.15.46:0/1211614110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436410b960 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 --2- 172.21.15.46:0/1211614110 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4364102b50 0x7f4364102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 -- 172.21.15.46:0/1211614110 >> 172.21.15.46:0/1211614110 conn(0x7f43640fc550 msgr2=0x7f43640fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:18.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 -- 172.21.15.46:0/1211614110 shutdown_connections 2024-09-10T04:36:18.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 -- 172.21.15.46:0/1211614110 wait complete. 2024-09-10T04:36:18.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 Processor -- start 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.485+0000 7f4369826640 1 -- start start 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f4369826640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4364102b50 0x7f436419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f4369826640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f4369826640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f436410ede0 con 0x7f4364103470 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f4369826640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f436410ec60 con 0x7f4364102b50 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f43627fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f43627fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436419db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58912/0 (socket says 172.21.15.46:58912) 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f43627fc640 1 -- 172.21.15.46:0/1579609073 learned_addr learned my addr 172.21.15.46:0/1579609073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f4362ffd640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4364102b50 0x7f436419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f43627fc640 1 -- 172.21.15.46:0/1579609073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4364102b50 msgr2=0x7f436419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f43627fc640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4364102b50 0x7f436419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f43627fc640 1 -- 172.21.15.46:0/1579609073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43641977b0 con 0x7f4364103470 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.486+0000 7f43627fc640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436419db70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4358004f90 tx=0x7f43580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.487+0000 7f4368824640 1 -- 172.21.15.46:0/1579609073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f435803f070 con 0x7f4364103470 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.487+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4364197a40 con 0x7f4364103470 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.487+0000 7f4368824640 1 -- 172.21.15.46:0/1579609073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4358004ac0 con 0x7f4364103470 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.487+0000 7f4368824640 1 -- 172.21.15.46:0/1579609073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4358035030 con 0x7f4364103470 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.487+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4364197f70 con 0x7f4364103470 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.488+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436406a4e0 con 0x7f4364103470 2024-09-10T04:36:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.489+0000 7f4368824640 1 -- 172.21.15.46:0/1579609073 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4358042070 con 0x7f4364103470 2024-09-10T04:36:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.489+0000 7f4368824640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434c07abf0 0x7f434c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.489+0000 7f4368824640 1 -- 172.21.15.46:0/1579609073 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f43580be530 con 0x7f4364103470 2024-09-10T04:36:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.489+0000 7f4362ffd640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434c07abf0 0x7f434c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.490+0000 7f4362ffd640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434c07abf0 0x7f434c07d0b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4354005fd0 tx=0x7f4354009500 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:18.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.494+0000 7f4368824640 1 -- 172.21.15.46:0/1579609073 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4358089c70 con 0x7f4364103470 2024-09-10T04:36:18.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.703+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4364198220 con 0x7f434c07abf0 2024-09-10T04:36:18.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.705+0000 7f4368824640 1 -- 172.21.15.46:0/1579609073 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f4364198220 con 0x7f434c07abf0 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434c07abf0 msgr2=0x7f434c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434c07abf0 0x7f434c07d0b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4354005fd0 tx=0x7f4354009500 comp rx=0 tx=0).stop 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 msgr2=0x7f436419db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436419db70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f4358004f90 tx=0x7f43580046c0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 shutdown_connections 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f434c07abf0 0x7f434c07d0b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4364103470 0x7f436419db70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 --2- 172.21.15.46:0/1579609073 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4364102b50 0x7f436419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 >> 172.21.15.46:0/1579609073 conn(0x7f43640fc550 msgr2=0x7f4364103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:18.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 shutdown_connections 2024-09-10T04:36:18.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.707+0000 7f4369826640 1 -- 172.21.15.46:0/1579609073 wait complete. 2024-09-10T04:36:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 -- 172.21.15.46:0/667449103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 msgr2=0x7f30e0101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 --2- 172.21.15.46:0/667449103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e0101620 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f30d4009880 tx=0x7f30d402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.793 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:18 smithi046 ceph-mon[109506]: pgmap v920: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 -- 172.21.15.46:0/667449103 shutdown_connections 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 --2- 172.21.15.46:0/667449103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e0101620 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 --2- 172.21.15.46:0/667449103 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e0106800 0x7f30e0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 -- 172.21.15.46:0/667449103 >> 172.21.15.46:0/667449103 conn(0x7f30e0077270 msgr2=0x7f30e0077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 -- 172.21.15.46:0/667449103 shutdown_connections 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.791+0000 7f30e6de7640 1 -- 172.21.15.46:0/667449103 wait complete. 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.792+0000 7f30e6de7640 1 Processor -- start 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.792+0000 7f30e6de7640 1 -- start start 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.792+0000 7f30e6de7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.792+0000 7f30e6de7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e0106800 0x7f30e019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.792+0000 7f30e6de7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30e010dcc0 con 0x7f30e01011e0 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.792+0000 7f30e6de7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f30e010db40 con 0x7f30e0106800 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.792+0000 7f30e4b83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30e4b83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58922/0 (socket says 172.21.15.46:58922) 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30e4b83640 1 -- 172.21.15.46:0/3743223579 learned_addr learned my addr 172.21.15.46:0/3743223579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30dffff640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e0106800 0x7f30e019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30e4b83640 1 -- 172.21.15.46:0/3743223579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e0106800 msgr2=0x7f30e019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30e4b83640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e0106800 0x7f30e019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30e4b83640 1 -- 172.21.15.46:0/3743223579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e01977f0 con 0x7f30e01011e0 2024-09-10T04:36:18.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30e4b83640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e019d670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f30d000cc60 tx=0x7f30d000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30ddffb640 1 -- 172.21.15.46:0/3743223579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f30d000ad30 con 0x7f30e01011e0 2024-09-10T04:36:18.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30ddffb640 1 -- 172.21.15.46:0/3743223579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f30d0005020 con 0x7f30e01011e0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30ddffb640 1 -- 172.21.15.46:0/3743223579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f30d000f660 con 0x7f30e01011e0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.793+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e0197ae0 con 0x7f30e01011e0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.794+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e0197fe0 con 0x7f30e01011e0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.795+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e0069f00 con 0x7f30e01011e0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.795+0000 7f30ddffb640 1 -- 172.21.15.46:0/3743223579 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f30d00051d0 con 0x7f30e01011e0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.796+0000 7f30ddffb640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30c807abf0 0x7f30c807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.796+0000 7f30ddffb640 1 -- 172.21.15.46:0/3743223579 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f30d009e190 con 0x7f30e01011e0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.796+0000 7f30dffff640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30c807abf0 0x7f30c807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.796+0000 7f30dffff640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30c807abf0 0x7f30c807d0b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f30e0198b40 tx=0x7f30d4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:18.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.800+0000 7f30ddffb640 1 -- 172.21.15.46:0/3743223579 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f30d0014030 con 0x7f30e01011e0 2024-09-10T04:36:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:18.994+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f30e0198350 con 0x7f30c807abf0 2024-09-10T04:36:19.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.003+0000 7f30ddffb640 1 -- 172.21.15.46:0/3743223579 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f30e0198350 con 0x7f30c807abf0 2024-09-10T04:36:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:36:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (27m) 6m ago 39m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:36:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (30m) 6m ago 39m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:36:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (30m) 7m ago 38m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:36:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (26m) 6m ago 39m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:36:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (26m) 7m ago 38m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (27m) 6m ago 39m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (22m) 6m ago 33m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (22m) 6m ago 33m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (22m) 7m ago 33m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (7m) 7m ago 33m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (29m) 6m ago 40m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (28m) 7m ago 38m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (26m) 6m ago 40m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (26m) 7m ago 38m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (28m) 6m ago 39m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (28m) 7m ago 38m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (25m) 6m ago 37m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:36:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (25m) 6m ago 36m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:36:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (24m) 6m ago 36m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:36:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (24m) 7m ago 35m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:36:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (23m) 7m ago 35m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:36:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (23m) 7m ago 34m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:36:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (28m) 6m ago 38m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:36:19.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30c807abf0 msgr2=0x7f30c807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30c807abf0 0x7f30c807d0b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f30e0198b40 tx=0x7f30d4031400 comp rx=0 tx=0).stop 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 msgr2=0x7f30e019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e019d670 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f30d000cc60 tx=0x7f30d000a620 comp rx=0 tx=0).stop 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 shutdown_connections 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30c807abf0 0x7f30c807d0b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30e0106800 0x7f30e019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 --2- 172.21.15.46:0/3743223579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30e01011e0 0x7f30e019d670 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.005+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 >> 172.21.15.46:0/3743223579 conn(0x7f30e0077270 msgr2=0x7f30e0104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.006+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 shutdown_connections 2024-09-10T04:36:19.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.006+0000 7f30e6de7640 1 -- 172.21.15.46:0/3743223579 wait complete. 2024-09-10T04:36:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:18 smithi070 ceph-mon[79774]: pgmap v920: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 -- 172.21.15.46:0/3840685787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000107900 msgr2=0x7fd000107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 --2- 172.21.15.46:0/3840685787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000107900 0x7fd000107ce0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009880 tx=0x7fcff002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 -- 172.21.15.46:0/3840685787 shutdown_connections 2024-09-10T04:36:19.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 --2- 172.21.15.46:0/3840685787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd000101900 0x7fd000101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 --2- 172.21.15.46:0/3840685787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000107900 0x7fd000107ce0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 -- 172.21.15.46:0/3840685787 >> 172.21.15.46:0/3840685787 conn(0x7fd0000fd620 msgr2=0x7fd0000ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 -- 172.21.15.46:0/3840685787 shutdown_connections 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.089+0000 7fd005d6e640 1 -- 172.21.15.46:0/3840685787 wait complete. 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fd005d6e640 1 Processor -- start 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fd005d6e640 1 -- start start 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fd005d6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000101900 0x7fd00019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fd005d6e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd000107900 0x7fd00019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fd005d6e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd00010edd0 con 0x7fd000101900 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fd005d6e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd00010ec50 con 0x7fd000107900 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fcfff7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000101900 0x7fd00019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fcfff7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000101900 0x7fd00019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46638/0 (socket says 172.21.15.46:46638) 2024-09-10T04:36:19.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fcfff7fe640 1 -- 172.21.15.46:0/578110750 learned_addr learned my addr 172.21.15.46:0/578110750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:19.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fcff7fff640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd000107900 0x7fd00019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fcfff7fe640 1 -- 172.21.15.46:0/578110750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd000107900 msgr2=0x7fd00019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fcfff7fe640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd000107900 0x7fd00019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.090+0000 7fcfff7fe640 1 -- 172.21.15.46:0/578110750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0001977a0 con 0x7fd000101900 2024-09-10T04:36:19.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.091+0000 7fcfff7fe640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000101900 0x7fd00019d620 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcff0031460 tx=0x7fcff00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:19.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.091+0000 7fcffd7fa640 1 -- 172.21.15.46:0/578110750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcff003f070 con 0x7fd000101900 2024-09-10T04:36:19.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.091+0000 7fcffd7fa640 1 -- 172.21.15.46:0/578110750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcff0004cf0 con 0x7fd000101900 2024-09-10T04:36:19.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.091+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd000197a30 con 0x7fd000101900 2024-09-10T04:36:19.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.091+0000 7fcffd7fa640 1 -- 172.21.15.46:0/578110750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcff0035030 con 0x7fd000101900 2024-09-10T04:36:19.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.091+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd000197f60 con 0x7fd000101900 2024-09-10T04:36:19.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.092+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd000103040 con 0x7fd000101900 2024-09-10T04:36:19.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.093+0000 7fcffd7fa640 1 -- 172.21.15.46:0/578110750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcff0042070 con 0x7fd000101900 2024-09-10T04:36:19.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.093+0000 7fcffd7fa640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfec07ac40 0x7fcfec07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.093+0000 7fcffd7fa640 1 -- 172.21.15.46:0/578110750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcff00be6b0 con 0x7fd000101900 2024-09-10T04:36:19.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.093+0000 7fcff7fff640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfec07ac40 0x7fcfec07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.094+0000 7fcff7fff640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfec07ac40 0x7fcfec07d100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005fd0 tx=0x7fcfe8005e20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:19.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.098+0000 7fcffd7fa640 1 -- 172.21.15.46:0/578110750 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fcff0089e70 con 0x7fd000101900 2024-09-10T04:36:19.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.356+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fd000109e70 con 0x7fd000101900 2024-09-10T04:36:19.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.357+0000 7fcffd7fa640 1 -- 172.21.15.46:0/578110750 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fcff008ed20 con 0x7fd000101900 2024-09-10T04:36:19.359 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:36:19.359 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:19.360 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:36:19.361 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:36:19.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:19.361 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:36:19.361 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:36:19.361 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:36:19.361 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:36:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfec07ac40 msgr2=0x7fcfec07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfec07ac40 0x7fcfec07d100 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8005fd0 tx=0x7fcfe8005e20 comp rx=0 tx=0).stop 2024-09-10T04:36:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000101900 msgr2=0x7fd00019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000101900 0x7fd00019d620 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcff0031460 tx=0x7fcff00048f0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 shutdown_connections 2024-09-10T04:36:19.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcfec07ac40 0x7fcfec07d100 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd000107900 0x7fd00019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 --2- 172.21.15.46:0/578110750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd000101900 0x7fd00019d620 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 >> 172.21.15.46:0/578110750 conn(0x7fd0000fd620 msgr2=0x7fd0000fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:19.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 shutdown_connections 2024-09-10T04:36:19.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.360+0000 7fd005d6e640 1 -- 172.21.15.46:0/578110750 wait complete. 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.444+0000 7f0bbc448640 1 -- 172.21.15.46:0/2692658825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb4102ac0 msgr2=0x7f0bb4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.444+0000 7f0bbc448640 1 --2- 172.21.15.46:0/2692658825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb4102ac0 0x7f0bb4102ea0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009880 tx=0x7f0bb002a530 comp rx=0 tx=0).stop 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 -- 172.21.15.46:0/2692658825 shutdown_connections 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 --2- 172.21.15.46:0/2692658825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb41033e0 0x7f0bb410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 --2- 172.21.15.46:0/2692658825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb4102ac0 0x7f0bb4102ea0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 -- 172.21.15.46:0/2692658825 >> 172.21.15.46:0/2692658825 conn(0x7f0bb40fc520 msgr2=0x7f0bb40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 -- 172.21.15.46:0/2692658825 shutdown_connections 2024-09-10T04:36:19.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 -- 172.21.15.46:0/2692658825 wait complete. 2024-09-10T04:36:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 Processor -- start 2024-09-10T04:36:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.445+0000 7f0bbc448640 1 -- start start 2024-09-10T04:36:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bbc448640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb4102ac0 0x7f0bb419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bbc448640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb41033e0 0x7f0bb419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bbc448640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb410edf0 con 0x7f0bb41033e0 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bbc448640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb410ec70 con 0x7f0bb4102ac0 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bb99e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb41033e0 0x7f0bb419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bb99e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb41033e0 0x7f0bb419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46656/0 (socket says 172.21.15.46:46656) 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bb99e3640 1 -- 172.21.15.46:0/4218671114 learned_addr learned my addr 172.21.15.46:0/4218671114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bba1e4640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb4102ac0 0x7f0bb419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bb99e3640 1 -- 172.21.15.46:0/4218671114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb4102ac0 msgr2=0x7f0bb419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bb99e3640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb4102ac0 0x7f0bb419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bb99e3640 1 -- 172.21.15.46:0/4218671114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb41977e0 con 0x7f0bb41033e0 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bb99e3640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb41033e0 0x7f0bb419dba0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e970 tx=0x7f0ba400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bab7fe640 1 -- 172.21.15.46:0/4218671114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ba400cac0 con 0x7f0bb41033e0 2024-09-10T04:36:19.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb4197ad0 con 0x7f0bb41033e0 2024-09-10T04:36:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bab7fe640 1 -- 172.21.15.46:0/4218671114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0ba40054e0 con 0x7f0bb41033e0 2024-09-10T04:36:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.446+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb4198060 con 0x7f0bb41033e0 2024-09-10T04:36:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.447+0000 7f0bab7fe640 1 -- 172.21.15.46:0/4218671114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0ba4004c10 con 0x7f0bb41033e0 2024-09-10T04:36:19.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.447+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb406aa90 con 0x7f0bb41033e0 2024-09-10T04:36:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.448+0000 7f0bab7fe640 1 -- 172.21.15.46:0/4218671114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0ba4004db0 con 0x7f0bb41033e0 2024-09-10T04:36:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.449+0000 7f0bab7fe640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ba007ac00 0x7f0ba007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.449+0000 7f0bab7fe640 1 -- 172.21.15.46:0/4218671114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0ba409e190 con 0x7f0bb41033e0 2024-09-10T04:36:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.449+0000 7f0bba1e4640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ba007ac00 0x7f0ba007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.449+0000 7f0bba1e4640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ba007ac00 0x7f0ba007d0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00096c0 tx=0x7f0bb0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:19.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.453+0000 7f0bab7fe640 1 -- 172.21.15.46:0/4218671114 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0ba40698d0 con 0x7f0bb41033e0 2024-09-10T04:36:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.676+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f0bb4105f90 con 0x7f0bb41033e0 2024-09-10T04:36:19.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.676+0000 7f0bab7fe640 1 -- 172.21.15.46:0/4218671114 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f0ba406e780 con 0x7f0bb41033e0 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:36:19.679 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:36:19.680 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:36:19.681 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:19.682 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:19.682 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:36:19.682 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:19.682 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:19.682 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:19.682 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:19.682 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.678+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ba007ac00 msgr2=0x7f0ba007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.678+0000 7f0bbc448640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ba007ac00 0x7f0ba007d0c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00096c0 tx=0x7f0bb0031400 comp rx=0 tx=0).stop 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.678+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb41033e0 msgr2=0x7f0bb419dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.678+0000 7f0bbc448640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb41033e0 0x7f0bb419dba0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0ba400e970 tx=0x7f0ba400ee30 comp rx=0 tx=0).stop 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.678+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 shutdown_connections 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.679+0000 7f0bbc448640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0ba007ac00 0x7f0ba007d0c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.679+0000 7f0bbc448640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb41033e0 0x7f0bb419dba0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.679+0000 7f0bbc448640 1 --2- 172.21.15.46:0/4218671114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0bb4102ac0 0x7f0bb419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.679+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 >> 172.21.15.46:0/4218671114 conn(0x7f0bb40fc520 msgr2=0x7f0bb4108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:19.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.679+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 shutdown_connections 2024-09-10T04:36:19.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.679+0000 7f0bbc448640 1 -- 172.21.15.46:0/4218671114 wait complete. 2024-09-10T04:36:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 -- 172.21.15.46:0/2450698106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 msgr2=0x7f72fc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 --2- 172.21.15.46:0/2450698106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc107ce0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009880 tx=0x7f72ec02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 -- 172.21.15.46:0/2450698106 shutdown_connections 2024-09-10T04:36:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 --2- 172.21.15.46:0/2450698106 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72fc101900 0x7f72fc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 --2- 172.21.15.46:0/2450698106 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc107ce0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 -- 172.21.15.46:0/2450698106 >> 172.21.15.46:0/2450698106 conn(0x7f72fc0fd600 msgr2=0x7f72fc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 -- 172.21.15.46:0/2450698106 shutdown_connections 2024-09-10T04:36:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.764+0000 7f7301e40640 1 -- 172.21.15.46:0/2450698106 wait complete. 2024-09-10T04:36:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.765+0000 7f7301e40640 1 Processor -- start 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.765+0000 7f7301e40640 1 -- start start 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.765+0000 7f7301e40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72fc101900 0x7f72fc19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.765+0000 7f7301e40640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.765+0000 7f7301e40640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc10edb0 con 0x7f72fc107900 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.765+0000 7f7301e40640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc10ec30 con 0x7f72fc101900 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72faffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72faffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc19dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46676/0 (socket says 172.21.15.46:46676) 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72faffd640 1 -- 172.21.15.46:0/973359341 learned_addr learned my addr 172.21.15.46:0/973359341 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72fb7fe640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72fc101900 0x7f72fc19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72faffd640 1 -- 172.21.15.46:0/973359341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72fc101900 msgr2=0x7f72fc19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72faffd640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72fc101900 0x7f72fc19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72faffd640 1 -- 172.21.15.46:0/973359341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc1977e0 con 0x7f72fc107900 2024-09-10T04:36:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72faffd640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc19dba0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d8d0 tx=0x7f72f000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72f8ff9640 1 -- 172.21.15.46:0/973359341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f72f0017030 con 0x7f72fc107900 2024-09-10T04:36:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc197ad0 con 0x7f72fc107900 2024-09-10T04:36:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72f8ff9640 1 -- 172.21.15.46:0/973359341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f72f0004020 con 0x7f72fc107900 2024-09-10T04:36:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f72f8ff9640 1 -- 172.21.15.46:0/973359341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f72f00047a0 con 0x7f72fc107900 2024-09-10T04:36:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.766+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc198060 con 0x7f72fc107900 2024-09-10T04:36:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.767+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc103040 con 0x7f72fc107900 2024-09-10T04:36:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.768+0000 7f72f8ff9640 1 -- 172.21.15.46:0/973359341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f72f00041d0 con 0x7f72fc107900 2024-09-10T04:36:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.768+0000 7f72f8ff9640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72e407ac00 0x7f72e407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.769+0000 7f72f8ff9640 1 -- 172.21.15.46:0/973359341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f72f0018070 con 0x7f72fc107900 2024-09-10T04:36:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.769+0000 7f72fb7fe640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72e407ac00 0x7f72e407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.769+0000 7f72fb7fe640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72e407ac00 0x7f72e407d0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0096c0 tx=0x7f72ec031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.773+0000 7f72f8ff9640 1 -- 172.21.15.46:0/973359341 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f72f0069110 con 0x7f72fc107900 2024-09-10T04:36:19.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.975+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f72fc198370 con 0x7f72e407ac00 2024-09-10T04:36:19.977 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:19 smithi046 ceph-mon[109506]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:19.977 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:19 smithi046 ceph-mon[109506]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:19.977 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:19 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/578110750' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.976+0000 7f72f8ff9640 1 -- 172.21.15.46:0/973359341 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f72fc198370 con 0x7f72e407ac00 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:36:19.978 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:36:19.979 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72e407ac00 msgr2=0x7f72e407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72e407ac00 0x7f72e407d0c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0096c0 tx=0x7f72ec031400 comp rx=0 tx=0).stop 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 msgr2=0x7f72fc19dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc19dba0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d8d0 tx=0x7f72f000dd90 comp rx=0 tx=0).stop 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 shutdown_connections 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f72e407ac00 0x7f72e407d0c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72fc107900 0x7f72fc19dba0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 --2- 172.21.15.46:0/973359341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72fc101900 0x7f72fc19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.978+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 >> 172.21.15.46:0/973359341 conn(0x7f72fc0fd600 msgr2=0x7f72fc0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.979+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 shutdown_connections 2024-09-10T04:36:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:19.979+0000 7f7301e40640 1 -- 172.21.15.46:0/973359341 wait complete. 2024-09-10T04:36:20.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 -- 172.21.15.46:0/4115696804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700106800 msgr2=0x7f9700106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:20.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 --2- 172.21.15.46:0/4115696804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700106800 0x7f9700106be0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009880 tx=0x7f96f002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 -- 172.21.15.46:0/4115696804 shutdown_connections 2024-09-10T04:36:20.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 --2- 172.21.15.46:0/4115696804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9700100800 0x7f9700100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 --2- 172.21.15.46:0/4115696804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700106800 0x7f9700106be0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 -- 172.21.15.46:0/4115696804 >> 172.21.15.46:0/4115696804 conn(0x7f97000fc520 msgr2=0x7f97000fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:20.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 -- 172.21.15.46:0/4115696804 shutdown_connections 2024-09-10T04:36:20.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.064+0000 7f97048a1640 1 -- 172.21.15.46:0/4115696804 wait complete. 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f97048a1640 1 Processor -- start 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f97048a1640 1 -- start start 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f97048a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100800 0x7f9700198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f97048a1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9700106800 0x7f9700198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f97048a1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f970010dcb0 con 0x7f9700100800 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f97048a1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f970010db30 con 0x7f9700106800 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100800 0x7f9700198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:20.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f96fe59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100800 0x7f9700198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46704/0 (socket says 172.21.15.46:46704) 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.065+0000 7f96fe59c640 1 -- 172.21.15.46:0/760748798 learned_addr learned my addr 172.21.15.46:0/760748798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96f7fff640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9700106800 0x7f9700198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96fe59c640 1 -- 172.21.15.46:0/760748798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9700106800 msgr2=0x7f9700198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96fe59c640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9700106800 0x7f9700198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96fe59c640 1 -- 172.21.15.46:0/760748798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9700199350 con 0x7f9700100800 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96fe59c640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100800 0x7f9700198810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f96f0031460 tx=0x7f96f00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96f77fe640 1 -- 172.21.15.46:0/760748798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f96f003f070 con 0x7f9700100800 2024-09-10T04:36:20.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97001995e0 con 0x7f9700100800 2024-09-10T04:36:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96f77fe640 1 -- 172.21.15.46:0/760748798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96f00035f0 con 0x7f9700100800 2024-09-10T04:36:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f96f77fe640 1 -- 172.21.15.46:0/760748798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f96f0035030 con 0x7f9700100800 2024-09-10T04:36:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.066+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001a1b90 con 0x7f9700100800 2024-09-10T04:36:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.067+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9700101f40 con 0x7f9700100800 2024-09-10T04:36:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.068+0000 7f96f77fe640 1 -- 172.21.15.46:0/760748798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f96f0042070 con 0x7f9700100800 2024-09-10T04:36:20.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.069+0000 7f96f77fe640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f96ec07abf0 0x7f96ec07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:20.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.069+0000 7f96f77fe640 1 -- 172.21.15.46:0/760748798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f96f00be720 con 0x7f9700100800 2024-09-10T04:36:20.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.069+0000 7f96f7fff640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f96ec07abf0 0x7f96ec07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:20.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.069+0000 7f96f7fff640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f96ec07abf0 0x7f96ec07d0b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f96e8005fd0 tx=0x7f96e8005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:20.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.073+0000 7f96f77fe640 1 -- 172.21.15.46:0/760748798 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f96f0089dc0 con 0x7f9700100800 2024-09-10T04:36:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:19 smithi070 ceph-mon[79774]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:19 smithi070 ceph-mon[79774]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:19 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/578110750' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:36:20.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.334+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9700103a10 con 0x7f9700100800 2024-09-10T04:36:20.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.334+0000 7f96f77fe640 1 -- 172.21.15.46:0/760748798 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f96f008ec70 con 0x7f9700100800 2024-09-10T04:36:20.336 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:36:20.337 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:36:20.338 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:36:20.339 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:36:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f96ec07abf0 msgr2=0x7f96ec07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f96ec07abf0 0x7f96ec07d0b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f96e8005fd0 tx=0x7f96e8005850 comp rx=0 tx=0).stop 2024-09-10T04:36:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100800 msgr2=0x7f9700198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100800 0x7f9700198810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f96f0031460 tx=0x7f96f00045e0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 shutdown_connections 2024-09-10T04:36:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f96ec07abf0 0x7f96ec07d0b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9700106800 0x7f9700198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 --2- 172.21.15.46:0/760748798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9700100800 0x7f9700198810 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.336+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 >> 172.21.15.46:0/760748798 conn(0x7f97000fc520 msgr2=0x7f97000fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.337+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 shutdown_connections 2024-09-10T04:36:20.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:20.337+0000 7f97048a1640 1 -- 172.21.15.46:0/760748798 wait complete. 2024-09-10T04:36:20.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:20 smithi070 ceph-mon[79774]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:20.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:20 smithi070 ceph-mon[79774]: pgmap v921: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:20.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:20 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4218671114' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:36:20.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:20 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/760748798' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:36:21.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:20 smithi046 ceph-mon[109506]: from='client.35522 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:21.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:20 smithi046 ceph-mon[109506]: pgmap v921: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:21.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:20 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4218671114' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:36:21.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:20 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/760748798' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:36:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:21 smithi070 ceph-mon[79774]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:21 smithi046 ceph-mon[109506]: from='client.35534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:22 smithi070 ceph-mon[79774]: pgmap v922: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:22 smithi046 ceph-mon[109506]: pgmap v922: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:24 smithi070 ceph-mon[79774]: pgmap v923: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:24 smithi046 ceph-mon[109506]: pgmap v923: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:26 smithi070 ceph-mon[79774]: pgmap v924: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:26 smithi046 ceph-mon[109506]: pgmap v924: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:28 smithi070 ceph-mon[79774]: pgmap v925: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:28 smithi046 ceph-mon[109506]: pgmap v925: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:30 smithi070 ceph-mon[79774]: pgmap v926: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:30 smithi046 ceph-mon[109506]: pgmap v926: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:32 smithi070 ceph-mon[79774]: pgmap v927: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:32 smithi046 ceph-mon[109506]: pgmap v927: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:34 smithi070 ceph-mon[79774]: pgmap v928: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:34 smithi046 ceph-mon[109506]: pgmap v928: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:36 smithi070 ceph-mon[79774]: pgmap v929: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:36 smithi046 ceph-mon[109506]: pgmap v929: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:39.005 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:38 smithi070 ceph-mon[79774]: pgmap v930: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:39.006 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:38 smithi046 ceph-mon[109506]: pgmap v930: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:40 smithi070 ceph-mon[79774]: pgmap v931: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:40 smithi046 ceph-mon[109506]: pgmap v931: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:43.022 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:42 smithi046 ceph-mon[109506]: pgmap v932: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:42 smithi070 ceph-mon[79774]: pgmap v932: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:44 smithi070 ceph-mon[79774]: pgmap v933: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:44 smithi046 ceph-mon[109506]: pgmap v933: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:46 smithi070 ceph-mon[79774]: pgmap v934: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:46 smithi046 ceph-mon[109506]: pgmap v934: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:48 smithi070 ceph-mon[79774]: pgmap v935: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:48 smithi046 ceph-mon[109506]: pgmap v935: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:50.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 -- 172.21.15.46:0/2677846626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 msgr2=0x7f0b1c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2677846626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c10b8d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009880 tx=0x7f0b0c02a530 comp rx=0 tx=0).stop 2024-09-10T04:36:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 -- 172.21.15.46:0/2677846626 shutdown_connections 2024-09-10T04:36:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2677846626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c10b8d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2677846626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b1c102ac0 0x7f0b1c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 -- 172.21.15.46:0/2677846626 >> 172.21.15.46:0/2677846626 conn(0x7f0b1c0fc520 msgr2=0x7f0b1c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 -- 172.21.15.46:0/2677846626 shutdown_connections 2024-09-10T04:36:50.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.434+0000 7f0b239b9640 1 -- 172.21.15.46:0/2677846626 wait complete. 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b239b9640 1 Processor -- start 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b239b9640 1 -- start start 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b239b9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b1c102ac0 0x7f0b1c19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b239b9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b239b9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c10edf0 con 0x7f0b1c1033e0 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b239b9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c10ec70 con 0x7f0b1c102ac0 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b20f54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:50.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.435+0000 7f0b20f54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60628/0 (socket says 172.21.15.46:60628) 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b20f54640 1 -- 172.21.15.46:0/2922288620 learned_addr learned my addr 172.21.15.46:0/2922288620 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b21755640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b1c102ac0 0x7f0b1c19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b20f54640 1 -- 172.21.15.46:0/2922288620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b1c102ac0 msgr2=0x7f0b1c19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b20f54640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b1c102ac0 0x7f0b1c19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b20f54640 1 -- 172.21.15.46:0/2922288620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c199ab0 con 0x7f0b1c1033e0 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b20f54640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c19fe30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c031460 tx=0x7f0b0c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b127fc640 1 -- 172.21.15.46:0/2922288620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b0c03f070 con 0x7f0b1c1033e0 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b127fc640 1 -- 172.21.15.46:0/2922288620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b0c0035f0 con 0x7f0b1c1033e0 2024-09-10T04:36:50.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c199d40 con 0x7f0b1c1033e0 2024-09-10T04:36:50.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b127fc640 1 -- 172.21.15.46:0/2922288620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0b0c035030 con 0x7f0b1c1033e0 2024-09-10T04:36:50.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.436+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c19a270 con 0x7f0b1c1033e0 2024-09-10T04:36:50.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.437+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1c06aa90 con 0x7f0b1c1033e0 2024-09-10T04:36:50.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.438+0000 7f0b127fc640 1 -- 172.21.15.46:0/2922288620 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0b0c042070 con 0x7f0b1c1033e0 2024-09-10T04:36:50.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.439+0000 7f0b127fc640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0b0807ac40 0x7f0b0807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:50.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.439+0000 7f0b21755640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0b0807ac40 0x7f0b0807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:50.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.439+0000 7f0b127fc640 1 -- 172.21.15.46:0/2922288620 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0b0c0be3a0 con 0x7f0b1c1033e0 2024-09-10T04:36:50.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.439+0000 7f0b21755640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0b0807ac40 0x7f0b0807d100 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c19abf0 tx=0x7f0b18005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:50.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.444+0000 7f0b127fc640 1 -- 172.21.15.46:0/2922288620 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0b0c089b60 con 0x7f0b1c1033e0 2024-09-10T04:36:50.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.646+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0b1c10b3e0 con 0x7f0b0807ac40 2024-09-10T04:36:50.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.648+0000 7f0b127fc640 1 -- 172.21.15.46:0/2922288620 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f0b1801b3f0 con 0x7f0b0807ac40 2024-09-10T04:36:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0b0807ac40 msgr2=0x7f0b0807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0b0807ac40 0x7f0b0807d100 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c19abf0 tx=0x7f0b18005f90 comp rx=0 tx=0).stop 2024-09-10T04:36:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 msgr2=0x7f0b1c19fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c19fe30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c031460 tx=0x7f0b0c0045c0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 shutdown_connections 2024-09-10T04:36:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0b0807ac40 0x7f0b0807d100 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b1c1033e0 0x7f0b1c19fe30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 --2- 172.21.15.46:0/2922288620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b1c102ac0 0x7f0b1c19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 >> 172.21.15.46:0/2922288620 conn(0x7f0b1c0fc520 msgr2=0x7f0b1c100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 shutdown_connections 2024-09-10T04:36:50.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.650+0000 7f0b239b9640 1 -- 172.21.15.46:0/2922288620 wait complete. 2024-09-10T04:36:50.661 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.736+0000 7f8cc2a45640 1 -- 172.21.15.46:0/3337770797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 msgr2=0x7f8cbc102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.736+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/3337770797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc102ea0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009880 tx=0x7f8cac02a530 comp rx=0 tx=0).stop 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 -- 172.21.15.46:0/3337770797 shutdown_connections 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/3337770797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cbc1033e0 0x7f8cbc10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/3337770797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc102ea0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 -- 172.21.15.46:0/3337770797 >> 172.21.15.46:0/3337770797 conn(0x7f8cbc0fc520 msgr2=0x7f8cbc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 -- 172.21.15.46:0/3337770797 shutdown_connections 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 -- 172.21.15.46:0/3337770797 wait complete. 2024-09-10T04:36:50.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 Processor -- start 2024-09-10T04:36:50.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.737+0000 7f8cc2a45640 1 -- start start 2024-09-10T04:36:50.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cc2a45640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:50.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cc2a45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cbc1033e0 0x7f8cbc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:50.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cc2a45640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc10edf0 con 0x7f8cbc102ac0 2024-09-10T04:36:50.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cc2a45640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc10ec70 con 0x7f8cbc1033e0 2024-09-10T04:36:50.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:50.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60640/0 (socket says 172.21.15.46:60640) 2024-09-10T04:36:50.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbbfff640 1 -- 172.21.15.46:0/936519462 learned_addr learned my addr 172.21.15.46:0/936519462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbb7fe640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cbc1033e0 0x7f8cbc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbbfff640 1 -- 172.21.15.46:0/936519462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cbc1033e0 msgr2=0x7f8cbc19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbbfff640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cbc1033e0 0x7f8cbc19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbbfff640 1 -- 172.21.15.46:0/936519462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc1977c0 con 0x7f8cbc102ac0 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbb7fe640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cbc1033e0 0x7f8cbc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cbbfff640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc19d640 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8cac031460 tx=0x7f8cac0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cb97fa640 1 -- 172.21.15.46:0/936519462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cac03f070 con 0x7f8cbc102ac0 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc197a50 con 0x7f8cbc102ac0 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cb97fa640 1 -- 172.21.15.46:0/936519462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cac0035f0 con 0x7f8cbc102ac0 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.738+0000 7f8cb97fa640 1 -- 172.21.15.46:0/936519462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8cac035030 con 0x7f8cbc102ac0 2024-09-10T04:36:50.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.739+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc197f80 con 0x7f8cbc102ac0 2024-09-10T04:36:50.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.740+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cbc06aa90 con 0x7f8cbc102ac0 2024-09-10T04:36:50.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.740+0000 7f8cb97fa640 1 -- 172.21.15.46:0/936519462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8cac042070 con 0x7f8cbc102ac0 2024-09-10T04:36:50.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.741+0000 7f8cb97fa640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca407ac40 0x7f8ca407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:50.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.741+0000 7f8cb97fa640 1 -- 172.21.15.46:0/936519462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8cac0be5a0 con 0x7f8cbc102ac0 2024-09-10T04:36:50.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.741+0000 7f8cbb7fe640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca407ac40 0x7f8ca407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:50.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.741+0000 7f8cbb7fe640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca407ac40 0x7f8ca407d100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005fd0 tx=0x7f8cb0005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:50.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.745+0000 7f8cb97fa640 1 -- 172.21.15.46:0/936519462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8cac089d60 con 0x7f8cbc102ac0 2024-09-10T04:36:50.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.948+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8cbc198230 con 0x7f8ca407ac40 2024-09-10T04:36:50.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:50 smithi046 ceph-mon[109506]: pgmap v936: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:50.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.951+0000 7f8cb97fa640 1 -- 172.21.15.46:0/936519462 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8cbc198230 con 0x7f8ca407ac40 2024-09-10T04:36:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.952+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca407ac40 msgr2=0x7f8ca407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca407ac40 0x7f8ca407d100 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0005fd0 tx=0x7f8cb0005850 comp rx=0 tx=0).stop 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 msgr2=0x7f8cbc19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc19d640 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8cac031460 tx=0x7f8cac0045e0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 shutdown_connections 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8ca407ac40 0x7f8ca407d100 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8cbc1033e0 0x7f8cbc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 --2- 172.21.15.46:0/936519462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cbc102ac0 0x7f8cbc19d640 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 >> 172.21.15.46:0/936519462 conn(0x7f8cbc0fc520 msgr2=0x7f8cbc10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 shutdown_connections 2024-09-10T04:36:50.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:50.953+0000 7f8cc2a45640 1 -- 172.21.15.46:0/936519462 wait complete. 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.037+0000 7fc5eb179640 1 -- 172.21.15.46:0/2279738673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 msgr2=0x7fc5e410b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.037+0000 7fc5eb179640 1 --2- 172.21.15.46:0/2279738673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e410b990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009880 tx=0x7fc5d802a530 comp rx=0 tx=0).stop 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.037+0000 7fc5eb179640 1 -- 172.21.15.46:0/2279738673 shutdown_connections 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.037+0000 7fc5eb179640 1 --2- 172.21.15.46:0/2279738673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e410b990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.037+0000 7fc5eb179640 1 --2- 172.21.15.46:0/2279738673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e4102b80 0x7fc5e4102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.037+0000 7fc5eb179640 1 -- 172.21.15.46:0/2279738673 >> 172.21.15.46:0/2279738673 conn(0x7fc5e40fc520 msgr2=0x7fc5e40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 -- 172.21.15.46:0/2279738673 shutdown_connections 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 -- 172.21.15.46:0/2279738673 wait complete. 2024-09-10T04:36:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 Processor -- start 2024-09-10T04:36:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 -- start start 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e4102b80 0x7fc5e419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410edd0 con 0x7fc5e41034a0 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.038+0000 7fc5eb179640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc5e410ec50 con 0x7fc5e4102b80 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60656/0 (socket says 172.21.15.46:60656) 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e3fff640 1 -- 172.21.15.46:0/4196543808 learned_addr learned my addr 172.21.15.46:0/4196543808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e8f15640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e4102b80 0x7fc5e419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e3fff640 1 -- 172.21.15.46:0/4196543808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e4102b80 msgr2=0x7fc5e419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e3fff640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e4102b80 0x7fc5e419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e3fff640 1 -- 172.21.15.46:0/4196543808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5e41977c0 con 0x7fc5e41034a0 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e3fff640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e419db80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8004f90 tx=0x7fc5d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e1ffb640 1 -- 172.21.15.46:0/4196543808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5d803f070 con 0x7fc5e41034a0 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e1ffb640 1 -- 172.21.15.46:0/4196543808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5d8004c70 con 0x7fc5e41034a0 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5e4197a50 con 0x7fc5e41034a0 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.039+0000 7fc5e1ffb640 1 -- 172.21.15.46:0/4196543808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5d8035030 con 0x7fc5e41034a0 2024-09-10T04:36:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.040+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5e4197f80 con 0x7fc5e41034a0 2024-09-10T04:36:51.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.041+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5e406a9d0 con 0x7fc5e41034a0 2024-09-10T04:36:51.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.041+0000 7fc5e1ffb640 1 -- 172.21.15.46:0/4196543808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc5d8042070 con 0x7fc5e41034a0 2024-09-10T04:36:51.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.042+0000 7fc5e1ffb640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5cc07abf0 0x7fc5cc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.042+0000 7fc5e1ffb640 1 -- 172.21.15.46:0/4196543808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc5d80be6f0 con 0x7fc5e41034a0 2024-09-10T04:36:51.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.042+0000 7fc5e8f15640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5cc07abf0 0x7fc5cc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.043+0000 7fc5e8f15640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5cc07abf0 0x7fc5cc07d0b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005fd0 tx=0x7fc5d4005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:51.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.047+0000 7fc5e1ffb640 1 -- 172.21.15.46:0/4196543808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc5d8089d90 con 0x7fc5e41034a0 2024-09-10T04:36:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:50 smithi070 ceph-mon[79774]: pgmap v936: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:51.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.240+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc5e41982f0 con 0x7fc5cc07abf0 2024-09-10T04:36:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.251+0000 7fc5e1ffb640 1 -- 172.21.15.46:0/4196543808 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fc5e41982f0 con 0x7fc5cc07abf0 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (28m) 7m ago 39m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (30m) 7m ago 40m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (30m) 7m ago 39m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (26m) 7m ago 39m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (26m) 7m ago 39m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (28m) 7m ago 39m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (23m) 7m ago 34m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:36:51.253 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (23m) 7m ago 34m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (22m) 7m ago 34m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (7m) 7m ago 34m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (30m) 7m ago 41m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (29m) 7m ago 38m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (27m) 7m ago 41m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (27m) 7m ago 38m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (29m) 7m ago 39m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (29m) 7m ago 39m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (26m) 7m ago 37m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (25m) 7m ago 37m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (25m) 7m ago 36m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:36:51.254 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (24m) 7m ago 36m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:36:51.255 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (24m) 7m ago 35m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:36:51.255 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (23m) 7m ago 35m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:36:51.255 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (28m) 7m ago 39m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:36:51.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5cc07abf0 msgr2=0x7fc5cc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5cc07abf0 0x7fc5cc07d0b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4005fd0 tx=0x7fc5d4005850 comp rx=0 tx=0).stop 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 msgr2=0x7fc5e419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e419db80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8004f90 tx=0x7fc5d80048f0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 shutdown_connections 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5cc07abf0 0x7fc5cc07d0b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5e41034a0 0x7fc5e419db80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 --2- 172.21.15.46:0/4196543808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5e4102b80 0x7fc5e419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 >> 172.21.15.46:0/4196543808 conn(0x7fc5e40fc520 msgr2=0x7fc5e4103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 shutdown_connections 2024-09-10T04:36:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.253+0000 7fc5eb179640 1 -- 172.21.15.46:0/4196543808 wait complete. 2024-09-10T04:36:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.336+0000 7fc604779640 1 -- 172.21.15.46:0/2966239896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 msgr2=0x7fc5fc102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.336+0000 7fc604779640 1 --2- 172.21.15.46:0/2966239896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc102e90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009880 tx=0x7fc5f802a530 comp rx=0 tx=0).stop 2024-09-10T04:36:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.336+0000 7fc604779640 1 -- 172.21.15.46:0/2966239896 shutdown_connections 2024-09-10T04:36:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.336+0000 7fc604779640 1 --2- 172.21.15.46:0/2966239896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc1033d0 0x7fc5fc10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.336+0000 7fc604779640 1 --2- 172.21.15.46:0/2966239896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc102e90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.336+0000 7fc604779640 1 -- 172.21.15.46:0/2966239896 >> 172.21.15.46:0/2966239896 conn(0x7fc5fc0fc550 msgr2=0x7fc5fc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 -- 172.21.15.46:0/2966239896 shutdown_connections 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 -- 172.21.15.46:0/2966239896 wait complete. 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 Processor -- start 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 -- start start 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc1033d0 0x7fc5fc19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc10ee40 con 0x7fc5fc102ab0 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc604779640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc5fc10ecc0 con 0x7fc5fc1033d0 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.337+0000 7fc602515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc602515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60678/0 (socket says 172.21.15.46:60678) 2024-09-10T04:36:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc602515640 1 -- 172.21.15.46:0/4273334550 learned_addr learned my addr 172.21.15.46:0/4273334550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc601d14640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc1033d0 0x7fc5fc19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc602515640 1 -- 172.21.15.46:0/4273334550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc1033d0 msgr2=0x7fc5fc19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc602515640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc1033d0 0x7fc5fc19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc602515640 1 -- 172.21.15.46:0/4273334550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5fc197810 con 0x7fc5fc102ab0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc602515640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc19d690 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8031460 tx=0x7fc5f80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc5f37fe640 1 -- 172.21.15.46:0/4273334550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5f803f070 con 0x7fc5fc102ab0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5fc197aa0 con 0x7fc5fc102ab0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc5f37fe640 1 -- 172.21.15.46:0/4273334550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc5f8004cf0 con 0x7fc5fc102ab0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc5f37fe640 1 -- 172.21.15.46:0/4273334550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc5f8035030 con 0x7fc5fc102ab0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.338+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5fc197fd0 con 0x7fc5fc102ab0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.339+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5fc06a580 con 0x7fc5fc102ab0 2024-09-10T04:36:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.340+0000 7fc5f37fe640 1 -- 172.21.15.46:0/4273334550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc5f8042070 con 0x7fc5fc102ab0 2024-09-10T04:36:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.340+0000 7fc5f37fe640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5e807ac40 0x7fc5e807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.341+0000 7fc5f37fe640 1 -- 172.21.15.46:0/4273334550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc5f80be6d0 con 0x7fc5fc102ab0 2024-09-10T04:36:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.341+0000 7fc601d14640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5e807ac40 0x7fc5e807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.341+0000 7fc601d14640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5e807ac40 0x7fc5e807d100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0feff0 tx=0x7fc5ec008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.345+0000 7fc5f37fe640 1 -- 172.21.15.46:0/4273334550 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc5f8089e90 con 0x7fc5fc102ab0 2024-09-10T04:36:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.604+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc5fc105f80 con 0x7fc5fc102ab0 2024-09-10T04:36:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.604+0000 7fc5f37fe640 1 -- 172.21.15.46:0/4273334550 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fc5f808ed40 con 0x7fc5fc102ab0 2024-09-10T04:36:51.606 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:36:51.606 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:36:51.607 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:36:51.608 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:36:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5e807ac40 msgr2=0x7fc5e807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5e807ac40 0x7fc5e807d100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0feff0 tx=0x7fc5ec008040 comp rx=0 tx=0).stop 2024-09-10T04:36:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 msgr2=0x7fc5fc19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc19d690 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8031460 tx=0x7fc5f80048f0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 shutdown_connections 2024-09-10T04:36:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc5e807ac40 0x7fc5e807d100 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5fc1033d0 0x7fc5fc19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 --2- 172.21.15.46:0/4273334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5fc102ab0 0x7fc5fc19d690 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 >> 172.21.15.46:0/4273334550 conn(0x7fc5fc0fc550 msgr2=0x7fc5fc108e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 shutdown_connections 2024-09-10T04:36:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.606+0000 7fc604779640 1 -- 172.21.15.46:0/4273334550 wait complete. 2024-09-10T04:36:51.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/3031648326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 msgr2=0x7ff5a4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/3031648326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a4101d60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff594009880 tx=0x7ff59402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/3031648326 shutdown_connections 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/3031648326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a4101d60 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/3031648326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4107900 0x7ff5a4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/3031648326 >> 172.21.15.46:0/3031648326 conn(0x7ff5a40fd600 msgr2=0x7ff5a40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/3031648326 shutdown_connections 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.690+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/3031648326 wait complete. 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5ac2ca640 1 Processor -- start 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5ac2ca640 1 -- start start 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5ac2ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5ac2ca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4107900 0x7ff5a4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5ac2ca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a410edb0 con 0x7ff5a4101900 2024-09-10T04:36:51.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5ac2ca640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a410ec30 con 0x7ff5a4107900 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5aa066640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5aa066640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60682/0 (socket says 172.21.15.46:60682) 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5aa066640 1 -- 172.21.15.46:0/1750592812 learned_addr learned my addr 172.21.15.46:0/1750592812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.691+0000 7ff5a9865640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4107900 0x7ff5a4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff5aa066640 1 -- 172.21.15.46:0/1750592812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4107900 msgr2=0x7ff5a4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff5aa066640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4107900 0x7ff5a4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff5aa066640 1 -- 172.21.15.46:0/1750592812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a4074f70 con 0x7ff5a4101900 2024-09-10T04:36:51.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff5aa066640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a40744f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000d8d0 tx=0x7ff5a000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:51.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff59b7fe640 1 -- 172.21.15.46:0/1750592812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff5a0017070 con 0x7ff5a4101900 2024-09-10T04:36:51.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff59b7fe640 1 -- 172.21.15.46:0/1750592812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5a0010460 con 0x7ff5a4101900 2024-09-10T04:36:51.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff59b7fe640 1 -- 172.21.15.46:0/1750592812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff5a000f5f0 con 0x7ff5a4101900 2024-09-10T04:36:51.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a4075260 con 0x7ff5a4101900 2024-09-10T04:36:51.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.692+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41a3e60 con 0x7ff5a4101900 2024-09-10T04:36:51.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.693+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a4103040 con 0x7ff5a4101900 2024-09-10T04:36:51.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.694+0000 7ff59b7fe640 1 -- 172.21.15.46:0/1750592812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff5a0009a30 con 0x7ff5a4101900 2024-09-10T04:36:51.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.694+0000 7ff59b7fe640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff59007ac40 0x7ff59007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:51.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.694+0000 7ff59b7fe640 1 -- 172.21.15.46:0/1750592812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff5a009d9e0 con 0x7ff5a4101900 2024-09-10T04:36:51.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.694+0000 7ff5a9865640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff59007ac40 0x7ff59007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:51.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.695+0000 7ff5a9865640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff59007ac40 0x7ff59007d100 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4075f80 tx=0x7ff594031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:51.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.699+0000 7ff59b7fe640 1 -- 172.21.15.46:0/1750592812 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff5a00691a0 con 0x7ff5a4101900 2024-09-10T04:36:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.922+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7ff5a4109e90 con 0x7ff5a4101900 2024-09-10T04:36:51.924 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:51 smithi046 ceph-mon[109506]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:51.924 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:51 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4273334550' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:36:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.923+0000 7ff59b7fe640 1 -- 172.21.15.46:0/1750592812 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7ff5a006e050 con 0x7ff5a4101900 2024-09-10T04:36:51.925 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:36:51.925 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:36:51.925 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:36:51.925 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:36:51.925 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:36:51.925 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:36:51.926 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:36:51.927 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:51.928 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:36:51.929 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:36:51.929 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:51.929 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:51.929 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:51.929 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff59007ac40 msgr2=0x7ff59007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff59007ac40 0x7ff59007d100 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4075f80 tx=0x7ff594031400 comp rx=0 tx=0).stop 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 msgr2=0x7ff5a40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a40744f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff5a000d8d0 tx=0x7ff5a000dd90 comp rx=0 tx=0).stop 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 shutdown_connections 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff59007ac40 0x7ff59007d100 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a4107900 0x7ff5a4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 --2- 172.21.15.46:0/1750592812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a4101900 0x7ff5a40744f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 >> 172.21.15.46:0/1750592812 conn(0x7ff5a40fd600 msgr2=0x7ff5a40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 shutdown_connections 2024-09-10T04:36:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:51.925+0000 7ff5ac2ca640 1 -- 172.21.15.46:0/1750592812 wait complete. 2024-09-10T04:36:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.009+0000 7fe46aea4640 1 -- 172.21.15.46:0/791715912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe464102ac0 msgr2=0x7fe464102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.009+0000 7fe46aea4640 1 --2- 172.21.15.46:0/791715912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe464102ac0 0x7fe464102ea0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe454009880 tx=0x7fe45402a530 comp rx=0 tx=0).stop 2024-09-10T04:36:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.009+0000 7fe46aea4640 1 -- 172.21.15.46:0/791715912 shutdown_connections 2024-09-10T04:36:52.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.009+0000 7fe46aea4640 1 --2- 172.21.15.46:0/791715912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4641033e0 0x7fe46410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.009+0000 7fe46aea4640 1 --2- 172.21.15.46:0/791715912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe464102ac0 0x7fe464102ea0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.009+0000 7fe46aea4640 1 -- 172.21.15.46:0/791715912 >> 172.21.15.46:0/791715912 conn(0x7fe4640fc520 msgr2=0x7fe4640fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.009+0000 7fe46aea4640 1 -- 172.21.15.46:0/791715912 shutdown_connections 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe46aea4640 1 -- 172.21.15.46:0/791715912 wait complete. 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe46aea4640 1 Processor -- start 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe46aea4640 1 -- start start 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe46aea4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe464102ac0 0x7fe46419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe46aea4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4641033e0 0x7fe46419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe46aea4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe46410edd0 con 0x7fe4641033e0 2024-09-10T04:36:52.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe46aea4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe46410ec50 con 0x7fe464102ac0 2024-09-10T04:36:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.010+0000 7fe463fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4641033e0 0x7fe46419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe468c40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe464102ac0 0x7fe46419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe463fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4641033e0 0x7fe46419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60698/0 (socket says 172.21.15.46:60698) 2024-09-10T04:36:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe463fff640 1 -- 172.21.15.46:0/2663014248 learned_addr learned my addr 172.21.15.46:0/2663014248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe463fff640 1 -- 172.21.15.46:0/2663014248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe464102ac0 msgr2=0x7fe46419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe463fff640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe464102ac0 0x7fe46419d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe463fff640 1 -- 172.21.15.46:0/2663014248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4641978e0 con 0x7fe4641033e0 2024-09-10T04:36:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe463fff640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4641033e0 0x7fe46419dc60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe45800d6c0 tx=0x7fe45800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe461ffb640 1 -- 172.21.15.46:0/2663014248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe458017070 con 0x7fe4641033e0 2024-09-10T04:36:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe461ffb640 1 -- 172.21.15.46:0/2663014248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe458009ce0 con 0x7fe4641033e0 2024-09-10T04:36:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe464197bd0 con 0x7fe4641033e0 2024-09-10T04:36:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe461ffb640 1 -- 172.21.15.46:0/2663014248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe45800f3c0 con 0x7fe4641033e0 2024-09-10T04:36:52.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.011+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe464198160 con 0x7fe4641033e0 2024-09-10T04:36:52.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.012+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe46406aa90 con 0x7fe4641033e0 2024-09-10T04:36:52.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.013+0000 7fe461ffb640 1 -- 172.21.15.46:0/2663014248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe458010460 con 0x7fe4641033e0 2024-09-10T04:36:52.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.014+0000 7fe461ffb640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe44c07ac40 0x7fe44c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:52.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.014+0000 7fe461ffb640 1 -- 172.21.15.46:0/2663014248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe45809d8d0 con 0x7fe4641033e0 2024-09-10T04:36:52.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.014+0000 7fe468c40640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe44c07ac40 0x7fe44c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:52.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.014+0000 7fe468c40640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe44c07ac40 0x7fe44c07d100 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe454000c00 tx=0x7fe454031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:52.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.018+0000 7fe461ffb640 1 -- 172.21.15.46:0/2663014248 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe458069090 con 0x7fe4641033e0 2024-09-10T04:36:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:51 smithi070 ceph-mon[79774]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:51 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4273334550' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:36:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.222+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe464106000 con 0x7fe44c07ac40 2024-09-10T04:36:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.223+0000 7fe461ffb640 1 -- 172.21.15.46:0/2663014248 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fe454004260 con 0x7fe44c07ac40 2024-09-10T04:36:52.225 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:36:52.225 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:36:52.225 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:36:52.226 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe44c07ac40 msgr2=0x7fe44c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe44c07ac40 0x7fe44c07d100 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe454000c00 tx=0x7fe454031400 comp rx=0 tx=0).stop 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4641033e0 msgr2=0x7fe46419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4641033e0 0x7fe46419dc60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe45800d6c0 tx=0x7fe45800db80 comp rx=0 tx=0).stop 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 shutdown_connections 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe44c07ac40 0x7fe44c07d100 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4641033e0 0x7fe46419dc60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 --2- 172.21.15.46:0/2663014248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe464102ac0 0x7fe46419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 >> 172.21.15.46:0/2663014248 conn(0x7fe4640fc520 msgr2=0x7fe464108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 shutdown_connections 2024-09-10T04:36:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.227+0000 7fe46aea4640 1 -- 172.21.15.46:0/2663014248 wait complete. 2024-09-10T04:36:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 -- 172.21.15.46:0/2907812207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328107910 msgr2=0x7fe328107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2907812207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328107910 0x7fe328107cf0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009880 tx=0x7fe31c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 -- 172.21.15.46:0/2907812207 shutdown_connections 2024-09-10T04:36:52.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2907812207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe328101910 0x7fe328101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2907812207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328107910 0x7fe328107cf0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 -- 172.21.15.46:0/2907812207 >> 172.21.15.46:0/2907812207 conn(0x7fe3280fd610 msgr2=0x7fe3280ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 -- 172.21.15.46:0/2907812207 shutdown_connections 2024-09-10T04:36:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 -- 172.21.15.46:0/2907812207 wait complete. 2024-09-10T04:36:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.311+0000 7fe32eff0640 1 Processor -- start 2024-09-10T04:36:52.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32eff0640 1 -- start start 2024-09-10T04:36:52.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32eff0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328101910 0x7fe32819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:52.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32eff0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe328107910 0x7fe32819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32eff0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe32810ede0 con 0x7fe328101910 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32eff0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe32810ec60 con 0x7fe328107910 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32cd8c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328101910 0x7fe32819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32cd8c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328101910 0x7fe32819d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60728/0 (socket says 172.21.15.46:60728) 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32cd8c640 1 -- 172.21.15.46:0/2761813626 learned_addr learned my addr 172.21.15.46:0/2761813626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe327fff640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe328107910 0x7fe32819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32cd8c640 1 -- 172.21.15.46:0/2761813626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe328107910 msgr2=0x7fe32819db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32cd8c640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe328107910 0x7fe32819db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe32cd8c640 1 -- 172.21.15.46:0/2761813626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3281977d0 con 0x7fe328101910 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.312+0000 7fe327fff640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe328107910 0x7fe32819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:36:52.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.313+0000 7fe32cd8c640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328101910 0x7fe32819d650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031460 tx=0x7fe31c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.313+0000 7fe325ffb640 1 -- 172.21.15.46:0/2761813626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe31c03f070 con 0x7fe328101910 2024-09-10T04:36:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.313+0000 7fe325ffb640 1 -- 172.21.15.46:0/2761813626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe31c0035f0 con 0x7fe328101910 2024-09-10T04:36:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.313+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe328197a60 con 0x7fe328101910 2024-09-10T04:36:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.313+0000 7fe325ffb640 1 -- 172.21.15.46:0/2761813626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe31c035030 con 0x7fe328101910 2024-09-10T04:36:52.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.313+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe328197f90 con 0x7fe328101910 2024-09-10T04:36:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.314+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe328103050 con 0x7fe328101910 2024-09-10T04:36:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.315+0000 7fe325ffb640 1 -- 172.21.15.46:0/2761813626 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe31c042070 con 0x7fe328101910 2024-09-10T04:36:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.315+0000 7fe325ffb640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe31807abf0 0x7fe31807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:36:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.315+0000 7fe325ffb640 1 -- 172.21.15.46:0/2761813626 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe31c0be680 con 0x7fe328101910 2024-09-10T04:36:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.315+0000 7fe327fff640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe31807abf0 0x7fe31807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:36:52.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.316+0000 7fe327fff640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe31807abf0 0x7fe31807d0b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe3140058c0 tx=0x7fe314005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:36:52.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.320+0000 7fe325ffb640 1 -- 172.21.15.46:0/2761813626 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe31c089dc0 con 0x7fe328101910 2024-09-10T04:36:52.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.576+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fe328109e80 con 0x7fe328101910 2024-09-10T04:36:52.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.577+0000 7fe325ffb640 1 -- 172.21.15.46:0/2761813626 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fe31c08ec70 con 0x7fe328101910 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:36:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:36:52.580 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:36:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe31807abf0 msgr2=0x7fe31807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe31807abf0 0x7fe31807d0b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe3140058c0 tx=0x7fe314005850 comp rx=0 tx=0).stop 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328101910 msgr2=0x7fe32819d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328101910 0x7fe32819d650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe31c031460 tx=0x7fe31c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 shutdown_connections 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe31807abf0 0x7fe31807d0b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe328107910 0x7fe32819db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 --2- 172.21.15.46:0/2761813626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe328101910 0x7fe32819d650 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 >> 172.21.15.46:0/2761813626 conn(0x7fe3280fd610 msgr2=0x7fe3280fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 shutdown_connections 2024-09-10T04:36:52.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:36:52.580+0000 7fe32eff0640 1 -- 172.21.15.46:0/2761813626 wait complete. 2024-09-10T04:36:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:52 smithi046 ceph-mon[109506]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:52 smithi046 ceph-mon[109506]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:52 smithi046 ceph-mon[109506]: pgmap v937: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:52 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1750592812' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:36:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:52 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2761813626' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:36:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:52 smithi070 ceph-mon[79774]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:52 smithi070 ceph-mon[79774]: from='client.35550 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:52 smithi070 ceph-mon[79774]: pgmap v937: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:52 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1750592812' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:36:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:52 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2761813626' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:36:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:53 smithi046 ceph-mon[109506]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:53.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:53 smithi070 ceph-mon[79774]: from='client.35562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:36:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:36:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:54 smithi070 ceph-mon[79774]: pgmap v938: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:54 smithi046 ceph-mon[109506]: pgmap v938: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:56 smithi070 ceph-mon[79774]: pgmap v939: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:56 smithi046 ceph-mon[109506]: pgmap v939: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:36:58 smithi070 ceph-mon[79774]: pgmap v940: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:36:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:36:58 smithi046 ceph-mon[109506]: pgmap v940: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:00 smithi070 ceph-mon[79774]: pgmap v941: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:00 smithi046 ceph-mon[109506]: pgmap v941: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:02 smithi070 ceph-mon[79774]: pgmap v942: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:02 smithi046 ceph-mon[109506]: pgmap v942: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:04 smithi070 ceph-mon[79774]: pgmap v943: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:04 smithi046 ceph-mon[109506]: pgmap v943: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:06 smithi070 ceph-mon[79774]: pgmap v944: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:06 smithi046 ceph-mon[109506]: pgmap v944: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:08 smithi070 ceph-mon[79774]: pgmap v945: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:08 smithi046 ceph-mon[109506]: pgmap v945: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:10 smithi070 ceph-mon[79774]: pgmap v946: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:10 smithi046 ceph-mon[109506]: pgmap v946: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:12 smithi070 ceph-mon[79774]: pgmap v947: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:13.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:12 smithi046 ceph-mon[109506]: pgmap v947: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:14 smithi070 ceph-mon[79774]: pgmap v948: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:14 smithi046 ceph-mon[109506]: pgmap v948: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:16 smithi070 ceph-mon[79774]: pgmap v949: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:16 smithi046 ceph-mon[109506]: pgmap v949: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:18 smithi046 ceph-mon[109506]: pgmap v950: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:18 smithi070 ceph-mon[79774]: pgmap v950: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:20 smithi070 ceph-mon[79774]: pgmap v951: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:20 smithi046 ceph-mon[109506]: pgmap v951: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 -- 172.21.15.46:0/2484080320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc102b60 msgr2=0x7f16bc102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 --2- 172.21.15.46:0/2484080320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc102b60 0x7f16bc102f40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009c20 tx=0x7f16ac02aa00 comp rx=0 tx=0).stop 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 -- 172.21.15.46:0/2484080320 shutdown_connections 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 --2- 172.21.15.46:0/2484080320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc103480 0x7f16bc10b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 --2- 172.21.15.46:0/2484080320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc102b60 0x7f16bc102f40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 -- 172.21.15.46:0/2484080320 >> 172.21.15.46:0/2484080320 conn(0x7f16bc0fc520 msgr2=0x7f16bc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 -- 172.21.15.46:0/2484080320 shutdown_connections 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.680+0000 7f16c15cc640 1 -- 172.21.15.46:0/2484080320 wait complete. 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16c15cc640 1 Processor -- start 2024-09-10T04:37:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16c15cc640 1 -- start start 2024-09-10T04:37:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16c15cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc102b60 0x7f16bc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16c15cc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc103480 0x7f16bc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16c15cc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10edd0 con 0x7f16bc103480 2024-09-10T04:37:22.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16c15cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc10ec50 con 0x7f16bc102b60 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16ba7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc103480 0x7f16bc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16ba7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc103480 0x7f16bc19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47544/0 (socket says 172.21.15.46:47544) 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.681+0000 7f16ba7fc640 1 -- 172.21.15.46:0/858421951 learned_addr learned my addr 172.21.15.46:0/858421951 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16baffd640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc102b60 0x7f16bc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16ba7fc640 1 -- 172.21.15.46:0/858421951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc102b60 msgr2=0x7f16bc19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16ba7fc640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc102b60 0x7f16bc19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16ba7fc640 1 -- 172.21.15.46:0/858421951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc1977c0 con 0x7f16bc103480 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16ba7fc640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc103480 0x7f16bc19db80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f16b000d8d0 tx=0x7f16b000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16a3fff640 1 -- 172.21.15.46:0/858421951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f16b0017070 con 0x7f16bc103480 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc197ab0 con 0x7f16bc103480 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16a3fff640 1 -- 172.21.15.46:0/858421951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16b0010460 con 0x7f16bc103480 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16a3fff640 1 -- 172.21.15.46:0/858421951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f16b000f5f0 con 0x7f16bc103480 2024-09-10T04:37:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.682+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc198040 con 0x7f16bc103480 2024-09-10T04:37:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.684+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc06a9f0 con 0x7f16bc103480 2024-09-10T04:37:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.684+0000 7f16a3fff640 1 -- 172.21.15.46:0/858421951 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f16b0009a30 con 0x7f16bc103480 2024-09-10T04:37:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.685+0000 7f16a3fff640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f16a407abf0 0x7f16a407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.685+0000 7f16a3fff640 1 -- 172.21.15.46:0/858421951 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f16b009db50 con 0x7f16bc103480 2024-09-10T04:37:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.685+0000 7f16baffd640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f16a407abf0 0x7f16a407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.686+0000 7f16baffd640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f16a407abf0 0x7f16a407d0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009430 tx=0x7f16ac03a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:22.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.690+0000 7f16a3fff640 1 -- 172.21.15.46:0/858421951 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f16b00691f0 con 0x7f16bc103480 2024-09-10T04:37:22.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.893+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f16bc1982f0 con 0x7f16a407abf0 2024-09-10T04:37:22.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.894+0000 7f16a3fff640 1 -- 172.21.15.46:0/858421951 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f16bc1982f0 con 0x7f16a407abf0 2024-09-10T04:37:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.896+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f16a407abf0 msgr2=0x7f16a407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.896+0000 7f16c15cc640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f16a407abf0 0x7f16a407d0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f16ac009430 tx=0x7f16ac03a040 comp rx=0 tx=0).stop 2024-09-10T04:37:22.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.896+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc103480 msgr2=0x7f16bc19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.896+0000 7f16c15cc640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc103480 0x7f16bc19db80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f16b000d8d0 tx=0x7f16b000dd90 comp rx=0 tx=0).stop 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.896+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 shutdown_connections 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.897+0000 7f16c15cc640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f16a407abf0 0x7f16a407d0b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.897+0000 7f16c15cc640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f16bc103480 0x7f16bc19db80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.897+0000 7f16c15cc640 1 --2- 172.21.15.46:0/858421951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16bc102b60 0x7f16bc19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.897+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 >> 172.21.15.46:0/858421951 conn(0x7f16bc0fc520 msgr2=0x7f16bc103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.897+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 shutdown_connections 2024-09-10T04:37:22.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.897+0000 7f16c15cc640 1 -- 172.21.15.46:0/858421951 wait complete. 2024-09-10T04:37:22.906 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:37:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 -- 172.21.15.46:0/2418398626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c101900 msgr2=0x7f088c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 --2- 172.21.15.46:0/2418398626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c101900 0x7f088c101d60 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0880009880 tx=0x7f088002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.983 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:22 smithi046 ceph-mon[109506]: pgmap v952: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 -- 172.21.15.46:0/2418398626 shutdown_connections 2024-09-10T04:37:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 --2- 172.21.15.46:0/2418398626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c101900 0x7f088c101d60 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 --2- 172.21.15.46:0/2418398626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f088c107900 0x7f088c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 -- 172.21.15.46:0/2418398626 >> 172.21.15.46:0/2418398626 conn(0x7f088c0fd600 msgr2=0x7f088c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 -- 172.21.15.46:0/2418398626 shutdown_connections 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.982+0000 7f0891f82640 1 -- 172.21.15.46:0/2418398626 wait complete. 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.983+0000 7f0891f82640 1 Processor -- start 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.983+0000 7f0891f82640 1 -- start start 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.983+0000 7f0891f82640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f088c101900 0x7f088c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.983+0000 7f0891f82640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c107900 0x7f088c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.983+0000 7f0891f82640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10edd0 con 0x7f088c107900 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.983+0000 7f0891f82640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f088c10ec50 con 0x7f088c101900 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.983+0000 7f088affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c107900 0x7f088c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f088affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c107900 0x7f088c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47552/0 (socket says 172.21.15.46:47552) 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f088affd640 1 -- 172.21.15.46:0/1466939900 learned_addr learned my addr 172.21.15.46:0/1466939900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f088b7fe640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f088c101900 0x7f088c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f088affd640 1 -- 172.21.15.46:0/1466939900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f088c101900 msgr2=0x7f088c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f088affd640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f088c101900 0x7f088c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f088affd640 1 -- 172.21.15.46:0/1466939900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088c074f70 con 0x7f088c107900 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f088affd640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c107900 0x7f088c074a30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0880031460 tx=0x7f0880004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f0888ff9640 1 -- 172.21.15.46:0/1466939900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f088003f070 con 0x7f088c107900 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f0888ff9640 1 -- 172.21.15.46:0/1466939900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f08800035f0 con 0x7f088c107900 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088c075200 con 0x7f088c107900 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.984+0000 7f0888ff9640 1 -- 172.21.15.46:0/1466939900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0880035030 con 0x7f088c107900 2024-09-10T04:37:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.985+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f088c1a3d60 con 0x7f088c107900 2024-09-10T04:37:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.986+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088c103040 con 0x7f088c107900 2024-09-10T04:37:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.986+0000 7f0888ff9640 1 -- 172.21.15.46:0/1466939900 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0880042070 con 0x7f088c107900 2024-09-10T04:37:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.987+0000 7f0888ff9640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f087407ac40 0x7f087407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.987+0000 7f0888ff9640 1 -- 172.21.15.46:0/1466939900 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f08800be4a0 con 0x7f088c107900 2024-09-10T04:37:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.987+0000 7f088b7fe640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f087407ac40 0x7f087407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.987+0000 7f088b7fe640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f087407ac40 0x7f087407d100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f087c009bd0 tx=0x7f087c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:22.991+0000 7f0888ff9640 1 -- 172.21.15.46:0/1466939900 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0880089c60 con 0x7f088c107900 2024-09-10T04:37:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:22 smithi070 ceph-mon[79774]: pgmap v952: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:23.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.198+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f088c062630 con 0x7f087407ac40 2024-09-10T04:37:23.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.200+0000 7f0888ff9640 1 -- 172.21.15.46:0/1466939900 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f088c062630 con 0x7f087407ac40 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f087407ac40 msgr2=0x7f087407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f087407ac40 0x7f087407d100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f087c009bd0 tx=0x7f087c005f90 comp rx=0 tx=0).stop 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c107900 msgr2=0x7f088c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c107900 0x7f088c074a30 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f0880031460 tx=0x7f0880004530 comp rx=0 tx=0).stop 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 shutdown_connections 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f087407ac40 0x7f087407d100 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f088c107900 0x7f088c074a30 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 --2- 172.21.15.46:0/1466939900 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f088c101900 0x7f088c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 >> 172.21.15.46:0/1466939900 conn(0x7f088c0fd600 msgr2=0x7f088c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 shutdown_connections 2024-09-10T04:37:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.202+0000 7f0891f82640 1 -- 172.21.15.46:0/1466939900 wait complete. 2024-09-10T04:37:23.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.286+0000 7f64d6147640 1 -- 172.21.15.46:0/2061696908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 msgr2=0x7f64d0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.286+0000 7f64d6147640 1 --2- 172.21.15.46:0/2061696908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0100cc0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009880 tx=0x7f64b802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.286+0000 7f64d6147640 1 -- 172.21.15.46:0/2061696908 shutdown_connections 2024-09-10T04:37:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.286+0000 7f64d6147640 1 --2- 172.21.15.46:0/2061696908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0100cc0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.286+0000 7f64d6147640 1 --2- 172.21.15.46:0/2061696908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d0106860 0x7f64d0106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.286+0000 7f64d6147640 1 -- 172.21.15.46:0/2061696908 >> 172.21.15.46:0/2061696908 conn(0x7f64d00fc580 msgr2=0x7f64d00fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.287+0000 7f64d6147640 1 -- 172.21.15.46:0/2061696908 shutdown_connections 2024-09-10T04:37:23.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.287+0000 7f64d6147640 1 -- 172.21.15.46:0/2061696908 wait complete. 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.287+0000 7f64d6147640 1 Processor -- start 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.287+0000 7f64d6147640 1 -- start start 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64d6147640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64d6147640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d0106860 0x7f64d0198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64d6147640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f64d010dd10 con 0x7f64d0100860 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64d6147640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f64d010db90 con 0x7f64d0106860 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47556/0 (socket says 172.21.15.46:47556) 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cf7fe640 1 -- 172.21.15.46:0/2466613726 learned_addr learned my addr 172.21.15.46:0/2466613726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64c6ffd640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d0106860 0x7f64d0198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cf7fe640 1 -- 172.21.15.46:0/2466613726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d0106860 msgr2=0x7f64d0198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cf7fe640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d0106860 0x7f64d0198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cf7fe640 1 -- 172.21.15.46:0/2466613726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d0199380 con 0x7f64d0100860 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64c6ffd640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d0106860 0x7f64d0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cf7fe640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0198840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c810 tx=0x7f64c000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.288+0000 7f64cd7fa640 1 -- 172.21.15.46:0/2466613726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f64c0004020 con 0x7f64d0100860 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.289+0000 7f64cd7fa640 1 -- 172.21.15.46:0/2466613726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f64c00041c0 con 0x7f64d0100860 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.289+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d0199670 con 0x7f64d0100860 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.289+0000 7f64cd7fa640 1 -- 172.21.15.46:0/2466613726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f64c00047a0 con 0x7f64d0100860 2024-09-10T04:37:23.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.289+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d01a1c70 con 0x7f64d0100860 2024-09-10T04:37:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.290+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d0101fa0 con 0x7f64d0100860 2024-09-10T04:37:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.290+0000 7f64cd7fa640 1 -- 172.21.15.46:0/2466613726 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f64c00079b0 con 0x7f64d0100860 2024-09-10T04:37:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.291+0000 7f64cd7fa640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f64bc07abf0 0x7f64bc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.291+0000 7f64cd7fa640 1 -- 172.21.15.46:0/2466613726 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f64c009dba0 con 0x7f64d0100860 2024-09-10T04:37:23.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.291+0000 7f64c6ffd640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f64bc07abf0 0x7f64bc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.292+0000 7f64c6ffd640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f64bc07abf0 0x7f64bc07d0b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64b8006010 tx=0x7f64b80095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.295+0000 7f64cd7fa640 1 -- 172.21.15.46:0/2466613726 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f64d0199670 con 0x7f64d0100860 2024-09-10T04:37:23.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.490+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f64d0062630 con 0x7f64bc07abf0 2024-09-10T04:37:23.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.499+0000 7f64cd7fa640 1 -- 172.21.15.46:0/2466613726 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f64d0062630 con 0x7f64bc07abf0 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (28m) 8m ago 40m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (31m) 8m ago 40m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (31m) 8m ago 39m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (27m) 8m ago 40m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (27m) 8m ago 39m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (28m) 8m ago 40m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:37:23.501 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (23m) 8m ago 34m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (23m) 8m ago 34m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (23m) 8m ago 34m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (8m) 8m ago 34m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (30m) 8m ago 41m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (30m) 8m ago 39m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (27m) 8m ago 41m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (27m) 8m ago 39m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (29m) 8m ago 40m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (29m) 8m ago 39m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (26m) 8m ago 38m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (26m) 8m ago 37m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (25m) 8m ago 37m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (25m) 8m ago 36m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:37:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (24m) 8m ago 36m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:37:23.503 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (24m) 8m ago 35m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:37:23.503 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (29m) 8m ago 39m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:37:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f64bc07abf0 msgr2=0x7f64bc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f64bc07abf0 0x7f64bc07d0b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f64b8006010 tx=0x7f64b80095f0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 msgr2=0x7f64d0198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0198840 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c810 tx=0x7f64c000ccd0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 shutdown_connections 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f64bc07abf0 0x7f64bc07d0b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64d0106860 0x7f64d0198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 --2- 172.21.15.46:0/2466613726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64d0100860 0x7f64d0198840 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 >> 172.21.15.46:0/2466613726 conn(0x7f64d00fc580 msgr2=0x7f64d00fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 shutdown_connections 2024-09-10T04:37:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.502+0000 7f64d6147640 1 -- 172.21.15.46:0/2466613726 wait complete. 2024-09-10T04:37:23.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 -- 172.21.15.46:0/3200996908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108280 msgr2=0x7feb6c108660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 --2- 172.21.15.46:0/3200996908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108280 0x7feb6c108660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7feb60009880 tx=0x7feb6002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 -- 172.21.15.46:0/3200996908 shutdown_connections 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 --2- 172.21.15.46:0/3200996908 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb6c108c30 0x7feb6c10cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 --2- 172.21.15.46:0/3200996908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108280 0x7feb6c108660 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 -- 172.21.15.46:0/3200996908 >> 172.21.15.46:0/3200996908 conn(0x7feb6c077930 msgr2=0x7feb6c079d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 -- 172.21.15.46:0/3200996908 shutdown_connections 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.586+0000 7feb72b98640 1 -- 172.21.15.46:0/3200996908 wait complete. 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb72b98640 1 Processor -- start 2024-09-10T04:37:23.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb72b98640 1 -- start start 2024-09-10T04:37:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb72b98640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb6c108280 0x7feb6c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb72b98640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108c30 0x7feb6c19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb72b98640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c115430 con 0x7feb6c108c30 2024-09-10T04:37:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb72b98640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feb6c1152b0 con 0x7feb6c108280 2024-09-10T04:37:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108c30 0x7feb6c19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108c30 0x7feb6c19e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47560/0 (socket says 172.21.15.46:47560) 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb6bfff640 1 -- 172.21.15.46:0/3132302289 learned_addr learned my addr 172.21.15.46:0/3132302289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.587+0000 7feb70934640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb6c108280 0x7feb6c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb6bfff640 1 -- 172.21.15.46:0/3132302289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb6c108280 msgr2=0x7feb6c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb6bfff640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb6c108280 0x7feb6c19dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb6bfff640 1 -- 172.21.15.46:0/3132302289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6c1a28b0 con 0x7feb6c108c30 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb70934640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb6c108280 0x7feb6c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb6bfff640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108c30 0x7feb6c19e520 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7feb5800d6c0 tx=0x7feb5800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb69ffb640 1 -- 172.21.15.46:0/3132302289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb58017070 con 0x7feb6c108c30 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6c1a2ba0 con 0x7feb6c108c30 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb69ffb640 1 -- 172.21.15.46:0/3132302289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb580054e0 con 0x7feb6c108c30 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb69ffb640 1 -- 172.21.15.46:0/3132302289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb58004b70 con 0x7feb6c108c30 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.588+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6c1a3130 con 0x7feb6c108c30 2024-09-10T04:37:23.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.589+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6c109670 con 0x7feb6c108c30 2024-09-10T04:37:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.590+0000 7feb69ffb640 1 -- 172.21.15.46:0/3132302289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7feb58004e30 con 0x7feb6c108c30 2024-09-10T04:37:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.590+0000 7feb69ffb640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb5c07abf0 0x7feb5c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.591+0000 7feb70934640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb5c07abf0 0x7feb5c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.591+0000 7feb69ffb640 1 -- 172.21.15.46:0/3132302289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7feb5809d8e0 con 0x7feb6c108c30 2024-09-10T04:37:23.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.591+0000 7feb70934640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb5c07abf0 0x7feb5c07d0b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb60000c00 tx=0x7feb60031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:23.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.595+0000 7feb69ffb640 1 -- 172.21.15.46:0/3132302289 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7feb58069020 con 0x7feb6c108c30 2024-09-10T04:37:23.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.868+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7feb6c108660 con 0x7feb6c108c30 2024-09-10T04:37:23.872 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:23.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.869+0000 7feb69ffb640 1 -- 172.21.15.46:0/3132302289 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7feb5806b020 con 0x7feb6c108c30 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:23.873 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb5c07abf0 msgr2=0x7feb5c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb5c07abf0 0x7feb5c07d0b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb60000c00 tx=0x7feb60031400 comp rx=0 tx=0).stop 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108c30 msgr2=0x7feb6c19e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108c30 0x7feb6c19e520 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7feb5800d6c0 tx=0x7feb5800db80 comp rx=0 tx=0).stop 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 shutdown_connections 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb5c07abf0 0x7feb5c07d0b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb6c108c30 0x7feb6c19e520 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 --2- 172.21.15.46:0/3132302289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb6c108280 0x7feb6c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 >> 172.21.15.46:0/3132302289 conn(0x7feb6c077930 msgr2=0x7feb6c078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 shutdown_connections 2024-09-10T04:37:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.871+0000 7feb72b98640 1 -- 172.21.15.46:0/3132302289 wait complete. 2024-09-10T04:37:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 -- 172.21.15.46:0/532003252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214106800 msgr2=0x7fb214106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 --2- 172.21.15.46:0/532003252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214106800 0x7fb214106be0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb204009880 tx=0x7fb20402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 -- 172.21.15.46:0/532003252 shutdown_connections 2024-09-10T04:37:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 --2- 172.21.15.46:0/532003252 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb214100800 0x7fb214100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 --2- 172.21.15.46:0/532003252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214106800 0x7fb214106be0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 -- 172.21.15.46:0/532003252 >> 172.21.15.46:0/532003252 conn(0x7fb2140fc520 msgr2=0x7fb2140fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 -- 172.21.15.46:0/532003252 shutdown_connections 2024-09-10T04:37:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.955+0000 7fb2196f0640 1 -- 172.21.15.46:0/532003252 wait complete. 2024-09-10T04:37:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb2196f0640 1 Processor -- start 2024-09-10T04:37:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb2196f0640 1 -- start start 2024-09-10T04:37:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb2196f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214100800 0x7fb21419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb2196f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb214106800 0x7fb21419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb2196f0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb21410dcd0 con 0x7fb214100800 2024-09-10T04:37:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb2196f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb21410db50 con 0x7fb214106800 2024-09-10T04:37:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb212ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214100800 0x7fb21419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb212ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214100800 0x7fb21419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47576/0 (socket says 172.21.15.46:47576) 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.956+0000 7fb212ffd640 1 -- 172.21.15.46:0/268553162 learned_addr learned my addr 172.21.15.46:0/268553162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb2127fc640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb214106800 0x7fb21419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb212ffd640 1 -- 172.21.15.46:0/268553162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb214106800 msgr2=0x7fb21419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb212ffd640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb214106800 0x7fb21419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb212ffd640 1 -- 172.21.15.46:0/268553162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2141999e0 con 0x7fb214100800 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb2127fc640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb214106800 0x7fb21419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb212ffd640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214100800 0x7fb21419f820 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb204031460 tx=0x7fb2040045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb1fbfff640 1 -- 172.21.15.46:0/268553162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb20403f070 con 0x7fb214100800 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb214199c70 con 0x7fb214100800 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb1fbfff640 1 -- 172.21.15.46:0/268553162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb2040035f0 con 0x7fb214100800 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb1fbfff640 1 -- 172.21.15.46:0/268553162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb204035030 con 0x7fb214100800 2024-09-10T04:37:23.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.957+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb21419a1a0 con 0x7fb214100800 2024-09-10T04:37:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.958+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb214101f40 con 0x7fb214100800 2024-09-10T04:37:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.959+0000 7fb1fbfff640 1 -- 172.21.15.46:0/268553162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb204042070 con 0x7fb214100800 2024-09-10T04:37:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.959+0000 7fb1fbfff640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1fc07abf0 0x7fb1fc07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.960+0000 7fb1fbfff640 1 -- 172.21.15.46:0/268553162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb2040be720 con 0x7fb214100800 2024-09-10T04:37:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.960+0000 7fb2127fc640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1fc07abf0 0x7fb1fc07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.960+0000 7fb2127fc640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1fc07abf0 0x7fb1fc07d0b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb2080058c0 tx=0x7fb208005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:23.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:23.964+0000 7fb1fbfff640 1 -- 172.21.15.46:0/268553162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb204089dc0 con 0x7fb214100800 2024-09-10T04:37:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:24.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.187+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb21419af70 con 0x7fb214100800 2024-09-10T04:37:24.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.187+0000 7fb1fbfff640 1 -- 172.21.15.46:0/268553162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fb20408ec70 con 0x7fb214100800 2024-09-10T04:37:24.188 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:37:24.188 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:37:24.189 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:37:24.190 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:24.191 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:37:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1fc07abf0 msgr2=0x7fb1fc07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1fc07abf0 0x7fb1fc07d0b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb2080058c0 tx=0x7fb208005850 comp rx=0 tx=0).stop 2024-09-10T04:37:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214100800 msgr2=0x7fb21419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214100800 0x7fb21419f820 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb204031460 tx=0x7fb2040045e0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 shutdown_connections 2024-09-10T04:37:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb1fc07abf0 0x7fb1fc07d0b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb214106800 0x7fb21419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 --2- 172.21.15.46:0/268553162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb214100800 0x7fb21419f820 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 >> 172.21.15.46:0/268553162 conn(0x7fb2140fc520 msgr2=0x7fb2140fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 shutdown_connections 2024-09-10T04:37:24.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.191+0000 7fb2196f0640 1 -- 172.21.15.46:0/268553162 wait complete. 2024-09-10T04:37:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 -- 172.21.15.46:0/4244558336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888102ac0 msgr2=0x7f9888102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 --2- 172.21.15.46:0/4244558336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888102ac0 0x7f9888102ea0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9878009880 tx=0x7f987802a530 comp rx=0 tx=0).stop 2024-09-10T04:37:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 -- 172.21.15.46:0/4244558336 shutdown_connections 2024-09-10T04:37:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 --2- 172.21.15.46:0/4244558336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98881033e0 0x7f988810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 --2- 172.21.15.46:0/4244558336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888102ac0 0x7f9888102ea0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 -- 172.21.15.46:0/4244558336 >> 172.21.15.46:0/4244558336 conn(0x7f98880fc520 msgr2=0x7f98880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 -- 172.21.15.46:0/4244558336 shutdown_connections 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.276+0000 7f988d0f1640 1 -- 172.21.15.46:0/4244558336 wait complete. 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.277+0000 7f988d0f1640 1 Processor -- start 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.277+0000 7f988d0f1640 1 -- start start 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.277+0000 7f988d0f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9888102ac0 0x7f9888100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.277+0000 7f988d0f1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98881033e0 0x7f9888100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.277+0000 7f988d0f1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f988810edd0 con 0x7f98881033e0 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.277+0000 7f988d0f1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f988810ec50 con 0x7f9888102ac0 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.277+0000 7f988659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98881033e0 0x7f9888100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98881033e0 0x7f9888100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47588/0 (socket says 172.21.15.46:47588) 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988659c640 1 -- 172.21.15.46:0/670674370 learned_addr learned my addr 172.21.15.46:0/670674370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:24.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f9886d9d640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9888102ac0 0x7f9888100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988659c640 1 -- 172.21.15.46:0/670674370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9888102ac0 msgr2=0x7f9888100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988659c640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9888102ac0 0x7f9888100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988659c640 1 -- 172.21.15.46:0/670674370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98881011a0 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988659c640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98881033e0 0x7f9888100c60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f987c00d6c0 tx=0x7f987c00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f986ffff640 1 -- 172.21.15.46:0/670674370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f987c017070 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f986ffff640 1 -- 172.21.15.46:0/670674370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f987c009ce0 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9888101490 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f986ffff640 1 -- 172.21.15.46:0/670674370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f987c00f3c0 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.278+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98881a3f60 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.279+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988806aa90 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.280+0000 7f986ffff640 1 -- 172.21.15.46:0/670674370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f987c010460 con 0x7f98881033e0 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.280+0000 7f986ffff640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f987007abf0 0x7f987007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:24.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.281+0000 7f986ffff640 1 -- 172.21.15.46:0/670674370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f987c09d9b0 con 0x7f98881033e0 2024-09-10T04:37:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.281+0000 7f9886d9d640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f987007abf0 0x7f987007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.281+0000 7f9886d9d640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f987007abf0 0x7f987007d0b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98780096c0 tx=0x7f9878031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:24.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.285+0000 7f986ffff640 1 -- 172.21.15.46:0/670674370 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f987c0690f0 con 0x7f98881033e0 2024-09-10T04:37:24.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.489+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f9888062630 con 0x7f987007abf0 2024-09-10T04:37:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.490+0000 7f986ffff640 1 -- 172.21.15.46:0/670674370 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f9888062630 con 0x7f987007abf0 2024-09-10T04:37:24.491 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:37:24.491 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:37:24.491 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:37:24.492 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:37:24.493 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:37:24.493 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f987007abf0 msgr2=0x7f987007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f987007abf0 0x7f987007d0b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f98780096c0 tx=0x7f9878031400 comp rx=0 tx=0).stop 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98881033e0 msgr2=0x7f9888100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98881033e0 0x7f9888100c60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f987c00d6c0 tx=0x7f987c00db80 comp rx=0 tx=0).stop 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 shutdown_connections 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f987007abf0 0x7f987007d0b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98881033e0 0x7f9888100c60 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 --2- 172.21.15.46:0/670674370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9888102ac0 0x7f9888100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 >> 172.21.15.46:0/670674370 conn(0x7f98880fc520 msgr2=0x7f9888108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 shutdown_connections 2024-09-10T04:37:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.493+0000 7f988d0f1640 1 -- 172.21.15.46:0/670674370 wait complete. 2024-09-10T04:37:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/3087091902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 msgr2=0x7fa1cc101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/3087091902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc101d60 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009880 tx=0x7fa1b802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/3087091902 shutdown_connections 2024-09-10T04:37:24.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/3087091902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc101d60 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/3087091902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1cc107900 0x7fa1cc107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/3087091902 >> 172.21.15.46:0/3087091902 conn(0x7fa1cc0fd600 msgr2=0x7fa1cc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/3087091902 shutdown_connections 2024-09-10T04:37:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/3087091902 wait complete. 2024-09-10T04:37:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.577+0000 7fa1d2e3d640 1 Processor -- start 2024-09-10T04:37:24.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d2e3d640 1 -- start start 2024-09-10T04:37:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d2e3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d2e3d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1cc107900 0x7fa1cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d2e3d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc10edd0 con 0x7fa1cc101900 2024-09-10T04:37:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d2e3d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc10ec50 con 0x7fa1cc107900 2024-09-10T04:37:24.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d0bd9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d0bd9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47608/0 (socket says 172.21.15.46:47608) 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d0bd9640 1 -- 172.21.15.46:0/1980641581 learned_addr learned my addr 172.21.15.46:0/1980641581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1cbfff640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1cc107900 0x7fa1cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d0bd9640 1 -- 172.21.15.46:0/1980641581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1cc107900 msgr2=0x7fa1cc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d0bd9640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1cc107900 0x7fa1cc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d0bd9640 1 -- 172.21.15.46:0/1980641581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc074f70 con 0x7fa1cc101900 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.578+0000 7fa1d0bd9640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc0744f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d6f0 tx=0x7fa1c000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.579+0000 7fa1c9ffb640 1 -- 172.21.15.46:0/1980641581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa1c0017070 con 0x7fa1cc101900 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.579+0000 7fa1c9ffb640 1 -- 172.21.15.46:0/1980641581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1c000de00 con 0x7fa1cc101900 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.579+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc075260 con 0x7fa1cc101900 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.579+0000 7fa1c9ffb640 1 -- 172.21.15.46:0/1980641581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa1c0004bb0 con 0x7fa1cc101900 2024-09-10T04:37:24.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.579+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc1a3e80 con 0x7fa1cc101900 2024-09-10T04:37:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.580+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1cc103040 con 0x7fa1cc101900 2024-09-10T04:37:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.580+0000 7fa1c9ffb640 1 -- 172.21.15.46:0/1980641581 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa1c0009a30 con 0x7fa1cc101900 2024-09-10T04:37:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.581+0000 7fa1c9ffb640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1bc07ac50 0x7fa1bc07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:24.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.581+0000 7fa1c9ffb640 1 -- 172.21.15.46:0/1980641581 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa1c009d7f0 con 0x7fa1cc101900 2024-09-10T04:37:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.581+0000 7fa1cbfff640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1bc07ac50 0x7fa1bc07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:24.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.582+0000 7fa1cbfff640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1bc07ac50 0x7fa1bc07d110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80096c0 tx=0x7fa1b8000bc0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:24.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.585+0000 7fa1c9ffb640 1 -- 172.21.15.46:0/1980641581 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa1c0068fb0 con 0x7fa1cc101900 2024-09-10T04:37:24.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.844+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fa1cc101d60 con 0x7fa1cc101900 2024-09-10T04:37:24.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.844+0000 7fa1c9ffb640 1 -- 172.21.15.46:0/1980641581 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fa1c006de60 con 0x7fa1cc101900 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:37:24.846 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:37:24.847 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:37:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1bc07ac50 msgr2=0x7fa1bc07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1bc07ac50 0x7fa1bc07d110 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80096c0 tx=0x7fa1b8000bc0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 msgr2=0x7fa1cc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc0744f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d6f0 tx=0x7fa1c000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 shutdown_connections 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1bc07ac50 0x7fa1bc07d110 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1cc107900 0x7fa1cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 --2- 172.21.15.46:0/1980641581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1cc101900 0x7fa1cc0744f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 >> 172.21.15.46:0/1980641581 conn(0x7fa1cc0fd600 msgr2=0x7fa1cc0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 shutdown_connections 2024-09-10T04:37:24.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:24.847+0000 7fa1d2e3d640 1 -- 172.21.15.46:0/1980641581 wait complete. 2024-09-10T04:37:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:24 smithi070 ceph-mon[79774]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:24 smithi070 ceph-mon[79774]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:24 smithi070 ceph-mon[79774]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:24 smithi070 ceph-mon[79774]: pgmap v953: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:24 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3132302289' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:37:25.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:24 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/268553162' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:37:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:24 smithi046 ceph-mon[109506]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:25.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:24 smithi046 ceph-mon[109506]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:25.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:24 smithi046 ceph-mon[109506]: from='client.35578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:25.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:24 smithi046 ceph-mon[109506]: pgmap v953: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:25.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:24 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3132302289' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:37:25.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:24 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/268553162' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:37:26.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:25 smithi070 ceph-mon[79774]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:26.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:25 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1980641581' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:37:26.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:25 smithi046 ceph-mon[109506]: from='client.35590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:26.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:25 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1980641581' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:37:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:26 smithi070 ceph-mon[79774]: pgmap v954: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:26 smithi046 ceph-mon[109506]: pgmap v954: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:28 smithi070 ceph-mon[79774]: pgmap v955: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:28 smithi046 ceph-mon[109506]: pgmap v955: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:30 smithi070 ceph-mon[79774]: pgmap v956: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:31.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:30 smithi046 ceph-mon[109506]: pgmap v956: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:32 smithi070 ceph-mon[79774]: pgmap v957: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:32 smithi046 ceph-mon[109506]: pgmap v957: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:34 smithi070 ceph-mon[79774]: pgmap v958: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:34 smithi046 ceph-mon[109506]: pgmap v958: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:36 smithi070 ceph-mon[79774]: pgmap v959: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:36 smithi046 ceph-mon[109506]: pgmap v959: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:38 smithi070 ceph-mon[79774]: pgmap v960: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:38 smithi046 ceph-mon[109506]: pgmap v960: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:40 smithi070 ceph-mon[79774]: pgmap v961: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:40 smithi046 ceph-mon[109506]: pgmap v961: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:42 smithi070 ceph-mon[79774]: pgmap v962: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:42 smithi046 ceph-mon[109506]: pgmap v962: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:44 smithi070 ceph-mon[79774]: pgmap v963: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:44 smithi046 ceph-mon[109506]: pgmap v963: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:46 smithi070 ceph-mon[79774]: pgmap v964: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:46 smithi046 ceph-mon[109506]: pgmap v964: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:48 smithi070 ceph-mon[79774]: pgmap v965: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:48 smithi046 ceph-mon[109506]: pgmap v965: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:50 smithi070 ceph-mon[79774]: pgmap v966: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:50 smithi046 ceph-mon[109506]: pgmap v966: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:52 smithi070 ceph-mon[79774]: pgmap v967: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:52 smithi046 ceph-mon[109506]: pgmap v967: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 -- 172.21.15.46:0/3603795102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074f40 msgr2=0x7f8a9410dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 --2- 172.21.15.46:0/3603795102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074f40 0x7f8a9410dc00 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009880 tx=0x7f8a7c02a530 comp rx=0 tx=0).stop 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 -- 172.21.15.46:0/3603795102 shutdown_connections 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 --2- 172.21.15.46:0/3603795102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074f40 0x7f8a9410dc00 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 --2- 172.21.15.46:0/3603795102 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a94074620 0x7f8a94074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 -- 172.21.15.46:0/3603795102 >> 172.21.15.46:0/3603795102 conn(0x7f8a940fd600 msgr2=0x7f8a940ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 -- 172.21.15.46:0/3603795102 shutdown_connections 2024-09-10T04:37:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.943+0000 7f8a99718640 1 -- 172.21.15.46:0/3603795102 wait complete. 2024-09-10T04:37:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.944+0000 7f8a99718640 1 Processor -- start 2024-09-10T04:37:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.944+0000 7f8a99718640 1 -- start start 2024-09-10T04:37:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a99718640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074620 0x7f8a9419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a99718640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a94074f40 0x7f8a9419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a99718640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94110fd0 con 0x7f8a94074620 2024-09-10T04:37:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a99718640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94110e50 con 0x7f8a94074f40 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074620 0x7f8a9419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a92ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074620 0x7f8a9419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52718/0 (socket says 172.21.15.46:52718) 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a92ffd640 1 -- 172.21.15.46:0/1942993105 learned_addr learned my addr 172.21.15.46:0/1942993105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a8a7fc640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a94074f40 0x7f8a9419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a92ffd640 1 -- 172.21.15.46:0/1942993105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a94074f40 msgr2=0x7f8a9419fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a92ffd640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a94074f40 0x7f8a9419fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a92ffd640 1 -- 172.21.15.46:0/1942993105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a94199a00 con 0x7f8a94074620 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a8a7fc640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a94074f40 0x7f8a9419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.945+0000 7f8a92ffd640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074620 0x7f8a9419f840 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400d8d0 tx=0x7f8a8400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.946+0000 7f8a90ff9640 1 -- 172.21.15.46:0/1942993105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8a84017030 con 0x7f8a94074620 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.946+0000 7f8a90ff9640 1 -- 172.21.15.46:0/1942993105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8a84004020 con 0x7f8a94074620 2024-09-10T04:37:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.946+0000 7f8a90ff9640 1 -- 172.21.15.46:0/1942993105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8a840047a0 con 0x7f8a94074620 2024-09-10T04:37:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.946+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a94199cf0 con 0x7f8a94074620 2024-09-10T04:37:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.946+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9419a250 con 0x7f8a94074620 2024-09-10T04:37:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.947+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a94075cd0 con 0x7f8a94074620 2024-09-10T04:37:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.948+0000 7f8a90ff9640 1 -- 172.21.15.46:0/1942993105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8a840041d0 con 0x7f8a94074620 2024-09-10T04:37:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.948+0000 7f8a90ff9640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8a8007abf0 0x7f8a8007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.948+0000 7f8a90ff9640 1 -- 172.21.15.46:0/1942993105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8a84018070 con 0x7f8a94074620 2024-09-10T04:37:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.948+0000 7f8a8a7fc640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8a8007abf0 0x7f8a8007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.949+0000 7f8a8a7fc640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8a8007abf0 0x7f8a8007d0b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8a9419b1f0 tx=0x7f8a7c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:54.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:54.953+0000 7f8a90ff9640 1 -- 172.21.15.46:0/1942993105 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8a840691e0 con 0x7f8a94074620 2024-09-10T04:37:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:54 smithi070 ceph-mon[79774]: pgmap v968: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:54 smithi046 ceph-mon[109506]: pgmap v968: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.158+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8a9419af30 con 0x7f8a8007abf0 2024-09-10T04:37:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.159+0000 7f8a90ff9640 1 -- 172.21.15.46:0/1942993105 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8a9419af30 con 0x7f8a8007abf0 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8a8007abf0 msgr2=0x7f8a8007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8a8007abf0 0x7f8a8007d0b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8a9419b1f0 tx=0x7f8a7c031400 comp rx=0 tx=0).stop 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074620 msgr2=0x7f8a9419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074620 0x7f8a9419f840 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8a8400d8d0 tx=0x7f8a8400dd90 comp rx=0 tx=0).stop 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 shutdown_connections 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8a8007abf0 0x7f8a8007d0b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a94074f40 0x7f8a9419fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 --2- 172.21.15.46:0/1942993105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a94074620 0x7f8a9419f840 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.161+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 >> 172.21.15.46:0/1942993105 conn(0x7f8a940fd600 msgr2=0x7f8a9410b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.162+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 shutdown_connections 2024-09-10T04:37:55.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.162+0000 7f8a99718640 1 -- 172.21.15.46:0/1942993105 wait complete. 2024-09-10T04:37:55.172 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:37:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 -- 172.21.15.46:0/3532854621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 msgr2=0x7f1030100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 --2- 172.21.15.46:0/3532854621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f1030100c60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1024009880 tx=0x7f102402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 -- 172.21.15.46:0/3532854621 shutdown_connections 2024-09-10T04:37:55.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 --2- 172.21.15.46:0/3532854621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f1030100c60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 --2- 172.21.15.46:0/3532854621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1030106800 0x7f1030106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 -- 172.21.15.46:0/3532854621 >> 172.21.15.46:0/3532854621 conn(0x7f10300fc520 msgr2=0x7f10300fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:55.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 -- 172.21.15.46:0/3532854621 shutdown_connections 2024-09-10T04:37:55.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.248+0000 7f1034b15640 1 -- 172.21.15.46:0/3532854621 wait complete. 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f1034b15640 1 Processor -- start 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f1034b15640 1 -- start start 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f1034b15640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f10300fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f1034b15640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1030106800 0x7f10300fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f1034b15640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f103010dcd0 con 0x7f1030100800 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f1034b15640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f103010db50 con 0x7f1030106800 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f102e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f10300fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f102e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f10300fcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52732/0 (socket says 172.21.15.46:52732) 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f102e59c640 1 -- 172.21.15.46:0/266447802 learned_addr learned my addr 172.21.15.46:0/266447802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:55.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.249+0000 7f102dd9b640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1030106800 0x7f10300fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f102e59c640 1 -- 172.21.15.46:0/266447802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1030106800 msgr2=0x7f10300fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f102e59c640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1030106800 0x7f10300fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f102e59c640 1 -- 172.21.15.46:0/266447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10300fe990 con 0x7f1030100800 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f102e59c640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f10300fcc10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f102000e970 tx=0x7f102000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f101f7fe640 1 -- 172.21.15.46:0/266447802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f102000cac0 con 0x7f1030100800 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f101f7fe640 1 -- 172.21.15.46:0/266447802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f10200054e0 con 0x7f1030100800 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10300fd690 con 0x7f1030100800 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f101f7fe640 1 -- 172.21.15.46:0/266447802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1020010430 con 0x7f1030100800 2024-09-10T04:37:55.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.250+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10301a8360 con 0x7f1030100800 2024-09-10T04:37:55.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.251+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1030101f40 con 0x7f1030100800 2024-09-10T04:37:55.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.252+0000 7f101f7fe640 1 -- 172.21.15.46:0/266447802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1020004ec0 con 0x7f1030100800 2024-09-10T04:37:55.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.252+0000 7f101f7fe640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f101807abf0 0x7f101807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.252+0000 7f101f7fe640 1 -- 172.21.15.46:0/266447802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f102009e310 con 0x7f1030100800 2024-09-10T04:37:55.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.252+0000 7f102dd9b640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f101807abf0 0x7f101807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.253+0000 7f102dd9b640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f101807abf0 0x7f101807d0b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1024000c00 tx=0x7f1024031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:55.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.257+0000 7f101f7fe640 1 -- 172.21.15.46:0/266447802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1020069a50 con 0x7f1030100800 2024-09-10T04:37:55.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.459+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f1030062630 con 0x7f101807abf0 2024-09-10T04:37:55.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.460+0000 7f101f7fe640 1 -- 172.21.15.46:0/266447802 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f1030062630 con 0x7f101807abf0 2024-09-10T04:37:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f101807abf0 msgr2=0x7f101807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f101807abf0 0x7f101807d0b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1024000c00 tx=0x7f1024031400 comp rx=0 tx=0).stop 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 msgr2=0x7f10300fcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f10300fcc10 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f102000e970 tx=0x7f102000ed70 comp rx=0 tx=0).stop 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 shutdown_connections 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f101807abf0 0x7f101807d0b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1030106800 0x7f10300fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 --2- 172.21.15.46:0/266447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1030100800 0x7f10300fcc10 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 >> 172.21.15.46:0/266447802 conn(0x7f10300fc520 msgr2=0x7f103006a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 shutdown_connections 2024-09-10T04:37:55.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.462+0000 7f1034b15640 1 -- 172.21.15.46:0/266447802 wait complete. 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 -- 172.21.15.46:0/1675208911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106800 msgr2=0x7fc504106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 --2- 172.21.15.46:0/1675208911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106800 0x7fc504106be0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009880 tx=0x7fc4f402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 -- 172.21.15.46:0/1675208911 shutdown_connections 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 --2- 172.21.15.46:0/1675208911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc504100800 0x7fc504100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 --2- 172.21.15.46:0/1675208911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504106800 0x7fc504106be0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 -- 172.21.15.46:0/1675208911 >> 172.21.15.46:0/1675208911 conn(0x7fc5040fc520 msgr2=0x7fc5040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 -- 172.21.15.46:0/1675208911 shutdown_connections 2024-09-10T04:37:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.547+0000 7fc508cc6640 1 -- 172.21.15.46:0/1675208911 wait complete. 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc508cc6640 1 Processor -- start 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc508cc6640 1 -- start start 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc508cc6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504100800 0x7fc504198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc508cc6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc504106800 0x7fc504198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc508cc6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc50410dcd0 con 0x7fc504100800 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc508cc6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc50410db50 con 0x7fc504106800 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc50259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504100800 0x7fc504198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc50259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504100800 0x7fc504198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52736/0 (socket says 172.21.15.46:52736) 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc50259c640 1 -- 172.21.15.46:0/3068614758 learned_addr learned my addr 172.21.15.46:0/3068614758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc4fbfff640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc504106800 0x7fc504198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc50259c640 1 -- 172.21.15.46:0/3068614758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc504106800 msgr2=0x7fc504198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc50259c640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc504106800 0x7fc504198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.548+0000 7fc50259c640 1 -- 172.21.15.46:0/3068614758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc504199350 con 0x7fc504100800 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.549+0000 7fc50259c640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504100800 0x7fc504198810 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4031460 tx=0x7fc4f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.549+0000 7fc4fb7fe640 1 -- 172.21.15.46:0/3068614758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc4f403f070 con 0x7fc504100800 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.549+0000 7fc4fb7fe640 1 -- 172.21.15.46:0/3068614758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4f40035f0 con 0x7fc504100800 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.549+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5041995e0 con 0x7fc504100800 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.549+0000 7fc4fb7fe640 1 -- 172.21.15.46:0/3068614758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc4f4035030 con 0x7fc504100800 2024-09-10T04:37:55.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.549+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5041a1b90 con 0x7fc504100800 2024-09-10T04:37:55.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.550+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc504101f40 con 0x7fc504100800 2024-09-10T04:37:55.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.550+0000 7fc4fb7fe640 1 -- 172.21.15.46:0/3068614758 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc4f4042070 con 0x7fc504100800 2024-09-10T04:37:55.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.551+0000 7fc4fb7fe640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f007abf0 0x7fc4f007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.551+0000 7fc4fb7fe640 1 -- 172.21.15.46:0/3068614758 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc4f40be5d0 con 0x7fc504100800 2024-09-10T04:37:55.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.551+0000 7fc4fbfff640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f007abf0 0x7fc4f007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.552+0000 7fc4fbfff640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f007abf0 0x7fc4f007d0b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc50406ad00 tx=0x7fc4ec00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:55.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.556+0000 7fc4fb7fe640 1 -- 172.21.15.46:0/3068614758 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc4f4089d10 con 0x7fc504100800 2024-09-10T04:37:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.753+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc504062630 con 0x7fc4f007abf0 2024-09-10T04:37:55.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.765+0000 7fc4fb7fe640 1 -- 172.21.15.46:0/3068614758 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fc504062630 con 0x7fc4f007abf0 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (29m) 8m ago 40m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (32m) 8m ago 41m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (31m) 8m ago 40m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (27m) 8m ago 41m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (27m) 8m ago 40m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (29m) 8m ago 40m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (24m) 8m ago 35m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (24m) 8m ago 35m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:37:55.767 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (23m) 8m ago 35m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (8m) 8m ago 35m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (31m) 8m ago 42m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (30m) 8m ago 40m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (28m) 8m ago 42m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (28m) 8m ago 39m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (30m) 8m ago 40m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (30m) 8m ago 40m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (27m) 8m ago 38m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (26m) 8m ago 38m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (26m) 8m ago 37m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:37:55.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (25m) 8m ago 37m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:37:55.769 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (25m) 8m ago 36m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:37:55.769 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (24m) 8m ago 36m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:37:55.769 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (29m) 8m ago 40m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:37:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f007abf0 msgr2=0x7fc4f007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f007abf0 0x7fc4f007d0b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc50406ad00 tx=0x7fc4ec00b390 comp rx=0 tx=0).stop 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504100800 msgr2=0x7fc504198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504100800 0x7fc504198810 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4031460 tx=0x7fc4f40045e0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 shutdown_connections 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc4f007abf0 0x7fc4f007d0b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc504106800 0x7fc504198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 --2- 172.21.15.46:0/3068614758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc504100800 0x7fc504198810 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 >> 172.21.15.46:0/3068614758 conn(0x7fc5040fc520 msgr2=0x7fc5040fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 shutdown_connections 2024-09-10T04:37:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.767+0000 7fc508cc6640 1 -- 172.21.15.46:0/3068614758 wait complete. 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 -- 172.21.15.46:0/2256158938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc107900 msgr2=0x7f2fbc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/2256158938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc107900 0x7f2fbc107ce0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009880 tx=0x7f2fa402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 -- 172.21.15.46:0/2256158938 shutdown_connections 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/2256158938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc101900 0x7f2fbc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/2256158938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc107900 0x7f2fbc107ce0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 -- 172.21.15.46:0/2256158938 >> 172.21.15.46:0/2256158938 conn(0x7f2fbc0fd600 msgr2=0x7f2fbc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 -- 172.21.15.46:0/2256158938 shutdown_connections 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.851+0000 7f2fc2eff640 1 -- 172.21.15.46:0/2256158938 wait complete. 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc2eff640 1 Processor -- start 2024-09-10T04:37:55.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc2eff640 1 -- start start 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc2eff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc101900 0x7f2fbc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc2eff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc107900 0x7f2fbc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc2eff640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc10ee30 con 0x7f2fbc101900 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc2eff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc10ecb0 con 0x7f2fbc107900 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc0c9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc101900 0x7f2fbc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc0c9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc101900 0x7f2fbc19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52752/0 (socket says 172.21.15.46:52752) 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc0c9b640 1 -- 172.21.15.46:0/1534679384 learned_addr learned my addr 172.21.15.46:0/1534679384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fbbfff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc107900 0x7f2fbc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc0c9b640 1 -- 172.21.15.46:0/1534679384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc107900 msgr2=0x7f2fbc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc0c9b640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc107900 0x7f2fbc19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc0c9b640 1 -- 172.21.15.46:0/1534679384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fbc197820 con 0x7f2fbc101900 2024-09-10T04:37:55.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fbbfff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc107900 0x7f2fbc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:37:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.852+0000 7f2fc0c9b640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc101900 0x7f2fbc19d6a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4031460 tx=0x7f2fa40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.853+0000 7f2fb9ffb640 1 -- 172.21.15.46:0/1534679384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2fa403f070 con 0x7f2fbc101900 2024-09-10T04:37:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.853+0000 7f2fb9ffb640 1 -- 172.21.15.46:0/1534679384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2fa4004cf0 con 0x7f2fbc101900 2024-09-10T04:37:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.853+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fbc197ab0 con 0x7f2fbc101900 2024-09-10T04:37:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.853+0000 7f2fb9ffb640 1 -- 172.21.15.46:0/1534679384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2fa4035030 con 0x7f2fbc101900 2024-09-10T04:37:55.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.853+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fbc197fe0 con 0x7f2fbc101900 2024-09-10T04:37:55.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.854+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fbc103040 con 0x7f2fbc101900 2024-09-10T04:37:55.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.855+0000 7f2fb9ffb640 1 -- 172.21.15.46:0/1534679384 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2fa4042070 con 0x7f2fbc101900 2024-09-10T04:37:55.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.855+0000 7f2fb9ffb640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2fa807ac40 0x7f2fa807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:55.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.855+0000 7f2fb9ffb640 1 -- 172.21.15.46:0/1534679384 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2fa40be770 con 0x7f2fbc101900 2024-09-10T04:37:55.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.855+0000 7f2fbbfff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2fa807ac40 0x7f2fa807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:55.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.856+0000 7f2fbbfff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2fa807ac40 0x7f2fa807d100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0ffc00 tx=0x7f2fac0073c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:55.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:55.860+0000 7f2fb9ffb640 1 -- 172.21.15.46:0/1534679384 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2fa4089e90 con 0x7f2fbc101900 2024-09-10T04:37:56.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:55 smithi070 ceph-mon[79774]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:56.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:55 smithi070 ceph-mon[79774]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:56.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:55 smithi070 ceph-mon[79774]: pgmap v969: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:56.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:55 smithi070 ceph-mon[79774]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:55 smithi046 ceph-mon[109506]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:55 smithi046 ceph-mon[109506]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:56.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:55 smithi046 ceph-mon[109506]: pgmap v969: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:56.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:55 smithi046 ceph-mon[109506]: from='client.35606 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:56.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.122+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f2fbc109e70 con 0x7f2fbc101900 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.123+0000 7f2fb9ffb640 1 -- 172.21.15.46:0/1534679384 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f2fa408ed40 con 0x7f2fbc101900 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:37:56.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:37:56.126 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:37:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2fa807ac40 msgr2=0x7f2fa807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2fa807ac40 0x7f2fa807d100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0ffc00 tx=0x7f2fac0073c0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc101900 msgr2=0x7f2fbc19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc101900 0x7f2fbc19d6a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4031460 tx=0x7f2fa40048f0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 shutdown_connections 2024-09-10T04:37:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2fa807ac40 0x7f2fa807d100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc107900 0x7f2fbc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 --2- 172.21.15.46:0/1534679384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fbc101900 0x7f2fbc19d6a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 >> 172.21.15.46:0/1534679384 conn(0x7f2fbc0fd600 msgr2=0x7f2fbc0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 shutdown_connections 2024-09-10T04:37:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.125+0000 7f2fc2eff640 1 -- 172.21.15.46:0/1534679384 wait complete. 2024-09-10T04:37:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.209+0000 7fa208d06640 1 -- 172.21.15.46:0/3901562682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 msgr2=0x7fa204106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.209+0000 7fa208d06640 1 --2- 172.21.15.46:0/3901562682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204106bd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4009880 tx=0x7fa1f402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.210+0000 7fa208d06640 1 -- 172.21.15.46:0/3901562682 shutdown_connections 2024-09-10T04:37:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.210+0000 7fa208d06640 1 --2- 172.21.15.46:0/3901562682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2041007f0 0x7fa204100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.210+0000 7fa208d06640 1 --2- 172.21.15.46:0/3901562682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204106bd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.210+0000 7fa208d06640 1 -- 172.21.15.46:0/3901562682 >> 172.21.15.46:0/3901562682 conn(0x7fa2040fc550 msgr2=0x7fa2040fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.210+0000 7fa208d06640 1 -- 172.21.15.46:0/3901562682 shutdown_connections 2024-09-10T04:37:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.210+0000 7fa208d06640 1 -- 172.21.15.46:0/3901562682 wait complete. 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.210+0000 7fa208d06640 1 Processor -- start 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa208d06640 1 -- start start 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa208d06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2041007f0 0x7fa2041987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa208d06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa208d06640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa20410dcc0 con 0x7fa2041067f0 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa208d06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa20410db40 con 0x7fa2041007f0 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa201d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa201d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52782/0 (socket says 172.21.15.46:52782) 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa201d9b640 1 -- 172.21.15.46:0/3906517734 learned_addr learned my addr 172.21.15.46:0/3906517734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa20259c640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2041007f0 0x7fa2041987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa201d9b640 1 -- 172.21.15.46:0/3906517734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2041007f0 msgr2=0x7fa2041987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa201d9b640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2041007f0 0x7fa2041987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.211+0000 7fa201d9b640 1 -- 172.21.15.46:0/3906517734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa204199360 con 0x7fa2041067f0 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.212+0000 7fa201d9b640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204198d10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00d6f0 tx=0x7fa1ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.212+0000 7fa1fb7fe640 1 -- 172.21.15.46:0/3906517734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa1ec017070 con 0x7fa2041067f0 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.212+0000 7fa1fb7fe640 1 -- 172.21.15.46:0/3906517734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa1ec00de00 con 0x7fa2041067f0 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.212+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2041a1840 con 0x7fa2041067f0 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.212+0000 7fa1fb7fe640 1 -- 172.21.15.46:0/3906517734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa1ec004bb0 con 0x7fa2041067f0 2024-09-10T04:37:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.212+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2041a1c40 con 0x7fa2041067f0 2024-09-10T04:37:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.213+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa204101f30 con 0x7fa2041067f0 2024-09-10T04:37:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.214+0000 7fa1fb7fe640 1 -- 172.21.15.46:0/3906517734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa1ec009a30 con 0x7fa2041067f0 2024-09-10T04:37:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.214+0000 7fa1fb7fe640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f007ac40 0x7fa1f007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.214+0000 7fa1fb7fe640 1 -- 172.21.15.46:0/3906517734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa1ec09d740 con 0x7fa2041067f0 2024-09-10T04:37:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.214+0000 7fa20259c640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f007ac40 0x7fa1f007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.215+0000 7fa20259c640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f007ac40 0x7fa1f007d100 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40096c0 tx=0x7fa1f4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.219+0000 7fa1fb7fe640 1 -- 172.21.15.46:0/3906517734 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa1ec068f00 con 0x7fa2041067f0 2024-09-10T04:37:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.443+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fa204108d60 con 0x7fa2041067f0 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.443+0000 7fa1fb7fe640 1 -- 172.21.15.46:0/3906517734 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fa1ec06ddb0 con 0x7fa2041067f0 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:56.445 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:37:56.446 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:37:56.447 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:56.448 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f007ac40 msgr2=0x7fa1f007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f007ac40 0x7fa1f007d100 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa1f40096c0 tx=0x7fa1f4031400 comp rx=0 tx=0).stop 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 msgr2=0x7fa204198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204198d10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec00d6f0 tx=0x7fa1ec00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 shutdown_connections 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa1f007ac40 0x7fa1f007d100 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa2041067f0 0x7fa204198d10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 --2- 172.21.15.46:0/3906517734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2041007f0 0x7fa2041987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 >> 172.21.15.46:0/3906517734 conn(0x7fa2040fc550 msgr2=0x7fa2040fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 shutdown_connections 2024-09-10T04:37:56.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.445+0000 7fa208d06640 1 -- 172.21.15.46:0/3906517734 wait complete. 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 -- 172.21.15.46:0/1618001386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 msgr2=0x7f75a810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 --2- 172.21.15.46:0/1618001386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a810b980 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f759c009880 tx=0x7f759c02a530 comp rx=0 tx=0).stop 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 -- 172.21.15.46:0/1618001386 shutdown_connections 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 --2- 172.21.15.46:0/1618001386 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a810b980 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 --2- 172.21.15.46:0/1618001386 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8102b70 0x7f75a8102f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 -- 172.21.15.46:0/1618001386 >> 172.21.15.46:0/1618001386 conn(0x7f75a80fc550 msgr2=0x7f75a80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 -- 172.21.15.46:0/1618001386 shutdown_connections 2024-09-10T04:37:56.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 -- 172.21.15.46:0/1618001386 wait complete. 2024-09-10T04:37:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 Processor -- start 2024-09-10T04:37:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.531+0000 7f75aef74640 1 -- start start 2024-09-10T04:37:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75aef74640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8102b70 0x7f75a819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75aef74640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a819db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75aef74640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75a810ede0 con 0x7f75a8103490 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75aef74640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f75a810ec60 con 0x7f75a8102b70 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a819db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a819db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52812/0 (socket says 172.21.15.46:52812) 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a7fff640 1 -- 172.21.15.46:0/625568579 learned_addr learned my addr 172.21.15.46:0/625568579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75acd10640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8102b70 0x7f75a819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a7fff640 1 -- 172.21.15.46:0/625568579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8102b70 msgr2=0x7f75a819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a7fff640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8102b70 0x7f75a819d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a7fff640 1 -- 172.21.15.46:0/625568579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a81977d0 con 0x7f75a8103490 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75acd10640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8102b70 0x7f75a819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a7fff640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a819db90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f759c004f90 tx=0x7f759c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a5ffb640 1 -- 172.21.15.46:0/625568579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f759c03f070 con 0x7f75a8103490 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a8197a60 con 0x7f75a8103490 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a5ffb640 1 -- 172.21.15.46:0/625568579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f759c004cf0 con 0x7f75a8103490 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.532+0000 7f75a5ffb640 1 -- 172.21.15.46:0/625568579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f759c035030 con 0x7f75a8103490 2024-09-10T04:37:56.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.533+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a8197f90 con 0x7f75a8103490 2024-09-10T04:37:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.534+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a806a4c0 con 0x7f75a8103490 2024-09-10T04:37:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.534+0000 7f75a5ffb640 1 -- 172.21.15.46:0/625568579 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f759c042070 con 0x7f75a8103490 2024-09-10T04:37:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.535+0000 7f75a5ffb640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f759007abf0 0x7f759007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.535+0000 7f75a5ffb640 1 -- 172.21.15.46:0/625568579 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f759c0be770 con 0x7f75a8103490 2024-09-10T04:37:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.535+0000 7f75acd10640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f759007abf0 0x7f759007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.536+0000 7f75acd10640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f759007abf0 0x7f759007d0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f75980058c0 tx=0x7f7598005d30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:56.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.539+0000 7f75a5ffb640 1 -- 172.21.15.46:0/625568579 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f759c089eb0 con 0x7f75a8103490 2024-09-10T04:37:56.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.742+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f75a8198240 con 0x7f759007abf0 2024-09-10T04:37:56.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.744+0000 7f75a5ffb640 1 -- 172.21.15.46:0/625568579 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f75a8198240 con 0x7f759007abf0 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:37:56.746 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:37:56.747 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:37:56.747 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:37:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f759007abf0 msgr2=0x7f759007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f759007abf0 0x7f759007d0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f75980058c0 tx=0x7f7598005d30 comp rx=0 tx=0).stop 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 msgr2=0x7f75a819db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a819db90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f759c004f90 tx=0x7f759c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 shutdown_connections 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f759007abf0 0x7f759007d0b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a8103490 0x7f75a819db90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 --2- 172.21.15.46:0/625568579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75a8102b70 0x7f75a819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.747+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 >> 172.21.15.46:0/625568579 conn(0x7f75a80fc550 msgr2=0x7f75a8103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.748+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 shutdown_connections 2024-09-10T04:37:56.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.748+0000 7f75aef74640 1 -- 172.21.15.46:0/625568579 wait complete. 2024-09-10T04:37:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 -- 172.21.15.46:0/3130012619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c101900 msgr2=0x7f160c101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 --2- 172.21.15.46:0/3130012619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c101900 0x7f160c101d60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1600009880 tx=0x7f160002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 -- 172.21.15.46:0/3130012619 shutdown_connections 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 --2- 172.21.15.46:0/3130012619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c101900 0x7f160c101d60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 --2- 172.21.15.46:0/3130012619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160c107900 0x7f160c107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 -- 172.21.15.46:0/3130012619 >> 172.21.15.46:0/3130012619 conn(0x7f160c0fd600 msgr2=0x7f160c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 -- 172.21.15.46:0/3130012619 shutdown_connections 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.832+0000 7f16122fa640 1 -- 172.21.15.46:0/3130012619 wait complete. 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f16122fa640 1 Processor -- start 2024-09-10T04:37:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f16122fa640 1 -- start start 2024-09-10T04:37:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f16122fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160c101900 0x7f160c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f16122fa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c107900 0x7f160c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f16122fa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f160c10edd0 con 0x7f160c107900 2024-09-10T04:37:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f16122fa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f160c10ec50 con 0x7f160c101900 2024-09-10T04:37:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f160b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c107900 0x7f160c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f160b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c107900 0x7f160c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52824/0 (socket says 172.21.15.46:52824) 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f160b7fe640 1 -- 172.21.15.46:0/2058985030 learned_addr learned my addr 172.21.15.46:0/2058985030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.833+0000 7f160bfff640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160c101900 0x7f160c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f160b7fe640 1 -- 172.21.15.46:0/2058985030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160c101900 msgr2=0x7f160c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f160b7fe640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160c101900 0x7f160c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f160b7fe640 1 -- 172.21.15.46:0/2058985030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c074f70 con 0x7f160c107900 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f160b7fe640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c107900 0x7f160c074a30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1600031460 tx=0x7f1600004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f16097fa640 1 -- 172.21.15.46:0/2058985030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f160003f070 con 0x7f160c107900 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f16097fa640 1 -- 172.21.15.46:0/2058985030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f16000035f0 con 0x7f160c107900 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c075200 con 0x7f160c107900 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f16097fa640 1 -- 172.21.15.46:0/2058985030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1600035030 con 0x7f160c107900 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.834+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c1a3d80 con 0x7f160c107900 2024-09-10T04:37:56.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.835+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160c103040 con 0x7f160c107900 2024-09-10T04:37:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.836+0000 7f16097fa640 1 -- 172.21.15.46:0/2058985030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1600042070 con 0x7f160c107900 2024-09-10T04:37:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.836+0000 7f16097fa640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15f407ac40 0x7f15f407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:37:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.837+0000 7f16097fa640 1 -- 172.21.15.46:0/2058985030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f16000be4a0 con 0x7f160c107900 2024-09-10T04:37:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.837+0000 7f160bfff640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15f407ac40 0x7f15f407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:37:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.837+0000 7f160bfff640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15f407ac40 0x7f15f407d100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009bd0 tx=0x7f15fc005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:37:56.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:56.841+0000 7f16097fa640 1 -- 172.21.15.46:0/2058985030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1600089c60 con 0x7f160c107900 2024-09-10T04:37:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:56 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1534679384' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:37:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:56 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3906517734' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:37:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:56 smithi070 ceph-mon[79774]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:56 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1534679384' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:37:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:56 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3906517734' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:37:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:56 smithi046 ceph-mon[109506]: from='client.35618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:37:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.099+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f160c109e70 con 0x7f160c107900 2024-09-10T04:37:57.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.099+0000 7f16097fa640 1 -- 172.21.15.46:0/2058985030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f160008eb10 con 0x7f160c107900 2024-09-10T04:37:57.101 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:37:57.102 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:37:57.103 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:37:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15f407ac40 msgr2=0x7f15f407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15f407ac40 0x7f15f407d100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009bd0 tx=0x7f15fc005f90 comp rx=0 tx=0).stop 2024-09-10T04:37:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c107900 msgr2=0x7f160c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:37:57.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c107900 0x7f160c074a30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1600031460 tx=0x7f1600004530 comp rx=0 tx=0).stop 2024-09-10T04:37:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 shutdown_connections 2024-09-10T04:37:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15f407ac40 0x7f15f407d100 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f160c107900 0x7f160c074a30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 --2- 172.21.15.46:0/2058985030 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f160c101900 0x7f160c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:37:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 >> 172.21.15.46:0/2058985030 conn(0x7f160c0fd600 msgr2=0x7f160c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:37:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 shutdown_connections 2024-09-10T04:37:57.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:37:57.102+0000 7f16122fa640 1 -- 172.21.15.46:0/2058985030 wait complete. 2024-09-10T04:37:58.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:57 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2058985030' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:37:58.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:37:57 smithi046 ceph-mon[109506]: pgmap v970: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:37:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:57 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2058985030' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:37:58.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:37:57 smithi070 ceph-mon[79774]: pgmap v970: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:00 smithi070 ceph-mon[79774]: pgmap v971: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:00 smithi046 ceph-mon[109506]: pgmap v971: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:02 smithi070 ceph-mon[79774]: pgmap v972: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:02 smithi046 ceph-mon[109506]: pgmap v972: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:04 smithi070 ceph-mon[79774]: pgmap v973: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:04 smithi046 ceph-mon[109506]: pgmap v973: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:06 smithi070 ceph-mon[79774]: pgmap v974: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:06 smithi046 ceph-mon[109506]: pgmap v974: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:08 smithi070 ceph-mon[79774]: pgmap v975: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:08 smithi046 ceph-mon[109506]: pgmap v975: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:10 smithi070 ceph-mon[79774]: pgmap v976: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:10 smithi046 ceph-mon[109506]: pgmap v976: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:12 smithi070 ceph-mon[79774]: pgmap v977: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:12 smithi046 ceph-mon[109506]: pgmap v977: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:14 smithi070 ceph-mon[79774]: pgmap v978: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:14 smithi046 ceph-mon[109506]: pgmap v978: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:16 smithi070 ceph-mon[79774]: pgmap v979: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:16 smithi046 ceph-mon[109506]: pgmap v979: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:18 smithi070 ceph-mon[79774]: pgmap v980: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:18 smithi046 ceph-mon[109506]: pgmap v980: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:20 smithi070 ceph-mon[79774]: pgmap v981: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:20 smithi046 ceph-mon[109506]: pgmap v981: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:22 smithi070 ceph-mon[79774]: pgmap v982: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:22 smithi046 ceph-mon[109506]: pgmap v982: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:24.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:24 smithi046 ceph-mon[109506]: pgmap v983: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:24 smithi070 ceph-mon[79774]: pgmap v983: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:26 smithi070 ceph-mon[79774]: pgmap v984: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:26 smithi046 ceph-mon[109506]: pgmap v984: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 -- 172.21.15.46:0/3085150836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 msgr2=0x7f6788106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 --2- 172.21.15.46:0/3085150836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f6788106be0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6778009880 tx=0x7f677802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 -- 172.21.15.46:0/3085150836 shutdown_connections 2024-09-10T04:38:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 --2- 172.21.15.46:0/3085150836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6788100800 0x7f6788100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 --2- 172.21.15.46:0/3085150836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f6788106be0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 -- 172.21.15.46:0/3085150836 >> 172.21.15.46:0/3085150836 conn(0x7f67880fc520 msgr2=0x7f67880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 -- 172.21.15.46:0/3085150836 shutdown_connections 2024-09-10T04:38:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 -- 172.21.15.46:0/3085150836 wait complete. 2024-09-10T04:38:27.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.200+0000 7f678dec2640 1 Processor -- start 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f678dec2640 1 -- start start 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f678dec2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6788100800 0x7f67880fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f678dec2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f67880fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f678dec2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f678810dcd0 con 0x7f6788106800 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f678dec2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f678810db50 con 0x7f6788100800 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f6786ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f67880fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f6786ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f67880fd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53360/0 (socket says 172.21.15.46:53360) 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f6786ffd640 1 -- 172.21.15.46:0/786775648 learned_addr learned my addr 172.21.15.46:0/786775648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.201+0000 7f67877fe640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6788100800 0x7f67880fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f6786ffd640 1 -- 172.21.15.46:0/786775648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6788100800 msgr2=0x7f67880fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f6786ffd640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6788100800 0x7f67880fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f6786ffd640 1 -- 172.21.15.46:0/786775648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67880fea20 con 0x7f6788106800 2024-09-10T04:38:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f6786ffd640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f67880fd1e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f677c00d6f0 tx=0x7f677c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f6784ff9640 1 -- 172.21.15.46:0/786775648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f677c017070 con 0x7f6788106800 2024-09-10T04:38:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67880fd720 con 0x7f6788106800 2024-09-10T04:38:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f6784ff9640 1 -- 172.21.15.46:0/786775648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f677c00de00 con 0x7f6788106800 2024-09-10T04:38:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f6784ff9640 1 -- 172.21.15.46:0/786775648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f677c004bb0 con 0x7f6788106800 2024-09-10T04:38:27.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.202+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a8350 con 0x7f6788106800 2024-09-10T04:38:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.203+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6788101f40 con 0x7f6788106800 2024-09-10T04:38:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.204+0000 7f6784ff9640 1 -- 172.21.15.46:0/786775648 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f677c004e60 con 0x7f6788106800 2024-09-10T04:38:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.205+0000 7f6784ff9640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677007abf0 0x7f677007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.205+0000 7f6784ff9640 1 -- 172.21.15.46:0/786775648 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f677c09d8c0 con 0x7f6788106800 2024-09-10T04:38:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.205+0000 7f67877fe640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677007abf0 0x7f677007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.205+0000 7f67877fe640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677007abf0 0x7f677007d0b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f67780096c0 tx=0x7f6778031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:27.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.209+0000 7f6784ff9640 1 -- 172.21.15.46:0/786775648 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f677c068f60 con 0x7f6788106800 2024-09-10T04:38:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.413+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6788062630 con 0x7f677007abf0 2024-09-10T04:38:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.414+0000 7f6784ff9640 1 -- 172.21.15.46:0/786775648 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f6788062630 con 0x7f677007abf0 2024-09-10T04:38:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677007abf0 msgr2=0x7f677007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677007abf0 0x7f677007d0b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f67780096c0 tx=0x7f6778031400 comp rx=0 tx=0).stop 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 msgr2=0x7f67880fd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f67880fd1e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f677c00d6f0 tx=0x7f677c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 shutdown_connections 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677007abf0 0x7f677007d0b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6788106800 0x7f67880fd1e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 --2- 172.21.15.46:0/786775648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6788100800 0x7f67880fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 >> 172.21.15.46:0/786775648 conn(0x7f67880fc520 msgr2=0x7f678806a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 shutdown_connections 2024-09-10T04:38:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.416+0000 7f678dec2640 1 -- 172.21.15.46:0/786775648 wait complete. 2024-09-10T04:38:27.427 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 -- 172.21.15.46:0/1671941672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 msgr2=0x7ffbc810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 --2- 172.21.15.46:0/1671941672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc810b8d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009880 tx=0x7ffbbc02a530 comp rx=0 tx=0).stop 2024-09-10T04:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 -- 172.21.15.46:0/1671941672 shutdown_connections 2024-09-10T04:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 --2- 172.21.15.46:0/1671941672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc810b8d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 --2- 172.21.15.46:0/1671941672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc8102ac0 0x7ffbc8102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 -- 172.21.15.46:0/1671941672 >> 172.21.15.46:0/1671941672 conn(0x7ffbc80fc520 msgr2=0x7ffbc80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 -- 172.21.15.46:0/1671941672 shutdown_connections 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.502+0000 7ffbd076c640 1 -- 172.21.15.46:0/1671941672 wait complete. 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbd076c640 1 Processor -- start 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbd076c640 1 -- start start 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbd076c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc8102ac0 0x7ffbc8100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbd076c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc8100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbd076c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc810edf0 con 0x7ffbc81033e0 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbd076c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc810ec70 con 0x7ffbc8102ac0 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbcdd07640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc8100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbcdd07640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc8100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53372/0 (socket says 172.21.15.46:53372) 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbcdd07640 1 -- 172.21.15.46:0/3068287561 learned_addr learned my addr 172.21.15.46:0/3068287561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbce508640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc8102ac0 0x7ffbc8100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbcdd07640 1 -- 172.21.15.46:0/3068287561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc8102ac0 msgr2=0x7ffbc8100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbcdd07640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc8102ac0 0x7ffbc8100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.503+0000 7ffbcdd07640 1 -- 172.21.15.46:0/3068287561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc81011a0 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.504+0000 7ffbcdd07640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc8100c60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc031460 tx=0x7ffbbc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.504+0000 7ffbc77fe640 1 -- 172.21.15.46:0/3068287561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffbbc03f070 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.504+0000 7ffbc77fe640 1 -- 172.21.15.46:0/3068287561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffbbc0035f0 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.504+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc8101430 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.504+0000 7ffbc77fe640 1 -- 172.21.15.46:0/3068287561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ffbbc035030 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.504+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc81a3de0 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.505+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc806aa90 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.506+0000 7ffbc77fe640 1 -- 172.21.15.46:0/3068287561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ffbbc042070 con 0x7ffbc81033e0 2024-09-10T04:38:27.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.506+0000 7ffbc77fe640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffbb007ac40 0x7ffbb007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.506+0000 7ffbce508640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffbb007ac40 0x7ffbb007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.506+0000 7ffbc77fe640 1 -- 172.21.15.46:0/3068287561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ffbbc0be650 con 0x7ffbc81033e0 2024-09-10T04:38:27.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.507+0000 7ffbce508640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffbb007ac40 0x7ffbb007d100 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8005fd0 tx=0x7ffbb8009680 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:27.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.512+0000 7ffbc77fe640 1 -- 172.21.15.46:0/3068287561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ffbbc089cf0 con 0x7ffbc81033e0 2024-09-10T04:38:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.715+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ffbc8062630 con 0x7ffbb007ac40 2024-09-10T04:38:27.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.716+0000 7ffbc77fe640 1 -- 172.21.15.46:0/3068287561 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ffbc8062630 con 0x7ffbb007ac40 2024-09-10T04:38:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffbb007ac40 msgr2=0x7ffbb007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffbb007ac40 0x7ffbb007d100 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8005fd0 tx=0x7ffbb8009680 comp rx=0 tx=0).stop 2024-09-10T04:38:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 msgr2=0x7ffbc8100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc8100c60 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc031460 tx=0x7ffbbc004530 comp rx=0 tx=0).stop 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 shutdown_connections 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ffbb007ac40 0x7ffbb007d100 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbc81033e0 0x7ffbc8100c60 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 --2- 172.21.15.46:0/3068287561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc8102ac0 0x7ffbc8100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 >> 172.21.15.46:0/3068287561 conn(0x7ffbc80fc520 msgr2=0x7ffbc8108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.718+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 shutdown_connections 2024-09-10T04:38:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.719+0000 7ffbd076c640 1 -- 172.21.15.46:0/3068287561 wait complete. 2024-09-10T04:38:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.802+0000 7f4c0c34a640 1 -- 172.21.15.46:0/3972845613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04100800 msgr2=0x7f4c04100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.802+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/3972845613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04100800 0x7f4c04100c60 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009880 tx=0x7f4bf402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.802+0000 7f4c0c34a640 1 -- 172.21.15.46:0/3972845613 shutdown_connections 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/3972845613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04100800 0x7f4c04100c60 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/3972845613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04106800 0x7f4c04106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 -- 172.21.15.46:0/3972845613 >> 172.21.15.46:0/3972845613 conn(0x7f4c040fc520 msgr2=0x7f4c040fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 -- 172.21.15.46:0/3972845613 shutdown_connections 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 -- 172.21.15.46:0/3972845613 wait complete. 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 Processor -- start 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 -- start start 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04100800 0x7f4c0419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04106800 0x7f4c0419dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.803+0000 7f4c0c34a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0410dcb0 con 0x7f4c04106800 2024-09-10T04:38:27.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c0c34a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0410db30 con 0x7f4c04100800 2024-09-10T04:38:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c098e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04106800 0x7f4c0419dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c098e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04106800 0x7f4c0419dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53384/0 (socket says 172.21.15.46:53384) 2024-09-10T04:38:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c0a0e6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04100800 0x7f4c0419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c098e5640 1 -- 172.21.15.46:0/1343249081 learned_addr learned my addr 172.21.15.46:0/1343249081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c098e5640 1 -- 172.21.15.46:0/1343249081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04100800 msgr2=0x7f4c0419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c098e5640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04100800 0x7f4c0419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c098e5640 1 -- 172.21.15.46:0/1343249081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c041977f0 con 0x7f4c04106800 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c098e5640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04106800 0x7f4c0419dbb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4031460 tx=0x7f4bf40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4bfb7fe640 1 -- 172.21.15.46:0/1343249081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4bf403f070 con 0x7f4c04106800 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c04197a80 con 0x7f4c04106800 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.804+0000 7f4bfb7fe640 1 -- 172.21.15.46:0/1343249081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4bf4004cf0 con 0x7f4c04106800 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.805+0000 7f4bfb7fe640 1 -- 172.21.15.46:0/1343249081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4bf4035030 con 0x7f4c04106800 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.805+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c04197fb0 con 0x7f4c04106800 2024-09-10T04:38:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.806+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c04101f40 con 0x7f4c04106800 2024-09-10T04:38:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.806+0000 7f4bfb7fe640 1 -- 172.21.15.46:0/1343249081 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4bf4042070 con 0x7f4c04106800 2024-09-10T04:38:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.807+0000 7f4bfb7fe640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4bf007abf0 0x7f4bf007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.807+0000 7f4bfb7fe640 1 -- 172.21.15.46:0/1343249081 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4bf40be630 con 0x7f4c04106800 2024-09-10T04:38:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.807+0000 7f4c0a0e6640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4bf007abf0 0x7f4bf007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:27.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.807+0000 7f4c0a0e6640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4bf007abf0 0x7f4bf007d0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000abe0 tx=0x7f4c0000a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:27.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:27.811+0000 7f4bfb7fe640 1 -- 172.21.15.46:0/1343249081 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4bf4089d70 con 0x7f4c04106800 2024-09-10T04:38:28.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.005+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4c04198d20 con 0x7f4bf007abf0 2024-09-10T04:38:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.014+0000 7f4bfb7fe640 1 -- 172.21.15.46:0/1343249081 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f4c04198d20 con 0x7f4bf007abf0 2024-09-10T04:38:28.016 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (30m) 9m ago 41m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (32m) 9m ago 41m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (32m) 9m ago 40m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (28m) 9m ago 41m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (28m) 9m ago 40m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (29m) 9m ago 41m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (24m) 9m ago 35m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (24m) 9m ago 36m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (24m) 9m ago 36m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (9m) 9m ago 35m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (32m) 9m ago 42m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (31m) 9m ago 40m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:38:28.017 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (28m) 9m ago 42m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (28m) 9m ago 40m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (30m) 9m ago 41m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (30m) 9m ago 40m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (28m) 9m ago 39m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (27m) 9m ago 38m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (26m) 9m ago 38m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (26m) 9m ago 37m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (25m) 9m ago 37m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (25m) 9m ago 37m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:38:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (30m) 9m ago 41m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:38:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4bf007abf0 msgr2=0x7f4bf007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4bf007abf0 0x7f4bf007d0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000abe0 tx=0x7f4c0000a2d0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04106800 msgr2=0x7f4c0419dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04106800 0x7f4c0419dbb0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4031460 tx=0x7f4bf40048f0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 shutdown_connections 2024-09-10T04:38:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4bf007abf0 0x7f4bf007d0b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c04106800 0x7f4c0419dbb0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 --2- 172.21.15.46:0/1343249081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c04100800 0x7f4c0419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.016+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 >> 172.21.15.46:0/1343249081 conn(0x7f4c040fc520 msgr2=0x7f4c0406a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.017+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 shutdown_connections 2024-09-10T04:38:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.017+0000 7f4c0c34a640 1 -- 172.21.15.46:0/1343249081 wait complete. 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 -- 172.21.15.46:0/3764365681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381067f0 msgr2=0x7fde38106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/3764365681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381067f0 0x7fde38106bd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fde28009880 tx=0x7fde2802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 -- 172.21.15.46:0/3764365681 shutdown_connections 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/3764365681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde381007f0 0x7fde38100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/3764365681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381067f0 0x7fde38106bd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 -- 172.21.15.46:0/3764365681 >> 172.21.15.46:0/3764365681 conn(0x7fde380fc550 msgr2=0x7fde380fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 -- 172.21.15.46:0/3764365681 shutdown_connections 2024-09-10T04:38:28.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.100+0000 7fde3cc7f640 1 -- 172.21.15.46:0/3764365681 wait complete. 2024-09-10T04:38:28.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3cc7f640 1 Processor -- start 2024-09-10T04:38:28.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3cc7f640 1 -- start start 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3cc7f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381007f0 0x7fde380fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3cc7f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde381067f0 0x7fde380fd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3cc7f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde3810dd20 con 0x7fde381007f0 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3cc7f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fde3810dba0 con 0x7fde381067f0 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381007f0 0x7fde380fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381007f0 0x7fde380fcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53398/0 (socket says 172.21.15.46:53398) 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3659c640 1 -- 172.21.15.46:0/2544052597 learned_addr learned my addr 172.21.15.46:0/2544052597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3659c640 1 -- 172.21.15.46:0/2544052597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde381067f0 msgr2=0x7fde380fd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde2ffff640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde381067f0 0x7fde380fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3659c640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde381067f0 0x7fde380fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.101+0000 7fde3659c640 1 -- 172.21.15.46:0/2544052597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde380fd7a0 con 0x7fde381007f0 2024-09-10T04:38:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.102+0000 7fde3659c640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381007f0 0x7fde380fcd20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fde28031460 tx=0x7fde280048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.102+0000 7fde2f7fe640 1 -- 172.21.15.46:0/2544052597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fde2803f070 con 0x7fde381007f0 2024-09-10T04:38:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.102+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde380fda30 con 0x7fde381007f0 2024-09-10T04:38:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.102+0000 7fde2f7fe640 1 -- 172.21.15.46:0/2544052597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde28004cf0 con 0x7fde381007f0 2024-09-10T04:38:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.102+0000 7fde2f7fe640 1 -- 172.21.15.46:0/2544052597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fde28035030 con 0x7fde381007f0 2024-09-10T04:38:28.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.102+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381a3dc0 con 0x7fde381007f0 2024-09-10T04:38:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.103+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde38101f30 con 0x7fde381007f0 2024-09-10T04:38:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.104+0000 7fde2f7fe640 1 -- 172.21.15.46:0/2544052597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fde28042070 con 0x7fde381007f0 2024-09-10T04:38:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.104+0000 7fde2f7fe640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde2407abf0 0x7fde2407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.104+0000 7fde2f7fe640 1 -- 172.21.15.46:0/2544052597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fde280be7b0 con 0x7fde381007f0 2024-09-10T04:38:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.104+0000 7fde2ffff640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde2407abf0 0x7fde2407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.105+0000 7fde2ffff640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde2407abf0 0x7fde2407d0b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fde380fe7b0 tx=0x7fde20008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:28.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.109+0000 7fde2f7fe640 1 -- 172.21.15.46:0/2544052597 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fde28089ef0 con 0x7fde381007f0 2024-09-10T04:38:28.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.367+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fde38109d70 con 0x7fde381007f0 2024-09-10T04:38:28.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.368+0000 7fde2f7fe640 1 -- 172.21.15.46:0/2544052597 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fde2808eda0 con 0x7fde381007f0 2024-09-10T04:38:28.370 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:38:28.370 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:38:28.370 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:38:28.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:38:28.371 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:38:28.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde2407abf0 msgr2=0x7fde2407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde2407abf0 0x7fde2407d0b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fde380fe7b0 tx=0x7fde20008040 comp rx=0 tx=0).stop 2024-09-10T04:38:28.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381007f0 msgr2=0x7fde380fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381007f0 0x7fde380fcd20 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fde28031460 tx=0x7fde280048f0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 shutdown_connections 2024-09-10T04:38:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fde2407abf0 0x7fde2407d0b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde381067f0 0x7fde380fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 --2- 172.21.15.46:0/2544052597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde381007f0 0x7fde380fcd20 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 >> 172.21.15.46:0/2544052597 conn(0x7fde380fc550 msgr2=0x7fde38069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:28.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 shutdown_connections 2024-09-10T04:38:28.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.371+0000 7fde3cc7f640 1 -- 172.21.15.46:0/2544052597 wait complete. 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 -- 172.21.15.46:0/2069711765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 msgr2=0x7fda9c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 --2- 172.21.15.46:0/2069711765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c100c60 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fda90009880 tx=0x7fda9002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 -- 172.21.15.46:0/2069711765 shutdown_connections 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 --2- 172.21.15.46:0/2069711765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c100c60 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 --2- 172.21.15.46:0/2069711765 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9c106800 0x7fda9c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 -- 172.21.15.46:0/2069711765 >> 172.21.15.46:0/2069711765 conn(0x7fda9c0fc520 msgr2=0x7fda9c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 -- 172.21.15.46:0/2069711765 shutdown_connections 2024-09-10T04:38:28.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 -- 172.21.15.46:0/2069711765 wait complete. 2024-09-10T04:38:28.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.456+0000 7fdaa175e640 1 Processor -- start 2024-09-10T04:38:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fdaa175e640 1 -- start start 2024-09-10T04:38:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fdaa175e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fdaa175e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9c106800 0x7fda9c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fdaa175e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10dcd0 con 0x7fda9c100800 2024-09-10T04:38:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fdaa175e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10db50 con 0x7fda9c106800 2024-09-10T04:38:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fda9affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fda9affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53428/0 (socket says 172.21.15.46:53428) 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fda9affd640 1 -- 172.21.15.46:0/897809623 learned_addr learned my addr 172.21.15.46:0/897809623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fda9a7fc640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9c106800 0x7fda9c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fda9affd640 1 -- 172.21.15.46:0/897809623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9c106800 msgr2=0x7fda9c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fda9affd640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9c106800 0x7fda9c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.457+0000 7fda9affd640 1 -- 172.21.15.46:0/897809623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c199a10 con 0x7fda9c100800 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.458+0000 7fda9affd640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c19f850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00d6f0 tx=0x7fda8c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.458+0000 7fda83fff640 1 -- 172.21.15.46:0/897809623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda8c017070 con 0x7fda9c100800 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.458+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c199d00 con 0x7fda9c100800 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.458+0000 7fda83fff640 1 -- 172.21.15.46:0/897809623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fda8c00de00 con 0x7fda9c100800 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.458+0000 7fda83fff640 1 -- 172.21.15.46:0/897809623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fda8c004bb0 con 0x7fda9c100800 2024-09-10T04:38:28.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.458+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c19a260 con 0x7fda9c100800 2024-09-10T04:38:28.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.459+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c101f40 con 0x7fda9c100800 2024-09-10T04:38:28.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.460+0000 7fda83fff640 1 -- 172.21.15.46:0/897809623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fda8c004e60 con 0x7fda9c100800 2024-09-10T04:38:28.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.460+0000 7fda83fff640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda8407abf0 0x7fda8407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.460+0000 7fda83fff640 1 -- 172.21.15.46:0/897809623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fda8c09d8d0 con 0x7fda9c100800 2024-09-10T04:38:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.460+0000 7fda9a7fc640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda8407abf0 0x7fda8407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.461+0000 7fda9a7fc640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda8407abf0 0x7fda8407d0b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda9c19b200 tx=0x7fda90031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:28.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.465+0000 7fda83fff640 1 -- 172.21.15.46:0/897809623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fda8c069010 con 0x7fda9c100800 2024-09-10T04:38:28.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.688+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fda9c19af40 con 0x7fda9c100800 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.689+0000 7fda83fff640 1 -- 172.21.15.46:0/897809623 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fda8c06dec0 con 0x7fda9c100800 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:38:28.691 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:38:28.692 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:38:28.693 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:38:28.694 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:38:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:38:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda8407abf0 msgr2=0x7fda8407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda8407abf0 0x7fda8407d0b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fda9c19b200 tx=0x7fda90031400 comp rx=0 tx=0).stop 2024-09-10T04:38:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 msgr2=0x7fda9c19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c19f850 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00d6f0 tx=0x7fda8c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 shutdown_connections 2024-09-10T04:38:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fda8407abf0 0x7fda8407d0b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9c106800 0x7fda9c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 --2- 172.21.15.46:0/897809623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda9c100800 0x7fda9c19f850 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 >> 172.21.15.46:0/897809623 conn(0x7fda9c0fc520 msgr2=0x7fda9c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 shutdown_connections 2024-09-10T04:38:28.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.691+0000 7fdaa175e640 1 -- 172.21.15.46:0/897809623 wait complete. 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.776+0000 7f526785a640 1 -- 172.21.15.46:0/1325576410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260101910 msgr2=0x7f5260101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.776+0000 7f526785a640 1 --2- 172.21.15.46:0/1325576410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260101910 0x7f5260101d70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5250009880 tx=0x7f525002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 -- 172.21.15.46:0/1325576410 shutdown_connections 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 --2- 172.21.15.46:0/1325576410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260101910 0x7f5260101d70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 --2- 172.21.15.46:0/1325576410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5260107910 0x7f5260107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 -- 172.21.15.46:0/1325576410 >> 172.21.15.46:0/1325576410 conn(0x7f52600fd630 msgr2=0x7f52600ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 -- 172.21.15.46:0/1325576410 shutdown_connections 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 -- 172.21.15.46:0/1325576410 wait complete. 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 Processor -- start 2024-09-10T04:38:28.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.777+0000 7f526785a640 1 -- start start 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f526785a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5260101910 0x7f526019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f526785a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260107910 0x7f526019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f526785a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f526010edc0 con 0x7f5260107910 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f526785a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f526010ec40 con 0x7f5260101910 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f5264df5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260107910 0x7f526019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f5264df5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260107910 0x7f526019dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53450/0 (socket says 172.21.15.46:53450) 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f5264df5640 1 -- 172.21.15.46:0/3471579423 learned_addr learned my addr 172.21.15.46:0/3471579423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:28.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f52655f6640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5260101910 0x7f526019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f5264df5640 1 -- 172.21.15.46:0/3471579423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5260101910 msgr2=0x7f526019d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f5264df5640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5260101910 0x7f526019d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f5264df5640 1 -- 172.21.15.46:0/3471579423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5260197770 con 0x7f5260107910 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f5264df5640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260107910 0x7f526019dbe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5250031460 tx=0x7f52500045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f52567fc640 1 -- 172.21.15.46:0/3471579423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f525003f070 con 0x7f5260107910 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f52567fc640 1 -- 172.21.15.46:0/3471579423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52500035f0 con 0x7f5260107910 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5260197a00 con 0x7f5260107910 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.778+0000 7f52567fc640 1 -- 172.21.15.46:0/3471579423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5250035030 con 0x7f5260107910 2024-09-10T04:38:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.779+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5260197f00 con 0x7f5260107910 2024-09-10T04:38:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.780+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5260103050 con 0x7f5260107910 2024-09-10T04:38:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.780+0000 7f52567fc640 1 -- 172.21.15.46:0/3471579423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5250042070 con 0x7f5260107910 2024-09-10T04:38:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.781+0000 7f52567fc640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f524c07abf0 0x7f524c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.781+0000 7f52567fc640 1 -- 172.21.15.46:0/3471579423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f52500be5d0 con 0x7f5260107910 2024-09-10T04:38:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.781+0000 7f52655f6640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f524c07abf0 0x7f524c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.781+0000 7f52655f6640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f524c07abf0 0x7f524c07d0b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f525c005fd0 tx=0x7f525c009680 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:28.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.785+0000 7f52567fc640 1 -- 172.21.15.46:0/3471579423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5250089d10 con 0x7f5260107910 2024-09-10T04:38:28.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.988+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f52601981b0 con 0x7f524c07abf0 2024-09-10T04:38:28.990 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:28 smithi046 ceph-mon[109506]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:28.991 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:28 smithi046 ceph-mon[109506]: pgmap v985: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:28.991 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:28 smithi046 ceph-mon[109506]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:28.991 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:28 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2544052597' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:38:28.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.989+0000 7f52567fc640 1 -- 172.21.15.46:0/3471579423 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f52601981b0 con 0x7f524c07abf0 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:38:28.992 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:38:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:38:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:38:28.993 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:38:28.993 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.991+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f524c07abf0 msgr2=0x7f524c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.991+0000 7f526785a640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f524c07abf0 0x7f524c07d0b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f525c005fd0 tx=0x7f525c009680 comp rx=0 tx=0).stop 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.991+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260107910 msgr2=0x7f526019dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.991+0000 7f526785a640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260107910 0x7f526019dbe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5250031460 tx=0x7f52500045e0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.992+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 shutdown_connections 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.992+0000 7f526785a640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f524c07abf0 0x7f524c07d0b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.992+0000 7f526785a640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5260107910 0x7f526019dbe0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.992+0000 7f526785a640 1 --2- 172.21.15.46:0/3471579423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5260101910 0x7f526019d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:28.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.992+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 >> 172.21.15.46:0/3471579423 conn(0x7f52600fd630 msgr2=0x7f52600fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.992+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 shutdown_connections 2024-09-10T04:38:28.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:28.992+0000 7f526785a640 1 -- 172.21.15.46:0/3471579423 wait complete. 2024-09-10T04:38:29.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.075+0000 7fcdf400e640 1 -- 172.21.15.46:0/998363772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069d90 msgr2=0x7fcdec100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.075+0000 7fcdf400e640 1 --2- 172.21.15.46:0/998363772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069d90 0x7fcdec100790 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009880 tx=0x7fcddc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 -- 172.21.15.46:0/998363772 shutdown_connections 2024-09-10T04:38:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 --2- 172.21.15.46:0/998363772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069d90 0x7fcdec100790 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 --2- 172.21.15.46:0/998363772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdec069920 0x7fcdec100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 -- 172.21.15.46:0/998363772 >> 172.21.15.46:0/998363772 conn(0x7fcdec0fbc90 msgr2=0x7fcdec0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 -- 172.21.15.46:0/998363772 shutdown_connections 2024-09-10T04:38:29.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 -- 172.21.15.46:0/998363772 wait complete. 2024-09-10T04:38:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 Processor -- start 2024-09-10T04:38:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 -- start start 2024-09-10T04:38:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.076+0000 7fcdf400e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069920 0x7fcdec0fc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf400e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdec069d90 0x7fcdec0fc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf400e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec10dd50 con 0x7fcdec069920 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf400e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec10dbd0 con 0x7fcdec069d90 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf1daa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069920 0x7fcdec0fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf1daa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069920 0x7fcdec0fc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47412/0 (socket says 172.21.15.46:47412) 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf1daa640 1 -- 172.21.15.46:0/1786453177 learned_addr learned my addr 172.21.15.46:0/1786453177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf15a9640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdec069d90 0x7fcdec0fc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf1daa640 1 -- 172.21.15.46:0/1786453177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdec069d90 msgr2=0x7fcdec0fc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf1daa640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdec069d90 0x7fcdec0fc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf1daa640 1 -- 172.21.15.46:0/1786453177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec0fcdc0 con 0x7fcdec069920 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf1daa640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069920 0x7fcdec0fc340 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e970 tx=0x7fcde800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcde2ffd640 1 -- 172.21.15.46:0/1786453177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcde800cac0 con 0x7fcdec069920 2024-09-10T04:38:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcde2ffd640 1 -- 172.21.15.46:0/1786453177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcde80054e0 con 0x7fcdec069920 2024-09-10T04:38:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcde2ffd640 1 -- 172.21.15.46:0/1786453177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcde8010430 con 0x7fcdec069920 2024-09-10T04:38:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.077+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec0fd0b0 con 0x7fcdec069920 2024-09-10T04:38:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.078+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1a3ec0 con 0x7fcdec069920 2024-09-10T04:38:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.078+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec101730 con 0x7fcdec069920 2024-09-10T04:38:29.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.079+0000 7fcde2ffd640 1 -- 172.21.15.46:0/1786453177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcde8004e80 con 0x7fcdec069920 2024-09-10T04:38:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.080+0000 7fcde2ffd640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdd807abf0 0x7fcdd807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.080+0000 7fcde2ffd640 1 -- 172.21.15.46:0/1786453177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcde809e420 con 0x7fcdec069920 2024-09-10T04:38:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.080+0000 7fcdf15a9640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdd807abf0 0x7fcdd807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:29.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.080+0000 7fcdf15a9640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdd807abf0 0x7fcdd807d0b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcddc031460 tx=0x7fcddc009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:28 smithi070 ceph-mon[79774]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:28 smithi070 ceph-mon[79774]: pgmap v985: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:28 smithi070 ceph-mon[79774]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:28 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2544052597' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:38:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.084+0000 7fcde2ffd640 1 -- 172.21.15.46:0/1786453177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fcde8069ac0 con 0x7fcdec069920 2024-09-10T04:38:29.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.345+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fcdec10a5f0 con 0x7fcdec069920 2024-09-10T04:38:29.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.346+0000 7fcde2ffd640 1 -- 172.21.15.46:0/1786453177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fcde806e970 con 0x7fcdec069920 2024-09-10T04:38:29.348 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:38:29.348 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:38:29.348 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:38:29.348 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:38:29.348 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:38:29.349 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:38:29.350 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdd807abf0 msgr2=0x7fcdd807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdd807abf0 0x7fcdd807d0b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcddc031460 tx=0x7fcddc009560 comp rx=0 tx=0).stop 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069920 msgr2=0x7fcdec0fc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069920 0x7fcdec0fc340 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e970 tx=0x7fcde800ed70 comp rx=0 tx=0).stop 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 shutdown_connections 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcdd807abf0 0x7fcdd807d0b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdec069d90 0x7fcdec0fc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 --2- 172.21.15.46:0/1786453177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdec069920 0x7fcdec0fc340 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 >> 172.21.15.46:0/1786453177 conn(0x7fcdec0fbc90 msgr2=0x7fcdec1035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:29.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 shutdown_connections 2024-09-10T04:38:29.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:29.348+0000 7fcdf400e640 1 -- 172.21.15.46:0/1786453177 wait complete. 2024-09-10T04:38:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:29 smithi070 ceph-mon[79774]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:29 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/897809623' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:38:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:29 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1786453177' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:38:30.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:29 smithi046 ceph-mon[109506]: from='client.35634 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:30.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:29 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/897809623' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:38:30.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:29 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1786453177' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:38:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:30 smithi070 ceph-mon[79774]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:30 smithi070 ceph-mon[79774]: pgmap v986: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:30 smithi046 ceph-mon[109506]: from='client.35646 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:38:31.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:30 smithi046 ceph-mon[109506]: pgmap v986: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:32 smithi070 ceph-mon[79774]: pgmap v987: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:32 smithi046 ceph-mon[109506]: pgmap v987: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:34 smithi070 ceph-mon[79774]: pgmap v988: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:34 smithi046 ceph-mon[109506]: pgmap v988: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:36 smithi070 ceph-mon[79774]: pgmap v989: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:36 smithi046 ceph-mon[109506]: pgmap v989: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:38 smithi070 ceph-mon[79774]: pgmap v990: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:38 smithi046 ceph-mon[109506]: pgmap v990: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:40 smithi070 ceph-mon[79774]: pgmap v991: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:40 smithi046 ceph-mon[109506]: pgmap v991: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:42 smithi070 ceph-mon[79774]: pgmap v992: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:42 smithi046 ceph-mon[109506]: pgmap v992: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:44 smithi070 ceph-mon[79774]: pgmap v993: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:44 smithi046 ceph-mon[109506]: pgmap v993: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:46 smithi070 ceph-mon[79774]: pgmap v994: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:46 smithi046 ceph-mon[109506]: pgmap v994: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:48 smithi070 ceph-mon[79774]: pgmap v995: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:48 smithi046 ceph-mon[109506]: pgmap v995: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:50.970 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:50 smithi070 ceph-mon[79774]: pgmap v996: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:50 smithi046 ceph-mon[109506]: pgmap v996: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:52 smithi070 ceph-mon[79774]: pgmap v997: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:52 smithi046 ceph-mon[109506]: pgmap v997: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:38:55.020 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:54 smithi046 ceph-mon[109506]: pgmap v998: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:54 smithi070 ceph-mon[79774]: pgmap v998: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:56 smithi070 ceph-mon[79774]: pgmap v999: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:56 smithi046 ceph-mon[109506]: pgmap v999: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:38:58 smithi070 ceph-mon[79774]: pgmap v1000: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:38:58 smithi046 ceph-mon[109506]: pgmap v1000: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:38:59.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 -- 172.21.15.46:0/3296018817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 msgr2=0x7ff348101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3296018817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff348101d60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff330009880 tx=0x7ff33002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 -- 172.21.15.46:0/3296018817 shutdown_connections 2024-09-10T04:38:59.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3296018817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff348101d60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3296018817 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff348107900 0x7ff348107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 -- 172.21.15.46:0/3296018817 >> 172.21.15.46:0/3296018817 conn(0x7ff3480fd600 msgr2=0x7ff3480ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:59.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 -- 172.21.15.46:0/3296018817 shutdown_connections 2024-09-10T04:38:59.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.448+0000 7ff34d76a640 1 -- 172.21.15.46:0/3296018817 wait complete. 2024-09-10T04:38:59.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff34d76a640 1 Processor -- start 2024-09-10T04:38:59.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff34d76a640 1 -- start start 2024-09-10T04:38:59.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff34d76a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff3480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff34d76a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff348107900 0x7ff348074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff34d76a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff34810edd0 con 0x7ff348101900 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff34d76a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff34810ec50 con 0x7ff348107900 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff346ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff3480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff346ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff3480744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59230/0 (socket says 172.21.15.46:59230) 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.449+0000 7ff346ffd640 1 -- 172.21.15.46:0/3964464044 learned_addr learned my addr 172.21.15.46:0/3964464044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff3467fc640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff348107900 0x7ff348074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff346ffd640 1 -- 172.21.15.46:0/3964464044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff348107900 msgr2=0x7ff348074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff346ffd640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff348107900 0x7ff348074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff346ffd640 1 -- 172.21.15.46:0/3964464044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff348074f70 con 0x7ff348101900 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff346ffd640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff3480744f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff34000eb10 tx=0x7ff34000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff33ffff640 1 -- 172.21.15.46:0/3964464044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff34000cce0 con 0x7ff348101900 2024-09-10T04:38:59.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff33ffff640 1 -- 172.21.15.46:0/3964464044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff340005020 con 0x7ff348101900 2024-09-10T04:38:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff348075260 con 0x7ff348101900 2024-09-10T04:38:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff33ffff640 1 -- 172.21.15.46:0/3964464044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff340010660 con 0x7ff348101900 2024-09-10T04:38:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.450+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3481a3e20 con 0x7ff348101900 2024-09-10T04:38:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.451+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff348103040 con 0x7ff348101900 2024-09-10T04:38:59.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.452+0000 7ff33ffff640 1 -- 172.21.15.46:0/3964464044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff3400051d0 con 0x7ff348101900 2024-09-10T04:38:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.453+0000 7ff33ffff640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33407ac00 0x7ff33407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.453+0000 7ff33ffff640 1 -- 172.21.15.46:0/3964464044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff34009e550 con 0x7ff348101900 2024-09-10T04:38:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.453+0000 7ff3467fc640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33407ac00 0x7ff33407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:59.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.453+0000 7ff3467fc640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33407ac00 0x7ff33407d0c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff348075f80 tx=0x7ff3300095c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:59.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.457+0000 7ff33ffff640 1 -- 172.21.15.46:0/3964464044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff340069c90 con 0x7ff348101900 2024-09-10T04:38:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.665+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7ff348062630 con 0x7ff33407ac00 2024-09-10T04:38:59.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.666+0000 7ff33ffff640 1 -- 172.21.15.46:0/3964464044 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7ff348062630 con 0x7ff33407ac00 2024-09-10T04:38:59.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33407ac00 msgr2=0x7ff33407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33407ac00 0x7ff33407d0c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff348075f80 tx=0x7ff3300095c0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 msgr2=0x7ff3480744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff3480744f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff34000eb10 tx=0x7ff34000c2d0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 shutdown_connections 2024-09-10T04:38:59.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff33407ac00 0x7ff33407d0c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff348107900 0x7ff348074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 --2- 172.21.15.46:0/3964464044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff348101900 0x7ff3480744f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 >> 172.21.15.46:0/3964464044 conn(0x7ff3480fd600 msgr2=0x7ff3480fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:59.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 shutdown_connections 2024-09-10T04:38:59.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.669+0000 7ff34d76a640 1 -- 172.21.15.46:0/3964464044 wait complete. 2024-09-10T04:38:59.679 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:38:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 -- 172.21.15.46:0/424145013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4106800 msgr2=0x7f5bf4106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/424145013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4106800 0x7f5bf4106be0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009880 tx=0x7f5bdc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 -- 172.21.15.46:0/424145013 shutdown_connections 2024-09-10T04:38:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/424145013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf4100800 0x7f5bf4100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/424145013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4106800 0x7f5bf4106be0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 -- 172.21.15.46:0/424145013 >> 172.21.15.46:0/424145013 conn(0x7f5bf40fc520 msgr2=0x7f5bf40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:59.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 -- 172.21.15.46:0/424145013 shutdown_connections 2024-09-10T04:38:59.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.756+0000 7f5bfae4d640 1 -- 172.21.15.46:0/424145013 wait complete. 2024-09-10T04:38:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bfae4d640 1 Processor -- start 2024-09-10T04:38:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bfae4d640 1 -- start start 2024-09-10T04:38:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bfae4d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4100800 0x7f5bf4107730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bfae4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf4106800 0x7f5bf4107c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:59.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bfae4d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410dcb0 con 0x7f5bf4100800 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bfae4d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf410db30 con 0x7f5bf4106800 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bf8be9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4100800 0x7f5bf4107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bf8be9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4100800 0x7f5bf4107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59254/0 (socket says 172.21.15.46:59254) 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bf8be9640 1 -- 172.21.15.46:0/821224308 learned_addr learned my addr 172.21.15.46:0/821224308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bf3fff640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf4106800 0x7f5bf4107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bf8be9640 1 -- 172.21.15.46:0/821224308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf4106800 msgr2=0x7f5bf4107c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bf8be9640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf4106800 0x7f5bf4107c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.757+0000 7f5bf8be9640 1 -- 172.21.15.46:0/821224308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf41081b0 con 0x7f5bf4100800 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.758+0000 7f5bf3fff640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf4106800 0x7f5bf4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.758+0000 7f5bf8be9640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4100800 0x7f5bf4107730 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc031460 tx=0x7f5bdc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.758+0000 7f5bf1ffb640 1 -- 172.21.15.46:0/821224308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5bdc03f070 con 0x7f5bf4100800 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.758+0000 7f5bf1ffb640 1 -- 172.21.15.46:0/821224308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5bdc0044d0 con 0x7f5bf4100800 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.758+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf4108440 con 0x7f5bf4100800 2024-09-10T04:38:59.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.758+0000 7f5bf1ffb640 1 -- 172.21.15.46:0/821224308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5bdc035030 con 0x7f5bf4100800 2024-09-10T04:38:59.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.758+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf41a3db0 con 0x7f5bf4100800 2024-09-10T04:38:59.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.759+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf4101f40 con 0x7f5bf4100800 2024-09-10T04:38:59.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.760+0000 7f5bf1ffb640 1 -- 172.21.15.46:0/821224308 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5bdc042070 con 0x7f5bf4100800 2024-09-10T04:38:59.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.760+0000 7f5bf1ffb640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5be007ac50 0x7f5be007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:38:59.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.760+0000 7f5bf1ffb640 1 -- 172.21.15.46:0/821224308 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5bdc0be700 con 0x7f5bf4100800 2024-09-10T04:38:59.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.761+0000 7f5bf3fff640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5be007ac50 0x7f5be007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:38:59.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.761+0000 7f5bf3fff640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5be007ac50 0x7f5be007d110 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bf41091c0 tx=0x7f5be4005d90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:38:59.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.765+0000 7f5bf1ffb640 1 -- 172.21.15.46:0/821224308 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5bdc004cf0 con 0x7f5bf4100800 2024-09-10T04:38:59.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.969+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5bf406a9f0 con 0x7f5be007ac50 2024-09-10T04:38:59.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.970+0000 7f5bf1ffb640 1 -- 172.21.15.46:0/821224308 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f5bf406a9f0 con 0x7f5be007ac50 2024-09-10T04:38:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5be007ac50 msgr2=0x7f5be007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5be007ac50 0x7f5be007d110 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bf41091c0 tx=0x7f5be4005d90 comp rx=0 tx=0).stop 2024-09-10T04:38:59.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4100800 msgr2=0x7f5bf4107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4100800 0x7f5bf4107730 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc031460 tx=0x7f5bdc0048f0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 shutdown_connections 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5be007ac50 0x7f5be007d110 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf4106800 0x7f5bf4107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 --2- 172.21.15.46:0/821224308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf4100800 0x7f5bf4107730 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.972+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 >> 172.21.15.46:0/821224308 conn(0x7f5bf40fc520 msgr2=0x7f5bf406a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.973+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 shutdown_connections 2024-09-10T04:38:59.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:38:59.973+0000 7f5bfae4d640 1 -- 172.21.15.46:0/821224308 wait complete. 2024-09-10T04:39:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 -- 172.21.15.46:0/3874426909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c100800 msgr2=0x7f773c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 --2- 172.21.15.46:0/3874426909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c100800 0x7f773c100c60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f772c009880 tx=0x7f772c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 -- 172.21.15.46:0/3874426909 shutdown_connections 2024-09-10T04:39:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 --2- 172.21.15.46:0/3874426909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c100800 0x7f773c100c60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 --2- 172.21.15.46:0/3874426909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773c106800 0x7f773c106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 -- 172.21.15.46:0/3874426909 >> 172.21.15.46:0/3874426909 conn(0x7f773c0fc520 msgr2=0x7f773c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 -- 172.21.15.46:0/3874426909 shutdown_connections 2024-09-10T04:39:00.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.057+0000 7f7743548640 1 -- 172.21.15.46:0/3874426909 wait complete. 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.058+0000 7f7743548640 1 Processor -- start 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.058+0000 7f7743548640 1 -- start start 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.058+0000 7f7743548640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773c100800 0x7f773c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.058+0000 7f7743548640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c106800 0x7f773c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.058+0000 7f7743548640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f773c10dcb0 con 0x7f773c106800 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.058+0000 7f7743548640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f773c10db30 con 0x7f773c100800 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.058+0000 7f7740ae3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c106800 0x7f773c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7740ae3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c106800 0x7f773c0fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59270/0 (socket says 172.21.15.46:59270) 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7740ae3640 1 -- 172.21.15.46:0/3879142583 learned_addr learned my addr 172.21.15.46:0/3879142583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f77412e4640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773c100800 0x7f773c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7740ae3640 1 -- 172.21.15.46:0/3879142583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773c100800 msgr2=0x7f773c0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7740ae3640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773c100800 0x7f773c0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7740ae3640 1 -- 172.21.15.46:0/3879142583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773c0fd650 con 0x7f773c106800 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7740ae3640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c106800 0x7f773c0fd110 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f772c031460 tx=0x7f772c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f77327fc640 1 -- 172.21.15.46:0/3879142583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f772c03f070 con 0x7f773c106800 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f77327fc640 1 -- 172.21.15.46:0/3879142583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f772c0035f0 con 0x7f773c106800 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773c0fd8e0 con 0x7f773c106800 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f77327fc640 1 -- 172.21.15.46:0/3879142583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f772c035030 con 0x7f773c106800 2024-09-10T04:39:00.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.059+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773c1a3db0 con 0x7f773c106800 2024-09-10T04:39:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.060+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773c101f40 con 0x7f773c106800 2024-09-10T04:39:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.061+0000 7f77327fc640 1 -- 172.21.15.46:0/3879142583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f772c042070 con 0x7f773c106800 2024-09-10T04:39:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.061+0000 7f77327fc640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f772807abf0 0x7f772807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.062+0000 7f77327fc640 1 -- 172.21.15.46:0/3879142583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f772c0be5d0 con 0x7f773c106800 2024-09-10T04:39:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.062+0000 7f77412e4640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f772807abf0 0x7f772807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.062+0000 7f77412e4640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f772807abf0 0x7f772807d0b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7738005d00 tx=0x7f773800b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:00.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.066+0000 7f77327fc640 1 -- 172.21.15.46:0/3879142583 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f772c089d10 con 0x7f773c106800 2024-09-10T04:39:00.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.261+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f773c062630 con 0x7f772807abf0 2024-09-10T04:39:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.270+0000 7f77327fc640 1 -- 172.21.15.46:0/3879142583 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f773c062630 con 0x7f772807abf0 2024-09-10T04:39:00.271 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (30m) 9m ago 41m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (33m) 9m ago 42m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (32m) 9m ago 41m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (29m) 9m ago 42m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (28m) 9m ago 41m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (30m) 9m ago 41m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (25m) 9m ago 36m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (25m) 9m ago 36m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (24m) 9m ago 36m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (9m) 9m ago 36m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (32m) 9m ago 43m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:39:00.272 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (31m) 9m ago 41m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (29m) 9m ago 43m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (29m) 9m ago 41m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (31m) 9m ago 42m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (31m) 9m ago 41m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (28m) 9m ago 39m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (27m) 9m ago 39m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (27m) 9m ago 38m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (26m) 9m ago 38m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (26m) 9m ago 38m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (25m) 9m ago 37m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:39:00.273 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (30m) 9m ago 41m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:39:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f772807abf0 msgr2=0x7f772807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f772807abf0 0x7f772807d0b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7738005d00 tx=0x7f773800b410 comp rx=0 tx=0).stop 2024-09-10T04:39:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c106800 msgr2=0x7f773c0fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c106800 0x7f773c0fd110 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f772c031460 tx=0x7f772c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 shutdown_connections 2024-09-10T04:39:00.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f772807abf0 0x7f772807d0b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f773c106800 0x7f773c0fd110 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 --2- 172.21.15.46:0/3879142583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773c100800 0x7f773c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 >> 172.21.15.46:0/3879142583 conn(0x7f773c0fc520 msgr2=0x7f773c06a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:00.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 shutdown_connections 2024-09-10T04:39:00.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.273+0000 7f7743548640 1 -- 172.21.15.46:0/3879142583 wait complete. 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 -- 172.21.15.46:0/2334338515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 msgr2=0x7f15d4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 --2- 172.21.15.46:0/2334338515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d4101d60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009c20 tx=0x7f15c802a970 comp rx=0 tx=0).stop 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 -- 172.21.15.46:0/2334338515 shutdown_connections 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 --2- 172.21.15.46:0/2334338515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d4101d60 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 --2- 172.21.15.46:0/2334338515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d4107900 0x7f15d4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 -- 172.21.15.46:0/2334338515 >> 172.21.15.46:0/2334338515 conn(0x7f15d40fd600 msgr2=0x7f15d40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 -- 172.21.15.46:0/2334338515 shutdown_connections 2024-09-10T04:39:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.356+0000 7f15db754640 1 -- 172.21.15.46:0/2334338515 wait complete. 2024-09-10T04:39:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15db754640 1 Processor -- start 2024-09-10T04:39:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15db754640 1 -- start start 2024-09-10T04:39:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15db754640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15db754640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d4107900 0x7f15d419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15db754640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15d410edd0 con 0x7f15d4101900 2024-09-10T04:39:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15db754640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f15d410ec50 con 0x7f15d4107900 2024-09-10T04:39:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15d94f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15d94f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d419f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59290/0 (socket says 172.21.15.46:59290) 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15d94f0640 1 -- 172.21.15.46:0/2926234705 learned_addr learned my addr 172.21.15.46:0/2926234705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15d8cef640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d4107900 0x7f15d419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15d94f0640 1 -- 172.21.15.46:0/2926234705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d4107900 msgr2=0x7f15d419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15d94f0640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d4107900 0x7f15d419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.357+0000 7f15d94f0640 1 -- 172.21.15.46:0/2926234705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d4199a10 con 0x7f15d4101900 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.358+0000 7f15d94f0640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d419f850 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f15d000d6f0 tx=0x7f15d000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.358+0000 7f15c67fc640 1 -- 172.21.15.46:0/2926234705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f15d0017070 con 0x7f15d4101900 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.358+0000 7f15c67fc640 1 -- 172.21.15.46:0/2926234705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f15d000de00 con 0x7f15d4101900 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.358+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d4199d00 con 0x7f15d4101900 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.358+0000 7f15c67fc640 1 -- 172.21.15.46:0/2926234705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f15d0004bb0 con 0x7f15d4101900 2024-09-10T04:39:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.358+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d419a290 con 0x7f15d4101900 2024-09-10T04:39:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.359+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d4103040 con 0x7f15d4101900 2024-09-10T04:39:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.360+0000 7f15c67fc640 1 -- 172.21.15.46:0/2926234705 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f15d0009a30 con 0x7f15d4101900 2024-09-10T04:39:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.360+0000 7f15c67fc640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15c007ac40 0x7f15c007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.360+0000 7f15d8cef640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15c007ac40 0x7f15c007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.360+0000 7f15c67fc640 1 -- 172.21.15.46:0/2926234705 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f15d009d7e0 con 0x7f15d4101900 2024-09-10T04:39:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.361+0000 7f15d8cef640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15c007ac40 0x7f15c007d100 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f15d40757e0 tx=0x7f15c803c040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:00.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.365+0000 7f15c67fc640 1 -- 172.21.15.46:0/2926234705 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f15d0068f00 con 0x7f15d4101900 2024-09-10T04:39:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.630+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f15d419a540 con 0x7f15d4101900 2024-09-10T04:39:00.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.630+0000 7f15c67fc640 1 -- 172.21.15.46:0/2926234705 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f15d006ddb0 con 0x7f15d4101900 2024-09-10T04:39:00.631 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:39:00.631 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:39:00.632 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:39:00.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:39:00.633 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15c007ac40 msgr2=0x7f15c007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15c007ac40 0x7f15c007d100 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f15d40757e0 tx=0x7f15c803c040 comp rx=0 tx=0).stop 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 msgr2=0x7f15d419f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d419f850 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f15d000d6f0 tx=0x7f15d000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 shutdown_connections 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f15c007ac40 0x7f15c007d100 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d4107900 0x7f15d419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 --2- 172.21.15.46:0/2926234705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15d4101900 0x7f15d419f850 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 >> 172.21.15.46:0/2926234705 conn(0x7f15d40fd600 msgr2=0x7f15d4074480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:00.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 shutdown_connections 2024-09-10T04:39:00.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.633+0000 7f15db754640 1 -- 172.21.15.46:0/2926234705 wait complete. 2024-09-10T04:39:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 -- 172.21.15.46:0/608961460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28101910 msgr2=0x7f8f28101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/608961460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28101910 0x7f8f28101d70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009880 tx=0x7f8f1c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 -- 172.21.15.46:0/608961460 shutdown_connections 2024-09-10T04:39:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/608961460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28101910 0x7f8f28101d70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/608961460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f28107910 0x7f8f28107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 -- 172.21.15.46:0/608961460 >> 172.21.15.46:0/608961460 conn(0x7f8f280fd630 msgr2=0x7f8f280ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:00.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 -- 172.21.15.46:0/608961460 shutdown_connections 2024-09-10T04:39:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.717+0000 7f8f2d85b640 1 -- 172.21.15.46:0/608961460 wait complete. 2024-09-10T04:39:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f2d85b640 1 Processor -- start 2024-09-10T04:39:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f2d85b640 1 -- start start 2024-09-10T04:39:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f2d85b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f28101910 0x7f8f2819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f2d85b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28107910 0x7f8f2819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f2d85b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810ede0 con 0x7f8f28107910 2024-09-10T04:39:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f2d85b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2810ec60 con 0x7f8f28101910 2024-09-10T04:39:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f267fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28107910 0x7f8f2819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f267fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28107910 0x7f8f2819db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59308/0 (socket says 172.21.15.46:59308) 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f267fc640 1 -- 172.21.15.46:0/173997533 learned_addr learned my addr 172.21.15.46:0/173997533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.718+0000 7f8f26ffd640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f28101910 0x7f8f2819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f267fc640 1 -- 172.21.15.46:0/173997533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f28101910 msgr2=0x7f8f2819d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f267fc640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f28101910 0x7f8f2819d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f267fc640 1 -- 172.21.15.46:0/173997533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f28197780 con 0x7f8f28107910 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f267fc640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28107910 0x7f8f2819db40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c031460 tx=0x7f8f1c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f2c859640 1 -- 172.21.15.46:0/173997533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f1c03f070 con 0x7f8f28107910 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f2c859640 1 -- 172.21.15.46:0/173997533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f1c0035f0 con 0x7f8f28107910 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f28197a10 con 0x7f8f28107910 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f2c859640 1 -- 172.21.15.46:0/173997533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8f1c035030 con 0x7f8f28107910 2024-09-10T04:39:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.719+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f28197f40 con 0x7f8f28107910 2024-09-10T04:39:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.720+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f28103050 con 0x7f8f28107910 2024-09-10T04:39:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.721+0000 7f8f2c859640 1 -- 172.21.15.46:0/173997533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8f1c042070 con 0x7f8f28107910 2024-09-10T04:39:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.721+0000 7f8f2c859640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f1007abf0 0x7f8f1007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.722+0000 7f8f2c859640 1 -- 172.21.15.46:0/173997533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8f1c0be5d0 con 0x7f8f28107910 2024-09-10T04:39:00.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.722+0000 7f8f26ffd640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f1007abf0 0x7f8f1007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:00.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.722+0000 7f8f26ffd640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f1007abf0 0x7f8f1007d0b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005fd0 tx=0x7f8f1800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.726+0000 7f8f2c859640 1 -- 172.21.15.46:0/173997533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8f1c089d10 con 0x7f8f28107910 2024-09-10T04:39:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.949+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8f28109e80 con 0x7f8f28107910 2024-09-10T04:39:00.950 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:00 smithi046 ceph-mon[109506]: pgmap v1001: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:00.950 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:00 smithi046 ceph-mon[109506]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:00.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:00 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2926234705' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:39:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.950+0000 7f8f2c859640 1 -- 172.21.15.46:0/173997533 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f8f1c08ebc0 con 0x7f8f28107910 2024-09-10T04:39:00.951 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:39:00.951 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:39:00.951 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:39:00.951 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:39:00.951 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:39:00.951 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:39:00.952 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:39:00.953 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:00.954 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:00.955 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:39:00.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f1007abf0 msgr2=0x7f8f1007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f1007abf0 0x7f8f1007d0b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005fd0 tx=0x7f8f1800b390 comp rx=0 tx=0).stop 2024-09-10T04:39:00.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28107910 msgr2=0x7f8f2819db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:00.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28107910 0x7f8f2819db40 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c031460 tx=0x7f8f1c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 shutdown_connections 2024-09-10T04:39:00.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8f1007abf0 0x7f8f1007d0b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f28107910 0x7f8f2819db40 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 --2- 172.21.15.46:0/173997533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f28101910 0x7f8f2819d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 >> 172.21.15.46:0/173997533 conn(0x7f8f280fd630 msgr2=0x7f8f280fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 shutdown_connections 2024-09-10T04:39:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:00.953+0000 7f8f2d85b640 1 -- 172.21.15.46:0/173997533 wait complete. 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 -- 172.21.15.46:0/2390450341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581067f0 msgr2=0x7f0058106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 --2- 172.21.15.46:0/2390450341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581067f0 0x7f0058106bd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0050009880 tx=0x7f005002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 -- 172.21.15.46:0/2390450341 shutdown_connections 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 --2- 172.21.15.46:0/2390450341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00581007f0 0x7f0058100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 --2- 172.21.15.46:0/2390450341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581067f0 0x7f0058106bd0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 -- 172.21.15.46:0/2390450341 >> 172.21.15.46:0/2390450341 conn(0x7f00580fc550 msgr2=0x7f00580fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 -- 172.21.15.46:0/2390450341 shutdown_connections 2024-09-10T04:39:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 -- 172.21.15.46:0/2390450341 wait complete. 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 Processor -- start 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.038+0000 7f005f7b3640 1 -- start start 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005f7b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581007f0 0x7f005819c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005f7b3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00581067f0 0x7f005819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005f7b3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f005810dca0 con 0x7f00581007f0 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005f7b3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f005810db20 con 0x7f00581067f0 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005d54f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581007f0 0x7f005819c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005d54f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581007f0 0x7f005819c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59336/0 (socket says 172.21.15.46:59336) 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005d54f640 1 -- 172.21.15.46:0/1125404994 learned_addr learned my addr 172.21.15.46:0/1125404994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005cd4e640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00581067f0 0x7f005819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005d54f640 1 -- 172.21.15.46:0/1125404994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00581067f0 msgr2=0x7f005819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005d54f640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00581067f0 0x7f005819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005d54f640 1 -- 172.21.15.46:0/1125404994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0058196790 con 0x7f00581007f0 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.039+0000 7f005d54f640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581007f0 0x7f005819c610 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0050031460 tx=0x7f0050004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.040+0000 7f00567fc640 1 -- 172.21.15.46:0/1125404994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f005003f070 con 0x7f00581007f0 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.040+0000 7f00567fc640 1 -- 172.21.15.46:0/1125404994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00500035f0 con 0x7f00581007f0 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.040+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0058196a20 con 0x7f00581007f0 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.040+0000 7f00567fc640 1 -- 172.21.15.46:0/1125404994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0050035030 con 0x7f00581007f0 2024-09-10T04:39:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.040+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0058196f50 con 0x7f00581007f0 2024-09-10T04:39:01.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.041+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0058101f30 con 0x7f00581007f0 2024-09-10T04:39:01.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.041+0000 7f00567fc640 1 -- 172.21.15.46:0/1125404994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0050042070 con 0x7f00581007f0 2024-09-10T04:39:01.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.042+0000 7f00567fc640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f004007ac40 0x7f004007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:01.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.042+0000 7f00567fc640 1 -- 172.21.15.46:0/1125404994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f00500be5f0 con 0x7f00581007f0 2024-09-10T04:39:01.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.042+0000 7f005cd4e640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f004007ac40 0x7f004007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:01.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.042+0000 7f005cd4e640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f004007ac40 0x7f004007d100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f00580ffca0 tx=0x7f004c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:01.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.046+0000 7f00567fc640 1 -- 172.21.15.46:0/1125404994 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0050089d10 con 0x7f00581007f0 2024-09-10T04:39:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:00 smithi070 ceph-mon[79774]: pgmap v1001: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:00 smithi070 ceph-mon[79774]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:00 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2926234705' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:39:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.250+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f0058197cc0 con 0x7f004007ac40 2024-09-10T04:39:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.252+0000 7f00567fc640 1 -- 172.21.15.46:0/1125404994 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f0058197cc0 con 0x7f004007ac40 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:39:01.253 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:39:01.254 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:39:01.254 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:39:01.254 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:39:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f004007ac40 msgr2=0x7f004007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f004007ac40 0x7f004007d100 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f00580ffca0 tx=0x7f004c005f90 comp rx=0 tx=0).stop 2024-09-10T04:39:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581007f0 msgr2=0x7f005819c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581007f0 0x7f005819c610 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0050031460 tx=0x7f0050004530 comp rx=0 tx=0).stop 2024-09-10T04:39:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 shutdown_connections 2024-09-10T04:39:01.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f004007ac40 0x7f004007d100 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00581067f0 0x7f005819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 --2- 172.21.15.46:0/1125404994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00581007f0 0x7f005819c610 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 >> 172.21.15.46:0/1125404994 conn(0x7f00580fc550 msgr2=0x7f00580fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 shutdown_connections 2024-09-10T04:39:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.254+0000 7f005f7b3640 1 -- 172.21.15.46:0/1125404994 wait complete. 2024-09-10T04:39:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 -- 172.21.15.46:0/2158502018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 msgr2=0x7fb4e0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 --2- 172.21.15.46:0/2158502018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e0107ce0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009880 tx=0x7fb4dc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 -- 172.21.15.46:0/2158502018 shutdown_connections 2024-09-10T04:39:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 --2- 172.21.15.46:0/2158502018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e0101900 0x7fb4e0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 --2- 172.21.15.46:0/2158502018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e0107ce0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 -- 172.21.15.46:0/2158502018 >> 172.21.15.46:0/2158502018 conn(0x7fb4e00fd600 msgr2=0x7fb4e00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 -- 172.21.15.46:0/2158502018 shutdown_connections 2024-09-10T04:39:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.338+0000 7fb4e7555640 1 -- 172.21.15.46:0/2158502018 wait complete. 2024-09-10T04:39:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e7555640 1 Processor -- start 2024-09-10T04:39:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e7555640 1 -- start start 2024-09-10T04:39:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e7555640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e0101900 0x7fb4e019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e7555640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e7555640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e010edb0 con 0x7fb4e0107900 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e7555640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e010ec30 con 0x7fb4e0101900 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e4af0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e4af0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e019db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59356/0 (socket says 172.21.15.46:59356) 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.339+0000 7fb4e4af0640 1 -- 172.21.15.46:0/1079547431 learned_addr learned my addr 172.21.15.46:0/1079547431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4e52f1640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e0101900 0x7fb4e019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4e4af0640 1 -- 172.21.15.46:0/1079547431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e0101900 msgr2=0x7fb4e019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4e4af0640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e0101900 0x7fb4e019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4e4af0640 1 -- 172.21.15.46:0/1079547431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e01977c0 con 0x7fb4e0107900 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4e4af0640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e019db80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d8d0 tx=0x7fb4d000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4d67fc640 1 -- 172.21.15.46:0/1079547431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4d0017070 con 0x7fb4e0107900 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4d67fc640 1 -- 172.21.15.46:0/1079547431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4d0010460 con 0x7fb4e0107900 2024-09-10T04:39:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e0197ab0 con 0x7fb4e0107900 2024-09-10T04:39:01.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4d67fc640 1 -- 172.21.15.46:0/1079547431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb4d000f5f0 con 0x7fb4e0107900 2024-09-10T04:39:01.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.340+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e0198040 con 0x7fb4e0107900 2024-09-10T04:39:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.341+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e0103040 con 0x7fb4e0107900 2024-09-10T04:39:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.342+0000 7fb4d67fc640 1 -- 172.21.15.46:0/1079547431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb4d0009a30 con 0x7fb4e0107900 2024-09-10T04:39:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.342+0000 7fb4d67fc640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb4cc07ac40 0x7fb4cc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.343+0000 7fb4d67fc640 1 -- 172.21.15.46:0/1079547431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb4d009da60 con 0x7fb4e0107900 2024-09-10T04:39:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.343+0000 7fb4e52f1640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb4cc07ac40 0x7fb4cc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:01.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.343+0000 7fb4e52f1640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb4cc07ac40 0x7fb4cc07d100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0096c0 tx=0x7fb4dc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.347+0000 7fb4d67fc640 1 -- 172.21.15.46:0/1079547431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb4d0069180 con 0x7fb4e0107900 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.603+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fb4e0104b10 con 0x7fb4e0107900 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.604+0000 7fb4d67fc640 1 -- 172.21.15.46:0/1079547431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fb4d006e030 con 0x7fb4e0107900 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:39:01.605 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:39:01.606 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:39:01.607 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:39:01.607 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:39:01.607 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:39:01.607 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:39:01.607 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:39:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb4cc07ac40 msgr2=0x7fb4cc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb4cc07ac40 0x7fb4cc07d100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0096c0 tx=0x7fb4dc031400 comp rx=0 tx=0).stop 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 msgr2=0x7fb4e019db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e019db80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000d8d0 tx=0x7fb4d000dd90 comp rx=0 tx=0).stop 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 shutdown_connections 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb4cc07ac40 0x7fb4cc07d100 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4e0107900 0x7fb4e019db80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 --2- 172.21.15.46:0/1079547431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4e0101900 0x7fb4e019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 >> 172.21.15.46:0/1079547431 conn(0x7fb4e00fd600 msgr2=0x7fb4e00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 shutdown_connections 2024-09-10T04:39:01.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:01.606+0000 7fb4e7555640 1 -- 172.21.15.46:0/1079547431 wait complete. 2024-09-10T04:39:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:01 smithi070 ceph-mon[79774]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:01 smithi070 ceph-mon[79774]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:01 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/173997533' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:39:02.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:01 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1079547431' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:39:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:01 smithi046 ceph-mon[109506]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:02.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:01 smithi046 ceph-mon[109506]: from='client.35662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:02.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:01 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/173997533' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:39:02.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:01 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1079547431' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:39:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:02 smithi070 ceph-mon[79774]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:02 smithi070 ceph-mon[79774]: pgmap v1002: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:02 smithi046 ceph-mon[109506]: from='client.35674 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:03.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:02 smithi046 ceph-mon[109506]: pgmap v1002: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:04 smithi070 ceph-mon[79774]: pgmap v1003: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:04 smithi046 ceph-mon[109506]: pgmap v1003: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:06 smithi070 ceph-mon[79774]: pgmap v1004: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:06 smithi046 ceph-mon[109506]: pgmap v1004: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:08 smithi070 ceph-mon[79774]: pgmap v1005: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:08 smithi046 ceph-mon[109506]: pgmap v1005: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:10 smithi070 ceph-mon[79774]: pgmap v1006: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:10 smithi046 ceph-mon[109506]: pgmap v1006: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:12.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:12 smithi070 ceph-mon[79774]: pgmap v1007: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:12.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:12 smithi046 ceph-mon[109506]: pgmap v1007: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:14 smithi070 ceph-mon[79774]: pgmap v1008: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:14 smithi046 ceph-mon[109506]: pgmap v1008: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:16 smithi070 ceph-mon[79774]: pgmap v1009: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:16 smithi046 ceph-mon[109506]: pgmap v1009: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:18 smithi070 ceph-mon[79774]: pgmap v1010: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:18 smithi046 ceph-mon[109506]: pgmap v1010: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:20 smithi070 ceph-mon[79774]: pgmap v1011: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:20 smithi046 ceph-mon[109506]: pgmap v1011: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:22 smithi070 ceph-mon[79774]: pgmap v1012: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:22 smithi046 ceph-mon[109506]: pgmap v1012: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:23.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:24.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:24 smithi070 ceph-mon[79774]: pgmap v1013: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:25.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:24 smithi046 ceph-mon[109506]: pgmap v1013: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:26 smithi070 ceph-mon[79774]: pgmap v1014: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:26 smithi046 ceph-mon[109506]: pgmap v1014: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:28 smithi070 ceph-mon[79774]: pgmap v1015: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:28 smithi046 ceph-mon[109506]: pgmap v1015: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:30 smithi046 ceph-mon[109506]: pgmap v1016: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:30 smithi070 ceph-mon[79774]: pgmap v1016: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.702+0000 7f8be9bdd640 1 -- 172.21.15.46:0/2740003424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 msgr2=0x7f8be4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.702+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/2740003424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4107ce0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4009880 tx=0x7f8bd402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.703+0000 7f8be9bdd640 1 -- 172.21.15.46:0/2740003424 shutdown_connections 2024-09-10T04:39:31.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.703+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/2740003424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8be4101900 0x7f8be4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.703+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/2740003424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4107ce0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.703+0000 7f8be9bdd640 1 -- 172.21.15.46:0/2740003424 >> 172.21.15.46:0/2740003424 conn(0x7f8be40fd600 msgr2=0x7f8be40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.703+0000 7f8be9bdd640 1 -- 172.21.15.46:0/2740003424 shutdown_connections 2024-09-10T04:39:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.703+0000 7f8be9bdd640 1 -- 172.21.15.46:0/2740003424 wait complete. 2024-09-10T04:39:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.703+0000 7f8be9bdd640 1 Processor -- start 2024-09-10T04:39:31.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be9bdd640 1 -- start start 2024-09-10T04:39:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be9bdd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8be4101900 0x7f8be40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:31.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be9bdd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be9bdd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8be410edb0 con 0x7f8be4107900 2024-09-10T04:39:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be9bdd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8be410ec30 con 0x7f8be4101900 2024-09-10T04:39:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52936/0 (socket says 172.21.15.46:52936) 2024-09-10T04:39:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be2ffd640 1 -- 172.21.15.46:0/3164064413 learned_addr learned my addr 172.21.15.46:0/3164064413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:31.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be37fe640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8be4101900 0x7f8be40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be2ffd640 1 -- 172.21.15.46:0/3164064413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8be4101900 msgr2=0x7f8be40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be2ffd640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8be4101900 0x7f8be40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.704+0000 7f8be2ffd640 1 -- 172.21.15.46:0/3164064413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be4074f70 con 0x7f8be4107900 2024-09-10T04:39:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.705+0000 7f8be2ffd640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4074a30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800e970 tx=0x7f8bd800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.705+0000 7f8be0ff9640 1 -- 172.21.15.46:0/3164064413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8bd800cac0 con 0x7f8be4107900 2024-09-10T04:39:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.705+0000 7f8be0ff9640 1 -- 172.21.15.46:0/3164064413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bd80054e0 con 0x7f8be4107900 2024-09-10T04:39:31.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.705+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be4075260 con 0x7f8be4107900 2024-09-10T04:39:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.705+0000 7f8be0ff9640 1 -- 172.21.15.46:0/3164064413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8bd8004e10 con 0x7f8be4107900 2024-09-10T04:39:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.705+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be41a3e20 con 0x7f8be4107900 2024-09-10T04:39:31.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.706+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be4103040 con 0x7f8be4107900 2024-09-10T04:39:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.707+0000 7f8be0ff9640 1 -- 172.21.15.46:0/3164064413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8bd8005020 con 0x7f8be4107900 2024-09-10T04:39:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.707+0000 7f8be0ff9640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bcc07ab90 0x7f8bcc07d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.707+0000 7f8be0ff9640 1 -- 172.21.15.46:0/3164064413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8bd809e460 con 0x7f8be4107900 2024-09-10T04:39:31.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.708+0000 7f8be37fe640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bcc07ab90 0x7f8bcc07d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.708+0000 7f8be37fe640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bcc07ab90 0x7f8bcc07d050 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4031460 tx=0x7f8bd40095c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:31.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.712+0000 7f8be0ff9640 1 -- 172.21.15.46:0/3164064413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8bd8069b00 con 0x7f8be4107900 2024-09-10T04:39:31.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.917+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8be4062630 con 0x7f8bcc07ab90 2024-09-10T04:39:31.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.918+0000 7f8be0ff9640 1 -- 172.21.15.46:0/3164064413 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8be4062630 con 0x7f8bcc07ab90 2024-09-10T04:39:31.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bcc07ab90 msgr2=0x7f8bcc07d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bcc07ab90 0x7f8bcc07d050 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8bd4031460 tx=0x7f8bd40095c0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 msgr2=0x7f8be4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4074a30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8bd800e970 tx=0x7f8bd800ee30 comp rx=0 tx=0).stop 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 shutdown_connections 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8bcc07ab90 0x7f8bcc07d050 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8be4107900 0x7f8be4074a30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 --2- 172.21.15.46:0/3164064413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8be4101900 0x7f8be40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.920+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 >> 172.21.15.46:0/3164064413 conn(0x7f8be40fd600 msgr2=0x7f8be40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.921+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 shutdown_connections 2024-09-10T04:39:31.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:31.921+0000 7f8be9bdd640 1 -- 172.21.15.46:0/3164064413 wait complete. 2024-09-10T04:39:31.931 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:39:32.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 -- 172.21.15.46:0/2726642731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec102ab0 msgr2=0x7f05ec102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2726642731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec102ab0 0x7f05ec102e90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009880 tx=0x7f05dc02a530 comp rx=0 tx=0).stop 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 -- 172.21.15.46:0/2726642731 shutdown_connections 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2726642731 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec1033d0 0x7f05ec10b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2726642731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec102ab0 0x7f05ec102e90 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 -- 172.21.15.46:0/2726642731 >> 172.21.15.46:0/2726642731 conn(0x7f05ec0fc550 msgr2=0x7f05ec0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 -- 172.21.15.46:0/2726642731 shutdown_connections 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.006+0000 7f05f2a54640 1 -- 172.21.15.46:0/2726642731 wait complete. 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05f2a54640 1 Processor -- start 2024-09-10T04:39:32.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05f2a54640 1 -- start start 2024-09-10T04:39:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05f2a54640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec102ab0 0x7f05ec19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05f2a54640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec1033d0 0x7f05ec19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05f2a54640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec10ede0 con 0x7f05ec1033d0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05f2a54640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec10ec60 con 0x7f05ec102ab0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05eb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec1033d0 0x7f05ec19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05eb7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec1033d0 0x7f05ec19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52948/0 (socket says 172.21.15.46:52948) 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05eb7fe640 1 -- 172.21.15.46:0/2578561887 learned_addr learned my addr 172.21.15.46:0/2578561887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.007+0000 7f05ebfff640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec102ab0 0x7f05ec19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05eb7fe640 1 -- 172.21.15.46:0/2578561887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec102ab0 msgr2=0x7f05ec19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05eb7fe640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec102ab0 0x7f05ec19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05eb7fe640 1 -- 172.21.15.46:0/2578561887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec1977b0 con 0x7f05ec1033d0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05ebfff640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec102ab0 0x7f05ec19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05eb7fe640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec1033d0 0x7f05ec19db70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f05e000e970 tx=0x7f05e000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05e97fa640 1 -- 172.21.15.46:0/2578561887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f05e000cac0 con 0x7f05ec1033d0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec197aa0 con 0x7f05ec1033d0 2024-09-10T04:39:32.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.008+0000 7f05e97fa640 1 -- 172.21.15.46:0/2578561887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f05e00054e0 con 0x7f05ec1033d0 2024-09-10T04:39:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.009+0000 7f05e97fa640 1 -- 172.21.15.46:0/2578561887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f05e0004dd0 con 0x7f05ec1033d0 2024-09-10T04:39:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.009+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec198030 con 0x7f05ec1033d0 2024-09-10T04:39:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.010+0000 7f05e97fa640 1 -- 172.21.15.46:0/2578561887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f05e00104e0 con 0x7f05ec1033d0 2024-09-10T04:39:32.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.010+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ec06a580 con 0x7f05ec1033d0 2024-09-10T04:39:32.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.010+0000 7f05e97fa640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f05d407abf0 0x7f05d407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.010+0000 7f05e97fa640 1 -- 172.21.15.46:0/2578561887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f05e009dc10 con 0x7f05ec1033d0 2024-09-10T04:39:32.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.010+0000 7f05ebfff640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f05d407abf0 0x7f05d407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.011+0000 7f05ebfff640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f05d407abf0 0x7f05d407d0b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000c00 tx=0x7f05dc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.015+0000 7f05e97fa640 1 -- 172.21.15.46:0/2578561887 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f05e0069340 con 0x7f05ec1033d0 2024-09-10T04:39:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.219+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f05ec198340 con 0x7f05d407abf0 2024-09-10T04:39:32.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.220+0000 7f05e97fa640 1 -- 172.21.15.46:0/2578561887 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f05ec198340 con 0x7f05d407abf0 2024-09-10T04:39:32.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f05d407abf0 msgr2=0x7f05d407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f05d407abf0 0x7f05d407d0b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000c00 tx=0x7f05dc031400 comp rx=0 tx=0).stop 2024-09-10T04:39:32.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec1033d0 msgr2=0x7f05ec19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec1033d0 0x7f05ec19db70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f05e000e970 tx=0x7f05e000ee30 comp rx=0 tx=0).stop 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 shutdown_connections 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f05d407abf0 0x7f05d407d0b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ec1033d0 0x7f05ec19db70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 --2- 172.21.15.46:0/2578561887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec102ab0 0x7f05ec19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 >> 172.21.15.46:0/2578561887 conn(0x7f05ec0fc550 msgr2=0x7f05ec10aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 shutdown_connections 2024-09-10T04:39:32.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.222+0000 7f05f2a54640 1 -- 172.21.15.46:0/2578561887 wait complete. 2024-09-10T04:39:32.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 -- 172.21.15.46:0/1555036919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 msgr2=0x7fb17c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 --2- 172.21.15.46:0/1555036919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c100c50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb170009880 tx=0x7fb17002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 -- 172.21.15.46:0/1555036919 shutdown_connections 2024-09-10T04:39:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 --2- 172.21.15.46:0/1555036919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c100c50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 --2- 172.21.15.46:0/1555036919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17c1067f0 0x7fb17c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 -- 172.21.15.46:0/1555036919 >> 172.21.15.46:0/1555036919 conn(0x7fb17c0fc550 msgr2=0x7fb17c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 -- 172.21.15.46:0/1555036919 shutdown_connections 2024-09-10T04:39:32.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.307+0000 7fb180b86640 1 -- 172.21.15.46:0/1555036919 wait complete. 2024-09-10T04:39:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.308+0000 7fb180b86640 1 Processor -- start 2024-09-10T04:39:32.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.308+0000 7fb180b86640 1 -- start start 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.308+0000 7fb180b86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.308+0000 7fb180b86640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17c1067f0 0x7fb17c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.308+0000 7fb180b86640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c10dcc0 con 0x7fb17c1007f0 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.308+0000 7fb180b86640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c10db40 con 0x7fb17c1067f0 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb17a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb17a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52972/0 (socket says 172.21.15.46:52972) 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb17a59c640 1 -- 172.21.15.46:0/3242247904 learned_addr learned my addr 172.21.15.46:0/3242247904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb179d9b640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17c1067f0 0x7fb17c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb17a59c640 1 -- 172.21.15.46:0/3242247904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17c1067f0 msgr2=0x7fb17c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb17a59c640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17c1067f0 0x7fb17c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb17a59c640 1 -- 172.21.15.46:0/3242247904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c199330 con 0x7fb17c1007f0 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb17a59c640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c1987f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00d6f0 tx=0x7fb16c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb16b7fe640 1 -- 172.21.15.46:0/3242247904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb16c017070 con 0x7fb17c1007f0 2024-09-10T04:39:32.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb16b7fe640 1 -- 172.21.15.46:0/3242247904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb16c00de00 con 0x7fb17c1007f0 2024-09-10T04:39:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c199620 con 0x7fb17c1007f0 2024-09-10T04:39:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb16b7fe640 1 -- 172.21.15.46:0/3242247904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb16c004bb0 con 0x7fb17c1007f0 2024-09-10T04:39:32.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.309+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c1a1c20 con 0x7fb17c1007f0 2024-09-10T04:39:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.310+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17c101f30 con 0x7fb17c1007f0 2024-09-10T04:39:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.311+0000 7fb16b7fe640 1 -- 172.21.15.46:0/3242247904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb16c009a30 con 0x7fb17c1007f0 2024-09-10T04:39:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.311+0000 7fb16b7fe640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.312+0000 7fb16b7fe640 1 -- 172.21.15.46:0/3242247904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb16c09d720 con 0x7fb17c1007f0 2024-09-10T04:39:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.312+0000 7fb179d9b640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.312+0000 7fb179d9b640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb17c06a7f0 tx=0x7fb170031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.316+0000 7fb16b7fe640 1 -- 172.21.15.46:0/3242247904 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb17c199620 con 0x7fb17c1007f0 2024-09-10T04:39:32.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.510+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb17c062630 con 0x7fb16407ac40 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.520+0000 7fb16b7fe640 1 -- 172.21.15.46:0/3242247904 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7fb17c062630 con 0x7fb16407ac40 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (31m) 10m ago 42m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (33m) 10m ago 42m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (33m) 10m ago 41m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (29m) 10m ago 42m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (29m) 10m ago 41m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (30m) 10m ago 42m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (25m) 10m ago 37m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:39:32.522 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (25m) 10m ago 37m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (25m) 10m ago 37m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (10m) 10m ago 36m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (33m) 10m ago 43m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (32m) 10m ago 41m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (30m) 10m ago 43m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (29m) 10m ago 41m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (31m) 10m ago 42m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (31m) 10m ago 41m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (29m) 10m ago 40m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (28m) 10m ago 39m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (27m) 10m ago 39m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (27m) 10m ago 39m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (26m) 10m ago 38m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:39:32.523 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (26m) 10m ago 38m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:39:32.524 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (31m) 10m ago 42m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:39:32.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 msgr2=0x7fb16407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb17c06a7f0 tx=0x7fb170031400 comp rx=0 tx=0).stop 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 msgr2=0x7fb17c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c1987f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00d6f0 tx=0x7fb16c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 shutdown_connections 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb17c1067f0 0x7fb17c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 --2- 172.21.15.46:0/3242247904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb17c1007f0 0x7fb17c1987f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 >> 172.21.15.46:0/3242247904 conn(0x7fb17c0fc550 msgr2=0x7fb17c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 shutdown_connections 2024-09-10T04:39:32.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.522+0000 7fb180b86640 1 -- 172.21.15.46:0/3242247904 wait complete. 2024-09-10T04:39:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 -- 172.21.15.46:0/2527920064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 msgr2=0x7f7ffc100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 --2- 172.21.15.46:0/2527920064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc100a30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009880 tx=0x7f7ff802a490 comp rx=0 tx=0).stop 2024-09-10T04:39:32.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 -- 172.21.15.46:0/2527920064 shutdown_connections 2024-09-10T04:39:32.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 --2- 172.21.15.46:0/2527920064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc101000 0x7f7ffc10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 --2- 172.21.15.46:0/2527920064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc100a30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 -- 172.21.15.46:0/2527920064 >> 172.21.15.46:0/2527920064 conn(0x7f7ffc0fc520 msgr2=0x7f7ffc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:32.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 -- 172.21.15.46:0/2527920064 shutdown_connections 2024-09-10T04:39:32.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.605+0000 7f80040b2640 1 -- 172.21.15.46:0/2527920064 wait complete. 2024-09-10T04:39:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f80040b2640 1 Processor -- start 2024-09-10T04:39:32.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f80040b2640 1 -- start start 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f80040b2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f80040b2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc101000 0x7f7ffc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f80040b2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc110fb0 con 0x7f7ffc100650 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f80040b2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc110e30 con 0x7f7ffc101000 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f8001e4e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f8001e4e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc19f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52988/0 (socket says 172.21.15.46:52988) 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.606+0000 7f8001e4e640 1 -- 172.21.15.46:0/1950724942 learned_addr learned my addr 172.21.15.46:0/1950724942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f800164d640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc101000 0x7f7ffc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f8001e4e640 1 -- 172.21.15.46:0/1950724942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc101000 msgr2=0x7f7ffc19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f8001e4e640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc101000 0x7f7ffc19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f8001e4e640 1 -- 172.21.15.46:0/1950724942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc199a40 con 0x7f7ffc100650 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f8001e4e640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc19f880 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8031460 tx=0x7f7ff80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f7ff2ffd640 1 -- 172.21.15.46:0/1950724942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ff803f070 con 0x7f7ffc100650 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f7ff2ffd640 1 -- 172.21.15.46:0/1950724942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ff8004cf0 con 0x7f7ffc100650 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc199cd0 con 0x7f7ffc100650 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f7ff2ffd640 1 -- 172.21.15.46:0/1950724942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ff8035030 con 0x7f7ffc100650 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.607+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc19a260 con 0x7f7ffc100650 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.608+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc06aa90 con 0x7f7ffc100650 2024-09-10T04:39:32.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.609+0000 7f7ff2ffd640 1 -- 172.21.15.46:0/1950724942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7ff8042070 con 0x7f7ffc100650 2024-09-10T04:39:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.609+0000 7f7ff2ffd640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fe807ac40 0x7f7fe807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.609+0000 7f800164d640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fe807ac40 0x7f7fe807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.609+0000 7f7ff2ffd640 1 -- 172.21.15.46:0/1950724942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7ff80be600 con 0x7f7ffc100650 2024-09-10T04:39:32.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.610+0000 7f800164d640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fe807ac40 0x7f7fe807d100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc102520 tx=0x7f7fec00a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.614+0000 7f7ff2ffd640 1 -- 172.21.15.46:0/1950724942 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7ff8089dc0 con 0x7f7ffc100650 2024-09-10T04:39:32.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.872+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f7ffc10b3a0 con 0x7f7ffc100650 2024-09-10T04:39:32.875 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:32 smithi046 ceph-mon[109506]: pgmap v1017: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:32.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.873+0000 7f7ff2ffd640 1 -- 172.21.15.46:0/1950724942 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f7ff808ec70 con 0x7f7ffc100650 2024-09-10T04:39:32.875 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:39:32.875 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:39:32.875 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:39:32.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:32.875 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:39:32.876 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:39:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fe807ac40 msgr2=0x7f7fe807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fe807ac40 0x7f7fe807d100 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc102520 tx=0x7f7fec00a250 comp rx=0 tx=0).stop 2024-09-10T04:39:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 msgr2=0x7f7ffc19f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc19f880 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8031460 tx=0x7f7ff80048f0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 shutdown_connections 2024-09-10T04:39:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7fe807ac40 0x7f7fe807d100 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ffc101000 0x7f7ffc19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 --2- 172.21.15.46:0/1950724942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ffc100650 0x7f7ffc19f880 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 >> 172.21.15.46:0/1950724942 conn(0x7f7ffc0fc520 msgr2=0x7f7ffc0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 shutdown_connections 2024-09-10T04:39:32.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.875+0000 7f80040b2640 1 -- 172.21.15.46:0/1950724942 wait complete. 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.959+0000 7fe2acda6640 1 -- 172.21.15.46:0/4280305917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 msgr2=0x7fe2a8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.959+0000 7fe2acda6640 1 --2- 172.21.15.46:0/4280305917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a8101d60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe29c009880 tx=0x7fe29c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 -- 172.21.15.46:0/4280305917 shutdown_connections 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 --2- 172.21.15.46:0/4280305917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a8101d60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 --2- 172.21.15.46:0/4280305917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2a8107900 0x7fe2a8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 -- 172.21.15.46:0/4280305917 >> 172.21.15.46:0/4280305917 conn(0x7fe2a80fd600 msgr2=0x7fe2a80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 -- 172.21.15.46:0/4280305917 shutdown_connections 2024-09-10T04:39:32.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 -- 172.21.15.46:0/4280305917 wait complete. 2024-09-10T04:39:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 Processor -- start 2024-09-10T04:39:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.960+0000 7fe2acda6640 1 -- start start 2024-09-10T04:39:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2acda6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2acda6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2a8107900 0x7fe2a819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2acda6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a810edd0 con 0x7fe2a8101900 2024-09-10T04:39:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2acda6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a810ec50 con 0x7fe2a8107900 2024-09-10T04:39:32.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53004/0 (socket says 172.21.15.46:53004) 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a659c640 1 -- 172.21.15.46:0/3094155193 learned_addr learned my addr 172.21.15.46:0/3094155193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a5d9b640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2a8107900 0x7fe2a819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a659c640 1 -- 172.21.15.46:0/3094155193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2a8107900 msgr2=0x7fe2a819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a659c640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2a8107900 0x7fe2a819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a659c640 1 -- 172.21.15.46:0/3094155193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a8197800 con 0x7fe2a8101900 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2a659c640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a819d680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e970 tx=0x7fe29800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2977fe640 1 -- 172.21.15.46:0/3094155193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe29800cac0 con 0x7fe2a8101900 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.961+0000 7fe2977fe640 1 -- 172.21.15.46:0/3094155193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2980054e0 con 0x7fe2a8101900 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.962+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a8197af0 con 0x7fe2a8101900 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.962+0000 7fe2977fe640 1 -- 172.21.15.46:0/3094155193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe298010430 con 0x7fe2a8101900 2024-09-10T04:39:32.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.962+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a8198050 con 0x7fe2a8101900 2024-09-10T04:39:32.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.963+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a8103040 con 0x7fe2a8101900 2024-09-10T04:39:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.963+0000 7fe2977fe640 1 -- 172.21.15.46:0/3094155193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe298004f00 con 0x7fe2a8101900 2024-09-10T04:39:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.964+0000 7fe2977fe640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe29007ac50 0x7fe29007d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.964+0000 7fe2977fe640 1 -- 172.21.15.46:0/3094155193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe29809e220 con 0x7fe2a8101900 2024-09-10T04:39:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.964+0000 7fe2a5d9b640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe29007ac50 0x7fe29007d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:32.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.964+0000 7fe2a5d9b640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe29007ac50 0x7fe29007d110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8198b50 tx=0x7fe29c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:32.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:32.968+0000 7fe2977fe640 1 -- 172.21.15.46:0/3094155193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe2980699e0 con 0x7fe2a8101900 2024-09-10T04:39:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:32 smithi070 ceph-mon[79774]: pgmap v1017: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:33.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.191+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe2a8109e70 con 0x7fe2a8101900 2024-09-10T04:39:33.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.192+0000 7fe2977fe640 1 -- 172.21.15.46:0/3094155193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fe29806e890 con 0x7fe2a8101900 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:39:33.194 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:39:33.195 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:39:33.196 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:33.197 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:33.197 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:33.197 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:39:33.197 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:39:33.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe29007ac50 msgr2=0x7fe29007d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe29007ac50 0x7fe29007d110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8198b50 tx=0x7fe29c031400 comp rx=0 tx=0).stop 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 msgr2=0x7fe2a819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a819d680 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe29800e970 tx=0x7fe29800ed70 comp rx=0 tx=0).stop 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 shutdown_connections 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe29007ac50 0x7fe29007d110 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2a8107900 0x7fe2a819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 --2- 172.21.15.46:0/3094155193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2a8101900 0x7fe2a819d680 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 >> 172.21.15.46:0/3094155193 conn(0x7fe2a80fd600 msgr2=0x7fe2a80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 shutdown_connections 2024-09-10T04:39:33.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.195+0000 7fe2acda6640 1 -- 172.21.15.46:0/3094155193 wait complete. 2024-09-10T04:39:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 -- 172.21.15.46:0/2005237028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 msgr2=0x7f5bf0106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 --2- 172.21.15.46:0/2005237028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf0106d80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009880 tx=0x7f5be002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 -- 172.21.15.46:0/2005237028 shutdown_connections 2024-09-10T04:39:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 --2- 172.21.15.46:0/2005237028 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf00ffdc0 0x7f5bf0100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 --2- 172.21.15.46:0/2005237028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf0106d80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 -- 172.21.15.46:0/2005237028 >> 172.21.15.46:0/2005237028 conn(0x7f5bf00fbc90 msgr2=0x7f5bf00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 -- 172.21.15.46:0/2005237028 shutdown_connections 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.281+0000 7f5bf547e640 1 -- 172.21.15.46:0/2005237028 wait complete. 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5bf547e640 1 Processor -- start 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5bf547e640 1 -- start start 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5bf547e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf00ffdc0 0x7f5bf0100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5bf547e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf01014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5bf547e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf010efa0 con 0x7f5bf01069a0 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5bf547e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf010ee20 con 0x7f5bf00ffdc0 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5be67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf01014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5be67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf01014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53016/0 (socket says 172.21.15.46:53016) 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.282+0000 7f5be67fc640 1 -- 172.21.15.46:0/660764228 learned_addr learned my addr 172.21.15.46:0/660764228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5beeffd640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf00ffdc0 0x7f5bf0100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5be67fc640 1 -- 172.21.15.46:0/660764228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf00ffdc0 msgr2=0x7f5bf0100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5be67fc640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf00ffdc0 0x7f5bf0100fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5be67fc640 1 -- 172.21.15.46:0/660764228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf0101a20 con 0x7f5bf01069a0 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5be67fc640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf01014e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b420 tx=0x7f5bd800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5becff9640 1 -- 172.21.15.46:0/660764228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5bd8011630 con 0x7f5bf01069a0 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5becff9640 1 -- 172.21.15.46:0/660764228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5bd8016070 con 0x7f5bf01069a0 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf0101d10 con 0x7f5bf01069a0 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5becff9640 1 -- 172.21.15.46:0/660764228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5bd8010d40 con 0x7f5bf01069a0 2024-09-10T04:39:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.283+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf01a3fd0 con 0x7f5bf01069a0 2024-09-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.284+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf01030e0 con 0x7f5bf01069a0 2024-09-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.285+0000 7f5becff9640 1 -- 172.21.15.46:0/660764228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5bd801b460 con 0x7f5bf01069a0 2024-09-10T04:39:33.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.285+0000 7f5becff9640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5bdc07ac40 0x7f5bdc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.285+0000 7f5becff9640 1 -- 172.21.15.46:0/660764228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5bd809d320 con 0x7f5bf01069a0 2024-09-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.286+0000 7f5beeffd640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5bdc07ac40 0x7f5bdc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.286+0000 7f5beeffd640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5bdc07ac40 0x7f5bdc07d100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5be0031460 tx=0x7f5be0009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.290+0000 7f5becff9640 1 -- 172.21.15.46:0/660764228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5bd8021250 con 0x7f5bf01069a0 2024-09-10T04:39:33.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.494+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5bf0062630 con 0x7f5bdc07ac40 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.495+0000 7f5becff9640 1 -- 172.21.15.46:0/660764228 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f5bf0062630 con 0x7f5bdc07ac40 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:39:33.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:39:33.498 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5bdc07ac40 msgr2=0x7f5bdc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5bdc07ac40 0x7f5bdc07d100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5be0031460 tx=0x7f5be0009560 comp rx=0 tx=0).stop 2024-09-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 msgr2=0x7f5bf01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf01014e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b420 tx=0x7f5bd800b8e0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 shutdown_connections 2024-09-10T04:39:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5bdc07ac40 0x7f5bdc07d100 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5bf01069a0 0x7f5bf01014e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 --2- 172.21.15.46:0/660764228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bf00ffdc0 0x7f5bf0100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 >> 172.21.15.46:0/660764228 conn(0x7f5bf00fbc90 msgr2=0x7f5bf00fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 shutdown_connections 2024-09-10T04:39:33.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.497+0000 7f5bf547e640 1 -- 172.21.15.46:0/660764228 wait complete. 2024-09-10T04:39:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 -- 172.21.15.46:0/871420991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1007f0 msgr2=0x7f0c7c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 --2- 172.21.15.46:0/871420991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1007f0 0x7f0c7c100c50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009880 tx=0x7f0c6c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 -- 172.21.15.46:0/871420991 shutdown_connections 2024-09-10T04:39:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 --2- 172.21.15.46:0/871420991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1007f0 0x7f0c7c100c50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 --2- 172.21.15.46:0/871420991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c7c1067f0 0x7f0c7c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 -- 172.21.15.46:0/871420991 >> 172.21.15.46:0/871420991 conn(0x7f0c7c0fc550 msgr2=0x7f0c7c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 -- 172.21.15.46:0/871420991 shutdown_connections 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.581+0000 7f0c83347640 1 -- 172.21.15.46:0/871420991 wait complete. 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c83347640 1 Processor -- start 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c83347640 1 -- start start 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c83347640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c7c1007f0 0x7f0c7c19d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c83347640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1067f0 0x7f0c7c19dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c83347640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c10dd20 con 0x7f0c7c1067f0 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c83347640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c10dba0 con 0x7f0c7c1007f0 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c808e2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1067f0 0x7f0c7c19dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c808e2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1067f0 0x7f0c7c19dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53032/0 (socket says 172.21.15.46:53032) 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.582+0000 7f0c808e2640 1 -- 172.21.15.46:0/345443167 learned_addr learned my addr 172.21.15.46:0/345443167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c810e3640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c7c1007f0 0x7f0c7c19d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c808e2640 1 -- 172.21.15.46:0/345443167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c7c1007f0 msgr2=0x7f0c7c19d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c808e2640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c7c1007f0 0x7f0c7c19d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c808e2640 1 -- 172.21.15.46:0/345443167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7c1978d0 con 0x7f0c7c1067f0 2024-09-10T04:39:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c808e2640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1067f0 0x7f0c7c19dc50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c031460 tx=0x7f0c6c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c727fc640 1 -- 172.21.15.46:0/345443167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0c6c03f070 con 0x7f0c7c1067f0 2024-09-10T04:39:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c727fc640 1 -- 172.21.15.46:0/345443167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0c6c004cf0 con 0x7f0c7c1067f0 2024-09-10T04:39:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7c197b60 con 0x7f0c7c1067f0 2024-09-10T04:39:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c727fc640 1 -- 172.21.15.46:0/345443167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f0c6c035030 con 0x7f0c7c1067f0 2024-09-10T04:39:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.583+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7c198090 con 0x7f0c7c1067f0 2024-09-10T04:39:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.584+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c7c101f30 con 0x7f0c7c1067f0 2024-09-10T04:39:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.585+0000 7f0c727fc640 1 -- 172.21.15.46:0/345443167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f0c6c042070 con 0x7f0c7c1067f0 2024-09-10T04:39:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.585+0000 7f0c727fc640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0c6807ac40 0x7f0c6807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:39:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.586+0000 7f0c810e3640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0c6807ac40 0x7f0c6807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:39:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.586+0000 7f0c727fc640 1 -- 172.21.15.46:0/345443167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f0c6c0be6d0 con 0x7f0c7c1067f0 2024-09-10T04:39:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.586+0000 7f0c810e3640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0c6807ac40 0x7f0c6807d100 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c198a10 tx=0x7f0c78008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:39:33.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.590+0000 7f0c727fc640 1 -- 172.21.15.46:0/345443167 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f0c6c089e90 con 0x7f0c7c1067f0 2024-09-10T04:39:33.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:33 smithi046 ceph-mon[109506]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:33.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:33 smithi046 ceph-mon[109506]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:33.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:33 smithi046 ceph-mon[109506]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:33.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:33 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1950724942' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:39:33.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:33 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3094155193' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:39:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.851+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f0c7c109d90 con 0x7f0c7c1067f0 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.851+0000 7f0c727fc640 1 -- 172.21.15.46:0/345443167 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f0c6c08ed40 con 0x7f0c7c1067f0 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:39:33.853 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:39:33.854 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:39:33.855 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:39:33.855 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:39:33.855 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:39:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0c6807ac40 msgr2=0x7f0c6807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0c6807ac40 0x7f0c6807d100 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c198a10 tx=0x7f0c78008040 comp rx=0 tx=0).stop 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1067f0 msgr2=0x7f0c7c19dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1067f0 0x7f0c7c19dc50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c031460 tx=0x7f0c6c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 shutdown_connections 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f0c6807ac40 0x7f0c6807d100 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c7c1067f0 0x7f0c7c19dc50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 --2- 172.21.15.46:0/345443167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c7c1007f0 0x7f0c7c19d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 >> 172.21.15.46:0/345443167 conn(0x7f0c7c0fc550 msgr2=0x7f0c7c069ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 shutdown_connections 2024-09-10T04:39:33.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:39:33.853+0000 7f0c83347640 1 -- 172.21.15.46:0/345443167 wait complete. 2024-09-10T04:39:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:33 smithi070 ceph-mon[79774]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:33 smithi070 ceph-mon[79774]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:33 smithi070 ceph-mon[79774]: from='client.35690 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:33 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1950724942' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:39:34.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:33 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3094155193' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:39:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:34 smithi070 ceph-mon[79774]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:34 smithi070 ceph-mon[79774]: pgmap v1018: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:34 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/345443167' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:39:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:34 smithi046 ceph-mon[109506]: from='client.35702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:39:35.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:34 smithi046 ceph-mon[109506]: pgmap v1018: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:35.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:34 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/345443167' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:39:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:36 smithi070 ceph-mon[79774]: pgmap v1019: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:36 smithi046 ceph-mon[109506]: pgmap v1019: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:38 smithi070 ceph-mon[79774]: pgmap v1020: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:38 smithi046 ceph-mon[109506]: pgmap v1020: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:40 smithi070 ceph-mon[79774]: pgmap v1021: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:40 smithi046 ceph-mon[109506]: pgmap v1021: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:42.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:42 smithi070 ceph-mon[79774]: pgmap v1022: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:42.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:42 smithi046 ceph-mon[109506]: pgmap v1022: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:44 smithi070 ceph-mon[79774]: pgmap v1023: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:44 smithi046 ceph-mon[109506]: pgmap v1023: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:46 smithi070 ceph-mon[79774]: pgmap v1024: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:46 smithi046 ceph-mon[109506]: pgmap v1024: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:48 smithi070 ceph-mon[79774]: pgmap v1025: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:48 smithi046 ceph-mon[109506]: pgmap v1025: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:50 smithi070 ceph-mon[79774]: pgmap v1026: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:50 smithi046 ceph-mon[109506]: pgmap v1026: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:52 smithi070 ceph-mon[79774]: pgmap v1027: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:53.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:52 smithi046 ceph-mon[109506]: pgmap v1027: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:39:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:54 smithi070 ceph-mon[79774]: pgmap v1028: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:54 smithi046 ceph-mon[109506]: pgmap v1028: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:56.947 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:56 smithi070 ceph-mon[79774]: pgmap v1029: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:56 smithi046 ceph-mon[109506]: pgmap v1029: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:39:58 smithi070 ceph-mon[79774]: pgmap v1030: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:39:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:39:58 smithi046 ceph-mon[109506]: pgmap v1030: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:00.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:00 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:39:59.999+0000 7f3ca9819640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:40:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:00 smithi070 ceph-mon[79774]: pgmap v1031: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:00 smithi070 ceph-mon[79774]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:40:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:00 smithi046 ceph-mon[109506]: pgmap v1031: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:00 smithi046 ceph-mon[109506]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:40:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:02 smithi070 ceph-mon[79774]: pgmap v1032: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:02 smithi046 ceph-mon[109506]: pgmap v1032: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.951+0000 7fdd863a8640 1 -- 172.21.15.46:0/4066463770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80100800 msgr2=0x7fdd80100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.951+0000 7fdd863a8640 1 --2- 172.21.15.46:0/4066463770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80100800 0x7fdd80100c60 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009880 tx=0x7fdd7402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.951+0000 7fdd863a8640 1 -- 172.21.15.46:0/4066463770 shutdown_connections 2024-09-10T04:40:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.951+0000 7fdd863a8640 1 --2- 172.21.15.46:0/4066463770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80100800 0x7fdd80100c60 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:03.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.951+0000 7fdd863a8640 1 --2- 172.21.15.46:0/4066463770 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd80106800 0x7fdd80106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.951+0000 7fdd863a8640 1 -- 172.21.15.46:0/4066463770 >> 172.21.15.46:0/4066463770 conn(0x7fdd800fc520 msgr2=0x7fdd800fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.952+0000 7fdd863a8640 1 -- 172.21.15.46:0/4066463770 shutdown_connections 2024-09-10T04:40:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.952+0000 7fdd863a8640 1 -- 172.21.15.46:0/4066463770 wait complete. 2024-09-10T04:40:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.952+0000 7fdd863a8640 1 Processor -- start 2024-09-10T04:40:03.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.952+0000 7fdd863a8640 1 -- start start 2024-09-10T04:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd863a8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd80100800 0x7fdd801987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd863a8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80106800 0x7fdd80198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd863a8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010dcb0 con 0x7fdd80106800 2024-09-10T04:40:03.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd863a8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8010db30 con 0x7fdd80100800 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80106800 0x7fdd80198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80106800 0x7fdd80198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46662/0 (socket says 172.21.15.46:46662) 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7f7fe640 1 -- 172.21.15.46:0/3942870691 learned_addr learned my addr 172.21.15.46:0/3942870691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7ffff640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd80100800 0x7fdd801987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7f7fe640 1 -- 172.21.15.46:0/3942870691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd80100800 msgr2=0x7fdd801987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7f7fe640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd80100800 0x7fdd801987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7f7fe640 1 -- 172.21.15.46:0/3942870691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd80199300 con 0x7fdd80106800 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.953+0000 7fdd7ffff640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd80100800 0x7fdd801987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.954+0000 7fdd7f7fe640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80106800 0x7fdd80198d00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdd74031460 tx=0x7fdd740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.954+0000 7fdd7d7fa640 1 -- 172.21.15.46:0/3942870691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdd7403f070 con 0x7fdd80106800 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.954+0000 7fdd7d7fa640 1 -- 172.21.15.46:0/3942870691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdd740035f0 con 0x7fdd80106800 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.954+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd80199590 con 0x7fdd80106800 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.954+0000 7fdd7d7fa640 1 -- 172.21.15.46:0/3942870691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdd74035030 con 0x7fdd80106800 2024-09-10T04:40:03.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.954+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801a1b40 con 0x7fdd80106800 2024-09-10T04:40:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.955+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd80101f40 con 0x7fdd80106800 2024-09-10T04:40:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.956+0000 7fdd7d7fa640 1 -- 172.21.15.46:0/3942870691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdd74042070 con 0x7fdd80106800 2024-09-10T04:40:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.956+0000 7fdd7d7fa640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd6807ac40 0x7fdd6807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:03.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.956+0000 7fdd7d7fa640 1 -- 172.21.15.46:0/3942870691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdd740be5a0 con 0x7fdd80106800 2024-09-10T04:40:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.956+0000 7fdd7ffff640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd6807ac40 0x7fdd6807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:03.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.957+0000 7fdd7ffff640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd6807ac40 0x7fdd6807d100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdd700058c0 tx=0x7fdd70005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:03.961+0000 7fdd7d7fa640 1 -- 172.21.15.46:0/3942870691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdd74089d60 con 0x7fdd80106800 2024-09-10T04:40:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.163+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fdd80062630 con 0x7fdd6807ac40 2024-09-10T04:40:04.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.164+0000 7fdd7d7fa640 1 -- 172.21.15.46:0/3942870691 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fdd80062630 con 0x7fdd6807ac40 2024-09-10T04:40:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd6807ac40 msgr2=0x7fdd6807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd6807ac40 0x7fdd6807d100 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdd700058c0 tx=0x7fdd70005850 comp rx=0 tx=0).stop 2024-09-10T04:40:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80106800 msgr2=0x7fdd80198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80106800 0x7fdd80198d00 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdd74031460 tx=0x7fdd740045e0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 shutdown_connections 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdd6807ac40 0x7fdd6807d100 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80106800 0x7fdd80198d00 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 --2- 172.21.15.46:0/3942870691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd80100800 0x7fdd801987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.166+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 >> 172.21.15.46:0/3942870691 conn(0x7fdd800fc520 msgr2=0x7fdd800fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.167+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 shutdown_connections 2024-09-10T04:40:04.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.167+0000 7fdd863a8640 1 -- 172.21.15.46:0/3942870691 wait complete. 2024-09-10T04:40:04.177 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:40:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 -- 172.21.15.46:0/1602851077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 msgr2=0x7faa3c104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 --2- 172.21.15.46:0/1602851077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c104c50 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7faa3000a990 tx=0x7faa3002b7d0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 -- 172.21.15.46:0/1602851077 shutdown_connections 2024-09-10T04:40:04.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 --2- 172.21.15.46:0/1602851077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c104c50 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 --2- 172.21.15.46:0/1602851077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c100250 0x7faa3c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 -- 172.21.15.46:0/1602851077 >> 172.21.15.46:0/1602851077 conn(0x7faa3c0fbc90 msgr2=0x7faa3c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 -- 172.21.15.46:0/1602851077 shutdown_connections 2024-09-10T04:40:04.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.252+0000 7faa4236d640 1 -- 172.21.15.46:0/1602851077 wait complete. 2024-09-10T04:40:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.253+0000 7faa4236d640 1 Processor -- start 2024-09-10T04:40:04.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.253+0000 7faa4236d640 1 -- start start 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.253+0000 7faa4236d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c100250 0x7faa3c195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.253+0000 7faa4236d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.253+0000 7faa4236d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c10cbd0 con 0x7faa3c100c00 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.253+0000 7faa4236d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7faa3c10ca50 con 0x7faa3c100250 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.253+0000 7faa3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa3b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46670/0 (socket says 172.21.15.46:46670) 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa3b7fe640 1 -- 172.21.15.46:0/1456226749 learned_addr learned my addr 172.21.15.46:0/1456226749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa3bfff640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c100250 0x7faa3c195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa3b7fe640 1 -- 172.21.15.46:0/1456226749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c100250 msgr2=0x7faa3c195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa3b7fe640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c100250 0x7faa3c195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa3b7fe640 1 -- 172.21.15.46:0/1456226749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3c1960c0 con 0x7faa3c100c00 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa3b7fe640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c195b80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7faa30006010 tx=0x7faa30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:04.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa397fa640 1 -- 172.21.15.46:0/1456226749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa30008e90 con 0x7faa3c100c00 2024-09-10T04:40:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3c196350 con 0x7faa3c100c00 2024-09-10T04:40:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa397fa640 1 -- 172.21.15.46:0/1456226749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa30045070 con 0x7faa3c100c00 2024-09-10T04:40:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa397fa640 1 -- 172.21.15.46:0/1456226749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa300035f0 con 0x7faa3c100c00 2024-09-10T04:40:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.254+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3c1a3e10 con 0x7faa3c100c00 2024-09-10T04:40:04.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.256+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa3c101730 con 0x7faa3c100c00 2024-09-10T04:40:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.256+0000 7faa397fa640 1 -- 172.21.15.46:0/1456226749 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7faa30032070 con 0x7faa3c100c00 2024-09-10T04:40:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.257+0000 7faa397fa640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa2407abf0 0x7faa2407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.257+0000 7faa397fa640 1 -- 172.21.15.46:0/1456226749 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7faa30008030 con 0x7faa3c100c00 2024-09-10T04:40:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.257+0000 7faa3bfff640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa2407abf0 0x7faa2407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.257+0000 7faa3bfff640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa2407abf0 0x7faa2407d0b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faa2c005fd0 tx=0x7faa2c005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:04.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.261+0000 7faa397fa640 1 -- 172.21.15.46:0/1456226749 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7faa3004f020 con 0x7faa3c100c00 2024-09-10T04:40:04.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.464+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7faa3c196b70 con 0x7faa2407abf0 2024-09-10T04:40:04.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.465+0000 7faa397fa640 1 -- 172.21.15.46:0/1456226749 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7faa3c196b70 con 0x7faa2407abf0 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa2407abf0 msgr2=0x7faa2407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa2407abf0 0x7faa2407d0b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faa2c005fd0 tx=0x7faa2c005850 comp rx=0 tx=0).stop 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 msgr2=0x7faa3c195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c195b80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7faa30006010 tx=0x7faa30004270 comp rx=0 tx=0).stop 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 shutdown_connections 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa2407abf0 0x7faa2407d0b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faa3c100c00 0x7faa3c195b80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 --2- 172.21.15.46:0/1456226749 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faa3c100250 0x7faa3c195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 >> 172.21.15.46:0/1456226749 conn(0x7faa3c0fbc90 msgr2=0x7faa3c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 shutdown_connections 2024-09-10T04:40:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.467+0000 7faa4236d640 1 -- 172.21.15.46:0/1456226749 wait complete. 2024-09-10T04:40:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 -- 172.21.15.46:0/1751118428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741067f0 msgr2=0x7f9274106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 --2- 172.21.15.46:0/1751118428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741067f0 0x7f9274106bd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9270009880 tx=0x7f927002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 -- 172.21.15.46:0/1751118428 shutdown_connections 2024-09-10T04:40:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 --2- 172.21.15.46:0/1751118428 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92741007f0 0x7f9274100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 --2- 172.21.15.46:0/1751118428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741067f0 0x7f9274106bd0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 -- 172.21.15.46:0/1751118428 >> 172.21.15.46:0/1751118428 conn(0x7f92740fc550 msgr2=0x7f92740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:04.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 -- 172.21.15.46:0/1751118428 shutdown_connections 2024-09-10T04:40:04.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.551+0000 7f927b5c5640 1 -- 172.21.15.46:0/1751118428 wait complete. 2024-09-10T04:40:04.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f927b5c5640 1 Processor -- start 2024-09-10T04:40:04.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f927b5c5640 1 -- start start 2024-09-10T04:40:04.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f927b5c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741007f0 0x7f927419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f927b5c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92741067f0 0x7f927419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f927b5c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f927410dca0 con 0x7f92741007f0 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f927b5c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f927410db20 con 0x7f92741067f0 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f9279361640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741007f0 0x7f927419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f9279361640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741007f0 0x7f927419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46694/0 (socket says 172.21.15.46:46694) 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f9279361640 1 -- 172.21.15.46:0/3128815302 learned_addr learned my addr 172.21.15.46:0/3128815302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.552+0000 7f9278b60640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92741067f0 0x7f927419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f9279361640 1 -- 172.21.15.46:0/3128815302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92741067f0 msgr2=0x7f927419db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f9279361640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92741067f0 0x7f927419db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f9279361640 1 -- 172.21.15.46:0/3128815302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9274197790 con 0x7f92741007f0 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f9278b60640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92741067f0 0x7f927419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:40:04.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f9279361640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741007f0 0x7f927419d610 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9270031460 tx=0x7f92700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f92667fc640 1 -- 172.21.15.46:0/3128815302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f927003f070 con 0x7f92741007f0 2024-09-10T04:40:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f92667fc640 1 -- 172.21.15.46:0/3128815302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9270004cf0 con 0x7f92741007f0 2024-09-10T04:40:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9274197a20 con 0x7f92741007f0 2024-09-10T04:40:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9274197f50 con 0x7f92741007f0 2024-09-10T04:40:04.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.553+0000 7f92667fc640 1 -- 172.21.15.46:0/3128815302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f9270035030 con 0x7f92741007f0 2024-09-10T04:40:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.554+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9258005290 con 0x7f92741007f0 2024-09-10T04:40:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.555+0000 7f92667fc640 1 -- 172.21.15.46:0/3128815302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f9270042070 con 0x7f92741007f0 2024-09-10T04:40:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.555+0000 7f92667fc640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f926007abf0 0x7f926007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.555+0000 7f92667fc640 1 -- 172.21.15.46:0/3128815302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f92700be680 con 0x7f92741007f0 2024-09-10T04:40:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.556+0000 7f9278b60640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f926007abf0 0x7f926007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.556+0000 7f9278b60640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f926007abf0 0x7f926007d0b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f92740ffb10 tx=0x7f9268005d90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:04.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.560+0000 7f92667fc640 1 -- 172.21.15.46:0/3128815302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f9270089dc0 con 0x7f92741007f0 2024-09-10T04:40:04.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.754+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f9258000bc0 con 0x7f926007abf0 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.764+0000 7f92667fc640 1 -- 172.21.15.46:0/3128815302 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f9258000bc0 con 0x7f926007abf0 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (31m) 10m ago 42m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (34m) 10m ago 43m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (34m) 10m ago 42m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (30m) 10m ago 43m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (29m) 10m ago 42m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:40:04.766 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (31m) 10m ago 42m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (26m) 10m ago 37m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (26m) 10m ago 37m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (25m) 10m ago 37m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (10m) 10m ago 37m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (33m) 10m ago 44m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (32m) 10m ago 42m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (30m) 10m ago 44m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (30m) 10m ago 42m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (32m) 10m ago 43m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:40:04.767 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (32m) 10m ago 42m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:40:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (29m) 10m ago 40m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:40:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (29m) 10m ago 40m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:40:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (28m) 10m ago 39m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:40:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (27m) 10m ago 39m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:40:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (27m) 10m ago 39m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:40:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (26m) 10m ago 38m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:40:04.768 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (31m) 10m ago 42m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:40:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f926007abf0 msgr2=0x7f926007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f926007abf0 0x7f926007d0b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f92740ffb10 tx=0x7f9268005d90 comp rx=0 tx=0).stop 2024-09-10T04:40:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741007f0 msgr2=0x7f927419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741007f0 0x7f927419d610 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9270031460 tx=0x7f92700048f0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 shutdown_connections 2024-09-10T04:40:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f926007abf0 0x7f926007d0b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92741067f0 0x7f927419db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 --2- 172.21.15.46:0/3128815302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92741007f0 0x7f927419d610 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 >> 172.21.15.46:0/3128815302 conn(0x7f92740fc550 msgr2=0x7f9274069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 shutdown_connections 2024-09-10T04:40:04.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.766+0000 7f927b5c5640 1 -- 172.21.15.46:0/3128815302 wait complete. 2024-09-10T04:40:04.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:04 smithi046 ceph-mon[109506]: pgmap v1033: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.849+0000 7f673e349640 1 -- 172.21.15.46:0/3518014246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 msgr2=0x7f6738101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.849+0000 7f673e349640 1 --2- 172.21.15.46:0/3518014246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f6738101d60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f672c009880 tx=0x7f672c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.849+0000 7f673e349640 1 -- 172.21.15.46:0/3518014246 shutdown_connections 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.849+0000 7f673e349640 1 --2- 172.21.15.46:0/3518014246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f6738101d60 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.849+0000 7f673e349640 1 --2- 172.21.15.46:0/3518014246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738107900 0x7f6738107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.849+0000 7f673e349640 1 -- 172.21.15.46:0/3518014246 >> 172.21.15.46:0/3518014246 conn(0x7f67380fd600 msgr2=0x7f67380ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.850+0000 7f673e349640 1 -- 172.21.15.46:0/3518014246 shutdown_connections 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.850+0000 7f673e349640 1 -- 172.21.15.46:0/3518014246 wait complete. 2024-09-10T04:40:04.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.850+0000 7f673e349640 1 Processor -- start 2024-09-10T04:40:04.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.850+0000 7f673e349640 1 -- start start 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f673e349640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f673819d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f673e349640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738107900 0x7f673819dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f673e349640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f673810ee30 con 0x7f6738101900 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f673e349640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f673810ecb0 con 0x7f6738107900 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f6737fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f673819d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f6737fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f673819d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46720/0 (socket says 172.21.15.46:46720) 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f6737fff640 1 -- 172.21.15.46:0/3368823994 learned_addr learned my addr 172.21.15.46:0/3368823994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:04.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f67377fe640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738107900 0x7f673819dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f6737fff640 1 -- 172.21.15.46:0/3368823994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738107900 msgr2=0x7f673819dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f6737fff640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738107900 0x7f673819dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f6737fff640 1 -- 172.21.15.46:0/3368823994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6738197840 con 0x7f6738101900 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f6737fff640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f673819d6c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f672800b420 tx=0x7f672800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f67357fa640 1 -- 172.21.15.46:0/3368823994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6728011630 con 0x7f6738101900 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f67357fa640 1 -- 172.21.15.46:0/3368823994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6728016070 con 0x7f6738101900 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6738197b30 con 0x7f6738101900 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.851+0000 7f67357fa640 1 -- 172.21.15.46:0/3368823994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6728010d40 con 0x7f6738101900 2024-09-10T04:40:04.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.852+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67381980c0 con 0x7f6738101900 2024-09-10T04:40:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.853+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6738103040 con 0x7f6738101900 2024-09-10T04:40:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.853+0000 7f67357fa640 1 -- 172.21.15.46:0/3368823994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f67280117d0 con 0x7f6738101900 2024-09-10T04:40:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.854+0000 7f67357fa640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f672007abf0 0x7f672007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.854+0000 7f67357fa640 1 -- 172.21.15.46:0/3368823994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f672809d4a0 con 0x7f6738101900 2024-09-10T04:40:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.854+0000 7f67377fe640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f672007abf0 0x7f672007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:04.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.854+0000 7f67377fe640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f672007abf0 0x7f672007d0b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f672c031460 tx=0x7f672c009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:04.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:04.858+0000 7f67357fa640 1 -- 172.21.15.46:0/3368823994 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6728068b40 con 0x7f6738101900 2024-09-10T04:40:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:04 smithi070 ceph-mon[79774]: pgmap v1033: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.120+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f6738109e90 con 0x7f6738101900 2024-09-10T04:40:05.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.121+0000 7f67357fa640 1 -- 172.21.15.46:0/3368823994 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f672806d9f0 con 0x7f6738101900 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:05.123 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:40:05.124 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:40:05.124 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:40:05.124 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f672007abf0 msgr2=0x7f672007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f672007abf0 0x7f672007d0b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f672c031460 tx=0x7f672c009560 comp rx=0 tx=0).stop 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 msgr2=0x7f673819d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f673819d6c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f672800b420 tx=0x7f672800b8e0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 shutdown_connections 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f672007abf0 0x7f672007d0b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6738107900 0x7f673819dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 --2- 172.21.15.46:0/3368823994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6738101900 0x7f673819d6c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 >> 172.21.15.46:0/3368823994 conn(0x7f67380fd600 msgr2=0x7f67380fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 shutdown_connections 2024-09-10T04:40:05.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.123+0000 7f673e349640 1 -- 172.21.15.46:0/3368823994 wait complete. 2024-09-10T04:40:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 -- 172.21.15.46:0/3255605068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc107900 msgr2=0x7f26cc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3255605068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc107900 0x7f26cc107ce0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009880 tx=0x7f26bc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 -- 172.21.15.46:0/3255605068 shutdown_connections 2024-09-10T04:40:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3255605068 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc101900 0x7f26cc101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3255605068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc107900 0x7f26cc107ce0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 -- 172.21.15.46:0/3255605068 >> 172.21.15.46:0/3255605068 conn(0x7f26cc0fd5e0 msgr2=0x7f26cc0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 -- 172.21.15.46:0/3255605068 shutdown_connections 2024-09-10T04:40:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.207+0000 7f26d25e8640 1 -- 172.21.15.46:0/3255605068 wait complete. 2024-09-10T04:40:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.208+0000 7f26d25e8640 1 Processor -- start 2024-09-10T04:40:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.208+0000 7f26d25e8640 1 -- start start 2024-09-10T04:40:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.208+0000 7f26d25e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc101900 0x7f26cc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.208+0000 7f26d25e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc107900 0x7f26cc19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.208+0000 7f26d25e8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc10edb0 con 0x7f26cc101900 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.208+0000 7f26d25e8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f26cc10ec30 con 0x7f26cc107900 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.208+0000 7f26cbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc101900 0x7f26cc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc101900 0x7f26cc19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46742/0 (socket says 172.21.15.46:46742) 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cbfff640 1 -- 172.21.15.46:0/3122541423 learned_addr learned my addr 172.21.15.46:0/3122541423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cb7fe640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc107900 0x7f26cc19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cbfff640 1 -- 172.21.15.46:0/3122541423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc107900 msgr2=0x7f26cc19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cbfff640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc107900 0x7f26cc19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cbfff640 1 -- 172.21.15.46:0/3122541423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26cc199a10 con 0x7f26cc101900 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cb7fe640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc107900 0x7f26cc19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:40:05.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26cbfff640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc101900 0x7f26cc19f850 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f26bc031460 tx=0x7f26bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26c97fa640 1 -- 172.21.15.46:0/3122541423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26bc03f070 con 0x7f26cc101900 2024-09-10T04:40:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26c97fa640 1 -- 172.21.15.46:0/3122541423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f26bc0035f0 con 0x7f26cc101900 2024-09-10T04:40:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26cc199ca0 con 0x7f26cc101900 2024-09-10T04:40:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26c97fa640 1 -- 172.21.15.46:0/3122541423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f26bc035030 con 0x7f26cc101900 2024-09-10T04:40:05.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.209+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26cc19a1d0 con 0x7f26cc101900 2024-09-10T04:40:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.210+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26cc103040 con 0x7f26cc101900 2024-09-10T04:40:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.211+0000 7f26c97fa640 1 -- 172.21.15.46:0/3122541423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f26bc042070 con 0x7f26cc101900 2024-09-10T04:40:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.211+0000 7f26c97fa640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26b407ac40 0x7f26b407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.212+0000 7f26c97fa640 1 -- 172.21.15.46:0/3122541423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f26bc0be640 con 0x7f26cc101900 2024-09-10T04:40:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.212+0000 7f26cb7fe640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26b407ac40 0x7f26b407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.212+0000 7f26cb7fe640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26b407ac40 0x7f26b407d100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f26cc100b50 tx=0x7f26c000b5f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:05.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.216+0000 7f26c97fa640 1 -- 172.21.15.46:0/3122541423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f26bc089d60 con 0x7f26cc101900 2024-09-10T04:40:05.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.439+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f26cc19b010 con 0x7f26cc101900 2024-09-10T04:40:05.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.439+0000 7f26c97fa640 1 -- 172.21.15.46:0/3122541423 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f26bc08ec10 con 0x7f26cc101900 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:40:05.442 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:40:05.443 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:05.444 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:05.445 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:40:05.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.441+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26b407ac40 msgr2=0x7f26b407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.441+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26b407ac40 0x7f26b407d100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f26cc100b50 tx=0x7f26c000b5f0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.441+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc101900 msgr2=0x7f26cc19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.441+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc101900 0x7f26cc19f850 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f26bc031460 tx=0x7f26bc0045e0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.442+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 shutdown_connections 2024-09-10T04:40:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.442+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f26b407ac40 0x7f26b407d100 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.442+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f26cc107900 0x7f26cc19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.442+0000 7f26d25e8640 1 --2- 172.21.15.46:0/3122541423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26cc101900 0x7f26cc19f850 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.442+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 >> 172.21.15.46:0/3122541423 conn(0x7f26cc0fd5e0 msgr2=0x7f26cc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.442+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 shutdown_connections 2024-09-10T04:40:05.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.442+0000 7f26d25e8640 1 -- 172.21.15.46:0/3122541423 wait complete. 2024-09-10T04:40:05.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/517330173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d841033e0 msgr2=0x7f6d8410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/517330173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d841033e0 0x7f6d8410b8d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009880 tx=0x7f6d7802a530 comp rx=0 tx=0).stop 2024-09-10T04:40:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/517330173 shutdown_connections 2024-09-10T04:40:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/517330173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d841033e0 0x7f6d8410b8d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/517330173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d84102ac0 0x7f6d84102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/517330173 >> 172.21.15.46:0/517330173 conn(0x7f6d840fc520 msgr2=0x7f6d840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/517330173 shutdown_connections 2024-09-10T04:40:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.526+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/517330173 wait complete. 2024-09-10T04:40:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.527+0000 7f6d8a2d7640 1 Processor -- start 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.527+0000 7f6d8a2d7640 1 -- start start 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.527+0000 7f6d8a2d7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d84102ac0 0x7f6d8419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.527+0000 7f6d8a2d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d841033e0 0x7f6d8419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d8a2d7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410edf0 con 0x7f6d84102ac0 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d8a2d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410ec70 con 0x7f6d841033e0 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d83fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d84102ac0 0x7f6d8419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d83fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d84102ac0 0x7f6d8419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46756/0 (socket says 172.21.15.46:46756) 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d83fff640 1 -- 172.21.15.46:0/2637127300 learned_addr learned my addr 172.21.15.46:0/2637127300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d837fe640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d841033e0 0x7f6d8419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d83fff640 1 -- 172.21.15.46:0/2637127300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d841033e0 msgr2=0x7f6d8419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d83fff640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d841033e0 0x7f6d8419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d83fff640 1 -- 172.21.15.46:0/2637127300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d84199a50 con 0x7f6d84102ac0 2024-09-10T04:40:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d83fff640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d84102ac0 0x7f6d8419f890 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e970 tx=0x7f6d7400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d817fa640 1 -- 172.21.15.46:0/2637127300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d7400cac0 con 0x7f6d84102ac0 2024-09-10T04:40:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d817fa640 1 -- 172.21.15.46:0/2637127300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6d740054e0 con 0x7f6d84102ac0 2024-09-10T04:40:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d84199d40 con 0x7f6d84102ac0 2024-09-10T04:40:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d817fa640 1 -- 172.21.15.46:0/2637127300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d74010430 con 0x7f6d84102ac0 2024-09-10T04:40:05.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.528+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8419a2a0 con 0x7f6d84102ac0 2024-09-10T04:40:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.530+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8406aa90 con 0x7f6d84102ac0 2024-09-10T04:40:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.530+0000 7f6d817fa640 1 -- 172.21.15.46:0/2637127300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6d74004f00 con 0x7f6d84102ac0 2024-09-10T04:40:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.531+0000 7f6d817fa640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d6c07ac40 0x7f6d6c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.531+0000 7f6d817fa640 1 -- 172.21.15.46:0/2637127300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6d7409e2d0 con 0x7f6d84102ac0 2024-09-10T04:40:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.531+0000 7f6d837fe640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d6c07ac40 0x7f6d6c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.531+0000 7f6d837fe640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d6c07ac40 0x7f6d6c07d100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6d8419b240 tx=0x7f6d78031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.535+0000 7f6d817fa640 1 -- 172.21.15.46:0/2637127300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6d74069a90 con 0x7f6d84102ac0 2024-09-10T04:40:05.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.741+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f6d8419af80 con 0x7f6d6c07ac40 2024-09-10T04:40:05.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.743+0000 7f6d817fa640 1 -- 172.21.15.46:0/2637127300 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f6d8419af80 con 0x7f6d6c07ac40 2024-09-10T04:40:05.744 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:40:05.744 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:40:05.745 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d6c07ac40 msgr2=0x7f6d6c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d6c07ac40 0x7f6d6c07d100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6d8419b240 tx=0x7f6d78031400 comp rx=0 tx=0).stop 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d84102ac0 msgr2=0x7f6d8419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d84102ac0 0x7f6d8419f890 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e970 tx=0x7f6d7400ed70 comp rx=0 tx=0).stop 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 shutdown_connections 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d6c07ac40 0x7f6d6c07d100 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d841033e0 0x7f6d8419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 --2- 172.21.15.46:0/2637127300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d84102ac0 0x7f6d8419f890 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 >> 172.21.15.46:0/2637127300 conn(0x7f6d840fc520 msgr2=0x7f6d84100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:05.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 shutdown_connections 2024-09-10T04:40:05.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.745+0000 7f6d8a2d7640 1 -- 172.21.15.46:0/2637127300 wait complete. 2024-09-10T04:40:05.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2676979131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018100800 msgr2=0x7fb018100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2676979131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018100800 0x7fb018100c60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009880 tx=0x7fb00c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2676979131 shutdown_connections 2024-09-10T04:40:05.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2676979131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018100800 0x7fb018100c60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2676979131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb018106800 0x7fb018106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2676979131 >> 172.21.15.46:0/2676979131 conn(0x7fb0180fc520 msgr2=0x7fb0180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:05.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2676979131 shutdown_connections 2024-09-10T04:40:05.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.829+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2676979131 wait complete. 2024-09-10T04:40:05.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.830+0000 7fb01c8ba640 1 Processor -- start 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.830+0000 7fb01c8ba640 1 -- start start 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.830+0000 7fb01c8ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb018100800 0x7fb01819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.830+0000 7fb01c8ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018106800 0x7fb01819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.830+0000 7fb01c8ba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb01810edd0 con 0x7fb018106800 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.830+0000 7fb01c8ba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb01810ec50 con 0x7fb018100800 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.830+0000 7fb015d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018106800 0x7fb01819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb015d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018106800 0x7fb01819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46766/0 (socket says 172.21.15.46:46766) 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb015d9b640 1 -- 172.21.15.46:0/2598799999 learned_addr learned my addr 172.21.15.46:0/2598799999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb01659c640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb018100800 0x7fb01819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb015d9b640 1 -- 172.21.15.46:0/2598799999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb018100800 msgr2=0x7fb01819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb015d9b640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb018100800 0x7fb01819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:05.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb015d9b640 1 -- 172.21.15.46:0/2598799999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0181977e0 con 0x7fb018106800 2024-09-10T04:40:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb015d9b640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018106800 0x7fb01819dba0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb00c031460 tx=0x7fb00c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb0077fe640 1 -- 172.21.15.46:0/2598799999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb00c03f070 con 0x7fb018106800 2024-09-10T04:40:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb018197a70 con 0x7fb018106800 2024-09-10T04:40:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb0077fe640 1 -- 172.21.15.46:0/2598799999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb00c004cf0 con 0x7fb018106800 2024-09-10T04:40:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb018197fa0 con 0x7fb018106800 2024-09-10T04:40:05.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.831+0000 7fb0077fe640 1 -- 172.21.15.46:0/2598799999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb00c035030 con 0x7fb018106800 2024-09-10T04:40:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.832+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb018101f40 con 0x7fb018106800 2024-09-10T04:40:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.833+0000 7fb0077fe640 1 -- 172.21.15.46:0/2598799999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb00c042070 con 0x7fb018106800 2024-09-10T04:40:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.834+0000 7fb0077fe640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb00007abf0 0x7fb00007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.834+0000 7fb0077fe640 1 -- 172.21.15.46:0/2598799999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb00c0be7f0 con 0x7fb018106800 2024-09-10T04:40:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.834+0000 7fb01659c640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb00007abf0 0x7fb00007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.834+0000 7fb01659c640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb00007abf0 0x7fb00007d0b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb008005e10 tx=0x7fb008005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:05.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:05.838+0000 7fb0077fe640 1 -- 172.21.15.46:0/2598799999 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb00c089e90 con 0x7fb018106800 2024-09-10T04:40:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:05 smithi070 ceph-mon[79774]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:05 smithi070 ceph-mon[79774]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:05 smithi070 ceph-mon[79774]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:06.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:05 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3368823994' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:40:06.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:05 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3122541423' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:40:06.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:05 smithi046 ceph-mon[109506]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:06.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:05 smithi046 ceph-mon[109506]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:06.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:05 smithi046 ceph-mon[109506]: from='client.35718 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:06.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:05 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3368823994' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:40:06.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:05 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3122541423' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:40:06.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.097+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fb018198d10 con 0x7fb018106800 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.098+0000 7fb0077fe640 1 -- 172.21.15.46:0/2598799999 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fb00c08ed40 con 0x7fb018106800 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:40:06.100 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:40:06.101 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:40:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:40:06.102 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:40:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:40:06.102 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:40:06.102 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb00007abf0 msgr2=0x7fb00007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb00007abf0 0x7fb00007d0b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb008005e10 tx=0x7fb008005da0 comp rx=0 tx=0).stop 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018106800 msgr2=0x7fb01819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018106800 0x7fb01819dba0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb00c031460 tx=0x7fb00c0048f0 comp rx=0 tx=0).stop 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 shutdown_connections 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb00007abf0 0x7fb00007d0b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb018106800 0x7fb01819dba0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 --2- 172.21.15.46:0/2598799999 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb018100800 0x7fb01819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.100+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 >> 172.21.15.46:0/2598799999 conn(0x7fb0180fc520 msgr2=0x7fb0180fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.101+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 shutdown_connections 2024-09-10T04:40:06.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:06.101+0000 7fb01c8ba640 1 -- 172.21.15.46:0/2598799999 wait complete. 2024-09-10T04:40:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:06 smithi070 ceph-mon[79774]: pgmap v1034: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:06 smithi070 ceph-mon[79774]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:06 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2598799999' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:40:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:06 smithi046 ceph-mon[109506]: pgmap v1034: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:06 smithi046 ceph-mon[109506]: from='client.35730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:06 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2598799999' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:40:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:08 smithi070 ceph-mon[79774]: pgmap v1035: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:08 smithi046 ceph-mon[109506]: pgmap v1035: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:10 smithi070 ceph-mon[79774]: pgmap v1036: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:10 smithi046 ceph-mon[109506]: pgmap v1036: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:12 smithi070 ceph-mon[79774]: pgmap v1037: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:12 smithi046 ceph-mon[109506]: pgmap v1037: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:14 smithi070 ceph-mon[79774]: pgmap v1038: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:14 smithi046 ceph-mon[109506]: pgmap v1038: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:16 smithi070 ceph-mon[79774]: pgmap v1039: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:16 smithi046 ceph-mon[109506]: pgmap v1039: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:18 smithi070 ceph-mon[79774]: pgmap v1040: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:18 smithi046 ceph-mon[109506]: pgmap v1040: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:20 smithi070 ceph-mon[79774]: pgmap v1041: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:21.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:20 smithi046 ceph-mon[109506]: pgmap v1041: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:22 smithi070 ceph-mon[79774]: pgmap v1042: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:22 smithi046 ceph-mon[109506]: pgmap v1042: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:23.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:24 smithi070 ceph-mon[79774]: pgmap v1043: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:24 smithi046 ceph-mon[109506]: pgmap v1043: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:26 smithi070 ceph-mon[79774]: pgmap v1044: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:26 smithi046 ceph-mon[109506]: pgmap v1044: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:28 smithi070 ceph-mon[79774]: pgmap v1045: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:28 smithi046 ceph-mon[109506]: pgmap v1045: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:30 smithi070 ceph-mon[79774]: pgmap v1046: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:30 smithi046 ceph-mon[109506]: pgmap v1046: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:32 smithi070 ceph-mon[79774]: pgmap v1047: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:32 smithi046 ceph-mon[109506]: pgmap v1047: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:34 smithi070 ceph-mon[79774]: pgmap v1048: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:34 smithi046 ceph-mon[109506]: pgmap v1048: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.200+0000 7f3a2e19b640 1 -- 172.21.15.46:0/2559129474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 msgr2=0x7f3a28100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.200+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/2559129474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a28100c50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009880 tx=0x7f3a1c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.200+0000 7f3a2e19b640 1 -- 172.21.15.46:0/2559129474 shutdown_connections 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.200+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/2559129474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a28100c50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.200+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/2559129474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a281067f0 0x7f3a28106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.200+0000 7f3a2e19b640 1 -- 172.21.15.46:0/2559129474 >> 172.21.15.46:0/2559129474 conn(0x7f3a280fc550 msgr2=0x7f3a280fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.201+0000 7f3a2e19b640 1 -- 172.21.15.46:0/2559129474 shutdown_connections 2024-09-10T04:40:36.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.201+0000 7f3a2e19b640 1 -- 172.21.15.46:0/2559129474 wait complete. 2024-09-10T04:40:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.201+0000 7f3a2e19b640 1 Processor -- start 2024-09-10T04:40:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a2e19b640 1 -- start start 2024-09-10T04:40:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a2e19b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a2819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a2e19b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a281067f0 0x7f3a2819db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a2e19b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2810dcc0 con 0x7f3a281007f0 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a2e19b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2810db40 con 0x7f3a281067f0 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a2819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a277fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a2819d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56020/0 (socket says 172.21.15.46:56020) 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a277fe640 1 -- 172.21.15.46:0/3164875377 learned_addr learned my addr 172.21.15.46:0/3164875377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a26ffd640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a281067f0 0x7f3a2819db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a277fe640 1 -- 172.21.15.46:0/3164875377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a281067f0 msgr2=0x7f3a2819db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a277fe640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a281067f0 0x7f3a2819db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.202+0000 7f3a277fe640 1 -- 172.21.15.46:0/3164875377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a28197780 con 0x7f3a281007f0 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.203+0000 7f3a277fe640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a2819d600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e970 tx=0x7f3a1800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.203+0000 7f3a24ff9640 1 -- 172.21.15.46:0/3164875377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3a1800cac0 con 0x7f3a281007f0 2024-09-10T04:40:36.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.203+0000 7f3a24ff9640 1 -- 172.21.15.46:0/3164875377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a180054e0 con 0x7f3a281007f0 2024-09-10T04:40:36.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.203+0000 7f3a24ff9640 1 -- 172.21.15.46:0/3164875377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3a18010430 con 0x7f3a281007f0 2024-09-10T04:40:36.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.203+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a28197a70 con 0x7f3a281007f0 2024-09-10T04:40:36.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.203+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a28197fd0 con 0x7f3a281007f0 2024-09-10T04:40:36.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.204+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a28101f30 con 0x7f3a281007f0 2024-09-10T04:40:36.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.205+0000 7f3a24ff9640 1 -- 172.21.15.46:0/3164875377 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3a18004f00 con 0x7f3a281007f0 2024-09-10T04:40:36.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.205+0000 7f3a24ff9640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3a1007ac40 0x7f3a1007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.205+0000 7f3a24ff9640 1 -- 172.21.15.46:0/3164875377 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3a1809e2d0 con 0x7f3a281007f0 2024-09-10T04:40:36.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.205+0000 7f3a26ffd640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3a1007ac40 0x7f3a1007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.206+0000 7f3a26ffd640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3a1007ac40 0x7f3a1007d100 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3a28198ad0 tx=0x7f3a1c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:36.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.210+0000 7f3a24ff9640 1 -- 172.21.15.46:0/3164875377 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3a18069a90 con 0x7f3a281007f0 2024-09-10T04:40:36.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.414+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f3a281982e0 con 0x7f3a1007ac40 2024-09-10T04:40:36.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.415+0000 7f3a24ff9640 1 -- 172.21.15.46:0/3164875377 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f3a281982e0 con 0x7f3a1007ac40 2024-09-10T04:40:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3a1007ac40 msgr2=0x7f3a1007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3a1007ac40 0x7f3a1007d100 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3a28198ad0 tx=0x7f3a1c031400 comp rx=0 tx=0).stop 2024-09-10T04:40:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 msgr2=0x7f3a2819d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a2819d600 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e970 tx=0x7f3a1800ed70 comp rx=0 tx=0).stop 2024-09-10T04:40:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 shutdown_connections 2024-09-10T04:40:36.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3a1007ac40 0x7f3a1007d100 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3a281067f0 0x7f3a2819db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 --2- 172.21.15.46:0/3164875377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a281007f0 0x7f3a2819d600 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 >> 172.21.15.46:0/3164875377 conn(0x7f3a280fc550 msgr2=0x7f3a28104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 shutdown_connections 2024-09-10T04:40:36.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.417+0000 7f3a2e19b640 1 -- 172.21.15.46:0/3164875377 wait complete. 2024-09-10T04:40:36.427 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:40:36.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.504+0000 7f8fefe62640 1 -- 172.21.15.46:0/1549583416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810cf10 msgr2=0x7f8fe8110f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.504+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1549583416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810cf10 0x7f8fe8110f60 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009880 tx=0x7f8fdc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.504+0000 7f8fefe62640 1 -- 172.21.15.46:0/1549583416 shutdown_connections 2024-09-10T04:40:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.504+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1549583416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810cf10 0x7f8fe8110f60 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.504+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1549583416 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe810c560 0x7f8fe810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.504+0000 7f8fefe62640 1 -- 172.21.15.46:0/1549583416 >> 172.21.15.46:0/1549583416 conn(0x7f8fe806c5b0 msgr2=0x7f8fe806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.505+0000 7f8fefe62640 1 -- 172.21.15.46:0/1549583416 shutdown_connections 2024-09-10T04:40:36.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.505+0000 7f8fefe62640 1 -- 172.21.15.46:0/1549583416 wait complete. 2024-09-10T04:40:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.505+0000 7f8fefe62640 1 Processor -- start 2024-09-10T04:40:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.505+0000 7f8fefe62640 1 -- start start 2024-09-10T04:40:36.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.505+0000 7f8fefe62640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810c560 0x7f8fe81a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.505+0000 7f8fefe62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe810cf10 0x7f8fe81a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fefe62640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81197c0 con 0x7f8fe810c560 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fefe62640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8119640 con 0x7f8fe810cf10 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fedbfe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810c560 0x7f8fe81a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fedbfe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810c560 0x7f8fe81a23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56040/0 (socket says 172.21.15.46:56040) 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fedbfe640 1 -- 172.21.15.46:0/1725609558 learned_addr learned my addr 172.21.15.46:0/1725609558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fed3fd640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe810cf10 0x7f8fe81a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fedbfe640 1 -- 172.21.15.46:0/1725609558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe810cf10 msgr2=0x7f8fe81a2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fedbfe640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe810cf10 0x7f8fe81a2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fedbfe640 1 -- 172.21.15.46:0/1725609558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe81a6ca0 con 0x7f8fe810c560 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fedbfe640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810c560 0x7f8fe81a23d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d8d0 tx=0x7f8fe400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:36.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fdaffd640 1 -- 172.21.15.46:0/1725609558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8fe4017030 con 0x7f8fe810c560 2024-09-10T04:40:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fdaffd640 1 -- 172.21.15.46:0/1725609558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8fe4004020 con 0x7f8fe810c560 2024-09-10T04:40:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fdaffd640 1 -- 172.21.15.46:0/1725609558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8fe40047a0 con 0x7f8fe810c560 2024-09-10T04:40:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe81a6f90 con 0x7f8fe810c560 2024-09-10T04:40:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.506+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81a7520 con 0x7f8fe810c560 2024-09-10T04:40:36.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.508+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fe810da40 con 0x7f8fe810c560 2024-09-10T04:40:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.508+0000 7f8fdaffd640 1 -- 172.21.15.46:0/1725609558 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8fe40041d0 con 0x7f8fe810c560 2024-09-10T04:40:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.509+0000 7f8fdaffd640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8fd407ac40 0x7f8fd407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.509+0000 7f8fdaffd640 1 -- 172.21.15.46:0/1725609558 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f8fe4018070 con 0x7f8fe810c560 2024-09-10T04:40:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.509+0000 7f8fed3fd640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8fd407ac40 0x7f8fd407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.509+0000 7f8fed3fd640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8fd407ac40 0x7f8fd407d100 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8fe810be20 tx=0x7f8fdc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:36.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.513+0000 7f8fdaffd640 1 -- 172.21.15.46:0/1725609558 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8fe40690c0 con 0x7f8fe810c560 2024-09-10T04:40:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.721+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8fe81a7830 con 0x7f8fd407ac40 2024-09-10T04:40:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.722+0000 7f8fdaffd640 1 -- 172.21.15.46:0/1725609558 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8fe81a7830 con 0x7f8fd407ac40 2024-09-10T04:40:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8fd407ac40 msgr2=0x7f8fd407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8fd407ac40 0x7f8fd407d100 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8fe810be20 tx=0x7f8fdc031400 comp rx=0 tx=0).stop 2024-09-10T04:40:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810c560 msgr2=0x7f8fe81a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810c560 0x7f8fe81a23d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d8d0 tx=0x7f8fe400dd90 comp rx=0 tx=0).stop 2024-09-10T04:40:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 shutdown_connections 2024-09-10T04:40:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f8fd407ac40 0x7f8fd407d100 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fe810cf10 0x7f8fe81a2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 --2- 172.21.15.46:0/1725609558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fe810c560 0x7f8fe81a23d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 >> 172.21.15.46:0/1725609558 conn(0x7f8fe806c5b0 msgr2=0x7f8fe810a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 shutdown_connections 2024-09-10T04:40:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.724+0000 7f8fefe62640 1 -- 172.21.15.46:0/1725609558 wait complete. 2024-09-10T04:40:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 -- 172.21.15.46:0/3476197802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581007f0 msgr2=0x7f8058100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3476197802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581007f0 0x7f8058100c50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f8048009880 tx=0x7f804802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.811 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:36 smithi046 ceph-mon[109506]: pgmap v1049: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:36.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 -- 172.21.15.46:0/3476197802 shutdown_connections 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3476197802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581007f0 0x7f8058100c50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3476197802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80581067f0 0x7f8058106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 -- 172.21.15.46:0/3476197802 >> 172.21.15.46:0/3476197802 conn(0x7f80580fc550 msgr2=0x7f80580fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 -- 172.21.15.46:0/3476197802 shutdown_connections 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.810+0000 7f805eb9c640 1 -- 172.21.15.46:0/3476197802 wait complete. 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f805eb9c640 1 Processor -- start 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f805eb9c640 1 -- start start 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f805eb9c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80581007f0 0x7f80580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f805eb9c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581067f0 0x7f80580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f805eb9c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f805810dca0 con 0x7f80581067f0 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f805eb9c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f805810db20 con 0x7f80581007f0 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f804ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581067f0 0x7f80580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f804ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581067f0 0x7f80580fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56048/0 (socket says 172.21.15.46:56048) 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f804ffff640 1 -- 172.21.15.46:0/3993882549 learned_addr learned my addr 172.21.15.46:0/3993882549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f805c938640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80581007f0 0x7f80580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f804ffff640 1 -- 172.21.15.46:0/3993882549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80581007f0 msgr2=0x7f80580fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.811+0000 7f804ffff640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80581007f0 0x7f80580fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f804ffff640 1 -- 172.21.15.46:0/3993882549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80580fd700 con 0x7f80581067f0 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f805c938640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80581007f0 0x7f80580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f804ffff640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581067f0 0x7f80580fd1c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8048031460 tx=0x7f80480045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f804dffb640 1 -- 172.21.15.46:0/3993882549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f804803f070 con 0x7f80581067f0 2024-09-10T04:40:36.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80580fd990 con 0x7f80581067f0 2024-09-10T04:40:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f804dffb640 1 -- 172.21.15.46:0/3993882549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80480035f0 con 0x7f80581067f0 2024-09-10T04:40:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f804dffb640 1 -- 172.21.15.46:0/3993882549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8048035030 con 0x7f80581067f0 2024-09-10T04:40:36.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.812+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80581a3d40 con 0x7f80581067f0 2024-09-10T04:40:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.813+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8058101f30 con 0x7f80581067f0 2024-09-10T04:40:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.814+0000 7f804dffb640 1 -- 172.21.15.46:0/3993882549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8048042070 con 0x7f80581067f0 2024-09-10T04:40:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.815+0000 7f804dffb640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f804007abf0 0x7f804007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.815+0000 7f804dffb640 1 -- 172.21.15.46:0/3993882549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f80480be720 con 0x7f80581067f0 2024-09-10T04:40:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.815+0000 7f805c938640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f804007abf0 0x7f804007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:36.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.815+0000 7f805c938640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f804007abf0 0x7f804007d0b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8054009b40 tx=0x7f8054005e50 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:36.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:36.819+0000 7f804dffb640 1 -- 172.21.15.46:0/3993882549 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8048089dc0 con 0x7f80581067f0 2024-09-10T04:40:37.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.011+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f8058062630 con 0x7f804007abf0 2024-09-10T04:40:37.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.020+0000 7f804dffb640 1 -- 172.21.15.46:0/3993882549 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f8058062630 con 0x7f804007abf0 2024-09-10T04:40:37.021 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (32m) 11m ago 43m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (34m) 11m ago 43m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (34m) 11m ago 42m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (30m) 11m ago 43m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (30m) 11m ago 42m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (31m) 11m ago 43m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (26m) 11m ago 38m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (27m) 11m ago 38m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (26m) 11m ago 38m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (11m) 11m ago 38m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (34m) 11m ago 44m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (33m) 11m ago 42m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:40:37.022 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (31m) 11m ago 45m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (30m) 11m ago 42m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (33m) 11m ago 43m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (32m) 11m ago 42m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (30m) 11m ago 41m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (29m) 11m ago 40m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (28m) 11m ago 40m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (28m) 11m ago 40m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (27m) 11m ago 39m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (27m) 11m ago 39m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:40:37.023 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (32m) 11m ago 43m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:40:37.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f804007abf0 msgr2=0x7f804007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f804007abf0 0x7f804007d0b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8054009b40 tx=0x7f8054005e50 comp rx=0 tx=0).stop 2024-09-10T04:40:37.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581067f0 msgr2=0x7f80580fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581067f0 0x7f80580fd1c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f8048031460 tx=0x7f80480045e0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 shutdown_connections 2024-09-10T04:40:37.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f804007abf0 0x7f804007d0b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80581067f0 0x7f80580fd1c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 --2- 172.21.15.46:0/3993882549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80581007f0 0x7f80580fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 >> 172.21.15.46:0/3993882549 conn(0x7f80580fc550 msgr2=0x7f8058069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:37.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 shutdown_connections 2024-09-10T04:40:37.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.023+0000 7f805eb9c640 1 -- 172.21.15.46:0/3993882549 wait complete. 2024-09-10T04:40:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:36 smithi070 ceph-mon[79774]: pgmap v1049: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:37.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 -- 172.21.15.46:0/1920228100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c1033e0 msgr2=0x7f339c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 --2- 172.21.15.46:0/1920228100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c1033e0 0x7f339c10b8d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3390009880 tx=0x7f339002a530 comp rx=0 tx=0).stop 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 -- 172.21.15.46:0/1920228100 shutdown_connections 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 --2- 172.21.15.46:0/1920228100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c1033e0 0x7f339c10b8d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 --2- 172.21.15.46:0/1920228100 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c102ac0 0x7f339c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 -- 172.21.15.46:0/1920228100 >> 172.21.15.46:0/1920228100 conn(0x7f339c0fc520 msgr2=0x7f339c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 -- 172.21.15.46:0/1920228100 shutdown_connections 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.106+0000 7f33a232c640 1 -- 172.21.15.46:0/1920228100 wait complete. 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f33a232c640 1 Processor -- start 2024-09-10T04:40:37.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f33a232c640 1 -- start start 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f33a232c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c102ac0 0x7f339c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f33a232c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c1033e0 0x7f339c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f33a232c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f339c10edf0 con 0x7f339c102ac0 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f33a232c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f339c10ec70 con 0x7f339c1033e0 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f339bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c102ac0 0x7f339c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f339bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c102ac0 0x7f339c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56074/0 (socket says 172.21.15.46:56074) 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.107+0000 7f339bfff640 1 -- 172.21.15.46:0/3601858015 learned_addr learned my addr 172.21.15.46:0/3601858015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f339b7fe640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c1033e0 0x7f339c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f339bfff640 1 -- 172.21.15.46:0/3601858015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c1033e0 msgr2=0x7f339c100c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f339bfff640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c1033e0 0x7f339c100c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f339bfff640 1 -- 172.21.15.46:0/3601858015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c1011a0 con 0x7f339c102ac0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f339bfff640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c102ac0 0x7f339c100720 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f338c00d6f0 tx=0x7f338c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f33997fa640 1 -- 172.21.15.46:0/3601858015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f338c017070 con 0x7f339c102ac0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c101490 con 0x7f339c102ac0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f33997fa640 1 -- 172.21.15.46:0/3601858015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f338c00de00 con 0x7f339c102ac0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f33997fa640 1 -- 172.21.15.46:0/3601858015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f338c004bb0 con 0x7f339c102ac0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.108+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c1a3e80 con 0x7f339c102ac0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.109+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c06aa90 con 0x7f339c102ac0 2024-09-10T04:40:37.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.110+0000 7f33997fa640 1 -- 172.21.15.46:0/3601858015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f338c004e60 con 0x7f339c102ac0 2024-09-10T04:40:37.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.110+0000 7f33997fa640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f338407abf0 0x7f338407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.111+0000 7f33997fa640 1 -- 172.21.15.46:0/3601858015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f338c09d8d0 con 0x7f339c102ac0 2024-09-10T04:40:37.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.111+0000 7f339b7fe640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f338407abf0 0x7f338407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.111+0000 7f339b7fe640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f338407abf0 0x7f338407d0b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f339c1021b0 tx=0x7f3390031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:37.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.115+0000 7f33997fa640 1 -- 172.21.15.46:0/3601858015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f338c069010 con 0x7f339c102ac0 2024-09-10T04:40:37.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.371+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f339c105f60 con 0x7f339c102ac0 2024-09-10T04:40:37.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.371+0000 7f33997fa640 1 -- 172.21.15.46:0/3601858015 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f338c06dec0 con 0x7f339c102ac0 2024-09-10T04:40:37.372 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:40:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:40:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:40:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:37.372 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:40:37.373 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f338407abf0 msgr2=0x7f338407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f338407abf0 0x7f338407d0b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f339c1021b0 tx=0x7f3390031400 comp rx=0 tx=0).stop 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c102ac0 msgr2=0x7f339c100720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c102ac0 0x7f339c100720 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f338c00d6f0 tx=0x7f338c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 shutdown_connections 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f338407abf0 0x7f338407d0b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f339c1033e0 0x7f339c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 --2- 172.21.15.46:0/3601858015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c102ac0 0x7f339c100720 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 >> 172.21.15.46:0/3601858015 conn(0x7f339c0fc520 msgr2=0x7f339c108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:37.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 shutdown_connections 2024-09-10T04:40:37.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.374+0000 7f33a232c640 1 -- 172.21.15.46:0/3601858015 wait complete. 2024-09-10T04:40:37.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.457+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1988632975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 msgr2=0x7fbcd4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.457+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1988632975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd4101d60 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009880 tx=0x7fbcc802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1988632975 shutdown_connections 2024-09-10T04:40:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1988632975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd4101d60 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1988632975 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd4107900 0x7fbcd4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1988632975 >> 172.21.15.46:0/1988632975 conn(0x7fbcd40fd620 msgr2=0x7fbcd40ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1988632975 shutdown_connections 2024-09-10T04:40:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1988632975 wait complete. 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 Processor -- start 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.458+0000 7fbcdb66d640 1 -- start start 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcdb66d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcdb66d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd4107900 0x7fbcd4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcdb66d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd410edb0 con 0x7fbcd4101900 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcdb66d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd410ec30 con 0x7fbcd4107900 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd9409640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd9409640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56084/0 (socket says 172.21.15.46:56084) 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd9409640 1 -- 172.21.15.46:0/1102314532 learned_addr learned my addr 172.21.15.46:0/1102314532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd9409640 1 -- 172.21.15.46:0/1102314532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd4107900 msgr2=0x7fbcd4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd8c08640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd4107900 0x7fbcd4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd9409640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd4107900 0x7fbcd4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd9409640 1 -- 172.21.15.46:0/1102314532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd4074f70 con 0x7fbcd4101900 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd8c08640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd4107900 0x7fbcd4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd9409640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd40744f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e970 tx=0x7fbccc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd27fc640 1 -- 172.21.15.46:0/1102314532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbccc00cac0 con 0x7fbcd4101900 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd27fc640 1 -- 172.21.15.46:0/1102314532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbccc00cc60 con 0x7fbcd4101900 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd4075260 con 0x7fbcd4101900 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.459+0000 7fbcd27fc640 1 -- 172.21.15.46:0/1102314532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbccc018720 con 0x7fbcd4101900 2024-09-10T04:40:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.460+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd41a3e20 con 0x7fbcd4101900 2024-09-10T04:40:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.461+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd4103040 con 0x7fbcd4101900 2024-09-10T04:40:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.461+0000 7fbcd27fc640 1 -- 172.21.15.46:0/1102314532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbccc010430 con 0x7fbcd4101900 2024-09-10T04:40:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.462+0000 7fbcd27fc640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcbc07ac40 0x7fbcbc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.462+0000 7fbcd27fc640 1 -- 172.21.15.46:0/1102314532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fbccc09e250 con 0x7fbcd4101900 2024-09-10T04:40:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.462+0000 7fbcd8c08640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcbc07ac40 0x7fbcbc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.463+0000 7fbcd8c08640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcbc07ac40 0x7fbcbc07d100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8006010 tx=0x7fbcc80095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:37.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.467+0000 7fbcd27fc640 1 -- 172.21.15.46:0/1102314532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbccc069a10 con 0x7fbcd4101900 2024-09-10T04:40:37.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.694+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fbcd4109ee0 con 0x7fbcd4101900 2024-09-10T04:40:37.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.695+0000 7fbcd27fc640 1 -- 172.21.15.46:0/1102314532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fbcd4109ee0 con 0x7fbcd4101900 2024-09-10T04:40:37.697 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:40:37.697 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:40:37.697 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:40:37.697 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:40:37.697 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:40:37.698 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:40:37.699 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:37.700 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:40:37.701 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:40:37.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcbc07ac40 msgr2=0x7fbcbc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcbc07ac40 0x7fbcbc07d100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8006010 tx=0x7fbcc80095f0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 msgr2=0x7fbcd40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd40744f0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e970 tx=0x7fbccc00ee30 comp rx=0 tx=0).stop 2024-09-10T04:40:37.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 shutdown_connections 2024-09-10T04:40:37.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcbc07ac40 0x7fbcbc07d100 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbcd4107900 0x7fbcd4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 --2- 172.21.15.46:0/1102314532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbcd4101900 0x7fbcd40744f0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.697+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 >> 172.21.15.46:0/1102314532 conn(0x7fbcd40fd620 msgr2=0x7fbcd40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:37.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.698+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 shutdown_connections 2024-09-10T04:40:37.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.698+0000 7fbcdb66d640 1 -- 172.21.15.46:0/1102314532 wait complete. 2024-09-10T04:40:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 -- 172.21.15.46:0/4225196266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 msgr2=0x7f7924102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 --2- 172.21.15.46:0/4225196266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f7924102ea0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7918009880 tx=0x7f791802a530 comp rx=0 tx=0).stop 2024-09-10T04:40:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 -- 172.21.15.46:0/4225196266 shutdown_connections 2024-09-10T04:40:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 --2- 172.21.15.46:0/4225196266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241033e0 0x7f792410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 --2- 172.21.15.46:0/4225196266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f7924102ea0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 -- 172.21.15.46:0/4225196266 >> 172.21.15.46:0/4225196266 conn(0x7f79240fc520 msgr2=0x7f79240fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 -- 172.21.15.46:0/4225196266 shutdown_connections 2024-09-10T04:40:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.785+0000 7f792adb4640 1 -- 172.21.15.46:0/4225196266 wait complete. 2024-09-10T04:40:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f792adb4640 1 Processor -- start 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f792adb4640 1 -- start start 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f792adb4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f792419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f792adb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241033e0 0x7f792419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f792adb4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f792410edd0 con 0x7f7924102ac0 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f792adb4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f792410ec50 con 0x7f79241033e0 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f7928b50640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f792419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f7928b50640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f792419d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56104/0 (socket says 172.21.15.46:56104) 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.786+0000 7f7928b50640 1 -- 172.21.15.46:0/4001489479 learned_addr learned my addr 172.21.15.46:0/4001489479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7923fff640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241033e0 0x7f792419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7928b50640 1 -- 172.21.15.46:0/4001489479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241033e0 msgr2=0x7f792419dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7928b50640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241033e0 0x7f792419dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7928b50640 1 -- 172.21.15.46:0/4001489479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79241978e0 con 0x7f7924102ac0 2024-09-10T04:40:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7928b50640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f792419d720 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7918031460 tx=0x7f79180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7921ffb640 1 -- 172.21.15.46:0/4001489479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f791803f070 con 0x7f7924102ac0 2024-09-10T04:40:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7921ffb640 1 -- 172.21.15.46:0/4001489479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f79180035f0 con 0x7f7924102ac0 2024-09-10T04:40:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7924197b70 con 0x7f7924102ac0 2024-09-10T04:40:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f7921ffb640 1 -- 172.21.15.46:0/4001489479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7918035030 con 0x7f7924102ac0 2024-09-10T04:40:37.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.787+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79241980a0 con 0x7f7924102ac0 2024-09-10T04:40:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.789+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792406aa90 con 0x7f7924102ac0 2024-09-10T04:40:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.789+0000 7f7921ffb640 1 -- 172.21.15.46:0/4001489479 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7918042070 con 0x7f7924102ac0 2024-09-10T04:40:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.790+0000 7f7921ffb640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f791407ac40 0x7f791407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.790+0000 7f7921ffb640 1 -- 172.21.15.46:0/4001489479 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f79180be4f0 con 0x7f7924102ac0 2024-09-10T04:40:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.790+0000 7f7923fff640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f791407ac40 0x7f791407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.791+0000 7f7923fff640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f791407ac40 0x7f791407d100 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f79240feec0 tx=0x7f791000b2e0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:37.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:37.794+0000 7f7921ffb640 1 -- 172.21.15.46:0/4001489479 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7918089cb0 con 0x7f7924102ac0 2024-09-10T04:40:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:37 smithi070 ceph-mon[79774]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:37 smithi070 ceph-mon[79774]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:37 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3601858015' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:40:37.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:37 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1102314532' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:40:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.002+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7924198e10 con 0x7f791407ac40 2024-09-10T04:40:38.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:37 smithi046 ceph-mon[109506]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:38.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:37 smithi046 ceph-mon[109506]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:38.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:37 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3601858015' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:40:38.005 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:37 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1102314532' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:40:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.003+0000 7f7921ffb640 1 -- 172.21.15.46:0/4001489479 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7924198e10 con 0x7f791407ac40 2024-09-10T04:40:38.005 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:40:38.006 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:40:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.006+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f791407ac40 msgr2=0x7f791407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.006+0000 7f792adb4640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f791407ac40 0x7f791407d100 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f79240feec0 tx=0x7f791000b2e0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.006+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 msgr2=0x7f792419d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.006+0000 7f792adb4640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f792419d720 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7918031460 tx=0x7f79180045e0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.007+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 shutdown_connections 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.007+0000 7f792adb4640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f791407ac40 0x7f791407d100 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.007+0000 7f792adb4640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79241033e0 0x7f792419dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.007+0000 7f792adb4640 1 --2- 172.21.15.46:0/4001489479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7924102ac0 0x7f792419d720 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.007+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 >> 172.21.15.46:0/4001489479 conn(0x7f79240fc520 msgr2=0x7f7924108ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.007+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 shutdown_connections 2024-09-10T04:40:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.007+0000 7f792adb4640 1 -- 172.21.15.46:0/4001489479 wait complete. 2024-09-10T04:40:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 -- 172.21.15.46:0/3838201671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0100800 msgr2=0x7f02e0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 --2- 172.21.15.46:0/3838201671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0100800 0x7f02e0100c60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f02d4009880 tx=0x7f02d402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 -- 172.21.15.46:0/3838201671 shutdown_connections 2024-09-10T04:40:38.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 --2- 172.21.15.46:0/3838201671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0100800 0x7f02e0100c60 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 --2- 172.21.15.46:0/3838201671 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e0106800 0x7f02e0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 -- 172.21.15.46:0/3838201671 >> 172.21.15.46:0/3838201671 conn(0x7f02e00fc520 msgr2=0x7f02e00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:38.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 -- 172.21.15.46:0/3838201671 shutdown_connections 2024-09-10T04:40:38.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.101+0000 7f02e6181640 1 -- 172.21.15.46:0/3838201671 wait complete. 2024-09-10T04:40:38.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.102+0000 7f02e6181640 1 Processor -- start 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.102+0000 7f02e6181640 1 -- start start 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.102+0000 7f02e6181640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e0100800 0x7f02e0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.102+0000 7f02e6181640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0106800 0x7f02e0198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.102+0000 7f02e6181640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02e010dcd0 con 0x7f02e0106800 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.102+0000 7f02e6181640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f02e010db50 con 0x7f02e0100800 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02deffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0106800 0x7f02e0198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02deffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0106800 0x7f02e0198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56122/0 (socket says 172.21.15.46:56122) 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02deffd640 1 -- 172.21.15.46:0/2573127913 learned_addr learned my addr 172.21.15.46:0/2573127913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02df7fe640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e0100800 0x7f02e0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02deffd640 1 -- 172.21.15.46:0/2573127913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e0100800 msgr2=0x7f02e0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02deffd640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e0100800 0x7f02e0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02deffd640 1 -- 172.21.15.46:0/2573127913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e0199360 con 0x7f02e0106800 2024-09-10T04:40:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02df7fe640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e0100800 0x7f02e0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:40:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02deffd640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0106800 0x7f02e0198d60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f02d4031460 tx=0x7f02d40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02dcff9640 1 -- 172.21.15.46:0/2573127913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02d403f070 con 0x7f02e0106800 2024-09-10T04:40:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e01995f0 con 0x7f02e0106800 2024-09-10T04:40:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02dcff9640 1 -- 172.21.15.46:0/2573127913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02d40035f0 con 0x7f02e0106800 2024-09-10T04:40:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.103+0000 7f02dcff9640 1 -- 172.21.15.46:0/2573127913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02d4035030 con 0x7f02e0106800 2024-09-10T04:40:38.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.104+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e01a1ba0 con 0x7f02e0106800 2024-09-10T04:40:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.105+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e0101f40 con 0x7f02e0106800 2024-09-10T04:40:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.105+0000 7f02dcff9640 1 -- 172.21.15.46:0/2573127913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f02d4042070 con 0x7f02e0106800 2024-09-10T04:40:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.106+0000 7f02dcff9640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807abf0 0x7f02c807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:40:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.106+0000 7f02dcff9640 1 -- 172.21.15.46:0/2573127913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f02d40be680 con 0x7f02e0106800 2024-09-10T04:40:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.106+0000 7f02df7fe640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807abf0 0x7f02c807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:40:38.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.106+0000 7f02df7fe640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807abf0 0x7f02c807d0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f02d00058c0 tx=0x7f02d0005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:40:38.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.110+0000 7f02dcff9640 1 -- 172.21.15.46:0/2573127913 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f02d4089dc0 con 0x7f02e0106800 2024-09-10T04:40:38.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.372+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f02e0108d70 con 0x7f02e0106800 2024-09-10T04:40:38.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.372+0000 7f02dcff9640 1 -- 172.21.15.46:0/2573127913 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f02d408ec70 con 0x7f02e0106800 2024-09-10T04:40:38.374 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:40:38.374 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:40:38.374 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:40:38.374 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:40:38.374 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:40:38.375 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:40:38.376 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:40:38.376 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:40:38.376 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:40:38.376 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:40:38.376 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:40:38.376 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807abf0 msgr2=0x7f02c807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807abf0 0x7f02c807d0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f02d00058c0 tx=0x7f02d0005850 comp rx=0 tx=0).stop 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0106800 msgr2=0x7f02e0198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0106800 0x7f02e0198d60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f02d4031460 tx=0x7f02d40045e0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 shutdown_connections 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02c807abf0 0x7f02c807d0b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02e0106800 0x7f02e0198d60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 --2- 172.21.15.46:0/2573127913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02e0100800 0x7f02e0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 >> 172.21.15.46:0/2573127913 conn(0x7f02e00fc520 msgr2=0x7f02e00fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 shutdown_connections 2024-09-10T04:40:38.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:40:38.374+0000 7f02e6181640 1 -- 172.21.15.46:0/2573127913 wait complete. 2024-09-10T04:40:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:38 smithi070 ceph-mon[79774]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:38 smithi070 ceph-mon[79774]: pgmap v1050: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:38 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2573127913' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:40:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:38 smithi046 ceph-mon[109506]: from='client.35746 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:38 smithi046 ceph-mon[109506]: pgmap v1050: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:38 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2573127913' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:40:40.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:39 smithi070 ceph-mon[79774]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:40.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:39 smithi046 ceph-mon[109506]: from='client.35758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:40:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:40 smithi070 ceph-mon[79774]: pgmap v1051: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:40 smithi046 ceph-mon[109506]: pgmap v1051: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:42 smithi070 ceph-mon[79774]: pgmap v1052: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:42 smithi046 ceph-mon[109506]: pgmap v1052: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:44 smithi070 ceph-mon[79774]: pgmap v1053: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:44 smithi046 ceph-mon[109506]: pgmap v1053: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:46 smithi070 ceph-mon[79774]: pgmap v1054: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:47.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:46 smithi046 ceph-mon[109506]: pgmap v1054: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:48 smithi070 ceph-mon[79774]: pgmap v1055: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:48 smithi046 ceph-mon[109506]: pgmap v1055: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:50 smithi070 ceph-mon[79774]: pgmap v1056: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:50 smithi046 ceph-mon[109506]: pgmap v1056: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:52 smithi070 ceph-mon[79774]: pgmap v1057: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:52 smithi046 ceph-mon[109506]: pgmap v1057: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:40:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:54 smithi070 ceph-mon[79774]: pgmap v1058: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:54 smithi046 ceph-mon[109506]: pgmap v1058: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:56 smithi070 ceph-mon[79774]: pgmap v1059: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:57.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:56 smithi046 ceph-mon[109506]: pgmap v1059: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:40:58 smithi070 ceph-mon[79774]: pgmap v1060: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:40:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:40:58 smithi046 ceph-mon[109506]: pgmap v1060: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:00 smithi070 ceph-mon[79774]: pgmap v1061: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:00 smithi046 ceph-mon[109506]: pgmap v1061: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:02 smithi070 ceph-mon[79774]: pgmap v1062: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:03.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:02 smithi046 ceph-mon[109506]: pgmap v1062: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:04 smithi070 ceph-mon[79774]: pgmap v1063: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:04 smithi046 ceph-mon[109506]: pgmap v1063: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:06 smithi070 ceph-mon[79774]: pgmap v1064: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:06 smithi046 ceph-mon[109506]: pgmap v1064: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:08.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 -- 172.21.15.46:0/3509219524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c102ac0 msgr2=0x7fa47c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 --2- 172.21.15.46:0/3509219524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c102ac0 0x7fa47c102ea0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009880 tx=0x7fa46c02a530 comp rx=0 tx=0).stop 2024-09-10T04:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 -- 172.21.15.46:0/3509219524 shutdown_connections 2024-09-10T04:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 --2- 172.21.15.46:0/3509219524 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa47c1033e0 0x7fa47c10b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 --2- 172.21.15.46:0/3509219524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c102ac0 0x7fa47c102ea0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 -- 172.21.15.46:0/3509219524 >> 172.21.15.46:0/3509219524 conn(0x7fa47c0fc520 msgr2=0x7fa47c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 -- 172.21.15.46:0/3509219524 shutdown_connections 2024-09-10T04:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.473+0000 7fa4815b1640 1 -- 172.21.15.46:0/3509219524 wait complete. 2024-09-10T04:41:08.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa4815b1640 1 Processor -- start 2024-09-10T04:41:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa4815b1640 1 -- start start 2024-09-10T04:41:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa4815b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa47c102ac0 0x7fa47c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa4815b1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c1033e0 0x7fa47c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:08.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa4815b1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c10edf0 con 0x7fa47c1033e0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa4815b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa47c10ec70 con 0x7fa47c102ac0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa47a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c1033e0 0x7fa47c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa47a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c1033e0 0x7fa47c19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58058/0 (socket says 172.21.15.46:58058) 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa47a7fc640 1 -- 172.21.15.46:0/1757296982 learned_addr learned my addr 172.21.15.46:0/1757296982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.474+0000 7fa47affd640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa47c102ac0 0x7fa47c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa47a7fc640 1 -- 172.21.15.46:0/1757296982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa47c102ac0 msgr2=0x7fa47c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa47a7fc640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa47c102ac0 0x7fa47c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa47a7fc640 1 -- 172.21.15.46:0/1757296982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47c1977c0 con 0x7fa47c1033e0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa47a7fc640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c1033e0 0x7fa47c19db80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e970 tx=0x7fa47000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa463fff640 1 -- 172.21.15.46:0/1757296982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa47000cac0 con 0x7fa47c1033e0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa463fff640 1 -- 172.21.15.46:0/1757296982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa4700054e0 con 0x7fa47c1033e0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa463fff640 1 -- 172.21.15.46:0/1757296982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa470004dd0 con 0x7fa47c1033e0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa47c197ab0 con 0x7fa47c1033e0 2024-09-10T04:41:08.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.475+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47c198040 con 0x7fa47c1033e0 2024-09-10T04:41:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.476+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47c06aa90 con 0x7fa47c1033e0 2024-09-10T04:41:08.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.477+0000 7fa463fff640 1 -- 172.21.15.46:0/1757296982 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa470005020 con 0x7fa47c1033e0 2024-09-10T04:41:08.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.478+0000 7fa463fff640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa46407abf0 0x7fa46407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:08.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.478+0000 7fa47affd640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa46407abf0 0x7fa46407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:08.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.478+0000 7fa463fff640 1 -- 172.21.15.46:0/1757296982 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa47009e1e0 con 0x7fa47c1033e0 2024-09-10T04:41:08.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.479+0000 7fa47affd640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa46407abf0 0x7fa46407d0b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0096c0 tx=0x7fa46c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:08.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.483+0000 7fa463fff640 1 -- 172.21.15.46:0/1757296982 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa4700699a0 con 0x7fa47c1033e0 2024-09-10T04:41:08.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.687+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa47c198350 con 0x7fa46407abf0 2024-09-10T04:41:08.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.688+0000 7fa463fff640 1 -- 172.21.15.46:0/1757296982 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fa47c198350 con 0x7fa46407abf0 2024-09-10T04:41:08.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa46407abf0 msgr2=0x7fa46407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa46407abf0 0x7fa46407d0b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0096c0 tx=0x7fa46c031400 comp rx=0 tx=0).stop 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c1033e0 msgr2=0x7fa47c19db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c1033e0 0x7fa47c19db80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fa47000e970 tx=0x7fa47000ee30 comp rx=0 tx=0).stop 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 shutdown_connections 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa46407abf0 0x7fa46407d0b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa47c1033e0 0x7fa47c19db80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 --2- 172.21.15.46:0/1757296982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa47c102ac0 0x7fa47c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.690+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 >> 172.21.15.46:0/1757296982 conn(0x7fa47c0fc520 msgr2=0x7fa47c10af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.691+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 shutdown_connections 2024-09-10T04:41:08.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.691+0000 7fa4815b1640 1 -- 172.21.15.46:0/1757296982 wait complete. 2024-09-10T04:41:08.701 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:41:08.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 -- 172.21.15.46:0/3418561660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 msgr2=0x7f4978100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 --2- 172.21.15.46:0/3418561660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f4978100520 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4968009880 tx=0x7f496802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 -- 172.21.15.46:0/3418561660 shutdown_connections 2024-09-10T04:41:08.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 --2- 172.21.15.46:0/3418561660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4978100af0 0x7f4978106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 --2- 172.21.15.46:0/3418561660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f4978100520 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 -- 172.21.15.46:0/3418561660 >> 172.21.15.46:0/3418561660 conn(0x7f49780fbc90 msgr2=0x7f49780fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:08.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 -- 172.21.15.46:0/3418561660 shutdown_connections 2024-09-10T04:41:08.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.776+0000 7f497e57c640 1 -- 172.21.15.46:0/3418561660 wait complete. 2024-09-10T04:41:08.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f497e57c640 1 Processor -- start 2024-09-10T04:41:08.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f497e57c640 1 -- start start 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f497e57c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f497819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f497e57c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4978100af0 0x7f497819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f497e57c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f497810eec0 con 0x7f4978100140 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f497e57c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f497810ed40 con 0x7f4978100af0 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f4977fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f497819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f4977fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f497819f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58080/0 (socket says 172.21.15.46:58080) 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.777+0000 7f4977fff640 1 -- 172.21.15.46:0/603596813 learned_addr learned my addr 172.21.15.46:0/603596813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f49777fe640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4978100af0 0x7f497819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f4977fff640 1 -- 172.21.15.46:0/603596813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4978100af0 msgr2=0x7f497819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f4977fff640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4978100af0 0x7f497819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f4977fff640 1 -- 172.21.15.46:0/603596813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4978199a60 con 0x7f4978100140 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f4977fff640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f497819f8a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4968031460 tx=0x7f49680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:08.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f49757fa640 1 -- 172.21.15.46:0/603596813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f496803f070 con 0x7f4978100140 2024-09-10T04:41:08.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f49757fa640 1 -- 172.21.15.46:0/603596813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4968004cf0 con 0x7f4978100140 2024-09-10T04:41:08.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4978199cf0 con 0x7f4978100140 2024-09-10T04:41:08.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f49757fa640 1 -- 172.21.15.46:0/603596813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4968035030 con 0x7f4978100140 2024-09-10T04:41:08.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.778+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497819a220 con 0x7f4978100140 2024-09-10T04:41:08.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.779+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4978103200 con 0x7f4978100140 2024-09-10T04:41:08.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.780+0000 7f49757fa640 1 -- 172.21.15.46:0/603596813 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4968042070 con 0x7f4978100140 2024-09-10T04:41:08.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.781+0000 7f49757fa640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f496007abf0 0x7f496007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:08.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.781+0000 7f49777fe640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f496007abf0 0x7f496007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:08.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.781+0000 7f49757fa640 1 -- 172.21.15.46:0/603596813 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f49680be750 con 0x7f4978100140 2024-09-10T04:41:08.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.781+0000 7f49777fe640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f496007abf0 0x7f496007d0b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4978106670 tx=0x7f496c009110 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:08.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.785+0000 7f49757fa640 1 -- 172.21.15.46:0/603596813 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4968089e90 con 0x7f4978100140 2024-09-10T04:41:08.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.988+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f497819af90 con 0x7f496007abf0 2024-09-10T04:41:08.990 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:08 smithi046 ceph-mon[109506]: pgmap v1065: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:08.990 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:08.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.989+0000 7f49757fa640 1 -- 172.21.15.46:0/603596813 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f497819af90 con 0x7f496007abf0 2024-09-10T04:41:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f496007abf0 msgr2=0x7f496007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f496007abf0 0x7f496007d0b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4978106670 tx=0x7f496c009110 comp rx=0 tx=0).stop 2024-09-10T04:41:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 msgr2=0x7f497819f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:08.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f497819f8a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4968031460 tx=0x7f49680048f0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 shutdown_connections 2024-09-10T04:41:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f496007abf0 0x7f496007d0b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4978100af0 0x7f497819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 --2- 172.21.15.46:0/603596813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4978100140 0x7f497819f8a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 >> 172.21.15.46:0/603596813 conn(0x7f49780fbc90 msgr2=0x7f49780fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 shutdown_connections 2024-09-10T04:41:08.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:08.992+0000 7f497e57c640 1 -- 172.21.15.46:0/603596813 wait complete. 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.076+0000 7ff3f64f7640 1 -- 172.21.15.46:0/3667841861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102ac0 msgr2=0x7ff3f0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.076+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/3667841861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102ac0 0x7ff3f0102ea0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009880 tx=0x7ff3e002a530 comp rx=0 tx=0).stop 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.076+0000 7ff3f64f7640 1 -- 172.21.15.46:0/3667841861 shutdown_connections 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.076+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/3667841861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f01033e0 0x7ff3f010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.076+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/3667841861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f0102ac0 0x7ff3f0102ea0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.076+0000 7ff3f64f7640 1 -- 172.21.15.46:0/3667841861 >> 172.21.15.46:0/3667841861 conn(0x7ff3f00fc520 msgr2=0x7ff3f00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.077+0000 7ff3f64f7640 1 -- 172.21.15.46:0/3667841861 shutdown_connections 2024-09-10T04:41:09.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.077+0000 7ff3f64f7640 1 -- 172.21.15.46:0/3667841861 wait complete. 2024-09-10T04:41:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.077+0000 7ff3f64f7640 1 Processor -- start 2024-09-10T04:41:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.077+0000 7ff3f64f7640 1 -- start start 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3f64f7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f0102ac0 0x7ff3f019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3f64f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f01033e0 0x7ff3f019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3f64f7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f010edf0 con 0x7ff3f01033e0 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3f64f7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f010ec70 con 0x7ff3f0102ac0 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f01033e0 0x7ff3f019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f01033e0 0x7ff3f019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44276/0 (socket says 172.21.15.46:44276) 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ef7fe640 1 -- 172.21.15.46:0/1781717218 learned_addr learned my addr 172.21.15.46:0/1781717218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3effff640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f0102ac0 0x7ff3f019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ef7fe640 1 -- 172.21.15.46:0/1781717218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f0102ac0 msgr2=0x7ff3f019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ef7fe640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f0102ac0 0x7ff3f019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ef7fe640 1 -- 172.21.15.46:0/1781717218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f01977e0 con 0x7ff3f01033e0 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ef7fe640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f01033e0 0x7ff3f019dba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c8f0 tx=0x7ff3e400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.078+0000 7ff3ed7fa640 1 -- 172.21.15.46:0/1781717218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff3e400ae50 con 0x7ff3f01033e0 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.079+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f0197ad0 con 0x7ff3f01033e0 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.079+0000 7ff3ed7fa640 1 -- 172.21.15.46:0/1781717218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff3e4005020 con 0x7ff3f01033e0 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.079+0000 7ff3ed7fa640 1 -- 172.21.15.46:0/1781717218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff3e4004d70 con 0x7ff3f01033e0 2024-09-10T04:41:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.079+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f0198060 con 0x7ff3f01033e0 2024-09-10T04:41:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.080+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f006aa90 con 0x7ff3f01033e0 2024-09-10T04:41:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.080+0000 7ff3ed7fa640 1 -- 172.21.15.46:0/1781717218 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff3e40079b0 con 0x7ff3f01033e0 2024-09-10T04:41:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.081+0000 7ff3ed7fa640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff3d807ada0 0x7ff3d807d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.081+0000 7ff3ed7fa640 1 -- 172.21.15.46:0/1781717218 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff3e409da30 con 0x7ff3f01033e0 2024-09-10T04:41:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.081+0000 7ff3effff640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff3d807ada0 0x7ff3d807d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.081+0000 7ff3effff640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff3d807ada0 0x7ff3d807d260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000c00 tx=0x7ff3e0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:08 smithi070 ceph-mon[79774]: pgmap v1065: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.085+0000 7ff3ed7fa640 1 -- 172.21.15.46:0/1781717218 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff3e4014030 con 0x7ff3f01033e0 2024-09-10T04:41:09.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.279+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff3f0198370 con 0x7ff3d807ada0 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.289+0000 7ff3ed7fa640 1 -- 172.21.15.46:0/1781717218 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7ff3f0198370 con 0x7ff3d807ada0 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (32m) 11m ago 44m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (35m) 11m ago 44m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (35m) 11m ago 43m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (31m) 11m ago 44m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (31m) 11m ago 43m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (32m) 11m ago 43m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:41:09.291 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (27m) 11m ago 38m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (27m) 11m ago 38m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (27m) 11m ago 38m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (11m) 11m ago 38m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (34m) 11m ago 45m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (33m) 11m ago 43m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (31m) 11m ago 45m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (31m) 11m ago 43m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (33m) 11m ago 44m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (33m) 11m ago 43m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (30m) 11m ago 42m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (30m) 11m ago 41m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (29m) 11m ago 41m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (28m) 11m ago 40m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:41:09.292 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (28m) 11m ago 40m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:41:09.293 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (27m) 11m ago 39m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:41:09.293 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (33m) 11m ago 43m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:41:09.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff3d807ada0 msgr2=0x7ff3d807d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff3d807ada0 0x7ff3d807d260 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0000c00 tx=0x7ff3e0031400 comp rx=0 tx=0).stop 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f01033e0 msgr2=0x7ff3f019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f01033e0 0x7ff3f019dba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c8f0 tx=0x7ff3e400cdb0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 shutdown_connections 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff3d807ada0 0x7ff3d807d260 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f01033e0 0x7ff3f019dba0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 --2- 172.21.15.46:0/1781717218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f0102ac0 0x7ff3f019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.291+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 >> 172.21.15.46:0/1781717218 conn(0x7ff3f00fc520 msgr2=0x7ff3f0108ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.292+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 shutdown_connections 2024-09-10T04:41:09.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.292+0000 7ff3f64f7640 1 -- 172.21.15.46:0/1781717218 wait complete. 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 -- 172.21.15.46:0/280823490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2574102ab0 msgr2=0x7f2574102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 --2- 172.21.15.46:0/280823490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2574102ab0 0x7f2574102e90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2570009880 tx=0x7f257002a530 comp rx=0 tx=0).stop 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 -- 172.21.15.46:0/280823490 shutdown_connections 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 --2- 172.21.15.46:0/280823490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25741033d0 0x7f257410b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 --2- 172.21.15.46:0/280823490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2574102ab0 0x7f2574102e90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 -- 172.21.15.46:0/280823490 >> 172.21.15.46:0/280823490 conn(0x7f25740fc550 msgr2=0x7f25740fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 -- 172.21.15.46:0/280823490 shutdown_connections 2024-09-10T04:41:09.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 -- 172.21.15.46:0/280823490 wait complete. 2024-09-10T04:41:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.375+0000 7f257b64a640 1 Processor -- start 2024-09-10T04:41:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f257b64a640 1 -- start start 2024-09-10T04:41:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f257b64a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2574102ab0 0x7f257419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f257b64a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741033d0 0x7f257419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f257b64a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2574110fe0 con 0x7f25741033d0 2024-09-10T04:41:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f257b64a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2574110e60 con 0x7f2574102ab0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f2578be5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741033d0 0x7f257419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f2578be5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741033d0 0x7f257419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44288/0 (socket says 172.21.15.46:44288) 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f2578be5640 1 -- 172.21.15.46:0/231395145 learned_addr learned my addr 172.21.15.46:0/231395145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f25793e6640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2574102ab0 0x7f257419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f2578be5640 1 -- 172.21.15.46:0/231395145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2574102ab0 msgr2=0x7f257419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f2578be5640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2574102ab0 0x7f257419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.376+0000 7f2578be5640 1 -- 172.21.15.46:0/231395145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25741999d0 con 0x7f25741033d0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.377+0000 7f2578be5640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741033d0 0x7f257419fd50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f256400c8f0 tx=0x7f256400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.377+0000 7f256a7fc640 1 -- 172.21.15.46:0/231395145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f256400ae50 con 0x7f25741033d0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.377+0000 7f256a7fc640 1 -- 172.21.15.46:0/231395145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2564005020 con 0x7f25741033d0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.377+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2574199cc0 con 0x7f25741033d0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.377+0000 7f256a7fc640 1 -- 172.21.15.46:0/231395145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2564004d70 con 0x7f25741033d0 2024-09-10T04:41:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.377+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257419a250 con 0x7f25741033d0 2024-09-10T04:41:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.378+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257406a580 con 0x7f25741033d0 2024-09-10T04:41:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.379+0000 7f256a7fc640 1 -- 172.21.15.46:0/231395145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f25640051d0 con 0x7f25741033d0 2024-09-10T04:41:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.379+0000 7f256a7fc640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f256007ada0 0x7f256007d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.379+0000 7f256a7fc640 1 -- 172.21.15.46:0/231395145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f256409da70 con 0x7f25741033d0 2024-09-10T04:41:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.379+0000 7f25793e6640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f256007ada0 0x7f256007d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.380+0000 7f25793e6640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f256007ada0 0x7f256007d260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2570000c00 tx=0x7f2570031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:09.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.384+0000 7f256a7fc640 1 -- 172.21.15.46:0/231395145 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2564014030 con 0x7f25741033d0 2024-09-10T04:41:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.644+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f2574105fe0 con 0x7f25741033d0 2024-09-10T04:41:09.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.644+0000 7f256a7fc640 1 -- 172.21.15.46:0/231395145 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f25640691b0 con 0x7f25741033d0 2024-09-10T04:41:09.646 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:41:09.646 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:41:09.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:41:09.648 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f256007ada0 msgr2=0x7f256007d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f256007ada0 0x7f256007d260 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2570000c00 tx=0x7f2570031400 comp rx=0 tx=0).stop 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741033d0 msgr2=0x7f257419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741033d0 0x7f257419fd50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f256400c8f0 tx=0x7f256400cdb0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 shutdown_connections 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f256007ada0 0x7f256007d260 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25741033d0 0x7f257419fd50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 --2- 172.21.15.46:0/231395145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2574102ab0 0x7f257419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 >> 172.21.15.46:0/231395145 conn(0x7f25740fc550 msgr2=0x7f2574108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:09.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 shutdown_connections 2024-09-10T04:41:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.647+0000 7f257b64a640 1 -- 172.21.15.46:0/231395145 wait complete. 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 -- 172.21.15.46:0/4055143802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106800 msgr2=0x7feb44106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 --2- 172.21.15.46:0/4055143802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106800 0x7feb44106be0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7feb40009880 tx=0x7feb4002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 -- 172.21.15.46:0/4055143802 shutdown_connections 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 --2- 172.21.15.46:0/4055143802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb44100800 0x7feb44100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 --2- 172.21.15.46:0/4055143802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44106800 0x7feb44106be0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 -- 172.21.15.46:0/4055143802 >> 172.21.15.46:0/4055143802 conn(0x7feb440fc520 msgr2=0x7feb440fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 -- 172.21.15.46:0/4055143802 shutdown_connections 2024-09-10T04:41:09.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 -- 172.21.15.46:0/4055143802 wait complete. 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.731+0000 7feb4c56e640 1 Processor -- start 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4c56e640 1 -- start start 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4c56e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44100800 0x7feb4419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4c56e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb44106800 0x7feb4419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4c56e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb4410dcd0 con 0x7feb44100800 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4c56e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7feb4410db50 con 0x7feb44106800 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4a30a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44100800 0x7feb4419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4a30a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44100800 0x7feb4419f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44300/0 (socket says 172.21.15.46:44300) 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4a30a640 1 -- 172.21.15.46:0/1697770649 learned_addr learned my addr 172.21.15.46:0/1697770649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb49b09640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb44106800 0x7feb4419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4a30a640 1 -- 172.21.15.46:0/1697770649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb44106800 msgr2=0x7feb4419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4a30a640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb44106800 0x7feb4419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4a30a640 1 -- 172.21.15.46:0/1697770649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb441999e0 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.732+0000 7feb4a30a640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44100800 0x7feb4419f820 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7feb40031460 tx=0x7feb400045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.733+0000 7feb3b7fe640 1 -- 172.21.15.46:0/1697770649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb4003f070 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.733+0000 7feb3b7fe640 1 -- 172.21.15.46:0/1697770649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7feb400035f0 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.733+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb44199c70 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.733+0000 7feb3b7fe640 1 -- 172.21.15.46:0/1697770649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7feb40035030 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.733+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4419a1a0 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.734+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb44101f40 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.734+0000 7feb3b7fe640 1 -- 172.21.15.46:0/1697770649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7feb40042070 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.735+0000 7feb3b7fe640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb3007abf0 0x7feb3007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.735+0000 7feb3b7fe640 1 -- 172.21.15.46:0/1697770649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7feb400be5d0 con 0x7feb44100800 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.735+0000 7feb49b09640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb3007abf0 0x7feb3007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:09.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.735+0000 7feb49b09640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb3007abf0 0x7feb3007d0b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feb440fe3b0 tx=0x7feb3400b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:09.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.739+0000 7feb3b7fe640 1 -- 172.21.15.46:0/1697770649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7feb40089d10 con 0x7feb44100800 2024-09-10T04:41:09.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.964+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7feb4419afa0 con 0x7feb44100800 2024-09-10T04:41:09.966 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:09 smithi046 ceph-mon[109506]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:09.966 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:09 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/231395145' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:41:09.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.964+0000 7feb3b7fe640 1 -- 172.21.15.46:0/1697770649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7feb4008ebc0 con 0x7feb44100800 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:41:09.967 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:41:09.968 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:09.969 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:09.970 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:09.970 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb3007abf0 msgr2=0x7feb3007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb3007abf0 0x7feb3007d0b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feb440fe3b0 tx=0x7feb3400b390 comp rx=0 tx=0).stop 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44100800 msgr2=0x7feb4419f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44100800 0x7feb4419f820 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7feb40031460 tx=0x7feb400045e0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 shutdown_connections 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7feb3007abf0 0x7feb3007d0b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb44106800 0x7feb4419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 --2- 172.21.15.46:0/1697770649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb44100800 0x7feb4419f820 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:09.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 >> 172.21.15.46:0/1697770649 conn(0x7feb440fc520 msgr2=0x7feb440fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:09.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.967+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 shutdown_connections 2024-09-10T04:41:09.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:09.968+0000 7feb4c56e640 1 -- 172.21.15.46:0/1697770649 wait complete. 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 -- 172.21.15.46:0/2749484400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc1033e0 msgr2=0x7f00cc10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/2749484400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc1033e0 0x7f00cc10b8d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009880 tx=0x7f00bc02a530 comp rx=0 tx=0).stop 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 -- 172.21.15.46:0/2749484400 shutdown_connections 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/2749484400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc1033e0 0x7f00cc10b8d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/2749484400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00cc102ac0 0x7f00cc102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 -- 172.21.15.46:0/2749484400 >> 172.21.15.46:0/2749484400 conn(0x7f00cc0fc520 msgr2=0x7f00cc0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 -- 172.21.15.46:0/2749484400 shutdown_connections 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.055+0000 7f00d3aa3640 1 -- 172.21.15.46:0/2749484400 wait complete. 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d3aa3640 1 Processor -- start 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d3aa3640 1 -- start start 2024-09-10T04:41:10.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d3aa3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc102ac0 0x7f00cc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d3aa3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00cc1033e0 0x7f00cc19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d3aa3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc10edf0 con 0x7f00cc102ac0 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d3aa3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f00cc10ec70 con 0x7f00cc1033e0 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d183f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc102ac0 0x7f00cc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d183f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc102ac0 0x7f00cc19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44330/0 (socket says 172.21.15.46:44330) 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d183f640 1 -- 172.21.15.46:0/855956241 learned_addr learned my addr 172.21.15.46:0/855956241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d103e640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00cc1033e0 0x7f00cc19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d183f640 1 -- 172.21.15.46:0/855956241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00cc1033e0 msgr2=0x7f00cc19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d183f640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00cc1033e0 0x7f00cc19dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.056+0000 7f00d183f640 1 -- 172.21.15.46:0/855956241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc197820 con 0x7f00cc102ac0 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.057+0000 7f00d183f640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc102ac0 0x7f00cc19d6a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d6f0 tx=0x7f00c800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:10.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.057+0000 7f00c2ffd640 1 -- 172.21.15.46:0/855956241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00c8017070 con 0x7f00cc102ac0 2024-09-10T04:41:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.057+0000 7f00c2ffd640 1 -- 172.21.15.46:0/855956241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f00c800de00 con 0x7f00cc102ac0 2024-09-10T04:41:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.057+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00cc197b10 con 0x7f00cc102ac0 2024-09-10T04:41:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.057+0000 7f00c2ffd640 1 -- 172.21.15.46:0/855956241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f00c8004bb0 con 0x7f00cc102ac0 2024-09-10T04:41:10.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.057+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00cc198070 con 0x7f00cc102ac0 2024-09-10T04:41:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.058+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00cc06aa90 con 0x7f00cc102ac0 2024-09-10T04:41:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.059+0000 7f00c2ffd640 1 -- 172.21.15.46:0/855956241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f00c8004e60 con 0x7f00cc102ac0 2024-09-10T04:41:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.059+0000 7f00c2ffd640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00b807abf0 0x7f00b807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.059+0000 7f00c2ffd640 1 -- 172.21.15.46:0/855956241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f00c809d8c0 con 0x7f00cc102ac0 2024-09-10T04:41:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.059+0000 7f00d103e640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00b807abf0 0x7f00b807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.060+0000 7f00d103e640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00b807abf0 0x7f00b807d0b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f00cc198b70 tx=0x7f00bc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.063+0000 7f00c2ffd640 1 -- 172.21.15.46:0/855956241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f00c8068f60 con 0x7f00cc102ac0 2024-09-10T04:41:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:09 smithi070 ceph-mon[79774]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:10.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:09 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/231395145' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:41:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.269+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f00cc198320 con 0x7f00b807abf0 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.270+0000 7f00c2ffd640 1 -- 172.21.15.46:0/855956241 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f00cc198320 con 0x7f00b807abf0 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:41:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:41:10.273 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:41:10.273 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:41:10.273 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:41:10.273 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:41:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00b807abf0 msgr2=0x7f00b807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00b807abf0 0x7f00b807d0b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f00cc198b70 tx=0x7f00bc031400 comp rx=0 tx=0).stop 2024-09-10T04:41:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc102ac0 msgr2=0x7f00cc19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc102ac0 0x7f00cc19d6a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d6f0 tx=0x7f00c800dbb0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 shutdown_connections 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f00b807abf0 0x7f00b807d0b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00cc1033e0 0x7f00cc19dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 --2- 172.21.15.46:0/855956241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc102ac0 0x7f00cc19d6a0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.272+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 >> 172.21.15.46:0/855956241 conn(0x7f00cc0fc520 msgr2=0x7f00cc108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.273+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 shutdown_connections 2024-09-10T04:41:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.273+0000 7f00d3aa3640 1 -- 172.21.15.46:0/855956241 wait complete. 2024-09-10T04:41:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 -- 172.21.15.46:0/2317645376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 msgr2=0x7fd034106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 --2- 172.21.15.46:0/2317645376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd034106d80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd028009880 tx=0x7fd02802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 -- 172.21.15.46:0/2317645376 shutdown_connections 2024-09-10T04:41:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 --2- 172.21.15.46:0/2317645376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0340ffdc0 0x7fd034100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 --2- 172.21.15.46:0/2317645376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd034106d80 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 -- 172.21.15.46:0/2317645376 >> 172.21.15.46:0/2317645376 conn(0x7fd0340fbc90 msgr2=0x7fd0340fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 -- 172.21.15.46:0/2317645376 shutdown_connections 2024-09-10T04:41:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.357+0000 7fd03acd8640 1 -- 172.21.15.46:0/2317645376 wait complete. 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd03acd8640 1 Processor -- start 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd03acd8640 1 -- start start 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd03acd8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0340ffdc0 0x7fd03419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd03acd8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd03419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd03acd8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410ef40 con 0x7fd0341069a0 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd03acd8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410edc0 con 0x7fd0340ffdc0 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd033fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd03419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:10.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd033fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd03419dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44358/0 (socket says 172.21.15.46:44358) 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd033fff640 1 -- 172.21.15.46:0/3719933660 learned_addr learned my addr 172.21.15.46:0/3719933660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.358+0000 7fd038a74640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0340ffdc0 0x7fd03419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd033fff640 1 -- 172.21.15.46:0/3719933660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0340ffdc0 msgr2=0x7fd03419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd033fff640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0340ffdc0 0x7fd03419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd033fff640 1 -- 172.21.15.46:0/3719933660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0341978d0 con 0x7fd0341069a0 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd038a74640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0340ffdc0 0x7fd03419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd033fff640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd03419dc50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd02000c8f0 tx=0x7fd02000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd031ffb640 1 -- 172.21.15.46:0/3719933660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd0200079b0 con 0x7fd0341069a0 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd031ffb640 1 -- 172.21.15.46:0/3719933660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd0200054e0 con 0x7fd0341069a0 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd034197bc0 con 0x7fd0341069a0 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd031ffb640 1 -- 172.21.15.46:0/3719933660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fd020004dd0 con 0x7fd0341069a0 2024-09-10T04:41:10.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.359+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd034198150 con 0x7fd0341069a0 2024-09-10T04:41:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.360+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0341030e0 con 0x7fd0341069a0 2024-09-10T04:41:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.361+0000 7fd031ffb640 1 -- 172.21.15.46:0/3719933660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fd020005020 con 0x7fd0341069a0 2024-09-10T04:41:10.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.361+0000 7fd031ffb640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd02407abf0 0x7fd02407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.362+0000 7fd031ffb640 1 -- 172.21.15.46:0/3719933660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fd02009db60 con 0x7fd0341069a0 2024-09-10T04:41:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.362+0000 7fd038a74640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd02407abf0 0x7fd02407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:10.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.362+0000 7fd038a74640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd02407abf0 0x7fd02407d0b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd028000c00 tx=0x7fd028031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:10.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.366+0000 7fd031ffb640 1 -- 172.21.15.46:0/3719933660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fd020014030 con 0x7fd0341069a0 2024-09-10T04:41:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.626+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fd034198e00 con 0x7fd0341069a0 2024-09-10T04:41:10.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.626+0000 7fd031ffb640 1 -- 172.21.15.46:0/3719933660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7fd020069320 con 0x7fd0341069a0 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:41:10.629 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:41:10.630 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:41:10.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd02407abf0 msgr2=0x7fd02407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd02407abf0 0x7fd02407d0b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd028000c00 tx=0x7fd028031420 comp rx=0 tx=0).stop 2024-09-10T04:41:10.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 msgr2=0x7fd03419dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:10.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd03419dc50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd02000c8f0 tx=0x7fd02000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 shutdown_connections 2024-09-10T04:41:10.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fd02407abf0 0x7fd02407d0b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0341069a0 0x7fd03419dc50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 --2- 172.21.15.46:0/3719933660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0340ffdc0 0x7fd03419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:10.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.628+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 >> 172.21.15.46:0/3719933660 conn(0x7fd0340fbc90 msgr2=0x7fd0340fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:10.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.629+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 shutdown_connections 2024-09-10T04:41:10.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:10.629+0000 7fd03acd8640 1 -- 172.21.15.46:0/3719933660 wait complete. 2024-09-10T04:41:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:10 smithi070 ceph-mon[79774]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:10 smithi070 ceph-mon[79774]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:10 smithi070 ceph-mon[79774]: pgmap v1066: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:10 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1697770649' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:41:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:10 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3719933660' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:41:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:10 smithi046 ceph-mon[109506]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:10 smithi046 ceph-mon[109506]: from='client.35774 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:11.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:10 smithi046 ceph-mon[109506]: pgmap v1066: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:11.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:10 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1697770649' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:41:11.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:10 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3719933660' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:41:12.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:11 smithi070 ceph-mon[79774]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:12.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:11 smithi046 ceph-mon[109506]: from='client.35786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:12 smithi070 ceph-mon[79774]: pgmap v1067: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:12 smithi046 ceph-mon[109506]: pgmap v1067: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:14 smithi070 ceph-mon[79774]: pgmap v1068: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:14 smithi046 ceph-mon[109506]: pgmap v1068: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:16 smithi070 ceph-mon[79774]: pgmap v1069: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:16 smithi046 ceph-mon[109506]: pgmap v1069: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:18 smithi070 ceph-mon[79774]: pgmap v1070: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:18 smithi046 ceph-mon[109506]: pgmap v1070: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:20 smithi070 ceph-mon[79774]: pgmap v1071: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:20 smithi046 ceph-mon[109506]: pgmap v1071: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:23.240 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:22 smithi070 ceph-mon[79774]: pgmap v1072: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:23.285 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:22 smithi046 ceph-mon[109506]: pgmap v1072: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:24.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:25.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:24 smithi070 ceph-mon[79774]: pgmap v1073: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:25.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:24 smithi046 ceph-mon[109506]: pgmap v1073: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:27.321 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:26 smithi046 ceph-mon[109506]: pgmap v1074: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:27.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:26 smithi070 ceph-mon[79774]: pgmap v1074: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:29.274 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:28 smithi046 ceph-mon[109506]: pgmap v1075: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:29.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:28 smithi070 ceph-mon[79774]: pgmap v1075: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:31.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:30 smithi070 ceph-mon[79774]: pgmap v1076: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:31.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:30 smithi046 ceph-mon[109506]: pgmap v1076: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:33.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:32 smithi070 ceph-mon[79774]: pgmap v1077: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:33.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:32 smithi046 ceph-mon[109506]: pgmap v1077: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:35.282 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:34 smithi046 ceph-mon[109506]: pgmap v1078: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:35.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:34 smithi070 ceph-mon[79774]: pgmap v1078: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:36.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:35 smithi070 ceph-mon[79774]: pgmap v1079: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:36.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:35 smithi046 ceph-mon[109506]: pgmap v1079: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:38 smithi070 ceph-mon[79774]: pgmap v1080: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:38 smithi046 ceph-mon[109506]: pgmap v1080: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.727+0000 7fec48a23640 1 -- 172.21.15.46:0/504932828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 msgr2=0x7fec44107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.727+0000 7fec48a23640 1 --2- 172.21.15.46:0/504932828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44107ce0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fec34009880 tx=0x7fec3402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.728+0000 7fec48a23640 1 -- 172.21.15.46:0/504932828 shutdown_connections 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.728+0000 7fec48a23640 1 --2- 172.21.15.46:0/504932828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec44101900 0x7fec44101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.728+0000 7fec48a23640 1 --2- 172.21.15.46:0/504932828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44107ce0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.728+0000 7fec48a23640 1 -- 172.21.15.46:0/504932828 >> 172.21.15.46:0/504932828 conn(0x7fec440fd600 msgr2=0x7fec440ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.728+0000 7fec48a23640 1 -- 172.21.15.46:0/504932828 shutdown_connections 2024-09-10T04:41:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.728+0000 7fec48a23640 1 -- 172.21.15.46:0/504932828 wait complete. 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec48a23640 1 Processor -- start 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec48a23640 1 -- start start 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec48a23640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec44101900 0x7fec440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec48a23640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec48a23640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fec4410edd0 con 0x7fec44107900 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec48a23640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fec4410ec50 con 0x7fec44101900 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec41d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec41d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36770/0 (socket says 172.21.15.46:36770) 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec41d9b640 1 -- 172.21.15.46:0/1308493538 learned_addr learned my addr 172.21.15.46:0/1308493538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:40.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec4259c640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec44101900 0x7fec440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec41d9b640 1 -- 172.21.15.46:0/1308493538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec44101900 msgr2=0x7fec440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec41d9b640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec44101900 0x7fec440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.729+0000 7fec41d9b640 1 -- 172.21.15.46:0/1308493538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec44074f70 con 0x7fec44107900 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.730+0000 7fec41d9b640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44074a30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fec3800e970 tx=0x7fec3800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.730+0000 7fec337fe640 1 -- 172.21.15.46:0/1308493538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fec3800cac0 con 0x7fec44107900 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.730+0000 7fec337fe640 1 -- 172.21.15.46:0/1308493538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec380054e0 con 0x7fec44107900 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.730+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec44075260 con 0x7fec44107900 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.730+0000 7fec337fe640 1 -- 172.21.15.46:0/1308493538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fec38010430 con 0x7fec44107900 2024-09-10T04:41:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.730+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec441a3e40 con 0x7fec44107900 2024-09-10T04:41:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.731+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec44103040 con 0x7fec44107900 2024-09-10T04:41:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.732+0000 7fec337fe640 1 -- 172.21.15.46:0/1308493538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fec38004e80 con 0x7fec44107900 2024-09-10T04:41:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.732+0000 7fec337fe640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fec2c07abf0 0x7fec2c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.732+0000 7fec337fe640 1 -- 172.21.15.46:0/1308493538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fec3809e370 con 0x7fec44107900 2024-09-10T04:41:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.732+0000 7fec4259c640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fec2c07abf0 0x7fec2c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.733+0000 7fec4259c640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fec2c07abf0 0x7fec2c07d0b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fec340096c0 tx=0x7fec34031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:40.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.737+0000 7fec337fe640 1 -- 172.21.15.46:0/1308493538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fec38069ab0 con 0x7fec44107900 2024-09-10T04:41:40.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.938+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fec44062630 con 0x7fec2c07abf0 2024-09-10T04:41:40.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.939+0000 7fec337fe640 1 -- 172.21.15.46:0/1308493538 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fec44062630 con 0x7fec2c07abf0 2024-09-10T04:41:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fec2c07abf0 msgr2=0x7fec2c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fec2c07abf0 0x7fec2c07d0b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fec340096c0 tx=0x7fec34031400 comp rx=0 tx=0).stop 2024-09-10T04:41:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 msgr2=0x7fec44074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:40.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44074a30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fec3800e970 tx=0x7fec3800ed70 comp rx=0 tx=0).stop 2024-09-10T04:41:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 shutdown_connections 2024-09-10T04:41:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fec2c07abf0 0x7fec2c07d0b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec44107900 0x7fec44074a30 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 --2- 172.21.15.46:0/1308493538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec44101900 0x7fec440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 >> 172.21.15.46:0/1308493538 conn(0x7fec440fd600 msgr2=0x7fec440fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 shutdown_connections 2024-09-10T04:41:40.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:40.941+0000 7fec48a23640 1 -- 172.21.15.46:0/1308493538 wait complete. 2024-09-10T04:41:40.951 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:41:40.951 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:40 smithi046 ceph-mon[109506]: pgmap v1081: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:41.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 -- 172.21.15.46:0/1950878552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c106800 msgr2=0x7f2f0c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1950878552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c106800 0x7f2f0c106be0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009c20 tx=0x7f2f0802a970 comp rx=0 tx=0).stop 2024-09-10T04:41:41.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 -- 172.21.15.46:0/1950878552 shutdown_connections 2024-09-10T04:41:41.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1950878552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f0c100800 0x7f2f0c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1950878552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c106800 0x7f2f0c106be0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 -- 172.21.15.46:0/1950878552 >> 172.21.15.46:0/1950878552 conn(0x7f2f0c0fc520 msgr2=0x7f2f0c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 -- 172.21.15.46:0/1950878552 shutdown_connections 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.027+0000 7f2f1416d640 1 -- 172.21.15.46:0/1950878552 wait complete. 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f1416d640 1 Processor -- start 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f1416d640 1 -- start start 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f1416d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c100800 0x7f2f0c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f1416d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f0c106800 0x7f2f0c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f1416d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c10dcb0 con 0x7f2f0c100800 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f1416d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0c10db30 con 0x7f2f0c106800 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11f09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c100800 0x7f2f0c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11f09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c100800 0x7f2f0c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36792/0 (socket says 172.21.15.46:36792) 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11f09640 1 -- 172.21.15.46:0/1150366924 learned_addr learned my addr 172.21.15.46:0/1150366924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11708640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f0c106800 0x7f2f0c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11f09640 1 -- 172.21.15.46:0/1150366924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f0c106800 msgr2=0x7f2f0c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11f09640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f0c106800 0x7f2f0c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11f09640 1 -- 172.21.15.46:0/1150366924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0c199330 con 0x7f2f0c100800 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11708640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f0c106800 0x7f2f0c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.028+0000 7f2f11f09640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c100800 0x7f2f0c1987a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2f0802ae90 tx=0x7f2f0802f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.029+0000 7f2efeffd640 1 -- 172.21.15.46:0/1150366924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f08047070 con 0x7f2f0c100800 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.029+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0c1995c0 con 0x7f2f0c100800 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.029+0000 7f2efeffd640 1 -- 172.21.15.46:0/1150366924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2f0802f7e0 con 0x7f2f0c100800 2024-09-10T04:41:41.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.029+0000 7f2efeffd640 1 -- 172.21.15.46:0/1150366924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2f08034070 con 0x7f2f0c100800 2024-09-10T04:41:41.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.029+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0c1a1b70 con 0x7f2f0c100800 2024-09-10T04:41:41.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.030+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0c101f40 con 0x7f2f0c100800 2024-09-10T04:41:41.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.030+0000 7f2efeffd640 1 -- 172.21.15.46:0/1150366924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2f0804a070 con 0x7f2f0c100800 2024-09-10T04:41:41.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.031+0000 7f2efeffd640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2ef807abf0 0x7f2ef807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.031+0000 7f2efeffd640 1 -- 172.21.15.46:0/1150366924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2f080be9d0 con 0x7f2f0c100800 2024-09-10T04:41:41.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.031+0000 7f2f11708640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2ef807abf0 0x7f2ef807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.031+0000 7f2f11708640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2ef807abf0 0x7f2ef807d0b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2f000058c0 tx=0x7f2f00005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.035+0000 7f2efeffd640 1 -- 172.21.15.46:0/1150366924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2f0803f070 con 0x7f2f0c100800 2024-09-10T04:41:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:40 smithi070 ceph-mon[79774]: pgmap v1081: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.241+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f2f0c062630 con 0x7f2ef807abf0 2024-09-10T04:41:41.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.243+0000 7f2efeffd640 1 -- 172.21.15.46:0/1150366924 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f2f0c062630 con 0x7f2ef807abf0 2024-09-10T04:41:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2ef807abf0 msgr2=0x7f2ef807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2ef807abf0 0x7f2ef807d0b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2f000058c0 tx=0x7f2f00005850 comp rx=0 tx=0).stop 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c100800 msgr2=0x7f2f0c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c100800 0x7f2f0c1987a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2f0802ae90 tx=0x7f2f0802f5f0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 shutdown_connections 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2ef807abf0 0x7f2ef807d0b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f0c106800 0x7f2f0c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 --2- 172.21.15.46:0/1150366924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f0c100800 0x7f2f0c1987a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 >> 172.21.15.46:0/1150366924 conn(0x7f2f0c0fc520 msgr2=0x7f2f0c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 shutdown_connections 2024-09-10T04:41:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.245+0000 7f2f1416d640 1 -- 172.21.15.46:0/1150366924 wait complete. 2024-09-10T04:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1512691074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 msgr2=0x7f7cd8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1512691074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd8102ea0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009880 tx=0x7f7cd402a530 comp rx=0 tx=0).stop 2024-09-10T04:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1512691074 shutdown_connections 2024-09-10T04:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1512691074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd81033e0 0x7f7cd810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1512691074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd8102ea0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1512691074 >> 172.21.15.46:0/1512691074 conn(0x7f7cd80fc520 msgr2=0x7f7cd80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:41.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1512691074 shutdown_connections 2024-09-10T04:41:41.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.330+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1512691074 wait complete. 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7ce01f2640 1 Processor -- start 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7ce01f2640 1 -- start start 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7ce01f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7ce01f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd81033e0 0x7f7cd819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7ce01f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810edf0 con 0x7f7cd8102ac0 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7ce01f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd810ec70 con 0x7f7cd81033e0 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7cddf8e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7cddf8e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd819f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36800/0 (socket says 172.21.15.46:36800) 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7cddf8e640 1 -- 172.21.15.46:0/1095040697 learned_addr learned my addr 172.21.15.46:0/1095040697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:41.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.331+0000 7f7cdd78d640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd81033e0 0x7f7cd819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7cddf8e640 1 -- 172.21.15.46:0/1095040697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd81033e0 msgr2=0x7f7cd819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7cddf8e640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd81033e0 0x7f7cd819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7cddf8e640 1 -- 172.21.15.46:0/1095040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd8199a30 con 0x7f7cd8102ac0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7cddf8e640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd819f820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4031460 tx=0x7f7cd40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7cceffd640 1 -- 172.21.15.46:0/1095040697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cd403f070 con 0x7f7cd8102ac0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd8199cc0 con 0x7f7cd8102ac0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7cceffd640 1 -- 172.21.15.46:0/1095040697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7cd4004cf0 con 0x7f7cd8102ac0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7cceffd640 1 -- 172.21.15.46:0/1095040697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7cd4035030 con 0x7f7cd8102ac0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.332+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd819a1f0 con 0x7f7cd8102ac0 2024-09-10T04:41:41.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.333+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd806aa90 con 0x7f7cd8102ac0 2024-09-10T04:41:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.334+0000 7f7cceffd640 1 -- 172.21.15.46:0/1095040697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7cd4042070 con 0x7f7cd8102ac0 2024-09-10T04:41:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.334+0000 7f7cceffd640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cc407abf0 0x7f7cc407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.335+0000 7f7cdd78d640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cc407abf0 0x7f7cc407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.335+0000 7f7cceffd640 1 -- 172.21.15.46:0/1095040697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7cd40be750 con 0x7f7cd8102ac0 2024-09-10T04:41:41.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.335+0000 7f7cdd78d640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cc407abf0 0x7f7cc407d0b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80fee80 tx=0x7f7cc8005ec0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.339+0000 7f7cceffd640 1 -- 172.21.15.46:0/1095040697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7cd4089e90 con 0x7f7cd8102ac0 2024-09-10T04:41:41.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.534+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7cd819af10 con 0x7f7cc407abf0 2024-09-10T04:41:41.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.544+0000 7f7cceffd640 1 -- 172.21.15.46:0/1095040697 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f7cd819af10 con 0x7f7cc407abf0 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (33m) 12m ago 44m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (35m) 12m ago 44m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (35m) 12m ago 43m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (31m) 12m ago 44m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (31m) 12m ago 43m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (32m) 12m ago 44m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (27m) 12m ago 39m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (28m) 12m ago 39m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:41:41.545 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (27m) 12m ago 39m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (12m) 12m ago 39m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (35m) 12m ago 45m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (34m) 12m ago 43m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (32m) 12m ago 46m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (31m) 12m ago 43m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (34m) 12m ago 44m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (33m) 12m ago 43m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (31m) 12m ago 42m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (30m) 12m ago 42m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (30m) 12m ago 41m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (29m) 12m ago 41m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (28m) 12m ago 40m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (28m) 12m ago 40m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:41:41.546 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (33m) 12m ago 44m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cc407abf0 msgr2=0x7f7cc407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cc407abf0 0x7f7cc407d0b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7cd80fee80 tx=0x7f7cc8005ec0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 msgr2=0x7f7cd819f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd819f820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4031460 tx=0x7f7cd40048f0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 shutdown_connections 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cc407abf0 0x7f7cc407d0b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7cd81033e0 0x7f7cd819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 --2- 172.21.15.46:0/1095040697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cd8102ac0 0x7f7cd819f820 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 >> 172.21.15.46:0/1095040697 conn(0x7f7cd80fc520 msgr2=0x7f7cd8100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:41.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 shutdown_connections 2024-09-10T04:41:41.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.547+0000 7f7ce01f2640 1 -- 172.21.15.46:0/1095040697 wait complete. 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.631+0000 7f1c730a7640 1 -- 172.21.15.46:0/136083000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c102b50 msgr2=0x7f1c6c102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.631+0000 7f1c730a7640 1 --2- 172.21.15.46:0/136083000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c102b50 0x7f1c6c102f30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009880 tx=0x7f1c5c02a530 comp rx=0 tx=0).stop 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.632+0000 7f1c730a7640 1 -- 172.21.15.46:0/136083000 shutdown_connections 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.632+0000 7f1c730a7640 1 --2- 172.21.15.46:0/136083000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6c103470 0x7f1c6c10b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.632+0000 7f1c730a7640 1 --2- 172.21.15.46:0/136083000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c102b50 0x7f1c6c102f30 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.632+0000 7f1c730a7640 1 -- 172.21.15.46:0/136083000 >> 172.21.15.46:0/136083000 conn(0x7f1c6c0fc550 msgr2=0x7f1c6c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.632+0000 7f1c730a7640 1 -- 172.21.15.46:0/136083000 shutdown_connections 2024-09-10T04:41:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.632+0000 7f1c730a7640 1 -- 172.21.15.46:0/136083000 wait complete. 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.632+0000 7f1c730a7640 1 Processor -- start 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c730a7640 1 -- start start 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c730a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6c102b50 0x7f1c6c1007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c730a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c103470 0x7f1c6c100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c730a7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10ede0 con 0x7f1c6c103470 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c730a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10ec60 con 0x7f1c6c102b50 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c103470 0x7f1c6c100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c6bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c103470 0x7f1c6c100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36824/0 (socket says 172.21.15.46:36824) 2024-09-10T04:41:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c6bfff640 1 -- 172.21.15.46:0/2801738994 learned_addr learned my addr 172.21.15.46:0/2801738994 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c70e43640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6c102b50 0x7f1c6c1007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c6bfff640 1 -- 172.21.15.46:0/2801738994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6c102b50 msgr2=0x7f1c6c1007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c6bfff640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6c102b50 0x7f1c6c1007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c6bfff640 1 -- 172.21.15.46:0/2801738994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c101230 con 0x7f1c6c103470 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.633+0000 7f1c6bfff640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c103470 0x7f1c6c100cf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c8f0 tx=0x7f1c6000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.634+0000 7f1c69ffb640 1 -- 172.21.15.46:0/2801738994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c600079b0 con 0x7f1c6c103470 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.634+0000 7f1c69ffb640 1 -- 172.21.15.46:0/2801738994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c600054e0 con 0x7f1c6c103470 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.634+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c101520 con 0x7f1c6c103470 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.634+0000 7f1c69ffb640 1 -- 172.21.15.46:0/2801738994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f1c60004dd0 con 0x7f1c6c103470 2024-09-10T04:41:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.634+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a3e50 con 0x7f1c6c103470 2024-09-10T04:41:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.635+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c06a4e0 con 0x7f1c6c103470 2024-09-10T04:41:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.635+0000 7f1c69ffb640 1 -- 172.21.15.46:0/2801738994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f1c60005020 con 0x7f1c6c103470 2024-09-10T04:41:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.636+0000 7f1c69ffb640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1c5407abf0 0x7f1c5407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.636+0000 7f1c69ffb640 1 -- 172.21.15.46:0/2801738994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f1c6009dc70 con 0x7f1c6c103470 2024-09-10T04:41:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.636+0000 7f1c70e43640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1c5407abf0 0x7f1c5407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.636+0000 7f1c70e43640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1c5407abf0 0x7f1c5407d0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c004f90 tx=0x7f1c5c005bd0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.640+0000 7f1c69ffb640 1 -- 172.21.15.46:0/2801738994 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f1c600a4080 con 0x7f1c6c103470 2024-09-10T04:41:41.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.902+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f1c6c105fb0 con 0x7f1c6c103470 2024-09-10T04:41:41.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.902+0000 7f1c69ffb640 1 -- 172.21.15.46:0/2801738994 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f1c60014030 con 0x7f1c6c103470 2024-09-10T04:41:41.903 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:41:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:41:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:41:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:41:41.903 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:41:41.904 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:41:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1c5407abf0 msgr2=0x7f1c5407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1c5407abf0 0x7f1c5407d0b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c004f90 tx=0x7f1c5c005bd0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c103470 msgr2=0x7f1c6c100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c103470 0x7f1c6c100cf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000c8f0 tx=0x7f1c6000cdb0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 shutdown_connections 2024-09-10T04:41:41.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f1c5407abf0 0x7f1c5407d0b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c103470 0x7f1c6c100cf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 --2- 172.21.15.46:0/2801738994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c6c102b50 0x7f1c6c1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 >> 172.21.15.46:0/2801738994 conn(0x7f1c6c0fc550 msgr2=0x7f1c6c108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 shutdown_connections 2024-09-10T04:41:41.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.904+0000 7f1c730a7640 1 -- 172.21.15.46:0/2801738994 wait complete. 2024-09-10T04:41:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 -- 172.21.15.46:0/2908573226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 msgr2=0x7fb398101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 --2- 172.21.15.46:0/2908573226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb398101d70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009880 tx=0x7fb38c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 -- 172.21.15.46:0/2908573226 shutdown_connections 2024-09-10T04:41:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 --2- 172.21.15.46:0/2908573226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb398101d70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 --2- 172.21.15.46:0/2908573226 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398107910 0x7fb398107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 -- 172.21.15.46:0/2908573226 >> 172.21.15.46:0/2908573226 conn(0x7fb3980fd630 msgr2=0x7fb3980ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 -- 172.21.15.46:0/2908573226 shutdown_connections 2024-09-10T04:41:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.989+0000 7fb39d660640 1 -- 172.21.15.46:0/2908573226 wait complete. 2024-09-10T04:41:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.990+0000 7fb39d660640 1 Processor -- start 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.990+0000 7fb39d660640 1 -- start start 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.990+0000 7fb39d660640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb39819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.990+0000 7fb39d660640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398107910 0x7fb39819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.990+0000 7fb39d660640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb39810edc0 con 0x7fb398101910 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.990+0000 7fb39d660640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb39810ec40 con 0x7fb398107910 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb396ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb39819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb396ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb39819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36834/0 (socket says 172.21.15.46:36834) 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb396ffd640 1 -- 172.21.15.46:0/55717034 learned_addr learned my addr 172.21.15.46:0/55717034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb3967fc640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398107910 0x7fb39819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb396ffd640 1 -- 172.21.15.46:0/55717034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398107910 msgr2=0x7fb39819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb396ffd640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398107910 0x7fb39819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb396ffd640 1 -- 172.21.15.46:0/55717034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb398197770 con 0x7fb398101910 2024-09-10T04:41:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb396ffd640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb39819d6a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d8d0 tx=0x7fb38800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb37ffff640 1 -- 172.21.15.46:0/55717034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb388017070 con 0x7fb398101910 2024-09-10T04:41:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb37ffff640 1 -- 172.21.15.46:0/55717034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb388010460 con 0x7fb398101910 2024-09-10T04:41:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb398197a60 con 0x7fb398101910 2024-09-10T04:41:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb37ffff640 1 -- 172.21.15.46:0/55717034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb38800f5f0 con 0x7fb398101910 2024-09-10T04:41:41.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.991+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb398197fc0 con 0x7fb398101910 2024-09-10T04:41:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.992+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb398103050 con 0x7fb398101910 2024-09-10T04:41:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.993+0000 7fb37ffff640 1 -- 172.21.15.46:0/55717034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb388009a30 con 0x7fb398101910 2024-09-10T04:41:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.994+0000 7fb37ffff640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb38007ac40 0x7fb38007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.994+0000 7fb3967fc640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb38007ac40 0x7fb38007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.994+0000 7fb37ffff640 1 -- 172.21.15.46:0/55717034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb38809da70 con 0x7fb398101910 2024-09-10T04:41:41.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.995+0000 7fb3967fc640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb38007ac40 0x7fb38007d100 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb398198fc0 tx=0x7fb38c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:41.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:41.999+0000 7fb37ffff640 1 -- 172.21.15.46:0/55717034 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb388069230 con 0x7fb398101910 2024-09-10T04:41:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.220+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb398109ea0 con 0x7fb398101910 2024-09-10T04:41:42.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.221+0000 7fb37ffff640 1 -- 172.21.15.46:0/55717034 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fb38806e0e0 con 0x7fb398101910 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:41:42.222 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:41:42.223 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:41:42.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:41:42.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:42.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:42.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:42.225 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:41:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:41:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb38007ac40 msgr2=0x7fb38007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb38007ac40 0x7fb38007d100 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb398198fc0 tx=0x7fb38c031400 comp rx=0 tx=0).stop 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 msgr2=0x7fb39819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb39819d6a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d8d0 tx=0x7fb38800dd90 comp rx=0 tx=0).stop 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 shutdown_connections 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb38007ac40 0x7fb38007d100 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398107910 0x7fb39819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 --2- 172.21.15.46:0/55717034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb398101910 0x7fb39819d6a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 >> 172.21.15.46:0/55717034 conn(0x7fb3980fd630 msgr2=0x7fb3980fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 shutdown_connections 2024-09-10T04:41:42.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.223+0000 7fb39d660640 1 -- 172.21.15.46:0/55717034 wait complete. 2024-09-10T04:41:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.309+0000 7fb17f65c640 1 -- 172.21.15.46:0/1800986922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 msgr2=0x7fb178107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.309+0000 7fb17f65c640 1 --2- 172.21.15.46:0/1800986922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb178107cf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb174009880 tx=0x7fb17402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.309+0000 7fb17f65c640 1 -- 172.21.15.46:0/1800986922 shutdown_connections 2024-09-10T04:41:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.309+0000 7fb17f65c640 1 --2- 172.21.15.46:0/1800986922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb178101910 0x7fb178101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 --2- 172.21.15.46:0/1800986922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb178107cf0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 -- 172.21.15.46:0/1800986922 >> 172.21.15.46:0/1800986922 conn(0x7fb1780fd630 msgr2=0x7fb1780ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 -- 172.21.15.46:0/1800986922 shutdown_connections 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 -- 172.21.15.46:0/1800986922 wait complete. 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 Processor -- start 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 -- start start 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb178101910 0x7fb17819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.310+0000 7fb17f65c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb17819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17f65c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810ede0 con 0x7fb178107910 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17f65c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb17810ec60 con 0x7fb178101910 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17cbf7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb17819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17cbf7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb17819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36842/0 (socket says 172.21.15.46:36842) 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17cbf7640 1 -- 172.21.15.46:0/4151556725 learned_addr learned my addr 172.21.15.46:0/4151556725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17d3f8640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb178101910 0x7fb17819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17cbf7640 1 -- 172.21.15.46:0/4151556725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb178101910 msgr2=0x7fb17819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17cbf7640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb178101910 0x7fb17819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17cbf7640 1 -- 172.21.15.46:0/4151556725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1781977b0 con 0x7fb178107910 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17d3f8640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb178101910 0x7fb17819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17cbf7640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb17819db70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e970 tx=0x7fb16800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb16e7fc640 1 -- 172.21.15.46:0/4151556725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb16800cac0 con 0x7fb178107910 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb16e7fc640 1 -- 172.21.15.46:0/4151556725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb1680054e0 con 0x7fb178107910 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb178197aa0 con 0x7fb178107910 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb16e7fc640 1 -- 172.21.15.46:0/4151556725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb168010430 con 0x7fb178107910 2024-09-10T04:41:42.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.311+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb178198030 con 0x7fb178107910 2024-09-10T04:41:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.312+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb178103050 con 0x7fb178107910 2024-09-10T04:41:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.313+0000 7fb16e7fc640 1 -- 172.21.15.46:0/4151556725 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb168004f00 con 0x7fb178107910 2024-09-10T04:41:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.314+0000 7fb16e7fc640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.314+0000 7fb17d3f8640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.314+0000 7fb16e7fc640 1 -- 172.21.15.46:0/4151556725 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb16809e380 con 0x7fb178107910 2024-09-10T04:41:42.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.314+0000 7fb17d3f8640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb1740095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:42.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.318+0000 7fb16e7fc640 1 -- 172.21.15.46:0/4151556725 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb168069b40 con 0x7fb178107910 2024-09-10T04:41:42.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.523+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb1781983a0 con 0x7fb16407ac40 2024-09-10T04:41:42.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.525+0000 7fb16e7fc640 1 -- 172.21.15.46:0/4151556725 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fb1781983a0 con 0x7fb16407ac40 2024-09-10T04:41:42.525 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:41:42.526 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:41:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 msgr2=0x7fb16407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb174006010 tx=0x7fb1740095f0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 msgr2=0x7fb17819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb17819db70 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb16800e970 tx=0x7fb16800ed70 comp rx=0 tx=0).stop 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 shutdown_connections 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb16407ac40 0x7fb16407d100 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb178107910 0x7fb17819db70 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 --2- 172.21.15.46:0/4151556725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb178101910 0x7fb17819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 >> 172.21.15.46:0/4151556725 conn(0x7fb1780fd630 msgr2=0x7fb1780fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 shutdown_connections 2024-09-10T04:41:42.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.528+0000 7fb17f65c640 1 -- 172.21.15.46:0/4151556725 wait complete. 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.616+0000 7f3399ff4640 1 -- 172.21.15.46:0/2949594232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 msgr2=0x7f339410b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.616+0000 7f3399ff4640 1 --2- 172.21.15.46:0/2949594232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339410b8c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3388009880 tx=0x7f338802a530 comp rx=0 tx=0).stop 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.617+0000 7f3399ff4640 1 -- 172.21.15.46:0/2949594232 shutdown_connections 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.617+0000 7f3399ff4640 1 --2- 172.21.15.46:0/2949594232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339410b8c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.617+0000 7f3399ff4640 1 --2- 172.21.15.46:0/2949594232 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3394102ab0 0x7f3394102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.617+0000 7f3399ff4640 1 -- 172.21.15.46:0/2949594232 >> 172.21.15.46:0/2949594232 conn(0x7f33940fc550 msgr2=0x7f33940fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.617+0000 7f3399ff4640 1 -- 172.21.15.46:0/2949594232 shutdown_connections 2024-09-10T04:41:42.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.617+0000 7f3399ff4640 1 -- 172.21.15.46:0/2949594232 wait complete. 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.617+0000 7f3399ff4640 1 Processor -- start 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3399ff4640 1 -- start start 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3399ff4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3394102ab0 0x7f339419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3399ff4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3399ff4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f339410ede0 con 0x7f33941033d0 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3399ff4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f339410ec60 con 0x7f3394102ab0 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3392ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3392ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36862/0 (socket says 172.21.15.46:36862) 2024-09-10T04:41:42.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3392ffd640 1 -- 172.21.15.46:0/1558502621 learned_addr learned my addr 172.21.15.46:0/1558502621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f33937fe640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3394102ab0 0x7f339419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3392ffd640 1 -- 172.21.15.46:0/1558502621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3394102ab0 msgr2=0x7f339419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3392ffd640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3394102ab0 0x7f339419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.618+0000 7f3392ffd640 1 -- 172.21.15.46:0/1558502621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3394197810 con 0x7f33941033d0 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.619+0000 7f3392ffd640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339419dbd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3388031460 tx=0x7f3388004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.619+0000 7f3390ff9640 1 -- 172.21.15.46:0/1558502621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f338803f070 con 0x7f33941033d0 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.619+0000 7f3390ff9640 1 -- 172.21.15.46:0/1558502621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33880035f0 con 0x7f33941033d0 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.619+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3394197aa0 con 0x7f33941033d0 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.619+0000 7f3390ff9640 1 -- 172.21.15.46:0/1558502621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3388035030 con 0x7f33941033d0 2024-09-10T04:41:42.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.619+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3394197fd0 con 0x7f33941033d0 2024-09-10T04:41:42.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.620+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339406a580 con 0x7f33941033d0 2024-09-10T04:41:42.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.621+0000 7f3390ff9640 1 -- 172.21.15.46:0/1558502621 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3388042070 con 0x7f33941033d0 2024-09-10T04:41:42.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.621+0000 7f3390ff9640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f337c07abf0 0x7f337c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:41:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.621+0000 7f3390ff9640 1 -- 172.21.15.46:0/1558502621 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f33880be5b0 con 0x7f33941033d0 2024-09-10T04:41:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.621+0000 7f33937fe640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f337c07abf0 0x7f337c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:41:42.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.622+0000 7f33937fe640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f337c07abf0 0x7f337c07d0b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3384005fd0 tx=0x7f338400b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:41:42.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.626+0000 7f3390ff9640 1 -- 172.21.15.46:0/1558502621 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3388089cf0 con 0x7f33941033d0 2024-09-10T04:41:42.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:42 smithi046 ceph-mon[109506]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:42 smithi046 ceph-mon[109506]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:42 smithi046 ceph-mon[109506]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:42 smithi046 ceph-mon[109506]: pgmap v1082: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:42 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2801738994' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:41:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:42 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/55717034' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:41:42.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.886+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f3394198d40 con 0x7f33941033d0 2024-09-10T04:41:42.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.887+0000 7f3390ff9640 1 -- 172.21.15.46:0/1558502621 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f338808eba0 con 0x7f33941033d0 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:41:42.888 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:41:42.889 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:41:42.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f337c07abf0 msgr2=0x7f337c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f337c07abf0 0x7f337c07d0b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3384005fd0 tx=0x7f338400b390 comp rx=0 tx=0).stop 2024-09-10T04:41:42.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 msgr2=0x7f339419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:41:42.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339419dbd0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3388031460 tx=0x7f3388004530 comp rx=0 tx=0).stop 2024-09-10T04:41:42.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 shutdown_connections 2024-09-10T04:41:42.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f337c07abf0 0x7f337c07d0b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33941033d0 0x7f339419dbd0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 --2- 172.21.15.46:0/1558502621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3394102ab0 0x7f339419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:41:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 >> 172.21.15.46:0/1558502621 conn(0x7f33940fc550 msgr2=0x7f3394108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:41:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 shutdown_connections 2024-09-10T04:41:42.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:41:42.889+0000 7f3399ff4640 1 -- 172.21.15.46:0/1558502621 wait complete. 2024-09-10T04:41:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:42 smithi070 ceph-mon[79774]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:42 smithi070 ceph-mon[79774]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:42 smithi070 ceph-mon[79774]: from='client.35802 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:42 smithi070 ceph-mon[79774]: pgmap v1082: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:42 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2801738994' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:41:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:42 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/55717034' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:41:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:43 smithi070 ceph-mon[79774]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:43.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:43 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1558502621' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:41:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:43 smithi046 ceph-mon[109506]: from='client.35814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:41:43.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:43 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1558502621' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:41:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:44 smithi070 ceph-mon[79774]: pgmap v1083: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:45.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:44 smithi046 ceph-mon[109506]: pgmap v1083: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:46 smithi070 ceph-mon[79774]: pgmap v1084: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:46 smithi046 ceph-mon[109506]: pgmap v1084: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:48 smithi070 ceph-mon[79774]: pgmap v1085: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:48 smithi046 ceph-mon[109506]: pgmap v1085: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:50 smithi070 ceph-mon[79774]: pgmap v1086: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:50 smithi046 ceph-mon[109506]: pgmap v1086: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:52 smithi070 ceph-mon[79774]: pgmap v1087: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:52 smithi046 ceph-mon[109506]: pgmap v1087: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:53.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:41:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:54 smithi070 ceph-mon[79774]: pgmap v1088: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:54 smithi046 ceph-mon[109506]: pgmap v1088: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:56 smithi070 ceph-mon[79774]: pgmap v1089: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:56 smithi046 ceph-mon[109506]: pgmap v1089: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:41:58 smithi070 ceph-mon[79774]: pgmap v1090: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:41:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:41:58 smithi046 ceph-mon[109506]: pgmap v1090: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:00 smithi070 ceph-mon[79774]: pgmap v1091: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:00 smithi046 ceph-mon[109506]: pgmap v1091: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:02 smithi070 ceph-mon[79774]: pgmap v1092: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:02 smithi046 ceph-mon[109506]: pgmap v1092: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:04 smithi070 ceph-mon[79774]: pgmap v1093: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:05.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:04 smithi046 ceph-mon[109506]: pgmap v1093: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:06 smithi070 ceph-mon[79774]: pgmap v1094: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:06 smithi046 ceph-mon[109506]: pgmap v1094: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:08 smithi070 ceph-mon[79774]: pgmap v1095: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:08 smithi046 ceph-mon[109506]: pgmap v1095: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:10 smithi070 ceph-mon[79774]: pgmap v1096: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:10 smithi046 ceph-mon[109506]: pgmap v1096: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 -- 172.21.15.46:0/2070212057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 msgr2=0x7fe2f4102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 --2- 172.21.15.46:0/2070212057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f4102ea0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009880 tx=0x7fe2e402a530 comp rx=0 tx=0).stop 2024-09-10T04:42:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 -- 172.21.15.46:0/2070212057 shutdown_connections 2024-09-10T04:42:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 --2- 172.21.15.46:0/2070212057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f41033e0 0x7fe2f410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:12.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 --2- 172.21.15.46:0/2070212057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f4102ea0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 -- 172.21.15.46:0/2070212057 >> 172.21.15.46:0/2070212057 conn(0x7fe2f40fc520 msgr2=0x7fe2f40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 -- 172.21.15.46:0/2070212057 shutdown_connections 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 -- 172.21.15.46:0/2070212057 wait complete. 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 Processor -- start 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.986+0000 7fe2fa169640 1 -- start start 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2fa169640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2fa169640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f41033e0 0x7fe2f419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2fa169640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410edf0 con 0x7fe2f4102ac0 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2fa169640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe2f410ec70 con 0x7fe2f41033e0 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2f37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2f37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35856/0 (socket says 172.21.15.46:35856) 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2f37fe640 1 -- 172.21.15.46:0/3786127455 learned_addr learned my addr 172.21.15.46:0/3786127455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:12.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2eaffd640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f41033e0 0x7fe2f419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2f37fe640 1 -- 172.21.15.46:0/3786127455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f41033e0 msgr2=0x7fe2f419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2f37fe640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f41033e0 0x7fe2f419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2f37fe640 1 -- 172.21.15.46:0/3786127455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2f4197780 con 0x7fe2f4102ac0 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.987+0000 7fe2eaffd640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f41033e0 0x7fe2f419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.988+0000 7fe2f37fe640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f419d600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4031460 tx=0x7fe2e40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.988+0000 7fe2f17fa640 1 -- 172.21.15.46:0/3786127455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe2e403f070 con 0x7fe2f4102ac0 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.988+0000 7fe2f17fa640 1 -- 172.21.15.46:0/3786127455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe2e4004b10 con 0x7fe2f4102ac0 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.988+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2f4197a10 con 0x7fe2f4102ac0 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.988+0000 7fe2f17fa640 1 -- 172.21.15.46:0/3786127455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe2e4035030 con 0x7fe2f4102ac0 2024-09-10T04:42:12.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.988+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2f4197f40 con 0x7fe2f4102ac0 2024-09-10T04:42:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.989+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2f406aa90 con 0x7fe2f4102ac0 2024-09-10T04:42:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.990+0000 7fe2f17fa640 1 -- 172.21.15.46:0/3786127455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe2e4042070 con 0x7fe2f4102ac0 2024-09-10T04:42:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.991+0000 7fe2f17fa640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe2e007ac00 0x7fe2e007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.991+0000 7fe2f17fa640 1 -- 172.21.15.46:0/3786127455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe2e40be7d0 con 0x7fe2f4102ac0 2024-09-10T04:42:12.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.991+0000 7fe2eaffd640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe2e007ac00 0x7fe2e007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:12.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.991+0000 7fe2eaffd640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe2e007ac00 0x7fe2e007d0c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4108900 tx=0x7fe2dc009670 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:12.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:12.995+0000 7fe2f17fa640 1 -- 172.21.15.46:0/3786127455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe2e4089f10 con 0x7fe2f4102ac0 2024-09-10T04:42:13.018 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:12 smithi046 ceph-mon[109506]: pgmap v1097: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:12 smithi070 ceph-mon[79774]: pgmap v1097: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:13.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.202+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe2f41981f0 con 0x7fe2e007ac00 2024-09-10T04:42:13.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.204+0000 7fe2f17fa640 1 -- 172.21.15.46:0/3786127455 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fe2f41981f0 con 0x7fe2e007ac00 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe2e007ac00 msgr2=0x7fe2e007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe2e007ac00 0x7fe2e007d0c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4108900 tx=0x7fe2dc009670 comp rx=0 tx=0).stop 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 msgr2=0x7fe2f419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f419d600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4031460 tx=0x7fe2e40045e0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 shutdown_connections 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe2e007ac00 0x7fe2e007d0c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe2f41033e0 0x7fe2f419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 --2- 172.21.15.46:0/3786127455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2f4102ac0 0x7fe2f419d600 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 >> 172.21.15.46:0/3786127455 conn(0x7fe2f40fc520 msgr2=0x7fe2f410af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 shutdown_connections 2024-09-10T04:42:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.206+0000 7fe2fa169640 1 -- 172.21.15.46:0/3786127455 wait complete. 2024-09-10T04:42:13.216 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:42:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 -- 172.21.15.46:0/3605725083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 msgr2=0x7fe11c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 --2- 172.21.15.46:0/3605725083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c10b8d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe104009880 tx=0x7fe10402a530 comp rx=0 tx=0).stop 2024-09-10T04:42:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 -- 172.21.15.46:0/3605725083 shutdown_connections 2024-09-10T04:42:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 --2- 172.21.15.46:0/3605725083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c10b8d0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 --2- 172.21.15.46:0/3605725083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c102ac0 0x7fe11c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 -- 172.21.15.46:0/3605725083 >> 172.21.15.46:0/3605725083 conn(0x7fe11c0fc520 msgr2=0x7fe11c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 -- 172.21.15.46:0/3605725083 shutdown_connections 2024-09-10T04:42:13.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.291+0000 7fe12278c640 1 -- 172.21.15.46:0/3605725083 wait complete. 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.292+0000 7fe12278c640 1 Processor -- start 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.292+0000 7fe12278c640 1 -- start start 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.292+0000 7fe12278c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c102ac0 0x7fe11c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.292+0000 7fe12278c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.292+0000 7fe12278c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c10edd0 con 0x7fe11c1033e0 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.292+0000 7fe12278c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe11c10ec50 con 0x7fe11c102ac0 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35864/0 (socket says 172.21.15.46:35864) 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11b7fe640 1 -- 172.21.15.46:0/3860511319 learned_addr learned my addr 172.21.15.46:0/3860511319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11bfff640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c102ac0 0x7fe11c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11b7fe640 1 -- 172.21.15.46:0/3860511319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c102ac0 msgr2=0x7fe11c100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11b7fe640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c102ac0 0x7fe11c100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11b7fe640 1 -- 172.21.15.46:0/3860511319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11c1011a0 con 0x7fe11c1033e0 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe11b7fe640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c100c60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe104031460 tx=0x7fe1040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe1197fa640 1 -- 172.21.15.46:0/3860511319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe10403f070 con 0x7fe11c1033e0 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11c101430 con 0x7fe11c1033e0 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe1197fa640 1 -- 172.21.15.46:0/3860511319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe104004ac0 con 0x7fe11c1033e0 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.293+0000 7fe1197fa640 1 -- 172.21.15.46:0/3860511319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe104035030 con 0x7fe11c1033e0 2024-09-10T04:42:13.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.294+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe11c1a3d50 con 0x7fe11c1033e0 2024-09-10T04:42:13.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.295+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe11c06aa90 con 0x7fe11c1033e0 2024-09-10T04:42:13.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.295+0000 7fe1197fa640 1 -- 172.21.15.46:0/3860511319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe104042070 con 0x7fe11c1033e0 2024-09-10T04:42:13.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.296+0000 7fe1197fa640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10807abf0 0x7fe10807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.296+0000 7fe1197fa640 1 -- 172.21.15.46:0/3860511319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe1040be5f0 con 0x7fe11c1033e0 2024-09-10T04:42:13.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.296+0000 7fe11bfff640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10807abf0 0x7fe10807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.296+0000 7fe11bfff640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10807abf0 0x7fe10807d0b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00abe0 tx=0x7fe10c00a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:13.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.300+0000 7fe1197fa640 1 -- 172.21.15.46:0/3860511319 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe104089d30 con 0x7fe11c1033e0 2024-09-10T04:42:13.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.505+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fe11c062630 con 0x7fe10807abf0 2024-09-10T04:42:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.506+0000 7fe1197fa640 1 -- 172.21.15.46:0/3860511319 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fe11c062630 con 0x7fe10807abf0 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.508+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10807abf0 msgr2=0x7fe10807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.508+0000 7fe12278c640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10807abf0 0x7fe10807d0b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00abe0 tx=0x7fe10c00a2d0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.508+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 msgr2=0x7fe11c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.508+0000 7fe12278c640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c100c60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe104031460 tx=0x7fe1040046c0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.509+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 shutdown_connections 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.509+0000 7fe12278c640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10807abf0 0x7fe10807d0b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.509+0000 7fe12278c640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe11c1033e0 0x7fe11c100c60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.509+0000 7fe12278c640 1 --2- 172.21.15.46:0/3860511319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe11c102ac0 0x7fe11c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.509+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 >> 172.21.15.46:0/3860511319 conn(0x7fe11c0fc520 msgr2=0x7fe11c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.509+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 shutdown_connections 2024-09-10T04:42:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.509+0000 7fe12278c640 1 -- 172.21.15.46:0/3860511319 wait complete. 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 -- 172.21.15.46:0/1829009014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074e30 msgr2=0x7f59f010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1829009014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074e30 0x7f59f010dc90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009880 tx=0x7f59e402a530 comp rx=0 tx=0).stop 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 -- 172.21.15.46:0/1829009014 shutdown_connections 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1829009014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074e30 0x7f59f010dc90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1829009014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59f0074510 0x7f59f00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 -- 172.21.15.46:0/1829009014 >> 172.21.15.46:0/1829009014 conn(0x7f59f00fd610 msgr2=0x7f59f00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 -- 172.21.15.46:0/1829009014 shutdown_connections 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.593+0000 7f59f61f2640 1 -- 172.21.15.46:0/1829009014 wait complete. 2024-09-10T04:42:13.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.594+0000 7f59f61f2640 1 Processor -- start 2024-09-10T04:42:13.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.594+0000 7f59f61f2640 1 -- start start 2024-09-10T04:42:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.594+0000 7f59f61f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074510 0x7f59f019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.594+0000 7f59f61f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59f0074e30 0x7f59f019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.594+0000 7f59f61f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0110fd0 con 0x7f59f0074510 2024-09-10T04:42:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.594+0000 7f59f61f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0110e50 con 0x7f59f0074e30 2024-09-10T04:42:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074510 0x7f59f019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ef7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074510 0x7f59f019f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35880/0 (socket says 172.21.15.46:35880) 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ef7fe640 1 -- 172.21.15.46:0/1069921138 learned_addr learned my addr 172.21.15.46:0/1069921138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59eeffd640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59f0074e30 0x7f59f019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ef7fe640 1 -- 172.21.15.46:0/1069921138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59f0074e30 msgr2=0x7f59f019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ef7fe640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59f0074e30 0x7f59f019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ef7fe640 1 -- 172.21.15.46:0/1069921138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f0199a80 con 0x7f59f0074510 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ef7fe640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074510 0x7f59f019f8c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d8d0 tx=0x7f59e000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ecff9640 1 -- 172.21.15.46:0/1069921138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f59e0017070 con 0x7f59f0074510 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f0199d70 con 0x7f59f0074510 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ecff9640 1 -- 172.21.15.46:0/1069921138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59e0010460 con 0x7f59f0074510 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59ecff9640 1 -- 172.21.15.46:0/1069921138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f59e000f5f0 con 0x7f59f0074510 2024-09-10T04:42:13.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.595+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f019a2d0 con 0x7f59f0074510 2024-09-10T04:42:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.596+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f0075c50 con 0x7f59f0074510 2024-09-10T04:42:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.597+0000 7f59ecff9640 1 -- 172.21.15.46:0/1069921138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f59e0009a30 con 0x7f59f0074510 2024-09-10T04:42:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.597+0000 7f59ecff9640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f59d807abf0 0x7f59d807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.597+0000 7f59ecff9640 1 -- 172.21.15.46:0/1069921138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f59e009db90 con 0x7f59f0074510 2024-09-10T04:42:13.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.598+0000 7f59eeffd640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f59d807abf0 0x7f59d807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.598+0000 7f59eeffd640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f59d807abf0 0x7f59d807d0b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f59f019b270 tx=0x7f59e4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:13.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.602+0000 7f59ecff9640 1 -- 172.21.15.46:0/1069921138 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f59e00692d0 con 0x7f59f0074510 2024-09-10T04:42:13.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.794+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f59f019afb0 con 0x7f59d807abf0 2024-09-10T04:42:13.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.804+0000 7f59ecff9640 1 -- 172.21.15.46:0/1069921138 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f59f019afb0 con 0x7f59d807abf0 2024-09-10T04:42:13.806 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:42:13.806 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (33m) 12m ago 45m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:42:13.806 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (36m) 12m ago 45m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:42:13.806 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (36m) 12m ago 44m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:42:13.806 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (32m) 12m ago 45m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:42:13.806 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (32m) 12m ago 44m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (33m) 12m ago 44m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (28m) 12m ago 39m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (28m) 12m ago 39m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (28m) 12m ago 39m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (12m) 12m ago 39m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (35m) 12m ago 46m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (34m) 12m ago 44m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (32m) 12m ago 46m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (32m) 12m ago 44m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (34m) 12m ago 45m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (34m) 12m ago 44m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (31m) 12m ago 43m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (31m) 12m ago 42m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:42:13.807 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (30m) 12m ago 42m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:42:13.808 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (30m) 12m ago 41m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:42:13.808 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (29m) 12m ago 41m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:42:13.808 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (28m) 12m ago 40m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:42:13.808 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (34m) 12m ago 44m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:42:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f59d807abf0 msgr2=0x7f59d807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f59d807abf0 0x7f59d807d0b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f59f019b270 tx=0x7f59e4031400 comp rx=0 tx=0).stop 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074510 msgr2=0x7f59f019f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074510 0x7f59f019f8c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d8d0 tx=0x7f59e000dd90 comp rx=0 tx=0).stop 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 shutdown_connections 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f59d807abf0 0x7f59d807d0b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59f0074e30 0x7f59f019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 --2- 172.21.15.46:0/1069921138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0074510 0x7f59f019f8c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 >> 172.21.15.46:0/1069921138 conn(0x7f59f00fd610 msgr2=0x7f59f010b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 shutdown_connections 2024-09-10T04:42:13.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.806+0000 7f59f61f2640 1 -- 172.21.15.46:0/1069921138 wait complete. 2024-09-10T04:42:13.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 -- 172.21.15.46:0/713507203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c107900 msgr2=0x7f678c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 --2- 172.21.15.46:0/713507203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c107900 0x7f678c107ce0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6780009880 tx=0x7f678002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 -- 172.21.15.46:0/713507203 shutdown_connections 2024-09-10T04:42:13.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 --2- 172.21.15.46:0/713507203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f678c101900 0x7f678c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 --2- 172.21.15.46:0/713507203 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c107900 0x7f678c107ce0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 -- 172.21.15.46:0/713507203 >> 172.21.15.46:0/713507203 conn(0x7f678c0fd600 msgr2=0x7f678c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:13.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 -- 172.21.15.46:0/713507203 shutdown_connections 2024-09-10T04:42:13.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.889+0000 7f6792ee4640 1 -- 172.21.15.46:0/713507203 wait complete. 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.890+0000 7f6792ee4640 1 Processor -- start 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.890+0000 7f6792ee4640 1 -- start start 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6792ee4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c101900 0x7f678c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6792ee4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f678c107900 0x7f678c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6792ee4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10edb0 con 0x7f678c101900 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6792ee4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f678c10ec30 con 0x7f678c107900 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6790c80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c101900 0x7f678c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6790c80640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c101900 0x7f678c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35900/0 (socket says 172.21.15.46:35900) 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6790c80640 1 -- 172.21.15.46:0/3221400146 learned_addr learned my addr 172.21.15.46:0/3221400146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f678bfff640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f678c107900 0x7f678c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6790c80640 1 -- 172.21.15.46:0/3221400146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f678c107900 msgr2=0x7f678c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6790c80640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f678c107900 0x7f678c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6790c80640 1 -- 172.21.15.46:0/3221400146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c1977c0 con 0x7f678c101900 2024-09-10T04:42:13.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.891+0000 7f6790c80640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c101900 0x7f678c19d640 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6780031460 tx=0x7f6780004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.892+0000 7f6789ffb640 1 -- 172.21.15.46:0/3221400146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f678003f070 con 0x7f678c101900 2024-09-10T04:42:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.892+0000 7f6789ffb640 1 -- 172.21.15.46:0/3221400146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f67800035f0 con 0x7f678c101900 2024-09-10T04:42:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.892+0000 7f6789ffb640 1 -- 172.21.15.46:0/3221400146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6780035030 con 0x7f678c101900 2024-09-10T04:42:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.892+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c197a50 con 0x7f678c101900 2024-09-10T04:42:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.892+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c197f80 con 0x7f678c101900 2024-09-10T04:42:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.893+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f678c103040 con 0x7f678c101900 2024-09-10T04:42:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.894+0000 7f6789ffb640 1 -- 172.21.15.46:0/3221400146 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6780042070 con 0x7f678c101900 2024-09-10T04:42:13.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.894+0000 7f6789ffb640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677c07ac40 0x7f677c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:13.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.894+0000 7f6789ffb640 1 -- 172.21.15.46:0/3221400146 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f67800be4a0 con 0x7f678c101900 2024-09-10T04:42:13.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.894+0000 7f678bfff640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677c07ac40 0x7f677c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.895+0000 7f678bfff640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677c07ac40 0x7f677c07d100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f678c100e00 tx=0x7f6778005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:13.899+0000 7f6789ffb640 1 -- 172.21.15.46:0/3221400146 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6780089c60 con 0x7f678c101900 2024-09-10T04:42:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.158+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f678c104b10 con 0x7f678c101900 2024-09-10T04:42:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.158+0000 7f6789ffb640 1 -- 172.21.15.46:0/3221400146 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f678008eb10 con 0x7f678c101900 2024-09-10T04:42:14.160 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:42:14.160 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:42:14.161 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:42:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677c07ac40 msgr2=0x7f677c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677c07ac40 0x7f677c07d100 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f678c100e00 tx=0x7f6778005f90 comp rx=0 tx=0).stop 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c101900 msgr2=0x7f678c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c101900 0x7f678c19d640 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6780031460 tx=0x7f6780004530 comp rx=0 tx=0).stop 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 shutdown_connections 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f677c07ac40 0x7f677c07d100 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f678c107900 0x7f678c19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 --2- 172.21.15.46:0/3221400146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f678c101900 0x7f678c19d640 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.160+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 >> 172.21.15.46:0/3221400146 conn(0x7f678c0fd600 msgr2=0x7f678c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.161+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 shutdown_connections 2024-09-10T04:42:14.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.161+0000 7f6792ee4640 1 -- 172.21.15.46:0/3221400146 wait complete. 2024-09-10T04:42:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 -- 172.21.15.46:0/206307825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c106800 msgr2=0x7fcf7c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 --2- 172.21.15.46:0/206307825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c106800 0x7fcf7c106be0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009880 tx=0x7fcf7802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 -- 172.21.15.46:0/206307825 shutdown_connections 2024-09-10T04:42:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 --2- 172.21.15.46:0/206307825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf7c100800 0x7fcf7c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 --2- 172.21.15.46:0/206307825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c106800 0x7fcf7c106be0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 -- 172.21.15.46:0/206307825 >> 172.21.15.46:0/206307825 conn(0x7fcf7c0fc520 msgr2=0x7fcf7c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 -- 172.21.15.46:0/206307825 shutdown_connections 2024-09-10T04:42:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.244+0000 7fcf83f36640 1 -- 172.21.15.46:0/206307825 wait complete. 2024-09-10T04:42:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf83f36640 1 Processor -- start 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf83f36640 1 -- start start 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf83f36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c100800 0x7fcf7c0fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf83f36640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf7c106800 0x7fcf7c0fd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf83f36640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c10dcd0 con 0x7fcf7c100800 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf83f36640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c10db50 con 0x7fcf7c106800 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf81cd2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c100800 0x7fcf7c0fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf81cd2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c100800 0x7fcf7c0fcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35910/0 (socket says 172.21.15.46:35910) 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf81cd2640 1 -- 172.21.15.46:0/1626960130 learned_addr learned my addr 172.21.15.46:0/1626960130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.245+0000 7fcf814d1640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf7c106800 0x7fcf7c0fd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf81cd2640 1 -- 172.21.15.46:0/1626960130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf7c106800 msgr2=0x7fcf7c0fd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf81cd2640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf7c106800 0x7fcf7c0fd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf81cd2640 1 -- 172.21.15.46:0/1626960130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c0fd6b0 con 0x7fcf7c100800 2024-09-10T04:42:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf81cd2640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c100800 0x7fcf7c0fcc30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcf78031460 tx=0x7fcf780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf72ffd640 1 -- 172.21.15.46:0/1626960130 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcf7803f070 con 0x7fcf7c100800 2024-09-10T04:42:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf72ffd640 1 -- 172.21.15.46:0/1626960130 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcf78004cf0 con 0x7fcf7c100800 2024-09-10T04:42:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c0fd940 con 0x7fcf7c100800 2024-09-10T04:42:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf72ffd640 1 -- 172.21.15.46:0/1626960130 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcf78035030 con 0x7fcf7c100800 2024-09-10T04:42:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.246+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1a3d80 con 0x7fcf7c100800 2024-09-10T04:42:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.247+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7c101f40 con 0x7fcf7c100800 2024-09-10T04:42:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.248+0000 7fcf72ffd640 1 -- 172.21.15.46:0/1626960130 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcf78042070 con 0x7fcf7c100800 2024-09-10T04:42:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.248+0000 7fcf72ffd640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcf6807abf0 0x7fcf6807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.249+0000 7fcf72ffd640 1 -- 172.21.15.46:0/1626960130 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcf780be740 con 0x7fcf7c100800 2024-09-10T04:42:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.249+0000 7fcf814d1640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcf6807abf0 0x7fcf6807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.249+0000 7fcf814d1640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcf6807abf0 0x7fcf6807d0b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0fe6c0 tx=0x7fcf6c00a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:14.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.253+0000 7fcf72ffd640 1 -- 172.21.15.46:0/1626960130 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fcf78089de0 con 0x7fcf7c100800 2024-09-10T04:42:14.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.487+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fcf7c109d80 con 0x7fcf7c100800 2024-09-10T04:42:14.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.487+0000 7fcf72ffd640 1 -- 172.21.15.46:0/1626960130 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fcf7808ec90 con 0x7fcf7c100800 2024-09-10T04:42:14.502 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:42:14.502 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:42:14.502 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:42:14.502 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:42:14.502 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:42:14.502 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:42:14.503 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:42:14.504 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:14.505 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:14.506 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:42:14.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcf6807abf0 msgr2=0x7fcf6807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcf6807abf0 0x7fcf6807d0b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0fe6c0 tx=0x7fcf6c00a250 comp rx=0 tx=0).stop 2024-09-10T04:42:14.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c100800 msgr2=0x7fcf7c0fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c100800 0x7fcf7c0fcc30 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcf78031460 tx=0x7fcf780048f0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 shutdown_connections 2024-09-10T04:42:14.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcf6807abf0 0x7fcf6807d0b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf7c106800 0x7fcf7c0fd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 --2- 172.21.15.46:0/1626960130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf7c100800 0x7fcf7c0fcc30 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 >> 172.21.15.46:0/1626960130 conn(0x7fcf7c0fc520 msgr2=0x7fcf7c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 shutdown_connections 2024-09-10T04:42:14.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.490+0000 7fcf83f36640 1 -- 172.21.15.46:0/1626960130 wait complete. 2024-09-10T04:42:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 -- 172.21.15.46:0/3006477081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc102b30 msgr2=0x7f5ccc102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 --2- 172.21.15.46:0/3006477081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc102b30 0x7f5ccc102f10 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009c20 tx=0x7f5cbc02aa00 comp rx=0 tx=0).stop 2024-09-10T04:42:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 -- 172.21.15.46:0/3006477081 shutdown_connections 2024-09-10T04:42:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 --2- 172.21.15.46:0/3006477081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ccc103450 0x7f5ccc10b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 --2- 172.21.15.46:0/3006477081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc102b30 0x7f5ccc102f10 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 -- 172.21.15.46:0/3006477081 >> 172.21.15.46:0/3006477081 conn(0x7f5ccc0fc550 msgr2=0x7f5ccc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 -- 172.21.15.46:0/3006477081 shutdown_connections 2024-09-10T04:42:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.575+0000 7f5cd1541640 1 -- 172.21.15.46:0/3006477081 wait complete. 2024-09-10T04:42:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.576+0000 7f5cd1541640 1 Processor -- start 2024-09-10T04:42:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.576+0000 7f5cd1541640 1 -- start start 2024-09-10T04:42:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.576+0000 7f5cd1541640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ccc102b30 0x7f5ccc19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cd1541640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc103450 0x7f5ccc19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cd1541640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10edc0 con 0x7f5ccc103450 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cd1541640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc10ec40 con 0x7f5ccc102b30 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cca7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc103450 0x7f5ccc19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cca7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc103450 0x7f5ccc19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35922/0 (socket says 172.21.15.46:35922) 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cca7fc640 1 -- 172.21.15.46:0/987198543 learned_addr learned my addr 172.21.15.46:0/987198543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5ccaffd640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ccc102b30 0x7f5ccc19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cca7fc640 1 -- 172.21.15.46:0/987198543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ccc102b30 msgr2=0x7f5ccc19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cca7fc640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ccc102b30 0x7f5ccc19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cca7fc640 1 -- 172.21.15.46:0/987198543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc1977b0 con 0x7f5ccc103450 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cca7fc640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc103450 0x7f5ccc19db70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c810 tx=0x7f5cc000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cb3fff640 1 -- 172.21.15.46:0/987198543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5cc0004020 con 0x7f5ccc103450 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cb3fff640 1 -- 172.21.15.46:0/987198543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5cc00041c0 con 0x7f5ccc103450 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.577+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc197aa0 con 0x7f5ccc103450 2024-09-10T04:42:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.578+0000 7f5cb3fff640 1 -- 172.21.15.46:0/987198543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5cc00047a0 con 0x7f5ccc103450 2024-09-10T04:42:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.578+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc198030 con 0x7f5ccc103450 2024-09-10T04:42:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.579+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc06a500 con 0x7f5ccc103450 2024-09-10T04:42:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.579+0000 7f5cb3fff640 1 -- 172.21.15.46:0/987198543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5cc00079b0 con 0x7f5ccc103450 2024-09-10T04:42:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.580+0000 7f5cb3fff640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5cb407abf0 0x7f5cb407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.580+0000 7f5cb3fff640 1 -- 172.21.15.46:0/987198543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5cc009db20 con 0x7f5ccc103450 2024-09-10T04:42:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.580+0000 7f5ccaffd640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5cb407abf0 0x7f5cb407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.580+0000 7f5ccaffd640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5cb407abf0 0x7f5cb407d0b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009430 tx=0x7f5cbc03a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.584+0000 7f5cb3fff640 1 -- 172.21.15.46:0/987198543 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5cc0014030 con 0x7f5ccc103450 2024-09-10T04:42:14.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.789+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5ccc198340 con 0x7f5cb407abf0 2024-09-10T04:42:14.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.790+0000 7f5cb3fff640 1 -- 172.21.15.46:0/987198543 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f5ccc198340 con 0x7f5cb407abf0 2024-09-10T04:42:14.792 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:42:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:42:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:42:14.792 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:42:14.793 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5cb407abf0 msgr2=0x7f5cb407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5cb407abf0 0x7f5cb407d0b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009430 tx=0x7f5cbc03a040 comp rx=0 tx=0).stop 2024-09-10T04:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc103450 msgr2=0x7f5ccc19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc103450 0x7f5ccc19db70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000c810 tx=0x7f5cc000ccd0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 shutdown_connections 2024-09-10T04:42:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5cb407abf0 0x7f5cb407d0b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ccc103450 0x7f5ccc19db70 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 --2- 172.21.15.46:0/987198543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ccc102b30 0x7f5ccc19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.792+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 >> 172.21.15.46:0/987198543 conn(0x7f5ccc0fc550 msgr2=0x7f5ccc108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.793+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 shutdown_connections 2024-09-10T04:42:14.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.793+0000 7f5cd1541640 1 -- 172.21.15.46:0/987198543 wait complete. 2024-09-10T04:42:14.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 -- 172.21.15.46:0/241848291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358102b20 msgr2=0x7f9358102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 --2- 172.21.15.46:0/241848291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358102b20 0x7f9358102f00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f93480098b0 tx=0x7f934802a560 comp rx=0 tx=0).stop 2024-09-10T04:42:14.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 -- 172.21.15.46:0/241848291 shutdown_connections 2024-09-10T04:42:14.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 --2- 172.21.15.46:0/241848291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9358103440 0x7f935810b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 --2- 172.21.15.46:0/241848291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358102b20 0x7f9358102f00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 -- 172.21.15.46:0/241848291 >> 172.21.15.46:0/241848291 conn(0x7f93580fc520 msgr2=0x7f93580fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 -- 172.21.15.46:0/241848291 shutdown_connections 2024-09-10T04:42:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 -- 172.21.15.46:0/241848291 wait complete. 2024-09-10T04:42:14.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.877+0000 7f935dbde640 1 Processor -- start 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f935dbde640 1 -- start start 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f935dbde640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9358102b20 0x7f935819d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f935dbde640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358103440 0x7f935819dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f935dbde640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f935810ee50 con 0x7f9358103440 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f935dbde640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f935810ecd0 con 0x7f9358102b20 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f9356ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358103440 0x7f935819dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f9356ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358103440 0x7f935819dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35942/0 (socket says 172.21.15.46:35942) 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f9356ffd640 1 -- 172.21.15.46:0/44031942 learned_addr learned my addr 172.21.15.46:0/44031942 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f93577fe640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9358102b20 0x7f935819d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f9356ffd640 1 -- 172.21.15.46:0/44031942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9358102b20 msgr2=0x7f935819d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:14.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f9356ffd640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9358102b20 0x7f935819d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.878+0000 7f9356ffd640 1 -- 172.21.15.46:0/44031942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93581978a0 con 0x7f9358103440 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.879+0000 7f9356ffd640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358103440 0x7f935819dc20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f934c00ba50 tx=0x7f934c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.879+0000 7f9354ff9640 1 -- 172.21.15.46:0/44031942 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f934c019070 con 0x7f9358103440 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.879+0000 7f9354ff9640 1 -- 172.21.15.46:0/44031942 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f934c00ce20 con 0x7f9358103440 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.879+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9358197b90 con 0x7f9358103440 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.879+0000 7f9354ff9640 1 -- 172.21.15.46:0/44031942 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f934c0075a0 con 0x7f9358103440 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.879+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9358198120 con 0x7f9358103440 2024-09-10T04:42:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.880+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935806aa30 con 0x7f9358103440 2024-09-10T04:42:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.881+0000 7f9354ff9640 1 -- 172.21.15.46:0/44031942 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f934c01c070 con 0x7f9358103440 2024-09-10T04:42:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.881+0000 7f9354ff9640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f934007ac40 0x7f934007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.881+0000 7f9354ff9640 1 -- 172.21.15.46:0/44031942 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f934c09d4e0 con 0x7f9358103440 2024-09-10T04:42:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.881+0000 7f93577fe640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f934007ac40 0x7f934007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.882+0000 7f93577fe640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f934007ac40 0x7f934007d100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9348009230 tx=0x7f9348031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:14.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:14.886+0000 7f9354ff9640 1 -- 172.21.15.46:0/44031942 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f934c068c00 con 0x7f9358103440 2024-09-10T04:42:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:14 smithi070 ceph-mon[79774]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:14 smithi070 ceph-mon[79774]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:14 smithi070 ceph-mon[79774]: pgmap v1098: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:14 smithi070 ceph-mon[79774]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:14 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3221400146' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:42:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:14 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1626960130' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:42:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:14 smithi046 ceph-mon[109506]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:14 smithi046 ceph-mon[109506]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:14 smithi046 ceph-mon[109506]: pgmap v1098: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:14 smithi046 ceph-mon[109506]: from='client.35830 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:14 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3221400146' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:42:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:14 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1626960130' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:42:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.145+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f9358105f80 con 0x7f9358103440 2024-09-10T04:42:15.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.145+0000 7f9354ff9640 1 -- 172.21.15.46:0/44031942 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f934c06dab0 con 0x7f9358103440 2024-09-10T04:42:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:42:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:42:15.147 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:42:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:42:15.147 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:42:15.148 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:42:15.149 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:42:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:42:15.149 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:42:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:42:15.149 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:42:15.149 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f934007ac40 msgr2=0x7f934007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f934007ac40 0x7f934007d100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9348009230 tx=0x7f9348031400 comp rx=0 tx=0).stop 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358103440 msgr2=0x7f935819dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358103440 0x7f935819dc20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f934c00ba50 tx=0x7f934c00bf10 comp rx=0 tx=0).stop 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 shutdown_connections 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f934007ac40 0x7f934007d100 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9358103440 0x7f935819dc20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 --2- 172.21.15.46:0/44031942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9358102b20 0x7f935819d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 >> 172.21.15.46:0/44031942 conn(0x7f93580fc520 msgr2=0x7f9358108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 shutdown_connections 2024-09-10T04:42:15.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:15.147+0000 7f935dbde640 1 -- 172.21.15.46:0/44031942 wait complete. 2024-09-10T04:42:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:15 smithi070 ceph-mon[79774]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:16.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:15 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/44031942' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:42:16.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:15 smithi046 ceph-mon[109506]: from='client.35842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:16.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:15 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/44031942' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:42:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:16 smithi070 ceph-mon[79774]: pgmap v1099: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:16 smithi046 ceph-mon[109506]: pgmap v1099: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:19.078 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:18 smithi070 ceph-mon[79774]: pgmap v1100: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:18 smithi046 ceph-mon[109506]: pgmap v1100: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:20 smithi070 ceph-mon[79774]: pgmap v1101: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:20 smithi046 ceph-mon[109506]: pgmap v1101: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:22 smithi070 ceph-mon[79774]: pgmap v1102: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:22 smithi046 ceph-mon[109506]: pgmap v1102: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:24.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:24 smithi070 ceph-mon[79774]: pgmap v1103: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:24 smithi046 ceph-mon[109506]: pgmap v1103: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:26 smithi070 ceph-mon[79774]: pgmap v1104: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:26 smithi046 ceph-mon[109506]: pgmap v1104: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:28 smithi070 ceph-mon[79774]: pgmap v1105: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:28 smithi046 ceph-mon[109506]: pgmap v1105: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:30 smithi070 ceph-mon[79774]: pgmap v1106: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:30 smithi046 ceph-mon[109506]: pgmap v1106: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:32 smithi070 ceph-mon[79774]: pgmap v1107: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:32 smithi046 ceph-mon[109506]: pgmap v1107: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:34 smithi070 ceph-mon[79774]: pgmap v1108: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:34 smithi046 ceph-mon[109506]: pgmap v1108: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:37.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:36 smithi070 ceph-mon[79774]: pgmap v1109: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:37.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:36 smithi046 ceph-mon[109506]: pgmap v1109: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:38 smithi070 ceph-mon[79774]: pgmap v1110: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:38 smithi046 ceph-mon[109506]: pgmap v1110: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:40 smithi070 ceph-mon[79774]: pgmap v1111: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:40 smithi046 ceph-mon[109506]: pgmap v1111: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:42 smithi070 ceph-mon[79774]: pgmap v1112: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:43.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:42 smithi046 ceph-mon[109506]: pgmap v1112: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:44 smithi070 ceph-mon[79774]: pgmap v1113: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:44 smithi046 ceph-mon[109506]: pgmap v1113: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1493445590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 msgr2=0x7f7ed0102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1493445590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed0102ea0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009880 tx=0x7f7ec002a530 comp rx=0 tx=0).stop 2024-09-10T04:42:45.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1493445590 shutdown_connections 2024-09-10T04:42:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1493445590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed01033e0 0x7f7ed010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1493445590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed0102ea0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1493445590 >> 172.21.15.46:0/1493445590 conn(0x7f7ed00fc520 msgr2=0x7f7ed00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1493445590 shutdown_connections 2024-09-10T04:42:45.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.245+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1493445590 wait complete. 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.246+0000 7f7ed4e63640 1 Processor -- start 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.246+0000 7f7ed4e63640 1 -- start start 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ed4e63640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ed4e63640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed01033e0 0x7f7ed019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ed4e63640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010edf0 con 0x7f7ed0102ac0 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ed4e63640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed010ec70 con 0x7f7ed01033e0 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ece59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ece59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed019d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41204/0 (socket says 172.21.15.46:41204) 2024-09-10T04:42:45.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ece59c640 1 -- 172.21.15.46:0/1886825662 learned_addr learned my addr 172.21.15.46:0/1886825662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ecdd9b640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed01033e0 0x7f7ed019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ece59c640 1 -- 172.21.15.46:0/1886825662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed01033e0 msgr2=0x7f7ed019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ece59c640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed01033e0 0x7f7ed019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.247+0000 7f7ece59c640 1 -- 172.21.15.46:0/1886825662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed01977c0 con 0x7f7ed0102ac0 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.248+0000 7f7ece59c640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed019d640 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0031460 tx=0x7f7ec00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.248+0000 7f7ebf7fe640 1 -- 172.21.15.46:0/1886825662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ec003f070 con 0x7f7ed0102ac0 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.248+0000 7f7ebf7fe640 1 -- 172.21.15.46:0/1886825662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7ec00035f0 con 0x7f7ed0102ac0 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.248+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed0197a50 con 0x7f7ed0102ac0 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.248+0000 7f7ebf7fe640 1 -- 172.21.15.46:0/1886825662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7ec0035030 con 0x7f7ed0102ac0 2024-09-10T04:42:45.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.248+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed0197f80 con 0x7f7ed0102ac0 2024-09-10T04:42:45.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.249+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed006aa90 con 0x7f7ed0102ac0 2024-09-10T04:42:45.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.250+0000 7f7ebf7fe640 1 -- 172.21.15.46:0/1886825662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7ec0042070 con 0x7f7ed0102ac0 2024-09-10T04:42:45.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.250+0000 7f7ebf7fe640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7eb807abf0 0x7f7eb807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.251+0000 7f7ebf7fe640 1 -- 172.21.15.46:0/1886825662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7ec00be670 con 0x7f7ed0102ac0 2024-09-10T04:42:45.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.251+0000 7f7ecdd9b640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7eb807abf0 0x7f7eb807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.251+0000 7f7ecdd9b640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7eb807abf0 0x7f7eb807d0b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0108900 tx=0x7f7ec400b2e0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:45.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.255+0000 7f7ebf7fe640 1 -- 172.21.15.46:0/1886825662 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7ec0089d10 con 0x7f7ed0102ac0 2024-09-10T04:42:45.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.461+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7ed01982f0 con 0x7f7eb807abf0 2024-09-10T04:42:45.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.462+0000 7f7ebf7fe640 1 -- 172.21.15.46:0/1886825662 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7ed01982f0 con 0x7f7eb807abf0 2024-09-10T04:42:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7eb807abf0 msgr2=0x7f7eb807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7eb807abf0 0x7f7eb807d0b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0108900 tx=0x7f7ec400b2e0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 msgr2=0x7f7ed019d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed019d640 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0031460 tx=0x7f7ec00045e0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 shutdown_connections 2024-09-10T04:42:45.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7eb807abf0 0x7f7eb807d0b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed01033e0 0x7f7ed019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 --2- 172.21.15.46:0/1886825662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ed0102ac0 0x7f7ed019d640 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 >> 172.21.15.46:0/1886825662 conn(0x7f7ed00fc520 msgr2=0x7f7ed010af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 shutdown_connections 2024-09-10T04:42:45.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.464+0000 7f7ed4e63640 1 -- 172.21.15.46:0/1886825662 wait complete. 2024-09-10T04:42:45.475 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:42:45.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 -- 172.21.15.46:0/2857445368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 msgr2=0x7f4b9810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/2857445368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9810b8d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009880 tx=0x7f4b8c02a530 comp rx=0 tx=0).stop 2024-09-10T04:42:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 -- 172.21.15.46:0/2857445368 shutdown_connections 2024-09-10T04:42:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/2857445368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9810b8d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/2857445368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98102ac0 0x7f4b98102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 -- 172.21.15.46:0/2857445368 >> 172.21.15.46:0/2857445368 conn(0x7f4b980fc520 msgr2=0x7f4b980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 -- 172.21.15.46:0/2857445368 shutdown_connections 2024-09-10T04:42:45.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.550+0000 7f4b9dc13640 1 -- 172.21.15.46:0/2857445368 wait complete. 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.551+0000 7f4b9dc13640 1 Processor -- start 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.551+0000 7f4b9dc13640 1 -- start start 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.551+0000 7f4b9dc13640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98102ac0 0x7f4b9819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.551+0000 7f4b9dc13640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.551+0000 7f4b9dc13640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810edd0 con 0x7f4b981033e0 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.551+0000 7f4b9dc13640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810ec50 con 0x7f4b98102ac0 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b96ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b96ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9819dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41222/0 (socket says 172.21.15.46:41222) 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b96ffd640 1 -- 172.21.15.46:0/3453723775 learned_addr learned my addr 172.21.15.46:0/3453723775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:45.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b977fe640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98102ac0 0x7f4b9819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b96ffd640 1 -- 172.21.15.46:0/3453723775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98102ac0 msgr2=0x7f4b9819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b96ffd640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98102ac0 0x7f4b9819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b96ffd640 1 -- 172.21.15.46:0/3453723775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b981977e0 con 0x7f4b981033e0 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b96ffd640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9819dba0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c031460 tx=0x7f4b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b94ff9640 1 -- 172.21.15.46:0/3453723775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b8c03f070 con 0x7f4b981033e0 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b98197a70 con 0x7f4b981033e0 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b94ff9640 1 -- 172.21.15.46:0/3453723775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b8c004ac0 con 0x7f4b981033e0 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.553+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b98197fa0 con 0x7f4b981033e0 2024-09-10T04:42:45.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.552+0000 7f4b94ff9640 1 -- 172.21.15.46:0/3453723775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b8c035030 con 0x7f4b981033e0 2024-09-10T04:42:45.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.554+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9806aa90 con 0x7f4b981033e0 2024-09-10T04:42:45.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.554+0000 7f4b94ff9640 1 -- 172.21.15.46:0/3453723775 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4b8c042070 con 0x7f4b981033e0 2024-09-10T04:42:45.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.555+0000 7f4b94ff9640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007abf0 0x7f4b8007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.555+0000 7f4b94ff9640 1 -- 172.21.15.46:0/3453723775 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4b8c0be690 con 0x7f4b981033e0 2024-09-10T04:42:45.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.555+0000 7f4b977fe640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007abf0 0x7f4b8007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.555+0000 7f4b977fe640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007abf0 0x7f4b8007d0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4b880058c0 tx=0x7f4b8800a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:45.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.560+0000 7f4b94ff9640 1 -- 172.21.15.46:0/3453723775 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4b8c089d70 con 0x7f4b981033e0 2024-09-10T04:42:45.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.764+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f4b98198250 con 0x7f4b8007abf0 2024-09-10T04:42:45.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.765+0000 7f4b94ff9640 1 -- 172.21.15.46:0/3453723775 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f4b98198250 con 0x7f4b8007abf0 2024-09-10T04:42:45.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007abf0 msgr2=0x7f4b8007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007abf0 0x7f4b8007d0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4b880058c0 tx=0x7f4b8800a2d0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 msgr2=0x7f4b9819dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9819dba0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c031460 tx=0x7f4b8c0046c0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 shutdown_connections 2024-09-10T04:42:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007abf0 0x7f4b8007d0b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b981033e0 0x7f4b9819dba0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 --2- 172.21.15.46:0/3453723775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98102ac0 0x7f4b9819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 >> 172.21.15.46:0/3453723775 conn(0x7f4b980fc520 msgr2=0x7f4b98108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:45.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.767+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 shutdown_connections 2024-09-10T04:42:45.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.768+0000 7f4b9dc13640 1 -- 172.21.15.46:0/3453723775 wait complete. 2024-09-10T04:42:45.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 -- 172.21.15.46:0/1216061633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81007f0 msgr2=0x7f3ed8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/1216061633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81007f0 0x7f3ed8100c50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009880 tx=0x7f3ecc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 -- 172.21.15.46:0/1216061633 shutdown_connections 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/1216061633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81007f0 0x7f3ed8100c50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/1216061633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ed81067f0 0x7f3ed8106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 -- 172.21.15.46:0/1216061633 >> 172.21.15.46:0/1216061633 conn(0x7f3ed80fc550 msgr2=0x7f3ed80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 -- 172.21.15.46:0/1216061633 shutdown_connections 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 -- 172.21.15.46:0/1216061633 wait complete. 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.853+0000 7f3eddc6d640 1 Processor -- start 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3eddc6d640 1 -- start start 2024-09-10T04:42:45.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3eddc6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ed81007f0 0x7f3ed80fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3eddc6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81067f0 0x7f3ed80fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3eddc6d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810dcc0 con 0x7f3ed81067f0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3eddc6d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed810db40 con 0x7f3ed81007f0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81067f0 0x7f3ed80fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81067f0 0x7f3ed80fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41254/0 (socket says 172.21.15.46:41254) 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed6ffd640 1 -- 172.21.15.46:0/2743624836 learned_addr learned my addr 172.21.15.46:0/2743624836 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed77fe640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ed81007f0 0x7f3ed80fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed6ffd640 1 -- 172.21.15.46:0/2743624836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ed81007f0 msgr2=0x7f3ed80fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed6ffd640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ed81007f0 0x7f3ed80fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed6ffd640 1 -- 172.21.15.46:0/2743624836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed80fd750 con 0x7f3ed81067f0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.854+0000 7f3ed6ffd640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81067f0 0x7f3ed80fd210 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc031460 tx=0x7f3ecc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.855+0000 7f3ed4ff9640 1 -- 172.21.15.46:0/2743624836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3ecc03f070 con 0x7f3ed81067f0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.855+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed80fd9e0 con 0x7f3ed81067f0 2024-09-10T04:42:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.855+0000 7f3ed4ff9640 1 -- 172.21.15.46:0/2743624836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3ecc0035f0 con 0x7f3ed81067f0 2024-09-10T04:42:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.855+0000 7f3ed4ff9640 1 -- 172.21.15.46:0/2743624836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3ecc035030 con 0x7f3ed81067f0 2024-09-10T04:42:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.855+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed81a3da0 con 0x7f3ed81067f0 2024-09-10T04:42:45.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.856+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed8101f30 con 0x7f3ed81067f0 2024-09-10T04:42:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.856+0000 7f3ed4ff9640 1 -- 172.21.15.46:0/2743624836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3ecc042070 con 0x7f3ed81067f0 2024-09-10T04:42:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.857+0000 7f3ed4ff9640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ec007abf0 0x7f3ec007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.857+0000 7f3ed4ff9640 1 -- 172.21.15.46:0/2743624836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f3ecc0be590 con 0x7f3ed81067f0 2024-09-10T04:42:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.857+0000 7f3ed77fe640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ec007abf0 0x7f3ec007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:45.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.857+0000 7f3ed77fe640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ec007abf0 0x7f3ec007d0b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005fd0 tx=0x7f3ec800b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:45.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:45.861+0000 7f3ed4ff9640 1 -- 172.21.15.46:0/2743624836 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3ecc089cd0 con 0x7f3ed81067f0 2024-09-10T04:42:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.056+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f3ed8062630 con 0x7f3ec007abf0 2024-09-10T04:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.066+0000 7f3ed4ff9640 1 -- 172.21.15.46:0/2743624836 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f3ed8062630 con 0x7f3ec007abf0 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (34m) 13m ago 45m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (36m) 13m ago 45m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (36m) 13m ago 45m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (32m) 13m ago 45m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (32m) 13m ago 44m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (33m) 13m ago 45m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (28m) 13m ago 40m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (29m) 13m ago 40m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (28m) 13m ago 40m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (13m) 13m ago 40m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (36m) 13m ago 46m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:42:46.068 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (35m) 13m ago 44m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (33m) 13m ago 47m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (32m) 13m ago 44m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (35m) 13m ago 45m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (35m) 13m ago 44m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (32m) 13m ago 43m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (31m) 13m ago 43m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (31m) 13m ago 42m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (30m) 13m ago 42m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (30m) 13m ago 41m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (29m) 13m ago 41m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:42:46.069 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (34m) 13m ago 45m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ec007abf0 msgr2=0x7f3ec007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ec007abf0 0x7f3ec007d0b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8005fd0 tx=0x7f3ec800b390 comp rx=0 tx=0).stop 2024-09-10T04:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81067f0 msgr2=0x7f3ed80fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81067f0 0x7f3ed80fd210 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc031460 tx=0x7f3ecc0045e0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 shutdown_connections 2024-09-10T04:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f3ec007abf0 0x7f3ec007d0b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ed81067f0 0x7f3ed80fd210 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 --2- 172.21.15.46:0/2743624836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ed81007f0 0x7f3ed80fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 >> 172.21.15.46:0/2743624836 conn(0x7f3ed80fc550 msgr2=0x7f3ed8069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 shutdown_connections 2024-09-10T04:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.068+0000 7f3eddc6d640 1 -- 172.21.15.46:0/2743624836 wait complete. 2024-09-10T04:42:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 -- 172.21.15.46:0/3266220142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 msgr2=0x7f03ac101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3266220142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac101d60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0394009880 tx=0x7f039402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 -- 172.21.15.46:0/3266220142 shutdown_connections 2024-09-10T04:42:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3266220142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac101d60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3266220142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac107900 0x7f03ac107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 -- 172.21.15.46:0/3266220142 >> 172.21.15.46:0/3266220142 conn(0x7f03ac0fd600 msgr2=0x7f03ac0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 -- 172.21.15.46:0/3266220142 shutdown_connections 2024-09-10T04:42:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.151+0000 7f03b2d26640 1 -- 172.21.15.46:0/3266220142 wait complete. 2024-09-10T04:42:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b2d26640 1 Processor -- start 2024-09-10T04:42:46.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b2d26640 1 -- start start 2024-09-10T04:42:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b2d26640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b2d26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac107900 0x7f03ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b2d26640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10ee30 con 0x7f03ac101900 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b2d26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10ecb0 con 0x7f03ac107900 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b0ac2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b0ac2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41258/0 (socket says 172.21.15.46:41258) 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b0ac2640 1 -- 172.21.15.46:0/3053157713 learned_addr learned my addr 172.21.15.46:0/3053157713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03abfff640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac107900 0x7f03ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b0ac2640 1 -- 172.21.15.46:0/3053157713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac107900 msgr2=0x7f03ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b0ac2640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac107900 0x7f03ac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.152+0000 7f03b0ac2640 1 -- 172.21.15.46:0/3053157713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac074f70 con 0x7f03ac101900 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.153+0000 7f03b0ac2640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac0744f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b420 tx=0x7f039c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.153+0000 7f03a9ffb640 1 -- 172.21.15.46:0/3053157713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f039c011630 con 0x7f03ac101900 2024-09-10T04:42:46.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.153+0000 7f03a9ffb640 1 -- 172.21.15.46:0/3053157713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f039c016070 con 0x7f03ac101900 2024-09-10T04:42:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.153+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac075260 con 0x7f03ac101900 2024-09-10T04:42:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.153+0000 7f03a9ffb640 1 -- 172.21.15.46:0/3053157713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f039c010d40 con 0x7f03ac101900 2024-09-10T04:42:46.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.153+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1a3ec0 con 0x7f03ac101900 2024-09-10T04:42:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.154+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac103040 con 0x7f03ac101900 2024-09-10T04:42:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.155+0000 7f03a9ffb640 1 -- 172.21.15.46:0/3053157713 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f039c01b460 con 0x7f03ac101900 2024-09-10T04:42:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.155+0000 7f03a9ffb640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f039807abf0 0x7f039807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.155+0000 7f03a9ffb640 1 -- 172.21.15.46:0/3053157713 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f039c09d450 con 0x7f03ac101900 2024-09-10T04:42:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.155+0000 7f03abfff640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f039807abf0 0x7f039807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.156+0000 7f03abfff640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f039807abf0 0x7f039807d0b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f03ac075f80 tx=0x7f0394031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:46.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.161+0000 7f03a9ffb640 1 -- 172.21.15.46:0/3053157713 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f039c021250 con 0x7f03ac101900 2024-09-10T04:42:46.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.420+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f03ac109e90 con 0x7f03ac101900 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.420+0000 7f03a9ffb640 1 -- 172.21.15.46:0/3053157713 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f039c068b90 con 0x7f03ac101900 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:46.423 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:42:46.424 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f039807abf0 msgr2=0x7f039807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f039807abf0 0x7f039807d0b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f03ac075f80 tx=0x7f0394031400 comp rx=0 tx=0).stop 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 msgr2=0x7f03ac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac0744f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f039c00b420 tx=0x7f039c00b8e0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 shutdown_connections 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f039807abf0 0x7f039807d0b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03ac107900 0x7f03ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 --2- 172.21.15.46:0/3053157713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03ac101900 0x7f03ac0744f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.422+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 >> 172.21.15.46:0/3053157713 conn(0x7f03ac0fd600 msgr2=0x7f03ac0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.423+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 shutdown_connections 2024-09-10T04:42:46.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.423+0000 7f03b2d26640 1 -- 172.21.15.46:0/3053157713 wait complete. 2024-09-10T04:42:46.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 -- 172.21.15.46:0/3168144693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 msgr2=0x7f734c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 --2- 172.21.15.46:0/3168144693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c100a30 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7348009880 tx=0x7f734802a490 comp rx=0 tx=0).stop 2024-09-10T04:42:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 -- 172.21.15.46:0/3168144693 shutdown_connections 2024-09-10T04:42:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 --2- 172.21.15.46:0/3168144693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c101000 0x7f734c10dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 --2- 172.21.15.46:0/3168144693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c100a30 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 -- 172.21.15.46:0/3168144693 >> 172.21.15.46:0/3168144693 conn(0x7f734c0fc520 msgr2=0x7f734c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 -- 172.21.15.46:0/3168144693 shutdown_connections 2024-09-10T04:42:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.507+0000 7f7353ed6640 1 -- 172.21.15.46:0/3168144693 wait complete. 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7353ed6640 1 Processor -- start 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7353ed6640 1 -- start start 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7353ed6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7353ed6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c101000 0x7f734c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7353ed6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f734c110fb0 con 0x7f734c100650 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7353ed6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f734c110e30 con 0x7f734c101000 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7351c72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7351c72640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c19f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41264/0 (socket says 172.21.15.46:41264) 2024-09-10T04:42:46.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.508+0000 7f7351c72640 1 -- 172.21.15.46:0/436134794 learned_addr learned my addr 172.21.15.46:0/436134794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f7351471640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c101000 0x7f734c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f7351c72640 1 -- 172.21.15.46:0/436134794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c101000 msgr2=0x7f734c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f7351c72640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c101000 0x7f734c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f7351c72640 1 -- 172.21.15.46:0/436134794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c199a40 con 0x7f734c100650 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f7351c72640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c19f880 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7348031460 tx=0x7f7348004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f733effd640 1 -- 172.21.15.46:0/436134794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f734803f070 con 0x7f734c100650 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f733effd640 1 -- 172.21.15.46:0/436134794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f73480035f0 con 0x7f734c100650 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c199cd0 con 0x7f734c100650 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f733effd640 1 -- 172.21.15.46:0/436134794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7348035030 con 0x7f734c100650 2024-09-10T04:42:46.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.509+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c19a260 con 0x7f734c100650 2024-09-10T04:42:46.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.510+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c06aa90 con 0x7f734c100650 2024-09-10T04:42:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.511+0000 7f733effd640 1 -- 172.21.15.46:0/436134794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7348042070 con 0x7f734c100650 2024-09-10T04:42:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.512+0000 7f733effd640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f733807ac40 0x7f733807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.512+0000 7f7351471640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f733807ac40 0x7f733807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.512+0000 7f733effd640 1 -- 172.21.15.46:0/436134794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f73480be5f0 con 0x7f734c100650 2024-09-10T04:42:46.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.512+0000 7f7351471640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f733807ac40 0x7f733807d100 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f734c102520 tx=0x7f7340005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:46.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.516+0000 7f733effd640 1 -- 172.21.15.46:0/436134794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7348089d10 con 0x7f734c100650 2024-09-10T04:42:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.741+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f734c19af70 con 0x7f734c100650 2024-09-10T04:42:46.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.741+0000 7f733effd640 1 -- 172.21.15.46:0/436134794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f734808ebc0 con 0x7f734c100650 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:42:46.744 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:42:46.745 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:42:46.746 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:46.747 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f733807ac40 msgr2=0x7f733807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f733807ac40 0x7f733807d100 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f734c102520 tx=0x7f7340005f90 comp rx=0 tx=0).stop 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 msgr2=0x7f734c19f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c19f880 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7348031460 tx=0x7f7348004530 comp rx=0 tx=0).stop 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 shutdown_connections 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f733807ac40 0x7f733807d100 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f734c101000 0x7f734c19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 --2- 172.21.15.46:0/436134794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f734c100650 0x7f734c19f880 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 >> 172.21.15.46:0/436134794 conn(0x7f734c0fc520 msgr2=0x7f734c0fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:46.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 shutdown_connections 2024-09-10T04:42:46.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.744+0000 7f7353ed6640 1 -- 172.21.15.46:0/436134794 wait complete. 2024-09-10T04:42:46.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 -- 172.21.15.46:0/2013503488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984101000 msgr2=0x7f798410dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2013503488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984101000 0x7f798410dcc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7974009880 tx=0x7f797402a490 comp rx=0 tx=0).stop 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 -- 172.21.15.46:0/2013503488 shutdown_connections 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2013503488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984101000 0x7f798410dcc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2013503488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7984100650 0x7f7984100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 -- 172.21.15.46:0/2013503488 >> 172.21.15.46:0/2013503488 conn(0x7f79840fc520 msgr2=0x7f79840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 -- 172.21.15.46:0/2013503488 shutdown_connections 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.829+0000 7f798b7ab640 1 -- 172.21.15.46:0/2013503488 wait complete. 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f798b7ab640 1 Processor -- start 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f798b7ab640 1 -- start start 2024-09-10T04:42:46.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f798b7ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984100650 0x7f798419f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f798b7ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7984101000 0x7f798419fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f798b7ab640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110ff0 con 0x7f7984100650 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f798b7ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7984110e70 con 0x7f7984101000 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f7989547640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984100650 0x7f798419f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f7989547640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984100650 0x7f798419f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41284/0 (socket says 172.21.15.46:41284) 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f7989547640 1 -- 172.21.15.46:0/2926559578 learned_addr learned my addr 172.21.15.46:0/2926559578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.830+0000 7f7988d46640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7984101000 0x7f798419fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f7989547640 1 -- 172.21.15.46:0/2926559578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7984101000 msgr2=0x7f798419fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f7989547640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7984101000 0x7f798419fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f7989547640 1 -- 172.21.15.46:0/2926559578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7984199a80 con 0x7f7984100650 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f7988d46640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7984101000 0x7f798419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f7989547640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984100650 0x7f798419f8c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f798000b420 tx=0x7f798000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:46.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f797a7fc640 1 -- 172.21.15.46:0/2926559578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7980011630 con 0x7f7984100650 2024-09-10T04:42:46.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f797a7fc640 1 -- 172.21.15.46:0/2926559578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7980016070 con 0x7f7984100650 2024-09-10T04:42:46.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7984199d70 con 0x7f7984100650 2024-09-10T04:42:46.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f797a7fc640 1 -- 172.21.15.46:0/2926559578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7980010d40 con 0x7f7984100650 2024-09-10T04:42:46.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.831+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798419a2d0 con 0x7f7984100650 2024-09-10T04:42:46.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.832+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798406aa90 con 0x7f7984100650 2024-09-10T04:42:46.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.833+0000 7f797a7fc640 1 -- 172.21.15.46:0/2926559578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f798001b460 con 0x7f7984100650 2024-09-10T04:42:46.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.833+0000 7f797a7fc640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f797007ac40 0x7f797007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:46.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.833+0000 7f797a7fc640 1 -- 172.21.15.46:0/2926559578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f798009d370 con 0x7f7984100650 2024-09-10T04:42:46.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.833+0000 7f7988d46640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f797007ac40 0x7f797007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:46.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.834+0000 7f7988d46640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f797007ac40 0x7f797007d100 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f798419b270 tx=0x7f7974031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:46.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:46.838+0000 7f797a7fc640 1 -- 172.21.15.46:0/2926559578 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7980021250 con 0x7f7984100650 2024-09-10T04:42:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.040+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f798419afb0 con 0x7f797007ac40 2024-09-10T04:42:47.043 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:46 smithi046 ceph-mon[109506]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:47.043 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:46 smithi046 ceph-mon[109506]: pgmap v1114: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:47.043 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:46 smithi046 ceph-mon[109506]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:47.043 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:46 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3053157713' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:42:47.043 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:46 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/436134794' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:42:47.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.042+0000 7f797a7fc640 1 -- 172.21.15.46:0/2926559578 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f798419afb0 con 0x7f797007ac40 2024-09-10T04:42:47.044 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:42:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:42:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:42:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:42:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:42:47.044 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:42:47.045 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:42:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f797007ac40 msgr2=0x7f797007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f797007ac40 0x7f797007d100 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f798419b270 tx=0x7f7974031400 comp rx=0 tx=0).stop 2024-09-10T04:42:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984100650 msgr2=0x7f798419f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984100650 0x7f798419f8c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f798000b420 tx=0x7f798000b8e0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 shutdown_connections 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f797007ac40 0x7f797007d100 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7984101000 0x7f798419fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 --2- 172.21.15.46:0/2926559578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7984100650 0x7f798419f8c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 >> 172.21.15.46:0/2926559578 conn(0x7f79840fc520 msgr2=0x7f79840fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 shutdown_connections 2024-09-10T04:42:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.044+0000 7f798b7ab640 1 -- 172.21.15.46:0/2926559578 wait complete. 2024-09-10T04:42:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:46 smithi070 ceph-mon[79774]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:46 smithi070 ceph-mon[79774]: pgmap v1114: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:46 smithi070 ceph-mon[79774]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:46 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3053157713' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:42:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:46 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/436134794' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.128+0000 7f030331c640 1 -- 172.21.15.46:0/1088570891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074eb0 msgr2=0x7f02fc1123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.128+0000 7f030331c640 1 --2- 172.21.15.46:0/1088570891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074eb0 0x7f02fc1123d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009880 tx=0x7f02ec02a530 comp rx=0 tx=0).stop 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 -- 172.21.15.46:0/1088570891 shutdown_connections 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 --2- 172.21.15.46:0/1088570891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074eb0 0x7f02fc1123d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 --2- 172.21.15.46:0/1088570891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02fc074590 0x7f02fc074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 -- 172.21.15.46:0/1088570891 >> 172.21.15.46:0/1088570891 conn(0x7f02fc0fd600 msgr2=0x7f02fc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 -- 172.21.15.46:0/1088570891 shutdown_connections 2024-09-10T04:42:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 -- 172.21.15.46:0/1088570891 wait complete. 2024-09-10T04:42:47.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 Processor -- start 2024-09-10T04:42:47.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 -- start start 2024-09-10T04:42:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074590 0x7f02fc19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02fc074eb0 0x7f02fc19e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.129+0000 7f030331c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc1153e0 con 0x7f02fc074590 2024-09-10T04:42:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f030331c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc115260 con 0x7f02fc074eb0 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03010b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074590 0x7f02fc19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03010b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074590 0x7f02fc19dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41296/0 (socket says 172.21.15.46:41296) 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03010b8640 1 -- 172.21.15.46:0/2775746017 learned_addr learned my addr 172.21.15.46:0/2775746017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03008b7640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02fc074eb0 0x7f02fc19e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03010b8640 1 -- 172.21.15.46:0/2775746017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02fc074eb0 msgr2=0x7f02fc19e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03010b8640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02fc074eb0 0x7f02fc19e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03010b8640 1 -- 172.21.15.46:0/2775746017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc1a28b0 con 0x7f02fc074590 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03008b7640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02fc074eb0 0x7f02fc19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f03010b8640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074590 0x7f02fc19dfe0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f02f800e970 tx=0x7f02f800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f02f27fc640 1 -- 172.21.15.46:0/2775746017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02f800cac0 con 0x7f02fc074590 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f02f27fc640 1 -- 172.21.15.46:0/2775746017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02f80054e0 con 0x7f02fc074590 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc1a2ba0 con 0x7f02fc074590 2024-09-10T04:42:47.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f02f27fc640 1 -- 172.21.15.46:0/2775746017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f02f8010430 con 0x7f02fc074590 2024-09-10T04:42:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.130+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc1a3130 con 0x7f02fc074590 2024-09-10T04:42:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.131+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc075cd0 con 0x7f02fc074590 2024-09-10T04:42:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.132+0000 7f02f27fc640 1 -- 172.21.15.46:0/2775746017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f02f8004f00 con 0x7f02fc074590 2024-09-10T04:42:47.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.133+0000 7f02f27fc640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02e807ac40 0x7f02e807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:42:47.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.133+0000 7f02f27fc640 1 -- 172.21.15.46:0/2775746017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f02f809e2d0 con 0x7f02fc074590 2024-09-10T04:42:47.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.133+0000 7f03008b7640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02e807ac40 0x7f02e807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:42:47.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.133+0000 7f03008b7640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02e807ac40 0x7f02e807d100 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f02ec006010 tx=0x7f02ec0095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:42:47.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.137+0000 7f02f27fc640 1 -- 172.21.15.46:0/2775746017 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f02f8069a90 con 0x7f02fc074590 2024-09-10T04:42:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.397+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f02fc10fac0 con 0x7f02fc074590 2024-09-10T04:42:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.397+0000 7f02f27fc640 1 -- 172.21.15.46:0/2775746017 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f02f806e940 con 0x7f02fc074590 2024-09-10T04:42:47.399 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:42:47.399 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:42:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:42:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:42:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:42:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02e807ac40 msgr2=0x7f02e807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02e807ac40 0x7f02e807d100 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f02ec006010 tx=0x7f02ec0095f0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074590 msgr2=0x7f02fc19dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:42:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074590 0x7f02fc19dfe0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f02f800e970 tx=0x7f02f800ed70 comp rx=0 tx=0).stop 2024-09-10T04:42:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 shutdown_connections 2024-09-10T04:42:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f02e807ac40 0x7f02e807d100 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02fc074eb0 0x7f02fc19e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 --2- 172.21.15.46:0/2775746017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f02fc074590 0x7f02fc19dfe0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:42:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.399+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 >> 172.21.15.46:0/2775746017 conn(0x7f02fc0fd600 msgr2=0x7f02fc104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:42:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.400+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 shutdown_connections 2024-09-10T04:42:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:42:47.400+0000 7f030331c640 1 -- 172.21.15.46:0/2775746017 wait complete. 2024-09-10T04:42:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:47 smithi070 ceph-mon[79774]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:47 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2775746017' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:42:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:47 smithi046 ceph-mon[109506]: from='client.35858 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:47 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2775746017' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:42:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:48 smithi070 ceph-mon[79774]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:48 smithi070 ceph-mon[79774]: pgmap v1115: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:48 smithi046 ceph-mon[109506]: from='client.35870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:42:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:48 smithi046 ceph-mon[109506]: pgmap v1115: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:50 smithi070 ceph-mon[79774]: pgmap v1116: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:50 smithi046 ceph-mon[109506]: pgmap v1116: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:52 smithi070 ceph-mon[79774]: pgmap v1117: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:52 smithi046 ceph-mon[109506]: pgmap v1117: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:42:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:54 smithi070 ceph-mon[79774]: pgmap v1118: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:55.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:54 smithi046 ceph-mon[109506]: pgmap v1118: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:56 smithi070 ceph-mon[79774]: pgmap v1119: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:56 smithi046 ceph-mon[109506]: pgmap v1119: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:59.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:42:58 smithi046 ceph-mon[109506]: pgmap v1120: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:42:59.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:42:58 smithi070 ceph-mon[79774]: pgmap v1120: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:01.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:00 smithi070 ceph-mon[79774]: pgmap v1121: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:01.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:00 smithi046 ceph-mon[109506]: pgmap v1121: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:03.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:02 smithi070 ceph-mon[79774]: pgmap v1122: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:03.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:02 smithi046 ceph-mon[109506]: pgmap v1122: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:04 smithi070 ceph-mon[79774]: pgmap v1123: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:05.280 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:04 smithi046 ceph-mon[109506]: pgmap v1123: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:07.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:06 smithi070 ceph-mon[79774]: pgmap v1124: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:07.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:06 smithi046 ceph-mon[109506]: pgmap v1124: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:08 smithi070 ceph-mon[79774]: pgmap v1125: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:09.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:09.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:08 smithi046 ceph-mon[109506]: pgmap v1125: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:09.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:11.286 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:10 smithi046 ceph-mon[109506]: pgmap v1126: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:11.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:10 smithi070 ceph-mon[79774]: pgmap v1126: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:13.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:12 smithi070 ceph-mon[79774]: pgmap v1127: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:13.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:12 smithi046 ceph-mon[109506]: pgmap v1127: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:15.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:14 smithi070 ceph-mon[79774]: pgmap v1128: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:15.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:14 smithi046 ceph-mon[109506]: pgmap v1128: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:17.285 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:16 smithi046 ceph-mon[109506]: pgmap v1129: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:17.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:16 smithi070 ceph-mon[79774]: pgmap v1129: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 -- 172.21.15.46:0/1353336933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20101900 msgr2=0x7f7c20101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1353336933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20101900 0x7f7c20101d60 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009880 tx=0x7f7c1402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 -- 172.21.15.46:0/1353336933 shutdown_connections 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1353336933 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20101900 0x7f7c20101d60 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1353336933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20107900 0x7f7c20107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 -- 172.21.15.46:0/1353336933 >> 172.21.15.46:0/1353336933 conn(0x7f7c200fd600 msgr2=0x7f7c200ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 -- 172.21.15.46:0/1353336933 shutdown_connections 2024-09-10T04:43:17.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.498+0000 7f7c2518f640 1 -- 172.21.15.46:0/1353336933 wait complete. 2024-09-10T04:43:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.499+0000 7f7c2518f640 1 Processor -- start 2024-09-10T04:43:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c2518f640 1 -- start start 2024-09-10T04:43:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c2518f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101900 0x7f7c2019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c2518f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20107900 0x7f7c2019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c2518f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2010edb0 con 0x7f7c20107900 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c2518f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2010ec30 con 0x7f7c20101900 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c1e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20107900 0x7f7c2019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c1e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20107900 0x7f7c2019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55756/0 (socket says 172.21.15.46:55756) 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c1e59c640 1 -- 172.21.15.46:0/1039309591 learned_addr learned my addr 172.21.15.46:0/1039309591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c1ed9d640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101900 0x7f7c2019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c1e59c640 1 -- 172.21.15.46:0/1039309591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101900 msgr2=0x7f7c2019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c1e59c640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101900 0x7f7c2019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.500+0000 7f7c1e59c640 1 -- 172.21.15.46:0/1039309591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201a03e0 con 0x7f7c20107900 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.501+0000 7f7c1e59c640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20107900 0x7f7c2019fd30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7c14031460 tx=0x7f7c140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.501+0000 7f7c07fff640 1 -- 172.21.15.46:0/1039309591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7c1403f070 con 0x7f7c20107900 2024-09-10T04:43:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.501+0000 7f7c07fff640 1 -- 172.21.15.46:0/1039309591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c14004cf0 con 0x7f7c20107900 2024-09-10T04:43:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.501+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c20199a70 con 0x7f7c20107900 2024-09-10T04:43:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.501+0000 7f7c07fff640 1 -- 172.21.15.46:0/1039309591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7c14035030 con 0x7f7c20107900 2024-09-10T04:43:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.501+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c20199f70 con 0x7f7c20107900 2024-09-10T04:43:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.502+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c20103040 con 0x7f7c20107900 2024-09-10T04:43:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.503+0000 7f7c07fff640 1 -- 172.21.15.46:0/1039309591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7c14042070 con 0x7f7c20107900 2024-09-10T04:43:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.503+0000 7f7c07fff640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7c0807abf0 0x7f7c0807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.504+0000 7f7c07fff640 1 -- 172.21.15.46:0/1039309591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7c140be730 con 0x7f7c20107900 2024-09-10T04:43:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.504+0000 7f7c1ed9d640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7c0807abf0 0x7f7c0807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.504+0000 7f7c1ed9d640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7c0807abf0 0x7f7c0807d0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7c2019aaf0 tx=0x7f7c1000a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.508+0000 7f7c07fff640 1 -- 172.21.15.46:0/1039309591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7c14089dd0 con 0x7f7c20107900 2024-09-10T04:43:17.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.713+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f7c2019a2e0 con 0x7f7c0807abf0 2024-09-10T04:43:17.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.714+0000 7f7c07fff640 1 -- 172.21.15.46:0/1039309591 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f7c2019a2e0 con 0x7f7c0807abf0 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7c0807abf0 msgr2=0x7f7c0807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7c0807abf0 0x7f7c0807d0b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7c2019aaf0 tx=0x7f7c1000a2d0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20107900 msgr2=0x7f7c2019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20107900 0x7f7c2019fd30 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7c14031460 tx=0x7f7c140048f0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 shutdown_connections 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7c0807abf0 0x7f7c0807d0b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c20107900 0x7f7c2019fd30 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 --2- 172.21.15.46:0/1039309591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c20101900 0x7f7c2019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 >> 172.21.15.46:0/1039309591 conn(0x7f7c200fd600 msgr2=0x7f7c200fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 shutdown_connections 2024-09-10T04:43:17.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.717+0000 7f7c2518f640 1 -- 172.21.15.46:0/1039309591 wait complete. 2024-09-10T04:43:17.727 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:43:17.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/1580263275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18107900 msgr2=0x7f5d18107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:17.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/1580263275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18107900 0x7f5d18107ce0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009880 tx=0x7f5d0802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/1580263275 shutdown_connections 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/1580263275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d18101900 0x7f5d18101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/1580263275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18107900 0x7f5d18107ce0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/1580263275 >> 172.21.15.46:0/1580263275 conn(0x7f5d180fd600 msgr2=0x7f5d180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/1580263275 shutdown_connections 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.802+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/1580263275 wait complete. 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d1e9b3640 1 Processor -- start 2024-09-10T04:43:17.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d1e9b3640 1 -- start start 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d1e9b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18101900 0x7f5d1819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d1e9b3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d18107900 0x7f5d1819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d1e9b3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1810edb0 con 0x7f5d18101900 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d1e9b3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1810ec30 con 0x7f5d18107900 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d17fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18101900 0x7f5d1819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d17fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18101900 0x7f5d1819d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55774/0 (socket says 172.21.15.46:55774) 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d17fff640 1 -- 172.21.15.46:0/3753146119 learned_addr learned my addr 172.21.15.46:0/3753146119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.803+0000 7f5d177fe640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d18107900 0x7f5d1819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d17fff640 1 -- 172.21.15.46:0/3753146119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d18107900 msgr2=0x7f5d1819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d17fff640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d18107900 0x7f5d1819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d17fff640 1 -- 172.21.15.46:0/3753146119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d181977e0 con 0x7f5d18101900 2024-09-10T04:43:17.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d17fff640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18101900 0x7f5d1819d660 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5d08031460 tx=0x7f5d080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d157fa640 1 -- 172.21.15.46:0/3753146119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d0803f070 con 0x7f5d18101900 2024-09-10T04:43:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d18197a70 con 0x7f5d18101900 2024-09-10T04:43:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d18197fa0 con 0x7f5d18101900 2024-09-10T04:43:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d157fa640 1 -- 172.21.15.46:0/3753146119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5d08004cf0 con 0x7f5d18101900 2024-09-10T04:43:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.804+0000 7f5d157fa640 1 -- 172.21.15.46:0/3753146119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5d08035030 con 0x7f5d18101900 2024-09-10T04:43:17.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.805+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d04005290 con 0x7f5d18101900 2024-09-10T04:43:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.807+0000 7f5d157fa640 1 -- 172.21.15.46:0/3753146119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5d08042070 con 0x7f5d18101900 2024-09-10T04:43:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.807+0000 7f5d157fa640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d0007ac40 0x7f5d0007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.807+0000 7f5d157fa640 1 -- 172.21.15.46:0/3753146119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5d080bec30 con 0x7f5d18101900 2024-09-10T04:43:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.807+0000 7f5d177fe640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d0007ac40 0x7f5d0007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:17.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.808+0000 7f5d177fe640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d0007ac40 0x7f5d0007d100 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5d18100d10 tx=0x7f5d0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:17.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:17.811+0000 7f5d157fa640 1 -- 172.21.15.46:0/3753146119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5d0808a3d0 con 0x7f5d18101900 2024-09-10T04:43:18.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.017+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f5d04000bc0 con 0x7f5d0007ac40 2024-09-10T04:43:18.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.019+0000 7f5d157fa640 1 -- 172.21.15.46:0/3753146119 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f5d04000bc0 con 0x7f5d0007ac40 2024-09-10T04:43:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d0007ac40 msgr2=0x7f5d0007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d0007ac40 0x7f5d0007d100 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5d18100d10 tx=0x7f5d0c00a2d0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18101900 msgr2=0x7f5d1819d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18101900 0x7f5d1819d660 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5d08031460 tx=0x7f5d080048f0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 shutdown_connections 2024-09-10T04:43:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5d0007ac40 0x7f5d0007d100 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d18107900 0x7f5d1819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 --2- 172.21.15.46:0/3753146119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18101900 0x7f5d1819d660 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 >> 172.21.15.46:0/3753146119 conn(0x7f5d180fd600 msgr2=0x7f5d180fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 shutdown_connections 2024-09-10T04:43:18.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.021+0000 7f5d1e9b3640 1 -- 172.21.15.46:0/3753146119 wait complete. 2024-09-10T04:43:18.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.105+0000 7f6613f15640 1 -- 172.21.15.46:0/51270934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 msgr2=0x7f660c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.105+0000 7f6613f15640 1 --2- 172.21.15.46:0/51270934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c10b8d0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f6608009880 tx=0x7f660802a530 comp rx=0 tx=0).stop 2024-09-10T04:43:18.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.105+0000 7f6613f15640 1 -- 172.21.15.46:0/51270934 shutdown_connections 2024-09-10T04:43:18.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.105+0000 7f6613f15640 1 --2- 172.21.15.46:0/51270934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c10b8d0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.105+0000 7f6613f15640 1 --2- 172.21.15.46:0/51270934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f660c102ac0 0x7f660c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.105+0000 7f6613f15640 1 -- 172.21.15.46:0/51270934 >> 172.21.15.46:0/51270934 conn(0x7f660c0fc520 msgr2=0x7f660c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:18.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 -- 172.21.15.46:0/51270934 shutdown_connections 2024-09-10T04:43:18.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 -- 172.21.15.46:0/51270934 wait complete. 2024-09-10T04:43:18.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 Processor -- start 2024-09-10T04:43:18.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 -- start start 2024-09-10T04:43:18.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f660c102ac0 0x7f660c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f660c10edf0 con 0x7f660c1033e0 2024-09-10T04:43:18.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.106+0000 7f6613f15640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f660c10ec70 con 0x7f660c102ac0 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f66114b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f66114b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c19dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55790/0 (socket says 172.21.15.46:55790) 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f66114b0640 1 -- 172.21.15.46:0/628464830 learned_addr learned my addr 172.21.15.46:0/628464830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f6611cb1640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f660c102ac0 0x7f660c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f66114b0640 1 -- 172.21.15.46:0/628464830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f660c102ac0 msgr2=0x7f660c19d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f66114b0640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f660c102ac0 0x7f660c19d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f66114b0640 1 -- 172.21.15.46:0/628464830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f660c197800 con 0x7f660c1033e0 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f66114b0640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c19dbc0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6608031460 tx=0x7f66080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f65feffd640 1 -- 172.21.15.46:0/628464830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f660803f070 con 0x7f660c1033e0 2024-09-10T04:43:18.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f65feffd640 1 -- 172.21.15.46:0/628464830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6608004ac0 con 0x7f660c1033e0 2024-09-10T04:43:18.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f660c197a90 con 0x7f660c1033e0 2024-09-10T04:43:18.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f65feffd640 1 -- 172.21.15.46:0/628464830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6608035030 con 0x7f660c1033e0 2024-09-10T04:43:18.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.107+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f660c197fc0 con 0x7f660c1033e0 2024-09-10T04:43:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.109+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f660c06aa90 con 0x7f660c1033e0 2024-09-10T04:43:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.109+0000 7f65feffd640 1 -- 172.21.15.46:0/628464830 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6608042070 con 0x7f660c1033e0 2024-09-10T04:43:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.110+0000 7f65feffd640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65f807abf0 0x7f65f807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.110+0000 7f65feffd640 1 -- 172.21.15.46:0/628464830 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f66080be630 con 0x7f660c1033e0 2024-09-10T04:43:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.110+0000 7f6611cb1640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65f807abf0 0x7f65f807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.110+0000 7f6611cb1640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65f807abf0 0x7f65f807d0b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f66000058c0 tx=0x7f660000a250 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:18.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.114+0000 7f65feffd640 1 -- 172.21.15.46:0/628464830 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6608089d70 con 0x7f660c1033e0 2024-09-10T04:43:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.312+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f660c198270 con 0x7f65f807abf0 2024-09-10T04:43:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.323+0000 7f65feffd640 1 -- 172.21.15.46:0/628464830 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f660c198270 con 0x7f65f807abf0 2024-09-10T04:43:18.324 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:43:18.324 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (34m) 13m ago 46m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:43:18.324 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (37m) 13m ago 46m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:43:18.324 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (37m) 13m ago 45m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:43:18.324 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (33m) 13m ago 46m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (33m) 13m ago 45m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (34m) 13m ago 46m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (29m) 13m ago 40m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (29m) 13m ago 40m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (29m) 13m ago 40m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (14m) 13m ago 40m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (36m) 13m ago 47m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (35m) 13m ago 45m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (33m) 13m ago 47m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (33m) 13m ago 45m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (35m) 13m ago 46m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (35m) 13m ago 45m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:43:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (32m) 13m ago 44m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:43:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (32m) 13m ago 43m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:43:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (31m) 13m ago 43m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:43:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (31m) 13m ago 42m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:43:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (30m) 13m ago 42m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:43:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (30m) 13m ago 41m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:43:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (35m) 13m ago 45m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65f807abf0 msgr2=0x7f65f807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65f807abf0 0x7f65f807d0b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f66000058c0 tx=0x7f660000a250 comp rx=0 tx=0).stop 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 msgr2=0x7f660c19dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c19dbc0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f6608031460 tx=0x7f66080046c0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 shutdown_connections 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f65f807abf0 0x7f65f807d0b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f660c1033e0 0x7f660c19dbc0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 --2- 172.21.15.46:0/628464830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f660c102ac0 0x7f660c19d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 >> 172.21.15.46:0/628464830 conn(0x7f660c0fc520 msgr2=0x7f660c108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 shutdown_connections 2024-09-10T04:43:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.325+0000 7f6613f15640 1 -- 172.21.15.46:0/628464830 wait complete. 2024-09-10T04:43:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 -- 172.21.15.46:0/4116685499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 msgr2=0x7f4a64101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 --2- 172.21.15.46:0/4116685499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a64101d60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4a58009880 tx=0x7f4a5802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 -- 172.21.15.46:0/4116685499 shutdown_connections 2024-09-10T04:43:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 --2- 172.21.15.46:0/4116685499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a64101d60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 --2- 172.21.15.46:0/4116685499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64107900 0x7f4a64107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 -- 172.21.15.46:0/4116685499 >> 172.21.15.46:0/4116685499 conn(0x7f4a640fd600 msgr2=0x7f4a640ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 -- 172.21.15.46:0/4116685499 shutdown_connections 2024-09-10T04:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.409+0000 7f4a69257640 1 -- 172.21.15.46:0/4116685499 wait complete. 2024-09-10T04:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a69257640 1 Processor -- start 2024-09-10T04:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a69257640 1 -- start start 2024-09-10T04:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a69257640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a69257640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64107900 0x7f4a64074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a69257640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410edd0 con 0x7f4a64101900 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a69257640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6410ec50 con 0x7f4a64107900 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a62d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a62d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a640744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55820/0 (socket says 172.21.15.46:55820) 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a62d9d640 1 -- 172.21.15.46:0/1723728342 learned_addr learned my addr 172.21.15.46:0/1723728342 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.410+0000 7f4a6259c640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64107900 0x7f4a64074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a62d9d640 1 -- 172.21.15.46:0/1723728342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64107900 msgr2=0x7f4a64074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a62d9d640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64107900 0x7f4a64074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a62d9d640 1 -- 172.21.15.46:0/1723728342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a64074f70 con 0x7f4a64101900 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a6259c640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64107900 0x7f4a64074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a62d9d640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a640744f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d6f0 tx=0x7f4a5400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a4bfff640 1 -- 172.21.15.46:0/1723728342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a54017070 con 0x7f4a64101900 2024-09-10T04:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a64075260 con 0x7f4a64101900 2024-09-10T04:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a4bfff640 1 -- 172.21.15.46:0/1723728342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4a5400de00 con 0x7f4a64101900 2024-09-10T04:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a4bfff640 1 -- 172.21.15.46:0/1723728342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4a54004bb0 con 0x7f4a64101900 2024-09-10T04:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.411+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641a3e80 con 0x7f4a64101900 2024-09-10T04:43:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.412+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a64103040 con 0x7f4a64101900 2024-09-10T04:43:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.413+0000 7f4a4bfff640 1 -- 172.21.15.46:0/1723728342 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4a54004e60 con 0x7f4a64101900 2024-09-10T04:43:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.413+0000 7f4a4bfff640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a4c07ac00 0x7f4a4c07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.413+0000 7f4a4bfff640 1 -- 172.21.15.46:0/1723728342 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4a5409d8b0 con 0x7f4a64101900 2024-09-10T04:43:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.413+0000 7f4a6259c640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a4c07ac00 0x7f4a4c07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.414+0000 7f4a6259c640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a4c07ac00 0x7f4a4c07d0c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4a58031460 tx=0x7f4a580041a0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:18.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.418+0000 7f4a4bfff640 1 -- 172.21.15.46:0/1723728342 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4a54068ff0 con 0x7f4a64101900 2024-09-10T04:43:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.680+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f4a64109e70 con 0x7f4a64101900 2024-09-10T04:43:18.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.680+0000 7f4a4bfff640 1 -- 172.21.15.46:0/1723728342 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f4a5406dea0 con 0x7f4a64101900 2024-09-10T04:43:18.681 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:43:18.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:43:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:43:18.683 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:43:18.683 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.682+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a4c07ac00 msgr2=0x7f4a4c07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.682+0000 7f4a69257640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a4c07ac00 0x7f4a4c07d0c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4a58031460 tx=0x7f4a580041a0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.682+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 msgr2=0x7f4a640744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.682+0000 7f4a69257640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a640744f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400d6f0 tx=0x7f4a5400dbb0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.683+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 shutdown_connections 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.683+0000 7f4a69257640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4a4c07ac00 0x7f4a4c07d0c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.683+0000 7f4a69257640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64107900 0x7f4a64074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.683+0000 7f4a69257640 1 --2- 172.21.15.46:0/1723728342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a64101900 0x7f4a640744f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.683+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 >> 172.21.15.46:0/1723728342 conn(0x7f4a640fd600 msgr2=0x7f4a640fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.683+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 shutdown_connections 2024-09-10T04:43:18.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.683+0000 7f4a69257640 1 -- 172.21.15.46:0/1723728342 wait complete. 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.766+0000 7fe12039f640 1 -- 172.21.15.46:0/1949506325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 msgr2=0x7fe118101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.766+0000 7fe12039f640 1 --2- 172.21.15.46:0/1949506325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe118101d60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe108009880 tx=0x7fe10802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 -- 172.21.15.46:0/1949506325 shutdown_connections 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 --2- 172.21.15.46:0/1949506325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe118101d60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 --2- 172.21.15.46:0/1949506325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe118107900 0x7fe118107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 -- 172.21.15.46:0/1949506325 >> 172.21.15.46:0/1949506325 conn(0x7fe1180fd600 msgr2=0x7fe1180ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 -- 172.21.15.46:0/1949506325 shutdown_connections 2024-09-10T04:43:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 -- 172.21.15.46:0/1949506325 wait complete. 2024-09-10T04:43:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 Processor -- start 2024-09-10T04:43:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 -- start start 2024-09-10T04:43:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.767+0000 7fe12039f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe11819d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe12039f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe118107900 0x7fe11819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe12039f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe11810edb0 con 0x7fe118101900 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe12039f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe11810ec30 con 0x7fe118107900 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11e13b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe11819d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11e13b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe11819d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55836/0 (socket says 172.21.15.46:55836) 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11e13b640 1 -- 172.21.15.46:0/3245323721 learned_addr learned my addr 172.21.15.46:0/3245323721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11d93a640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe118107900 0x7fe11819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11e13b640 1 -- 172.21.15.46:0/3245323721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe118107900 msgr2=0x7fe11819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11e13b640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe118107900 0x7fe11819dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11e13b640 1 -- 172.21.15.46:0/3245323721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe118197800 con 0x7fe118101900 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11d93a640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe118107900 0x7fe11819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe11e13b640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe11819d680 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c810 tx=0x7fe11400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe10f7fe640 1 -- 172.21.15.46:0/3245323721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe114004020 con 0x7fe118101900 2024-09-10T04:43:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe10f7fe640 1 -- 172.21.15.46:0/3245323721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1140041c0 con 0x7fe118101900 2024-09-10T04:43:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe118197af0 con 0x7fe118101900 2024-09-10T04:43:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe10f7fe640 1 -- 172.21.15.46:0/3245323721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe1140047a0 con 0x7fe118101900 2024-09-10T04:43:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.768+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe118198080 con 0x7fe118101900 2024-09-10T04:43:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.769+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe118103040 con 0x7fe118101900 2024-09-10T04:43:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.770+0000 7fe10f7fe640 1 -- 172.21.15.46:0/3245323721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe1140079b0 con 0x7fe118101900 2024-09-10T04:43:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.771+0000 7fe10f7fe640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10407abf0 0x7fe10407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.771+0000 7fe10f7fe640 1 -- 172.21.15.46:0/3245323721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe11409dc60 con 0x7fe118101900 2024-09-10T04:43:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.771+0000 7fe11d93a640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10407abf0 0x7fe10407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.771+0000 7fe11d93a640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10407abf0 0x7fe10407d0b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe108031460 tx=0x7fe1080095f0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:18.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:18.775+0000 7fe10f7fe640 1 -- 172.21.15.46:0/3245323721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe114014030 con 0x7fe118101900 2024-09-10T04:43:19.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.002+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe118109e90 con 0x7fe118101900 2024-09-10T04:43:19.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:18 smithi046 ceph-mon[109506]: pgmap v1130: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:19.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:18 smithi046 ceph-mon[109506]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:19.004 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:18 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1723728342' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:43:19.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.003+0000 7fe10f7fe640 1 -- 172.21.15.46:0/3245323721 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7fe114069300 con 0x7fe118101900 2024-09-10T04:43:19.004 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:43:19.004 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:43:19.005 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:43:19.006 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:19.007 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:19.008 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:43:19.008 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:19.008 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:19.008 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:19.008 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:19.008 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:43:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10407abf0 msgr2=0x7fe10407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10407abf0 0x7fe10407d0b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe108031460 tx=0x7fe1080095f0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 msgr2=0x7fe11819d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe11819d680 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fe11400c810 tx=0x7fe11400ccd0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 shutdown_connections 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe10407abf0 0x7fe10407d0b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe118107900 0x7fe11819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 --2- 172.21.15.46:0/3245323721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe118101900 0x7fe11819d680 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 >> 172.21.15.46:0/3245323721 conn(0x7fe1180fd600 msgr2=0x7fe1180fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 shutdown_connections 2024-09-10T04:43:19.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.005+0000 7fe12039f640 1 -- 172.21.15.46:0/3245323721 wait complete. 2024-09-10T04:43:19.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 -- 172.21.15.46:0/1564085638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528102b30 msgr2=0x7f8528102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 --2- 172.21.15.46:0/1564085638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528102b30 0x7f8528102f10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f8518009880 tx=0x7f851802a530 comp rx=0 tx=0).stop 2024-09-10T04:43:19.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 -- 172.21.15.46:0/1564085638 shutdown_connections 2024-09-10T04:43:19.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 --2- 172.21.15.46:0/1564085638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8528103450 0x7f852810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 --2- 172.21.15.46:0/1564085638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528102b30 0x7f8528102f10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 -- 172.21.15.46:0/1564085638 >> 172.21.15.46:0/1564085638 conn(0x7f85280fc550 msgr2=0x7f85280fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:19.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 -- 172.21.15.46:0/1564085638 shutdown_connections 2024-09-10T04:43:19.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.092+0000 7f852dd9d640 1 -- 172.21.15.46:0/1564085638 wait complete. 2024-09-10T04:43:19.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.093+0000 7f852dd9d640 1 Processor -- start 2024-09-10T04:43:19.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.093+0000 7f852dd9d640 1 -- start start 2024-09-10T04:43:19.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.093+0000 7f852dd9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8528102b30 0x7f8528100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:19.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.093+0000 7f852dd9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528103450 0x7f8528100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:19.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.093+0000 7f852dd9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f852810ede0 con 0x7f8528103450 2024-09-10T04:43:19.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.093+0000 7f852dd9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f852810ec60 con 0x7f8528102b30 2024-09-10T04:43:19.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8526ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528103450 0x7f8528100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:19.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8526ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528103450 0x7f8528100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49632/0 (socket says 172.21.15.46:49632) 2024-09-10T04:43:19.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8526ffd640 1 -- 172.21.15.46:0/2426561104 learned_addr learned my addr 172.21.15.46:0/2426561104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f85277fe640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8528102b30 0x7f8528100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8526ffd640 1 -- 172.21.15.46:0/2426561104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8528102b30 msgr2=0x7f8528100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8526ffd640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8528102b30 0x7f8528100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8526ffd640 1 -- 172.21.15.46:0/2426561104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8528101210 con 0x7f8528103450 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f85277fe640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8528102b30 0x7f8528100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8526ffd640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528103450 0x7f8528100cd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f851c00e970 tx=0x7f851c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8524ff9640 1 -- 172.21.15.46:0/2426561104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f851c00cac0 con 0x7f8528103450 2024-09-10T04:43:19.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8524ff9640 1 -- 172.21.15.46:0/2426561104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f851c0054e0 con 0x7f8528103450 2024-09-10T04:43:19.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8528101500 con 0x7f8528103450 2024-09-10T04:43:19.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f8524ff9640 1 -- 172.21.15.46:0/2426561104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f851c010430 con 0x7f8528103450 2024-09-10T04:43:19.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.094+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85281a3e50 con 0x7f8528103450 2024-09-10T04:43:19.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.096+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852806a500 con 0x7f8528103450 2024-09-10T04:43:19.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.096+0000 7f8524ff9640 1 -- 172.21.15.46:0/2426561104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f851c004ec0 con 0x7f8528103450 2024-09-10T04:43:19.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.097+0000 7f8524ff9640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f851007abf0 0x7f851007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:19.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.097+0000 7f8524ff9640 1 -- 172.21.15.46:0/2426561104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f851c09e340 con 0x7f8528103450 2024-09-10T04:43:19.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.097+0000 7f85277fe640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f851007abf0 0x7f851007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:19.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.097+0000 7f85277fe640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f851007abf0 0x7f851007d0b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f85180096c0 tx=0x7f8518004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:19.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.101+0000 7f8524ff9640 1 -- 172.21.15.46:0/2426561104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f851c069a80 con 0x7f8528103450 2024-09-10T04:43:19.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.305+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f8528062630 con 0x7f851007abf0 2024-09-10T04:43:19.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.306+0000 7f8524ff9640 1 -- 172.21.15.46:0/2426561104 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7f8528062630 con 0x7f851007abf0 2024-09-10T04:43:19.307 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:43:19.307 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:43:19.307 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:43:19.307 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:43:19.307 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:43:19.308 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:43:19.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f851007abf0 msgr2=0x7f851007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f851007abf0 0x7f851007d0b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f85180096c0 tx=0x7f8518004f30 comp rx=0 tx=0).stop 2024-09-10T04:43:19.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528103450 msgr2=0x7f8528100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528103450 0x7f8528100cd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f851c00e970 tx=0x7f851c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:43:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 shutdown_connections 2024-09-10T04:43:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f851007abf0 0x7f851007d0b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8528103450 0x7f8528100cd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 --2- 172.21.15.46:0/2426561104 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8528102b30 0x7f8528100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 >> 172.21.15.46:0/2426561104 conn(0x7f85280fc550 msgr2=0x7f8528108f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 shutdown_connections 2024-09-10T04:43:19.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.308+0000 7f852dd9d640 1 -- 172.21.15.46:0/2426561104 wait complete. 2024-09-10T04:43:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:18 smithi070 ceph-mon[79774]: pgmap v1130: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:18 smithi070 ceph-mon[79774]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:19.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:18 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1723728342' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:43:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 -- 172.21.15.46:0/189755586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 msgr2=0x7efd3410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 --2- 172.21.15.46:0/189755586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3410b8d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7efd28009880 tx=0x7efd2802a530 comp rx=0 tx=0).stop 2024-09-10T04:43:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 -- 172.21.15.46:0/189755586 shutdown_connections 2024-09-10T04:43:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 --2- 172.21.15.46:0/189755586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3410b8d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 --2- 172.21.15.46:0/189755586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd34102ac0 0x7efd34102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 -- 172.21.15.46:0/189755586 >> 172.21.15.46:0/189755586 conn(0x7efd340fc520 msgr2=0x7efd340fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 -- 172.21.15.46:0/189755586 shutdown_connections 2024-09-10T04:43:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.393+0000 7efd39496640 1 -- 172.21.15.46:0/189755586 wait complete. 2024-09-10T04:43:19.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd39496640 1 Processor -- start 2024-09-10T04:43:19.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd39496640 1 -- start start 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd39496640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd34102ac0 0x7efd3419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd39496640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd39496640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efd3410edd0 con 0x7efd341033e0 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd39496640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efd3410ec50 con 0x7efd34102ac0 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd327fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd327fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3419dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49642/0 (socket says 172.21.15.46:49642) 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd327fc640 1 -- 172.21.15.46:0/2451962718 learned_addr learned my addr 172.21.15.46:0/2451962718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.394+0000 7efd32ffd640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd34102ac0 0x7efd3419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd327fc640 1 -- 172.21.15.46:0/2451962718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd34102ac0 msgr2=0x7efd3419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd327fc640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd34102ac0 0x7efd3419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd327fc640 1 -- 172.21.15.46:0/2451962718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd341977e0 con 0x7efd341033e0 2024-09-10T04:43:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd327fc640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3419dba0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efd28031460 tx=0x7efd28004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd1bfff640 1 -- 172.21.15.46:0/2451962718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efd2803f070 con 0x7efd341033e0 2024-09-10T04:43:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd1bfff640 1 -- 172.21.15.46:0/2451962718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efd280035f0 con 0x7efd341033e0 2024-09-10T04:43:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd1bfff640 1 -- 172.21.15.46:0/2451962718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efd28035030 con 0x7efd341033e0 2024-09-10T04:43:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd34197a70 con 0x7efd341033e0 2024-09-10T04:43:19.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.395+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd34197fa0 con 0x7efd341033e0 2024-09-10T04:43:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.396+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3406aa90 con 0x7efd341033e0 2024-09-10T04:43:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.397+0000 7efd1bfff640 1 -- 172.21.15.46:0/2451962718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efd28042070 con 0x7efd341033e0 2024-09-10T04:43:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.398+0000 7efd1bfff640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd1c07abf0 0x7efd1c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.398+0000 7efd1bfff640 1 -- 172.21.15.46:0/2451962718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efd280be650 con 0x7efd341033e0 2024-09-10T04:43:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.398+0000 7efd32ffd640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd1c07abf0 0x7efd1c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:19.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.398+0000 7efd32ffd640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd1c07abf0 0x7efd1c07d0b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd2400bbe0 tx=0x7efd2400b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.402+0000 7efd1bfff640 1 -- 172.21.15.46:0/2451962718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efd28089cf0 con 0x7efd341033e0 2024-09-10T04:43:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.663+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7efd34105f60 con 0x7efd341033e0 2024-09-10T04:43:19.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.663+0000 7efd1bfff640 1 -- 172.21.15.46:0/2451962718 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7efd2808eba0 con 0x7efd341033e0 2024-09-10T04:43:19.664 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:43:19.665 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:43:19.666 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:43:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd1c07abf0 msgr2=0x7efd1c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd1c07abf0 0x7efd1c07d0b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd2400bbe0 tx=0x7efd2400b390 comp rx=0 tx=0).stop 2024-09-10T04:43:19.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 msgr2=0x7efd3419dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3419dba0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7efd28031460 tx=0x7efd28004530 comp rx=0 tx=0).stop 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 shutdown_connections 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efd1c07abf0 0x7efd1c07d0b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd341033e0 0x7efd3419dba0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 --2- 172.21.15.46:0/2451962718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd34102ac0 0x7efd3419d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 >> 172.21.15.46:0/2451962718 conn(0x7efd340fc520 msgr2=0x7efd34108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 shutdown_connections 2024-09-10T04:43:19.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:19.666+0000 7efd39496640 1 -- 172.21.15.46:0/2451962718 wait complete. 2024-09-10T04:43:20.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:19 smithi046 ceph-mon[109506]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:19 smithi046 ceph-mon[109506]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:19 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3245323721' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:43:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:19 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2451962718' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:43:20.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:19 smithi070 ceph-mon[79774]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:20.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:19 smithi070 ceph-mon[79774]: from='client.35886 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:20.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:19 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3245323721' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:43:20.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:19 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2451962718' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:43:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:20 smithi070 ceph-mon[79774]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:21.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:20 smithi070 ceph-mon[79774]: pgmap v1131: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:21.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:20 smithi046 ceph-mon[109506]: from='client.35898 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:21.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:20 smithi046 ceph-mon[109506]: pgmap v1131: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:23.137 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:22 smithi070 ceph-mon[79774]: pgmap v1132: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:23.139 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:22 smithi046 ceph-mon[109506]: pgmap v1132: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:24.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:25.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:24 smithi070 ceph-mon[79774]: pgmap v1133: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:25.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:24 smithi046 ceph-mon[109506]: pgmap v1133: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:26.284 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:25 smithi046 ceph-mon[109506]: pgmap v1134: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:26.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:25 smithi070 ceph-mon[79774]: pgmap v1134: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:28 smithi070 ceph-mon[79774]: pgmap v1135: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:28 smithi046 ceph-mon[109506]: pgmap v1135: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:30 smithi070 ceph-mon[79774]: pgmap v1136: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:30 smithi046 ceph-mon[109506]: pgmap v1136: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:32 smithi070 ceph-mon[79774]: pgmap v1137: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:33.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:32 smithi046 ceph-mon[109506]: pgmap v1137: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:34 smithi070 ceph-mon[79774]: pgmap v1138: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:35.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:34 smithi046 ceph-mon[109506]: pgmap v1138: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:36 smithi070 ceph-mon[79774]: pgmap v1139: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:36 smithi046 ceph-mon[109506]: pgmap v1139: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:38 smithi070 ceph-mon[79774]: pgmap v1140: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:38 smithi046 ceph-mon[109506]: pgmap v1140: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:40 smithi070 ceph-mon[79774]: pgmap v1141: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:40 smithi046 ceph-mon[109506]: pgmap v1141: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:42.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:42 smithi070 ceph-mon[79774]: pgmap v1142: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:42.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:42 smithi046 ceph-mon[109506]: pgmap v1142: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:45.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:44 smithi070 ceph-mon[79774]: pgmap v1143: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:45.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:44 smithi046 ceph-mon[109506]: pgmap v1143: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:46 smithi070 ceph-mon[79774]: pgmap v1144: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:46 smithi046 ceph-mon[109506]: pgmap v1144: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:48 smithi070 ceph-mon[79774]: pgmap v1145: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:48 smithi046 ceph-mon[109506]: pgmap v1145: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 -- 172.21.15.46:0/2718324542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8107900 msgr2=0x7fc6e8107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/2718324542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8107900 0x7fc6e8107ce0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8009880 tx=0x7fc6d802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 -- 172.21.15.46:0/2718324542 shutdown_connections 2024-09-10T04:43:49.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/2718324542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e8101900 0x7fc6e8101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/2718324542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8107900 0x7fc6e8107ce0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 -- 172.21.15.46:0/2718324542 >> 172.21.15.46:0/2718324542 conn(0x7fc6e80fd600 msgr2=0x7fc6e80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:49.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 -- 172.21.15.46:0/2718324542 shutdown_connections 2024-09-10T04:43:49.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.758+0000 7fc6edfb2640 1 -- 172.21.15.46:0/2718324542 wait complete. 2024-09-10T04:43:49.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6edfb2640 1 Processor -- start 2024-09-10T04:43:49.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6edfb2640 1 -- start start 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6edfb2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8101900 0x7fc6e819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6edfb2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e8107900 0x7fc6e819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6edfb2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e810edd0 con 0x7fc6e8101900 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6edfb2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e810ec50 con 0x7fc6e8107900 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6e77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8101900 0x7fc6e819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6e77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8101900 0x7fc6e819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60814/0 (socket says 172.21.15.46:60814) 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6e77fe640 1 -- 172.21.15.46:0/1884128044 learned_addr learned my addr 172.21.15.46:0/1884128044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.759+0000 7fc6e6ffd640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e8107900 0x7fc6e819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6e77fe640 1 -- 172.21.15.46:0/1884128044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e8107900 msgr2=0x7fc6e819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6e77fe640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e8107900 0x7fc6e819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6e77fe640 1 -- 172.21.15.46:0/1884128044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e81977c0 con 0x7fc6e8101900 2024-09-10T04:43:49.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6e77fe640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8101900 0x7fc6e819d640 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8031460 tx=0x7fc6d80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:49.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6e4ff9640 1 -- 172.21.15.46:0/1884128044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc6d803f070 con 0x7fc6e8101900 2024-09-10T04:43:49.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e8197a50 con 0x7fc6e8101900 2024-09-10T04:43:49.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6e4ff9640 1 -- 172.21.15.46:0/1884128044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6d80035f0 con 0x7fc6e8101900 2024-09-10T04:43:49.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6e4ff9640 1 -- 172.21.15.46:0/1884128044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc6d8035030 con 0x7fc6e8101900 2024-09-10T04:43:49.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.760+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e8197f80 con 0x7fc6e8101900 2024-09-10T04:43:49.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.761+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e8103040 con 0x7fc6e8101900 2024-09-10T04:43:49.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.762+0000 7fc6e4ff9640 1 -- 172.21.15.46:0/1884128044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc6d8042070 con 0x7fc6e8101900 2024-09-10T04:43:49.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.763+0000 7fc6e4ff9640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6d007abf0 0x7fc6d007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:49.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.763+0000 7fc6e6ffd640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6d007abf0 0x7fc6d007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:49.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.763+0000 7fc6e4ff9640 1 -- 172.21.15.46:0/1884128044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc6d80be5d0 con 0x7fc6e8101900 2024-09-10T04:43:49.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.763+0000 7fc6e6ffd640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6d007abf0 0x7fc6d007d0b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80ffbf0 tx=0x7fc6dc00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:49.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.767+0000 7fc6e4ff9640 1 -- 172.21.15.46:0/1884128044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc6d8089d10 con 0x7fc6e8101900 2024-09-10T04:43:49.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.974+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc6e8198230 con 0x7fc6d007abf0 2024-09-10T04:43:49.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.975+0000 7fc6e4ff9640 1 -- 172.21.15.46:0/1884128044 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fc6e8198230 con 0x7fc6d007abf0 2024-09-10T04:43:49.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.977+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6d007abf0 msgr2=0x7fc6d007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.977+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6d007abf0 0x7fc6d007d0b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80ffbf0 tx=0x7fc6dc00b390 comp rx=0 tx=0).stop 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.977+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8101900 msgr2=0x7fc6e819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.977+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8101900 0x7fc6e819d640 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8031460 tx=0x7fc6d80045e0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.978+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 shutdown_connections 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.978+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc6d007abf0 0x7fc6d007d0b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.978+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e8107900 0x7fc6e819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.978+0000 7fc6edfb2640 1 --2- 172.21.15.46:0/1884128044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6e8101900 0x7fc6e819d640 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.978+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 >> 172.21.15.46:0/1884128044 conn(0x7fc6e80fd600 msgr2=0x7fc6e80fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.978+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 shutdown_connections 2024-09-10T04:43:49.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:49.978+0000 7fc6edfb2640 1 -- 172.21.15.46:0/1884128044 wait complete. 2024-09-10T04:43:49.988 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:43:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/1555975431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 msgr2=0x7fcbd4101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/1555975431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd4101d60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0009880 tx=0x7fcbc002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/1555975431 shutdown_connections 2024-09-10T04:43:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/1555975431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd4101d60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/1555975431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbd4107900 0x7fcbd4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/1555975431 >> 172.21.15.46:0/1555975431 conn(0x7fcbd40fd600 msgr2=0x7fcbd40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/1555975431 shutdown_connections 2024-09-10T04:43:50.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.063+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/1555975431 wait complete. 2024-09-10T04:43:50.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.064+0000 7fcbdb0f3640 1 Processor -- start 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.064+0000 7fcbdb0f3640 1 -- start start 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.064+0000 7fcbdb0f3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.064+0000 7fcbdb0f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbd4107900 0x7fcbd4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.064+0000 7fcbdb0f3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd410edd0 con 0x7fcbd4101900 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.064+0000 7fcbdb0f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd410ec50 con 0x7fcbd4107900 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd8e8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd8e8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60838/0 (socket says 172.21.15.46:60838) 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd8e8f640 1 -- 172.21.15.46:0/2341367236 learned_addr learned my addr 172.21.15.46:0/2341367236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd3fff640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbd4107900 0x7fcbd4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd8e8f640 1 -- 172.21.15.46:0/2341367236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbd4107900 msgr2=0x7fcbd4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd8e8f640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbd4107900 0x7fcbd4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd8e8f640 1 -- 172.21.15.46:0/2341367236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd4074f70 con 0x7fcbd4101900 2024-09-10T04:43:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd8e8f640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd40744f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800e970 tx=0x7fcbc800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd1ffb640 1 -- 172.21.15.46:0/2341367236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcbc800cac0 con 0x7fcbd4101900 2024-09-10T04:43:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd1ffb640 1 -- 172.21.15.46:0/2341367236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcbc80054e0 con 0x7fcbd4101900 2024-09-10T04:43:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd4075260 con 0x7fcbd4101900 2024-09-10T04:43:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.065+0000 7fcbd1ffb640 1 -- 172.21.15.46:0/2341367236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fcbc8010430 con 0x7fcbd4101900 2024-09-10T04:43:50.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.066+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd41a3e20 con 0x7fcbd4101900 2024-09-10T04:43:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.067+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd4103040 con 0x7fcbd4101900 2024-09-10T04:43:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.067+0000 7fcbd1ffb640 1 -- 172.21.15.46:0/2341367236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fcbc8004f00 con 0x7fcbd4101900 2024-09-10T04:43:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.068+0000 7fcbd1ffb640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbc407abf0 0x7fcbc407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.068+0000 7fcbd1ffb640 1 -- 172.21.15.46:0/2341367236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fcbc809e3c0 con 0x7fcbd4101900 2024-09-10T04:43:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.068+0000 7fcbd3fff640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbc407abf0 0x7fcbc407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.069+0000 7fcbd3fff640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbc407abf0 0x7fcbc407d0b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4075f80 tx=0x7fcbc0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:50.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.073+0000 7fcbd1ffb640 1 -- 172.21.15.46:0/2341367236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fcbc8069b00 con 0x7fcbd4101900 2024-09-10T04:43:50.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.285+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fcbd4062630 con 0x7fcbc407abf0 2024-09-10T04:43:50.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.286+0000 7fcbd1ffb640 1 -- 172.21.15.46:0/2341367236 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fcbd4062630 con 0x7fcbc407abf0 2024-09-10T04:43:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbc407abf0 msgr2=0x7fcbc407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbc407abf0 0x7fcbc407d0b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4075f80 tx=0x7fcbc0031400 comp rx=0 tx=0).stop 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 msgr2=0x7fcbd40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd40744f0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fcbc800e970 tx=0x7fcbc800ed70 comp rx=0 tx=0).stop 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 shutdown_connections 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fcbc407abf0 0x7fcbc407d0b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbd4107900 0x7fcbd4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 --2- 172.21.15.46:0/2341367236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd4101900 0x7fcbd40744f0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.288+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 >> 172.21.15.46:0/2341367236 conn(0x7fcbd40fd600 msgr2=0x7fcbd40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.289+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 shutdown_connections 2024-09-10T04:43:50.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.289+0000 7fcbdb0f3640 1 -- 172.21.15.46:0/2341367236 wait complete. 2024-09-10T04:43:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/989296520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 msgr2=0x7f4b98106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/989296520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98106be0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009880 tx=0x7f4b8802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/989296520 shutdown_connections 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/989296520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98100800 0x7f4b98100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/989296520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98106be0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/989296520 >> 172.21.15.46:0/989296520 conn(0x7f4b980fc520 msgr2=0x7f4b980fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/989296520 shutdown_connections 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/989296520 wait complete. 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.373+0000 7f4b9dcbc640 1 Processor -- start 2024-09-10T04:43:50.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b9dcbc640 1 -- start start 2024-09-10T04:43:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b9dcbc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98100800 0x7f4b98198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b9dcbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b9dcbc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810dcb0 con 0x7f4b98106800 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b9dcbc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810db30 con 0x7f4b98100800 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b96ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b96ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60846/0 (socket says 172.21.15.46:60846) 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b96ffd640 1 -- 172.21.15.46:0/2068023593 learned_addr learned my addr 172.21.15.46:0/2068023593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b977fe640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98100800 0x7f4b98198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b96ffd640 1 -- 172.21.15.46:0/2068023593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98100800 msgr2=0x7f4b98198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b96ffd640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98100800 0x7f4b98198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.374+0000 7f4b96ffd640 1 -- 172.21.15.46:0/2068023593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b98199350 con 0x7f4b98106800 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.375+0000 7f4b96ffd640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98198d50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d8d0 tx=0x7f4b8c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.375+0000 7f4b94ff9640 1 -- 172.21.15.46:0/2068023593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b8c017030 con 0x7f4b98106800 2024-09-10T04:43:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.375+0000 7f4b94ff9640 1 -- 172.21.15.46:0/2068023593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4b8c004020 con 0x7f4b98106800 2024-09-10T04:43:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.375+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b98199640 con 0x7f4b98106800 2024-09-10T04:43:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.375+0000 7f4b94ff9640 1 -- 172.21.15.46:0/2068023593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4b8c0047a0 con 0x7f4b98106800 2024-09-10T04:43:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.375+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b981a1c40 con 0x7f4b98106800 2024-09-10T04:43:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.376+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b98101f40 con 0x7f4b98106800 2024-09-10T04:43:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.377+0000 7f4b94ff9640 1 -- 172.21.15.46:0/2068023593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4b8c0041d0 con 0x7f4b98106800 2024-09-10T04:43:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.377+0000 7f4b94ff9640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007ac40 0x7f4b8007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.377+0000 7f4b94ff9640 1 -- 172.21.15.46:0/2068023593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4b8c018070 con 0x7f4b98106800 2024-09-10T04:43:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.377+0000 7f4b977fe640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007ac40 0x7f4b8007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.378+0000 7f4b977fe640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007ac40 0x7f4b8007d100 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4b880096c0 tx=0x7f4b88031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.382+0000 7f4b94ff9640 1 -- 172.21.15.46:0/2068023593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4b98199640 con 0x7f4b98106800 2024-09-10T04:43:50.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.578+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f4b98062630 con 0x7f4b8007ac40 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.588+0000 7f4b94ff9640 1 -- 172.21.15.46:0/2068023593 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f4b98062630 con 0x7f4b8007ac40 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (35m) 14m ago 46m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (37m) 14m ago 46m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (37m) 14m ago 46m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (33m) 14m ago 46m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (33m) 14m ago 46m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:43:50.590 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (35m) 14m ago 46m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (29m) 14m ago 41m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (30m) 14m ago 41m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (29m) 14m ago 41m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (14m) 14m ago 41m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (37m) 14m ago 48m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (36m) 14m ago 45m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (34m) 14m ago 48m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (34m) 14m ago 45m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (36m) 14m ago 46m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (36m) 14m ago 45m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (33m) 14m ago 44m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (32m) 14m ago 44m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (32m) 14m ago 43m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:43:50.591 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (31m) 14m ago 43m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:43:50.592 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (31m) 14m ago 42m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:43:50.592 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (30m) 14m ago 42m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:43:50.592 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (35m) 14m ago 46m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:43:50.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007ac40 msgr2=0x7f4b8007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007ac40 0x7f4b8007d100 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4b880096c0 tx=0x7f4b88031400 comp rx=0 tx=0).stop 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 msgr2=0x7f4b98198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98198d50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c00d8d0 tx=0x7f4b8c00dd90 comp rx=0 tx=0).stop 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 shutdown_connections 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4b8007ac40 0x7f4b8007d100 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b98106800 0x7f4b98198d50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 --2- 172.21.15.46:0/2068023593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b98100800 0x7f4b98198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.590+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 >> 172.21.15.46:0/2068023593 conn(0x7f4b980fc520 msgr2=0x7f4b980fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.591+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 shutdown_connections 2024-09-10T04:43:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.591+0000 7f4b9dcbc640 1 -- 172.21.15.46:0/2068023593 wait complete. 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.673+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/3530176171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 msgr2=0x7f5a18100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.673+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/3530176171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a18100c90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009880 tx=0x7f5a1402a530 comp rx=0 tx=0).stop 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.673+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/3530176171 shutdown_connections 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.673+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/3530176171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a181011d0 0x7f5a181056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.673+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/3530176171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a18100c90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.673+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/3530176171 >> 172.21.15.46:0/3530176171 conn(0x7f5a180fc520 msgr2=0x7f5a180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.674+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/3530176171 shutdown_connections 2024-09-10T04:43:50.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.674+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/3530176171 wait complete. 2024-09-10T04:43:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.674+0000 7f5a1fd7b640 1 Processor -- start 2024-09-10T04:43:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1fd7b640 1 -- start start 2024-09-10T04:43:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1fd7b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a1819e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1fd7b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a181011d0 0x7f5a1819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1fd7b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5a18115450 con 0x7f5a181008b0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1fd7b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5a181152d0 con 0x7f5a181011d0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1db17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a1819e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1db17640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a1819e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60874/0 (socket says 172.21.15.46:60874) 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1db17640 1 -- 172.21.15.46:0/2252369194 learned_addr learned my addr 172.21.15.46:0/2252369194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1d316640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a181011d0 0x7f5a1819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1db17640 1 -- 172.21.15.46:0/2252369194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a181011d0 msgr2=0x7f5a1819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1db17640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a181011d0 0x7f5a1819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1db17640 1 -- 172.21.15.46:0/2252369194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a181a2900 con 0x7f5a181008b0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.675+0000 7f5a1d316640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a181011d0 0x7f5a1819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.676+0000 7f5a1db17640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a1819e030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5a14031460 tx=0x7f5a140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.676+0000 7f5a0effd640 1 -- 172.21.15.46:0/2252369194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5a1403f070 con 0x7f5a181008b0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.676+0000 7f5a0effd640 1 -- 172.21.15.46:0/2252369194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5a14004cf0 con 0x7f5a181008b0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.676+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a181a2b90 con 0x7f5a181008b0 2024-09-10T04:43:50.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.676+0000 7f5a0effd640 1 -- 172.21.15.46:0/2252369194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5a14035030 con 0x7f5a181008b0 2024-09-10T04:43:50.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.676+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a181a30c0 con 0x7f5a181008b0 2024-09-10T04:43:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.677+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1806aa90 con 0x7f5a181008b0 2024-09-10T04:43:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.678+0000 7f5a0effd640 1 -- 172.21.15.46:0/2252369194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5a14042070 con 0x7f5a181008b0 2024-09-10T04:43:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.678+0000 7f5a0effd640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5a0407abf0 0x7f5a0407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.678+0000 7f5a0effd640 1 -- 172.21.15.46:0/2252369194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5a140be7b0 con 0x7f5a181008b0 2024-09-10T04:43:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.678+0000 7f5a1d316640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5a0407abf0 0x7f5a0407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:50.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.679+0000 7f5a1d316640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5a0407abf0 0x7f5a0407d0b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5a181124f0 tx=0x7f5a08008040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:50.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.683+0000 7f5a0effd640 1 -- 172.21.15.46:0/2252369194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5a14089ef0 con 0x7f5a181008b0 2024-09-10T04:43:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.943+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f5a18102d70 con 0x7f5a181008b0 2024-09-10T04:43:50.945 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:50 smithi046 ceph-mon[109506]: pgmap v1146: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.943+0000 7f5a0effd640 1 -- 172.21.15.46:0/2252369194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f5a1408eda0 con 0x7f5a181008b0 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:43:50.946 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:43:50.947 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5a0407abf0 msgr2=0x7f5a0407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5a0407abf0 0x7f5a0407d0b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5a181124f0 tx=0x7f5a08008040 comp rx=0 tx=0).stop 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 msgr2=0x7f5a1819e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a1819e030 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5a14031460 tx=0x7f5a140048f0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 shutdown_connections 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5a0407abf0 0x7f5a0407d0b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a181011d0 0x7f5a1819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 --2- 172.21.15.46:0/2252369194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5a181008b0 0x7f5a1819e030 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 >> 172.21.15.46:0/2252369194 conn(0x7f5a180fc520 msgr2=0x7f5a18112b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 shutdown_connections 2024-09-10T04:43:50.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:50.945+0000 7f5a1fd7b640 1 -- 172.21.15.46:0/2252369194 wait complete. 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 -- 172.21.15.46:0/1757311560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100120 msgr2=0x7f7bd0100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/1757311560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100120 0x7f7bd0100500 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009880 tx=0x7f7bcc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 -- 172.21.15.46:0/1757311560 shutdown_connections 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/1757311560 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bd0100ad0 0x7f7bd0106610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/1757311560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100120 0x7f7bd0100500 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 -- 172.21.15.46:0/1757311560 >> 172.21.15.46:0/1757311560 conn(0x7f7bd00fbc90 msgr2=0x7f7bd00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 -- 172.21.15.46:0/1757311560 shutdown_connections 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.030+0000 7f7bd73fe640 1 -- 172.21.15.46:0/1757311560 wait complete. 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd73fe640 1 Processor -- start 2024-09-10T04:43:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd73fe640 1 -- start start 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd73fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bd0100120 0x7f7bd019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd73fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100ad0 0x7f7bd019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd73fe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010ee60 con 0x7f7bd0100ad0 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd73fe640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd010ece0 con 0x7f7bd0100120 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd4999640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100ad0 0x7f7bd019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd4999640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100ad0 0x7f7bd019dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60882/0 (socket says 172.21.15.46:60882) 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd4999640 1 -- 172.21.15.46:0/4073918291 learned_addr learned my addr 172.21.15.46:0/4073918291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd519a640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bd0100120 0x7f7bd019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd4999640 1 -- 172.21.15.46:0/4073918291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bd0100120 msgr2=0x7f7bd019d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd4999640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bd0100120 0x7f7bd019d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd4999640 1 -- 172.21.15.46:0/4073918291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd0197890 con 0x7f7bd0100ad0 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.031+0000 7f7bd4999640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100ad0 0x7f7bd019dc10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000d6f0 tx=0x7f7bc000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:51.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.032+0000 7f7bc67fc640 1 -- 172.21.15.46:0/4073918291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7bc0017070 con 0x7f7bd0100ad0 2024-09-10T04:43:51.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.032+0000 7f7bc67fc640 1 -- 172.21.15.46:0/4073918291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7bc000de00 con 0x7f7bd0100ad0 2024-09-10T04:43:51.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.032+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd0197b80 con 0x7f7bd0100ad0 2024-09-10T04:43:51.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.032+0000 7f7bc67fc640 1 -- 172.21.15.46:0/4073918291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7bc000f3c0 con 0x7f7bd0100ad0 2024-09-10T04:43:51.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.032+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd0198110 con 0x7f7bd0100ad0 2024-09-10T04:43:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.033+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd01031e0 con 0x7f7bd0100ad0 2024-09-10T04:43:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.034+0000 7f7bc67fc640 1 -- 172.21.15.46:0/4073918291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7bc001a070 con 0x7f7bd0100ad0 2024-09-10T04:43:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.034+0000 7f7bc67fc640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7bbc07ac40 0x7f7bbc07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.034+0000 7f7bc67fc640 1 -- 172.21.15.46:0/4073918291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7bc009d820 con 0x7f7bd0100ad0 2024-09-10T04:43:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.034+0000 7f7bd519a640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7bbc07ac40 0x7f7bbc07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.035+0000 7f7bd519a640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7bbc07ac40 0x7f7bbc07d100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc000c00 tx=0x7f7bcc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.038+0000 7f7bc67fc640 1 -- 172.21.15.46:0/4073918291 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7bc0021030 con 0x7f7bd0100ad0 2024-09-10T04:43:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:50 smithi070 ceph-mon[79774]: pgmap v1146: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:51.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.260+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7bd0109f70 con 0x7f7bd0100ad0 2024-09-10T04:43:51.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.260+0000 7f7bc67fc640 1 -- 172.21.15.46:0/4073918291 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+1996 (secure 0 0 0) 0x7f7bc0068fe0 con 0x7f7bd0100ad0 2024-09-10T04:43:51.262 INFO:teuthology.orchestra.run.smithi046.stdout:e27 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:29:16:475094+0000 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 18 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:13:25.852309+0000 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:43:51.263 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:up {} 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:failed 0,1 2024-09-10T04:43:51.264 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:43:51.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{-1:34328} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:51.266 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{-1:34360} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:51.266 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:51.266 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:43:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:43:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7bbc07ac40 msgr2=0x7f7bbc07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7bbc07ac40 0x7f7bbc07d100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc000c00 tx=0x7f7bcc031400 comp rx=0 tx=0).stop 2024-09-10T04:43:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100ad0 msgr2=0x7f7bd019dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100ad0 0x7f7bd019dc10 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7bc000d6f0 tx=0x7f7bc000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 shutdown_connections 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7bbc07ac40 0x7f7bbc07d100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bd0100ad0 0x7f7bd019dc10 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 --2- 172.21.15.46:0/4073918291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bd0100120 0x7f7bd019d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 >> 172.21.15.46:0/4073918291 conn(0x7f7bd00fbc90 msgr2=0x7f7bd00fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 shutdown_connections 2024-09-10T04:43:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.262+0000 7f7bd73fe640 1 -- 172.21.15.46:0/4073918291 wait complete. 2024-09-10T04:43:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 -- 172.21.15.46:0/2743271067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 msgr2=0x7fa234101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 --2- 172.21.15.46:0/2743271067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa234101d60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009880 tx=0x7fa21c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 -- 172.21.15.46:0/2743271067 shutdown_connections 2024-09-10T04:43:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 --2- 172.21.15.46:0/2743271067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa234101d60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 --2- 172.21.15.46:0/2743271067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa234107900 0x7fa234107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 -- 172.21.15.46:0/2743271067 >> 172.21.15.46:0/2743271067 conn(0x7fa2340fd600 msgr2=0x7fa2340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 -- 172.21.15.46:0/2743271067 shutdown_connections 2024-09-10T04:43:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.348+0000 7fa238d37640 1 -- 172.21.15.46:0/2743271067 wait complete. 2024-09-10T04:43:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa238d37640 1 Processor -- start 2024-09-10T04:43:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa238d37640 1 -- start start 2024-09-10T04:43:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa238d37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa23419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa238d37640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa234107900 0x7fa23419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa238d37640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410edd0 con 0x7fa234101900 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa238d37640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa23410ec50 con 0x7fa234107900 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa23259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa23419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa23259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa23419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60890/0 (socket says 172.21.15.46:60890) 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa23259c640 1 -- 172.21.15.46:0/2381458682 learned_addr learned my addr 172.21.15.46:0/2381458682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.349+0000 7fa22bfff640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa234107900 0x7fa23419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa23259c640 1 -- 172.21.15.46:0/2381458682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa234107900 msgr2=0x7fa23419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa23259c640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa234107900 0x7fa23419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa23259c640 1 -- 172.21.15.46:0/2381458682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2341977a0 con 0x7fa234101900 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa23259c640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa23419d620 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c8f0 tx=0x7fa22400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa22b7fe640 1 -- 172.21.15.46:0/2381458682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa2240079b0 con 0x7fa234101900 2024-09-10T04:43:51.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa22b7fe640 1 -- 172.21.15.46:0/2381458682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa2240054e0 con 0x7fa234101900 2024-09-10T04:43:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa234197a90 con 0x7fa234101900 2024-09-10T04:43:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa22b7fe640 1 -- 172.21.15.46:0/2381458682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa224004dd0 con 0x7fa234101900 2024-09-10T04:43:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.350+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa234197ff0 con 0x7fa234101900 2024-09-10T04:43:51.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.351+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa234103040 con 0x7fa234101900 2024-09-10T04:43:51.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.352+0000 7fa22b7fe640 1 -- 172.21.15.46:0/2381458682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa224005020 con 0x7fa234101900 2024-09-10T04:43:51.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.352+0000 7fa22b7fe640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa22007abf0 0x7fa22007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.352+0000 7fa22b7fe640 1 -- 172.21.15.46:0/2381458682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa22409db80 con 0x7fa234101900 2024-09-10T04:43:51.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.352+0000 7fa22bfff640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa22007abf0 0x7fa22007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.353+0000 7fa22bfff640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa22007abf0 0x7fa22007d0b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa234198af0 tx=0x7fa21c031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:51.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.357+0000 7fa22b7fe640 1 -- 172.21.15.46:0/2381458682 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa224014030 con 0x7fa234101900 2024-09-10T04:43:51.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.560+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa234198360 con 0x7fa22007abf0 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.561+0000 7fa22b7fe640 1 -- 172.21.15.46:0/2381458682 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+431 (secure 0 0 0) 0x7fa234198360 con 0x7fa22007abf0 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:43:51.563 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:43:51.564 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:43:51.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa22007abf0 msgr2=0x7fa22007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa22007abf0 0x7fa22007d0b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa234198af0 tx=0x7fa21c031420 comp rx=0 tx=0).stop 2024-09-10T04:43:51.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 msgr2=0x7fa23419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa23419d620 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa22400c8f0 tx=0x7fa22400cdb0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 shutdown_connections 2024-09-10T04:43:51.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa22007abf0 0x7fa22007d0b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa234107900 0x7fa23419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 --2- 172.21.15.46:0/2381458682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa234101900 0x7fa23419d620 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 >> 172.21.15.46:0/2381458682 conn(0x7fa2340fd600 msgr2=0x7fa2340fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:51.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 shutdown_connections 2024-09-10T04:43:51.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.564+0000 7fa238d37640 1 -- 172.21.15.46:0/2381458682 wait complete. 2024-09-10T04:43:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 -- 172.21.15.46:0/68170296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4107900 msgr2=0x7f30c4107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 --2- 172.21.15.46:0/68170296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4107900 0x7f30c4107ce0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f30b4009880 tx=0x7f30b402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 -- 172.21.15.46:0/68170296 shutdown_connections 2024-09-10T04:43:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 --2- 172.21.15.46:0/68170296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30c4101900 0x7f30c4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 --2- 172.21.15.46:0/68170296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4107900 0x7f30c4107ce0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 -- 172.21.15.46:0/68170296 >> 172.21.15.46:0/68170296 conn(0x7f30c40fd600 msgr2=0x7f30c40ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:51.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 -- 172.21.15.46:0/68170296 shutdown_connections 2024-09-10T04:43:51.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.648+0000 7f30ca034640 1 -- 172.21.15.46:0/68170296 wait complete. 2024-09-10T04:43:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30ca034640 1 Processor -- start 2024-09-10T04:43:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30ca034640 1 -- start start 2024-09-10T04:43:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30ca034640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4101900 0x7f30c419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30ca034640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30c4107900 0x7f30c419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30ca034640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f30c410edb0 con 0x7f30c4101900 2024-09-10T04:43:51.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30ca034640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f30c410ec30 con 0x7f30c4107900 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30c37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4101900 0x7f30c419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30c37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4101900 0x7f30c419d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60898/0 (socket says 172.21.15.46:60898) 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30c37fe640 1 -- 172.21.15.46:0/1492896660 learned_addr learned my addr 172.21.15.46:0/1492896660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30c2ffd640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30c4107900 0x7f30c419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.649+0000 7f30c37fe640 1 -- 172.21.15.46:0/1492896660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30c4107900 msgr2=0x7f30c419db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.650+0000 7f30c37fe640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30c4107900 0x7f30c419db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.650+0000 7f30c37fe640 1 -- 172.21.15.46:0/1492896660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c41977c0 con 0x7f30c4101900 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.650+0000 7f30c37fe640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4101900 0x7f30c419d640 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f30b4031460 tx=0x7f30b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.650+0000 7f30c0ff9640 1 -- 172.21.15.46:0/1492896660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f30b403f070 con 0x7f30c4101900 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.650+0000 7f30c0ff9640 1 -- 172.21.15.46:0/1492896660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f30b40044d0 con 0x7f30c4101900 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.650+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c4197a50 con 0x7f30c4101900 2024-09-10T04:43:51.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.650+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c4197f80 con 0x7f30c4101900 2024-09-10T04:43:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.651+0000 7f30c0ff9640 1 -- 172.21.15.46:0/1492896660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f30b4035030 con 0x7f30c4101900 2024-09-10T04:43:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.652+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c4103040 con 0x7f30c4101900 2024-09-10T04:43:51.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.652+0000 7f30c0ff9640 1 -- 172.21.15.46:0/1492896660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f30b4042070 con 0x7f30c4101900 2024-09-10T04:43:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.653+0000 7f30c0ff9640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30ac07ac50 0x7f30ac07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:43:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.653+0000 7f30c0ff9640 1 -- 172.21.15.46:0/1492896660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f30b40be710 con 0x7f30c4101900 2024-09-10T04:43:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.653+0000 7f30c2ffd640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30ac07ac50 0x7f30ac07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:43:51.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.653+0000 7f30c2ffd640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30ac07ac50 0x7f30ac07d110 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f30b8005e10 tx=0x7f30b8005da0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:43:51.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.657+0000 7f30c0ff9640 1 -- 172.21.15.46:0/1492896660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f30b4004cf0 con 0x7f30c4101900 2024-09-10T04:43:51.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:51 smithi046 ceph-mon[109506]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:51.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:51 smithi046 ceph-mon[109506]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:51.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:51 smithi046 ceph-mon[109506]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:51.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:51 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2252369194' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:43:51.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:51 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4073918291' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:43:51.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.918+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f30c4198cf0 con 0x7f30c4101900 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.919+0000 7f30c0ff9640 1 -- 172.21.15.46:0/1492896660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3795 (secure 0 0 0) 0x7f30b4089ed0 con 0x7f30c4101900 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:43:51.921 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmprit13xpm:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:43:51.922 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:43:51.923 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:43:51.923 INFO:teuthology.orchestra.run.smithi046.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T04:43:51.923 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T04:43:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.920+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30ac07ac50 msgr2=0x7f30ac07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.920+0000 7f30ca034640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30ac07ac50 0x7f30ac07d110 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f30b8005e10 tx=0x7f30b8005da0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.920+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4101900 msgr2=0x7f30c419d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.920+0000 7f30ca034640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4101900 0x7f30c419d640 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f30b4031460 tx=0x7f30b40048f0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.921+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 shutdown_connections 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.921+0000 7f30ca034640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f30ac07ac50 0x7f30ac07d110 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.921+0000 7f30ca034640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30c4107900 0x7f30c419db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.921+0000 7f30ca034640 1 --2- 172.21.15.46:0/1492896660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30c4101900 0x7f30c419d640 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.921+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 >> 172.21.15.46:0/1492896660 conn(0x7f30c40fd600 msgr2=0x7f30c40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.921+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 shutdown_connections 2024-09-10T04:43:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:43:51.921+0000 7f30ca034640 1 -- 172.21.15.46:0/1492896660 wait complete. 2024-09-10T04:43:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:51 smithi070 ceph-mon[79774]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:51 smithi070 ceph-mon[79774]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:51 smithi070 ceph-mon[79774]: from='client.35914 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:51 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2252369194' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:43:52.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:51 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4073918291' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:43:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:52 smithi070 ceph-mon[79774]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:52 smithi070 ceph-mon[79774]: pgmap v1147: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:52 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1492896660' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:43:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:52 smithi046 ceph-mon[109506]: from='client.35926 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:43:53.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:52 smithi046 ceph-mon[109506]: pgmap v1147: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:53.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:52 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1492896660' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:43:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:43:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:54 smithi046 ceph-mon[109506]: pgmap v1148: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:55.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:54 smithi070 ceph-mon[79774]: pgmap v1148: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:56 smithi070 ceph-mon[79774]: pgmap v1149: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:56 smithi046 ceph-mon[109506]: pgmap v1149: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:43:58 smithi070 ceph-mon[79774]: pgmap v1150: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:43:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:43:58 smithi046 ceph-mon[109506]: pgmap v1150: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:01.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:00 smithi070 ceph-mon[79774]: pgmap v1151: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:01.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:00 smithi046 ceph-mon[109506]: pgmap v1151: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:02 smithi070 ceph-mon[79774]: pgmap v1152: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:02 smithi046 ceph-mon[109506]: pgmap v1152: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:05.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:04 smithi070 ceph-mon[79774]: pgmap v1153: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:04 smithi046 ceph-mon[109506]: pgmap v1153: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:06 smithi070 ceph-mon[79774]: pgmap v1154: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:06 smithi046 ceph-mon[109506]: pgmap v1154: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:08 smithi070 ceph-mon[79774]: pgmap v1155: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:08 smithi046 ceph-mon[109506]: pgmap v1155: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:10 smithi070 ceph-mon[79774]: pgmap v1156: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:10 smithi046 ceph-mon[109506]: pgmap v1156: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:12 smithi070 ceph-mon[79774]: pgmap v1157: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:12 smithi046 ceph-mon[109506]: pgmap v1157: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:14 smithi070 ceph-mon[79774]: pgmap v1158: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:14 smithi046 ceph-mon[109506]: pgmap v1158: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: pgmap v1159: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.cpwyud"} : dispatch 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.cpwyud"}]': finished 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.hasdap"}]': finished 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.ehvjgr"} : dispatch 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.ehvjgr"}]': finished 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.myztiz"} : dispatch 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.myztiz"}]': finished 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: Standby daemon mds.cephfs.smithi046.hasdap assigned to filesystem cephfs as rank 0 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: Standby daemon mds.cephfs.smithi046.cpwyud assigned to filesystem cephfs as rank 1 2024-09-10T04:44:17.087 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T04:44:17.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T04:44:17.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:replay,1=cephfs.smithi046.cpwyud=up:replay} 2 up:standby 2024-09-10T04:44:17.088 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:16 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"} : dispatch 2024-09-10T04:44:17.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: pgmap v1159: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:17.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:17.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:17.088 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.cpwyud"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.cpwyud"}]': finished 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.hasdap"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi046.hasdap"}]': finished 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.ehvjgr"} : dispatch 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.ehvjgr"}]': finished 2024-09-10T04:44:17.089 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.myztiz"} : dispatch 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi070.myztiz"}]': finished 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: Standby daemon mds.cephfs.smithi046.hasdap assigned to filesystem cephfs as rank 0 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: Standby daemon mds.cephfs.smithi046.cpwyud assigned to filesystem cephfs as rank 1 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:replay,1=cephfs.smithi046.cpwyud=up:replay} 2 up:standby 2024-09-10T04:44:17.090 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:16 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"} : dispatch 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: pgmap v1160: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: pgmap v1161: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all mds 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:resolve 2024-09-10T04:44:17.802 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:replay,1=cephfs.smithi046.cpwyud=up:resolve} 2 up:standby 2024-09-10T04:44:17.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:17.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:17.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:17.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:17.803 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:17 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:18.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: pgmap v1160: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:18.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: pgmap v1161: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:18.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all mds 2024-09-10T04:44:18.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T04:44:18.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:resolve 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:replay,1=cephfs.smithi046.cpwyud=up:resolve} 2 up:standby 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:18.086 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:17 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:18 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all rgw 2024-09-10T04:44:18.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:18 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T04:44:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:18 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all rgw 2024-09-10T04:44:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:18 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T04:44:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:19 smithi070 ceph-mon[79774]: pgmap v1162: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:19 smithi070 ceph-mon[79774]: Upgrade: Updating ceph-exporter.smithi046 (1/2) 2024-09-10T04:44:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-10T04:44:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:19 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:20.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:19 smithi070 ceph-mon[79774]: Deploying daemon ceph-exporter.smithi046 on smithi046 2024-09-10T04:44:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:19 smithi046 ceph-mon[109506]: pgmap v1162: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-10T04:44:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:19 smithi046 ceph-mon[109506]: Upgrade: Updating ceph-exporter.smithi046 (1/2) 2024-09-10T04:44:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi046", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-10T04:44:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:19 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:20.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:19 smithi046 ceph-mon[109506]: Deploying daemon ceph-exporter.smithi046 on smithi046 2024-09-10T04:44:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 -- 172.21.15.46:0/661848993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9501033f0 msgr2=0x7fa95010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 --2- 172.21.15.46:0/661848993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9501033f0 0x7fa95010b8e0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fa944009880 tx=0x7fa94402a530 comp rx=0 tx=0).stop 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 -- 172.21.15.46:0/661848993 shutdown_connections 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 --2- 172.21.15.46:0/661848993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9501033f0 0x7fa95010b8e0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 --2- 172.21.15.46:0/661848993 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa950102ad0 0x7fa950102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 -- 172.21.15.46:0/661848993 >> 172.21.15.46:0/661848993 conn(0x7fa9500fc510 msgr2=0x7fa9500fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 -- 172.21.15.46:0/661848993 shutdown_connections 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 -- 172.21.15.46:0/661848993 wait complete. 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 Processor -- start 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.013+0000 7fa956897640 1 -- start start 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa956897640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa950102ad0 0x7fa95019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa956897640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9501033f0 0x7fa95019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa956897640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa95010ee00 con 0x7fa950102ad0 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa956897640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa95010ec80 con 0x7fa9501033f0 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa94ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa950102ad0 0x7fa95019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa94ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa950102ad0 0x7fa95019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56742/0 (socket says 172.21.15.46:56742) 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa94ffff640 1 -- 172.21.15.46:0/833293885 learned_addr learned my addr 172.21.15.46:0/833293885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.014+0000 7fa94f7fe640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9501033f0 0x7fa95019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94ffff640 1 -- 172.21.15.46:0/833293885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9501033f0 msgr2=0x7fa95019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94ffff640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9501033f0 0x7fa95019dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94ffff640 1 -- 172.21.15.46:0/833293885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa950197830 con 0x7fa950102ad0 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94f7fe640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9501033f0 0x7fa95019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:44:22.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94ffff640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa950102ad0 0x7fa95019d6b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa94000d6f0 tx=0x7fa94000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94d7fa640 1 -- 172.21.15.46:0/833293885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa940017070 con 0x7fa950102ad0 2024-09-10T04:44:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94d7fa640 1 -- 172.21.15.46:0/833293885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa94000de00 con 0x7fa950102ad0 2024-09-10T04:44:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa950197b20 con 0x7fa950102ad0 2024-09-10T04:44:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa94d7fa640 1 -- 172.21.15.46:0/833293885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa940004bb0 con 0x7fa950102ad0 2024-09-10T04:44:22.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.015+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa950198080 con 0x7fa950102ad0 2024-09-10T04:44:22.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.016+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95006ada0 con 0x7fa950102ad0 2024-09-10T04:44:22.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.018+0000 7fa94d7fa640 1 -- 172.21.15.46:0/833293885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa940009a30 con 0x7fa950102ad0 2024-09-10T04:44:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.018+0000 7fa94d7fa640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa93807ac40 0x7fa93807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.018+0000 7fa94d7fa640 1 -- 172.21.15.46:0/833293885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa94009d7f0 con 0x7fa950102ad0 2024-09-10T04:44:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.019+0000 7fa94f7fe640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa93807ac40 0x7fa93807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.019+0000 7fa94f7fe640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa93807ac40 0x7fa93807d100 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa950198b80 tx=0x7fa944031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.023+0000 7fa94d7fa640 1 -- 172.21.15.46:0/833293885 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa940068fb0 con 0x7fa950102ad0 2024-09-10T04:44:22.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:21 smithi070 ceph-mon[79774]: pgmap v1163: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-09-10T04:44:22.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:21 smithi046 ceph-mon[109506]: pgmap v1163: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 9.2 MiB/s rd, 3 op/s 2024-09-10T04:44:22.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.253+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fa950198330 con 0x7fa93807ac40 2024-09-10T04:44:22.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.254+0000 7fa94d7fa640 1 -- 172.21.15.46:0/833293885 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+441 (secure 0 0 0) 0x7fa950198330 con 0x7fa93807ac40 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa93807ac40 msgr2=0x7fa93807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa93807ac40 0x7fa93807d100 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa950198b80 tx=0x7fa944031400 comp rx=0 tx=0).stop 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa950102ad0 msgr2=0x7fa95019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa950102ad0 0x7fa95019d6b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fa94000d6f0 tx=0x7fa94000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 shutdown_connections 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa93807ac40 0x7fa93807d100 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9501033f0 0x7fa95019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 --2- 172.21.15.46:0/833293885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa950102ad0 0x7fa95019d6b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 >> 172.21.15.46:0/833293885 conn(0x7fa9500fc510 msgr2=0x7fa950103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 shutdown_connections 2024-09-10T04:44:22.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.257+0000 7fa956897640 1 -- 172.21.15.46:0/833293885 wait complete. 2024-09-10T04:44:22.267 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:44:22.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 -- 172.21.15.46:0/605224772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978102b50 msgr2=0x7f6978102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 --2- 172.21.15.46:0/605224772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978102b50 0x7f6978102f30 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6968009880 tx=0x7f696802a530 comp rx=0 tx=0).stop 2024-09-10T04:44:22.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 -- 172.21.15.46:0/605224772 shutdown_connections 2024-09-10T04:44:22.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 --2- 172.21.15.46:0/605224772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6978103470 0x7f697810b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 --2- 172.21.15.46:0/605224772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978102b50 0x7f6978102f30 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 -- 172.21.15.46:0/605224772 >> 172.21.15.46:0/605224772 conn(0x7f69780fc550 msgr2=0x7f69780fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:22.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 -- 172.21.15.46:0/605224772 shutdown_connections 2024-09-10T04:44:22.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.343+0000 7f697cb83640 1 -- 172.21.15.46:0/605224772 wait complete. 2024-09-10T04:44:22.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f697cb83640 1 Processor -- start 2024-09-10T04:44:22.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f697cb83640 1 -- start start 2024-09-10T04:44:22.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f697cb83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6978102b50 0x7f697819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f697cb83640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978103470 0x7f697819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f697cb83640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f697810edc0 con 0x7f6978103470 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f697cb83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f697810ec40 con 0x7f6978102b50 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f6975d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978103470 0x7f697819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f6975d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978103470 0x7f697819db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56762/0 (socket says 172.21.15.46:56762) 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f6975d9b640 1 -- 172.21.15.46:0/1102508866 learned_addr learned my addr 172.21.15.46:0/1102508866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.344+0000 7f697659c640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6978102b50 0x7f697819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f6975d9b640 1 -- 172.21.15.46:0/1102508866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6978102b50 msgr2=0x7f697819d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f6975d9b640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6978102b50 0x7f697819d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f6975d9b640 1 -- 172.21.15.46:0/1102508866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69781977b0 con 0x7f6978103470 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f6975d9b640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978103470 0x7f697819db70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f696c00e970 tx=0x7f696c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f69677fe640 1 -- 172.21.15.46:0/1102508866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f696c00cac0 con 0x7f6978103470 2024-09-10T04:44:22.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f69677fe640 1 -- 172.21.15.46:0/1102508866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f696c00cc60 con 0x7f6978103470 2024-09-10T04:44:22.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6978197aa0 con 0x7f6978103470 2024-09-10T04:44:22.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f69677fe640 1 -- 172.21.15.46:0/1102508866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f696c018720 con 0x7f6978103470 2024-09-10T04:44:22.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.345+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6978198030 con 0x7f6978103470 2024-09-10T04:44:22.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.346+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697806a4e0 con 0x7f6978103470 2024-09-10T04:44:22.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.347+0000 7f69677fe640 1 -- 172.21.15.46:0/1102508866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f696c0188e0 con 0x7f6978103470 2024-09-10T04:44:22.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.347+0000 7f69677fe640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696007ac40 0x7f696007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.348+0000 7f69677fe640 1 -- 172.21.15.46:0/1102508866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f696c09e250 con 0x7f6978103470 2024-09-10T04:44:22.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.348+0000 7f697659c640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696007ac40 0x7f696007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.348+0000 7f697659c640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696007ac40 0x7f696007d100 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f69680096c0 tx=0x7f6968004f30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.352+0000 7f69677fe640 1 -- 172.21.15.46:0/1102508866 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f696c0699b0 con 0x7f6978103470 2024-09-10T04:44:22.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.556+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f69781983a0 con 0x7f696007ac40 2024-09-10T04:44:22.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.557+0000 7f69677fe640 1 -- 172.21.15.46:0/1102508866 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+441 (secure 0 0 0) 0x7f69781983a0 con 0x7f696007ac40 2024-09-10T04:44:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696007ac40 msgr2=0x7f696007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696007ac40 0x7f696007d100 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f69680096c0 tx=0x7f6968004f30 comp rx=0 tx=0).stop 2024-09-10T04:44:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978103470 msgr2=0x7f697819db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978103470 0x7f697819db70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f696c00e970 tx=0x7f696c00ee30 comp rx=0 tx=0).stop 2024-09-10T04:44:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 shutdown_connections 2024-09-10T04:44:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f696007ac40 0x7f696007d100 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6978103470 0x7f697819db70 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 --2- 172.21.15.46:0/1102508866 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6978102b50 0x7f697819d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 >> 172.21.15.46:0/1102508866 conn(0x7f69780fc550 msgr2=0x7f6978108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:22.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 shutdown_connections 2024-09-10T04:44:22.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.561+0000 7f697cb83640 1 -- 172.21.15.46:0/1102508866 wait complete. 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 -- 172.21.15.46:0/2122752316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70102ab0 msgr2=0x7f2e70102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 --2- 172.21.15.46:0/2122752316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70102ab0 0x7f2e70102e90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2e60009880 tx=0x7f2e6002a530 comp rx=0 tx=0).stop 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 -- 172.21.15.46:0/2122752316 shutdown_connections 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 --2- 172.21.15.46:0/2122752316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e701033d0 0x7f2e7010b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 --2- 172.21.15.46:0/2122752316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e70102ab0 0x7f2e70102e90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 -- 172.21.15.46:0/2122752316 >> 172.21.15.46:0/2122752316 conn(0x7f2e700fc550 msgr2=0x7f2e700fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 -- 172.21.15.46:0/2122752316 shutdown_connections 2024-09-10T04:44:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.651+0000 7f2e7544b640 1 -- 172.21.15.46:0/2122752316 wait complete. 2024-09-10T04:44:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e7544b640 1 Processor -- start 2024-09-10T04:44:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e7544b640 1 -- start start 2024-09-10T04:44:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e7544b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e70102ab0 0x7f2e7019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e7544b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e701033d0 0x7f2e7019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e7544b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7010edc0 con 0x7f2e701033d0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e7544b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7010ec40 con 0x7f2e70102ab0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e6e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e701033d0 0x7f2e7019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e6e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e701033d0 0x7f2e7019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56776/0 (socket says 172.21.15.46:56776) 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e6e7fc640 1 -- 172.21.15.46:0/820759088 learned_addr learned my addr 172.21.15.46:0/820759088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e6effd640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e70102ab0 0x7f2e7019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e6e7fc640 1 -- 172.21.15.46:0/820759088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e70102ab0 msgr2=0x7f2e7019d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e6e7fc640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e70102ab0 0x7f2e7019d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.652+0000 7f2e6e7fc640 1 -- 172.21.15.46:0/820759088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e701977b0 con 0x7f2e701033d0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.653+0000 7f2e6e7fc640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e701033d0 0x7f2e7019db70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400e970 tx=0x7f2e6400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.653+0000 7f2e57fff640 1 -- 172.21.15.46:0/820759088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e64009590 con 0x7f2e701033d0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.653+0000 7f2e57fff640 1 -- 172.21.15.46:0/820759088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2e64010430 con 0x7f2e701033d0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.653+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e70197aa0 con 0x7f2e701033d0 2024-09-10T04:44:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.653+0000 7f2e57fff640 1 -- 172.21.15.46:0/820759088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f2e64018630 con 0x7f2e701033d0 2024-09-10T04:44:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.653+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e70198030 con 0x7f2e701033d0 2024-09-10T04:44:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.655+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7006a580 con 0x7f2e701033d0 2024-09-10T04:44:22.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.656+0000 7f2e57fff640 1 -- 172.21.15.46:0/820759088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f2e640188f0 con 0x7f2e701033d0 2024-09-10T04:44:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.656+0000 7f2e57fff640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e5807abf0 0x7f2e5807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.656+0000 7f2e57fff640 1 -- 172.21.15.46:0/820759088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f2e6409e330 con 0x7f2e701033d0 2024-09-10T04:44:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.656+0000 7f2e6effd640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e5807abf0 0x7f2e5807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.657+0000 7f2e6effd640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e5807abf0 0x7f2e5807d0b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2e600096c0 tx=0x7f2e60031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.661+0000 7f2e57fff640 1 -- 172.21.15.46:0/820759088 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f2e6406e990 con 0x7f2e701033d0 2024-09-10T04:44:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.868+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2e70198250 con 0x7f2e5807abf0 2024-09-10T04:44:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.878+0000 7f2e57fff640 1 -- 172.21.15.46:0/820759088 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f2e70198250 con 0x7f2e5807abf0 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (35m) 15m ago 47m 20.9M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (38m) 15m ago 47m 10.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee9bd458f469 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (38m) 15m ago 46m 10.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 1b46dc2a5146 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (34m) 15m ago 47m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (34m) 15m ago 46m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (35m) 15m ago 47m 79.4M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (30m) 15m ago 41m 23.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (30m) 15m ago 41m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (30m) 15m ago 41m 20.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:44:22.881 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (15m) 15m ago 41m 14.2M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (37m) 15m ago 48m 533M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (37m) 15m ago 46m 463M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (34m) 15m ago 48m 68.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (34m) 15m ago 46m 50.0M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (36m) 15m ago 47m 11.2M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (36m) 15m ago 46m 12.4M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (33m) 15m ago 45m 91.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (33m) 15m ago 44m 95.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (32m) 15m ago 44m 87.0M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (32m) 15m ago 43m 95.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (31m) 15m ago 43m 89.7M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (31m) 15m ago 42m 82.6M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:44:22.882 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (36m) 15m ago 46m 69.2M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e5807abf0 msgr2=0x7f2e5807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e5807abf0 0x7f2e5807d0b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2e600096c0 tx=0x7f2e60031400 comp rx=0 tx=0).stop 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e701033d0 msgr2=0x7f2e7019db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e701033d0 0x7f2e7019db70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2e6400e970 tx=0x7f2e6400ee30 comp rx=0 tx=0).stop 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 shutdown_connections 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f2e5807abf0 0x7f2e5807d0b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e701033d0 0x7f2e7019db70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 --2- 172.21.15.46:0/820759088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e70102ab0 0x7f2e7019d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.881+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 >> 172.21.15.46:0/820759088 conn(0x7f2e700fc550 msgr2=0x7f2e70108ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.882+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 shutdown_connections 2024-09-10T04:44:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.882+0000 7f2e7544b640 1 -- 172.21.15.46:0/820759088 wait complete. 2024-09-10T04:44:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.978+0000 7fade06f0640 1 -- 172.21.15.46:0/1169958642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 msgr2=0x7fadd8100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.978+0000 7fade06f0640 1 --2- 172.21.15.46:0/1169958642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd8100a80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009880 tx=0x7fadd402a490 comp rx=0 tx=0).stop 2024-09-10T04:44:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 -- 172.21.15.46:0/1169958642 shutdown_connections 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 --2- 172.21.15.46:0/1169958642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8101050 0x7fadd810dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 --2- 172.21.15.46:0/1169958642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd8100a80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 -- 172.21.15.46:0/1169958642 >> 172.21.15.46:0/1169958642 conn(0x7fadd80fc550 msgr2=0x7fadd80fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 -- 172.21.15.46:0/1169958642 shutdown_connections 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 -- 172.21.15.46:0/1169958642 wait complete. 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 Processor -- start 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 -- start start 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.979+0000 7fade06f0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8101050 0x7fadd819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fade06f0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8110f10 con 0x7fadd81006a0 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fade06f0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8110d90 con 0x7fadd8101050 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadde48c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadde48c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56786/0 (socket says 172.21.15.46:56786) 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadde48c640 1 -- 172.21.15.46:0/187304198 learned_addr learned my addr 172.21.15.46:0/187304198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadddc8b640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8101050 0x7fadd819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadde48c640 1 -- 172.21.15.46:0/187304198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8101050 msgr2=0x7fadd819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadde48c640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8101050 0x7fadd819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadde48c640 1 -- 172.21.15.46:0/187304198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd81999a0 con 0x7fadd81006a0 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadde48c640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd819f7e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fadd4031460 tx=0x7fadd40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadcf7fe640 1 -- 172.21.15.46:0/187304198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fadd403f070 con 0x7fadd81006a0 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.980+0000 7fadcf7fe640 1 -- 172.21.15.46:0/187304198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fadd4004b10 con 0x7fadd81006a0 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.981+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd8199c90 con 0x7fadd81006a0 2024-09-10T04:44:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.981+0000 7fadcf7fe640 1 -- 172.21.15.46:0/187304198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fadd4035030 con 0x7fadd81006a0 2024-09-10T04:44:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.981+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd819a220 con 0x7fadd81006a0 2024-09-10T04:44:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.982+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd806a520 con 0x7fadd81006a0 2024-09-10T04:44:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.982+0000 7fadcf7fe640 1 -- 172.21.15.46:0/187304198 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fadd4042070 con 0x7fadd81006a0 2024-09-10T04:44:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.983+0000 7fadcf7fe640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fadc407ac50 0x7fadc407d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.983+0000 7fadcf7fe640 1 -- 172.21.15.46:0/187304198 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fadd40be650 con 0x7fadd81006a0 2024-09-10T04:44:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.983+0000 7fadddc8b640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fadc407ac50 0x7fadc407d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.984+0000 7fadddc8b640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fadc407ac50 0x7fadc407d110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fadd80fef00 tx=0x7fadc800b410 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:22.988+0000 7fadcf7fe640 1 -- 172.21.15.46:0/187304198 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fadd4089e10 con 0x7fadd81006a0 2024-09-10T04:44:23.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.351+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fadd819aed0 con 0x7fadd81006a0 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:44:23.356 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:44:23.357 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:44:23.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:44:23.357 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:44:23.357 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:44:23.357 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:44:23.357 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:44:23.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.353+0000 7fadcf7fe640 1 -- 172.21.15.46:0/187304198 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fadd408ecc0 con 0x7fadd81006a0 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fadc407ac50 msgr2=0x7fadc407d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fadc407ac50 0x7fadc407d110 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fadd80fef00 tx=0x7fadc800b410 comp rx=0 tx=0).stop 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 msgr2=0x7fadd819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd819f7e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fadd4031460 tx=0x7fadd40045e0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 shutdown_connections 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fadc407ac50 0x7fadc407d110 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadd8101050 0x7fadd819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 --2- 172.21.15.46:0/187304198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fadd81006a0 0x7fadd819f7e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 >> 172.21.15.46:0/187304198 conn(0x7fadd80fc550 msgr2=0x7fadd80fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 shutdown_connections 2024-09-10T04:44:23.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.357+0000 7fade06f0640 1 -- 172.21.15.46:0/187304198 wait complete. 2024-09-10T04:44:23.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 -- 172.21.15.46:0/528455994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0101900 msgr2=0x7faaa0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 --2- 172.21.15.46:0/528455994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0101900 0x7faaa0101d60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7faa900097e0 tx=0x7faa9002a3f0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 -- 172.21.15.46:0/528455994 shutdown_connections 2024-09-10T04:44:23.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 --2- 172.21.15.46:0/528455994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0101900 0x7faaa0101d60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 --2- 172.21.15.46:0/528455994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0107900 0x7faaa0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 -- 172.21.15.46:0/528455994 >> 172.21.15.46:0/528455994 conn(0x7faaa00fd5e0 msgr2=0x7faaa00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:23.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 -- 172.21.15.46:0/528455994 shutdown_connections 2024-09-10T04:44:23.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.490+0000 7faaa8260640 1 -- 172.21.15.46:0/528455994 wait complete. 2024-09-10T04:44:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa8260640 1 Processor -- start 2024-09-10T04:44:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa8260640 1 -- start start 2024-09-10T04:44:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa8260640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0101900 0x7faaa019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa8260640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0107900 0x7faaa019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa8260640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faaa010edd0 con 0x7faaa0101900 2024-09-10T04:44:23.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa8260640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7faaa010ec50 con 0x7faaa0107900 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa5ffc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0101900 0x7faaa019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.491+0000 7faaa5ffc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0101900 0x7faaa019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56804/0 (socket says 172.21.15.46:56804) 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faaa5ffc640 1 -- 172.21.15.46:0/217015902 learned_addr learned my addr 172.21.15.46:0/217015902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faaa57fb640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0107900 0x7faaa019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faaa5ffc640 1 -- 172.21.15.46:0/217015902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0107900 msgr2=0x7faaa019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faaa5ffc640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0107900 0x7faaa019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faaa5ffc640 1 -- 172.21.15.46:0/217015902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa01977a0 con 0x7faaa0101900 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faaa5ffc640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0101900 0x7faaa019d620 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00d6f0 tx=0x7faa9c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faa96ffd640 1 -- 172.21.15.46:0/217015902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa9c017070 con 0x7faaa0101900 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faa96ffd640 1 -- 172.21.15.46:0/217015902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faa9c00de00 con 0x7faaa0101900 2024-09-10T04:44:23.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.492+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa0197a90 con 0x7faaa0101900 2024-09-10T04:44:23.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.493+0000 7faa96ffd640 1 -- 172.21.15.46:0/217015902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7faa9c004b50 con 0x7faaa0101900 2024-09-10T04:44:23.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.493+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa0197ff0 con 0x7faaa0101900 2024-09-10T04:44:23.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.494+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa0103040 con 0x7faaa0101900 2024-09-10T04:44:23.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.495+0000 7faa96ffd640 1 -- 172.21.15.46:0/217015902 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7faa9c0053b0 con 0x7faaa0101900 2024-09-10T04:44:23.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.496+0000 7faa96ffd640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa8c07abf0 0x7faa8c07d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:23.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.496+0000 7faaa57fb640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa8c07abf0 0x7faa8c07d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:23.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.496+0000 7faa96ffd640 1 -- 172.21.15.46:0/217015902 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7faa9c09d430 con 0x7faaa0101900 2024-09-10T04:44:23.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.497+0000 7faaa57fb640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa8c07abf0 0x7faa8c07d0b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faaa0198af0 tx=0x7faa900058b0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:23.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.505+0000 7faa96ffd640 1 -- 172.21.15.46:0/217015902 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7faa9c068bf0 con 0x7faaa0101900 2024-09-10T04:44:23.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.754+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7faaa0109e70 con 0x7faaa0101900 2024-09-10T04:44:23.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.755+0000 7faa96ffd640 1 -- 172.21.15.46:0/217015902 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+2037 (secure 0 0 0) 0x7faa9c06daa0 con 0x7faaa0101900 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:e29 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:44:17:123852+0000 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:44:23.757 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 29 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:44:16.133219+0000 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:44:23.758 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=34328,1=34360} 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:44:23.759 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 0 members: 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:34328} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{1:34360} state up:resolve seq 458 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:44:23.760 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{-1:44341} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:44:23.761 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{-1:44967} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:44:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T04:44:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa8c07abf0 msgr2=0x7faa8c07d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa8c07abf0 0x7faa8c07d0b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faaa0198af0 tx=0x7faa900058b0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0101900 msgr2=0x7faaa019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0101900 0x7faaa019d620 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00d6f0 tx=0x7faa9c00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 shutdown_connections 2024-09-10T04:44:23.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7faa8c07abf0 0x7faa8c07d0b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faaa0107900 0x7faaa019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 --2- 172.21.15.46:0/217015902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faaa0101900 0x7faaa019d620 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 >> 172.21.15.46:0/217015902 conn(0x7faaa00fd5e0 msgr2=0x7faaa00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:23.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 shutdown_connections 2024-09-10T04:44:23.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.757+0000 7faaa8260640 1 -- 172.21.15.46:0/217015902 wait complete. 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 -- 172.21.15.46:0/3281154914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 msgr2=0x7fc52010b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 --2- 172.21.15.46:0/3281154914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52010b960 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009880 tx=0x7fc50c02a530 comp rx=0 tx=0).stop 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 -- 172.21.15.46:0/3281154914 shutdown_connections 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 --2- 172.21.15.46:0/3281154914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52010b960 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 --2- 172.21.15.46:0/3281154914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520102b50 0x7fc520102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 -- 172.21.15.46:0/3281154914 >> 172.21.15.46:0/3281154914 conn(0x7fc5200fc550 msgr2=0x7fc5200fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 -- 172.21.15.46:0/3281154914 shutdown_connections 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.845+0000 7fc527101640 1 -- 172.21.15.46:0/3281154914 wait complete. 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc527101640 1 Processor -- start 2024-09-10T04:44:23.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc527101640 1 -- start start 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc527101640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520102b50 0x7fc52019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc527101640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc527101640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc52010ede0 con 0x7fc520103470 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc527101640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc52010ec60 con 0x7fc520102b50 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52019db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56826/0 (socket says 172.21.15.46:56826) 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc524e9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520102b50 0x7fc52019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.846+0000 7fc51ffff640 1 -- 172.21.15.46:0/976765702 learned_addr learned my addr 172.21.15.46:0/976765702 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:23.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc524e9d640 1 -- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 msgr2=0x7fc52019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc524e9d640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc524e9d640 1 -- 172.21.15.46:0/976765702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5201977b0 con 0x7fc520102b50 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc51ffff640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc524e9d640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520102b50 0x7fc52019d630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc514009b50 tx=0x7fc51400b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc51dffb640 1 -- 172.21.15.46:0/976765702 <== mon.1 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc51401e070 con 0x7fc520102b50 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc51dffb640 1 -- 172.21.15.46:0/976765702 <== mon.1 v2:172.21.15.70:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc514009e50 con 0x7fc520102b50 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.847+0000 7fc51dffb640 1 -- 172.21.15.46:0/976765702 <== mon.1 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc514019450 con 0x7fc520102b50 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.848+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc520197a40 con 0x7fc520102b50 2024-09-10T04:44:23.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.848+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc520197f40 con 0x7fc520102b50 2024-09-10T04:44:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.849+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52006a4e0 con 0x7fc520102b50 2024-09-10T04:44:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.849+0000 7fc51dffb640 1 -- 172.21.15.46:0/976765702 <== mon.1 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc514021070 con 0x7fc520102b50 2024-09-10T04:44:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.850+0000 7fc51dffb640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc51007ac40 0x7fc51007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.850+0000 7fc51dffb640 1 -- 172.21.15.46:0/976765702 <== mon.1 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc514016070 con 0x7fc520102b50 2024-09-10T04:44:23.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.850+0000 7fc51ffff640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc51007ac40 0x7fc51007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.851+0000 7fc51ffff640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc51007ac40 0x7fc51007d100 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc520198b00 tx=0x7fc50c0058d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:23.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:23.854+0000 7fc51dffb640 1 -- 172.21.15.46:0/976765702 <== mon.1 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc51406cfb0 con 0x7fc520102b50 2024-09-10T04:44:24.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.063+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fc5201982b0 con 0x7fc51007ac40 2024-09-10T04:44:24.066 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: pgmap v1164: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 21 MiB/s rd, 8 op/s 2024-09-10T04:44:24.066 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:24.066 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:24.066 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:24.066 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:24.067 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:24.067 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/187304198' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:24.067 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:23 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/217015902' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:44:24.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.065+0000 7fc51dffb640 1 -- 172.21.15.46:0/976765702 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+441 (secure 0 0 0) 0x7fc5201982b0 con 0x7fc51007ac40 2024-09-10T04:44:24.068 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:44:24.068 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca", 2024-09-10T04:44:24.068 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": true, 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [ 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mon", 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: "osd", 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr", 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: "mds", 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: "crash" 2024-09-10T04:44:24.069 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-09-10T04:44:24.070 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T04:44:24.070 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-09-10T04:44:24.070 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:44:24.070 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:44:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc51007ac40 msgr2=0x7fc51007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc51007ac40 0x7fc51007d100 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc520198b00 tx=0x7fc50c0058d0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520102b50 msgr2=0x7fc52019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520102b50 0x7fc52019d630 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc514009b50 tx=0x7fc51400b690 comp rx=0 tx=0).stop 2024-09-10T04:44:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 shutdown_connections 2024-09-10T04:44:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc51007ac40 0x7fc51007d100 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc520103470 0x7fc52019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 --2- 172.21.15.46:0/976765702 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc520102b50 0x7fc52019d630 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 >> 172.21.15.46:0/976765702 conn(0x7fc5200fc550 msgr2=0x7fc520103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 shutdown_connections 2024-09-10T04:44:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.068+0000 7fc527101640 1 -- 172.21.15.46:0/976765702 wait complete. 2024-09-10T04:44:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 -- 172.21.15.46:0/2196590912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 msgr2=0x7f4e0c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 --2- 172.21.15.46:0/2196590912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c107ce0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009880 tx=0x7f4df402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 -- 172.21.15.46:0/2196590912 shutdown_connections 2024-09-10T04:44:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 --2- 172.21.15.46:0/2196590912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c101900 0x7f4e0c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 --2- 172.21.15.46:0/2196590912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c107ce0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 -- 172.21.15.46:0/2196590912 >> 172.21.15.46:0/2196590912 conn(0x7f4e0c0fd600 msgr2=0x7f4e0c0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 -- 172.21.15.46:0/2196590912 shutdown_connections 2024-09-10T04:44:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 -- 172.21.15.46:0/2196590912 wait complete. 2024-09-10T04:44:24.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.156+0000 7f4e11de2640 1 Processor -- start 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e11de2640 1 -- start start 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e11de2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c101900 0x7f4e0c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e11de2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e11de2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c10edd0 con 0x7f4e0c107900 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e11de2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c10ec50 con 0x7f4e0c101900 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56852/0 (socket says 172.21.15.46:56852) 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0affd640 1 -- 172.21.15.46:0/3228107361 learned_addr learned my addr 172.21.15.46:0/3228107361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:24.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0b7fe640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c101900 0x7f4e0c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0affd640 1 -- 172.21.15.46:0/3228107361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c101900 msgr2=0x7f4e0c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0affd640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c101900 0x7f4e0c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0affd640 1 -- 172.21.15.46:0/3228107361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c074f70 con 0x7f4e0c107900 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.157+0000 7f4e0b7fe640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c101900 0x7f4e0c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.158+0000 7f4e0affd640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c074a30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00d6f0 tx=0x7f4dfc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.158+0000 7f4e08ff9640 1 -- 172.21.15.46:0/3228107361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4dfc017070 con 0x7f4e0c107900 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.158+0000 7f4e08ff9640 1 -- 172.21.15.46:0/3228107361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4dfc00de00 con 0x7f4e0c107900 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.158+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c075260 con 0x7f4e0c107900 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.158+0000 7f4e08ff9640 1 -- 172.21.15.46:0/3228107361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4dfc004bb0 con 0x7f4e0c107900 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.158+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c1a3e40 con 0x7f4e0c107900 2024-09-10T04:44:24.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.159+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0c103040 con 0x7f4e0c107900 2024-09-10T04:44:24.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.160+0000 7f4e08ff9640 1 -- 172.21.15.46:0/3228107361 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4dfc004e60 con 0x7f4e0c107900 2024-09-10T04:44:24.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.161+0000 7f4e08ff9640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4df807abf0 0x7f4df807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:24.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.161+0000 7f4e08ff9640 1 -- 172.21.15.46:0/3228107361 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4dfc09d8d0 con 0x7f4e0c107900 2024-09-10T04:44:24.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.161+0000 7f4e0b7fe640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4df807abf0 0x7f4df807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:24.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.161+0000 7f4e0b7fe640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4df807abf0 0x7f4df807d0b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c00 tx=0x7f4df4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:24.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.165+0000 7f4e08ff9640 1 -- 172.21.15.46:0/3228107361 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4dfc069010 con 0x7f4e0c107900 2024-09-10T04:44:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: pgmap v1164: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 21 MiB/s rd, 8 op/s 2024-09-10T04:44:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: from='client.35942 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:24.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:24.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:24.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/187304198' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:24.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:23 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/217015902' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:44:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.423+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7f4e0c109e70 con 0x7f4e0c107900 2024-09-10T04:44:24.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.423+0000 7f4e08ff9640 1 -- 172.21.15.46:0/3228107361 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+3669 (secure 0 0 0) 0x7f4dfc06dec0 con 0x7f4e0c107900 2024-09-10T04:44:24.426 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-09-10T04:44:24.426 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T04:44:24.426 INFO:teuthology.orchestra.run.smithi046.stdout: host smithi070 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi070/config 2024-09-10T04:44:24.426 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T04:44:24.426 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp71ave4iu:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:44:24.426 INFO:teuthology.orchestra.run.smithi046.stdout:Traceback (most recent call last): 2024-09-10T04:44:24.426 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: return _run_code(code, main_globals, None, 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: exec(code, run_globals) 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5628, in 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 5616, in main 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 414, in _infer_config 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 329, in _infer_fsid 2024-09-10T04:44:24.427 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 442, in _infer_image 2024-09-10T04:44:24.428 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 316, in _validate_fsid 2024-09-10T04:44:24.428 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/__main__.py", line 3353, in command_ceph_volume 2024-09-10T04:44:24.428 INFO:teuthology.orchestra.run.smithi046.stdout: File "/tmp/tmp59vjo1x_.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T04:44:24.428 INFO:teuthology.orchestra.run.smithi046.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca -e NODE_NAME=smithi070 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp71ave4iu:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca inventory --format=json-pretty --filter-for-batch 2024-09-10T04:44:24.428 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T04:44:24.428 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs is degraded 2024-09-10T04:44:24.428 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:44:24.429 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:44:24.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.425+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4df807abf0 msgr2=0x7f4df807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.425+0000 7f4e11de2640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4df807abf0 0x7f4df807d0b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c00 tx=0x7f4df4031400 comp rx=0 tx=0).stop 2024-09-10T04:44:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.425+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 msgr2=0x7f4e0c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.425+0000 7f4e11de2640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c074a30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00d6f0 tx=0x7f4dfc00dbb0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.426+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 shutdown_connections 2024-09-10T04:44:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.426+0000 7f4e11de2640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4df807abf0 0x7f4df807d0b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.426+0000 7f4e11de2640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e0c107900 0x7f4e0c074a30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.426+0000 7f4e11de2640 1 --2- 172.21.15.46:0/3228107361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c101900 0x7f4e0c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.426+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 >> 172.21.15.46:0/3228107361 conn(0x7f4e0c0fd600 msgr2=0x7f4e0c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.426+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 shutdown_connections 2024-09-10T04:44:24.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:24.426+0000 7f4e11de2640 1 -- 172.21.15.46:0/3228107361 wait complete. 2024-09-10T04:44:25.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:24 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3228107361' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:44:25.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:24 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3228107361' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:44:26.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:25 smithi046 ceph-mon[109506]: pgmap v1165: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 22 MiB/s rd, 8 op/s 2024-09-10T04:44:26.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:25 smithi046 ceph-mon[109506]: from='client.45599 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:26.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:25 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:resolve 2024-09-10T04:44:26.278 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:25 smithi046 ceph-mon[109506]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:resolve,1=cephfs.smithi046.cpwyud=up:resolve} 2 up:standby 2024-09-10T04:44:26.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:25 smithi070 ceph-mon[79774]: pgmap v1165: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 22 MiB/s rd, 8 op/s 2024-09-10T04:44:26.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:25 smithi070 ceph-mon[79774]: from='client.45599 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:26.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:25 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:resolve 2024-09-10T04:44:26.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:25 smithi070 ceph-mon[79774]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:resolve,1=cephfs.smithi046.cpwyud=up:resolve} 2 up:standby 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: reconnect by client.24367 192.168.0.1:0/2709434340 after 0.00199998 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: reconnect by client.24367 192.168.0.1:0/2709434340 after 0.00199998 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: reconnect by client.14606 192.168.0.1:0/1648660616 after 0.00199998 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: reconnect by client.14606 192.168.0.1:0/1648660616 after 0.00199998 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:reconnect 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:reconnect 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:reconnect,1=cephfs.smithi046.cpwyud=up:reconnect} 2 up:standby 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:27.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:26 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:27.168 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: reconnect by client.24367 192.168.0.1:0/2709434340 after 0.00199998 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: reconnect by client.24367 192.168.0.1:0/2709434340 after 0.00199998 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: reconnect by client.14606 192.168.0.1:0/1648660616 after 0.00199998 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: reconnect by client.14606 192.168.0.1:0/1648660616 after 0.00199998 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:reconnect 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:reconnect 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:reconnect,1=cephfs.smithi046.cpwyud=up:reconnect} 2 up:standby 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:27.169 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:26 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:28.176 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:27 smithi070 ceph-mon[79774]: pgmap v1166: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-09-10T04:44:28.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:27 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:rejoin 2024-09-10T04:44:28.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:27 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:rejoin 2024-09-10T04:44:28.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:27 smithi070 ceph-mon[79774]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:rejoin,1=cephfs.smithi046.cpwyud=up:rejoin} 2 up:standby 2024-09-10T04:44:28.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:27 smithi070 ceph-mon[79774]: daemon mds.cephfs.smithi046.cpwyud is now active in filesystem cephfs as rank 1 2024-09-10T04:44:28.177 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:27 smithi070 ceph-mon[79774]: daemon mds.cephfs.smithi046.hasdap is now active in filesystem cephfs as rank 0 2024-09-10T04:44:28.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:27 smithi046 ceph-mon[109506]: pgmap v1166: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-09-10T04:44:28.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:27 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:rejoin 2024-09-10T04:44:28.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:27 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:rejoin 2024-09-10T04:44:28.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:27 smithi046 ceph-mon[109506]: fsmap cephfs:2/2 {0=cephfs.smithi046.hasdap=up:rejoin,1=cephfs.smithi046.cpwyud=up:rejoin} 2 up:standby 2024-09-10T04:44:28.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:27 smithi046 ceph-mon[109506]: daemon mds.cephfs.smithi046.cpwyud is now active in filesystem cephfs as rank 1 2024-09-10T04:44:28.338 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:27 smithi046 ceph-mon[109506]: daemon mds.cephfs.smithi046.hasdap is now active in filesystem cephfs as rank 0 2024-09-10T04:44:29.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T04:44:29.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:active 2024-09-10T04:44:29.001 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:active 2024-09-10T04:44:29.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi046.cpwyud=up:active} 2 up:standby 2024-09-10T04:44:29.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi046.cpwyud=up:active} 2 up:standby-replay 2024-09-10T04:44:29.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:29.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-10T04:44:29.002 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:28 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] up:active 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: mds.? [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] up:active 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi046.cpwyud=up:active} 2 up:standby 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi046.cpwyud=up:active} 2 up:standby-replay 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi070", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-10T04:44:29.289 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:28 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:29 smithi070 ceph-mon[79774]: pgmap v1167: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-09-10T04:44:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:29 smithi070 ceph-mon[79774]: Upgrade: Updating ceph-exporter.smithi070 (2/2) 2024-09-10T04:44:30.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:29 smithi070 ceph-mon[79774]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-09-10T04:44:30.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:29 smithi046 ceph-mon[109506]: pgmap v1167: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-09-10T04:44:30.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:29 smithi046 ceph-mon[109506]: Upgrade: Updating ceph-exporter.smithi070 (2/2) 2024-09-10T04:44:30.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:29 smithi046 ceph-mon[109506]: Deploying daemon ceph-exporter.smithi070 on smithi070 2024-09-10T04:44:32.186 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:31 smithi070 ceph-mon[79774]: pgmap v1168: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 24 MiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T04:44:32.284 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:31 smithi046 ceph-mon[109506]: pgmap v1168: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 24 MiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T04:44:34.242 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: pgmap v1169: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 24 MiB/s rd, 4.5 KiB/s wr, 19 op/s 2024-09-10T04:44:34.242 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.242 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.242 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:34.242 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.242 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.243 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.243 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:33 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: pgmap v1169: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 24 MiB/s rd, 4.5 KiB/s wr, 19 op/s 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:34.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:33 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:36.138 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:35 smithi046 ceph-mon[109506]: pgmap v1170: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-10T04:44:36.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:35 smithi070 ceph-mon[79774]: pgmap v1170: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.5 KiB/s wr, 16 op/s 2024-09-10T04:44:38.290 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:37 smithi046 ceph-mon[109506]: pgmap v1171: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 25 MiB/s rd, 4.5 KiB/s wr, 19 op/s 2024-09-10T04:44:38.324 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:37 smithi070 ceph-mon[79774]: pgmap v1171: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 25 MiB/s rd, 4.5 KiB/s wr, 19 op/s 2024-09-10T04:44:39.325 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:39 smithi046 ceph-mon[109506]: pgmap v1172: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 4.5 KiB/s wr, 18 op/s 2024-09-10T04:44:39.325 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.326 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:39.326 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.326 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.326 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.326 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:39 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:39 smithi070 ceph-mon[79774]: pgmap v1172: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 4.5 KiB/s wr, 18 op/s 2024-09-10T04:44:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:39.336 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:39 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:40.877 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:40.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:40.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:40.878 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:40 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:41.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:40 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:41.831 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:41 smithi070 ceph-mon[79774]: pgmap v1173: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 4.5 KiB/s wr, 18 op/s 2024-09-10T04:44:41.935 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:41 smithi046 ceph-mon[109506]: pgmap v1173: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 20 MiB/s rd, 4.5 KiB/s wr, 18 op/s 2024-09-10T04:44:43.216 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:42 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:43.216 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:42 smithi070 ceph-mon[79774]: pgmap v1174: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-09-10T04:44:43.216 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:42 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:43.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:42 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:43.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:42 smithi046 ceph-mon[109506]: pgmap v1174: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-09-10T04:44:43.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:42 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: pgmap v1175: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 5 op/s 2024-09-10T04:44:44.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:44 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-10T04:44:44.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: pgmap v1175: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 5 op/s 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T04:44:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:44 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} : dispatch 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi046.cpwyud=up:active} 2 up:standby-replay 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.586 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all ceph-exporter 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi046"} : dispatch 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi046"}]': finished 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"} : dispatch 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"}]': finished 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all iscsi 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.587 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all nfs 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: Upgrade: Setting container_image for all nvmeof 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: Upgrade: Finalizing container_image settings 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-10T04:44:45.589 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-10T04:44:45.590 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.591 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: Upgrade: Complete! 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:45.592 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.593 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:45.593 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:45.593 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:45.593 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: fsmap cephfs:2 {0=cephfs.smithi046.hasdap=up:active,1=cephfs.smithi046.cpwyud=up:active} 2 up:standby-replay 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.594 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all ceph-exporter 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi046"} : dispatch 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi046"}]': finished 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"} : dispatch 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi070"}]': finished 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all iscsi 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.595 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all nfs 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: Upgrade: Setting container_image for all nvmeof 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.596 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: Upgrade: Finalizing container_image settings 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-10T04:44:45.597 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T04:44:45.598 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: Upgrade: Complete! 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:45.599 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:45.600 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:45.600 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:45.600 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:44:45.600 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:44:45.600 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:44:45.600 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:46.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:46 smithi070 ceph-mon[79774]: pgmap v1176: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 5 op/s 2024-09-10T04:44:46.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:46 smithi046 ceph-mon[109506]: pgmap v1176: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 5 op/s 2024-09-10T04:44:48.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:48 smithi070 ceph-mon[79774]: pgmap v1177: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s rd, 3 op/s 2024-09-10T04:44:48.588 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:48.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:48 smithi046 ceph-mon[109506]: pgmap v1177: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s rd, 3 op/s 2024-09-10T04:44:48.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:50.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:50 smithi070 ceph-mon[79774]: pgmap v1178: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s rd, 3 op/s 2024-09-10T04:44:50.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:50 smithi046 ceph-mon[109506]: pgmap v1178: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 5.0 KiB/s rd, 3 op/s 2024-09-10T04:44:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:52 smithi070 ceph-mon[79774]: pgmap v1179: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 5.5 KiB/s rd, 4 op/s 2024-09-10T04:44:52.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:52 smithi046 ceph-mon[109506]: pgmap v1179: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 5.5 KiB/s rd, 4 op/s 2024-09-10T04:44:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:54 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:54 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:54.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:54 smithi070 ceph-mon[79774]: pgmap v1180: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T04:44:54.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:54 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:44:54.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:54 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:44:54.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:54 smithi046 ceph-mon[109506]: pgmap v1180: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T04:44:54.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 -- 172.21.15.46:0/1082146846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 msgr2=0x7f4820100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:54.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 --2- 172.21.15.46:0/1082146846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f4820100c50 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f4808009880 tx=0x7f480802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 -- 172.21.15.46:0/1082146846 shutdown_connections 2024-09-10T04:44:54.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 --2- 172.21.15.46:0/1082146846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f4820100c50 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 --2- 172.21.15.46:0/1082146846 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48201067f0 0x7f4820106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 -- 172.21.15.46:0/1082146846 >> 172.21.15.46:0/1082146846 conn(0x7f48200fc550 msgr2=0x7f48200fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:54.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 -- 172.21.15.46:0/1082146846 shutdown_connections 2024-09-10T04:44:54.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.515+0000 7f482522e640 1 -- 172.21.15.46:0/1082146846 wait complete. 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.516+0000 7f482522e640 1 Processor -- start 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.516+0000 7f482522e640 1 -- start start 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f482522e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f482019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f482522e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48201067f0 0x7f482019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f482522e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f482010dcc0 con 0x7f48201007f0 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f482522e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f482010db40 con 0x7f48201067f0 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f482019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f482019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48900/0 (socket says 172.21.15.46:48900) 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481ed9d640 1 -- 172.21.15.46:0/3127041979 learned_addr learned my addr 172.21.15.46:0/3127041979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:54.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481e59c640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48201067f0 0x7f482019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481ed9d640 1 -- 172.21.15.46:0/3127041979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48201067f0 msgr2=0x7f482019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481ed9d640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48201067f0 0x7f482019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481ed9d640 1 -- 172.21.15.46:0/3127041979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48201977e0 con 0x7f48201007f0 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.517+0000 7f481ed9d640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f482019d660 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f481000d6f0 tx=0x7f481000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.518+0000 7f4817fff640 1 -- 172.21.15.46:0/3127041979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4810017070 con 0x7f48201007f0 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.518+0000 7f4817fff640 1 -- 172.21.15.46:0/3127041979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f481000de00 con 0x7f48201007f0 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.518+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4820197ad0 con 0x7f48201007f0 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.518+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4820198030 con 0x7f48201007f0 2024-09-10T04:44:54.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.518+0000 7f4817fff640 1 -- 172.21.15.46:0/3127041979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4810004bb0 con 0x7f48201007f0 2024-09-10T04:44:54.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.519+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4820101f30 con 0x7f48201007f0 2024-09-10T04:44:54.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.519+0000 7f4817fff640 1 -- 172.21.15.46:0/3127041979 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4810009a30 con 0x7f48201007f0 2024-09-10T04:44:54.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.520+0000 7f4817fff640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f480c07ac50 0x7f480c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:54.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.520+0000 7f4817fff640 1 -- 172.21.15.46:0/3127041979 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f481009d740 con 0x7f48201007f0 2024-09-10T04:44:54.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.520+0000 7f481e59c640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f480c07ac50 0x7f480c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:54.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.521+0000 7f481e59c640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f480c07ac50 0x7f480c07d110 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4820198b30 tx=0x7f4808031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:54.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.525+0000 7f4817fff640 1 -- 172.21.15.46:0/3127041979 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4810068f00 con 0x7f48201007f0 2024-09-10T04:44:54.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.728+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7f48201982e0 con 0x7f480c07ac50 2024-09-10T04:44:54.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.729+0000 7f4817fff640 1 -- 172.21.15.46:0/3127041979 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+175 (secure 0 0 0) 0x7f48201982e0 con 0x7f480c07ac50 2024-09-10T04:44:54.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f480c07ac50 msgr2=0x7f480c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:54.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f480c07ac50 0x7f480c07d110 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4820198b30 tx=0x7f4808031400 comp rx=0 tx=0).stop 2024-09-10T04:44:54.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 msgr2=0x7f482019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:54.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f482019d660 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f481000d6f0 tx=0x7f481000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 shutdown_connections 2024-09-10T04:44:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f480c07ac50 0x7f480c07d110 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48201067f0 0x7f482019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 --2- 172.21.15.46:0/3127041979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48201007f0 0x7f482019d660 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 >> 172.21.15.46:0/3127041979 conn(0x7f48200fc550 msgr2=0x7f4820069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 shutdown_connections 2024-09-10T04:44:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:54.731+0000 7f482522e640 1 -- 172.21.15.46:0/3127041979 wait complete. 2024-09-10T04:44:55.317 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:55 smithi070 ceph-mon[79774]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:55.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:55 smithi046 ceph-mon[109506]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:55.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph orch ps' 2024-09-10T04:44:55.781 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:44:56.177 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:56 smithi046 ceph-mon[109506]: pgmap v1181: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:44:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 -- 172.21.15.46:0/2747259723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14901033e0 msgr2=0x7f149010b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 --2- 172.21.15.46:0/2747259723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14901033e0 0x7f149010b8d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1480009880 tx=0x7f148002a530 comp rx=0 tx=0).stop 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 -- 172.21.15.46:0/2747259723 shutdown_connections 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 --2- 172.21.15.46:0/2747259723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14901033e0 0x7f149010b8d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 --2- 172.21.15.46:0/2747259723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1490102ac0 0x7f1490102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 -- 172.21.15.46:0/2747259723 >> 172.21.15.46:0/2747259723 conn(0x7f14900fc520 msgr2=0x7f14900fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 -- 172.21.15.46:0/2747259723 shutdown_connections 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.263+0000 7f1498456640 1 -- 172.21.15.46:0/2747259723 wait complete. 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f1498456640 1 Processor -- start 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f1498456640 1 -- start start 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f1498456640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1490102ac0 0x7f149019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f1498456640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14901033e0 0x7f149019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f1498456640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f149010edf0 con 0x7f1490102ac0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f1498456640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f149010ec70 con 0x7f14901033e0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f14961f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1490102ac0 0x7f149019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f14961f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1490102ac0 0x7f149019d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48924/0 (socket says 172.21.15.46:48924) 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f14961f2640 1 -- 172.21.15.46:0/463838518 learned_addr learned my addr 172.21.15.46:0/463838518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.264+0000 7f14959f1640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14901033e0 0x7f149019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f14961f2640 1 -- 172.21.15.46:0/463838518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14901033e0 msgr2=0x7f149019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f14961f2640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14901033e0 0x7f149019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f14961f2640 1 -- 172.21.15.46:0/463838518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1490197800 con 0x7f1490102ac0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f14961f2640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1490102ac0 0x7f149019d680 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e970 tx=0x7f148c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f14877fe640 1 -- 172.21.15.46:0/463838518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f148c00cac0 con 0x7f1490102ac0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f14877fe640 1 -- 172.21.15.46:0/463838518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f148c0054e0 con 0x7f1490102ac0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1490197af0 con 0x7f1490102ac0 2024-09-10T04:44:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f14877fe640 1 -- 172.21.15.46:0/463838518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f148c010430 con 0x7f1490102ac0 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.265+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1490198050 con 0x7f1490102ac0 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.266+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149006aa90 con 0x7f1490102ac0 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.267+0000 7f14877fe640 1 -- 172.21.15.46:0/463838518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f148c004f00 con 0x7f1490102ac0 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.268+0000 7f14877fe640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f147c07ac50 0x7f147c07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.268+0000 7f14959f1640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f147c07ac50 0x7f147c07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.268+0000 7f14877fe640 1 -- 172.21.15.46:0/463838518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f148c09e220 con 0x7f1490102ac0 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.268+0000 7f14959f1640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f147c07ac50 0x7f147c07d110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1490198b50 tx=0x7f1480031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.272+0000 7f14877fe640 1 -- 172.21.15.46:0/463838518 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f148c0699e0 con 0x7f1490102ac0 2024-09-10T04:44:56.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.467+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f1490198360 con 0x7f147c07ac50 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.477+0000 7f14877fe640 1 -- 172.21.15.46:0/463838518 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3816 (secure 0 0 0) 0x7f1490198360 con 0x7f147c07ac50 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (36m) 18s ago 47m 21.0M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (33s) 18s ago 48m 8220k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 404196f2cc20 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (23s) 16s ago 47m 7545k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 6972232ad67e 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (34m) 18s ago 48m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (34m) 16s ago 47m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (36m) 18s ago 47m 78.1M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:44:56.478 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (31m) 18s ago 42m 28.4M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (31m) 18s ago 42m 89.3M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (30m) 16s ago 42m 236M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (15m) 16s ago 42m 24.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (38m) 18s ago 49m 537M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (37m) 16s ago 47m 464M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (35m) 18s ago 49m 87.3M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (35m) 16s ago 46m 60.7M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (37m) 18s ago 47m 11.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (37m) 16s ago 47m 10.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (34m) 18s ago 45m 95.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (33m) 18s ago 45m 105M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (33m) 18s ago 44m 96.4M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (32m) 16s ago 44m 105M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:44:56.479 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (32m) 16s ago 43m 97.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:44:56.480 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (31m) 16s ago 43m 92.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:44:56.480 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (36m) 18s ago 47m 73.7M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:44:56.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f147c07ac50 msgr2=0x7f147c07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:56.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f147c07ac50 0x7f147c07d110 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1490198b50 tx=0x7f1480031400 comp rx=0 tx=0).stop 2024-09-10T04:44:56.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1490102ac0 msgr2=0x7f149019d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:56.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1490102ac0 0x7f149019d680 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e970 tx=0x7f148c00ed70 comp rx=0 tx=0).stop 2024-09-10T04:44:56.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 shutdown_connections 2024-09-10T04:44:56.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f147c07ac50 0x7f147c07d110 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:56.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14901033e0 0x7f149019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:56.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 --2- 172.21.15.46:0/463838518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1490102ac0 0x7f149019d680 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:56.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 >> 172.21.15.46:0/463838518 conn(0x7f14900fc520 msgr2=0x7f1490108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:56.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 shutdown_connections 2024-09-10T04:44:56.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:56.479+0000 7f1498456640 1 -- 172.21.15.46:0/463838518 wait complete. 2024-09-10T04:44:56.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:56 smithi070 ceph-mon[79774]: pgmap v1181: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:44:57.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph orch upgrade status' 2024-09-10T04:44:57.336 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:57 smithi046 ceph-mon[109506]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:57.361 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:44:57.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:57 smithi070 ceph-mon[79774]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 -- 172.21.15.46:0/3044746839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948101900 msgr2=0x7fb948101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 --2- 172.21.15.46:0/3044746839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948101900 0x7fb948101d60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009880 tx=0x7fb93c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 -- 172.21.15.46:0/3044746839 shutdown_connections 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 --2- 172.21.15.46:0/3044746839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948101900 0x7fb948101d60 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 --2- 172.21.15.46:0/3044746839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb948107900 0x7fb948107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 -- 172.21.15.46:0/3044746839 >> 172.21.15.46:0/3044746839 conn(0x7fb9480fd600 msgr2=0x7fb9480ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 -- 172.21.15.46:0/3044746839 shutdown_connections 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.892+0000 7fb94d534640 1 -- 172.21.15.46:0/3044746839 wait complete. 2024-09-10T04:44:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.893+0000 7fb94d534640 1 Processor -- start 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.893+0000 7fb94d534640 1 -- start start 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb94d534640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb948101900 0x7fb94819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb94d534640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948107900 0x7fb94819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb94d534640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810edd0 con 0x7fb948107900 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb94d534640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fb94810ec50 con 0x7fb948101900 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb9467fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948107900 0x7fb94819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb9467fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948107900 0x7fb94819db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48942/0 (socket says 172.21.15.46:48942) 2024-09-10T04:44:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb9467fc640 1 -- 172.21.15.46:0/2041879265 learned_addr learned my addr 172.21.15.46:0/2041879265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb946ffd640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb948101900 0x7fb94819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb9467fc640 1 -- 172.21.15.46:0/2041879265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb948101900 msgr2=0x7fb94819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb9467fc640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb948101900 0x7fb94819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb9467fc640 1 -- 172.21.15.46:0/2041879265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9481977a0 con 0x7fb948107900 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb9467fc640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948107900 0x7fb94819db60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb93c031460 tx=0x7fb93c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb92ffff640 1 -- 172.21.15.46:0/2041879265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb93c03f070 con 0x7fb948107900 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb92ffff640 1 -- 172.21.15.46:0/2041879265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb93c0035f0 con 0x7fb948107900 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.894+0000 7fb92ffff640 1 -- 172.21.15.46:0/2041879265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fb93c035030 con 0x7fb948107900 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.895+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb948197a30 con 0x7fb948107900 2024-09-10T04:44:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.895+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb948197f60 con 0x7fb948107900 2024-09-10T04:44:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.896+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb948103040 con 0x7fb948107900 2024-09-10T04:44:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.896+0000 7fb92ffff640 1 -- 172.21.15.46:0/2041879265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fb93c042070 con 0x7fb948107900 2024-09-10T04:44:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.897+0000 7fb92ffff640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb93007ac40 0x7fb93007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.897+0000 7fb946ffd640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb93007ac40 0x7fb93007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.897+0000 7fb92ffff640 1 -- 172.21.15.46:0/2041879265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fb93c0be5a0 con 0x7fb948107900 2024-09-10T04:44:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.897+0000 7fb946ffd640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb93007ac40 0x7fb93007d100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb938005fd0 tx=0x7fb938005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:57.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:57.901+0000 7fb92ffff640 1 -- 172.21.15.46:0/2041879265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fb93c089d60 con 0x7fb948107900 2024-09-10T04:44:58.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.107+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) -- 0x7fb948198210 con 0x7fb93007ac40 2024-09-10T04:44:58.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.108+0000 7fb92ffff640 1 -- 172.21.15.46:0/2041879265 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+175 (secure 0 0 0) 0x7fb948198210 con 0x7fb93007ac40 2024-09-10T04:44:58.109 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:44:58.109 INFO:teuthology.orchestra.run.smithi046.stdout: "target_image": null, 2024-09-10T04:44:58.109 INFO:teuthology.orchestra.run.smithi046.stdout: "in_progress": false, 2024-09-10T04:44:58.109 INFO:teuthology.orchestra.run.smithi046.stdout: "which": "", 2024-09-10T04:44:58.109 INFO:teuthology.orchestra.run.smithi046.stdout: "services_complete": [], 2024-09-10T04:44:58.109 INFO:teuthology.orchestra.run.smithi046.stdout: "progress": null, 2024-09-10T04:44:58.110 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-09-10T04:44:58.110 INFO:teuthology.orchestra.run.smithi046.stdout: "is_paused": false 2024-09-10T04:44:58.110 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:44:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb93007ac40 msgr2=0x7fb93007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:58.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb93007ac40 0x7fb93007d100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb938005fd0 tx=0x7fb938005850 comp rx=0 tx=0).stop 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948107900 msgr2=0x7fb94819db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948107900 0x7fb94819db60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb93c031460 tx=0x7fb93c0045e0 comp rx=0 tx=0).stop 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 shutdown_connections 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fb93007ac40 0x7fb93007d100 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb948107900 0x7fb94819db60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 --2- 172.21.15.46:0/2041879265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb948101900 0x7fb94819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.110+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 >> 172.21.15.46:0/2041879265 conn(0x7fb9480fd600 msgr2=0x7fb9480fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.111+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 shutdown_connections 2024-09-10T04:44:58.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:58.111+0000 7fb94d534640 1 -- 172.21.15.46:0/2041879265 wait complete. 2024-09-10T04:44:58.518 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:58 smithi046 ceph-mon[109506]: pgmap v1182: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:44:58.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:58 smithi070 ceph-mon[79774]: pgmap v1182: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:44:58.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph health detail' 2024-09-10T04:44:59.043 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:44:59.417 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:44:59 smithi046 ceph-mon[109506]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2108412105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 msgr2=0x7fbce0100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2108412105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce0100c60 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009880 tx=0x7fbccc02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2108412105 shutdown_connections 2024-09-10T04:44:59.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2108412105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce0100c60 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2108412105 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce0106800 0x7fbce0106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2108412105 >> 172.21.15.46:0/2108412105 conn(0x7fbce00fc520 msgr2=0x7fbce00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2108412105 shutdown_connections 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.499+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2108412105 wait complete. 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce6e8b640 1 Processor -- start 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce6e8b640 1 -- start start 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce6e8b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce6e8b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce0106800 0x7fbce019fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:59.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce6e8b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010dcd0 con 0x7fbce0100800 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce6e8b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fbce010db50 con 0x7fbce0106800 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce4c27640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce4c27640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45522/0 (socket says 172.21.15.46:45522) 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbce4c27640 1 -- 172.21.15.46:0/2814541537 learned_addr learned my addr 172.21.15.46:0/2814541537 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.500+0000 7fbcdffff640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce0106800 0x7fbce019fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbce4c27640 1 -- 172.21.15.46:0/2814541537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce0106800 msgr2=0x7fbce019fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbce4c27640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce0106800 0x7fbce019fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbce4c27640 1 -- 172.21.15.46:0/2814541537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce0199a10 con 0x7fbce0100800 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbce4c27640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce019f850 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c8f0 tx=0x7fbcd400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:59.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbcddffb640 1 -- 172.21.15.46:0/2814541537 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbcd40079b0 con 0x7fbce0100800 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce0199d00 con 0x7fbce0100800 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbcddffb640 1 -- 172.21.15.46:0/2814541537 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcd40054e0 con 0x7fbce0100800 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbcddffb640 1 -- 172.21.15.46:0/2814541537 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fbcd4004dd0 con 0x7fbce0100800 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.501+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce019a260 con 0x7fbce0100800 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.502+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce0101f40 con 0x7fbce0100800 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.503+0000 7fbcddffb640 1 -- 172.21.15.46:0/2814541537 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fbcd4005020 con 0x7fbce0100800 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.503+0000 7fbcddffb640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcd007abf0 0x7fbcd007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:44:59.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.503+0000 7fbcddffb640 1 -- 172.21.15.46:0/2814541537 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fbcd409db40 con 0x7fbce0100800 2024-09-10T04:44:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.503+0000 7fbcdffff640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcd007abf0 0x7fbcd007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:44:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.504+0000 7fbcdffff640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcd007abf0 0x7fbcd007d0b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbce019b200 tx=0x7fbccc031420 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:44:59.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.508+0000 7fbcddffb640 1 -- 172.21.15.46:0/2814541537 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fbcd4014030 con 0x7fbce0100800 2024-09-10T04:44:59.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:44:59 smithi070 ceph-mon[79774]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:44:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.769+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) -- 0x7fbce019a570 con 0x7fbce0100800 2024-09-10T04:44:59.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.770+0000 7fbcddffb640 1 -- 172.21.15.46:0/2814541537 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) ==== 74+0+201 (secure 0 0 0) 0x7fbcd4069300 con 0x7fbce0100800 2024-09-10T04:44:59.771 INFO:teuthology.orchestra.run.smithi046.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:44:59.771 INFO:teuthology.orchestra.run.smithi046.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:44:59.771 INFO:teuthology.orchestra.run.smithi046.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:44:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcd007abf0 msgr2=0x7fbcd007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcd007abf0 0x7fbcd007d0b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbce019b200 tx=0x7fbccc031420 comp rx=0 tx=0).stop 2024-09-10T04:44:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 msgr2=0x7fbce019f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:44:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce019f850 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400c8f0 tx=0x7fbcd400cdb0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 shutdown_connections 2024-09-10T04:44:59.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fbcd007abf0 0x7fbcd007d0b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbce0106800 0x7fbce019fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 --2- 172.21.15.46:0/2814541537 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbce0100800 0x7fbce019f850 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:44:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 >> 172.21.15.46:0/2814541537 conn(0x7fbce00fc520 msgr2=0x7fbce00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:44:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 shutdown_connections 2024-09-10T04:44:59.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:44:59.772+0000 7fbce6e8b640 1 -- 172.21.15.46:0/2814541537 wait complete. 2024-09-10T04:45:00.446 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:00 smithi046 ceph-mon[109506]: pgmap v1183: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:00.446 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:00 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2814541537' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:45:00.477 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph versions' 2024-09-10T04:45:00.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:00 smithi070 ceph-mon[79774]: pgmap v1183: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:00.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:00 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2814541537' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-10T04:45:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:45:01.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 -- 172.21.15.46:0/403505745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0107900 msgr2=0x7f97b0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:45:01.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 --2- 172.21.15.46:0/403505745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0107900 0x7f97b0107ce0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009880 tx=0x7f97a402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 -- 172.21.15.46:0/403505745 shutdown_connections 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 --2- 172.21.15.46:0/403505745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b0101900 0x7f97b0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 --2- 172.21.15.46:0/403505745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0107900 0x7f97b0107ce0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 -- 172.21.15.46:0/403505745 >> 172.21.15.46:0/403505745 conn(0x7f97b00fd600 msgr2=0x7f97b00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 -- 172.21.15.46:0/403505745 shutdown_connections 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.223+0000 7f97b6edc640 1 -- 172.21.15.46:0/403505745 wait complete. 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b6edc640 1 Processor -- start 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b6edc640 1 -- start start 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b6edc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0101900 0x7f97b019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b6edc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b0107900 0x7f97b019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b6edc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010edd0 con 0x7f97b0101900 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b6edc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f97b010ec50 con 0x7f97b0107900 2024-09-10T04:45:01.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b4c78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0101900 0x7f97b019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b4c78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0101900 0x7f97b019d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45538/0 (socket says 172.21.15.46:45538) 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97b4c78640 1 -- 172.21.15.46:0/1412666323 learned_addr learned my addr 172.21.15.46:0/1412666323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.224+0000 7f97affff640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b0107900 0x7f97b019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97b4c78640 1 -- 172.21.15.46:0/1412666323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b0107900 msgr2=0x7f97b019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97b4c78640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b0107900 0x7f97b019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97b4c78640 1 -- 172.21.15.46:0/1412666323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b01977a0 con 0x7f97b0101900 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97b4c78640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0101900 0x7f97b019d620 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f97a4031460 tx=0x7f97a40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97adffb640 1 -- 172.21.15.46:0/1412666323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97a403f070 con 0x7f97b0101900 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97adffb640 1 -- 172.21.15.46:0/1412666323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f97a4004cf0 con 0x7f97b0101900 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b0197a30 con 0x7f97b0101900 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97adffb640 1 -- 172.21.15.46:0/1412666323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f97a4035030 con 0x7f97b0101900 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.225+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b0197f60 con 0x7f97b0101900 2024-09-10T04:45:01.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.226+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b0103040 con 0x7f97b0101900 2024-09-10T04:45:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.227+0000 7f97adffb640 1 -- 172.21.15.46:0/1412666323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f97a4042070 con 0x7f97b0101900 2024-09-10T04:45:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.227+0000 7f97adffb640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97a007abf0 0x7f97a007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:45:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.228+0000 7f97adffb640 1 -- 172.21.15.46:0/1412666323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f97a40be750 con 0x7f97b0101900 2024-09-10T04:45:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.228+0000 7f97affff640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97a007abf0 0x7f97a007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:45:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.228+0000 7f97affff640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97a007abf0 0x7f97a007d0b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c005e20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:45:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.232+0000 7f97adffb640 1 -- 172.21.15.46:0/1412666323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f97a4089e90 con 0x7f97b0101900 2024-09-10T04:45:01.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.490+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7f97b0109e70 con 0x7f97b0101900 2024-09-10T04:45:01.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.491+0000 7f97adffb640 1 -- 172.21.15.46:0/1412666323 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7f97a408ed40 con 0x7f97b0101900 2024-09-10T04:45:01.492 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:45:01.492 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:45:01.493 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:45:01.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:45:01.494 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:45:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97a007abf0 msgr2=0x7f97a007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:45:01.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97a007abf0 0x7f97a007d0b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f979c005fd0 tx=0x7f979c005e20 comp rx=0 tx=0).stop 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0101900 msgr2=0x7f97b019d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0101900 0x7f97b019d620 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f97a4031460 tx=0x7f97a40048f0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 shutdown_connections 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f97a007abf0 0x7f97a007d0b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f97b0107900 0x7f97b019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 --2- 172.21.15.46:0/1412666323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97b0101900 0x7f97b019d620 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 >> 172.21.15.46:0/1412666323 conn(0x7f97b00fd600 msgr2=0x7f97b00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 shutdown_connections 2024-09-10T04:45:01.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:45:01.493+0000 7f97b6edc640 1 -- 172.21.15.46:0/1412666323 wait complete. 2024-09-10T04:45:02.169 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-10T04:45:02.435 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:45:02.558 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:02 smithi046 ceph-mon[109506]: pgmap v1184: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:02.558 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:02 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1412666323' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:45:02.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:02 smithi070 ceph-mon[79774]: pgmap v1184: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:02.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:02 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1412666323' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:45:02.881 INFO:teuthology.orchestra.run.smithi046.stdout:wait for servicemap items w/ changing names to refresh 2024-09-10T04:45:03.594 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'sleep 60' 2024-09-10T04:45:03.857 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:45:04.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:04 smithi070 ceph-mon[79774]: pgmap v1185: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:04.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:04 smithi046 ceph-mon[109506]: pgmap v1185: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:06 smithi070 ceph-mon[79774]: pgmap v1186: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:06.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:06 smithi046 ceph-mon[109506]: pgmap v1186: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:08.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:08 smithi070 ceph-mon[79774]: pgmap v1187: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:08.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:08 smithi046 ceph-mon[109506]: pgmap v1187: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:10 smithi070 ceph-mon[79774]: pgmap v1188: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:10.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:10 smithi046 ceph-mon[109506]: pgmap v1188: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:12.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:12 smithi070 ceph-mon[79774]: pgmap v1189: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:12.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:12 smithi046 ceph-mon[109506]: pgmap v1189: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:14.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:14 smithi070 ceph-mon[79774]: pgmap v1190: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:14.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:14 smithi046 ceph-mon[109506]: pgmap v1190: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:16 smithi070 ceph-mon[79774]: pgmap v1191: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:16 smithi046 ceph-mon[109506]: pgmap v1191: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:18 smithi070 ceph-mon[79774]: pgmap v1192: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:18 smithi046 ceph-mon[109506]: pgmap v1192: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:20.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:20 smithi046 ceph-mon[109506]: pgmap v1193: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:20 smithi070 ceph-mon[79774]: pgmap v1193: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:22 smithi070 ceph-mon[79774]: pgmap v1194: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:22 smithi046 ceph-mon[109506]: pgmap v1194: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:23.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:24.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:24 smithi070 ceph-mon[79774]: pgmap v1195: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:24.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:24 smithi046 ceph-mon[109506]: pgmap v1195: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:26.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:26 smithi046 ceph-mon[109506]: pgmap v1196: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:26 smithi070 ceph-mon[79774]: pgmap v1196: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:28 smithi070 ceph-mon[79774]: pgmap v1197: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:28.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:28 smithi046 ceph-mon[109506]: pgmap v1197: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:30 smithi070 ceph-mon[79774]: pgmap v1198: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:30 smithi046 ceph-mon[109506]: pgmap v1198: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:32.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:32 smithi046 ceph-mon[109506]: pgmap v1199: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:32 smithi070 ceph-mon[79774]: pgmap v1199: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:34 smithi070 ceph-mon[79774]: pgmap v1200: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:34 smithi046 ceph-mon[109506]: pgmap v1200: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:36 smithi070 ceph-mon[79774]: pgmap v1201: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:36.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:36 smithi046 ceph-mon[109506]: pgmap v1201: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:38 smithi046 ceph-mon[109506]: pgmap v1202: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:38 smithi070 ceph-mon[79774]: pgmap v1202: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:40 smithi070 ceph-mon[79774]: pgmap v1203: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:40 smithi046 ceph-mon[109506]: pgmap v1203: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:42.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:42 smithi070 ceph-mon[79774]: pgmap v1204: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:42.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:42 smithi046 ceph-mon[109506]: pgmap v1204: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:44 smithi046 ceph-mon[109506]: pgmap v1205: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:44.656 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:44 smithi070 ceph-mon[79774]: pgmap v1205: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:45:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:45:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:45:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:45:45.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:45:45.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:45:45.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:45:45.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:45:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:46 smithi070 ceph-mon[79774]: pgmap v1206: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:46 smithi046 ceph-mon[109506]: pgmap v1206: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:48 smithi070 ceph-mon[79774]: pgmap v1207: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:48.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:48 smithi046 ceph-mon[109506]: pgmap v1207: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:50 smithi046 ceph-mon[109506]: pgmap v1208: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:50 smithi070 ceph-mon[79774]: pgmap v1208: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:52 smithi070 ceph-mon[79774]: pgmap v1209: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:52.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:52 smithi046 ceph-mon[109506]: pgmap v1209: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:45:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:45:54.736 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:54 smithi070 ceph-mon[79774]: pgmap v1210: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:54 smithi046 ceph-mon[109506]: pgmap v1210: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:45:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:56 smithi046 ceph-mon[109506]: pgmap v1211: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:56 smithi070 ceph-mon[79774]: pgmap v1211: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:45:58.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:45:58 smithi046 ceph-mon[109506]: pgmap v1212: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:45:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:45:58 smithi070 ceph-mon[79774]: pgmap v1212: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:00 smithi070 ceph-mon[79774]: pgmap v1213: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:00 smithi046 ceph-mon[109506]: pgmap v1213: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:02.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:02 smithi046 ceph-mon[109506]: pgmap v1214: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:02 smithi070 ceph-mon[79774]: pgmap v1214: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:04.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:04 smithi046 ceph-mon[109506]: pgmap v1215: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:04 smithi070 ceph-mon[79774]: pgmap v1215: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:04.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph orch ps' 2024-09-10T04:46:05.170 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:05.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/1809908219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4100800 msgr2=0x7ff5d4100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/1809908219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4100800 0x7ff5d4100c60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009c20 tx=0x7ff5bc02a970 comp rx=0 tx=0).stop 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/1809908219 shutdown_connections 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/1809908219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4100800 0x7ff5d4100c60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/1809908219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5d4106800 0x7ff5d4106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/1809908219 >> 172.21.15.46:0/1809908219 conn(0x7ff5d40fc520 msgr2=0x7ff5d40fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/1809908219 shutdown_connections 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.612+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/1809908219 wait complete. 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d8bc9640 1 Processor -- start 2024-09-10T04:46:05.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d8bc9640 1 -- start start 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d8bc9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5d4100800 0x7ff5d4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d8bc9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4106800 0x7ff5d4198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d8bc9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d410dd30 con 0x7ff5d4106800 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d8bc9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d410dbb0 con 0x7ff5d4100800 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d1d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4106800 0x7ff5d4198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d1d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4106800 0x7ff5d4198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36416/0 (socket says 172.21.15.46:36416) 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d1d9b640 1 -- 172.21.15.46:0/2968808635 learned_addr learned my addr 172.21.15.46:0/2968808635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.613+0000 7ff5d259c640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5d4100800 0x7ff5d4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5d1d9b640 1 -- 172.21.15.46:0/2968808635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5d4100800 msgr2=0x7ff5d4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5d1d9b640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5d4100800 0x7ff5d4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:05.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5d1d9b640 1 -- 172.21.15.46:0/2968808635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d4199440 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5d1d9b640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4106800 0x7ff5d4198e40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005cc0 tx=0x7ff5bc02f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5cb7fe640 1 -- 172.21.15.46:0/2968808635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff5bc047070 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5cb7fe640 1 -- 172.21.15.46:0/2968808635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5bc02f7e0 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d41996d0 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5cb7fe640 1 -- 172.21.15.46:0/2968808635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff5bc034070 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.614+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d41a1c80 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.615+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d4101f40 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.616+0000 7ff5cb7fe640 1 -- 172.21.15.46:0/2968808635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff5bc04a070 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.616+0000 7ff5cb7fe640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff5c007ac40 0x7ff5c007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.617+0000 7ff5d259c640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff5c007ac40 0x7ff5c007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.617+0000 7ff5cb7fe640 1 -- 172.21.15.46:0/2968808635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff5bc0be980 con 0x7ff5d4106800 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.617+0000 7ff5d259c640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff5c007ac40 0x7ff5c007d100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009990 tx=0x7ff5c4009110 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:05.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.621+0000 7ff5cb7fe640 1 -- 172.21.15.46:0/2968808635 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff5bc03f070 con 0x7ff5d4106800 2024-09-10T04:46:05.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.815+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 --> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff5d4062630 con 0x7ff5c007ac40 2024-09-10T04:46:05.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.825+0000 7ff5cb7fe640 1 -- 172.21.15.46:0/2968808635 <== mgr.14758 v2:172.21.15.46:6800/3636798080 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3840 (secure 0 0 0) 0x7ff5d4062630 con 0x7ff5c007ac40 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:alertmanager.smithi046 smithi046 *:9093,9094 running (37m) 87s ago 48m 21.0M - 0.27.0 11f11916f8cd d3c6f52628f7 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi046 smithi046 running (102s) 87s ago 49m 8220k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 404196f2cc20 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-exporter.smithi070 smithi070 running (93s) 86s ago 48m 7545k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 6972232ad67e 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi046 smithi046 running (36m) 87s ago 49m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b3ed1ecbc666 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:crash.smithi070 smithi070 running (36m) 86s ago 48m 6891k - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 15a04578c6c9 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:grafana.smithi046 smithi046 *:3000 running (37m) 87s ago 48m 78.1M - 10.4.0 c8b91775d855 40f598b869a6 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.cpwyud smithi046 running (32m) 87s ago 43m 28.4M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e71223b8cac 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi046.hasdap smithi046 running (32m) 87s ago 43m 89.3M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b b39c68c27cae 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.ehvjgr smithi070 running (31m) 86s ago 43m 236M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 85e7a4e42276 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:mds.cephfs.smithi070.myztiz smithi070 running (16m) 86s ago 43m 24.7M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ef1c4ea1d11b 2024-09-10T04:46:05.827 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi046.zvckjg smithi046 *:8443,9283,8765 running (39m) 87s ago 50m 537M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 2c997124b25b 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.smithi070.tezbzn smithi070 *:8443,9283,8765 running (38m) 86s ago 48m 464M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9dde280af70e 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi046 smithi046 running (36m) 87s ago 50m 87.3M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b da8d2333cd82 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:mon.smithi070 smithi070 running (36m) 86s ago 48m 60.7M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 9c38930714e4 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi046 smithi046 *:9100 running (38m) 87s ago 49m 11.6M - 1.7.0 72c9c2088986 0ff0decf440c 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:node-exporter.smithi070 smithi070 *:9100 running (38m) 86s ago 48m 10.5M - 1.7.0 72c9c2088986 49c35165c676 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (35m) 87s ago 47m 95.1M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 5359fceff12d 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi046 running (35m) 87s ago 46m 105M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 4a8fa4298755 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi046 running (34m) 87s ago 45m 96.4M 1217M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 66743aa5ec84 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi070 running (33m) 86s ago 45m 105M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 0313330c72b6 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi070 running (33m) 86s ago 45m 97.2M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3bf61d5adfbe 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi070 running (32m) 86s ago 44m 92.3M 2240M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 79aa94488c3a 2024-09-10T04:46:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:prometheus.smithi046 smithi046 *:9095 running (38m) 87s ago 48m 73.7M - 2.51.0 1d3b7f56885b f94f26bb10fa 2024-09-10T04:46:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff5c007ac40 msgr2=0x7ff5c007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff5c007ac40 0x7ff5c007d100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009990 tx=0x7ff5c4009110 comp rx=0 tx=0).stop 2024-09-10T04:46:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4106800 msgr2=0x7ff5d4198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4106800 0x7ff5d4198e40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc005cc0 tx=0x7ff5bc02f5f0 comp rx=0 tx=0).stop 2024-09-10T04:46:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 shutdown_connections 2024-09-10T04:46:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff5c007ac40 0x7ff5c007d100 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:05.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5d4106800 0x7ff5d4198e40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 --2- 172.21.15.46:0/2968808635 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5d4100800 0x7ff5d4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 >> 172.21.15.46:0/2968808635 conn(0x7ff5d40fc520 msgr2=0x7ff5d40fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 shutdown_connections 2024-09-10T04:46:05.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:05.827+0000 7ff5d8bc9640 1 -- 172.21.15.46:0/2968808635 wait complete. 2024-09-10T04:46:06.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph versions' 2024-09-10T04:46:06.723 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:06.736 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:06 smithi046 ceph-mon[109506]: pgmap v1216: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:06.736 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:06 smithi046 ceph-mon[109506]: from='client.35984 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:46:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:06 smithi070 ceph-mon[79774]: pgmap v1216: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:06 smithi070 ceph-mon[79774]: from='client.35984 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:46:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 -- 172.21.15.46:0/3226709196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc1033d0 msgr2=0x7fc9dc10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 --2- 172.21.15.46:0/3226709196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc1033d0 0x7fc9dc10b8c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009880 tx=0x7fc9cc02a530 comp rx=0 tx=0).stop 2024-09-10T04:46:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 -- 172.21.15.46:0/3226709196 shutdown_connections 2024-09-10T04:46:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 --2- 172.21.15.46:0/3226709196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc1033d0 0x7fc9dc10b8c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:07.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 --2- 172.21.15.46:0/3226709196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9dc102ab0 0x7fc9dc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 -- 172.21.15.46:0/3226709196 >> 172.21.15.46:0/3226709196 conn(0x7fc9dc0fc550 msgr2=0x7fc9dc0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 -- 172.21.15.46:0/3226709196 shutdown_connections 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.219+0000 7fc9e3266640 1 -- 172.21.15.46:0/3226709196 wait complete. 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e3266640 1 Processor -- start 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e3266640 1 -- start start 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e3266640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc102ab0 0x7fc9dc19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e3266640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9dc1033d0 0x7fc9dc19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e3266640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc10ede0 con 0x7fc9dc102ab0 2024-09-10T04:46:07.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e3266640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc10ec60 con 0x7fc9dc1033d0 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e1002640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc102ab0 0x7fc9dc19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e1002640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc102ab0 0x7fc9dc19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36446/0 (socket says 172.21.15.46:36446) 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e1002640 1 -- 172.21.15.46:0/776525956 learned_addr learned my addr 172.21.15.46:0/776525956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e0801640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9dc1033d0 0x7fc9dc19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e1002640 1 -- 172.21.15.46:0/776525956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9dc1033d0 msgr2=0x7fc9dc19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e1002640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9dc1033d0 0x7fc9dc19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e1002640 1 -- 172.21.15.46:0/776525956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc197810 con 0x7fc9dc102ab0 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.220+0000 7fc9e0801640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9dc1033d0 0x7fc9dc19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.221+0000 7fc9e1002640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc102ab0 0x7fc9dc19d690 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800e970 tx=0x7fc9d800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.221+0000 7fc9d27fc640 1 -- 172.21.15.46:0/776525956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc9d800cac0 con 0x7fc9dc102ab0 2024-09-10T04:46:07.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.221+0000 7fc9d27fc640 1 -- 172.21.15.46:0/776525956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9d80054e0 con 0x7fc9dc102ab0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.221+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9dc197b00 con 0x7fc9dc102ab0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.221+0000 7fc9d27fc640 1 -- 172.21.15.46:0/776525956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc9d8010430 con 0x7fc9dc102ab0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.221+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9dc198060 con 0x7fc9dc102ab0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.223+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9dc06a580 con 0x7fc9dc102ab0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.223+0000 7fc9d27fc640 1 -- 172.21.15.46:0/776525956 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc9d8004e80 con 0x7fc9dc102ab0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.223+0000 7fc9d27fc640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc9c807ac00 0x7fc9c807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.223+0000 7fc9d27fc640 1 -- 172.21.15.46:0/776525956 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc9d809e370 con 0x7fc9dc102ab0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.223+0000 7fc9e0801640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc9c807ac00 0x7fc9c807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.224+0000 7fc9e0801640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc9c807ac00 0x7fc9c807d0c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc198b60 tx=0x7fc9cc031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:07.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.228+0000 7fc9d27fc640 1 -- 172.21.15.46:0/776525956 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc9d8069ab0 con 0x7fc9dc102ab0 2024-09-10T04:46:07.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.494+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc9dc105f50 con 0x7fc9dc102ab0 2024-09-10T04:46:07.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.494+0000 7fc9d27fc640 1 -- 172.21.15.46:0/776525956 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fc9d806e960 con 0x7fc9dc102ab0 2024-09-10T04:46:07.495 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-09-10T04:46:07.495 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": { 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": { 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 2 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": { 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 6 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": { 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 4 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-09-10T04:46:07.496 INFO:teuthology.orchestra.run.smithi046.stdout: "overall": { 2024-09-10T04:46:07.497 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)": 14 2024-09-10T04:46:07.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-09-10T04:46:07.497 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-09-10T04:46:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc9c807ac00 msgr2=0x7fc9c807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc9c807ac00 0x7fc9c807d0c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc198b60 tx=0x7fc9cc031400 comp rx=0 tx=0).stop 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc102ab0 msgr2=0x7fc9dc19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc102ab0 0x7fc9dc19d690 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800e970 tx=0x7fc9d800ed70 comp rx=0 tx=0).stop 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 shutdown_connections 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc9c807ac00 0x7fc9c807d0c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9dc1033d0 0x7fc9dc19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 --2- 172.21.15.46:0/776525956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9dc102ab0 0x7fc9dc19d690 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 >> 172.21.15.46:0/776525956 conn(0x7fc9dc0fc550 msgr2=0x7fc9dc108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 shutdown_connections 2024-09-10T04:46:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:07.497+0000 7fc9e3266640 1 -- 172.21.15.46:0/776525956 wait complete. 2024-09-10T04:46:08.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-10T04:46:08.410 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:08.678 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:08 smithi046 ceph-mon[109506]: pgmap v1217: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:08.678 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:08 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/776525956' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:46:08.678 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:08 smithi070 ceph-mon[79774]: pgmap v1217: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:08 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/776525956' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:46:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:08.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.844+0000 7fc76deed640 1 -- 172.21.15.46:0/4001885875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 msgr2=0x7fc768106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:08.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.844+0000 7fc76deed640 1 --2- 172.21.15.46:0/4001885875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768106bd0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc758009880 tx=0x7fc75802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:08.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 -- 172.21.15.46:0/4001885875 shutdown_connections 2024-09-10T04:46:08.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 --2- 172.21.15.46:0/4001885875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7681007f0 0x7fc768100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:08.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 --2- 172.21.15.46:0/4001885875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768106bd0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 -- 172.21.15.46:0/4001885875 >> 172.21.15.46:0/4001885875 conn(0x7fc7680fc550 msgr2=0x7fc7680fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 -- 172.21.15.46:0/4001885875 shutdown_connections 2024-09-10T04:46:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 -- 172.21.15.46:0/4001885875 wait complete. 2024-09-10T04:46:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 Processor -- start 2024-09-10T04:46:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.845+0000 7fc76deed640 1 -- start start 2024-09-10T04:46:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc76deed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7681007f0 0x7fc768198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc76deed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc76deed640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc76810dcc0 con 0x7fc7681067f0 2024-09-10T04:46:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc76deed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc76810db40 con 0x7fc7681007f0 2024-09-10T04:46:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc766ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc766ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58210/0 (socket says 172.21.15.46:58210) 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc766ffd640 1 -- 172.21.15.46:0/3866315945 learned_addr learned my addr 172.21.15.46:0/3866315945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc7677fe640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7681007f0 0x7fc768198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc766ffd640 1 -- 172.21.15.46:0/3866315945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7681007f0 msgr2=0x7fc768198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc766ffd640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7681007f0 0x7fc768198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc766ffd640 1 -- 172.21.15.46:0/3866315945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc768199350 con 0x7fc7681067f0 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.846+0000 7fc766ffd640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768198d50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc75000d6f0 tx=0x7fc75000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:08.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.847+0000 7fc764ff9640 1 -- 172.21.15.46:0/3866315945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc750017070 con 0x7fc7681067f0 2024-09-10T04:46:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.847+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc768199640 con 0x7fc7681067f0 2024-09-10T04:46:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.847+0000 7fc764ff9640 1 -- 172.21.15.46:0/3866315945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc75000de00 con 0x7fc7681067f0 2024-09-10T04:46:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.847+0000 7fc764ff9640 1 -- 172.21.15.46:0/3866315945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc750004bb0 con 0x7fc7681067f0 2024-09-10T04:46:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.847+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7681a1c40 con 0x7fc7681067f0 2024-09-10T04:46:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.848+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc768101f30 con 0x7fc7681067f0 2024-09-10T04:46:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.848+0000 7fc764ff9640 1 -- 172.21.15.46:0/3866315945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc750004e60 con 0x7fc7681067f0 2024-09-10T04:46:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.849+0000 7fc764ff9640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc75407abf0 0x7fc75407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.849+0000 7fc764ff9640 1 -- 172.21.15.46:0/3866315945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc75009d820 con 0x7fc7681067f0 2024-09-10T04:46:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.849+0000 7fc7677fe640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc75407abf0 0x7fc75407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:08.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.850+0000 7fc7677fe640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc75407abf0 0x7fc75407d0b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7580096c0 tx=0x7fc758031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:08.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:08.853+0000 7fc764ff9640 1 -- 172.21.15.46:0/3866315945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc768199640 con 0x7fc7681067f0 2024-09-10T04:46:09.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.118+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fc768108d60 con 0x7fc7681067f0 2024-09-10T04:46:09.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.118+0000 7fc764ff9640 1 -- 172.21.15.46:0/3866315945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fc750068f60 con 0x7fc7681067f0 2024-09-10T04:46:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc75407abf0 msgr2=0x7fc75407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc75407abf0 0x7fc75407d0b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7580096c0 tx=0x7fc758031400 comp rx=0 tx=0).stop 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 msgr2=0x7fc768198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768198d50 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc75000d6f0 tx=0x7fc75000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 shutdown_connections 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc75407abf0 0x7fc75407d0b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7681067f0 0x7fc768198d50 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 --2- 172.21.15.46:0/3866315945 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7681007f0 0x7fc768198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 >> 172.21.15.46:0/3866315945 conn(0x7fc7680fc550 msgr2=0x7fc7680fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 shutdown_connections 2024-09-10T04:46:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:09.121+0000 7fc76deed640 1 -- 172.21.15.46:0/3866315945 wait complete. 2024-09-10T04:46:09.130 INFO:teuthology.orchestra.run.smithi046.stdout:true 2024-09-10T04:46:09.767 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:09 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3866315945' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:46:09.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -e sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-09-10T04:46:09.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:09 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3866315945' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:46:10.058 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:10.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:10 smithi070 ceph-mon[79774]: pgmap v1218: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:10.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:10 smithi046 ceph-mon[109506]: pgmap v1218: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 -- 172.21.15.46:0/2987705449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0106800 msgr2=0x7fa7b0106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2987705449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0106800 0x7fa7b0106be0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009880 tx=0x7fa7a002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 -- 172.21.15.46:0/2987705449 shutdown_connections 2024-09-10T04:46:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2987705449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7b0100800 0x7fa7b0100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2987705449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0106800 0x7fa7b0106be0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 -- 172.21.15.46:0/2987705449 >> 172.21.15.46:0/2987705449 conn(0x7fa7b00fc520 msgr2=0x7fa7b00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 -- 172.21.15.46:0/2987705449 shutdown_connections 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.678+0000 7fa7b690d640 1 -- 172.21.15.46:0/2987705449 wait complete. 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7b690d640 1 Processor -- start 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7b690d640 1 -- start start 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7b690d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0100800 0x7fa7b01987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7b690d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7b0106800 0x7fa7b0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7b690d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010dcb0 con 0x7fa7b0100800 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7b690d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa7b010db30 con 0x7fa7b0106800 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7affff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0100800 0x7fa7b01987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7affff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0100800 0x7fa7b01987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58236/0 (socket says 172.21.15.46:58236) 2024-09-10T04:46:10.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7affff640 1 -- 172.21.15.46:0/2847168744 learned_addr learned my addr 172.21.15.46:0/2847168744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.679+0000 7fa7af7fe640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7b0106800 0x7fa7b0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7affff640 1 -- 172.21.15.46:0/2847168744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7b0106800 msgr2=0x7fa7b0198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7affff640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7b0106800 0x7fa7b0198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7affff640 1 -- 172.21.15.46:0/2847168744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7b01992d0 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7affff640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0100800 0x7fa7b01987d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0031460 tx=0x7fa7a00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7ad7fa640 1 -- 172.21.15.46:0/2847168744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa7a003f070 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7b0199560 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7ad7fa640 1 -- 172.21.15.46:0/2847168744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa7a0004cf0 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7ad7fa640 1 -- 172.21.15.46:0/2847168744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa7a0035030 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.680+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7b01a1b20 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.681+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7b0101f40 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.682+0000 7fa7ad7fa640 1 -- 172.21.15.46:0/2847168744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa7a0042070 con 0x7fa7b0100800 2024-09-10T04:46:10.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.682+0000 7fa7ad7fa640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa79807abf0 0x7fa79807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.683+0000 7fa7ad7fa640 1 -- 172.21.15.46:0/2847168744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa7a00be690 con 0x7fa7b0100800 2024-09-10T04:46:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.683+0000 7fa7af7fe640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa79807abf0 0x7fa79807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.683+0000 7fa7af7fe640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa79807abf0 0x7fa79807d0b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa7b006ad40 tx=0x7fa7a400a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:10.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.687+0000 7fa7ad7fa640 1 -- 172.21.15.46:0/2847168744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa7a0089dd0 con 0x7fa7b0100800 2024-09-10T04:46:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.952+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "versions"} v 0) -- 0x7fa7b0103a10 con 0x7fa7b0100800 2024-09-10T04:46:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.952+0000 7fa7ad7fa640 1 -- 172.21.15.46:0/2847168744 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) ==== 56+0+623 (secure 0 0 0) 0x7fa7a008ec80 con 0x7fa7b0100800 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa79807abf0 msgr2=0x7fa79807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa79807abf0 0x7fa79807d0b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa7b006ad40 tx=0x7fa7a400a2d0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0100800 msgr2=0x7fa7b01987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0100800 0x7fa7b01987d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0031460 tx=0x7fa7a00048f0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 shutdown_connections 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa79807abf0 0x7fa79807d0b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7b0106800 0x7fa7b0198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 --2- 172.21.15.46:0/2847168744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7b0100800 0x7fa7b01987d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:10.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 >> 172.21.15.46:0/2847168744 conn(0x7fa7b00fc520 msgr2=0x7fa7b00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 shutdown_connections 2024-09-10T04:46:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:10.956+0000 7fa7b690d640 1 -- 172.21.15.46:0/2847168744 wait complete. 2024-09-10T04:46:10.965 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)" 2024-09-10T04:46:11.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:11 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2847168744' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:46:11.633 DEBUG:teuthology.parallel:result is None 2024-09-10T04:46:11.634 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:46:11.647 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:46:11.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T04:46:11.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:11 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2847168744' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-10T04:46:11.910 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 -- 172.21.15.46:0/2279531622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 msgr2=0x7fc1d0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2279531622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d0101d60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009880 tx=0x7fc1c402a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 -- 172.21.15.46:0/2279531622 shutdown_connections 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2279531622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d0101d60 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2279531622 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d0107900 0x7fc1d0107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 -- 172.21.15.46:0/2279531622 >> 172.21.15.46:0/2279531622 conn(0x7fc1d00fd620 msgr2=0x7fc1d00ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 -- 172.21.15.46:0/2279531622 shutdown_connections 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.514+0000 7fc1d6057640 1 -- 172.21.15.46:0/2279531622 wait complete. 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1d6057640 1 Processor -- start 2024-09-10T04:46:12.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1d6057640 1 -- start start 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1d6057640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1d6057640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d0107900 0x7fc1d019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1d6057640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d010edd0 con 0x7fc1d0101900 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1d6057640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d010ec50 con 0x7fc1d0107900 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58268/0 (socket says 172.21.15.46:58268) 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.515+0000 7fc1cf7fe640 1 -- 172.21.15.46:0/2970867858 learned_addr learned my addr 172.21.15.46:0/2970867858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1ceffd640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d0107900 0x7fc1d019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1cf7fe640 1 -- 172.21.15.46:0/2970867858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d0107900 msgr2=0x7fc1d019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1cf7fe640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d0107900 0x7fc1d019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1cf7fe640 1 -- 172.21.15.46:0/2970867858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d0197820 con 0x7fc1d0101900 2024-09-10T04:46:12.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1cf7fe640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d019d6a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000d6f0 tx=0x7fc1c000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1ccff9640 1 -- 172.21.15.46:0/2970867858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1c0017070 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d0197b10 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1ccff9640 1 -- 172.21.15.46:0/2970867858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc1c000de00 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1ccff9640 1 -- 172.21.15.46:0/2970867858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc1c0004bb0 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.516+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d0198070 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.517+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d0103040 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.518+0000 7fc1ccff9640 1 -- 172.21.15.46:0/2970867858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc1c0009a30 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.518+0000 7fc1ccff9640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1b807abf0 0x7fc1b807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.518+0000 7fc1ccff9640 1 -- 172.21.15.46:0/2970867858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc1c009d8d0 con 0x7fc1d0101900 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.518+0000 7fc1ceffd640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1b807abf0 0x7fc1b807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:12.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.519+0000 7fc1ceffd640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1b807abf0 0x7fc1b807d0b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0198b70 tx=0x7fc1c4031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.523+0000 7fc1ccff9640 1 -- 172.21.15.46:0/2970867858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc1c0068f70 con 0x7fc1d0101900 2024-09-10T04:46:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.749+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc1d0109e70 con 0x7fc1d0101900 2024-09-10T04:46:12.750 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:12 smithi046 ceph-mon[109506]: pgmap v1219: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.749+0000 7fc1ccff9640 1 -- 172.21.15.46:0/2970867858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) ==== 76+0+2046 (secure 0 0 0) 0x7fc1c006de20 con 0x7fc1d0101900 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:e35 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:btime 2024-09-10T04:44:44:259333+0000 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 35 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T04:46:12.751 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-09-10T04:02:20.933717+0000 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-09-10T04:44:43.355403+0000 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 60 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 75 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 2 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1 2024-09-10T04:46:12.752 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=34328,1=34360} 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data enabled 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 1 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 34328 members: 34328,34360 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.hasdap{0:34328} state up:active seq 467 join_fscid=1 addr [v2:172.21.15.46:6826/4182274028,v1:172.21.15.46:6827/4182274028] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.ehvjgr{0:44341} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.70:6824/798617823,v1:172.21.15.70:6825/798617823] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi046.cpwyud{1:34360} state up:active seq 463 join_fscid=1 addr [v2:172.21.15.46:6828/2230370623,v1:172.21.15.46:6829/2230370623] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:46:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.cephfs.smithi070.myztiz{1:44967} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.70:6826/1908756829,v1:172.21.15.70:6827/1908756829] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T04:46:12.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:12.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 35 2024-09-10T04:46:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.752+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1b807abf0 msgr2=0x7fc1b807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.752+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1b807abf0 0x7fc1b807d0b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0198b70 tx=0x7fc1c4031400 comp rx=0 tx=0).stop 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.752+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 msgr2=0x7fc1d019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.752+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d019d6a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000d6f0 tx=0x7fc1c000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.753+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 shutdown_connections 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.753+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc1b807abf0 0x7fc1b807d0b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.753+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1d0107900 0x7fc1d019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.753+0000 7fc1d6057640 1 --2- 172.21.15.46:0/2970867858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1d0101900 0x7fc1d019d6a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.753+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 >> 172.21.15.46:0/2970867858 conn(0x7fc1d00fd620 msgr2=0x7fc1d00fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.753+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 shutdown_connections 2024-09-10T04:46:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:12.753+0000 7fc1d6057640 1 -- 172.21.15.46:0/2970867858 wait complete. 2024-09-10T04:46:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:12 smithi070 ceph-mon[79774]: pgmap v1219: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:13.457 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-09-10T04:46:13.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T04:46:13.735 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:13.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:13 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2970867858' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:46:13.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:13 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2970867858' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 -- 172.21.15.46:0/501021310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 msgr2=0x7f8218100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 --2- 172.21.15.46:0/501021310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f8218100a30 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8214009880 tx=0x7f821402a490 comp rx=0 tx=0).stop 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 -- 172.21.15.46:0/501021310 shutdown_connections 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 --2- 172.21.15.46:0/501021310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8218101000 0x7f821810dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 --2- 172.21.15.46:0/501021310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f8218100a30 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 -- 172.21.15.46:0/501021310 >> 172.21.15.46:0/501021310 conn(0x7f82180fc520 msgr2=0x7f82180fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 -- 172.21.15.46:0/501021310 shutdown_connections 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.318+0000 7f82201d8640 1 -- 172.21.15.46:0/501021310 wait complete. 2024-09-10T04:46:14.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f82201d8640 1 Processor -- start 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f82201d8640 1 -- start start 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f82201d8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f821819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f82201d8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8218101000 0x7f821819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f82201d8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8218110f90 con 0x7f8218100650 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f82201d8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f8218110e10 con 0x7f8218101000 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f821df74640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f821819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f821df74640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f821819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58288/0 (socket says 172.21.15.46:58288) 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.319+0000 7f821df74640 1 -- 172.21.15.46:0/3989846336 learned_addr learned my addr 172.21.15.46:0/3989846336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f821d773640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8218101000 0x7f821819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f821df74640 1 -- 172.21.15.46:0/3989846336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8218101000 msgr2=0x7f821819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f821df74640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8218101000 0x7f821819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f821df74640 1 -- 172.21.15.46:0/3989846336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8218199980 con 0x7f8218100650 2024-09-10T04:46:14.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f821d773640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8218101000 0x7f821819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f821df74640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f821819f7c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8214031460 tx=0x7f8214004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f820effd640 1 -- 172.21.15.46:0/3989846336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f821403f070 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f820effd640 1 -- 172.21.15.46:0/3989846336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f82140035f0 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f820effd640 1 -- 172.21.15.46:0/3989846336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f8214035030 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8218199c10 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.320+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821819a1a0 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.321+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821806aa90 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.322+0000 7f820effd640 1 -- 172.21.15.46:0/3989846336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f8214042070 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.322+0000 7f820effd640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f820407ac40 0x7f820407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.322+0000 7f820effd640 1 -- 172.21.15.46:0/3989846336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f82140be550 con 0x7f8218100650 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.322+0000 7f821d773640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f820407ac40 0x7f820407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.323+0000 7f821d773640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f820407ac40 0x7f820407d100 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8218102530 tx=0x7f8208005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:14.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.327+0000 7f820effd640 1 -- 172.21.15.46:0/3989846336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f8214089d10 con 0x7f8218100650 2024-09-10T04:46:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.550+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f821819aeb0 con 0x7f8218100650 2024-09-10T04:46:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.551+0000 7f820effd640 1 -- 172.21.15.46:0/3989846336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5355 (secure 0 0 0) 0x7f821408ebc0 con 0x7f8218100650 2024-09-10T04:46:14.552 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:14.552 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"btime":"2024-09-10T04:44:44:259333+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:43.355403+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:active","state_seq":467,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:active","state_seq":463,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44341":{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44967":{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34328,"qdb_cluster":[34328,34360]},"id":1}]} 2024-09-10T04:46:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 35 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f820407ac40 msgr2=0x7f820407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f820407ac40 0x7f820407d100 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8218102530 tx=0x7f8208005f90 comp rx=0 tx=0).stop 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 msgr2=0x7f821819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f821819f7c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8214031460 tx=0x7f8214004530 comp rx=0 tx=0).stop 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 shutdown_connections 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f820407ac40 0x7f820407d100 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8218101000 0x7f821819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 --2- 172.21.15.46:0/3989846336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8218100650 0x7f821819f7c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 >> 172.21.15.46:0/3989846336 conn(0x7f82180fc520 msgr2=0x7f82180fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 shutdown_connections 2024-09-10T04:46:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:14.553+0000 7f82201d8640 1 -- 172.21.15.46:0/3989846336 wait complete. 2024-09-10T04:46:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:14 smithi070 ceph-mon[79774]: pgmap v1220: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:14 smithi046 ceph-mon[109506]: pgmap v1220: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:15.161 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 16, 'max_mds': 2, 'flags': 50} 2024-09-10T04:46:15.161 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 17 2024-09-10T04:46:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:15.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:15 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3989846336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T04:46:15.700 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:15 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3989846336' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T04:46:15.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 -- 172.21.15.46:0/1160960018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c1069a0 msgr2=0x7fc66c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 --2- 172.21.15.46:0/1160960018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c1069a0 0x7fc66c106d80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc668009880 tx=0x7fc66802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 -- 172.21.15.46:0/1160960018 shutdown_connections 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 --2- 172.21.15.46:0/1160960018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c0ffdc0 0x7fc66c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 --2- 172.21.15.46:0/1160960018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c1069a0 0x7fc66c106d80 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 -- 172.21.15.46:0/1160960018 >> 172.21.15.46:0/1160960018 conn(0x7fc66c0fbc90 msgr2=0x7fc66c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 -- 172.21.15.46:0/1160960018 shutdown_connections 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.901+0000 7fc673aae640 1 -- 172.21.15.46:0/1160960018 wait complete. 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc673aae640 1 Processor -- start 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc673aae640 1 -- start start 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc673aae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c0ffdc0 0x7fc66c19d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc673aae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c1069a0 0x7fc66c19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:15.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc673aae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c10ef40 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc673aae640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fc66c10edc0 con 0x7fc66c1069a0 2024-09-10T04:46:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc67184a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c0ffdc0 0x7fc66c19d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc67184a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c0ffdc0 0x7fc66c19d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58310/0 (socket says 172.21.15.46:58310) 2024-09-10T04:46:15.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc67184a640 1 -- 172.21.15.46:0/1885419920 learned_addr learned my addr 172.21.15.46:0/1885419920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc671049640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c1069a0 0x7fc66c19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc67184a640 1 -- 172.21.15.46:0/1885419920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c1069a0 msgr2=0x7fc66c19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc67184a640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c1069a0 0x7fc66c19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc67184a640 1 -- 172.21.15.46:0/1885419920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc66c1978f0 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.902+0000 7fc67184a640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c0ffdc0 0x7fc66c19d730 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc668031460 tx=0x7fc6680048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.903+0000 7fc662ffd640 1 -- 172.21.15.46:0/1885419920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc66803f070 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.903+0000 7fc662ffd640 1 -- 172.21.15.46:0/1885419920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc668004cf0 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.903+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc66c197b80 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.903+0000 7fc662ffd640 1 -- 172.21.15.46:0/1885419920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fc668035030 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.903+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc66c1980b0 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.904+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc66c1030e0 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.904+0000 7fc662ffd640 1 -- 172.21.15.46:0/1885419920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fc668042070 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.905+0000 7fc662ffd640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65807ac40 0x7fc65807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.905+0000 7fc671049640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65807ac40 0x7fc65807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.905+0000 7fc662ffd640 1 -- 172.21.15.46:0/1885419920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fc6680be6d0 con 0x7fc66c0ffdc0 2024-09-10T04:46:15.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.905+0000 7fc671049640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65807ac40 0x7fc65807d100 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005fd0 tx=0x7fc65c005e20 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:15.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:15.910+0000 7fc662ffd640 1 -- 172.21.15.46:0/1885419920 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fc668089e90 con 0x7fc66c0ffdc0 2024-09-10T04:46:16.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.130+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) -- 0x7fc66c109ee0 con 0x7fc66c0ffdc0 2024-09-10T04:46:16.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.130+0000 7fc662ffd640 1 -- 172.21.15.46:0/1885419920 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v35) ==== 107+0+3644 (secure 0 0 0) 0x7fc66808ed40 con 0x7fc66c0ffdc0 2024-09-10T04:46:16.132 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:16.132 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"btime":"2024-09-10T04:13:25:742600+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:24.745137+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14558,"mds_1":24325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14558":{"gid":14558,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6827/4038650990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4038650990},{"type":"v1","addr":"172.21.15.46:6827","nonce":4038650990}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24325":{"gid":24325,"name":"cephfs.smithi070.ehvjgr","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.70:6825/1155442666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":1155442666},{"type":"v1","addr":"172.21.15.70:6825","nonce":1155442666}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14558,"qdb_cluster":[24325,14558]},"id":1}]} 2024-09-10T04:46:16.132 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 17 2024-09-10T04:46:16.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65807ac40 msgr2=0x7fc65807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:16.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65807ac40 0x7fc65807d100 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc65c005fd0 tx=0x7fc65c005e20 comp rx=0 tx=0).stop 2024-09-10T04:46:16.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c0ffdc0 msgr2=0x7fc66c19d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:16.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c0ffdc0 0x7fc66c19d730 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc668031460 tx=0x7fc6680048f0 comp rx=0 tx=0).stop 2024-09-10T04:46:16.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 shutdown_connections 2024-09-10T04:46:16.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fc65807ac40 0x7fc65807d100 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:16.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c1069a0 0x7fc66c19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:16.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 --2- 172.21.15.46:0/1885419920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc66c0ffdc0 0x7fc66c19d730 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:16.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 >> 172.21.15.46:0/1885419920 conn(0x7fc66c0fbc90 msgr2=0x7fc66c0fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:16.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 shutdown_connections 2024-09-10T04:46:16.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:16.133+0000 7fc673aae640 1 -- 172.21.15.46:0/1885419920 wait complete. 2024-09-10T04:46:16.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:16 smithi046 ceph-mon[109506]: pgmap v1221: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:16.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:16 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1885419920' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 17, "format": "json"} : dispatch 2024-09-10T04:46:16.813 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 17 2024-09-10T04:46:16.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 18 2024-09-10T04:46:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:16 smithi070 ceph-mon[79774]: pgmap v1221: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:16 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1885419920' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 17, "format": "json"} : dispatch 2024-09-10T04:46:17.077 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 -- 172.21.15.46:0/1823534294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88106800 msgr2=0x7f5b88106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 --2- 172.21.15.46:0/1823534294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88106800 0x7f5b88106be0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009880 tx=0x7f5b7802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 -- 172.21.15.46:0/1823534294 shutdown_connections 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 --2- 172.21.15.46:0/1823534294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b88100800 0x7f5b88100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 --2- 172.21.15.46:0/1823534294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88106800 0x7f5b88106be0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 -- 172.21.15.46:0/1823534294 >> 172.21.15.46:0/1823534294 conn(0x7f5b880fc520 msgr2=0x7f5b880fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 -- 172.21.15.46:0/1823534294 shutdown_connections 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.561+0000 7f5b8e531640 1 -- 172.21.15.46:0/1823534294 wait complete. 2024-09-10T04:46:17.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b8e531640 1 Processor -- start 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b8e531640 1 -- start start 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b8e531640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88100800 0x7f5b8819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b8e531640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b88106800 0x7f5b8819dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b8e531640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8810dcb0 con 0x7f5b88100800 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b8e531640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8810db30 con 0x7f5b88106800 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b87fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88100800 0x7f5b8819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b87fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88100800 0x7f5b8819d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58324/0 (socket says 172.21.15.46:58324) 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b87fff640 1 -- 172.21.15.46:0/3188468800 learned_addr learned my addr 172.21.15.46:0/3188468800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.562+0000 7f5b877fe640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b88106800 0x7f5b8819dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b87fff640 1 -- 172.21.15.46:0/3188468800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b88106800 msgr2=0x7f5b8819dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b87fff640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b88106800 0x7f5b8819dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b87fff640 1 -- 172.21.15.46:0/3188468800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b88197740 con 0x7f5b88100800 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b87fff640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88100800 0x7f5b8819d670 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031460 tx=0x7f5b780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:17.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b857fa640 1 -- 172.21.15.46:0/3188468800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b7803f070 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b881979d0 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b857fa640 1 -- 172.21.15.46:0/3188468800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b78004cf0 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b857fa640 1 -- 172.21.15.46:0/3188468800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5b78035030 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.563+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b88197f00 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.564+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b88101f40 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.565+0000 7f5b857fa640 1 -- 172.21.15.46:0/3188468800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5b78042070 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.565+0000 7f5b857fa640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5b7007abf0 0x7f5b7007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.565+0000 7f5b877fe640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5b7007abf0 0x7f5b7007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.565+0000 7f5b857fa640 1 -- 172.21.15.46:0/3188468800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f5b780be690 con 0x7f5b88100800 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.566+0000 7f5b877fe640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5b7007abf0 0x7f5b7007d0b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5b88198f90 tx=0x7f5b7c00a2d0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:17.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.570+0000 7f5b857fa640 1 -- 172.21.15.46:0/3188468800 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5b78089dd0 con 0x7f5b88100800 2024-09-10T04:46:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.796+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) -- 0x7f5b881981b0 con 0x7f5b88100800 2024-09-10T04:46:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.796+0000 7f5b857fa640 1 -- 172.21.15.46:0/3188468800 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v35) ==== 107+0+3603 (secure 0 0 0) 0x7f5b7808ec80 con 0x7f5b88100800 2024-09-10T04:46:17.797 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:17.798 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"btime":"2024-09-10T04:13:25:852311+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34316,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/184543618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":184543618},{"type":"v1","addr":"172.21.15.46:6829","nonce":184543618}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:17.798 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5b7007abf0 msgr2=0x7f5b7007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5b7007abf0 0x7f5b7007d0b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5b88198f90 tx=0x7f5b7c00a2d0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88100800 msgr2=0x7f5b8819d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88100800 0x7f5b8819d670 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5b78031460 tx=0x7f5b780048f0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 shutdown_connections 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f5b7007abf0 0x7f5b7007d0b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b88106800 0x7f5b8819dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 --2- 172.21.15.46:0/3188468800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b88100800 0x7f5b8819d670 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 >> 172.21.15.46:0/3188468800 conn(0x7f5b880fc520 msgr2=0x7f5b8806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 shutdown_connections 2024-09-10T04:46:17.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:17.800+0000 7f5b8e531640 1 -- 172.21.15.46:0/3188468800 wait complete. 2024-09-10T04:46:18.485 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:18 smithi046 ceph-mon[109506]: pgmap v1222: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:18.486 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:18 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3188468800' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 18, "format": "json"} : dispatch 2024-09-10T04:46:18.487 DEBUG:tasks.fs:max_mds not reduced in epoch 18 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:18.487 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 18 2024-09-10T04:46:18.487 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 19 2024-09-10T04:46:18.756 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:18 smithi070 ceph-mon[79774]: pgmap v1222: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:18 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3188468800' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 18, "format": "json"} : dispatch 2024-09-10T04:46:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 -- 172.21.15.46:0/4164526608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4428102ac0 msgr2=0x7f4428102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 --2- 172.21.15.46:0/4164526608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4428102ac0 0x7f4428102ea0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4418009880 tx=0x7f441802a530 comp rx=0 tx=0).stop 2024-09-10T04:46:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 -- 172.21.15.46:0/4164526608 shutdown_connections 2024-09-10T04:46:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 --2- 172.21.15.46:0/4164526608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44281033e0 0x7f442810b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 --2- 172.21.15.46:0/4164526608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4428102ac0 0x7f4428102ea0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 -- 172.21.15.46:0/4164526608 >> 172.21.15.46:0/4164526608 conn(0x7f44280fc520 msgr2=0x7f44280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 -- 172.21.15.46:0/4164526608 shutdown_connections 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.207+0000 7f442cb1f640 1 -- 172.21.15.46:0/4164526608 wait complete. 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.208+0000 7f442cb1f640 1 Processor -- start 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.208+0000 7f442cb1f640 1 -- start start 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.208+0000 7f442cb1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4428102ac0 0x7f442819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.208+0000 7f442cb1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44281033e0 0x7f442819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.208+0000 7f442cb1f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f442810edf0 con 0x7f44281033e0 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.208+0000 7f442cb1f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f442810ec70 con 0x7f4428102ac0 2024-09-10T04:46:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44281033e0 0x7f442819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44281033e0 0x7f442819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50374/0 (socket says 172.21.15.46:50374) 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441ffff640 1 -- 172.21.15.46:0/2886778787 learned_addr learned my addr 172.21.15.46:0/2886778787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f442659c640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4428102ac0 0x7f442819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441ffff640 1 -- 172.21.15.46:0/2886778787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4428102ac0 msgr2=0x7f442819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441ffff640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4428102ac0 0x7f442819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441ffff640 1 -- 172.21.15.46:0/2886778787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44281977c0 con 0x7f44281033e0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441ffff640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44281033e0 0x7f442819db80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f441000d6d0 tx=0x7f441000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441f7fe640 1 -- 172.21.15.46:0/2886778787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4410017070 con 0x7f44281033e0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441f7fe640 1 -- 172.21.15.46:0/2886778787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f44100054e0 con 0x7f44281033e0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4428197ab0 con 0x7f44281033e0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f441f7fe640 1 -- 172.21.15.46:0/2886778787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4410004ae0 con 0x7f44281033e0 2024-09-10T04:46:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.209+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4428198040 con 0x7f44281033e0 2024-09-10T04:46:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.210+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f442806aa90 con 0x7f44281033e0 2024-09-10T04:46:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.211+0000 7f441f7fe640 1 -- 172.21.15.46:0/2886778787 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4410004de0 con 0x7f44281033e0 2024-09-10T04:46:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.212+0000 7f441f7fe640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f441407ac00 0x7f441407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.212+0000 7f441f7fe640 1 -- 172.21.15.46:0/2886778787 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f441009d7c0 con 0x7f44281033e0 2024-09-10T04:46:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.212+0000 7f442659c640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f441407ac00 0x7f441407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.212+0000 7f442659c640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f441407ac00 0x7f441407d0c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f44180096c0 tx=0x7f4418031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.216+0000 7f441f7fe640 1 -- 172.21.15.46:0/2886778787 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4410068f00 con 0x7f44281033e0 2024-09-10T04:46:19.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.435+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) -- 0x7f44281982f0 con 0x7f44281033e0 2024-09-10T04:46:19.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.436+0000 7f441f7fe640 1 -- 172.21.15.46:0/2886778787 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v35) ==== 107+0+5297 (secure 0 0 0) 0x7f441006ddb0 con 0x7f44281033e0 2024-09-10T04:46:19.437 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:19.437 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":19,"btime":"2024-09-10T04:13:26:854128+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34316,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/184543618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":184543618},{"type":"v1","addr":"172.21.15.46:6829","nonce":184543618}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44303,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/2786504298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2786504298},{"type":"v1","addr":"172.21.15.70:6825","nonce":2786504298}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":44307,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/3836504153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":3836504153},{"type":"v1","addr":"172.21.15.46:6827","nonce":3836504153}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:19.438 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 19 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f441407ac00 msgr2=0x7f441407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f441407ac00 0x7f441407d0c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f44180096c0 tx=0x7f4418031400 comp rx=0 tx=0).stop 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44281033e0 msgr2=0x7f442819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44281033e0 0x7f442819db80 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f441000d6d0 tx=0x7f441000db90 comp rx=0 tx=0).stop 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 shutdown_connections 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f441407ac00 0x7f441407d0c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44281033e0 0x7f442819db80 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 --2- 172.21.15.46:0/2886778787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4428102ac0 0x7f442819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 >> 172.21.15.46:0/2886778787 conn(0x7f44280fc520 msgr2=0x7f442810af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 shutdown_connections 2024-09-10T04:46:19.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:19.439+0000 7f442cb1f640 1 -- 172.21.15.46:0/2886778787 wait complete. 2024-09-10T04:46:19.755 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:19 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2886778787' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 19, "format": "json"} : dispatch 2024-09-10T04:46:19.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:19 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2886778787' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 19, "format": "json"} : dispatch 2024-09-10T04:46:20.077 DEBUG:tasks.fs:max_mds not reduced in epoch 19 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:20.077 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-09-10T04:46:20.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 20 2024-09-10T04:46:20.343 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:20.733 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:20 smithi046 ceph-mon[109506]: pgmap v1223: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:20 smithi070 ceph-mon[79774]: pgmap v1223: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 -- 172.21.15.46:0/324826916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c1033d0 msgr2=0x7fa27c10b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 --2- 172.21.15.46:0/324826916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c1033d0 0x7fa27c10b8c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa264009880 tx=0x7fa26402a530 comp rx=0 tx=0).stop 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 -- 172.21.15.46:0/324826916 shutdown_connections 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 --2- 172.21.15.46:0/324826916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c1033d0 0x7fa27c10b8c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 --2- 172.21.15.46:0/324826916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa27c102ab0 0x7fa27c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 -- 172.21.15.46:0/324826916 >> 172.21.15.46:0/324826916 conn(0x7fa27c0fc550 msgr2=0x7fa27c0fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 -- 172.21.15.46:0/324826916 shutdown_connections 2024-09-10T04:46:20.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.815+0000 7fa281482640 1 -- 172.21.15.46:0/324826916 wait complete. 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.816+0000 7fa281482640 1 Processor -- start 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.816+0000 7fa281482640 1 -- start start 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.816+0000 7fa281482640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c102ab0 0x7fa27c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.816+0000 7fa281482640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa27c1033d0 0x7fa27c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.816+0000 7fa281482640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10edc0 con 0x7fa27c102ab0 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.816+0000 7fa281482640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa27c10ec40 con 0x7fa27c1033d0 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c102ab0 0x7fa27c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c102ab0 0x7fa27c19d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50398/0 (socket says 172.21.15.46:50398) 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27affd640 1 -- 172.21.15.46:0/3455087035 learned_addr learned my addr 172.21.15.46:0/3455087035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27a7fc640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa27c1033d0 0x7fa27c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27affd640 1 -- 172.21.15.46:0/3455087035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa27c1033d0 msgr2=0x7fa27c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27affd640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa27c1033d0 0x7fa27c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27affd640 1 -- 172.21.15.46:0/3455087035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27c197790 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa27affd640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c102ab0 0x7fa27c19d610 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ea30 tx=0x7fa26c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa273fff640 1 -- 172.21.15.46:0/3455087035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa26c00cbe0 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa273fff640 1 -- 172.21.15.46:0/3455087035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa26c00cd80 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27c197a80 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa273fff640 1 -- 172.21.15.46:0/3455087035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa26c01a920 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.817+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27c197fe0 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.818+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27c06a580 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.819+0000 7fa273fff640 1 -- 172.21.15.46:0/3455087035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa26c01aac0 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.819+0000 7fa273fff640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa26807ac40 0x7fa26807d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.820+0000 7fa273fff640 1 -- 172.21.15.46:0/3455087035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa26c09e2b0 con 0x7fa27c102ab0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.820+0000 7fa27a7fc640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa26807ac40 0x7fa26807d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.820+0000 7fa27a7fc640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa26807ac40 0x7fa26807d100 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa27c198ae0 tx=0x7fa264031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:20.824+0000 7fa273fff640 1 -- 172.21.15.46:0/3455087035 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa26c016030 con 0x7fa27c102ab0 2024-09-10T04:46:21.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.047+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) -- 0x7fa27c1982f0 con 0x7fa27c102ab0 2024-09-10T04:46:21.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.048+0000 7fa273fff640 1 -- 172.21.15.46:0/3455087035 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v35) ==== 107+0+4450 (secure 0 0 0) 0x7fa26c069a70 con 0x7fa27c102ab0 2024-09-10T04:46:21.049 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:21.049 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"btime":"2024-09-10T04:13:29:414168+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34316,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/184543618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":184543618},{"type":"v1","addr":"172.21.15.46:6829","nonce":184543618}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44303,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/2786504298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2786504298},{"type":"v1","addr":"172.21.15.70:6825","nonce":2786504298}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:21.049 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 20 2024-09-10T04:46:21.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa26807ac40 msgr2=0x7fa26807d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa26807ac40 0x7fa26807d100 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa27c198ae0 tx=0x7fa264031400 comp rx=0 tx=0).stop 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c102ab0 msgr2=0x7fa27c19d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c102ab0 0x7fa27c19d610 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00ea30 tx=0x7fa26c00eef0 comp rx=0 tx=0).stop 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 shutdown_connections 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa26807ac40 0x7fa26807d100 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa27c1033d0 0x7fa27c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 --2- 172.21.15.46:0/3455087035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c102ab0 0x7fa27c19d610 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.050+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 >> 172.21.15.46:0/3455087035 conn(0x7fa27c0fc550 msgr2=0x7fa27c10aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.051+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 shutdown_connections 2024-09-10T04:46:21.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:21.051+0000 7fa281482640 1 -- 172.21.15.46:0/3455087035 wait complete. 2024-09-10T04:46:21.645 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:21 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3455087035' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-10T04:46:21.676 DEBUG:tasks.fs:max_mds not reduced in epoch 20 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:21.676 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-09-10T04:46:21.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 21 2024-09-10T04:46:21.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:21 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3455087035' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-10T04:46:21.936 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 -- 172.21.15.46:0/3465296468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec1033e0 msgr2=0x7f62ec10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 --2- 172.21.15.46:0/3465296468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec1033e0 0x7f62ec10b8d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009880 tx=0x7f62e002a530 comp rx=0 tx=0).stop 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 -- 172.21.15.46:0/3465296468 shutdown_connections 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 --2- 172.21.15.46:0/3465296468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec1033e0 0x7f62ec10b8d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 --2- 172.21.15.46:0/3465296468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec102ac0 0x7f62ec102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 -- 172.21.15.46:0/3465296468 >> 172.21.15.46:0/3465296468 conn(0x7f62ec0fc520 msgr2=0x7f62ec0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 -- 172.21.15.46:0/3465296468 shutdown_connections 2024-09-10T04:46:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.431+0000 7f62f288a640 1 -- 172.21.15.46:0/3465296468 wait complete. 2024-09-10T04:46:22.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.432+0000 7f62f288a640 1 Processor -- start 2024-09-10T04:46:22.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.432+0000 7f62f288a640 1 -- start start 2024-09-10T04:46:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.432+0000 7f62f288a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec102ac0 0x7f62ec19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.432+0000 7f62f288a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec1033e0 0x7f62ec19dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:22.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.432+0000 7f62f288a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec10edd0 con 0x7f62ec102ac0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.432+0000 7f62f288a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec10ec50 con 0x7f62ec1033e0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62ebfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec102ac0 0x7f62ec19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62ebfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec102ac0 0x7f62ec19d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50414/0 (socket says 172.21.15.46:50414) 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62ebfff640 1 -- 172.21.15.46:0/1392422882 learned_addr learned my addr 172.21.15.46:0/1392422882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62eb7fe640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec1033e0 0x7f62ec19dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62ebfff640 1 -- 172.21.15.46:0/1392422882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec1033e0 msgr2=0x7f62ec19dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62ebfff640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec1033e0 0x7f62ec19dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62ebfff640 1 -- 172.21.15.46:0/1392422882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec1977e0 con 0x7f62ec102ac0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62ebfff640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec102ac0 0x7f62ec19d660 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d8d0 tx=0x7f62dc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62e97fa640 1 -- 172.21.15.46:0/1392422882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62dc017070 con 0x7f62ec102ac0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62e97fa640 1 -- 172.21.15.46:0/1392422882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f62dc010460 con 0x7f62ec102ac0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec197ad0 con 0x7f62ec102ac0 2024-09-10T04:46:22.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62e97fa640 1 -- 172.21.15.46:0/1392422882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f62dc00f5f0 con 0x7f62ec102ac0 2024-09-10T04:46:22.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.433+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec198030 con 0x7f62ec102ac0 2024-09-10T04:46:22.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.435+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec06aa90 con 0x7f62ec102ac0 2024-09-10T04:46:22.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.435+0000 7f62e97fa640 1 -- 172.21.15.46:0/1392422882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f62dc009a30 con 0x7f62ec102ac0 2024-09-10T04:46:22.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.436+0000 7f62e97fa640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62d407ac40 0x7f62d407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:22.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.436+0000 7f62e97fa640 1 -- 172.21.15.46:0/1392422882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f62dc09da60 con 0x7f62ec102ac0 2024-09-10T04:46:22.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.436+0000 7f62eb7fe640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62d407ac40 0x7f62d407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:22.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.436+0000 7f62eb7fe640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62d407ac40 0x7f62d407d100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f62ec198b30 tx=0x7f62e0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:22.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.441+0000 7f62e97fa640 1 -- 172.21.15.46:0/1392422882 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f62dc069220 con 0x7f62ec102ac0 2024-09-10T04:46:22.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.667+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) -- 0x7f62ec105f60 con 0x7f62ec102ac0 2024-09-10T04:46:22.669 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:22 smithi046 ceph-mon[109506]: pgmap v1224: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.668+0000 7f62e97fa640 1 -- 172.21.15.46:0/1392422882 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v35) ==== 107+0+5297 (secure 0 0 0) 0x7f62dc06e0d0 con 0x7f62ec102ac0 2024-09-10T04:46:22.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:22.670 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"btime":"2024-09-10T04:13:35:433662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34316,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/184543618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":184543618},{"type":"v1","addr":"172.21.15.46:6829","nonce":184543618}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44303,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/2786504298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2786504298},{"type":"v1","addr":"172.21.15.70:6825","nonce":2786504298}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 21 2024-09-10T04:46:22.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62d407ac40 msgr2=0x7f62d407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62d407ac40 0x7f62d407d100 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f62ec198b30 tx=0x7f62e0031400 comp rx=0 tx=0).stop 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec102ac0 msgr2=0x7f62ec19d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec102ac0 0x7f62ec19d660 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00d8d0 tx=0x7f62dc00dd90 comp rx=0 tx=0).stop 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 shutdown_connections 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f62d407ac40 0x7f62d407d100 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62ec1033e0 0x7f62ec19dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 --2- 172.21.15.46:0/1392422882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62ec102ac0 0x7f62ec19d660 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 >> 172.21.15.46:0/1392422882 conn(0x7f62ec0fc520 msgr2=0x7f62ec108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 shutdown_connections 2024-09-10T04:46:22.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:22.670+0000 7f62f288a640 1 -- 172.21.15.46:0/1392422882 wait complete. 2024-09-10T04:46:22.836 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:22 smithi070 ceph-mon[79774]: pgmap v1224: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:23.343 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:23.343 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-09-10T04:46:23.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 22 2024-09-10T04:46:23.609 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:23.671 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:23 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1392422882' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-10T04:46:23.671 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:23 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1392422882' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-10T04:46:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 -- 172.21.15.46:0/925237198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100250 msgr2=0x7fdad8100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 --2- 172.21.15.46:0/925237198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100250 0x7fdad8100630 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009880 tx=0x7fdac802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 -- 172.21.15.46:0/925237198 shutdown_connections 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 --2- 172.21.15.46:0/925237198 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdad8100c00 0x7fdad8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 --2- 172.21.15.46:0/925237198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100250 0x7fdad8100630 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 -- 172.21.15.46:0/925237198 >> 172.21.15.46:0/925237198 conn(0x7fdad80fbc90 msgr2=0x7fdad80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 -- 172.21.15.46:0/925237198 shutdown_connections 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.114+0000 7fdadd784640 1 -- 172.21.15.46:0/925237198 wait complete. 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.115+0000 7fdadd784640 1 Processor -- start 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.115+0000 7fdadd784640 1 -- start start 2024-09-10T04:46:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdadd784640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdad8100250 0x7fdad8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdadd784640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100c00 0x7fdad8102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdadd784640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdad810cc40 con 0x7fdad8100c00 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdadd784640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fdad810cac0 con 0x7fdad8100250 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100c00 0x7fdad8102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad67fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100c00 0x7fdad8102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50428/0 (socket says 172.21.15.46:50428) 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad67fc640 1 -- 172.21.15.46:0/3270929510 learned_addr learned my addr 172.21.15.46:0/3270929510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad6ffd640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdad8100250 0x7fdad8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad67fc640 1 -- 172.21.15.46:0/3270929510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdad8100250 msgr2=0x7fdad8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad67fc640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdad8100250 0x7fdad8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad67fc640 1 -- 172.21.15.46:0/3270929510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad8102770 con 0x7fdad8100c00 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdad67fc640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100c00 0x7fdad8102230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00db40 tx=0x7fdacc00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdabffff640 1 -- 172.21.15.46:0/3270929510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdacc00ac90 con 0x7fdad8100c00 2024-09-10T04:46:24.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad8102a60 con 0x7fdad8100c00 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdabffff640 1 -- 172.21.15.46:0/3270929510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdacc00ae30 con 0x7fdad8100c00 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdabffff640 1 -- 172.21.15.46:0/3270929510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fdacc00e680 con 0x7fdad8100c00 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.116+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad81a3e90 con 0x7fdad8100c00 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.117+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad8101730 con 0x7fdad8100c00 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.118+0000 7fdabffff640 1 -- 172.21.15.46:0/3270929510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fdacc00e980 con 0x7fdad8100c00 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.118+0000 7fdabffff640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdac007ac00 0x7fdac007d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.118+0000 7fdabffff640 1 -- 172.21.15.46:0/3270929510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fdacc09dd70 con 0x7fdad8100c00 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.119+0000 7fdad6ffd640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdac007ac00 0x7fdac007d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.119+0000 7fdad6ffd640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdac007ac00 0x7fdac007d0c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdac8031460 tx=0x7fdac80095c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.123+0000 7fdabffff640 1 -- 172.21.15.46:0/3270929510 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fdacc0694b0 con 0x7fdad8100c00 2024-09-10T04:46:24.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.346+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) -- 0x7fdad8100630 con 0x7fdad8100c00 2024-09-10T04:46:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.347+0000 7fdabffff640 1 -- 172.21.15.46:0/3270929510 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v35) ==== 107+0+4453 (secure 0 0 0) 0x7fdacc06e360 con 0x7fdad8100c00 2024-09-10T04:46:24.348 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"btime":"2024-09-10T04:13:47:401677+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44303,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/2786504298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2786504298},{"type":"v1","addr":"172.21.15.70:6825","nonce":2786504298}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:24.348 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 22 2024-09-10T04:46:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdac007ac00 msgr2=0x7fdac007d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdac007ac00 0x7fdac007d0c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdac8031460 tx=0x7fdac80095c0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100c00 msgr2=0x7fdad8102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100c00 0x7fdad8102230 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00db40 tx=0x7fdacc00a250 comp rx=0 tx=0).stop 2024-09-10T04:46:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 shutdown_connections 2024-09-10T04:46:24.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fdac007ac00 0x7fdac007d0c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad8100c00 0x7fdad8102230 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 --2- 172.21.15.46:0/3270929510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdad8100250 0x7fdad8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:24.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 >> 172.21.15.46:0/3270929510 conn(0x7fdad80fbc90 msgr2=0x7fdad80fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:24.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 shutdown_connections 2024-09-10T04:46:24.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:24.349+0000 7fdadd784640 1 -- 172.21.15.46:0/3270929510 wait complete. 2024-09-10T04:46:24.640 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:24 smithi046 ceph-mon[109506]: pgmap v1225: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:24.640 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:24 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3270929510' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-10T04:46:24.736 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:24 smithi070 ceph-mon[79774]: pgmap v1225: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:24.736 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:24 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3270929510' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-10T04:46:25.042 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:25.042 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-09-10T04:46:25.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 23 2024-09-10T04:46:25.305 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 -- 172.21.15.46:0/3919381522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54181033d0 msgr2=0x7f541810b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3919381522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54181033d0 0x7f541810b8c0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f540c009880 tx=0x7f540c02a530 comp rx=0 tx=0).stop 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 -- 172.21.15.46:0/3919381522 shutdown_connections 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3919381522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54181033d0 0x7f541810b8c0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3919381522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5418102ab0 0x7f5418102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 -- 172.21.15.46:0/3919381522 >> 172.21.15.46:0/3919381522 conn(0x7f54180fc550 msgr2=0x7f54180fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 -- 172.21.15.46:0/3919381522 shutdown_connections 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 -- 172.21.15.46:0/3919381522 wait complete. 2024-09-10T04:46:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 Processor -- start 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.774+0000 7f541fc1b640 1 -- start start 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541fc1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5418102ab0 0x7f541819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541fc1b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54181033d0 0x7f541819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541fc1b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f541810ede0 con 0x7f5418102ab0 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541fc1b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f541810ec60 con 0x7f54181033d0 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d9b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5418102ab0 0x7f541819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d9b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5418102ab0 0x7f541819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50446/0 (socket says 172.21.15.46:50446) 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d9b7640 1 -- 172.21.15.46:0/3583733265 learned_addr learned my addr 172.21.15.46:0/3583733265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d1b6640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54181033d0 0x7f541819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d9b7640 1 -- 172.21.15.46:0/3583733265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54181033d0 msgr2=0x7f541819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d9b7640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54181033d0 0x7f541819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d9b7640 1 -- 172.21.15.46:0/3583733265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5418197810 con 0x7f5418102ab0 2024-09-10T04:46:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541d9b7640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5418102ab0 0x7f541819d690 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f541000e970 tx=0x7f541000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f5416ffd640 1 -- 172.21.15.46:0/3583733265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f541000cac0 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5418197b00 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.775+0000 7f5416ffd640 1 -- 172.21.15.46:0/3583733265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f54100054e0 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.776+0000 7f5416ffd640 1 -- 172.21.15.46:0/3583733265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f5410010430 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.776+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5418198060 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.777+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541806a580 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.777+0000 7f5416ffd640 1 -- 172.21.15.46:0/3583733265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f5410004f00 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.778+0000 7f5416ffd640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f540007abf0 0x7f540007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.778+0000 7f5416ffd640 1 -- 172.21.15.46:0/3583733265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f541009e3f0 con 0x7f5418102ab0 2024-09-10T04:46:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.778+0000 7f541d1b6640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f540007abf0 0x7f540007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.778+0000 7f541d1b6640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f540007abf0 0x7f540007d0b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5418198b60 tx=0x7f540c031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:25.782+0000 7f5416ffd640 1 -- 172.21.15.46:0/3583733265 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f5410069b30 con 0x7f5418102ab0 2024-09-10T04:46:26.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.007+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) -- 0x7f5418105f50 con 0x7f5418102ab0 2024-09-10T04:46:26.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.008+0000 7f5416ffd640 1 -- 172.21.15.46:0/3583733265 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v35) ==== 107+0+5300 (secure 0 0 0) 0x7f541006ce20 con 0x7f5418102ab0 2024-09-10T04:46:26.009 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:26.009 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"btime":"2024-09-10T04:13:52:416515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44303,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/2786504298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2786504298},{"type":"v1","addr":"172.21.15.70:6825","nonce":2786504298}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:26.009 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 23 2024-09-10T04:46:26.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f540007abf0 msgr2=0x7f540007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:26.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f540007abf0 0x7f540007d0b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5418198b60 tx=0x7f540c031400 comp rx=0 tx=0).stop 2024-09-10T04:46:26.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5418102ab0 msgr2=0x7f541819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5418102ab0 0x7f541819d690 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f541000e970 tx=0x7f541000ed70 comp rx=0 tx=0).stop 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 shutdown_connections 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f540007abf0 0x7f540007d0b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54181033d0 0x7f541819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 --2- 172.21.15.46:0/3583733265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5418102ab0 0x7f541819d690 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.010+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 >> 172.21.15.46:0/3583733265 conn(0x7f54180fc550 msgr2=0x7f5418108ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.011+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 shutdown_connections 2024-09-10T04:46:26.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:26.011+0000 7f541fc1b640 1 -- 172.21.15.46:0/3583733265 wait complete. 2024-09-10T04:46:26.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:26 smithi046 ceph-mon[109506]: pgmap v1226: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:26.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:26 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3583733265' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-10T04:46:26.768 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:26.768 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-09-10T04:46:26.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 24 2024-09-10T04:46:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:26 smithi070 ceph-mon[79774]: pgmap v1226: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:26 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3583733265' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-10T04:46:27.031 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:27.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 -- 172.21.15.46:0/35415582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078107900 msgr2=0x7fa078107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:27.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 --2- 172.21.15.46:0/35415582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078107900 0x7fa078107ce0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fa070009c20 tx=0x7fa07002a970 comp rx=0 tx=0).stop 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 -- 172.21.15.46:0/35415582 shutdown_connections 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 --2- 172.21.15.46:0/35415582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa078101900 0x7fa078101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 --2- 172.21.15.46:0/35415582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078107900 0x7fa078107ce0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 -- 172.21.15.46:0/35415582 >> 172.21.15.46:0/35415582 conn(0x7fa0780fd600 msgr2=0x7fa0780ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 -- 172.21.15.46:0/35415582 shutdown_connections 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 -- 172.21.15.46:0/35415582 wait complete. 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 Processor -- start 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.507+0000 7fa07ee45640 1 -- start start 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07ee45640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078101900 0x7fa07819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:27.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07ee45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa078107900 0x7fa07819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07ee45640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780756c0 con 0x7fa078101900 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07ee45640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fa078075540 con 0x7fa078107900 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07cbe1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078101900 0x7fa07819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07cbe1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078101900 0x7fa07819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50462/0 (socket says 172.21.15.46:50462) 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07cbe1640 1 -- 172.21.15.46:0/960755665 learned_addr learned my addr 172.21.15.46:0/960755665 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa077fff640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa078107900 0x7fa07819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07cbe1640 1 -- 172.21.15.46:0/960755665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa078107900 msgr2=0x7fa07819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07cbe1640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa078107900 0x7fa07819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07cbe1640 1 -- 172.21.15.46:0/960755665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa078199a10 con 0x7fa078101900 2024-09-10T04:46:27.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa077fff640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa078107900 0x7fa07819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07cbe1640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078101900 0x7fa07819f850 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa07002ae90 tx=0x7fa07002f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa075ffb640 1 -- 172.21.15.46:0/960755665 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa070047070 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa075ffb640 1 -- 172.21.15.46:0/960755665 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa07002f7e0 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa078199ca0 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.508+0000 7fa075ffb640 1 -- 172.21.15.46:0/960755665 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fa070034070 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.509+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa07819a1d0 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.510+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078103040 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.510+0000 7fa075ffb640 1 -- 172.21.15.46:0/960755665 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fa07004a070 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.511+0000 7fa075ffb640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa06007ac40 0x7fa06007d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.511+0000 7fa075ffb640 1 -- 172.21.15.46:0/960755665 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fa0700be9b0 con 0x7fa078101900 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.511+0000 7fa077fff640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa06007ac40 0x7fa06007d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.511+0000 7fa077fff640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa06007ac40 0x7fa06007d100 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0058c0 tx=0x7fa06c005850 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:27.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.515+0000 7fa075ffb640 1 -- 172.21.15.46:0/960755665 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fa07003f070 con 0x7fa078101900 2024-09-10T04:46:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.736+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) -- 0x7fa07819afe0 con 0x7fa078101900 2024-09-10T04:46:27.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.737+0000 7fa075ffb640 1 -- 172.21.15.46:0/960755665 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v35) ==== 107+0+4453 (secure 0 0 0) 0x7fa07008a0d0 con 0x7fa078101900 2024-09-10T04:46:27.738 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:27.738 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"btime":"2024-09-10T04:14:03:997349+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa06007ac40 msgr2=0x7fa06007d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa06007ac40 0x7fa06007d100 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0058c0 tx=0x7fa06c005850 comp rx=0 tx=0).stop 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078101900 msgr2=0x7fa07819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078101900 0x7fa07819f850 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fa07002ae90 tx=0x7fa07002f5f0 comp rx=0 tx=0).stop 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 shutdown_connections 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fa06007ac40 0x7fa06007d100 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa078107900 0x7fa07819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 --2- 172.21.15.46:0/960755665 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa078101900 0x7fa07819f850 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 >> 172.21.15.46:0/960755665 conn(0x7fa0780fd600 msgr2=0x7fa0780fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 shutdown_connections 2024-09-10T04:46:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:27.739+0000 7fa07ee45640 1 -- 172.21.15.46:0/960755665 wait complete. 2024-09-10T04:46:28.417 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:28.417 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-09-10T04:46:28.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 25 2024-09-10T04:46:28.678 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:28.767 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:28 smithi046 ceph-mon[109506]: pgmap v1227: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:28.767 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:28 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/960755665' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-10T04:46:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:28 smithi070 ceph-mon[79774]: pgmap v1227: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:28 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/960755665' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-10T04:46:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 -- 172.21.15.46:0/3557948520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 msgr2=0x7f7d10100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 --2- 172.21.15.46:0/3557948520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d10100c50 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009880 tx=0x7f7cf802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 -- 172.21.15.46:0/3557948520 shutdown_connections 2024-09-10T04:46:29.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 --2- 172.21.15.46:0/3557948520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d10100c50 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 --2- 172.21.15.46:0/3557948520 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d101067f0 0x7f7d10106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 -- 172.21.15.46:0/3557948520 >> 172.21.15.46:0/3557948520 conn(0x7f7d100fc550 msgr2=0x7f7d100fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 -- 172.21.15.46:0/3557948520 shutdown_connections 2024-09-10T04:46:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.179+0000 7f7d15792640 1 -- 172.21.15.46:0/3557948520 wait complete. 2024-09-10T04:46:29.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d15792640 1 Processor -- start 2024-09-10T04:46:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d15792640 1 -- start start 2024-09-10T04:46:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d15792640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d100fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d15792640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d101067f0 0x7f7d100fd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d15792640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010dcc0 con 0x7f7d101007f0 2024-09-10T04:46:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d15792640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010db40 con 0x7f7d101067f0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d0effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d100fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d0effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d100fccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58798/0 (socket says 172.21.15.46:58798) 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d0effd640 1 -- 172.21.15.46:0/3786226378 learned_addr learned my addr 172.21.15.46:0/3786226378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d0e7fc640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d101067f0 0x7f7d100fd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.180+0000 7f7d0effd640 1 -- 172.21.15.46:0/3786226378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d101067f0 msgr2=0x7f7d100fd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d0effd640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d101067f0 0x7f7d100fd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d0effd640 1 -- 172.21.15.46:0/3786226378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d100fd750 con 0x7f7d101007f0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d0effd640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d100fccd0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000cc60 tx=0x7f7d0000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d07fff640 1 -- 172.21.15.46:0/3786226378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7d0000ad30 con 0x7f7d101007f0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d100fda40 con 0x7f7d101007f0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d07fff640 1 -- 172.21.15.46:0/3786226378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7d00005020 con 0x7f7d101007f0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d07fff640 1 -- 172.21.15.46:0/3786226378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7d0000f640 con 0x7f7d101007f0 2024-09-10T04:46:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.181+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d101a3e60 con 0x7f7d101007f0 2024-09-10T04:46:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.182+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d10101f30 con 0x7f7d101007f0 2024-09-10T04:46:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.183+0000 7f7d07fff640 1 -- 172.21.15.46:0/3786226378 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7d000051d0 con 0x7f7d101007f0 2024-09-10T04:46:29.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.183+0000 7f7d07fff640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cfc07ac50 0x7f7cfc07d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.183+0000 7f7d07fff640 1 -- 172.21.15.46:0/3786226378 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f7d0009de70 con 0x7f7d101007f0 2024-09-10T04:46:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.183+0000 7f7d0e7fc640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cfc07ac50 0x7f7cfc07d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:29.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.184+0000 7f7d0e7fc640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cfc07ac50 0x7f7cfc07d110 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80096c0 tx=0x7f7cf8000bc0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:29.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.188+0000 7f7d07fff640 1 -- 172.21.15.46:0/3786226378 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7d00014030 con 0x7f7d101007f0 2024-09-10T04:46:29.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.410+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) -- 0x7f7d10100c50 con 0x7f7d101007f0 2024-09-10T04:46:29.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.410+0000 7f7d07fff640 1 -- 172.21.15.46:0/3786226378 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v35) ==== 107+0+5297 (secure 0 0 0) 0x7f7d00069630 con 0x7f7d101007f0 2024-09-10T04:46:29.412 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:29.412 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"btime":"2024-09-10T04:14:07:776361+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44295,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/470769983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":470769983},{"type":"v1","addr":"172.21.15.70:6827","nonce":470769983}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:29.412 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-09-10T04:46:29.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cfc07ac50 msgr2=0x7f7cfc07d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cfc07ac50 0x7f7cfc07d110 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80096c0 tx=0x7f7cf8000bc0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 msgr2=0x7f7d100fccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d100fccd0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000cc60 tx=0x7f7d0000a620 comp rx=0 tx=0).stop 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 shutdown_connections 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f7cfc07ac50 0x7f7cfc07d110 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d101067f0 0x7f7d100fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 --2- 172.21.15.46:0/3786226378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d101007f0 0x7f7d100fccd0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 >> 172.21.15.46:0/3786226378 conn(0x7f7d100fc550 msgr2=0x7f7d10069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 shutdown_connections 2024-09-10T04:46:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:29.413+0000 7f7d15792640 1 -- 172.21.15.46:0/3786226378 wait complete. 2024-09-10T04:46:29.718 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:29 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3786226378' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-10T04:46:29.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:29 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3786226378' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-10T04:46:30.032 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:30.033 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-09-10T04:46:30.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 26 2024-09-10T04:46:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:30.738 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:30 smithi046 ceph-mon[109506]: pgmap v1228: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:30 smithi070 ceph-mon[79774]: pgmap v1228: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:30.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3558246693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38107900 msgr2=0x7f6d38107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3558246693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38107900 0x7f6d38107ce0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009880 tx=0x7f6d2802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3558246693 shutdown_connections 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3558246693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d38101900 0x7f6d38101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3558246693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38107900 0x7f6d38107ce0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3558246693 >> 172.21.15.46:0/3558246693 conn(0x7f6d380fd600 msgr2=0x7f6d380ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3558246693 shutdown_connections 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.820+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3558246693 wait complete. 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3cbd4640 1 Processor -- start 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3cbd4640 1 -- start start 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3cbd4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38101900 0x7f6d380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3cbd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d38107900 0x7f6d38074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3cbd4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810edd0 con 0x7f6d38101900 2024-09-10T04:46:30.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3cbd4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810ec50 con 0x7f6d38107900 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38101900 0x7f6d380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d2ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d38107900 0x7f6d38074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38101900 0x7f6d380744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58814/0 (socket says 172.21.15.46:58814) 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.821+0000 7f6d3659c640 1 -- 172.21.15.46:0/3916991550 learned_addr learned my addr 172.21.15.46:0/3916991550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d3659c640 1 -- 172.21.15.46:0/3916991550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d38107900 msgr2=0x7f6d38074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d3659c640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d38107900 0x7f6d38074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d3659c640 1 -- 172.21.15.46:0/3916991550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d38074f70 con 0x7f6d38101900 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d3659c640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38101900 0x7f6d380744f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6d28031460 tx=0x7f6d280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d2f7fe640 1 -- 172.21.15.46:0/3916991550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d2803f070 con 0x7f6d38101900 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d2f7fe640 1 -- 172.21.15.46:0/3916991550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6d280035f0 con 0x7f6d38101900 2024-09-10T04:46:30.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d38075200 con 0x7f6d38101900 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d2f7fe640 1 -- 172.21.15.46:0/3916991550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6d28035030 con 0x7f6d38101900 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.822+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d381a3d80 con 0x7f6d38101900 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.823+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d38103040 con 0x7f6d38101900 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.824+0000 7f6d2f7fe640 1 -- 172.21.15.46:0/3916991550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6d28042070 con 0x7f6d38101900 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.824+0000 7f6d2f7fe640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d2407abf0 0x7f6d2407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.825+0000 7f6d2ffff640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d2407abf0 0x7f6d2407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.825+0000 7f6d2f7fe640 1 -- 172.21.15.46:0/3916991550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6d280be600 con 0x7f6d38101900 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.825+0000 7f6d2ffff640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d2407abf0 0x7f6d2407d0b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d38075f80 tx=0x7f6d2000b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:30.829+0000 7f6d2f7fe640 1 -- 172.21.15.46:0/3916991550 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6d28089d40 con 0x7f6d38101900 2024-09-10T04:46:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.053+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) -- 0x7f6d38109e70 con 0x7f6d38101900 2024-09-10T04:46:31.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.053+0000 7f6d2f7fe640 1 -- 172.21.15.46:0/3916991550 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v35) ==== 107+0+4453 (secure 0 0 0) 0x7f6d2808ebf0 con 0x7f6d38101900 2024-09-10T04:46:31.054 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:31.055 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"btime":"2024-09-10T04:29:11:083458+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:31.055 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-09-10T04:46:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d2407abf0 msgr2=0x7f6d2407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d2407abf0 0x7f6d2407d0b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f6d38075f80 tx=0x7f6d2000b390 comp rx=0 tx=0).stop 2024-09-10T04:46:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38101900 msgr2=0x7f6d380744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38101900 0x7f6d380744f0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6d28031460 tx=0x7f6d280045e0 comp rx=0 tx=0).stop 2024-09-10T04:46:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 shutdown_connections 2024-09-10T04:46:31.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6d2407abf0 0x7f6d2407d0b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d38107900 0x7f6d38074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 --2- 172.21.15.46:0/3916991550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d38101900 0x7f6d380744f0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 >> 172.21.15.46:0/3916991550 conn(0x7f6d380fd600 msgr2=0x7f6d380fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 shutdown_connections 2024-09-10T04:46:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:31.056+0000 7f6d3cbd4640 1 -- 172.21.15.46:0/3916991550 wait complete. 2024-09-10T04:46:31.659 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:31 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3916991550' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-10T04:46:31.692 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:31.692 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-09-10T04:46:31.692 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 27 2024-09-10T04:46:31.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:31 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3916991550' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-10T04:46:31.955 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.454+0000 7f38a0fd6640 1 -- 172.21.15.46:0/2918003681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 msgr2=0x7f389c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.454+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/2918003681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c10b8d0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3884009880 tx=0x7f388402a530 comp rx=0 tx=0).stop 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.454+0000 7f38a0fd6640 1 -- 172.21.15.46:0/2918003681 shutdown_connections 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.454+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/2918003681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c10b8d0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.454+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/2918003681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f389c102ac0 0x7f389c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.454+0000 7f38a0fd6640 1 -- 172.21.15.46:0/2918003681 >> 172.21.15.46:0/2918003681 conn(0x7f389c0fc520 msgr2=0x7f389c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.455+0000 7f38a0fd6640 1 -- 172.21.15.46:0/2918003681 shutdown_connections 2024-09-10T04:46:32.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.455+0000 7f38a0fd6640 1 -- 172.21.15.46:0/2918003681 wait complete. 2024-09-10T04:46:32.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.455+0000 7f38a0fd6640 1 Processor -- start 2024-09-10T04:46:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.455+0000 7f38a0fd6640 1 -- start start 2024-09-10T04:46:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f38a0fd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f389c102ac0 0x7f389c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f38a0fd6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c100c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f38a0fd6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f389c10edf0 con 0x7f389c1033e0 2024-09-10T04:46:32.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f38a0fd6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f389c10ec70 con 0x7f389c102ac0 2024-09-10T04:46:32.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f3899d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:32.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f3899d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58832/0 (socket says 172.21.15.46:58832) 2024-09-10T04:46:32.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f3899d9b640 1 -- 172.21.15.46:0/4099000557 learned_addr learned my addr 172.21.15.46:0/4099000557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:32.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f389a59c640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f389c102ac0 0x7f389c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:32.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f3899d9b640 1 -- 172.21.15.46:0/4099000557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f389c102ac0 msgr2=0x7f389c100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:32.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f3899d9b640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f389c102ac0 0x7f389c100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:32.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f3899d9b640 1 -- 172.21.15.46:0/4099000557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c1011a0 con 0x7f389c1033e0 2024-09-10T04:46:32.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f3899d9b640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c100c60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3884031460 tx=0x7f3884004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:32.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.456+0000 7f38937fe640 1 -- 172.21.15.46:0/4099000557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f388403f070 con 0x7f389c1033e0 2024-09-10T04:46:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.457+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c101430 con 0x7f389c1033e0 2024-09-10T04:46:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.457+0000 7f38937fe640 1 -- 172.21.15.46:0/4099000557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3884004a90 con 0x7f389c1033e0 2024-09-10T04:46:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.457+0000 7f38937fe640 1 -- 172.21.15.46:0/4099000557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3884035030 con 0x7f389c1033e0 2024-09-10T04:46:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.457+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c1a3de0 con 0x7f389c1033e0 2024-09-10T04:46:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.458+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c06aa90 con 0x7f389c1033e0 2024-09-10T04:46:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.458+0000 7f38937fe640 1 -- 172.21.15.46:0/4099000557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3884042070 con 0x7f389c1033e0 2024-09-10T04:46:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.459+0000 7f38937fe640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f388807ac00 0x7f388807d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.459+0000 7f38937fe640 1 -- 172.21.15.46:0/4099000557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f38840be6e0 con 0x7f389c1033e0 2024-09-10T04:46:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.459+0000 7f389a59c640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f388807ac00 0x7f388807d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.459+0000 7f389a59c640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f388807ac00 0x7f388807d0c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f388c005fd0 tx=0x7f388c00b390 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:32.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.463+0000 7f38937fe640 1 -- 172.21.15.46:0/4099000557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3884089e20 con 0x7f389c1033e0 2024-09-10T04:46:32.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.686+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) -- 0x7f389c105f60 con 0x7f389c1033e0 2024-09-10T04:46:32.687 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:32 smithi046 ceph-mon[109506]: pgmap v1229: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.687+0000 7f38937fe640 1 -- 172.21.15.46:0/4099000557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v35) ==== 107+0+5300 (secure 0 0 0) 0x7f388408ecd0 con 0x7f389c1033e0 2024-09-10T04:46:32.688 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:32.688 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":27,"btime":"2024-09-10T04:29:16:475094+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:13:25.852309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:32.688 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 27 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f388807ac00 msgr2=0x7f388807d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f388807ac00 0x7f388807d0c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f388c005fd0 tx=0x7f388c00b390 comp rx=0 tx=0).stop 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 msgr2=0x7f389c100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c100c60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3884031460 tx=0x7f3884004530 comp rx=0 tx=0).stop 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 shutdown_connections 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f388807ac00 0x7f388807d0c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f389c1033e0 0x7f389c100c60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:32.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 --2- 172.21.15.46:0/4099000557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f389c102ac0 0x7f389c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 >> 172.21.15.46:0/4099000557 conn(0x7f389c0fc520 msgr2=0x7f389c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 shutdown_connections 2024-09-10T04:46:32.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:32.689+0000 7f38a0fd6640 1 -- 172.21.15.46:0/4099000557 wait complete. 2024-09-10T04:46:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:32 smithi070 ceph-mon[79774]: pgmap v1229: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:33.384 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T04:46:33.385 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-09-10T04:46:33.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 28 2024-09-10T04:46:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:33.768 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:33 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/4099000557' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-10T04:46:33.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:33 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/4099000557' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-10T04:46:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 -- 172.21.15.46:0/2424365070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4841034a0 msgr2=0x7ff48410b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 --2- 172.21.15.46:0/2424365070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4841034a0 0x7ff48410b990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff478009c20 tx=0x7ff47802aa00 comp rx=0 tx=0).stop 2024-09-10T04:46:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 -- 172.21.15.46:0/2424365070 shutdown_connections 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 --2- 172.21.15.46:0/2424365070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4841034a0 0x7ff48410b990 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 --2- 172.21.15.46:0/2424365070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff484102b80 0x7ff484102f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 -- 172.21.15.46:0/2424365070 >> 172.21.15.46:0/2424365070 conn(0x7ff4840fc520 msgr2=0x7ff4840fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 -- 172.21.15.46:0/2424365070 shutdown_connections 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.109+0000 7ff48c752640 1 -- 172.21.15.46:0/2424365070 wait complete. 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48c752640 1 Processor -- start 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48c752640 1 -- start start 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48c752640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff484102b80 0x7ff48419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48c752640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4841034a0 0x7ff48419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48c752640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff48410edd0 con 0x7ff484102b80 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48c752640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7ff48410ec50 con 0x7ff4841034a0 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48a4ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff484102b80 0x7ff48419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:34.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48a4ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff484102b80 0x7ff48419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58850/0 (socket says 172.21.15.46:58850) 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff48a4ee640 1 -- 172.21.15.46:0/1077294738 learned_addr learned my addr 172.21.15.46:0/1077294738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.110+0000 7ff489ced640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4841034a0 0x7ff48419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff48a4ee640 1 -- 172.21.15.46:0/1077294738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4841034a0 msgr2=0x7ff48419db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff48a4ee640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4841034a0 0x7ff48419db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff48a4ee640 1 -- 172.21.15.46:0/1077294738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4841977a0 con 0x7ff484102b80 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff48a4ee640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff484102b80 0x7ff48419d620 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00c8a0 tx=0x7ff47c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff4837fe640 1 -- 172.21.15.46:0/1077294738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff47c0079b0 con 0x7ff484102b80 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff4837fe640 1 -- 172.21.15.46:0/1077294738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff47c007b50 con 0x7ff484102b80 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff484197a90 con 0x7ff484102b80 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff4837fe640 1 -- 172.21.15.46:0/1077294738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7ff47c018700 con 0x7ff484102b80 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.111+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff484198020 con 0x7ff484102b80 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.112+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48406a9d0 con 0x7ff484102b80 2024-09-10T04:46:34.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.113+0000 7ff4837fe640 1 -- 172.21.15.46:0/1077294738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7ff47c007d00 con 0x7ff484102b80 2024-09-10T04:46:34.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.113+0000 7ff4837fe640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff46c07ac00 0x7ff46c07d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:34.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.113+0000 7ff4837fe640 1 -- 172.21.15.46:0/1077294738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7ff47c09dea0 con 0x7ff484102b80 2024-09-10T04:46:34.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.113+0000 7ff489ced640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff46c07ac00 0x7ff46c07d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:34.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.114+0000 7ff489ced640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff46c07ac00 0x7ff46c07d0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff484108980 tx=0x7ff47803a040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:34.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.118+0000 7ff4837fe640 1 -- 172.21.15.46:0/1077294738 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7ff47c014030 con 0x7ff484102b80 2024-09-10T04:46:34.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.340+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) -- 0x7ff484198d00 con 0x7ff484102b80 2024-09-10T04:46:34.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.340+0000 7ff4837fe640 1 -- 172.21.15.46:0/1077294738 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v35) ==== 107+0+5322 (secure 0 0 0) 0x7ff484198d00 con 0x7ff484102b80 2024-09-10T04:46:34.342 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:34.343 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":28,"btime":"2024-09-10T04:44:16:122757+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:16.122744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:34.343 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 28 2024-09-10T04:46:34.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff46c07ac00 msgr2=0x7ff46c07d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:34.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff46c07ac00 0x7ff46c07d0c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff484108980 tx=0x7ff47803a040 comp rx=0 tx=0).stop 2024-09-10T04:46:34.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff484102b80 msgr2=0x7ff48419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff484102b80 0x7ff48419d620 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00c8a0 tx=0x7ff47c00cd60 comp rx=0 tx=0).stop 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 shutdown_connections 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7ff46c07ac00 0x7ff46c07d0c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4841034a0 0x7ff48419db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 --2- 172.21.15.46:0/1077294738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff484102b80 0x7ff48419d620 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 >> 172.21.15.46:0/1077294738 conn(0x7ff4840fc520 msgr2=0x7ff484103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.342+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 shutdown_connections 2024-09-10T04:46:34.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:34.343+0000 7ff48c752640 1 -- 172.21.15.46:0/1077294738 wait complete. 2024-09-10T04:46:34.674 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:34 smithi046 ceph-mon[109506]: pgmap v1230: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:34.674 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:34 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1077294738' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-10T04:46:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:34 smithi070 ceph-mon[79774]: pgmap v1230: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:34 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1077294738' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-10T04:46:34.997 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-09-10T04:46:34.997 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 29 2024-09-10T04:46:35.260 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 -- 172.21.15.46:0/2894668698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 msgr2=0x7f49fc100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 --2- 172.21.15.46:0/2894668698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc100c70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009880 tx=0x7f49ec02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 -- 172.21.15.46:0/2894668698 shutdown_connections 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 --2- 172.21.15.46:0/2894668698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc100c70 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 --2- 172.21.15.46:0/2894668698 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc106810 0x7f49fc106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 -- 172.21.15.46:0/2894668698 >> 172.21.15.46:0/2894668698 conn(0x7f49fc0fc510 msgr2=0x7f49fc0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 -- 172.21.15.46:0/2894668698 shutdown_connections 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 -- 172.21.15.46:0/2894668698 wait complete. 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.809+0000 7f4a037f2640 1 Processor -- start 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a037f2640 1 -- start start 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a037f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:35.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a037f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc106810 0x7f49fc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a037f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10dcc0 con 0x7f49fc100810 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a037f2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc10db40 con 0x7f49fc106810 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a0158e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a0158e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58864/0 (socket says 172.21.15.46:58864) 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a0158e640 1 -- 172.21.15.46:0/1335075177 learned_addr learned my addr 172.21.15.46:0/1335075177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a00d8d640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc106810 0x7f49fc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a0158e640 1 -- 172.21.15.46:0/1335075177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc106810 msgr2=0x7f49fc198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a0158e640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc106810 0x7f49fc198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a0158e640 1 -- 172.21.15.46:0/1335075177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc199330 con 0x7f49fc100810 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.810+0000 7f4a0158e640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc1987f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f49f800d8d0 tx=0x7f49f800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.811+0000 7f49f27fc640 1 -- 172.21.15.46:0/1335075177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49f8017070 con 0x7f49fc100810 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.811+0000 7f49f27fc640 1 -- 172.21.15.46:0/1335075177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f49f8010460 con 0x7f49fc100810 2024-09-10T04:46:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.811+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc199620 con 0x7f49fc100810 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.811+0000 7f49f27fc640 1 -- 172.21.15.46:0/1335075177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f49f800f5f0 con 0x7f49fc100810 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.811+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc1a1c20 con 0x7f49fc100810 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.812+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc101f50 con 0x7f49fc100810 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.812+0000 7f49f27fc640 1 -- 172.21.15.46:0/1335075177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f49f8009a30 con 0x7f49fc100810 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.813+0000 7f49f27fc640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807ac50 0x7f49e807d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.813+0000 7f49f27fc640 1 -- 172.21.15.46:0/1335075177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f49f809d9c0 con 0x7f49fc100810 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.813+0000 7f4a00d8d640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807ac50 0x7f49e807d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.813+0000 7f4a00d8d640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807ac50 0x7f49e807d110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f49fc06b010 tx=0x7f49ec031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:35.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:35.817+0000 7f49f27fc640 1 -- 172.21.15.46:0/1335075177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f49fc199620 con 0x7f49fc100810 2024-09-10T04:46:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.041+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) -- 0x7f49fc108da0 con 0x7f49fc100810 2024-09-10T04:46:36.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.041+0000 7f49f27fc640 1 -- 172.21.15.46:0/1335075177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v35) ==== 107+0+5324 (secure 0 0 0) 0x7f49f8069180 con 0x7f49fc100810 2024-09-10T04:46:36.044 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:36.044 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"btime":"2024-09-10T04:44:17:123852+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:16.133219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:resolve","state_seq":458,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:36.044 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 29 2024-09-10T04:46:36.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.043+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807ac50 msgr2=0x7f49e807d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.043+0000 7f4a037f2640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807ac50 0x7f49e807d110 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f49fc06b010 tx=0x7f49ec031400 comp rx=0 tx=0).stop 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.043+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 msgr2=0x7f49fc1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.043+0000 7f4a037f2640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc1987f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f49f800d8d0 tx=0x7f49f800dd90 comp rx=0 tx=0).stop 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.044+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 shutdown_connections 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.044+0000 7f4a037f2640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f49e807ac50 0x7f49e807d110 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.044+0000 7f4a037f2640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49fc106810 0x7f49fc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.044+0000 7f4a037f2640 1 --2- 172.21.15.46:0/1335075177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49fc100810 0x7f49fc1987f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.044+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 >> 172.21.15.46:0/1335075177 conn(0x7f49fc0fc510 msgr2=0x7f49fc0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.044+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 shutdown_connections 2024-09-10T04:46:36.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:36.044+0000 7f4a037f2640 1 -- 172.21.15.46:0/1335075177 wait complete. 2024-09-10T04:46:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:36 smithi046 ceph-mon[109506]: pgmap v1231: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:36 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1335075177' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-10T04:46:36.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 30 2024-09-10T04:46:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:36 smithi070 ceph-mon[79774]: pgmap v1231: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:36 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1335075177' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-10T04:46:36.877 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:37.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.377+0000 7f337147f640 1 -- 172.21.15.46:0/1316934240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 msgr2=0x7f336c10b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.377+0000 7f337147f640 1 --2- 172.21.15.46:0/1316934240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c10b930 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f3360009880 tx=0x7f336002a530 comp rx=0 tx=0).stop 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.378+0000 7f337147f640 1 -- 172.21.15.46:0/1316934240 shutdown_connections 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.378+0000 7f337147f640 1 --2- 172.21.15.46:0/1316934240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c10b930 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.378+0000 7f337147f640 1 --2- 172.21.15.46:0/1316934240 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c102b20 0x7f336c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.378+0000 7f337147f640 1 -- 172.21.15.46:0/1316934240 >> 172.21.15.46:0/1316934240 conn(0x7f336c0fc580 msgr2=0x7f336c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.378+0000 7f337147f640 1 -- 172.21.15.46:0/1316934240 shutdown_connections 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.378+0000 7f337147f640 1 -- 172.21.15.46:0/1316934240 wait complete. 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.378+0000 7f337147f640 1 Processor -- start 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f337147f640 1 -- start start 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f337147f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c102b20 0x7f336c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f337147f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:37.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f337147f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f336c10ee50 con 0x7f336c103440 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f337147f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f336c10ecd0 con 0x7f336c102b20 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f336a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f336a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c19dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58874/0 (socket says 172.21.15.46:58874) 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f336a7fc640 1 -- 172.21.15.46:0/983666158 learned_addr learned my addr 172.21.15.46:0/983666158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.379+0000 7f336affd640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c102b20 0x7f336c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f336a7fc640 1 -- 172.21.15.46:0/983666158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c102b20 msgr2=0x7f336c19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f336a7fc640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c102b20 0x7f336c19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f336a7fc640 1 -- 172.21.15.46:0/983666158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c197860 con 0x7f336c103440 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f336affd640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c102b20 0x7f336c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f336a7fc640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c19dbe0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3360031460 tx=0x7f33600045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f3353fff640 1 -- 172.21.15.46:0/983666158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f336003f070 con 0x7f336c103440 2024-09-10T04:46:37.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c197af0 con 0x7f336c103440 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f3353fff640 1 -- 172.21.15.46:0/983666158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33600035f0 con 0x7f336c103440 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f3353fff640 1 -- 172.21.15.46:0/983666158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f3360035030 con 0x7f336c103440 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.380+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c198020 con 0x7f336c103440 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.382+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c06ada0 con 0x7f336c103440 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.382+0000 7f3353fff640 1 -- 172.21.15.46:0/983666158 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f3360042070 con 0x7f336c103440 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.382+0000 7f3353fff640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f335407abf0 0x7f335407d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.382+0000 7f3353fff640 1 -- 172.21.15.46:0/983666158 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f33600be4a0 con 0x7f336c103440 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.382+0000 7f336affd640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f335407abf0 0x7f335407d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.383+0000 7f336affd640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f335407abf0 0x7f335407d0b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f336c1989a0 tx=0x7f335c005f90 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:37.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.387+0000 7f3353fff640 1 -- 172.21.15.46:0/983666158 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f3360089be0 con 0x7f336c103440 2024-09-10T04:46:37.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.611+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) -- 0x7f336c105fc0 con 0x7f336c103440 2024-09-10T04:46:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.612+0000 7f3353fff640 1 -- 172.21.15.46:0/983666158 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v35) ==== 107+0+5327 (secure 0 0 0) 0x7f3360086020 con 0x7f336c103440 2024-09-10T04:46:37.614 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:37.614 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"btime":"2024-09-10T04:44:24:834799+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:24.104638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:resolve","state_seq":464,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:resolve","state_seq":458,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:37.614 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 30 2024-09-10T04:46:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f335407abf0 msgr2=0x7f335407d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f335407abf0 0x7f335407d0b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f336c1989a0 tx=0x7f335c005f90 comp rx=0 tx=0).stop 2024-09-10T04:46:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 msgr2=0x7f336c19dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c19dbe0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f3360031460 tx=0x7f33600045c0 comp rx=0 tx=0).stop 2024-09-10T04:46:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 shutdown_connections 2024-09-10T04:46:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f335407abf0 0x7f335407d0b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f336c103440 0x7f336c19dbe0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 --2- 172.21.15.46:0/983666158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f336c102b20 0x7f336c19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 >> 172.21.15.46:0/983666158 conn(0x7f336c0fc580 msgr2=0x7f336c108f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.614+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 shutdown_connections 2024-09-10T04:46:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:37.615+0000 7f337147f640 1 -- 172.21.15.46:0/983666158 wait complete. 2024-09-10T04:46:38.220 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 31 2024-09-10T04:46:38.483 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:38 smithi070 ceph-mon[79774]: pgmap v1232: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:38 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/983666158' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-10T04:46:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:38 smithi046 ceph-mon[109506]: pgmap v1232: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:38 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/983666158' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-10T04:46:38.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 -- 172.21.15.46:0/3615189050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 msgr2=0x7f7490101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 --2- 172.21.15.46:0/3615189050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f7490101d60 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7478009880 tx=0x7f747802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 -- 172.21.15.46:0/3615189050 shutdown_connections 2024-09-10T04:46:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 --2- 172.21.15.46:0/3615189050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f7490101d60 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 --2- 172.21.15.46:0/3615189050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7490107900 0x7f7490107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 -- 172.21.15.46:0/3615189050 >> 172.21.15.46:0/3615189050 conn(0x7f74900fd600 msgr2=0x7f74900ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 -- 172.21.15.46:0/3615189050 shutdown_connections 2024-09-10T04:46:38.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.917+0000 7f74951e4640 1 -- 172.21.15.46:0/3615189050 wait complete. 2024-09-10T04:46:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.918+0000 7f74951e4640 1 Processor -- start 2024-09-10T04:46:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.918+0000 7f74951e4640 1 -- start start 2024-09-10T04:46:38.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.918+0000 7f74951e4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f74900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.918+0000 7f74951e4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7490107900 0x7f7490074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.918+0000 7f74951e4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f749010edd0 con 0x7f7490101900 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.918+0000 7f74951e4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f749010ec50 con 0x7f7490107900 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f74900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f74900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32892/0 (socket says 172.21.15.46:32892) 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748ed9d640 1 -- 172.21.15.46:0/3408327685 learned_addr learned my addr 172.21.15.46:0/3408327685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748e59c640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7490107900 0x7f7490074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748ed9d640 1 -- 172.21.15.46:0/3408327685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7490107900 msgr2=0x7f7490074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748ed9d640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7490107900 0x7f7490074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748ed9d640 1 -- 172.21.15.46:0/3408327685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7490074f70 con 0x7f7490101900 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f748ed9d640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f74900744f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f748000d6f0 tx=0x7f748000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f7487fff640 1 -- 172.21.15.46:0/3408327685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7480017070 con 0x7f7490101900 2024-09-10T04:46:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f7487fff640 1 -- 172.21.15.46:0/3408327685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f748000de00 con 0x7f7490101900 2024-09-10T04:46:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7490075260 con 0x7f7490101900 2024-09-10T04:46:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f7487fff640 1 -- 172.21.15.46:0/3408327685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f7480004bb0 con 0x7f7490101900 2024-09-10T04:46:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.919+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a3e20 con 0x7f7490101900 2024-09-10T04:46:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.920+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7490103040 con 0x7f7490101900 2024-09-10T04:46:38.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.921+0000 7f7487fff640 1 -- 172.21.15.46:0/3408327685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f7480009a30 con 0x7f7490101900 2024-09-10T04:46:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.922+0000 7f7487fff640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f747c07ac40 0x7f747c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.922+0000 7f7487fff640 1 -- 172.21.15.46:0/3408327685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f748009d7e0 con 0x7f7490101900 2024-09-10T04:46:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.922+0000 7f748e59c640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f747c07ac40 0x7f747c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.923+0000 7f748e59c640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f747c07ac40 0x7f747c07d100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7490075f80 tx=0x7f7478031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:38.926+0000 7f7487fff640 1 -- 172.21.15.46:0/3408327685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f7480068f00 con 0x7f7490101900 2024-09-10T04:46:39.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.174+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) -- 0x7f7490109e70 con 0x7f7490101900 2024-09-10T04:46:39.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.175+0000 7f7487fff640 1 -- 172.21.15.46:0/3408327685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v35) ==== 107+0+5331 (secure 0 0 0) 0x7f748006ddb0 con 0x7f7490101900 2024-09-10T04:46:39.177 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:39.177 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":31,"btime":"2024-09-10T04:44:25:840150+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:24.889903+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:reconnect","state_seq":465,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:reconnect","state_seq":461,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:39.178 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 31 2024-09-10T04:46:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f747c07ac40 msgr2=0x7f747c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f747c07ac40 0x7f747c07d100 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7490075f80 tx=0x7f7478031400 comp rx=0 tx=0).stop 2024-09-10T04:46:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 msgr2=0x7f74900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:39.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f74900744f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f748000d6f0 tx=0x7f748000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:46:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 shutdown_connections 2024-09-10T04:46:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f747c07ac40 0x7f747c07d100 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7490107900 0x7f7490074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 --2- 172.21.15.46:0/3408327685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7490101900 0x7f74900744f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.177+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 >> 172.21.15.46:0/3408327685 conn(0x7f74900fd600 msgr2=0x7f74900fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.178+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 shutdown_connections 2024-09-10T04:46:39.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:39.178+0000 7f74951e4640 1 -- 172.21.15.46:0/3408327685 wait complete. 2024-09-10T04:46:39.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:39 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3408327685' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-10T04:46:39.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:39 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3408327685' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-10T04:46:39.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 32 2024-09-10T04:46:40.132 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:40.510 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:40 smithi046 ceph-mon[109506]: pgmap v1233: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 -- 172.21.15.46:0/3035858773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 msgr2=0x7efe84107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3035858773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe84107ce0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efe78009880 tx=0x7efe7802a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 -- 172.21.15.46:0/3035858773 shutdown_connections 2024-09-10T04:46:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3035858773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe84101900 0x7efe84101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3035858773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe84107ce0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 -- 172.21.15.46:0/3035858773 >> 172.21.15.46:0/3035858773 conn(0x7efe840fd600 msgr2=0x7efe840ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:40.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 -- 172.21.15.46:0/3035858773 shutdown_connections 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.591+0000 7efe8aafe640 1 -- 172.21.15.46:0/3035858773 wait complete. 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe8aafe640 1 Processor -- start 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe8aafe640 1 -- start start 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe8aafe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe84101900 0x7efe8419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe8aafe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe8419db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe8aafe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410edd0 con 0x7efe84107900 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe8aafe640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7efe8410ec50 con 0x7efe84101900 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe83fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe8419db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe83fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe8419db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32920/0 (socket says 172.21.15.46:32920) 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.592+0000 7efe83fff640 1 -- 172.21.15.46:0/3865626472 learned_addr learned my addr 172.21.15.46:0/3865626472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe8889a640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe84101900 0x7efe8419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:40.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe83fff640 1 -- 172.21.15.46:0/3865626472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe84101900 msgr2=0x7efe8419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe83fff640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe84101900 0x7efe8419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe83fff640 1 -- 172.21.15.46:0/3865626472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe841977c0 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe83fff640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe8419db80 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7efe7000d6f0 tx=0x7efe7000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe81ffb640 1 -- 172.21.15.46:0/3865626472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe70017070 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe84197ab0 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe81ffb640 1 -- 172.21.15.46:0/3865626472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe7000de00 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe81ffb640 1 -- 172.21.15.46:0/3865626472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7efe70004bb0 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.593+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe84198040 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.594+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe84103040 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.595+0000 7efe81ffb640 1 -- 172.21.15.46:0/3865626472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7efe70009a30 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.595+0000 7efe81ffb640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe7407ac40 0x7efe7407d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.596+0000 7efe81ffb640 1 -- 172.21.15.46:0/3865626472 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7efe7009d7e0 con 0x7efe84107900 2024-09-10T04:46:40.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.596+0000 7efe8889a640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe7407ac40 0x7efe7407d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.596+0000 7efe8889a640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe7407ac40 0x7efe7407d100 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efe780096c0 tx=0x7efe78031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:40.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.600+0000 7efe81ffb640 1 -- 172.21.15.46:0/3865626472 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7efe70068f00 con 0x7efe84107900 2024-09-10T04:46:40.803 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:40 smithi070 ceph-mon[79774]: pgmap v1233: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.821+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) -- 0x7efe84109e70 con 0x7efe84107900 2024-09-10T04:46:40.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.822+0000 7efe81ffb640 1 -- 172.21.15.46:0/3865626472 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v35) ==== 107+0+5325 (secure 0 0 0) 0x7efe7006ddb0 con 0x7efe84107900 2024-09-10T04:46:40.824 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:40.824 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"btime":"2024-09-10T04:44:26:845848+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:25.849709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:rejoin","state_seq":466,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:rejoin","state_seq":462,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T04:46:40.824 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-09-10T04:46:40.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe7407ac40 msgr2=0x7efe7407d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe7407ac40 0x7efe7407d100 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efe780096c0 tx=0x7efe78031400 comp rx=0 tx=0).stop 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 msgr2=0x7efe8419db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe8419db80 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7efe7000d6f0 tx=0x7efe7000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 shutdown_connections 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7efe7407ac40 0x7efe7407d100 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe84107900 0x7efe8419db80 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 --2- 172.21.15.46:0/3865626472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe84101900 0x7efe8419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.824+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 >> 172.21.15.46:0/3865626472 conn(0x7efe840fd600 msgr2=0x7efe840fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.825+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 shutdown_connections 2024-09-10T04:46:40.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:40.825+0000 7efe8aafe640 1 -- 172.21.15.46:0/3865626472 wait complete. 2024-09-10T04:46:41.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 33 2024-09-10T04:46:41.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:41 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/3865626472' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-10T04:46:41.710 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:41.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:41 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/3865626472' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-10T04:46:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 -- 172.21.15.46:0/3042074542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8101900 msgr2=0x7f42f8101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 --2- 172.21.15.46:0/3042074542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8101900 0x7f42f8101d60 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f42e0009880 tx=0x7f42e002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 -- 172.21.15.46:0/3042074542 shutdown_connections 2024-09-10T04:46:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 --2- 172.21.15.46:0/3042074542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8101900 0x7f42f8101d60 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 --2- 172.21.15.46:0/3042074542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42f8107900 0x7f42f8107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 -- 172.21.15.46:0/3042074542 >> 172.21.15.46:0/3042074542 conn(0x7f42f80fd600 msgr2=0x7f42f80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 -- 172.21.15.46:0/3042074542 shutdown_connections 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.148+0000 7f42fef06640 1 -- 172.21.15.46:0/3042074542 wait complete. 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fef06640 1 Processor -- start 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fef06640 1 -- start start 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fef06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42f8101900 0x7f42f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fef06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8107900 0x7f42f8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fef06640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810ee30 con 0x7f42f8107900 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fef06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f42f810ecb0 con 0x7f42f8101900 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8107900 0x7f42f8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42f7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8107900 0x7f42f8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32930/0 (socket says 172.21.15.46:32930) 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42f7fff640 1 -- 172.21.15.46:0/295406990 learned_addr learned my addr 172.21.15.46:0/295406990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fcca2640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42f8101900 0x7f42f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42f7fff640 1 -- 172.21.15.46:0/295406990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42f8101900 msgr2=0x7f42f80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42f7fff640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42f8101900 0x7f42f80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42f7fff640 1 -- 172.21.15.46:0/295406990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f8074f70 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.149+0000 7f42fcca2640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42f8101900 0x7f42f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.150+0000 7f42f7fff640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8107900 0x7f42f8074a30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f42e0031460 tx=0x7f42e00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.150+0000 7f42f5ffb640 1 -- 172.21.15.46:0/295406990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42e003f070 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.150+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f8075200 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.150+0000 7f42f5ffb640 1 -- 172.21.15.46:0/295406990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f42e00044d0 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.150+0000 7f42f5ffb640 1 -- 172.21.15.46:0/295406990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f42e0035030 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.150+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f81a3e00 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.151+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f8103040 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.152+0000 7f42f5ffb640 1 -- 172.21.15.46:0/295406990 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f42e0042070 con 0x7f42f8107900 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.152+0000 7f42f5ffb640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42e407ac00 0x7f42e407d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.152+0000 7f42f5ffb640 1 -- 172.21.15.46:0/295406990 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f42e00be820 con 0x7f42f8107900 2024-09-10T04:46:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.152+0000 7f42fcca2640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42e407ac00 0x7f42e407d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.153+0000 7f42fcca2640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42e407ac00 0x7f42e407d0c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007eb0 tx=0x7f42e80073c0 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.157+0000 7f42f5ffb640 1 -- 172.21.15.46:0/295406990 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f42e0004cf0 con 0x7f42f8107900 2024-09-10T04:46:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.378+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) -- 0x7f42f8109e90 con 0x7f42f8107900 2024-09-10T04:46:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.379+0000 7f42f5ffb640 1 -- 172.21.15.46:0/295406990 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v35) ==== 107+0+5340 (secure 0 0 0) 0x7f42e0089f60 con 0x7f42f8107900 2024-09-10T04:46:42.381 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:42.381 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":33,"btime":"2024-09-10T04:44:27:850471+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:27.850464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:active","state_seq":467,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:active","state_seq":463,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34328,"qdb_cluster":[34328,34360]},"id":1}]} 2024-09-10T04:46:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 33 2024-09-10T04:46:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42e407ac00 msgr2=0x7f42e407d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42e407ac00 0x7f42e407d0c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007eb0 tx=0x7f42e80073c0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8107900 msgr2=0x7f42f8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8107900 0x7f42f8074a30 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f42e0031460 tx=0x7f42e00048f0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 shutdown_connections 2024-09-10T04:46:42.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f42e407ac00 0x7f42e407d0c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42f8107900 0x7f42f8074a30 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 --2- 172.21.15.46:0/295406990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42f8101900 0x7f42f80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 >> 172.21.15.46:0/295406990 conn(0x7f42f80fd600 msgr2=0x7f42f80fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 shutdown_connections 2024-09-10T04:46:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:42.381+0000 7f42fef06640 1 -- 172.21.15.46:0/295406990 wait complete. 2024-09-10T04:46:42.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:42 smithi070 ceph-mon[79774]: pgmap v1234: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:42.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:42 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/295406990' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-10T04:46:42.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:42 smithi046 ceph-mon[109506]: pgmap v1234: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:42.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:42 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/295406990' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-10T04:46:43.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 34 2024-09-10T04:46:43.323 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 -- 172.21.15.46:0/2116124012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0103440 msgr2=0x7f4fa010b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2116124012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0103440 0x7f4fa010b930 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4f880098b0 tx=0x7f4f8802a560 comp rx=0 tx=0).stop 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 -- 172.21.15.46:0/2116124012 shutdown_connections 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2116124012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0103440 0x7f4fa010b930 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2116124012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa0102b20 0x7f4fa0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 -- 172.21.15.46:0/2116124012 >> 172.21.15.46:0/2116124012 conn(0x7f4fa00fc520 msgr2=0x7f4fa00fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 -- 172.21.15.46:0/2116124012 shutdown_connections 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.813+0000 7f4fa51df640 1 -- 172.21.15.46:0/2116124012 wait complete. 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4fa51df640 1 Processor -- start 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4fa51df640 1 -- start start 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4fa51df640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0102b20 0x7f4fa0100780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4fa51df640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa0103440 0x7f4fa0100cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4fa51df640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa010edf0 con 0x7f4fa0102b20 2024-09-10T04:46:43.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4fa51df640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa010ec70 con 0x7f4fa0103440 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4f9ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0102b20 0x7f4fa0100780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4f9ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0102b20 0x7f4fa0100780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32944/0 (socket says 172.21.15.46:32944) 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.814+0000 7f4f9ed9d640 1 -- 172.21.15.46:0/2868722771 learned_addr learned my addr 172.21.15.46:0/2868722771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f97fff640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa0103440 0x7f4fa0100cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f9ed9d640 1 -- 172.21.15.46:0/2868722771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa0103440 msgr2=0x7f4fa0100cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f9ed9d640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa0103440 0x7f4fa0100cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f9ed9d640 1 -- 172.21.15.46:0/2868722771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa0101200 con 0x7f4fa0102b20 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f9ed9d640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0102b20 0x7f4fa0100780 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000d6f0 tx=0x7f4f9000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f9cd99640 1 -- 172.21.15.46:0/2868722771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f90017070 con 0x7f4fa0102b20 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f9cd99640 1 -- 172.21.15.46:0/2868722771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f9000de00 con 0x7f4fa0102b20 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa01014f0 con 0x7f4fa0102b20 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4f9cd99640 1 -- 172.21.15.46:0/2868722771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f4f90004bb0 con 0x7f4fa0102b20 2024-09-10T04:46:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.815+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa01a3f20 con 0x7f4fa0102b20 2024-09-10T04:46:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.816+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa006aa30 con 0x7f4fa0102b20 2024-09-10T04:46:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.817+0000 7f4f9cd99640 1 -- 172.21.15.46:0/2868722771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f4f90009a30 con 0x7f4fa0102b20 2024-09-10T04:46:43.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.817+0000 7f4f9cd99640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f8c07ac40 0x7f4f8c07d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.817+0000 7f4f9cd99640 1 -- 172.21.15.46:0/2868722771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f4f9009d760 con 0x7f4fa0102b20 2024-09-10T04:46:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.818+0000 7f4f97fff640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f8c07ac40 0x7f4f8c07d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:43.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.818+0000 7f4f97fff640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f8c07ac40 0x7f4f8c07d100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f88031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:43.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:43.822+0000 7f4f9cd99640 1 -- 172.21.15.46:0/2868722771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f4f90068f20 con 0x7f4fa0102b20 2024-09-10T04:46:44.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.045+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) -- 0x7f4fa0102f00 con 0x7f4fa0102b20 2024-09-10T04:46:44.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.046+0000 7f4f9cd99640 1 -- 172.21.15.46:0/2868722771 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v35) ==== 107+0+5355 (secure 0 0 0) 0x7f4f9006ddd0 con 0x7f4fa0102b20 2024-09-10T04:46:44.048 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-09-10T04:46:44.048 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"btime":"2024-09-10T04:44:27:859763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T04:02:20.933717+0000","modified":"2024-09-10T04:44:27.859744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":75,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34328,"mds_1":34360},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34328":{"gid":34328,"name":"cephfs.smithi046.hasdap","rank":0,"incarnation":28,"state":"up:active","state_seq":467,"addr":"172.21.15.46:6827/4182274028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":4182274028},{"type":"v1","addr":"172.21.15.46:6827","nonce":4182274028}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34360":{"gid":34360,"name":"cephfs.smithi046.cpwyud","rank":1,"incarnation":28,"state":"up:active","state_seq":463,"addr":"172.21.15.46:6829/2230370623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2230370623},{"type":"v1","addr":"172.21.15.46:6829","nonce":2230370623}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44341":{"gid":44341,"name":"cephfs.smithi070.ehvjgr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6825/798617823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":798617823},{"type":"v1","addr":"172.21.15.70:6825","nonce":798617823}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44967":{"gid":44967,"name":"cephfs.smithi070.myztiz","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.70:6827/1908756829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":1908756829},{"type":"v1","addr":"172.21.15.70:6827","nonce":1908756829}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34328,"qdb_cluster":[34360,34328]},"id":1}]} 2024-09-10T04:46:44.049 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-09-10T04:46:44.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f8c07ac40 msgr2=0x7f4f8c07d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:44.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f8c07ac40 0x7f4f8c07d100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4f88000c00 tx=0x7f4f88031400 comp rx=0 tx=0).stop 2024-09-10T04:46:44.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0102b20 msgr2=0x7f4fa0100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0102b20 0x7f4fa0100780 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000d6f0 tx=0x7f4f9000dbb0 comp rx=0 tx=0).stop 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 shutdown_connections 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f4f8c07ac40 0x7f4f8c07d100 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4fa0103440 0x7f4fa0100cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 --2- 172.21.15.46:0/2868722771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4fa0102b20 0x7f4fa0100780 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 >> 172.21.15.46:0/2868722771 conn(0x7f4fa00fc520 msgr2=0x7f4fa0103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 shutdown_connections 2024-09-10T04:46:44.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:44.048+0000 7f4fa51df640 1 -- 172.21.15.46:0/2868722771 wait complete. 2024-09-10T04:46:44.741 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:44 smithi046 ceph-mon[109506]: pgmap v1235: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:44.742 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:44 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/2868722771' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 34, "format": "json"} : dispatch 2024-09-10T04:46:44.743 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T04:46:44.754 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T04:46:44.754 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T04:46:44.754 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T04:46:44.774 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T04:46:44.774 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T04:46:44.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T04:46:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:44 smithi070 ceph-mon[79774]: pgmap v1235: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:44 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/2868722771' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 34, "format": "json"} : dispatch 2024-09-10T04:46:45.136 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.617+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/940693227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 msgr2=0x7fe3f0107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.617+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/940693227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f0107ce0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009880 tx=0x7fe3e002a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.617+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/940693227 shutdown_connections 2024-09-10T04:46:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.617+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/940693227 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f0101900 0x7fe3f0101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.617+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/940693227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f0107ce0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.617+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/940693227 >> 172.21.15.46:0/940693227 conn(0x7fe3f00fd600 msgr2=0x7fe3f00ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.617+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/940693227 shutdown_connections 2024-09-10T04:46:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.618+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/940693227 wait complete. 2024-09-10T04:46:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.618+0000 7fe3f5d9d640 1 Processor -- start 2024-09-10T04:46:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.618+0000 7fe3f5d9d640 1 -- start start 2024-09-10T04:46:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3f5d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f0101900 0x7fe3f019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3f5d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3f5d9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f010edb0 con 0x7fe3f0107900 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3f5d9d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f010ec30 con 0x7fe3f0101900 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3eeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3eeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f019dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:32974/0 (socket says 172.21.15.46:32974) 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3eeffd640 1 -- 172.21.15.46:0/13364780 learned_addr learned my addr 172.21.15.46:0/13364780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3ef7fe640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f0101900 0x7fe3f019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3eeffd640 1 -- 172.21.15.46:0/13364780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f0101900 msgr2=0x7fe3f019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3eeffd640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f0101900 0x7fe3f019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3eeffd640 1 -- 172.21.15.46:0/13364780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f01977e0 con 0x7fe3f0107900 2024-09-10T04:46:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.619+0000 7fe3ef7fe640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f0101900 0x7fe3f019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T04:46:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.620+0000 7fe3eeffd640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f019dba0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400d8d0 tx=0x7fe3e400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.620+0000 7fe3ecff9640 1 -- 172.21.15.46:0/13364780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3e4017030 con 0x7fe3f0107900 2024-09-10T04:46:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.620+0000 7fe3ecff9640 1 -- 172.21.15.46:0/13364780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe3e4004020 con 0x7fe3f0107900 2024-09-10T04:46:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.620+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f0197a70 con 0x7fe3f0107900 2024-09-10T04:46:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.620+0000 7fe3ecff9640 1 -- 172.21.15.46:0/13364780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7fe3e40047a0 con 0x7fe3f0107900 2024-09-10T04:46:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.620+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f0198000 con 0x7fe3f0107900 2024-09-10T04:46:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.621+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f0103040 con 0x7fe3f0107900 2024-09-10T04:46:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.622+0000 7fe3ecff9640 1 -- 172.21.15.46:0/13364780 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7fe3e40041d0 con 0x7fe3f0107900 2024-09-10T04:46:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.622+0000 7fe3ecff9640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3d807abf0 0x7fe3d807d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.623+0000 7fe3ecff9640 1 -- 172.21.15.46:0/13364780 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7fe3e4018070 con 0x7fe3f0107900 2024-09-10T04:46:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.623+0000 7fe3ef7fe640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3d807abf0 0x7fe3d807d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.623+0000 7fe3ef7fe640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3d807abf0 0x7fe3d807d0b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00096c0 tx=0x7fe3e0031400 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:45.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.627+0000 7fe3ecff9640 1 -- 172.21.15.46:0/13364780 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7fe3e4069110 con 0x7fe3f0107900 2024-09-10T04:46:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.826+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3f0104b10 con 0x7fe3f0107900 2024-09-10T04:46:45.829 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:45 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:46:45.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.826+0000 7fe3ecff9640 1 -- 172.21.15.46:0/13364780 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 33 entries v79) ==== 81+0+1952 (secure 0 0 0) 0x7fe3e406dfc0 con 0x7fe3f0107900 2024-09-10T04:46:45.829 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6827/4038650990 2024-09-11T04:13:25.852111+0000 2024-09-10T04:46:45.829 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/653182070 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:45.829 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2101940292 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:45.829 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2504291949 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4068095835 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2101940292 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6826/4038650990 2024-09-11T04:13:25.852111+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/2174023799 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/653182070 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6828/3363961098 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/3538810660 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2984295913 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2203092367 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3785267557 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/532189864 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1078369413 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6825/1155442666 2024-09-11T04:13:25.852123+0000 2024-09-10T04:46:45.830 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/3539336086 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2676338366 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/697320365 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4048021550 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/2918420410 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6829/3363961098 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1023279878 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/1411809474 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2611013585 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3311822035 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/1804840410 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6824/1155442666 2024-09-11T04:13:25.852123+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3014162921 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:45.831 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/1804840410 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:45.832 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/3538810660 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:45.832 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1609399519 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:45.832 INFO:teuthology.orchestra.run.smithi046.stderr:listed 33 entries 2024-09-10T04:46:45.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3d807abf0 msgr2=0x7fe3d807d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:45.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3d807abf0 0x7fe3d807d0b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe3e00096c0 tx=0x7fe3e0031400 comp rx=0 tx=0).stop 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 msgr2=0x7fe3f019dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f019dba0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400d8d0 tx=0x7fe3e400dd90 comp rx=0 tx=0).stop 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 shutdown_connections 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7fe3d807abf0 0x7fe3d807d0b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f0107900 0x7fe3f019dba0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 --2- 172.21.15.46:0/13364780 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3f0101900 0x7fe3f019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 >> 172.21.15.46:0/13364780 conn(0x7fe3f00fd600 msgr2=0x7fe3f00fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 shutdown_connections 2024-09-10T04:46:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:45.829+0000 7fe3f5d9d640 1 -- 172.21.15.46:0/13364780 wait complete. 2024-09-10T04:46:45.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:45 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:46:46.510 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-09-10T04:46:46.510 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T04:46:46.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:squid shell --fsid 628edc22-6f28-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T04:46:46.832 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/mon.smithi046/config 2024-09-10T04:46:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:46 smithi070 ceph-mon[79774]: pgmap v1236: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:46:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:46:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:46:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:46 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/13364780' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T04:46:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:46 smithi046 ceph-mon[109506]: pgmap v1236: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:46:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:46:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:46:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:46 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/13364780' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 -- 172.21.15.46:0/2367758391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 msgr2=0x7f6b94100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 --2- 172.21.15.46:0/2367758391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b94100c60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c009880 tx=0x7f6b7c02a4a0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 -- 172.21.15.46:0/2367758391 shutdown_connections 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 --2- 172.21.15.46:0/2367758391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b94100c60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 --2- 172.21.15.46:0/2367758391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b94106800 0x7f6b94106be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 -- 172.21.15.46:0/2367758391 >> 172.21.15.46:0/2367758391 conn(0x7f6b940fc520 msgr2=0x7f6b940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 -- 172.21.15.46:0/2367758391 shutdown_connections 2024-09-10T04:46:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 -- 172.21.15.46:0/2367758391 wait complete. 2024-09-10T04:46:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.316+0000 7f6b988a7640 1 Processor -- start 2024-09-10T04:46:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b988a7640 1 -- start start 2024-09-10T04:46:47.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b988a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b9419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b988a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b94106800 0x7f6b9419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b988a7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9410dcd0 con 0x7f6b94100800 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b988a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9410db50 con 0x7f6b94106800 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b9419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b9259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b9419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33002/0 (socket says 172.21.15.46:33002) 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b9259c640 1 -- 172.21.15.46:0/1909386067 learned_addr learned my addr 172.21.15.46:0/1909386067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b9259c640 1 -- 172.21.15.46:0/1909386067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b94106800 msgr2=0x7f6b9419cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b91d9b640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b94106800 0x7f6b9419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b9259c640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b94106800 0x7f6b9419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b9259c640 1 -- 172.21.15.46:0/1909386067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b941966a0 con 0x7f6b94100800 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.317+0000 7f6b9259c640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b9419c5d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400ea00 tx=0x7f6b8400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.318+0000 7f6b8b7fe640 1 -- 172.21.15.46:0/1909386067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b8400cb80 con 0x7f6b94100800 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.318+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b94196990 con 0x7f6b94100800 2024-09-10T04:46:47.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.318+0000 7f6b8b7fe640 1 -- 172.21.15.46:0/1909386067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b8400cd20 con 0x7f6b94100800 2024-09-10T04:46:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.318+0000 7f6b8b7fe640 1 -- 172.21.15.46:0/1909386067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 357+0+0 (secure 0 0 0) 0x7f6b84011470 con 0x7f6b94100800 2024-09-10T04:46:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.318+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b94196ef0 con 0x7f6b94100800 2024-09-10T04:46:47.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.319+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b94101f40 con 0x7f6b94100800 2024-09-10T04:46:47.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.319+0000 7f6b8b7fe640 1 -- 172.21.15.46:0/1909386067 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 32) ==== 103166+0+0 (secure 0 0 0) 0x7f6b84011750 con 0x7f6b94100800 2024-09-10T04:46:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.320+0000 7f6b8b7fe640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6b8007abf0 0x7f6b8007d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T04:46:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.320+0000 7f6b8b7fe640 1 -- 172.21.15.46:0/1909386067 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6242+0+0 (secure 0 0 0) 0x7f6b8409e1f0 con 0x7f6b94100800 2024-09-10T04:46:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.320+0000 7f6b91d9b640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6b8007abf0 0x7f6b8007d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T04:46:47.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.321+0000 7f6b91d9b640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6b8007abf0 0x7f6b8007d0b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c031460 tx=0x7f6b7c009560 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T04:46:47.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.324+0000 7f6b8b7fe640 1 -- 172.21.15.46:0/1909386067 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197410 (secure 0 0 0) 0x7f6b84016030 con 0x7f6b94100800 2024-09-10T04:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.526+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b94108d70 con 0x7f6b94100800 2024-09-10T04:46:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.526+0000 7f6b8b7fe640 1 -- 172.21.15.46:0/1909386067 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 33 entries v79) ==== 81+0+1952 (secure 0 0 0) 0x7f6b84069890 con 0x7f6b94100800 2024-09-10T04:46:47.528 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6827/4038650990 2024-09-11T04:13:25.852111+0000 2024-09-10T04:46:47.528 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/653182070 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2101940292 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2504291949 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4068095835 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2101940292 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6826/4038650990 2024-09-11T04:13:25.852111+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/2174023799 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/653182070 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6828/3363961098 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/3538810660 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2984295913 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2203092367 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3785267557 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/532189864 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.529 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1078369413 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6825/1155442666 2024-09-11T04:13:25.852123+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/3539336086 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2676338366 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/697320365 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/4048021550 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/2918420410 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6829/3363961098 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1023279878 2024-09-11T03:56:35.886608+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:0/1411809474 2024-09-11T04:06:52.000528+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2611013585 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3311822035 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/1804840410 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.70:6824/1155442666 2024-09-11T04:13:25.852123+0000 2024-09-10T04:46:47.530 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3014162921 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:47.531 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/1804840410 2024-09-11T03:56:03.190805+0000 2024-09-10T04:46:47.531 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/3538810660 2024-09-11T04:05:37.650060+0000 2024-09-10T04:46:47.531 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1609399519 2024-09-11T03:57:33.701336+0000 2024-09-10T04:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:listed 33 entries 2024-09-10T04:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6b8007abf0 msgr2=0x7f6b8007d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6b8007abf0 0x7f6b8007d0b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c031460 tx=0x7f6b7c009560 comp rx=0 tx=0).stop 2024-09-10T04:46:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 msgr2=0x7f6b9419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b9419c5d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400ea00 tx=0x7f6b8400eec0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 shutdown_connections 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:6800/3636798080,v1:172.21.15.46:6801/3636798080] conn(0x7f6b8007abf0 0x7f6b8007d0b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b94106800 0x7f6b9419cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 --2- 172.21.15.46:0/1909386067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b94100800 0x7f6b9419c5d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 >> 172.21.15.46:0/1909386067 conn(0x7f6b940fc520 msgr2=0x7f6b940fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 shutdown_connections 2024-09-10T04:46:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-09-10T04:46:47.528+0000 7f6b988a7640 1 -- 172.21.15.46:0/1909386067 wait complete. 2024-09-10T04:46:48.320 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-09-10T04:46:48.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T04:46:48.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T04:46:48.352 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T04:46:48.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:48 smithi046 ceph-mon[109506]: pgmap v1237: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:48.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:48 smithi046 ceph-mon[109506]: from='client.? 172.21.15.46:0/1909386067' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T04:46:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:48 smithi070 ceph-mon[79774]: pgmap v1237: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:48 smithi070 ceph-mon[79774]: from='client.? 172.21.15.46:0/1909386067' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T04:46:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:50 smithi046 ceph-mon[109506]: pgmap v1238: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:50 smithi070 ceph-mon[79774]: pgmap v1238: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:52 smithi070 ceph-mon[79774]: pgmap v1239: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:52.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:52 smithi046 ceph-mon[109506]: pgmap v1239: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:46:53.824 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:46:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:54 smithi070 ceph-mon[79774]: pgmap v1240: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:54 smithi046 ceph-mon[109506]: pgmap v1240: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:46:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:56 smithi070 ceph-mon[79774]: pgmap v1241: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:56.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:56 smithi046 ceph-mon[109506]: pgmap v1241: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:46:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:46:58 smithi070 ceph-mon[79774]: pgmap v1242: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:46:58.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:46:58 smithi046 ceph-mon[109506]: pgmap v1242: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:00 smithi070 ceph-mon[79774]: pgmap v1243: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:00 smithi046 ceph-mon[109506]: pgmap v1243: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:02 smithi070 ceph-mon[79774]: pgmap v1244: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:02.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:02 smithi046 ceph-mon[109506]: pgmap v1244: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:04.589 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:04 smithi046 ceph-mon[109506]: pgmap v1245: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:04 smithi070 ceph-mon[79774]: pgmap v1245: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:06.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:06 smithi070 ceph-mon[79774]: pgmap v1246: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:06.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:06 smithi046 ceph-mon[109506]: pgmap v1246: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:08 smithi070 ceph-mon[79774]: pgmap v1247: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:08 smithi046 ceph-mon[109506]: pgmap v1247: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:08.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:10 smithi070 ceph-mon[79774]: pgmap v1248: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:10.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:10 smithi046 ceph-mon[109506]: pgmap v1248: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:12 smithi070 ceph-mon[79774]: pgmap v1249: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:12 smithi046 ceph-mon[109506]: pgmap v1249: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:14 smithi070 ceph-mon[79774]: pgmap v1250: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:14.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:14 smithi046 ceph-mon[109506]: pgmap v1250: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:16 smithi070 ceph-mon[79774]: pgmap v1251: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:16 smithi046 ceph-mon[109506]: pgmap v1251: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:18 smithi070 ceph-mon[79774]: pgmap v1252: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:18 smithi046 ceph-mon[109506]: pgmap v1252: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:20 smithi070 ceph-mon[79774]: pgmap v1253: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:20.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:20 smithi046 ceph-mon[109506]: pgmap v1253: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:22 smithi070 ceph-mon[79774]: pgmap v1254: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:22 smithi046 ceph-mon[109506]: pgmap v1254: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:24 smithi070 ceph-mon[79774]: pgmap v1255: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:24.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:24 smithi046 ceph-mon[109506]: pgmap v1255: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:26.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:26 smithi046 ceph-mon[109506]: pgmap v1256: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:26 smithi070 ceph-mon[79774]: pgmap v1256: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:28 smithi070 ceph-mon[79774]: pgmap v1257: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:28 smithi046 ceph-mon[109506]: pgmap v1257: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:30 smithi070 ceph-mon[79774]: pgmap v1258: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:30 smithi046 ceph-mon[109506]: pgmap v1258: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:32 smithi070 ceph-mon[79774]: pgmap v1259: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:32.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:32 smithi046 ceph-mon[109506]: pgmap v1259: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:34 smithi070 ceph-mon[79774]: pgmap v1260: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:34 smithi046 ceph-mon[109506]: pgmap v1260: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:36 smithi070 ceph-mon[79774]: pgmap v1261: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:36.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:36 smithi046 ceph-mon[109506]: pgmap v1261: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:38 smithi046 ceph-mon[109506]: pgmap v1262: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:38 smithi070 ceph-mon[79774]: pgmap v1262: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:40.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:40 smithi046 ceph-mon[109506]: pgmap v1263: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:40 smithi070 ceph-mon[79774]: pgmap v1263: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:42.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:42 smithi070 ceph-mon[79774]: pgmap v1264: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:42.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:42 smithi046 ceph-mon[109506]: pgmap v1264: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:44.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:44 smithi046 ceph-mon[109506]: pgmap v1265: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:44 smithi070 ceph-mon[79774]: pgmap v1265: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:46 smithi070 ceph-mon[79774]: pgmap v1266: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:47:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:47:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:47:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:47:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:46 smithi046 ceph-mon[109506]: pgmap v1266: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:47:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:47:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:47:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:47:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:48 smithi070 ceph-mon[79774]: pgmap v1267: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:48.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:48 smithi046 ceph-mon[109506]: pgmap v1267: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:50 smithi046 ceph-mon[109506]: pgmap v1268: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:50 smithi070 ceph-mon[79774]: pgmap v1268: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:52.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:52 smithi070 ceph-mon[79774]: pgmap v1269: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:52 smithi046 ceph-mon[109506]: pgmap v1269: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:47:53.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:47:54.736 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:54 smithi070 ceph-mon[79774]: pgmap v1270: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:54.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:54 smithi046 ceph-mon[109506]: pgmap v1270: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:47:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:56 smithi046 ceph-mon[109506]: pgmap v1271: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:56 smithi070 ceph-mon[79774]: pgmap v1271: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:47:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:47:58 smithi070 ceph-mon[79774]: pgmap v1272: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:47:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:47:58 smithi046 ceph-mon[109506]: pgmap v1272: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:00 smithi070 ceph-mon[79774]: pgmap v1273: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:00 smithi046 ceph-mon[109506]: pgmap v1273: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:02.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:02 smithi046 ceph-mon[109506]: pgmap v1274: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:02 smithi070 ceph-mon[79774]: pgmap v1274: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:04 smithi070 ceph-mon[79774]: pgmap v1275: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:04.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:04 smithi046 ceph-mon[109506]: pgmap v1275: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:06 smithi070 ceph-mon[79774]: pgmap v1276: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:06.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:06 smithi046 ceph-mon[109506]: pgmap v1276: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:08 smithi046 ceph-mon[109506]: pgmap v1277: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:08 smithi070 ceph-mon[79774]: pgmap v1277: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:10.746 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:10 smithi046 ceph-mon[109506]: pgmap v1278: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:10 smithi070 ceph-mon[79774]: pgmap v1278: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:12.669 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:12 smithi070 ceph-mon[79774]: pgmap v1279: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:12.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:12 smithi046 ceph-mon[109506]: pgmap v1279: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:14.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:14 smithi046 ceph-mon[109506]: pgmap v1280: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:14 smithi070 ceph-mon[79774]: pgmap v1280: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:16 smithi070 ceph-mon[79774]: pgmap v1281: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:16.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:16 smithi046 ceph-mon[109506]: pgmap v1281: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:18 smithi070 ceph-mon[79774]: pgmap v1282: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:18 smithi046 ceph-mon[109506]: pgmap v1282: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:20 smithi046 ceph-mon[109506]: pgmap v1283: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:20 smithi070 ceph-mon[79774]: pgmap v1283: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:22 smithi070 ceph-mon[79774]: pgmap v1284: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:22 smithi046 ceph-mon[109506]: pgmap v1284: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:24.736 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:24 smithi070 ceph-mon[79774]: pgmap v1285: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:24.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:24 smithi046 ceph-mon[109506]: pgmap v1285: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:26 smithi070 ceph-mon[79774]: pgmap v1286: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:26.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:26 smithi046 ceph-mon[109506]: pgmap v1286: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:28 smithi070 ceph-mon[79774]: pgmap v1287: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:28.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:28 smithi046 ceph-mon[109506]: pgmap v1287: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:30 smithi070 ceph-mon[79774]: pgmap v1288: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:30.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:30 smithi046 ceph-mon[109506]: pgmap v1288: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:32.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:32 smithi046 ceph-mon[109506]: pgmap v1289: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:32 smithi070 ceph-mon[79774]: pgmap v1289: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:34 smithi070 ceph-mon[79774]: pgmap v1290: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:34 smithi046 ceph-mon[109506]: pgmap v1290: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:36.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:36 smithi046 ceph-mon[109506]: pgmap v1291: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:36 smithi070 ceph-mon[79774]: pgmap v1291: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:38 smithi046 ceph-mon[109506]: pgmap v1292: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:38 smithi070 ceph-mon[79774]: pgmap v1292: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:40 smithi070 ceph-mon[79774]: pgmap v1293: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:40 smithi046 ceph-mon[109506]: pgmap v1293: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:42.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:42 smithi070 ceph-mon[79774]: pgmap v1294: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:42 smithi046 ceph-mon[109506]: pgmap v1294: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:44.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:44 smithi046 ceph-mon[109506]: pgmap v1295: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:44 smithi070 ceph-mon[79774]: pgmap v1295: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:46 smithi070 ceph-mon[79774]: pgmap v1296: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:48:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:46 smithi046 ceph-mon[109506]: pgmap v1296: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:46.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:48:47.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:47 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:48:47.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:47 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:48:47.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:47 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:48:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:47 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:48:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:47 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:48:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:47 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:48:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:48 smithi070 ceph-mon[79774]: pgmap v1297: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:48.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:48 smithi046 ceph-mon[109506]: pgmap v1297: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:50 smithi070 ceph-mon[79774]: pgmap v1298: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:50.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:50 smithi046 ceph-mon[109506]: pgmap v1298: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:52.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:52 smithi070 ceph-mon[79774]: pgmap v1299: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:52.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:52 smithi046 ceph-mon[109506]: pgmap v1299: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:48:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:53.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:48:54.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:54 smithi070 ceph-mon[79774]: pgmap v1300: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:54 smithi046 ceph-mon[109506]: pgmap v1300: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:48:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:56 smithi070 ceph-mon[79774]: pgmap v1301: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:56.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:56 smithi046 ceph-mon[109506]: pgmap v1301: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:48:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:48:58 smithi070 ceph-mon[79774]: pgmap v1302: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:48:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:48:58 smithi046 ceph-mon[109506]: pgmap v1302: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:00 smithi070 ceph-mon[79774]: pgmap v1303: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:00 smithi046 ceph-mon[109506]: pgmap v1303: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:02 smithi070 ceph-mon[79774]: pgmap v1304: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:02.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:02 smithi046 ceph-mon[109506]: pgmap v1304: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:04 smithi070 ceph-mon[79774]: pgmap v1305: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:04.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:04 smithi046 ceph-mon[109506]: pgmap v1305: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:06 smithi070 ceph-mon[79774]: pgmap v1306: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:07.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:06 smithi046 ceph-mon[109506]: pgmap v1306: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:08 smithi070 ceph-mon[79774]: pgmap v1307: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:08 smithi046 ceph-mon[109506]: pgmap v1307: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:10 smithi070 ceph-mon[79774]: pgmap v1308: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:10 smithi046 ceph-mon[109506]: pgmap v1308: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:12 smithi070 ceph-mon[79774]: pgmap v1309: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:12 smithi046 ceph-mon[109506]: pgmap v1309: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:14 smithi070 ceph-mon[79774]: pgmap v1310: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:15.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:14 smithi046 ceph-mon[109506]: pgmap v1310: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:17.017 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:16 smithi046 ceph-mon[109506]: pgmap v1311: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:16 smithi070 ceph-mon[79774]: pgmap v1311: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:19.014 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:18 smithi070 ceph-mon[79774]: pgmap v1312: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:19.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:18 smithi046 ceph-mon[109506]: pgmap v1312: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:20 smithi070 ceph-mon[79774]: pgmap v1313: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:20 smithi046 ceph-mon[109506]: pgmap v1313: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:22 smithi070 ceph-mon[79774]: pgmap v1314: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:23.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:22 smithi046 ceph-mon[109506]: pgmap v1314: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:23.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:25.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:24 smithi070 ceph-mon[79774]: pgmap v1315: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:25.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:24 smithi046 ceph-mon[109506]: pgmap v1315: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:26 smithi070 ceph-mon[79774]: pgmap v1316: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:26 smithi046 ceph-mon[109506]: pgmap v1316: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:28 smithi070 ceph-mon[79774]: pgmap v1317: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:28 smithi046 ceph-mon[109506]: pgmap v1317: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:30 smithi070 ceph-mon[79774]: pgmap v1318: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:30 smithi046 ceph-mon[109506]: pgmap v1318: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:33.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:32 smithi070 ceph-mon[79774]: pgmap v1319: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:33.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:32 smithi046 ceph-mon[109506]: pgmap v1319: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:35.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:34 smithi070 ceph-mon[79774]: pgmap v1320: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:35.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:34 smithi046 ceph-mon[109506]: pgmap v1320: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:37.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:36 smithi070 ceph-mon[79774]: pgmap v1321: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:37.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:36 smithi046 ceph-mon[109506]: pgmap v1321: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:38 smithi070 ceph-mon[79774]: pgmap v1322: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:39.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:39.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:38 smithi046 ceph-mon[109506]: pgmap v1322: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:39.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:41.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:40 smithi070 ceph-mon[79774]: pgmap v1323: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:41.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:40 smithi046 ceph-mon[109506]: pgmap v1323: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:42 smithi046 ceph-mon[109506]: pgmap v1324: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:43.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:42 smithi070 ceph-mon[79774]: pgmap v1324: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:44 smithi070 ceph-mon[79774]: pgmap v1325: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:45.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:44 smithi046 ceph-mon[109506]: pgmap v1325: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:46 smithi070 ceph-mon[79774]: pgmap v1326: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:47.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:46 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:49:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:46 smithi046 ceph-mon[109506]: pgmap v1326: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:47.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:46 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:49:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:47 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:49:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:47 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:49:48.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:47 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:49:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:47 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:49:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:47 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:49:48.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:47 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:49:49.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:48 smithi070 ceph-mon[79774]: pgmap v1327: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:49.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:48 smithi046 ceph-mon[109506]: pgmap v1327: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:51.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:50 smithi070 ceph-mon[79774]: pgmap v1328: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:51.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:50 smithi046 ceph-mon[109506]: pgmap v1328: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:53.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:52 smithi070 ceph-mon[79774]: pgmap v1329: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:53.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:52 smithi046 ceph-mon[109506]: pgmap v1329: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:49:54.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:54.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:49:54.985 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:54 smithi070 ceph-mon[79774]: pgmap v1330: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:55.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:54 smithi046 ceph-mon[109506]: pgmap v1330: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:49:57.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:56 smithi070 ceph-mon[79774]: pgmap v1331: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:57.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:56 smithi046 ceph-mon[109506]: pgmap v1331: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:49:59.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:49:58 smithi070 ceph-mon[79774]: pgmap v1332: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:49:59.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:49:58 smithi046 ceph-mon[109506]: pgmap v1332: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:00 smithi070 ceph-mon[79774]: pgmap v1333: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:00 smithi070 ceph-mon[79774]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:50:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:00 smithi070 ceph-mon[79774]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:50:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:00 smithi070 ceph-mon[79774]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:50:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:00 smithi046 ceph-mon[109506]: pgmap v1333: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:00 smithi046 ceph-mon[109506]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T04:50:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:00 smithi046 ceph-mon[109506]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T04:50:00.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:00 smithi046 ceph-mon[109506]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T04:50:03.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:02 smithi070 ceph-mon[79774]: pgmap v1334: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:03.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:02 smithi046 ceph-mon[109506]: pgmap v1334: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:04.938 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:04 smithi070 ceph-mon[79774]: pgmap v1335: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:05.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:04 smithi046 ceph-mon[109506]: pgmap v1335: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:07.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:06 smithi070 ceph-mon[79774]: pgmap v1336: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:07.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:06 smithi046 ceph-mon[109506]: pgmap v1336: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:08 smithi070 ceph-mon[79774]: pgmap v1337: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:09.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:09.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:08 smithi046 ceph-mon[109506]: pgmap v1337: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:09.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:11.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:10 smithi070 ceph-mon[79774]: pgmap v1338: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:11.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:10 smithi046 ceph-mon[109506]: pgmap v1338: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:13.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:12 smithi070 ceph-mon[79774]: pgmap v1339: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:13.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:12 smithi046 ceph-mon[109506]: pgmap v1339: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:15.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:14 smithi070 ceph-mon[79774]: pgmap v1340: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:15.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:14 smithi046 ceph-mon[109506]: pgmap v1340: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:17.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:16 smithi070 ceph-mon[79774]: pgmap v1341: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:17.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:16 smithi046 ceph-mon[109506]: pgmap v1341: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:19.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:18 smithi070 ceph-mon[79774]: pgmap v1342: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:19.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:18 smithi046 ceph-mon[109506]: pgmap v1342: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:21.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:20 smithi070 ceph-mon[79774]: pgmap v1343: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:21.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:20 smithi046 ceph-mon[109506]: pgmap v1343: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:23.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:22 smithi070 ceph-mon[79774]: pgmap v1344: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:23.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:22 smithi046 ceph-mon[109506]: pgmap v1344: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:24.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:24.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:24.986 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:24 smithi070 ceph-mon[79774]: pgmap v1345: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:25.087 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:24 smithi046 ceph-mon[109506]: pgmap v1345: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:27.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:26 smithi070 ceph-mon[79774]: pgmap v1346: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:27.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:26 smithi046 ceph-mon[109506]: pgmap v1346: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:29.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:28 smithi070 ceph-mon[79774]: pgmap v1347: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:29.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:28 smithi046 ceph-mon[109506]: pgmap v1347: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:31.085 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:30 smithi070 ceph-mon[79774]: pgmap v1348: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:31.086 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:30 smithi046 ceph-mon[109506]: pgmap v1348: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:32.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:32 smithi070 ceph-mon[79774]: pgmap v1349: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:32.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:32 smithi046 ceph-mon[109506]: pgmap v1349: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:34 smithi070 ceph-mon[79774]: pgmap v1350: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:34 smithi046 ceph-mon[109506]: pgmap v1350: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:36.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:36 smithi070 ceph-mon[79774]: pgmap v1351: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:36.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:36 smithi046 ceph-mon[109506]: pgmap v1351: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:50:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:38 smithi046 ceph-mon[109506]: pgmap v1352: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:38.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:38 smithi070 ceph-mon[79774]: pgmap v1352: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:40 smithi070 ceph-mon[79774]: pgmap v1353: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:40 smithi046 ceph-mon[109506]: pgmap v1353: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:42.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:42 smithi070 ceph-mon[79774]: pgmap v1354: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:42.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:42 smithi046 ceph-mon[109506]: pgmap v1354: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:50:44.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:44 smithi046 ceph-mon[109506]: pgmap v1355: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:44 smithi070 ceph-mon[79774]: pgmap v1355: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:50:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:46 smithi070 ceph-mon[79774]: pgmap v1356: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:50:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:46 smithi046 ceph-mon[109506]: pgmap v1356: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:50:47.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:47 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:50:47.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:47 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:50:48.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:48 smithi046 ceph-mon[109506]: pgmap v1357: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-10T04:50:48.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:50:48.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:50:48.781 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:48 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:50:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:48 smithi070 ceph-mon[79774]: pgmap v1357: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-10T04:50:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:50:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:50:48.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:48 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' 2024-09-10T04:50:50.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:50 smithi046 ceph-mon[109506]: pgmap v1358: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:50:50.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:50 smithi070 ceph-mon[79774]: pgmap v1358: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:50:52.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:52 smithi046 ceph-mon[109506]: pgmap v1359: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:50:52.837 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:52 smithi070 ceph-mon[79774]: pgmap v1359: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:50:53.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:53 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:53.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:53 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:50:54.736 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:54 smithi070 ceph-mon[79774]: pgmap v1360: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:50:54.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:54 smithi046 ceph-mon[109506]: pgmap v1360: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:50:56.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:56 smithi046 ceph-mon[109506]: pgmap v1361: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:50:56.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:56 smithi070 ceph-mon[79774]: pgmap v1361: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:50:58.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:50:58 smithi070 ceph-mon[79774]: pgmap v1362: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:50:58.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:50:58 smithi046 ceph-mon[109506]: pgmap v1362: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:00.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:00 smithi070 ceph-mon[79774]: pgmap v1363: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:00.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:00 smithi046 ceph-mon[109506]: pgmap v1363: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:02.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:02 smithi046 ceph-mon[109506]: pgmap v1364: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:02.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:02 smithi070 ceph-mon[79774]: pgmap v1364: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:04.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:04 smithi070 ceph-mon[79774]: pgmap v1365: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:04.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:04 smithi046 ceph-mon[109506]: pgmap v1365: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:06.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:06 smithi070 ceph-mon[79774]: pgmap v1366: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:06.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:06 smithi046 ceph-mon[109506]: pgmap v1366: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:08.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:08 smithi046 ceph-mon[109506]: pgmap v1367: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:08.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:08 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:51:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:08 smithi070 ceph-mon[79774]: pgmap v1367: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:08.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:08 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:51:10.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:10 smithi070 ceph-mon[79774]: pgmap v1368: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:10.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:10 smithi046 ceph-mon[109506]: pgmap v1368: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:12.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:12 smithi070 ceph-mon[79774]: pgmap v1369: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:12.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:12 smithi046 ceph-mon[109506]: pgmap v1369: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:14.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:14 smithi046 ceph-mon[109506]: pgmap v1370: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:14.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:14 smithi070 ceph-mon[79774]: pgmap v1370: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:16.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:16 smithi070 ceph-mon[79774]: pgmap v1371: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:16.837 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:16 smithi046 ceph-mon[109506]: pgmap v1371: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:18.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:18 smithi070 ceph-mon[79774]: pgmap v1372: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:18.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:18 smithi046 ceph-mon[109506]: pgmap v1372: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:20.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:20 smithi046 ceph-mon[109506]: pgmap v1373: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:20.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:20 smithi070 ceph-mon[79774]: pgmap v1373: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:22.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:22 smithi070 ceph-mon[79774]: pgmap v1374: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:22.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:22 smithi046 ceph-mon[109506]: pgmap v1374: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:23.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:23 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:51:23.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:23 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:51:24.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:24 smithi070 ceph-mon[79774]: pgmap v1375: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:24.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:24 smithi046 ceph-mon[109506]: pgmap v1375: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:26.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:26 smithi046 ceph-mon[109506]: pgmap v1376: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:26.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:26 smithi070 ceph-mon[79774]: pgmap v1376: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:28.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:28 smithi070 ceph-mon[79774]: pgmap v1377: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:28.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:28 smithi046 ceph-mon[109506]: pgmap v1377: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:30.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:30 smithi070 ceph-mon[79774]: pgmap v1378: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:30.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:30 smithi046 ceph-mon[109506]: pgmap v1378: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:32.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:32 smithi046 ceph-mon[109506]: pgmap v1379: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:32.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:32 smithi070 ceph-mon[79774]: pgmap v1379: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T04:51:34.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:34 smithi070 ceph-mon[79774]: pgmap v1380: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:34.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:34 smithi046 ceph-mon[109506]: pgmap v1380: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:36.780 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:36 smithi070 ceph-mon[79774]: pgmap v1381: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:36.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:36 smithi046 ceph-mon[109506]: pgmap v1381: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T04:51:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:38 smithi046 ceph-mon[109506]: pgmap v1382: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:38.587 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:38 smithi046 ceph-mon[109506]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:51:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:38 smithi070 ceph-mon[79774]: pgmap v1382: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T04:51:38.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:38 smithi070 ceph-mon[79774]: from='mgr.14758 172.21.15.46:0/3447456367' entity='mgr.smithi046.zvckjg' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:51:40.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:40 smithi070 ceph-mon[79774]: pgmap v1383: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:40.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:40 smithi046 ceph-mon[109506]: pgmap v1383: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T04:51:42.585 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:42 smithi070 ceph-mon[79774]: pgmap v1384: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:51:42.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:42 smithi046 ceph-mon[109506]: pgmap v1384: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-09-10T04:51:44.586 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:44 smithi046 ceph-mon[109506]: pgmap v1385: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:51:44.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:44 smithi070 ceph-mon[79774]: pgmap v1385: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-09-10T04:51:46.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:46 smithi070 ceph-mon[79774]: pgmap v1386: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:51:46.836 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:46 smithi046 ceph-mon[109506]: pgmap v1386: 65 pgs: 65 active+clean; 245 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T04:51:47.423 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-09-10T04:51:47.423 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T04:51:47.425 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T04:51:47.438 INFO:tasks.cephadm:Teardown begin 2024-09-10T04:51:47.439 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/cephadm.py", line 2374, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T04:51:47.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:51:47.477 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:51:47.509 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T04:51:47.510 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T04:51:47.535 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T04:51:47.566 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T04:51:47.567 INFO:tasks.cephadm.mon.smithi046:Stopping mon.smithi046... 2024-09-10T04:51:47.567 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046 2024-09-10T04:51:47.941 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:47 smithi046 systemd[1]: Stopping Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:51:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:47 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:51:47.938+0000 7f3cac81f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi046 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:51:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:47 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046[109502]: 2024-09-10T04:51:47.939+0000 7f3cac81f640 -1 mon.smithi046@0(leader) e2 *** Got Signal Terminated *** 2024-09-10T04:51:48.337 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:48 smithi046 podman[160897]: 2024-09-10 04:51:48.04169184 +0000 UTC m=+0.238793696 container died da8d2333cd825834448ae8d2e4f822b655d9c5cc53b4cd09afa71510c3d7553e (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731) 2024-09-10T04:51:48.766 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:48 smithi046 podman[160897]: 2024-09-10 04:51:48.57238797 +0000 UTC m=+0.769489823 container cleanup da8d2333cd825834448ae8d2e4f822b655d9c5cc53b4cd09afa71510c3d7553e (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:51:48.766 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:48 smithi046 bash[160897]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046 2024-09-10T04:51:49.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service' 2024-09-10T04:51:49.027 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:48 smithi046 podman[160948]: 2024-09-10 04:51:48.76553789 +0000 UTC m=+0.720158965 container remove da8d2333cd825834448ae8d2e4f822b655d9c5cc53b4cd09afa71510c3d7553e (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi046, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T04:51:49.028 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:49 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service: Deactivated successfully. 2024-09-10T04:51:49.028 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:49 smithi046 systemd[1]: Stopped Ceph mon.smithi046 for 628edc22-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:51:49.028 INFO:journalctl@ceph.mon.smithi046.smithi046.stdout:Sep 10 04:51:49 smithi046 systemd[1]: ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi046.service: Consumed 27.870s CPU time. 2024-09-10T04:51:49.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:51:49.059 INFO:tasks.cephadm.mon.smithi046:Stopped mon.smithi046 2024-09-10T04:51:49.059 INFO:tasks.cephadm.mon.smithi070:Stopping mon.smithi070... 2024-09-10T04:51:49.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi070 2024-09-10T04:51:49.563 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:49 smithi070 systemd[1]: Stopping Ceph mon.smithi070 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:51:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:49 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070[79769]: 2024-09-10T04:51:49.560+0000 7fc28e344640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi070 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:51:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:49 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070[79769]: 2024-09-10T04:51:49.560+0000 7fc28e344640 -1 mon.smithi070@1(peon) e2 *** Got Signal Terminated *** 2024-09-10T04:51:49.835 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:49 smithi070 podman[113188]: 2024-09-10 04:51:49.643413922 +0000 UTC m=+0.380515906 container died 9c38930714e4a15eb2c3bdd945213e59e04dc1a82f9441d3e030504125a70bfd (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T04:51:50.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:50 smithi070 podman[113188]: 2024-09-10 04:51:50.111156456 +0000 UTC m=+0.848258436 container cleanup 9c38930714e4a15eb2c3bdd945213e59e04dc1a82f9441d3e030504125a70bfd (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T04:51:50.335 INFO:journalctl@ceph.mon.smithi070.smithi070.stdout:Sep 10 04:51:50 smithi070 bash[113188]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-mon-smithi070 2024-09-10T04:51:50.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@mon.smithi070.service' 2024-09-10T04:51:50.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:51:50.573 INFO:tasks.cephadm.mon.smithi070:Stopped mon.smithi070 2024-09-10T04:51:50.573 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T04:51:50.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.0 2024-09-10T04:51:50.861 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:50 smithi046 systemd[1]: Stopping Ceph osd.0 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:51:51.337 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:50 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[118266]: 2024-09-10T04:51:50.859+0000 7fe9f7a96640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:51:51.337 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:50 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[118266]: 2024-09-10T04:51:50.859+0000 7fe9f7a96640 -1 osd.0 79 *** Got signal Terminated *** 2024-09-10T04:51:51.337 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:50 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0[118266]: 2024-09-10T04:51:50.859+0000 7fe9f7a96640 -1 osd.0 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:51:56.538 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:56 smithi046 podman[161060]: 2024-09-10 04:51:56.221367212 +0000 UTC m=+5.450406138 container died 5359fceff12da4da4dd517379ba1ee50b1cd396b9f648f58ac6fc4f7b76c80f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T04:51:56.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:56 smithi046 podman[161060]: 2024-09-10 04:51:56.635842149 +0000 UTC m=+5.864881075 container cleanup 5359fceff12da4da4dd517379ba1ee50b1cd396b9f648f58ac6fc4f7b76c80f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T04:51:56.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:56 smithi046 bash[161060]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0 2024-09-10T04:51:57.208 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:56 smithi046 podman[161089]: 2024-09-10 04:51:56.881392824 +0000 UTC m=+0.656516666 container remove 5359fceff12da4da4dd517379ba1ee50b1cd396b9f648f58ac6fc4f7b76c80f1 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T04:51:57.466 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:57 smithi046 podman[161176]: 2024-09-10 04:51:57.110996048 +0000 UTC m=+0.016560394 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:51:57.466 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:57 smithi046 podman[161176]: 2024-09-10 04:51:57.229847439 +0000 UTC m=+0.135411825 container create 030e7b24e028a3c1cee7dcd9c782285baf643a040f257e83df6b353a531d9c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T04:51:57.466 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:57 smithi046 podman[161176]: 2024-09-10 04:51:57.465670527 +0000 UTC m=+0.371234871 container init 030e7b24e028a3c1cee7dcd9c782285baf643a040f257e83df6b353a531d9c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T04:51:57.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:57 smithi046 podman[161176]: 2024-09-10 04:51:57.469450264 +0000 UTC m=+0.375014612 container start 030e7b24e028a3c1cee7dcd9c782285baf643a040f257e83df6b353a531d9c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2) 2024-09-10T04:51:57.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:57 smithi046 podman[161176]: 2024-09-10 04:51:57.557322303 +0000 UTC m=+0.462886653 container attach 030e7b24e028a3c1cee7dcd9c782285baf643a040f257e83df6b353a531d9c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T04:51:57.837 INFO:journalctl@ceph.osd.0.smithi046.stdout:Sep 10 04:51:57 smithi046 podman[161176]: 2024-09-10 04:51:57.660371103 +0000 UTC m=+0.565935457 container died 030e7b24e028a3c1cee7dcd9c782285baf643a040f257e83df6b353a531d9c01 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:51:58.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T04:51:58.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:51:58.321 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T04:51:58.322 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T04:51:58.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.1 2024-09-10T04:51:58.587 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:51:58 smithi046 systemd[1]: Stopping Ceph osd.1 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:51:59.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:51:58 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[123892]: 2024-09-10T04:51:58.689+0000 7f1a7d857640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:51:59.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:51:58 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[123892]: 2024-09-10T04:51:58.689+0000 7f1a7d857640 -1 osd.1 79 *** Got signal Terminated *** 2024-09-10T04:51:59.087 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:51:58 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1[123892]: 2024-09-10T04:51:58.689+0000 7f1a7d857640 -1 osd.1 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:52:04.337 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:04 smithi046 podman[161280]: 2024-09-10 04:52:04.077971394 +0000 UTC m=+5.527239118 container stop 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T04:52:04.337 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:04 smithi046 podman[161280]: 2024-09-10 04:52:04.078793348 +0000 UTC m=+5.528061073 container died 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T04:52:04.742 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:04 smithi046 podman[161280]: 2024-09-10 04:52:04.456809624 +0000 UTC m=+5.906077389 container cleanup 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:52:04.742 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:04 smithi046 bash[161280]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1 2024-09-10T04:52:05.086 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:04 smithi046 podman[161295]: 2024-09-10 04:52:04.741260868 +0000 UTC m=+0.654250081 container remove 4a8fa4298755ddf26f577e4b1d869d832ab49e8fbeb0f8d731cf4fa8a490002a (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T04:52:05.362 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:05 smithi046 podman[161385]: 2024-09-10 04:52:05.002338587 +0000 UTC m=+0.016405608 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:52:05.362 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:05 smithi046 podman[161385]: 2024-09-10 04:52:05.134187435 +0000 UTC m=+0.148254459 container create 88072d5fd31ffb1a3e2ae47d5dfca303cfa2d1a55c1e1ee3dccebd99ca674462 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T04:52:05.362 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:05 smithi046 podman[161385]: 2024-09-10 04:52:05.361885629 +0000 UTC m=+0.375952653 container init 88072d5fd31ffb1a3e2ae47d5dfca303cfa2d1a55c1e1ee3dccebd99ca674462 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD) 2024-09-10T04:52:05.837 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:05 smithi046 podman[161385]: 2024-09-10 04:52:05.365412569 +0000 UTC m=+0.379479599 container start 88072d5fd31ffb1a3e2ae47d5dfca303cfa2d1a55c1e1ee3dccebd99ca674462 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T04:52:05.837 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:05 smithi046 podman[161385]: 2024-09-10 04:52:05.486761581 +0000 UTC m=+0.500828603 container attach 88072d5fd31ffb1a3e2ae47d5dfca303cfa2d1a55c1e1ee3dccebd99ca674462 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T04:52:05.837 INFO:journalctl@ceph.osd.1.smithi046.stdout:Sep 10 04:52:05 smithi046 podman[161385]: 2024-09-10 04:52:05.564926183 +0000 UTC m=+0.578993210 container died 88072d5fd31ffb1a3e2ae47d5dfca303cfa2d1a55c1e1ee3dccebd99ca674462 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T04:52:06.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T04:52:06.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:52:06.330 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T04:52:06.330 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T04:52:06.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.2 2024-09-10T04:52:06.586 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:06 smithi046 systemd[1]: Stopping Ceph osd.2 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:52:07.087 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:06 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[129483]: 2024-09-10T04:52:06.685+0000 7f7a3ed39640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:52:07.087 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:06 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[129483]: 2024-09-10T04:52:06.685+0000 7f7a3ed39640 -1 osd.2 79 *** Got signal Terminated *** 2024-09-10T04:52:07.087 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:06 smithi046 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2[129483]: 2024-09-10T04:52:06.685+0000 7f7a3ed39640 -1 osd.2 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:52:12.533 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:12 smithi046 podman[161488]: 2024-09-10 04:52:12.230465464 +0000 UTC m=+5.667252835 container died 66743aa5ec84046f37d3767154c5471cbd1f4f6db10244795d4e666ae86f15b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-10T04:52:12.836 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:12 smithi046 podman[161488]: 2024-09-10 04:52:12.612210127 +0000 UTC m=+6.048997498 container cleanup 66743aa5ec84046f37d3767154c5471cbd1f4f6db10244795d4e666ae86f15b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T04:52:12.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:12 smithi046 bash[161488]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2 2024-09-10T04:52:13.198 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:12 smithi046 podman[161506]: 2024-09-10 04:52:12.855185203 +0000 UTC m=+0.622164536 container remove 66743aa5ec84046f37d3767154c5471cbd1f4f6db10244795d4e666ae86f15b0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T04:52:13.459 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:13 smithi046 podman[161596]: 2024-09-10 04:52:13.100369053 +0000 UTC m=+0.016407504 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:52:13.459 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:13 smithi046 podman[161596]: 2024-09-10 04:52:13.224405641 +0000 UTC m=+0.140444127 container create b0ff75bc938dd898f6ee1d8b7453837854df50247946ac80d9e43ae4c73cb4a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:52:13.460 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:13 smithi046 podman[161596]: 2024-09-10 04:52:13.458877552 +0000 UTC m=+0.374916004 container init b0ff75bc938dd898f6ee1d8b7453837854df50247946ac80d9e43ae4c73cb4a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:52:13.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:13 smithi046 podman[161596]: 2024-09-10 04:52:13.462460883 +0000 UTC m=+0.378499323 container start b0ff75bc938dd898f6ee1d8b7453837854df50247946ac80d9e43ae4c73cb4a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:52:13.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:13 smithi046 podman[161596]: 2024-09-10 04:52:13.55053576 +0000 UTC m=+0.466574205 container attach b0ff75bc938dd898f6ee1d8b7453837854df50247946ac80d9e43ae4c73cb4a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T04:52:13.837 INFO:journalctl@ceph.osd.2.smithi046.stdout:Sep 10 04:52:13 smithi046 podman[161596]: 2024-09-10 04:52:13.659578581 +0000 UTC m=+0.575617034 container died b0ff75bc938dd898f6ee1d8b7453837854df50247946ac80d9e43ae4c73cb4a8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux ) 2024-09-10T04:52:14.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T04:52:14.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:52:14.356 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T04:52:14.356 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T04:52:14.356 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.3 2024-09-10T04:52:14.680 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:14 smithi070 systemd[1]: Stopping Ceph osd.3 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:52:15.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:14 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:52:14.678+0000 7fc57ef07640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:52:15.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:14 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:52:14.678+0000 7fc57ef07640 -1 osd.3 79 *** Got signal Terminated *** 2024-09-10T04:52:15.085 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:14 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:52:14.678+0000 7fc57ef07640 -1 osd.3 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:52:18.335 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:17 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:17.835+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:18.835 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:18 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:52:18.491+0000 7fc57b51c640 -1 osd.3 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:53.964546+0000 front 2024-09-10T04:51:53.964415+0000 (oldest deadline 2024-09-10T04:52:18.064316+0000) 2024-09-10T04:52:19.271 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:18 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:18.834+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:19.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:19 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3[87181]: 2024-09-10T04:52:19.457+0000 7fc57b51c640 -1 osd.3 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:53.964546+0000 front 2024-09-10T04:51:53.964415+0000 (oldest deadline 2024-09-10T04:52:18.064316+0000) 2024-09-10T04:52:19.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:19 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:19.269+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:20.296 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:20 smithi070 podman[113322]: 2024-09-10 04:52:20.007998558 +0000 UTC m=+5.447553066 container died 0313330c72b68a051f416fc8f5252c102f5735c4f01895f720730cc11b9044cb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T04:52:20.297 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:19 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:19.857+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:20.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:20 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:20.294+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:20.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:20 smithi070 podman[113322]: 2024-09-10 04:52:20.428579281 +0000 UTC m=+5.868133788 container cleanup 0313330c72b68a051f416fc8f5252c102f5735c4f01895f720730cc11b9044cb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T04:52:20.585 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:20 smithi070 bash[113322]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3 2024-09-10T04:52:21.047 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:20 smithi070 podman[113338]: 2024-09-10 04:52:20.630150759 +0000 UTC m=+0.624633222 container remove 0313330c72b68a051f416fc8f5252c102f5735c4f01895f720730cc11b9044cb (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T04:52:21.048 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:20 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:20.831+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:21.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:21 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:21.286+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:21.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:21 smithi070 podman[113430]: 2024-09-10 04:52:20.949584668 +0000 UTC m=+0.015868445 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:52:21.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:21 smithi070 podman[113430]: 2024-09-10 04:52:21.064300445 +0000 UTC m=+0.130584258 container create f598022059b85f0bf9ef553f022209a3267a2110d809b780844aa1c416b87d19 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T04:52:21.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:21 smithi070 podman[113430]: 2024-09-10 04:52:21.291829831 +0000 UTC m=+0.358113601 container init f598022059b85f0bf9ef553f022209a3267a2110d809b780844aa1c416b87d19 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T04:52:21.335 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:21 smithi070 podman[113430]: 2024-09-10 04:52:21.295136716 +0000 UTC m=+0.361420489 container start f598022059b85f0bf9ef553f022209a3267a2110d809b780844aa1c416b87d19 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:52:21.803 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:21 smithi070 podman[113430]: 2024-09-10 04:52:21.383509354 +0000 UTC m=+0.449793140 container attach f598022059b85f0bf9ef553f022209a3267a2110d809b780844aa1c416b87d19 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:52:21.804 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 10 04:52:21 smithi070 podman[113430]: 2024-09-10 04:52:21.488955584 +0000 UTC m=+0.555239357 container died f598022059b85f0bf9ef553f022209a3267a2110d809b780844aa1c416b87d19 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T04:52:22.085 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:21 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:21.873+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:22.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T04:52:22.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:52:22.191 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T04:52:22.191 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T04:52:22.191 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.4 2024-09-10T04:52:22.411 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:22 smithi070 systemd[1]: Stopping Ceph osd.4 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:52:22.411 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:22 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:22.237+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:22.835 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:22 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:22.540+0000 7f0508891640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:52:22.835 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:22 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:22.540+0000 7f0508891640 -1 osd.4 79 *** Got signal Terminated *** 2024-09-10T04:52:22.835 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:22 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:22.540+0000 7f0508891640 -1 osd.4 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:52:23.255 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:22 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:22.892+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:23.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:23 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:23.253+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:24.300 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:23 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:23.941+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:24.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:24 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:24.299+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:25.335 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:24 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:24.969+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:25.335 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:24 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:24.969+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:01.481528+0000 front 2024-09-10T04:52:01.481563+0000 (oldest deadline 2024-09-10T04:52:24.881611+0000) 2024-09-10T04:52:25.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:25 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:25.334+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:26.289 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:25 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:25.984+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:26.290 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:25 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:25.984+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:01.481528+0000 front 2024-09-10T04:52:01.481563+0000 (oldest deadline 2024-09-10T04:52:24.881611+0000) 2024-09-10T04:52:26.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:26 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:26.288+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:27.317 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:26 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:26.984+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:52.880117+0000 front 2024-09-10T04:51:52.879977+0000 (oldest deadline 2024-09-10T04:52:16.979922+0000) 2024-09-10T04:52:27.317 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:26 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4[92144]: 2024-09-10T04:52:26.984+0000 7f0504ea6640 -1 osd.4 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:01.481528+0000 front 2024-09-10T04:52:01.481563+0000 (oldest deadline 2024-09-10T04:52:24.881611+0000) 2024-09-10T04:52:27.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:27 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:27.316+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:27.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:27 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:27.316+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:28.085 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:27 smithi070 podman[113532]: 2024-09-10 04:52:27.865525444 +0000 UTC m=+5.437227788 container died 3bf61d5adfbe6a23955ba0a3bf6f086dba640c212485bb04bc0d0b892d04d6c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2) 2024-09-10T04:52:28.559 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:28 smithi070 podman[113532]: 2024-09-10 04:52:28.265276901 +0000 UTC m=+5.836979244 container cleanup 3bf61d5adfbe6a23955ba0a3bf6f086dba640c212485bb04bc0d0b892d04d6c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T04:52:28.559 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:28 smithi070 bash[113532]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4 2024-09-10T04:52:28.559 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:28 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:28.346+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:28.559 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:28 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:28.346+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:28.835 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:28 smithi070 podman[113562]: 2024-09-10 04:52:28.558305078 +0000 UTC m=+0.683700546 container remove 3bf61d5adfbe6a23955ba0a3bf6f086dba640c212485bb04bc0d0b892d04d6c0 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:52:29.162 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:28 smithi070 podman[113648]: 2024-09-10 04:52:28.774996042 +0000 UTC m=+0.015579303 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:52:29.162 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:28 smithi070 podman[113648]: 2024-09-10 04:52:28.87577177 +0000 UTC m=+0.116355022 container create 6f54eebe0840cb8e76069d396fc1afbdd102e510a1f8b7d6b60e66e609ffbe55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:52:29.162 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:29 smithi070 podman[113648]: 2024-09-10 04:52:29.078924392 +0000 UTC m=+0.319507646 container init 6f54eebe0840cb8e76069d396fc1afbdd102e510a1f8b7d6b60e66e609ffbe55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T04:52:29.162 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:29 smithi070 podman[113648]: 2024-09-10 04:52:29.082533345 +0000 UTC m=+0.323116606 container start 6f54eebe0840cb8e76069d396fc1afbdd102e510a1f8b7d6b60e66e609ffbe55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-10T04:52:29.162 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:29 smithi070 podman[113648]: 2024-09-10 04:52:29.161658305 +0000 UTC m=+0.402241570 container attach 6f54eebe0840cb8e76069d396fc1afbdd102e510a1f8b7d6b60e66e609ffbe55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T04:52:29.583 INFO:journalctl@ceph.osd.4.smithi070.stdout:Sep 10 04:52:29 smithi070 podman[113648]: 2024-09-10 04:52:29.2710743 +0000 UTC m=+0.511657552 container died 6f54eebe0840cb8e76069d396fc1afbdd102e510a1f8b7d6b60e66e609ffbe55 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T04:52:29.583 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:29 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:29.344+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:29.583 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:29 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:29.344+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:29.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T04:52:29.948 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:52:29.948 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T04:52:29.948 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T04:52:29.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.5 2024-09-10T04:52:30.277 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:30 smithi070 systemd[1]: Stopping Ceph osd.5 for 628edc22-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:52:30.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:30 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:30.275+0000 7fa727db7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:52:30.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:30 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:30.275+0000 7fa727db7640 -1 osd.5 79 *** Got signal Terminated *** 2024-09-10T04:52:30.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:30 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:30.275+0000 7fa727db7640 -1 osd.5 79 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:52:30.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:30 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:30.361+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:30.585 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:30 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:30.361+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:31.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:31 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:31.368+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:31.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:31 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:31.368+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:32.782 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:32 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:32.355+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:32.782 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:32 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:32.355+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:33.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:33 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:33.396+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:33.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:33 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:33.396+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:33.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:33 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:33.396+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-09-10T04:52:07.110632+0000 front 2024-09-10T04:52:07.110606+0000 (oldest deadline 2024-09-10T04:52:33.010503+0000) 2024-09-10T04:52:34.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:34 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:34.367+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-09-10T04:51:54.909561+0000 front 2024-09-10T04:51:54.909638+0000 (oldest deadline 2024-09-10T04:52:18.309603+0000) 2024-09-10T04:52:34.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:34 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:34.367+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-09-10T04:52:02.410270+0000 front 2024-09-10T04:52:02.410315+0000 (oldest deadline 2024-09-10T04:52:27.110200+0000) 2024-09-10T04:52:34.835 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:34 smithi070 ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5[97106]: 2024-09-10T04:52:34.367+0000 7fa7203c4640 -1 osd.5 79 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-09-10T04:52:07.110632+0000 front 2024-09-10T04:52:07.110606+0000 (oldest deadline 2024-09-10T04:52:33.010503+0000) 2024-09-10T04:52:36.012 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:35 smithi070 podman[113754]: 2024-09-10 04:52:35.723170418 +0000 UTC m=+5.552117751 container died 79aa94488c3a319444330beaa647efd701421d655d9679709f6d305ef38ec020 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:52:36.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:36 smithi070 podman[113754]: 2024-09-10 04:52:36.084159737 +0000 UTC m=+5.913107072 container cleanup 79aa94488c3a319444330beaa647efd701421d655d9679709f6d305ef38ec020 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T04:52:36.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:36 smithi070 bash[113754]: ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5 2024-09-10T04:52:36.701 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:36 smithi070 podman[113769]: 2024-09-10 04:52:36.38568067 +0000 UTC m=+0.656836345 container remove 79aa94488c3a319444330beaa647efd701421d655d9679709f6d305ef38ec020 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T04:52:36.972 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:36 smithi070 podman[113861]: 2024-09-10 04:52:36.603860438 +0000 UTC m=+0.015698869 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:52:36.973 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:36 smithi070 podman[113861]: 2024-09-10 04:52:36.736389286 +0000 UTC m=+0.148227716 container create 778a939a9f0c02873104f3366c90c1424fcfe19dfbb68de8485abca8c12729c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:52:36.973 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:36 smithi070 podman[113861]: 2024-09-10 04:52:36.972150874 +0000 UTC m=+0.383989304 container init 778a939a9f0c02873104f3366c90c1424fcfe19dfbb68de8485abca8c12729c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T04:52:37.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:36 smithi070 podman[113861]: 2024-09-10 04:52:36.97543971 +0000 UTC m=+0.387278147 container start 778a939a9f0c02873104f3366c90c1424fcfe19dfbb68de8485abca8c12729c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T04:52:37.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:37 smithi070 podman[113861]: 2024-09-10 04:52:37.055525993 +0000 UTC m=+0.467364443 container attach 778a939a9f0c02873104f3366c90c1424fcfe19dfbb68de8485abca8c12729c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T04:52:37.335 INFO:journalctl@ceph.osd.5.smithi070.stdout:Sep 10 04:52:37 smithi070 podman[113861]: 2024-09-10 04:52:37.168302472 +0000 UTC m=+0.580140904 container died 778a939a9f0c02873104f3366c90c1424fcfe19dfbb68de8485abca8c12729c8 (image=quay.ceph.io/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-628edc22-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:52:37.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-628edc22-6f28-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T04:52:37.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:52:37.795 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T04:52:37.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 628edc22-6f28-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T04:52:38.009 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:52:42.674 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[76450]: fuse finished with error 0 and tester_r 0 2024-09-10T04:53:01.474 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 628edc22-6f28-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T04:53:01.675 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:53:17.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:53:17.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:53:17.231 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T04:53:17.233 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603/remote/smithi046/crash 2024-09-10T04:53:17.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash -- . 2024-09-10T04:53:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T04:53:17.274 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-09-10T04:53:17.276 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603/remote/smithi070/crash 2024-09-10T04:53:17.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash -- . 2024-09-10T04:53:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/628edc22-6f28-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T04:53:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2024-09-10T04:53:17.302 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T04:53:17.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-10T04:53:17.354 INFO:teuthology.orchestra.run.smithi046.stdout:2024-09-10T04:20:00.000432+0000 mon.smithi046 (mon.0) 568 : cluster [ERR] fs cephfs is offline because no MDS is active for it. 2024-09-10T04:53:17.355 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-10T04:53:17.355 INFO:tasks.cephadm:Compressing logs... 2024-09-10T04:53:17.355 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T04:53:17.397 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T04:53:17.422 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T04:53:17.422 INFO:teuthology.orchestra.run.smithi070.stderr:: No such file or directory 2024-09-10T04:53:17.422 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T04:53:17.422 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T04:53:17.422 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mon.smithi070.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:: No such file or directory 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mon.smithi046.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mon.smithi046.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mgr.smithi046.zvckjg.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mgr.smithi046.zvckjg.log: /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi046.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi046.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T04:53:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.log 2024-09-10T04:53:17.431 INFO:teuthology.orchestra.run.smithi046.stderr: 86.4% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T04:53:17.431 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi070.log 2024-09-10T04:53:17.431 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mon.smithi070.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi070.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.log 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mgr.smithi070.tezbzn.log 85.1% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi070.myztiz.log 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mgr.smithi070.tezbzn.log: /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.log: 94.5% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi070.log.gz 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr: 91.4% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi070.myztiz.log: gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi070.ehvjgr.log 2024-09-10T04:53:17.432 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.3.log: 89.6% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T04:53:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.2.log: 94.4% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi046.log.gz 2024-09-10T04:53:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi046.hasdap.log 2024-09-10T04:53:17.434 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.log: 91.3% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T04:53:17.437 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T04:53:17.438 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-09-10T04:53:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi046.cpwyud.log 2024-09-10T04:53:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi046.hasdap.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-10T04:53:17.444 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi070.ehvjgr.log: 91.0% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T04:53:17.445 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T04:53:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi046.cpwyud.log: 91.0% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T04:53:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T04:53:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T04:53:17.480 INFO:teuthology.orchestra.run.smithi070.stderr: 90.1% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mgr.smithi070.tezbzn.log.gz 2024-09-10T04:53:17.515 INFO:teuthology.orchestra.run.smithi070.stderr: 94.6% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi070.myztiz.log.gz 2024-09-10T04:53:17.556 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 91.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T04:53:18.081 INFO:teuthology.orchestra.run.smithi046.stderr: 90.1% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mgr.smithi046.zvckjg.log.gz 2024-09-10T04:53:18.314 INFO:teuthology.orchestra.run.smithi070.stderr: 92.5% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mon.smithi070.log.gz 2024-09-10T04:53:19.846 INFO:teuthology.orchestra.run.smithi046.stderr: 89.8% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mon.smithi046.log.gz 2024-09-10T04:53:21.501 INFO:teuthology.orchestra.run.smithi070.stderr: 93.8% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T04:53:22.424 INFO:teuthology.orchestra.run.smithi046.stderr: 93.7% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T04:53:22.517 INFO:teuthology.orchestra.run.smithi046.stderr: 93.7% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T04:53:23.207 INFO:teuthology.orchestra.run.smithi070.stderr: 94.8% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi070.ehvjgr.log.gz 2024-09-10T04:53:23.941 INFO:teuthology.orchestra.run.smithi070.stderr: 93.8% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T04:53:24.051 INFO:teuthology.orchestra.run.smithi070.stderr: 93.8% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T04:53:24.860 INFO:teuthology.orchestra.run.smithi046.stderr: 93.8% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T04:53:26.335 INFO:teuthology.orchestra.run.smithi046.stderr: 94.9% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi046.cpwyud.log.gz 2024-09-10T04:53:32.562 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-09-10T04:53:32.562 INFO:teuthology.orchestra.run.smithi070.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-09-10T04:53:32.564 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-10T04:53:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m15.152s 2024-09-10T04:53:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m37.979s 2024-09-10T04:53:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m1.152s 2024-09-10T04:53:34.015 INFO:teuthology.orchestra.run.smithi046.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-10T04:55:07.973 INFO:teuthology.orchestra.run.smithi046.stderr: 93.0% -- replaced with /var/log/ceph/628edc22-6f28-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi046.hasdap.log.gz 2024-09-10T04:55:07.974 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-09-10T04:55:07.974 INFO:teuthology.orchestra.run.smithi046.stderr:real 1m50.560s 2024-09-10T04:55:07.974 INFO:teuthology.orchestra.run.smithi046.stderr:user 2m31.813s 2024-09-10T04:55:07.974 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m4.525s 2024-09-10T04:55:07.975 INFO:tasks.cephadm:Archiving logs... 2024-09-10T04:55:07.976 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603/remote/smithi046/log 2024-09-10T04:55:07.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T04:55:28.488 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603/remote/smithi070/log 2024-09-10T04:55:28.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T04:55:33.481 INFO:tasks.cephadm:Removing cluster... 2024-09-10T04:55:33.481 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 628edc22-6f28-11ef-bcea-c7b262605968 --force 2024-09-10T04:55:33.697 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:55:34.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 628edc22-6f28-11ef-bcea-c7b262605968 --force 2024-09-10T04:55:34.319 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: 628edc22-6f28-11ef-bcea-c7b262605968 2024-09-10T04:55:34.594 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T04:55:34.594 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T04:55:34.611 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T04:55:34.626 INFO:tasks.cephadm:Teardown complete 2024-09-10T04:55:34.627 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T04:55:34.638 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T04:55:34.638 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T04:55:34.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T04:55:34.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T04:55:34.735 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T04:55:34.745 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T04:55:34.746 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T04:55:34.748 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T04:55:34.761 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-09-10T04:55:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-10T04:55:34.767 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T04:55:34.767 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-09-10T04:55:34.767 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +17us[ +14us] +/- 52ms 2024-09-10T04:55:34.767 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 72 -304us[ -308us] +/- 58ms 2024-09-10T04:55:34.767 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +456us[ +456us] +/- 88ms 2024-09-10T04:55:34.767 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T04:55:34.768 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T04:55:34.768 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-10T04:55:34.768 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv01.front.sepia.ceph.com 3 7 377 78 +15us[ +18us] +/- 51ms 2024-09-10T04:55:34.769 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 11 -307us[ -307us] +/- 59ms 2024-09-10T04:55:34.769 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +443us[ +443us] +/- 88ms 2024-09-10T04:55:34.769 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T04:55:34.769 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T04:55:34.781 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T04:55:34.781 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T04:55:34.832 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T04:55:34.916 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T04:55:34.988 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T04:55:35.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:55:35.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T04:55:35.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:55:35.046 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T04:55:35.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T04:55:35.081 INFO:teuthology.task.internal:Duration was 4060.193934 seconds 2024-09-10T04:55:35.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T04:55:35.131 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T04:55:35.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T04:55:35.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T04:55:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T04:55:35.172 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T04:55:35.381 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T04:55:35.381 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-09-10T04:55:35.381 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T04:55:35.445 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-09-10T04:55:35.445 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T04:55:35.465 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T04:55:35.465 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T04:55:35.488 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T04:55:35.531 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T04:55:35.531 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T04:55:35.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T04:55:36.789 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T04:55:36.801 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T04:55:36.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T04:55:36.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T04:55:36.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T04:55:36.869 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:55:36.871 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:55:36.893 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-09-10T04:55:36.925 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-09-10T04:55:37.042 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:55:37.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:55:37.057 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:55:37.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:55:37.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T04:55:37.112 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T04:55:37.113 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603/remote/smithi046 2024-09-10T04:55:37.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T04:55:37.171 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898603/remote/smithi070 2024-09-10T04:55:37.172 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T04:55:37.212 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T04:55:37.213 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T04:55:37.215 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T04:55:37.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T04:55:37.276 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T04:55:37.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T04:55:37.300 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T04:55:37.300 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T04:55:37.302 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T04:55:37.315 INFO:teuthology.orchestra.run.smithi046.stdout: 265324 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 10 04:55 /home/ubuntu/cephtest 2024-09-10T04:55:37.316 INFO:teuthology.orchestra.run.smithi046.stdout: 289472 4 d--------- 2 ubuntu ubuntu 4096 Sep 10 04:02 /home/ubuntu/cephtest/mnt.0 2024-09-10T04:55:37.316 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-09-10T04:55:37.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:55:37.317 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-09-10T04:55:37.317 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/ceph_fuse.py", line 248, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-09-10T04:55:37.318 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T04:55:37.349 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T04:55:37.402 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T04:55:37.405 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/squid 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 4060.1939339637756 failure_reason: reached maximum tries (50) after waiting for 300 seconds flavor: default owner: scheduled_adking@teuthology status: fail success: false 2024-09-10T04:55:37.405 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T04:55:37.487 INFO:teuthology.run:FAIL